Multi-device support

[Krzmbrzl]

Context

Mumble uses certificates for user authentication. That's great, but causes issues as soon as one is trying to use multiple devices as per default each of them will use a different certificate and therefore these devices will be considered different users.

Of course one can copy the original certificate to the new device(s), but this solution is clearly suboptimal (from a usability as well as from a security point of view).

Description

It would be neat, if we were able to support multiple devices that are all associated with a single registered user on a given server.

This will require handling authentication with different certificates as well as a way to add and remove devices (certificates really) to/from a list of known devices.

As an extension, it'd be nice for a user to be online with different devices at the same time. In the UI these would display only as a single connected client but if that client is receiving audio that stream should either get sent to all currently connected devices or (if configured by the user) only to one specific device at a time.

Mumble component

Server

OS-specific?

No

Additional information

Related:

• Allow multiple certificates per user #2107
• Allow multiple login with the same certificate, but several different devices / resources #1832

[Krzmbrzl]

Note: I will work on implementing this feature as soon as I have finished the database rewrite.

[Hartmnt]

As an extension, it'd be nice for a user to be online with different devices at the same time. In the UI these would display only as a single connected client but if that client is receiving audio that stream should either get sent to all currently connected devices or (if configured by the user) only to one specific device at a time.

I feel like this needs further consideration. I have to regularly deal with the following scenario: A user switches devices between desktop and mobile with two certificates. Ideally, I want the user to have the same registered ID on both devices, as in the "multi device support" you propose here. However, due to the extreme quality differences between Mumble desktop and the desktop microphone and Mumla and the mobile headset, I would really really also want to be able to set different, persistent local volume adjustments for both devices independently.

[Krzmbrzl]

I would really really also want to be able to set different, persistent local volume adjustments for both devices independently.

That seems to be impossible. Either clients consider these devices as the same registered user or not. Exposing the client's currently connected device(s) (which would be required as volume adjustments are handled on the client-side) does not seem like a good idea either.
Using a hashed device-ID to share with other clients might solve the privacy issue but considering that a user can be online with different devices at the same time (once this feature is implemented) we'd have to send that hash with every single audio packet, which seems to incur too much overhead in terms of bandwidth to be worth it 👀

[Hartmnt]

In my scenario the clients would be still displayed as two different nodes in the tree, but both have the same registered ID, which is different from the client slot id, right?

I would rather have that than merging devices into the same visual client to be honest.

[Hartmnt]

The volume adjustment on the client side happens by certificate hash. The devices, all belonging to the same registered user, would still have a different hash. Just the same registered id. It should be possible to visually have two distinct clients for the same registered user with different certificate hashes.

My user-story only clashes with the last part of your proposal of joining the clients visually, if both are connected. A compromise would be to introduce different behavior only when both devices are connected simultaneously.

[Krzmbrzl]

In my scenario the clients would be still displayed as two different nodes in the tree, but both have the same registered ID, which is different from the client slot id, right?

If by client slot id you mean the client's session ID, then yes.

Not merging them in the UI seems rather pointless though. Why combine them in the first place then? Why treat them as the same user on the server but not in the UI? That seems inconsistent. 👀

The devices, all belonging to the same registered user, would still have a different hash.

That will have to be abstracted away as we really want it to be completely transparent what device a user is using. And this still doesn't solve the issue of having two (or more) devices online at the same time. How should the client know which adjustments to apply?

A compromise would be to introduce different behavior only when both devices are connected simultaneously.

How do you imagine things to work if only one device is connected? Registered users have a fixed name that gets assigned to them. So if you authenticate as that user, you'll get that name. And other than the name there isn't really anything that visually distinguishes clients 🤔

[Hartmnt]

That will have to be abstracted away as we really want it to be completely transparent what device a user is using

So there is the requirement that there will be absolutely no indication to the client whatsoever that the user is using a different device? Not even a number from 0 to N?

Then this will be sadly pretty much impossible... Just please take into consideration that users might want different volume adjustments for different devices, but do not know that yet. 🥲

[Krzmbrzl]

Not even a number from 0 to N?

Something like that would be possible. Only the certificate hash would likely have to be abstracted in order for old features to remain functioning.

[Hartmnt]

Something like that would be possible. Only the certificate hash would likely have to be abstracted in order for old features to remain functioning.

If it is for backwards compatibility, you could still send the device hash as an additional field which older clients would simply ignore.

[peacememories]

Hi, I'm new to this discussion, but this is also a feature I would really like. And I'd like to speak in support of differentiating the clients to other participants. Ideally I'd even want the ability to join different rooms from different devices at the same time, although that is not a necessary feature.

Not merging them in the UI seems rather pointless though. Why combine them in the first place then? Why treat them as the same user on the server but not in the UI? That seems inconsistent. 👀

For me, the main point of combining the clients is a shared identity and, most importantly, shared permissions. This benefit still exists even if the clients can be distinguished by other parties.

I hadn't thought about the possibility of multiple volume levels per user, but now that it has been mentioned, that would be useful sometimes.

[skobkin]

I hadn't thought about the possibility of multiple volume levels per user, but now that it has been mentioned, that would be useful sometimes.

Volume levels are most likely being stored locally, so no synchronization would be possible unless specifically implemented in the server.

[Hartmnt]

Ideally I'd even want the ability to join different rooms from different devices at the same time, although that is not a necessary feature.

I think that is not even remotely what this issue is going for. The idea would be that your multiple devices behave as a single client, only capable of joining a single channel at the same time. If you switch channels on one device, you switch on all devices.
My proposal here was that the devices would be displayed as individual "user indicators", such that different devices of the same user can have different client side properties (local volume, but also maybe local mute etc.). At no point was it intended to split those "dots", such that the different devices can join different channels.

Volume levels are most likely being stored locally, so no synchronization would be possible unless specifically implemented in the server.

Local volume levels are stored locally based on certificate hash. Always have been, and probably will be in the future.

If the last proposal of this issue is implemented as described, multiple vastly different devices (cellphone and PC for example) would be seen as the same client with the same certificate hash and same ID etc. While having some nice advantages, it would mean that you would NOT be able to set a different local volume adjustment per device.

And this will more often than not either mean that the client will quickly change between much too quite or - even worse - much too loud.

---

There was also some internal discussion about this a year ago:

Using network messages it should be pretty easy to tell the clients, which devices are currently connected per user. Making clients aware of the existence of multiple devices was deemed to not be the problem.
The main problem here is how to address the UI ambiguity of "What happens if the user drags the local volume adjustment slider, if the target is connected with multiple devices"

We have noticed that separate user indicators are not strictly necessary to fix the local volume adjustment problem, if we get a little creative. Imagine we have implemented multiple device support, but there is only one user entry in the list for every user and we can not change that.

1. Store the local volume based on the set of devices that are currently connected. Instead of "volume of user0" we would have "volume of user0-device0-device1" and "volume of user0-device1" etc. as separate entries. When you adjust the local volume, you adjust it for the currently connected set of devices for the user.

   • Advantage: Devices not simultaneously connected behave as "separate users" as far as local volume is concerned. Works well in the multiple device scenario, if the user does not constantly adds or removes devices. (Switching devices is not a problem). No need for a audio packet scheme extension.
   • Disadvantage: A new device combination would mean the local volume adjustment would be entirely fresh. If connected devices constantly change on the fly, this could get annoying. Can not have different volumes for different devices, when multiple devices are connected.

2. Keep track of devices of a user separately "user0-device0", "user0-device1" etc. (not as a set) and adjust all currently connected devices when the slider is dragged.

   • Advantage: Devices not simultaneously connected behave as "separate users" as far as local volume is concerned. Works well in the multiple device scenario, if the user does not constantly adds or removes devices. (Switching devices is not a problem). No need for a audio packet scheme extension.
   • Disadvantage: Slightly confusing if multiple connected devices have a vastly different audio record volume -> Might be impossible to properly adjust. New devices would need volume adjustment, but adding or removing "known" devices would not. Can not adjust the volume for individual devices if and only if there are multiple connected at the same time

3. Keep track of devices of a user separately "user0-device0", "user1-device1" etc. (not as a set) and track the last device the user was last talking from and adjust only that device

   • Advantage: Devices not simultaneously connected behave as "separate users" as far as local volume is concerned.
     Works very well in the multiple device scenario: User would have to adjust at most once for any device.
   • Disadvantage: The volume might jump when the user switches devices on the fly. At least until the receiver has adjusted the volume for all devices. Would need to implement a device id into the audio packet scheme, because we need to know the audio packet -> device relation client-side.

[Krzmbrzl]

We talked about how we could resolve the issue with volume adjustments on a per-device basis and what we came up with is something that we believe would be a general improvement to how Mumble works: #6428

We believe that this feature should implicitly take care of all issues regarding different devices potentially having different mic configurations.