Skip to content

Commit

Permalink
Remove separate codec, HDR, and remote bitrate adjustments
Browse files Browse the repository at this point in the history 
We currently scale bitrate based on both remote vs local, SDR vs HDR, and H.264 vs
HEVC vs AV1. This has led to a lot of user confusion wondering why the bitrate
doesn't seem to match their selection in some configurations.

In H.264 local streams, we will currently overshoot the selected bitrate by about
20% due to FEC, while remote streams will be right around the selected bitrate due
to remote-specific FEC bitrate adjustments.

HEVC and AV1 streams (as configured by most clients) basically behave similarly
between local and remote, since the codec bitrate adjustment factor of 75% is nearly
the same as the FEC bitrate adjustment factor of 80%. However, this adjustment was
only performed for SDR streams so local HDR streams would overshoot like H.264.

This change cleans up all this mess by using a single non-codec-specific video
bitrate adjustment for FEC in all cases. It also allows Sunshine to perform the FEC
adjustment on its end if the default FEC value of 20% has been overridden by the
user or if we implement dynamic FEC support in the future.

The net result is HEVC and AV1 SDR streams will only see a tiny bitrate increase,
but HDR and H.264 may see noticable 20% bitrate reductions that may require the
user to adjust their bitrate setting to reach the effective value they got before.
However, the new behavior should be more intuitive for users going forward since
changing codecs, using a VPN, or enabling HDR won't cause significant changes to the
video bitrate.
  • Loading branch information
@cgutman
cgutman committed Dec 4, 2023
1 parent 5de4a5b commit 3ed3ba6
Show file tree
Hide file tree
Showing 5 changed files with 21 additions and 48 deletions.
2 changes: 0 additions & 2 deletions src/Connection.c
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,6 @@ bool HighQualitySurroundSupported;
bool HighQualitySurroundEnabled;
OPUS_MULTISTREAM_CONFIGURATION NormalQualityOpusConfig;
OPUS_MULTISTREAM_CONFIGURATION HighQualityOpusConfig;
int OriginalVideoBitrate;
int AudioPacketDuration;
bool AudioEncryptionEnabled;
bool ReferenceFrameInvalidationSupported;
Expand Down Expand Up @@ -258,7 +257,6 @@ int LiStartConnection(PSERVER_INFORMATION serverInfo, PSTREAM_CONFIGURATION stre
memset(&LocalAddr, 0, sizeof(LocalAddr));
NegotiatedVideoFormat = 0;
memcpy(&StreamConfig, streamConfig, sizeof(StreamConfig));
OriginalVideoBitrate = streamConfig->bitrate;
RemoteAddrString = strdup(serverInfo->address);

// The values in RTSP SETUP will be used to populate these.
Expand Down
1 change: 0 additions & 1 deletion src/Limelight-internal.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,6 @@ extern bool HighQualitySurroundSupported;
extern bool HighQualitySurroundEnabled;
extern OPUS_MULTISTREAM_CONFIGURATION NormalQualityOpusConfig;
extern OPUS_MULTISTREAM_CONFIGURATION HighQualityOpusConfig;
extern int OriginalVideoBitrate;
extern int AudioPacketDuration;
extern bool AudioEncryptionEnabled;
extern bool ReferenceFrameInvalidationSupported;
Expand Down
16 changes: 3 additions & 13 deletions src/Limelight.h
View file
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,9 @@ typedef struct _STREAM_CONFIGURATION {
// FPS of the desired video stream
int fps;

// Bitrate of the desired video stream (audio adds another ~1 Mbps)
// Bitrate of the desired video stream (audio adds another ~1 Mbps). This
// includes error correction data, so the actual encoder bitrate will be
// about 20% lower when using the standard 20% FEC configuration.
int bitrate;

// Max video packet size in bytes (use 1024 if unsure). If STREAM_CFG_AUTO
Expand All @@ -65,18 +67,6 @@ typedef struct _STREAM_CONFIGURATION {
// See VIDEO_FORMAT constants below.
int supportedVideoFormats;

// Specifies the percentage that the specified bitrate will be adjusted
// when an HEVC stream will be delivered. This allows clients to opt to
// reduce bandwidth when HEVC is chosen as the video codec rather than
// (or in addition to) improving image quality.
int hevcBitratePercentageMultiplier;

// Specifies the percentage that the specified bitrate will be adjusted
// when an AV1 stream will be delivered. This allows clients to opt to
// reduce bandwidth when AV1 is chosen as the video codec rather than
// (or in addition to) improving image quality.
int av1BitratePercentageMultiplier;

// If specified, the client's display refresh rate x 100. For example,
// 59.94 Hz would be specified as 5994. This is used by recent versions
// of GFE for enhanced frame pacing.
Expand Down
14 changes: 1 addition & 13 deletions src/RtspConnection.c
View file
Original file line number Diff line number Diff line change
Expand Up @@ -785,7 +785,7 @@ int performRtspHandshake(PSERVER_INFORMATION serverInfo) {
// 2. The audio decoder has not declared that it is slow
// 3. The stream is either local or not surround sound (to prevent MTU issues over the Internet)
LC_ASSERT(StreamConfig.streamingRemotely != STREAM_CFG_AUTO);
if (OriginalVideoBitrate >= HIGH_AUDIO_BITRATE_THRESHOLD &&
if (StreamConfig.bitrate >= HIGH_AUDIO_BITRATE_THRESHOLD &&
(AudioCallbacks.capabilities & CAPABILITY_SLOW_OPUS_DECODER) == 0 &&
(StreamConfig.streamingRemotely != STREAM_CFG_REMOTE || CHANNEL_COUNT_FROM_AUDIO_CONFIGURATION(StreamConfig.audioConfiguration) <= 2)) {
// If we have an RTSP URL string and it was successfully parsed and copied, use that string
Expand Down Expand Up @@ -906,12 +906,6 @@ int performRtspHandshake(PSERVER_INFORMATION serverInfo) {
}
else {
NegotiatedVideoFormat = VIDEO_FORMAT_AV1_MAIN8;

// Apply bitrate adjustment for SDR AV1 if the client requested one
if (StreamConfig.av1BitratePercentageMultiplier != 0) {
StreamConfig.bitrate *= StreamConfig.av1BitratePercentageMultiplier;
StreamConfig.bitrate /= 100;
}
}
}
else if ((StreamConfig.supportedVideoFormats & VIDEO_FORMAT_MASK_H265) && strstr(response.payload, "sprop-parameter-sets=AAAAAU")) {
Expand All @@ -926,12 +920,6 @@ int performRtspHandshake(PSERVER_INFORMATION serverInfo) {
}
else {
NegotiatedVideoFormat = VIDEO_FORMAT_H265;

// Apply bitrate adjustment for SDR HEVC if the client requested one
if (StreamConfig.hevcBitratePercentageMultiplier != 0) {
StreamConfig.bitrate *= StreamConfig.hevcBitratePercentageMultiplier;
StreamConfig.bitrate /= 100;
}
}
}
else {
Expand Down
36 changes: 17 additions & 19 deletions src/SdpGenerator.c
View file
Original file line number Diff line number Diff line change
Expand Up @@ -258,7 +258,7 @@ static PSDP_OPTION getAttributesList(char*urlSafeAddr) {
int audioChannelCount;
int audioChannelMask;
int err;
int bitrate;
int adjustedBitrate;

// This must have been resolved to either local or remote by now
LC_ASSERT(StreamConfig.streamingRemotely != STREAM_CFG_AUTO);
Expand Down Expand Up @@ -289,52 +289,51 @@ static PSDP_OPTION getAttributesList(char*urlSafeAddr) {
err |= addAttributeString(&optionHead, "x-nv-video[0].timeoutLengthMs", "7000");
err |= addAttributeString(&optionHead, "x-nv-video[0].framesWithInvalidRefThreshold", "0");

// 20% of the video bitrate will added to the user-specified bitrate for FEC
adjustedBitrate = (int)(StreamConfig.bitrate * 0.80);

// Use more strict bitrate logic when streaming remotely. The theory here is that remote
// streaming is much more bandwidth sensitive. Someone might select 5 Mbps because that's
// really all they have, so we need to be careful not to exceed the cap, even counting
// things like audio and control data.
if (StreamConfig.streamingRemotely == STREAM_CFG_REMOTE) {
// 20% of the video bitrate will added to the user-specified bitrate for FEC
bitrate = (int)(OriginalVideoBitrate * 0.80);

// Subtract 500 Kbps to leave room for audio and control. On remote streams,
// GFE will use 96Kbps stereo audio. For local streams, it will choose 512Kbps.
if (bitrate > 500) {
bitrate -= 500;
if (adjustedBitrate > 500) {
adjustedBitrate -= 500;
}
}
else {
bitrate = StreamConfig.bitrate;
}

// If the calculated bitrate (with the HEVC multiplier in effect) is less than this,
// use the lower of the two bitrate values.
bitrate = StreamConfig.bitrate < bitrate ? StreamConfig.bitrate : bitrate;

// GFE currently imposes a limit of 100 Mbps for the video bitrate. It will automatically
// impose that on maximumBitrateKbps but not on initialBitrateKbps. We will impose the cap
// ourselves so initialBitrateKbps does not exceed maximumBitrateKbps.
bitrate = bitrate > 100000 ? 100000 : bitrate;
adjustedBitrate = adjustedBitrate > 100000 ? 100000 : adjustedBitrate;

// We don't support dynamic bitrate scaling properly (it tends to bounce between min and max and never
// settle on the optimal bitrate if it's somewhere in the middle), so we'll just latch the bitrate
// to the requested value.
if (AppVersionQuad[0] >= 5) {
snprintf(payloadStr, sizeof(payloadStr), "%d", bitrate);
snprintf(payloadStr, sizeof(payloadStr), "%d", adjustedBitrate);

err |= addAttributeString(&optionHead, "x-nv-video[0].initialBitrateKbps", payloadStr);
err |= addAttributeString(&optionHead, "x-nv-video[0].initialPeakBitrateKbps", payloadStr);

err |= addAttributeString(&optionHead, "x-nv-vqos[0].bw.minimumBitrateKbps", payloadStr);
err |= addAttributeString(&optionHead, "x-nv-vqos[0].bw.maximumBitrateKbps", payloadStr);

// Send the configured bitrate to Sunshine hosts, so they can adjust for dynamic FEC percentage
if (IS_SUNSHINE()) {
snprintf(payloadStr, sizeof(payloadStr), "%u", StreamConfig.bitrate);
err |= addAttributeString(&optionHead, "x-ml-video.configuredBitrateKbps", payloadStr);
}
}
else {
if (StreamConfig.streamingRemotely == STREAM_CFG_REMOTE) {
err |= addAttributeString(&optionHead, "x-nv-video[0].averageBitrate", "4");
err |= addAttributeString(&optionHead, "x-nv-video[0].peakBitrate", "4");
}

snprintf(payloadStr, sizeof(payloadStr), "%d", bitrate);
snprintf(payloadStr, sizeof(payloadStr), "%d", adjustedBitrate);
err |= addAttributeString(&optionHead, "x-nv-vqos[0].bw.minimumBitrate", payloadStr);
err |= addAttributeString(&optionHead, "x-nv-vqos[0].bw.maximumBitrate", payloadStr);
}
Expand Down Expand Up @@ -447,8 +446,7 @@ static PSDP_OPTION getAttributesList(char*urlSafeAddr) {
}

if (AppVersionQuad[0] >= 7) {
// Decide to use HQ audio based on the original video bitrate, not the HEVC-adjusted value
if (OriginalVideoBitrate >= HIGH_AUDIO_BITRATE_THRESHOLD && audioChannelCount > 2 &&
if (StreamConfig.bitrate >= HIGH_AUDIO_BITRATE_THRESHOLD && audioChannelCount > 2 &&
HighQualitySurroundSupported && (AudioCallbacks.capabilities & CAPABILITY_SLOW_OPUS_DECODER) == 0) {
// Enable high quality mode for surround sound
err |= addAttributeString(&optionHead, "x-nv-audio.surround.AudioQuality", "1");
Expand All @@ -466,7 +464,7 @@ static PSDP_OPTION getAttributesList(char*urlSafeAddr) {

if ((AudioCallbacks.capabilities & CAPABILITY_SLOW_OPUS_DECODER) ||
((AudioCallbacks.capabilities & CAPABILITY_SUPPORTS_ARBITRARY_AUDIO_DURATION) != 0 &&
OriginalVideoBitrate < LOW_AUDIO_BITRATE_TRESHOLD)) {
StreamConfig.bitrate < LOW_AUDIO_BITRATE_TRESHOLD)) {
// Use 10 ms packets for slow devices and networks to balance latency and bandwidth usage
AudioPacketDuration = 10;
}
Expand Down

0 comments on commit 3ed3ba6

Please sign in to comment.