High resolutions don't work

[DarkoGNU]

My phone is Galaxy S7, Android 8.0.0 stock. 352x288, 30 FPS seems to be the best I can get.

Here's what I get when the preview in VLC works, but FPS and resolution don't match:
[darkognu@DarkoGNU ~]$ vlc "rtsp://192.168.0.104:8086?h264=5000-60-480-360"
VLC media player 3.0.9.2 Vetinari (revision 3.0.9.2-0-gd4c1aefe4d)
[0000555cb84e65b0] main libvlc: Running vlc with the default interface. Use 'cvlc' to use vlc without interface.
Created new TCP socket 26 for connection
[00007f279000b6b0] avcodec decoder: Using NVIDIA VDPAU Driver Shared Library 440.82 Wed Apr 1 19:39:06 UTC 2020 for hardware decoding
[00007f279000b6b0] main decoder error: buffer deadlock prevented

Here's what I get when it doesn't work at all:
[darkognu@DarkoGNU ~]$ vlc "rtsp://192.168.0.104:8086?h264=5000-60-1920-1080"
VLC media player 3.0.9.2 Vetinari (revision 3.0.9.2-0-gd4c1aefe4d)
[0000561997f515b0] main libvlc: Running vlc with the default interface. Use 'cvlc' to use vlc without interface.
Created new TCP socket 26 for connection
[00007f08ec001140] live555 demux error: SETUP of'video/H264' failed 500 Internal Server Error
[00007f08ec001140] live555 demux error: Nothing to play for rtsp://192.168.0.104:8086?h264=5000-60-1920-1080
[00007f08ec002f50] satip stream error: Failed to setup RTSP session
Created new TCP socket 27 for connection
[00007f08ec009710] live555 demux error: SETUP of'video/H264' failed 500 Internal Server Error

And it repeats. Any ideas?

[javaing]

maybe try to mark the mMode =, that line:
if (mQuality.resX>=640) {
// Using the MediaCodec API with the buffer method for high resolutions is too slow
Log.e(TAG,"Using the MediaCodec API with the buffer method for high resolutions is too slow");
//mMode = MODE_MEDIARECORDER_API;
}