BC Media Format - Streaming audio to the cameras #187

entropin · 2021-08-01T13:37:25Z

entropin
Aug 1, 2021

Hi, Am working towards adding "TalkAbility" functionality to Neolink, but am having a problem getting my camera to accept the audio bites am sending @QuantumEntangledAndy has helped me out quite a bit, but I wanted to see if anyone else got some insights regarding the audio recordings.

This is the <TalkConfig> that my code and the client is sending to the camera:

<?xml version="1.0" encoding="UTF-8" ?>
<body>
<TalkConfig version="1.1">
<channelId>0</channelId>
<duplex>FDX</duplex>
<audioStreamMode>followVideoStream</audioStreamMode>
<audioConfig>
<audioType>adpcm</audioType>
<sampleRate>16000</sampleRate>
<samplePrecision>16</samplePrecision>
<lengthPerEncoder>1024</lengthPerEncoder>
<soundTrack>mono</soundTrack>
</audioConfig>
</TalkConfig>
</body>

Here is a short WireShark capture of me streaming to the camera via the old Reolink client.
https://drive.google.com/file/d/14256PW40V2ZrgIHPISStMmbYJV0gOkoY/view?usp=sharing

The packages that the client is sending are in BC Media format, like how the camera is sending video to the client. So the most likely is that i need to encode by audio to DVI-4 adpcm.

This is how I convert the audio am working with right now ffmpeg -i bensound-dubstep.mp3 -f wav -ar 16000 -ac 1 -acodec adpcm_ms OUTPUT.wav

Wikipedia list that ffmpeg has DVI support but i cant find any information that i can understand on how to specify DVI4. Dose someone know more about this?

Finally am also am confused about this part of the protocol documentation:

`### ADPCM

After the ADPCM header is another header of the form

2 bytes Magic either 0x00 0x01 or 0x00 0x7a
2 Bytes DVI-4 Block size
`
What is a Block in this context? I cant find any good information about Block size and audio encoding :/

Any and all insight is welcomed, right now it feels like am just throwing random bits at the camera haha

Answered by QuantumEntangledAndy

Aug 1, 2021

Ok so I do not believe ffmpeg supports dvi4 adpcm (it is not the same as DVI movies). You can check which ones it supports with assuming linux/osx term with grep

ffmpeg -codecs | grep adpcm

The closest format that ffmpeg supports is the IMA one which is the same as DVI but with a different block header.

So few things support it that we created a adpcm decoder in rust src/bc_protocol/adpcm.rs. It may be necessary to write the encoder then you can just use more standard formats.

It is definitely NOT adpcm_ms

Now about the block part

A block is a small chunk of audio. Every byte in adpcm depends on the previous byte. So what do you do if you want to start playing at 80% through the track? D…

View full answer

QuantumEntangledAndy · 2021-08-01T14:43:36Z

QuantumEntangledAndy
Aug 1, 2021
Collaborator

Ok so I do not believe ffmpeg supports dvi4 adpcm (it is not the same as DVI movies). You can check which ones it supports with assuming linux/osx term with grep

ffmpeg -codecs | grep adpcm

The closest format that ffmpeg supports is the IMA one which is the same as DVI but with a different block header.

So few things support it that we created a adpcm decoder in rust src/bc_protocol/adpcm.rs. It may be necessary to write the encoder then you can just use more standard formats.

It is definitely NOT adpcm_ms

Now about the block part

A block is a small chunk of audio. Every byte in adpcm depends on the previous byte. So what do you do if you want to start playing at 80% through the track? Do you load every byte before? No instead you only need to go back to the beginning of the block. In the block header is effectively a checkpoint saying this byte is this value no need to look further backwards to calculate it.

How often this checkpoint is made depends on how the adpcm was implemented and may be configurable. You may want to read our discussion in #68 this is where we actually reverse engineered this stuff and I learnt all I know about ADPCM during that PR is the BC Media format.

@twistedddx I remember you are good at this sort of less common audio formats do you have any recommended way to encode in DVI4 adpcm?

69 replies

twistedddx Aug 5, 2021

New commit for 8000 works good.

Btw in your typo correction you added 2x comments that include the typo ;)

QuantumEntangledAndy Aug 5, 2021
Collaborator

Typo fixed. I am going to squash those commits so if you pull this branch again it will probably error. Best to remove and repull the branch

entropin Aug 5, 2021
Author

Nice, works fine for me! :) I Do have C# code with login/decryption and fully working talk functionality now if anyone is interested, but I think I'll stick with the Rust code for my project. I just have to learn GStreamer.

With C# I feel like I need to fight the bytes at every level, clearly not meant for bit manipulation :) Thanks a thousand for all the help and insight! I'll be back with a PR for pan/zoom when am feel like I got the basics of Rust down.

QuantumEntangledAndy Aug 5, 2021
Collaborator

I am glad it works well for you too. It's rather fun to work on this sort of thing. Good luck with the pan and if you need more help decoding you know where to find us :)

twistedddx Aug 5, 2021

Maybe we(98% your effort @QuantumEntangledAndy ) stole the puzzle win fun from @entropin

Oops sorry for ruining the game entropion :(

entropin · 2021-08-03T06:58:51Z

entropin
Aug 3, 2021
Author

Recompiling the audio files
So, just to help me fully understand the data I wished to recompile the audio file from the capture, this is my understanding as of now. Please help me make it inline with reality :D

I'll take the first 202 pack, find the position of the payload, in this case starting at byte poss 132.
Splitting that binary payload stream every time it finds the bytes 01bw (30317762) this will give me 3 chunks of bytes in the case of the first 202 package
Removing the first 8 byte from every chunk, to get rid of the BCMedia header (The 01bw is still present on every chunk after the split).
Removing 4 more bytes since we don't care about the second magic.

Q: Should i also remove the adpcm state? or should that still be in the recompiled audio file?

5, concatenating all the bytes and saving to file (repeat the process and writing more data to the file for every 202 type found in the capture)?

Is this correct ?

Also, do we know of any desktop media player that can play this format (to validate that the file works before sending it back to the camera)?

20 replies

twistedddx Aug 3, 2021

you likely would want to use adpcmenc to create this adpcm data from more common formats.
Eg
Feed Neolink a wav, aac or MP3.
Neolink uses gstreamer to convert to DVI adpcm

QuantumEntangledAndy Aug 3, 2021
Collaborator

Yes see if I can get it to stream from the microphone or stdin too.

Also that will let us match the format to exactly what TalkAbility reports is the best format.

entropin Aug 4, 2021
Author

Oki, so is still having problems playing back the files created from the dump on my computer, the files generated seams fine and all bytes lines up. So I have a few questions.

This command generates the adpcm file. Is the consensus that this now should be blockalign=516? and not 512?

gst-launch-1.0 filesrc location=data.aac ! decodebin ! audio/x-raw ! audioresample ! "audio/x-raw,rate=16000" ! audioconvert ! "audio/x-raw,channels=1" ! queue ! adpcmenc blockalign=512 layout=dvi ! filesink location=out.adpcm

Anyhow, how do I simply reverse this command? Turning the file back to aac (gst documentation is wast.., bin stuck her for a while).

@twistedddx You mentioned wave headers, can I just stick that in the begining of to the out.adpcm file and play it in vlc? Or dose the file out.adpcm have a header that i need to replace?

entropin Aug 4, 2021
Author

the wave header you mentioned worked like a charm on my recompiled audio, than you!
52494646 A4910000 57415645 666D7420 10000000 11000100 803e0000 00FA7D00 04020400 64617461 ec372200

QuantumEntangledAndy Aug 5, 2021
Collaborator

Yes consensus is adpcm is 516 block-align that seems to work and give good audio on the camera while 512 did not.

This is my encoding

gst-launch-1.0 filesrc location=target/aac/nero.aac ! decodebin ! audioconvert ! audioresample ! "audio/x-raw,rate=16000,channels=1" ! volume volume=2.0 ! adpcmenc blockalign=516 layout=dvi ! filesink location=target/aac/out.adpcm

Playing it from the out.adpcm is difficult. It lacks a header that describes the data so most progams like gst and ffmpeg cannot deduce it's format.

You can try this one

gst-launch-1.0 filesrc location=target/aac/out.adpcm ! typefind force-caps=audio/x-adpcm,layout=dvi,block_align=516,channels=1,rate=16000 ! adpcmdec ! audioconvert ! autoaudiosink

Which should use the force-caps part of typefind to set the format but I have never gotten it to work. :(

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

BC Media Format - Streaming audio to the cameras #187

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 2 comments 89 replies

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{title}}

{{editor}}'s edit

{{editor}}'s edit

{{title}}

{{title}}

Select a reply

BC Media Format - Streaming audio to the cameras #187

entropin Aug 1, 2021

Replies: 2 comments · 89 replies

QuantumEntangledAndy Aug 1, 2021 Collaborator

twistedddx Aug 5, 2021

QuantumEntangledAndy Aug 5, 2021 Collaborator

entropin Aug 5, 2021 Author

QuantumEntangledAndy Aug 5, 2021 Collaborator

twistedddx Aug 5, 2021

entropin Aug 3, 2021 Author

twistedddx Aug 3, 2021

QuantumEntangledAndy Aug 3, 2021 Collaborator

entropin Aug 4, 2021 Author

entropin Aug 4, 2021 Author

QuantumEntangledAndy Aug 5, 2021 Collaborator

entropin
Aug 1, 2021

Replies: 2 comments 89 replies

QuantumEntangledAndy
Aug 1, 2021
Collaborator

QuantumEntangledAndy Aug 5, 2021
Collaborator

entropin Aug 5, 2021
Author

QuantumEntangledAndy Aug 5, 2021
Collaborator

entropin
Aug 3, 2021
Author

QuantumEntangledAndy Aug 3, 2021
Collaborator

entropin Aug 4, 2021
Author

entropin Aug 4, 2021
Author

QuantumEntangledAndy Aug 5, 2021
Collaborator