Using USBTMC for streaming data

[kmatch98]

tinyUSB questions for @hathach and @pigrew regarding USBTMC:

Many thanks for an awesome library. There is a lot of good capabilities here, now I just need to figure out how best to use them!

——————

I’m working to extend tannewt’s TinyLogicFriend project that uses tinyUSB to make microcontroller boards into basic digital logic analyzers.

Where I can use help: I am struggling to understand the best way of streaming the data from the device back to sigrok host via the USBTMC connection. I would like to get your recommendation on structuring this based on the USBTMC example that you provided.

(I have read that a “circular” buffer would be the best method for this kind of streaming operation, but I’m not sure if there is an easy way to implement in tinyUSB.)

My main questions:

• How do I best stream data from the logic pulse measurements onto the USB BulkIn endpoint, even when the number of data points may be unknown?
• How do I send an “end of data” command so the host knows the measurement data is finished?
• How do I let the host know that the measurements are lagging behind, so wait for a bit (this is probably just setting the right timeout on the host side0?
• How can the host best interrupt the measurements and then the buffer can get flushed?

I’d appreciate if you can suggest which callbacks functions (and the _iter function) should handle which key parts of the actions. I can use help understanding how to buffer properly and when to send the data. Even any general guidance is appreciated!

—————-

A few other questions, please don’t feel like you have to answer them all:

1. I don’t really understand the sequence of all the callbacks that occur (defined in the example in usbtmc_app.c)
2. The functiontud_usbtmc_msgBulkIn_request_cb seems to immediately send some data from the buffer back immediately. Why is this?

—————-

For reference I’m tracking my project here.

Since this is a test and measurement tool, I’m using the tinyUSB’s USBTMC example as the framework. (With BulkIn and BulkOut endpoints). I’m integrating the TinyLogicFriend with the sigrok logic analyzer software package.

Status: I have sigrok successfully sending and receiving SCPI text commands and measurement settings to the TinyLogicFriend via USBTMC. Now I am turning to the main communication for the logic analyzer data streaming. I would like to operate in either capture or streaming mode, but my first work is to setup streaming mode (where the data is measured from the pins and buffered onto the USB bus at the same time).

Capabilities I want:

• When host sends the “RUN” command, the device should measure and then stream measurement data to the host via BulkIn endpoint to the host.
• When sample count is reached, the device should send some message to let the host know the data is finished.
• At any time the host can send a STOP command and the device should stop measuring and clear the measurement buffer to the host (with same end of data message as in USB3.0 (xHCI) driver support #2 above)
• When not RUNning, respond to normal SCPI commands.

[kmatch98]

I forgot to mention, here is my hacked up USBTMC example with the SCPI parser integrated to decode and respond to commands: https://github.com/kmatch98/tinyusb/tree/master/examples/device/usbtmc/src

Now my main focus is on figuring out how to RUN and stream back the data.

[pigrew]

The TinyUSB USBTMC driver is not optimized for streaming operations (e.g., all of a message's data must be available before a message is sent, and data is copied between buffers a few times without DMA). However, it is definitely usable at lower data rates.

USBTMC is fundamentally different than CDC. CDC is two streams of data, in and out. USBTMC is message based. For SCPI, each IN message is a reply to an OUT query. In and OUT messages do not happen simultaneously, one has to wait for the other to finish (which is different than CDC). With that in mind, a good SCPI syntax can be developed.

How do I best stream data from the logic pulse measurements onto the USB BulkIn endpoint, even when the number of data points may be unknown?

I would suggest finding an existing commercial instrument, and copying its SCPI syntax. You'll find that results are generally broken down into smaller "messages". For example, the KeySight 34465A can store 10,000 points in its internal buffers. The host sends a "R?" query that will return as many datapoints as are already measured (and remove those from the internal buffer).

For USBTMC during a read, the host specifies a maximum message length. If the reply message is that exact length, it can be specified if the message is the full thing or a partial reply (EOM bit). There is a limitation in the TinyUSB USBTMC class driver that the full reply message must be known and in memory when the reply starts being sent. Since the host specifies the length, you shouldn't depend on the max length being short.

Circular buffers or ping-pong buffers are appropriate data structures for you to stream the data into. You could use the tinyUSB queue structure or an interrupt-safe circular buffer.

With the above class driver limitation, I would suggest following what the 34465A does, and have a "R?" command that fetches all available already streamed data or a "DATA:REMOVE?" command which will first waits for a particular amount of data to be available before initiating the reply. You could implement a "DATA:POINts:EVENt:THReshold" event register to allow a SRQ to be thrown when the proper amount of data is available before a read.

So, I can imagine a transaction like:

DATA:POINts:EVENt:THReshold 1000
<configure SRQ to trigger when threshold is reached>
SAMP:COUNT 10000000 # max number of samples
RUN
<wait for SRQ>
R? # read available data
<wait for SRQ>
R? # read available data
STOP
R? # read remainder of data
STAT:QUES:COND? # check for data buffer overflow

How do I let the host know that the measurements are lagging behind, so wait for a bit (this is probably just setting the right timeout on the host side0?

The USBTMC device can either NOT reply until the data is available (causing the host viRead to block) or it can use a service-request (SRQ), and the host waits for the SRQ before it requests data. In general, avoid timeouts for SCPI read/write. (Timeouts are OK when waiting for a SRQ).

How do I send an “end of data” command so the host knows the measurement data is finished?

Use a bit of a status register, for a "complete" indication.

Be aware that a lot of the USBTMC class driver is trying to be able to handle abort scenarios. USBTMC messages can be aborted at any time, using a "request and acknowledge" system.

So, study the USBTMC and USB488 specifications, and study example instruments. Streaming is more complicated than how I've used the class driver so far.

I hope that this will clear up some of the issues you've been having. Without me actually writing your program, it's hard to say exactly what you need to write in your "usbtmc_iter" function. I had contemplated including libscpi into the example, but at that time I thought I shouldn't include any external dependencies. It might be a good idea for the future, so that the example is more easily used.

[kmatch98]

Thanks for the kind reply, this helps clarify a lot of questions. Here's some things that got clarified for me along with a few questions:

---

Buffering (this is my primary confusion)

The TinyUSB USBTMC driver is not optimized for streaming operations (e.g., all of a message's data must be available before a message is sent, and data is copied between buffers a few times without DMA). However, it is definitely usable at lower data rates.

I see this note that you included in the tud_usbtmc_transmit_dev_msg_data:

// called from app
// We keep a reference to the buffer, so it MUST not change until the app is
// notified that the transfer is complete.
// length of data is specified in the hdr.

// We can't just send the whole thing at once because we need to concatanate the
// header with the data.
bool tud_usbtmc_transmit_dev_msg_data(
    const void * data, size_t len,
    bool endOfMessage,
    bool usingTermChar)
{

and later in the function...

memcpy((uint8_t*)(usbtmc_state.ep_bulk_in_buf) + headerLen, data, dataLen);

Do I need to verify the data is sent before reusing the input data buffer? If so, which callback detects that the message was actually sent? (Maybe I will need two buffers if I need to wait for the buffer to be sent?)

With seeing the memcpy, I assumed that I would be free to reuse the data buffer after calling tud_usbtmc_transmit_dev_msg_data, but your note in the comment confuses me. Any clarification is appreciated!

---

Buffer construction and example?

Circular buffers or ping-pong buffers are appropriate data structures for you to stream the data into. You could use the tinyUSB queue structure or an interrupt-safe circular buffer.

Is there a suggested example for this?

---

End of Measurement signal from device to host

Once I trigger the device to "RUN", it can keep pushing data on BulkIn if I keep the third parameter (bool endOfMessage) to the transmit function as false:

tud_usbtmc_transmit_dev_msg_data(sample, sample_len, false, false);

Once the measurements are complete, the device can send the endOfMessage third parameter as true. Then on the next host read, the host will get nothing back so it can detect the end of the datastream.

---

SRQ's

Currently, I'm not following your suggested method of SRQ call and response. Perhaps not including this will cause some control flow issues due to buffer overruns. Will need to consider how to deal with that.

Right now I just send a "RUN" command to the device and the host just continuously reads the BulkIn data until there is no more data. I'm suspect there are some buffers in between (?). Maybe I will need to watch out for buffer overflows and/or timeouts.

---

Thanks again for this awesome contribution and for your clarifications.

[pigrew]

Do I need to verify the data is sent before reusing the input data buffer? If so, which callback detects that the message was actually sent? (Maybe I will need two buffers if I need to wait for the buffer to be sent?)

If you read a few lines down, the memcpy only copies up to length sizeof(usbtmc_state.ep_bulk_in_buf). If your buffer (plus the message header) is longer than this, it will be copied later after USB processes the first ep_bulk_in_buf length of data.

Buffer construction and example?

Circular buffers or ping-pong buffers are appropriate data structures for you to stream the data into. You could use the tinyUSB queue structure or an interrupt-safe circular buffer.

Is there a suggested example for this?

No, I don't really have an example in mind. But, now that I look at it, TinyUSB has a tu_fifo (which I have not used) that should do what you want in terms of buffering. It's used in the CDC class examples.

End of Measurement signal from device to host

Once I trigger the device to "RUN", it can keep pushing data on BulkIn if I keep the third parameter (bool endOfMessage) to the transmit function as false:

tud_usbtmc_transmit_dev_msg_data(sample, sample_len, false, false);

Once the measurements are complete, the device can send the endOfMessage third parameter as true. Then on the next host read, the host will get nothing back so it can detect the end of the datastream.

Yes, I think this could work. I believe the host VISA library would read the entire message (possibly multiple transactions) before the viRead() returns and the host application gets anything back (but I could be wrong).

Remember that nothing is "push from the device" in SCPI. Everything is a response to a query from the host. Once you "RUN", you should have a "FETCH?" or "R?" command that gets the data back.

If you want aborts to be easier for host-side programming, your fetch command should happen only once the data is actually available to be read back.

bool tud_usbtmc_msgBulkIn_complete_cb(void); is called once the transaction is complete.

I didn't think about having one message contain multiple transactions when I wrote the driver (and have not tried it), but it should work....

SRQ's

Currently, I'm not following your suggested method of SRQ call and response. Perhaps not including this will cause some control flow issues due to buffer overruns. Will need to consider how to deal with that.

Right now I just send a "RUN" command to the device and the host just continuously reads the BulkIn data until there is no more data. I'm suspect there are some buffers in between (?). Maybe I will need to watch out for buffer overflows and/or timeouts.

The SRQ is an out-of-band signaling interface (like an "interrupt request"). SCPI devices have a "status model" which stores bit-fields of event and status registers. SCPI allows devices to be configured to raise the SRQ when particular event or status bits are set. SRQ was signaled through a dedicated wire in GPIB, and USBTMC has an adapted form of it.

So, I'm suggesting that you use the status model to set a bit when "enough" data is buffered in the MCU (like when the buffer is half full). The host could detect that service request and then ask for the data. An bit could also be created to signify when the logic analyzer acquisition is complete, and also if there was a MCU buffer overrun.

[kmatch98]

Thanks so much for the detailed suggestions and for giving me your ideas on how to solve my issues. Special thanks for pointing out that fifo buffer. I definitely will check that out to see whether I can make use of that.

Thanks for the help for a relative newbie to the USB world. And thanks again for making all these cool tools to use.

[kmatch98]

bool tud_usbtmc_msgBulkIn_complete_cb(void); is called once the transaction is complete.

I didn't think about having one message contain multiple transactions when I wrote the driver (and have not tried it), but it should work....

I'm trying to send multiple messages with tud_usbtmc_transmit_dev_msg_data(). I think I'm running into the issue where I am calling tud_usbtmc_transmit_dev_msg_data() before it sent the previous data. Per your previous comment, I don't think this function is currently setup to queue up a bunch of messages. Seems like there are few options

• make a version of tud_usbtmc_transmit_dev_msg_data() that automatically pushes this onto a buffer
• add a buffer to something like tud_usbtmc_transmit_dev_msg_data()
• make tud_usbtmc_transmit_dev_msg_data() give a callback so that something else can know when this function has sent its data so I can send more.

I definitely understand that your code was designed more for the call/response and I'm trying to something outside of that. At this point I'm wondering whether I should fall back to the host call/device response flow control, or if I should spend some more time on this "open-ended" streaming of BulkIn data. On reflection, the call-response is more typical and already working in the package, so it's probably the best idea. I'll sleep on it and probably focus my efforts in that direction. Thanks for pointing mein that direction, since it's probably the easiest path to trod (rather than inventing something totally new).

Again, thanks for your comments clarifying the capabilities of the USBTMC functions.

[kmatch98]

I’m making this as answered. Thank you!

[kmatch98]

bool tud_usbtmc_msgBulkIn_complete_cb(void); is called once the transaction is complete.

I didn't think about having one message contain multiple transactions when I wrote the driver (and have not tried it), but it should work....

@pigrew I spent some time trying to send multiple messages with tud_usbtmc_transmit_dev_msg_data(). I think I'm running into the issue where I am calling tud_usbtmc_transmit_dev_msg_data() before it sent the previous data. Per your previous comment, I don't think this function is currently setup to queue up a bunch of messages. Seems like there are few options:

• make a version of tud_usbtmc_transmit_dev_msg_data() that automatically pushes this onto a buffer
• add a buffer to something like tud_usbtmc_transmit_dev_msg_data()
• make tud_usbtmc_transmit_dev_msg_data() give a callback so that something else can know when this function has sent its data so I can send more.
• use call and response method that you suggested

I definitely understand that your code was designed more for the call/response and I'm trying to something outside of that. At this point I'm leaning toward following your suggestion and use the host call/device response flow control, or if I should spend some effort on this "open-ended" streaming of BulkIn data.

On reflection, the call-response is more typical and already working in the package, so it's probably the best idea. I'll sleep on it and probably focus my efforts in that direction. Thanks for pointing mein that direction, since it's probably the easiest path to trod (rather than inventing something totally new).

Again, thanks for your comments clarifying the capabilities of the USBTMC functions.

[pigrew]

I've been reflecting this discussion, and unfortunately do not know what the true answer is.

You are correct that the class driver has no queue. You must wait for an abort or tud_usbtmc_msgBulkIn_complete_cb callback before sending more data. (If that callback isn't working, that's a bug, and should be fixed). I wrote the class using a fairly dumb API for the USBTMC operations, and it'd take a bunch of thinking to improve the API. Some limitations of TinyUSB at the time (like no DMA abstration and no high-speed USB support).

I also have not been able to find any examples of high data-rate streaming using USBTMC. Perhaps this is because the protocol is half-duplex (as far as I can tell from the specification). One cannot have multiple actions happening simultaneously. The device is only in a write state or a read state, but can't accept new bulk-out commands while there is bulk-in data happening. Performance will also be limited because there must be a REQUEST_DEV_DEP_MSG_IN on the bulk-out pipe for every message read back.

The other concern is if you want to be SCPI compliant. If not, you could make it so an "unsolicited" read will always return the buffered data. This would speed things up a bit since there would be no need to send the "R?" command before each message getting data back.

Another difficulty is that I'm not sure how well the host-side VISA libraries will support reads with large messages. I'm only familiar with read-operations that are blocking. I don't think VISA has any sort of streaming API, so the host side support isn't really available, either.

One possibility is to, as you suggest, make usbtmc_transmit_dev_msg_data take a FIFO (and the expected buffer size) instead of a buffer. In that case, I'm not sure if it'd needed to split the message into multiple transfers (or not). I'm completely open to this plan (and it certainly will help streaming applications). Propose an API if you want to go this route. It's double-good if the device driver API could be extended to support reading from the FIFO (so there would be only a single copy in the TinyUSB code when the device drivers copies the data from the application FIFO to the device buffer).

So, what would I suggest:

Live with the speed limitations induced by a SCPI-compliant USBTMC call-response command set. It will never be as fast as a purpose-built USB class, but should be good enough for most cases.

Define a command set that is usable for both streaming and block-buffers. Start by implementing block-buffers, and see how fast you can get. Optimize transfer rate and implement streaming if and when you realize that you have a performance bottleneck. I expect that implementing DMA for memory-to-memory copies will be the first thing to try.