Need to read a serial data stream and process

[Steve Weidman]

Good Morning.

I'm new to LabView, having just gotten the version 8 full development suite.

Overall, everything has worked out great, but I'm having trouble reading a processing a serial string.

Here is my situation:

I have a three wire serial port (no flow control) running at 38400.

Twice per second, my instrument sends out a comma delimited data stream with a CR on the end. I need to capture the whole data string and take it apart at the commas. I know how to take it apart, so no problems there.

By using the "bytes at port" connected to the standard serial read, I can see my data. At times though, it gets "out of sync" for lack of a better term and doesn't capture the whole string.

I thought that maybe using events, might capture it, but no luck there at all.

Any ideas or futher clarifications for me?

Thanks!!

-Steve Weidman

[orko]

Can you provide an example VI? It would help if you save it for LV 7.1 so more people could look at it (I'm hoping to upgrade to 8.0 next month).

What are your settings for the serial connection set at? You probably already verified that your port settings are set exactly to what your instrument specifies, although sometimes it is a matter of tweaking the timeout value, if it is set too low (I think it defaults to 10 seconds though). Are there any errors set when the incomplete reads happen?

Another thing to look at is if your data reads are being slowed down by other logic in your VI. It may help to stream incoming data into a queue in one while loop, and perform operations out of the queue in another. Since you have no flow control lines, you may be running into a condition where you miss data when your logic takes more than 500 msec to run.

Joe (orko)

[Steve Weidman]

Can you provide an example VI? It would help if you save it for LV 7.1 so more people could look at it (I'm hoping to upgrade to 8.0 next month).

What are your settings for the serial connection set at? You probably already verified that your port settings are set exactly to what your instrument specifies, although sometimes it is a matter of tweaking the timeout value, if it is set too low (I think it defaults to 10 seconds though). Are there any errors set when the incomplete reads happen?

Another thing to look at is if your data reads are being slowed down by other logic in your VI. It may help to stream incoming data into a queue in one while loop, and perform operations out of the queue in another. Since you have no flow control lines, you may be running into a condition where you miss data when your logic takes more than 500 msec to run.

Joe (orko)

That's an interesting idea concerning the missed data. That may be what's happening.

While I pull together the other information you asked for, could you explain how I can make a queue to store the intermediate data?

Thanks!

-Steve

[orko]

That's an interesting idea concerning the missed data. That may be what's happening.

While I pull together the other information you asked for, could you explain how I can make a queue to store the intermediate data?

Thanks!

-Steve

Here's an example. It may need some refinement, but it explains the idea behind "producer/consumer" loops. The consumer loop is where the work gets done.

Notice that even if the consumer loop takes a long time to process, the producer loop continues to gather data and store it in the queue.

Download File:post-3266-1140219841.vi

Hope this helps. I think there are some examples in the help under the keywords of "queues" and "serial" if you need more explanation.

Joe (orko)

[Steve Weidman]

Here's an example. It may need some refinement, but it explains the idea behind "producer/consumer" loops. The consumer loop is where the work gets done.

Notice that even if the consumer loop takes a long time to process, the producer loop continues to gather data and store it in the queue.

Download File:post-3266-1140219841.vi

Hope this helps. I think there are some examples in the help under the keywords of "queues" and "serial" if you need more explanation.

Joe (orko)

Sorry for the delay in getting back to you. Lots of things to do here.

Thank you so much for the VI. That appears to be exactly what I'm after. I should be able to test it later today.

Again, thanks so much for the help!!

-Steve W.

[WMassey]

Good Morning.

Twice per second, my instrument sends out a comma delimited data stream with a CR on the end.

By using the "bytes at port" connected to the standard serial read, I can see my data.

At times though, it gets "out of sync" for lack of a better term and doesn't capture the whole string.

Take advantage of the fact that the VISA system can buffer the data for you until a complete message is received and then return that message to you. Don't use "Bytes at Port" except for possibly checking to see when it is non-zero which would tell you when it was OK to start attempting to get or to wait on a message. Enable the VISA "read termination character" feature to look for the carriage return that each message contains. It will be used as a signal to VISA when it has a complete message available to return to your code for processing. If you specify a large enough read buffer to the VISA read function then it can buffer multiple messages from the instrument and return the oldest one each time you do a call to it.

You should only get out of sync if your message processing routine cannot keep up with the rate of incoming messages. (in that case I would suggest, as a last resort, you read two or more VISA messages for every one you pass off to your processing routine, dumping the extras).

Download File:post-2800-1140640688.zip

[Steve Weidman]

Take advantage of the fact that the VISA system can buffer the data for you until a complete message is received and then return that message to you. Don't use "Bytes at Port" except for possibly checking to see when it is non-zero which would tell you when it was OK to start attempting to get or to wait on a message. Enable the VISA "read termination character" feature to look for the carriage return that each message contains. It will be used as a signal to VISA when it has a complete message available to return to your code for processing. If you specify a large enough read buffer to the VISA read function then it can buffer multiple messages from the instrument and return the oldest one each time you do a call to it.

You should only get out of sync if your message processing routine cannot keep up with the rate of incoming messages. (in that case I would suggest, as a last resort, you read two or more VISA messages for every one you pass off to your processing routine, dumping the extras).

That seems to be working well too. I put a while loop around it and it seems to be kicking my data stream out perfect every time.

Thanks so much for the help!!

-Steve W.

[Steve Weidman]

Take advantage of the fact that the VISA system can buffer the data for you until a complete message is received and then return that message to you. Don't use "Bytes at Port" except for possibly checking to see when it is non-zero which would tell you when it was OK to start attempting to get or to wait on a message. Enable the VISA "read termination character" feature to look for the carriage return that each message contains. It will be used as a signal to VISA when it has a complete message available to return to your code for processing. If you specify a large enough read buffer to the VISA read function then it can buffer multiple messages from the instrument and return the oldest one each time you do a call to it.

You should only get out of sync if your message processing routine cannot keep up with the rate of incoming messages. (in that case I would suggest, as a last resort, you read two or more VISA messages for every one you pass off to your processing routine, dumping the extras).

I may have to set my "read" constant fairly high (like 1000 or more). Is there a problem with doing that?

Thanks!!

-Steve W.

[WMassey]

I may have to set my "read" constant fairly high (like 1000 or more). Is there a problem with doing that?

Thanks!!

-Steve W.

If you are speaking of the read buffer size ...

( "Number of Bytes to Read" input to the VISA Read function or "Max Allowed Byte Count" input on the VI I posted )

... then there should be no problem with a 1000-byte buffer. Heck, it's a U32 number so it should be able to take a really BIG value if you wanted.

How much memory do you have in your machine?

[Steve Weidman]

If you are speaking of the read buffer size ...

( "Number of Bytes to Read" input to the VISA Read function or "Max Allowed Byte Count" input on the VI I posted )

... then there should be no problem with a 1000-byte buffer. Heck, it's a U32 number so it should be able to take a really BIG value if you wanted.

How much memory do you have in your machine?

The number of bytes to read on the VISA read function.

I didn't figure it would hurt anything, but wanted to make sure.

Thanks,

-Steve W.