Serial Buffer (UART) Overrun

[Justin Goeres]

I have two computers connected via a null modem cable. One is the received for my serial data (where my application runs), while the other is simulating a sensor I will eventually have to talk to. The sensor continuously broadcasts updated readings (so this is not a query-response situation, it's more like drinking from a firehose). Each message from the sensor is about 20 bytes long, and the sensor wants to send around 500 messages per second (so we're around 10kBps here).

The serial ports on both computers are set to 115200N81 (although slower speeds still exhibit the problem; see below).

Here is the problem: Any significant use of the receiving computer (switching applications or even just clicking in the window's title bar causes the receiving computer to return a "VISA Error -1073807252 (0xBFFF006C): An overrun error occurred during transfer. A character was not read from the hardware before the next character arrived." I have replicated the problem with both my laptop (USB->RS232 adapter) on the receiving end and with a desktop machine (old-school RS232 on the motherboard). This happens even with the simplest of serial read/write code (e.g. NI's examples).

I was able to find a pretty good overview of the meaning of the error in this page on the Developer Zone. Basically, it looks to me like the speed of the data stream is overrunning the capacity of the UART in the receiving computer before Windows comes around to service the IRQ and move the data out of there. I have tried all of the suggestions on the NI page without success (switching to RTS/CTS handshaking seemed to help a little, but the error still occurred). Reducing the baud rate to 9600 (laptop receiver) helped quite a bit, but the error still occurs with moderate abuse of Windows.

So I'm bringing my problem here. My questions are as follows:

• Is there any way to mitigate this in software? Given that the sensor is just broadcasting data freely, I can't tell it to just wait for me to catch up.
  
• Is this really normal, for the UART on a modern laptop (and a modern desktop) to overrun at only Ten Kilobytes Per Second??? :headbang:
  
• Where (probably) is the UART in the laptop system? Is it on the motherboard, or is it in the USB->RS232 adapter? I've consulted all the documentation I have for both, and I can't find a clear answer.
  

[Grampa_of_Oliva_n_Eden]

QUOTE(Justin Goeres @ Sep 25 2007, 09:54 AM)

...(switching to RTS/CTS handshaking seemed to help a little, but the error still occurred). ...

Could you put a scope on the lines and make sure this is not a noise or bad ground issue?

Hardware handshaking should flag the transmitter to spot transmitting until CTS goes high again.

Ben

[Justin Goeres]

QUOTE(Ben @ Sep 25 2007, 07:00 AM)

Could you put a scope on the lines and make sure this is not a noise or bad ground issue?

Sadly, no. That's due to lack of breakouts for the cable and lack of a scope.

QUOTE

Hardware handshaking should flag the transmitter to spot transmitting until CTS goes high again.

What I can tell you that the handshaking definitely seems to be working, in the sense that the transmitter and the receiver each time out if the other isn't running. But I guess if there's a noise issue then the problem would be that the handshake hiccups every so often....

[Louis Manfredi]

Hi Justin:

I've had bad luck controlling serial hardware through a USB-serial adaptor. Changing the computer to one with a real, built in serial port made the problem go away.

If you are working with a computer with a built in serial port, try changing the hardware buffering settings under the Advanced... control for setting the serial port. Not sure if that trick might also help with a USB based port.

Best Luck, Louis

[shoneill]

QUOTE(Justin Goeres @ Sep 25 2007, 03:54 PM)

I have two computers connected via a null modem cable. One is the received for my serial data (where my application runs), while the other is simulating a sensor I will eventually have to talk to. The sensor continuously broadcasts updated readings (so this is not a query-response situation, it's more like drinking from a firehose). Each message from the sensor is about 20 bytes long, and the sensor wants to send around 500 messages per second (so we're around 10kBps here).

The serial ports on both computers are set to 115200N81 (although slower speeds still exhibit the problem; see below).

Here is the problem: Any significant use of the receiving computer (switching applications or even just clicking in the window's title bar causes the receiving computer to return a "VISA Error -1073807252 (0xBFFF006C): An overrun error occurred during transfer. A character was not read from the hardware before the next character arrived." I have replicated the problem with both my laptop (USB->RS232 adapter) on the receiving end and with a desktop machine (old-school RS232 on the motherboard). This happens even with the simplest of serial read/write code (e.g. NI's examples).

I was able to find a pretty good overview of the meaning of the error in this http://zone.ni.com/devzone/cda/tut/p/id/4052' target="_blank">page on the Developer Zone. Basically, it looks to me like the speed of the data stream is overrunning the capacity of the UART in the receiving computer before Windows comes around to service the IRQ and move the data out of there. I have tried all of the suggestions on the NI page without success (switching to RTS/CTS handshaking seemed to help a little, but the error still occurred). Reducing the baud rate to 9600 (laptop receiver) helped quite a bit, but the error still occurs with moderate abuse of Windows.

So I'm bringing my problem here. My questions are as follows:

• Is there any way to mitigate this in software? Given that the sensor is just broadcasting data freely, I can't tell it to just wait for me to catch up.
  
• Is this really normal, for the UART on a modern laptop (and a modern desktop) to overrun at only Ten Kilobytes Per Second??? :headbang:
  
• Where (probably) is the UART in the laptop system? Is it on the motherboard, or is it in the USB->RS232 adapter? I've consulted all the documentation I have for both, and I can't find a clear answer.
  

20 Bytes at 500 Hz is indeed 10 KBps, but Serial transmission is normally lsited as Kbps (note small and large "b"). This, your 10 kBps becomes 80kbps without adding stop bits, parity and so on. Adding a stop bit and a parity bit moves this to 100kbps.

This should still be below your 112500 kbps serial speed, but maybe it's just too close to the theoretical limit.

Can you slow down the tranmission to only 250Hz and see if the problem goes away? edit: I see you tried this already.

Perhaps it's a simple flow control problem? Check a different com port. I have good experience with Exsys USB-RS-232 controllers (prolific Chipset). Cheap and fully-featured.

Shane.

[eaolson]

There's another KB article which offers a few suggestions. There's also a software buffer, I think. Have you tried increasing the size of that with VISA Set I/O Buffer Size?

I've also had trouble with one USB-to-serial adapter. What brand are you using?

[Grampa_of_Oliva_n_Eden]

QUOTE(Justin Goeres @ Sep 25 2007, 10:12 AM)

Sadly, no. That's due to lack of breakouts for the cable and lack of a scope.

What I can tell you that the handshaking definitely seems to be working, in the sense that the transmitter and the receiver each time out if the other isn't running. But I guess if there's a noise issue then the problem would be that the handshake hiccups every so often....

Well if you can't look then poke.

Put a good ground between the RTN lines of both ends.

I have seen this issue (bad grounds) affect Laptops (because they are isolated from ground via the battery) and USB-Serial adapters.

Another point:

You are reading ALL of the bytes at the port every time and NOT just a subset, CORRECT?

Ben

[Justin Goeres]

QUOTE(shoneill @ Sep 25 2007, 07:25 AM)

20 Bytes at 500 Hz is indeed 10 KBps, but Serial transmission is normally lsited as Kbps (note small and large "b"). This, your 10 kBps becomes 80kbps without adding stop bits, parity and so on. Adding a stop bit and a parity bit moves this to 100kbps.

Oh, for Pete's sake... I hate it when I do that. I even used the capital B on purpose, not realizing I was doing the math entirely wrong. (Problem is still unsolved, but at least I've had my daily dose of reminder that I'm still not the smartest person in the room :headbang: )

QUOTE(Louis Manfredi @ Sep 25 2007, 07:20 AM)

If you are working with a computer with a built in serial port, try changing the hardware buffering settings under the Advanced... control for setting the serial port. Not sure if that trick might also help with a USB based port.

My USB adapter supports this (which leads me to believe the UART is in the adapter), but changing its value had no effect.

QUOTE(eaolson @ Sep 25 2007, 07:32 AM)

There's

another KB article

which offers a few suggestions. There's also a software buffer, I think. Have you tried increasing the size of that with VISA Set I/O Buffer Size?

Yep, tried that, too. It really seems to be the hardware buffer that's overrunning.

QUOTE

I've also had trouble with one USB-to-serial adapter. What brand are you using?

It's a Keyspan 19Qi. I'm using driver version 3.2, which is the latest (although it's dated 2002). Unfortunately this is the only USB adapter I have -- but it has performed admirably in many other projects, so I'm fairly trusting of it (and I know how hard it is to find one to trust). However, I don't recall that I've ever used it with flow control.

QUOTE(Ben @ Sep 25 2007, 07:33 AM)

You are reading ALL of the bytes at the port every time and NOT just a subset, CORRECT?

Yes, but thanks for asking. Given that I can't reliably multiply and divide by 8, that's a fair question .

[LAVA 1.0 Content]

How are you offloading the buffer? Is the front panel of the reader visible?

I don't see any example code, but you might want to use VI Server to start a high priority VI in the background that just pulls data as fast as possible from the resource (VISA?) and and then use a queue to pass the data to your parsing/analysis code.

The other idea might be to replace the USB-RS232 adapter with a Ethernet-RS232 device. Most laptops will have an Ethernet port and the Ethernet-RS232 adapters have a dedicated processor and memory that would help buffer the data. The OS buffer would be much larger than the few bytes that a UART has and would help alot.

[shoneill]

QUOTE(Justin Goeres @ Sep 25 2007, 04:58 PM)

Oh, for Pete's sake... I hate it when I do that. I even used the capital B on purpose, not realizing I was doing the math entirely wrong. (Problem is still unsolved, but at least I've had my daily dose of reminder that I'm still not the smartest person in the room :headbang: )

My USB adapter supports this (which leads me to believe the UART is in the adapter), but changing its value had no effect.

Yep, tried that, too. It really seems to be the hardware buffer that's overrunning.

It's a Keyspan 19Qi. I'm using driver version 3.2, which is the latest (although it's dated 2002). Unfortunately this is the only USB adapter I have -- but it has performed admirably in many other projects, so I'm fairly trusting of it (and I know how hard it is to find one to trust). However, I don't recall that I've ever used it with flow control.

Yes, but thanks for asking. Given that I can't reliably multiply and divide by 8, that's a fair question .

It seems that your adapter does no hardware buffering at all. I know there are quite a lot of adapters out there which have a few bytes of buffer on-board. Not much, just enough to stop things like this happening.

If you keep the Baud high, but lower the frequency of the data transmission (you're doing a simulation, right?), do you receive everything OK?

If you reduce the packet size to 8 Bytes, do you receive everything OK?

I don't think software buffers are going to help, because it sounds like the bytes are being overwitten on the Hardware judging by the error message.....

http://www.exsys.ch/deutsch/produkte/ex_1334.html' target="_blank">Here's a link to the adapter I usually use, I'm pretty happy until now. I tried some Moxa (good reputation) but they bombed on me - on multiple occasions. They apparently don't support software flushing of their buffers! 64 Bytes of buffer. Really not much. I've tried to search for the 19Qi, but I've yet to find anything of use (like Specs). Maybe it's missing the HW buffer......

Shane.

[Justin Goeres]

QUOTE(LV Punk @ Sep 25 2007, 12:12 PM)

How are you offloading the buffer? Is the front panel of the reader visible?

Currently, it's just going to a front panel string control (which is visible, yes). That's the arrangement I've fallen back to after simplifying things back during testing.

QUOTE

I don't see any example code, but you might want to use VI Server to start a high priority VI in the background that just pulls data as fast as possible from the resource (VISA?) and and then use a queue to pass the data to your parsing/analysis code.

The example code would actually just be the Advanced Serial Write and Read.vi from the LabVIEW examples. I have it running in "Read" mode on the receiver and "Write" mode on the transmitter.

Actually, using the VI Server and a queue was exactly where I started. I've been peeling back the layers ever since .

QUOTE

The other idea might be to replace the USB-RS232 adapter with a Ethernet-RS232 device. Most laptops will have an Ethernet port and the Ethernet-RS232 adapters have a dedicated processor and memory that would help buffer the data. The OS buffer would be much larger than the few bytes that a UART has and would help alot.

Yep, that option is on the table. I'm just trying to make sure that I'm not completely out in left field before going down that route.

QUOTE(shoneill @ Sep 25 2007, 12:20 PM)

If you keep the Baud high, but lower the frequency of the data transmission (you're doing a simulation, right?), do you receive everything OK?

Yes, a high baud rate but a lower transmission frequency makes the problem go away. In fact, you can gradually increase the transmission speed and see the system get more and more sensitive to smaller and smaller interruptions. It's fun, in that oh-look-this-is-really-broken kind of way.

QUOTE

If you reduce the packet size to 8 Bytes, do you receive everything OK?

I have not tested 8 bytes specifically, but I have noted that shorter test messages seem to be more resistant to problems. So I think the answer is somewhere in the neighborhood of yes.

QUOTE

Here's

a link to the adapter I usually use, I'm pretty happy until now. I tried some Moxa (good reputation) but they bombed on me - on multiple occasions. They apparently don't support software flushing of their buffers! 64 Bytes of buffer. Really not much. I've tried to search for the 19Qi, but I've yet to find anything of use (like Specs). Maybe it's missing the HW buffer......

Thanks for the link. I've been reasonably happy with Keyspan over the years, but a USB->RS232 adapter is one of those things that sometimes you just need to pony up and buy the really good one, so you can stop worrying about it. Perhaps it's time to step up and get a new one.

[shoneill]

QUOTE(Justin Goeres @ Sep 26 2007, 02:57 AM)

Thanks for the link. I've been reasonably happy with Keyspan over the years, but a USB->RS232 adapter is one of those things that sometimes you just need to pony up and buy the really good one, so you can stop worrying about it. Perhaps it's time to step up and get a new one.

Yup, I'd have to agree with that one. I mentioned it because the Moxa series "should" be really good, but I had some problems with them. They have since released a new line, so perhaps they work better, I dunno.

If it would be of interest to you, I could try out a sample transmission on one of my adapters (Code?)........ I'd be interested as to how it holds up. I also have a MOXA unit available, so a comparison may be interesting.

Shane.

[Grampa_of_Oliva_n_Eden]

Other ideas

Is the USB port on the PC a USB-2?

There are VISA Events available that I have never had to resort to. Perhaps you can an event for each character ????

Just thinking out loud,

Ben

[PeterB]

QUOTE(Justin Goeres @ Sep 25 2007, 11:54 PM)

Here is the problem: Any significant use of the receiving computer (switching applications or even just clicking in the window's title bar causes the receiving computer to return a "VISA Error -1073807252 (0xBFFF006C): An overrun error occurred during transfer. A character was not read from the hardware before the next character arrived." I have replicated the problem with both my laptop (USB->RS232 adapter) on the receiving end and with a desktop machine (old-school RS232 on the motherboard). This happens even with the simplest of serial read/write code (e.g. NI's examples).

Hi Justin,

I think your problem is solved by reading http://keyspan.custhelp.com/cgi-bin/keyspan.cfg/php/enduser/std_adp.php?p_sid=-S7T5JMi&p_lva=&p_faqid=403&p_created=1116093562&p_sp=cF9ncmlkc29ydD0mcF9yb3dfY250PTEwNyZwX3NlYXJjaF90ZXh0PSZwX3NlYXJjaF90eXBlPTMmcF9wcm9kX2x2bDECZwX3NvcnRfYnk9ZGZsdCZwX3BhZ2U9Mg**&p_li=' target="_blank">this article from Keyspan.

In short you need their high speed adapter, your Qi ain't designed to do what you want.

regards

Peter

"Over the years, Keyspan has made several adapters in the USA-19 line. The USA-19 line is defined as sb erial [A]dapters with [1] DB[9] port (these characteristics make up the part number [uSA-19]). The following information outlines the difference between each variant of the USA-19 line:

Product Name: USB PDA Adapter

Model Numbers: USA-19, USA-19x, USA-19Q, USA-19Qi

- - - - - - - - - - - - - - - - - - - - - - - - - -

The models in the USB PDA Adapter line (models USA-19, USA-19x, USA-19Q, USA-19Qi) were designed for connecting Palm Pilots, Wacom tablets, and other PDAs to Mac and Windows computers. These adapters only had enough serial capabilities to communicate with PDAs and graphics tablets and therefore using these adapters with serial devices that aren't PDAs or graphics tablets IS NOT RECOMMENDED (the USA-19HS is the recommended adapter for any serial device).

"