Jump to content

UDP Read of Variable Size Packets


Recommended Posts

Hi all,

I had a couple of questions related to reading udp packets in Labview. I am receiving 100 - 200 Mbps of variably sized packets. The first byte in the packet specifies its size in bytes*4.

1) When I perform a read, I have no idea how much data to request, so I set an arbitrary size. It seems in doing this that I am getting multiple packets together and that I subsequently need to parse the data into packets after reading. Doing so seems to slow down the machine a bit and I'm nervous I'm dropping packets (see next point). I also tried reading only 1 byte, then reading X number of bytes based on that 1 byte, but that threw errors (I think it was error 113 about more data being available than I read).

2) How do I tell if I am overflowing the socket buffer? I am getting weird behavior that seems to indicate dropped packets occasionally but no error is coming out of the udp read function.

Thanks,

Jim

Link to comment

Reading first byte to determine the number of bytes to read, then doing a second read of that number of bytes is the correct way to do this. Could you check what error you're receiving, and attach the code that demonstrates it? There's no reason that should return an error, and code 113 does not match your description of it.

Link to comment

And to add to that, based on the description you should just ignore it as your software is designed for this. You can check for this error code and clear the error cluster if it appears to ignore it.

VISA throws something similar as well.

(null)

Link to comment

Hi all,

Thanks so much for the help. I had some errors in my implemenation (U8 vs I32) and bytes vs quadlets (bytes*4).

After fixing them the problem remains. I wasn't sure of the error off hand, but I confirmed it is 113. Unfortunately ignoring it seems not to work as there is simply no data returned. My guess is this is a Labview implementation that if this error occurs it chooses to retain the data for a subsequent read rather than require concatenation.

This error occurs on the first read. I thought it might be a startup issue but on subsequent read loops I still get nothing.

Thoughts? Thanks again for the help.

Jim

Link to comment

I am not. There is no error input to the first udp read. The second udp read has an input but I clear the error if it is 113. However, I never get to the second udp read because the first udp read is always empty and I have a case structure around the second one in which it doesn't run if the first one returns no data.

Link to comment

I'm not sure why that's not working for you. You should probably post your VI, just for clarity.

An alternative, though not as "correct" of a solution, would be to have a parallel loop constantly reading as much data as possible (if you can get that to work) and then dumping it onto a queue which is processed by another loop. The consumer loop would have to be able to figure out where packet boundaries are and I don't know how tenuous that would be for your application. If you wanted to add complexity, you could package the UDP data with a boolean that lets the producer hint at whether it's a packet boundary or not.

Link to comment

I looked at the documentation for the UDP read VI, and I believe you are getting error 113 and no data on that first read because the one byte you attempt to read is part of a larger packet. From the documentation of the Max Size control of that VI:

max size is the maximum number of bytes to read. The default is 548. (Windows) If you wire a value other than 548 to this input, Windows might return an error because the function cannot read fewer bytes than are in a packet.

  • Like 1
Link to comment

Way to read the documenation! Thanks.

I'm not sure how to improve the reading I have. I've tried two options:

1) Processing an entire "read" and concatenating a partial packet (if it exists) to the next read

2) Implementing a circular buffer.

The first was a lot simpler but performance was not great. The second was much more difficult and I didn't see much performance gain (if any). I think it is likely due to memory copying that isn't immediately clear to me. Ideally I would only have one fixed space in memory that I am ever working with for all packets, in addition to the memory generated for each packet that I pull out of the buffer.

I was sort of hoping this was a solved problem with a accepted way of handling it.

Finally, any ideas why I am not getting buffer overflow errors? Does Labview report this?

Link to comment

I think you are not getting overflow errors because the operating system drops packets silently once the buffer is full. I have no idea why you are getting partial packets - the documentation seems to imply that LabVIEW only reads complete packets (I may be wrong on this). I think asbo's solution is probably the best approach to take with the issue of not reading fast enough.

Link to comment

My apologies for misleading you, I suggested the standard approach for TCP and missed the note in the documentation that makes it unsuitable for UDP.

If you post your circular buffer approach I'm sure someone will be happy to suggest how it can be made more efficient.

Link to comment

Thanks all for the feedback.

I haven't changed the default buffer size yet so I'll give that a try:

http://digital.ni.com/public.nsf/allkb/D5AC7E8AE545322D8625730100604F2D

I'm also doing minimal processing (read a couple of bytes and conditionally throw onto a queue) in addition to the packet splitting, so I hadn't split it up yet into separate read and split loops but I'll try that next.

After some brief web searching it wasn't immediately obvious if there was some system way to detect buffer overflow. I'll probably just do some testing and convince myself that at high rates exceeding my typical usage the system seems stable.

The partial packets may be indicative of a buffer overflow. I'm going to work on getting the system to be more reliable then see if the problem still exists.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.