Jump to content

Recommended Posts

my brother, i'm not angry :) i'm just beginner in labview and i don't have enough knowledge with it to understand exactely what my friends here are trying to explain for me, moreover, i mentioned earlier this is not a home work, it is rather a research, and if you google the term IEC61850, you will understand why we insist on UDP!!

There is nothing wrong with your explaining skills! There are however people who do not want things to be explained to them but instead things being done for them. Or English is not the native language but the posts look pretty ok to me in grammar, so why someone with such a grasp of the language never has heard about "chunks" is beyond me.

Also while reading a file in chunks and sending it over UDP will probably work in a local network, I'm pretty sure the resulting file on the receiver end will be damaged after being transmitted like that over a more complex network infrastructure. That is unless one adds also packet sequencing code to the transmission (and in doing so more or less reimplements the TCP protocol on top of UDP).

i knwo what "chunks" are, but how to build them in labview!!, i don't want people to do work for me, i just need some help :) i already said i'm not expert in labview, i have a good theoritical background with both TCP and UDP protocols, and in fact they are just a part in a larger research about IEC61850 standard.

Link to post
Share on other sites

wow, now it is working, but the received file seems to be corrupted due to packet duplication or miss-order, i tried to improve the performance using queues but i didn't get what i want, any suggerstions so far??

UDP is not a reliable protocol. Unless you intend to do A LOT of extra error handling, packet sequence management, lost packet retransmission and what else, you can forget to use it for transfering a file that needs to stay consistent.

  • Like 2
Link to post
Share on other sites

Decided to Google IEC61850 and UDP. I found this interesting PowerPoint presentation in PDF form.

http://www.pes-psrc....%2002282012.pdf

I liked the next to the last slide best:

The woods are lovely, dark, and deep:

After several weeks of research, the conclusions are:
  • Results are significant but not statistically so.
    • It is clear that UDP can be made reliable.
    • Packet loss was due to buffer issues and not network congestion.
    • Software architecture has an impact on buffer maintenance.

    [*]More discovery/research needs to be done regarding:

    • ​Optimum signaling for threads receiving UDP.
    • What is the interaction of buffering between VMs and the Host.
    • What is the impact of different NIC driver parameters on buffering/responsiveness.
    • Need for education (e.g. back to basics)

The author indicates that dropped messages are statistically insignificant. They were never able to eliminate them completely.

If you're just starting out with LabVIEW, this is going to take you some time, and I would wager that you will never achieve zero data loss using UDP.

Edited by Phillip Brooks
  • Like 2
Link to post
Share on other sites

wow, now it is working, but the received file seems to be corrupted due to packet duplication or miss-order, i tried to improve the performance using queues but i didn't get what i want, any suggerstions so far??

Really?

.........

UDP does not guarantee ordering, non-duplication or even delivery of "chunks", so you would need to make sure each write contains enough information to completely identify which part of which file it is. That way you could reassemble them on the other side. Even then you may not get them all, so probably want a way to ask for some part of the file to be resent.

These are things that TCP would take care of for you. Do you really need to use UDP?

Just read 65k bytes (or 1500 bytes is better for the reasons Rolf outlined) from the file at a time and write it using the UDP write. However. As GregR stated. It is not guaranteed to be recieved in the correct order, or indeed a chunk to be received at all! So that's not a lot of good for files, but usually acceptable for audio or video where a lost frame or two doesn't matter too much.

The easy solution, as others are saying, is to use TCPIP where all the ordering and receiving reassembly is handled for you (as well as other things UDP doesn't do out-of-the-box such as congestion control, re-transmission of lost packets etc). If you really must use UDP then you will have to code all that yourself (unless missing or incorrectly ordered file chunks is acceptable-for TDMS it wouldn't be). There are a few examples of reliable UDP (RDP). Noteably UDT and NORM. I am not aware of anyone doing this in Labview however, since it is just easier to use TCPIP.

And going beyond 1500 bytes MTU is only an option if you are on a controlled network where you do know about all routers involved between the two parties. Once you go outside of that, such as an internet connection, the maximum MTU is entirely out of your hands and going beyond the default of 1500 bytes is really playing chances that the communication will actually work, unless you do a lot extra work in reassembling the incoming packages into the right order on the receiving end. And that is something that is even on embedded devices usually handled much more efficiently by the TCP protocol level, so why go to the hassle of UDP in that case?

  • Like 2
Link to post
Share on other sites

Decided to Google IEC61850 and UDP. I found this interesting PowerPoint presentation in PDF form.

http://www.pes-psrc....%2002282012.pdf

To me it looks like IEC61850 is not about transfering large files consistently over the network. To do that over UDP is really not a smart idea as you have to implement on top of UDP something similar what TCP does already for you.

An interesting side node to this: A while back I was looking at the network protocol used by Second Life and other compatible virtual world environments. They used to have the entire communication over UDP with their own sequence handling, acknowledgement and retransmission requests. The reason to go for UDP was supposedly the low overhead that was required to make movement and interaction in a 3D virtual world possible at all. However a few years ago they did introduce a new mechanisme for non-realtime critical messages and are porting slowly more and more of the messages over to this new mechanisme. It is basically a HTTP based protocol where the packet load is usually a data structure in their own LLSD format, which is in many ways similar to JSON.

And yes I have implemented some basic messages in LabVIEW that got me as far as logging into a virtual world server. :cool:

  • Like 1
Link to post
Share on other sites

To me it looks like IEC61850 is not about transfering large files consistently over the network.

I googled it too and that was my conclusion. There were clearly tasks that cried out for UDP, broadcasting time-critical small pieces of information, but there was nothing suggesting one had to use UDP for sending large files.

Link to post
Share on other sites

Decided to Google IEC61850 and UDP. I found this interesting PowerPoint presentation in PDF form.

http://www.pes-psrc....%2002282012.pdf

I liked the next to the last slide best:

The author indicates that dropped messages are statistically insignificant. They were never able to eliminate them completely.

If you're just starting out with LabVIEW, this is going to take you some time, and I would wager that you will never achieve zero data loss using UDP.

Finally it seems like someone understands what is our focus from insisting on UDP, this is really encouraging; in fact our approach starts from the GOOSE model, we want to make UDP as reliable as possible by adding a small algorythm, just a light one to not decrease the speed of transfer. we have found that labview is the best tool to do so. IEC61850 is new standard that still needs a lot of work to be fully accepted in industry, we know it may look meaningless to insist on UDP, but at least, let's try :) this is the spirit

To me it looks like IEC61850 is not about transfering large files consistently over the network. To do that over UDP is really not a smart idea as you have to implement on top of UDP something similar what TCP does already for you.

An interesting side node to this: A while back I was looking at the network protocol used by Second Life and other compatible virtual world environments. They used to have the entire communication over UDP with their own sequence handling, acknowledgement and retransmission requests. The reason to go for UDP was supposedly the low overhead that was required to make movement and interaction in a 3D virtual world possible at all. However a few years ago they did introduce a new mechanisme for non-realtime critical messages and are porting slowly more and more of the messages over to this new mechanisme. It is basically a HTTP based protocol where the packet load is usually a data structure in their own LLSD format, which is in many ways similar to JSON.

And yes I have implemented some basic messages in LabVIEW that got me as far as logging into a virtual world server. :cool:

And this is the point my friend, we want to build anew UDP based protocol, you can say like UDT, to be used in power substations just like the GOOSE, i already said that i have a good theoritical knowledge with both TCP and UDP and the differences between them, however, i just don't know how to implement them using labview at the time my supervisor insists on using this programming language

I googled it too and that was my conclusion. There were clearly tasks that cried out for UDP, broadcasting time-critical small pieces of information, but there was nothing suggesting one had to use UDP for sending large files.

because you have just read what other people suggested for this new technology, IEC61850 is still under development, and the trend now is toward UDP based protocols, just look for more details about IEC and you will understand why we are insisting on UDP instead of TCP

Link to post
Share on other sites

And this is the point my friend, we want to build anew UDP based protocol, you can say like UDT, to be used in power substations just like the GOOSE, i already said that i have a good theoritical knowledge with both TCP and UDP and the differences between them, however, i just don't know how to implement them using labview at the time my supervisor insists on using this programming language

You haven't shown much so far that would have made me believe that you have much of a technical understanding of those protocols.

And wanting to do something is a nice thing, but if it is a useful exercise an entirely different one. Considering some notices in the paper about how the difference in calling socket functions can have a significant influence in lost buffers, I think LabVIEW might be not the ideal environment to tackle this research. You have no control whatsoever, in how LabVIEW handles the socket calls. In addition it adds it's own network data buffering that you can only influence in limited ways from the LabVIEW nodes. And no I'm not talking about controlling socket attributes by calling socket API functions directly, but how LabVIEW calls recvfrom() and friends itself.

  • Like 1
Link to post
Share on other sites

I still don't see why you want to build the file transfer on UDP, but I could imagine that if you know your packet loss rates and patterns, you could build a FEC (like RS codes) to counter the losses. At least it would be a fun thing to try ;)

Link to post
Share on other sites

I still don't see why you want to build the file transfer on UDP, but I could imagine that if you know your packet loss rates and patterns, you could build a FEC (like RS codes) to counter the losses. At least it would be a fun thing to try ;)

To me it seems like an academic research project, and as such literally an "academic exercise". I'm sure such work is needed but I wouldn't even bother to think about using UDP to transfer larger entities that need to stay consistent. TCP is tried and proven for such things, so why it needs to be UDP is beyond me. It's not like UDP would be easier to route in restricted networks or such, so I simply don't see the benefit of going through the hassle of reimplementing some of the TCP handling on top of UDP.

Link to post
Share on other sites

To me it seems like an academic research project, and as such literally an "academic exercise". I'm sure such work is needed but I wouldn't even bother to think about using UDP to transfer larger entities that need to stay consistent. TCP is tried and proven for such things, so why it needs to be UDP is beyond me. It's not like UDP would be easier to route in restricted networks or such, so I simply don't see the benefit of going through the hassle of reimplementing some of the TCP handling on top of UDP.

Indeed. However that "tried and proven" doesn't extend to multicast and a control/monitoring system could definitely take advantage of that.

That said. This type of requirement is what Ethercat is for.

Edited by ShaunR
Link to post
Share on other sites

Indeed. However that "tried and proven" doesn't extend to multicast and a control/monitoring system could definitely take advantage of that.

That said. This type of requirement is what Ethercat is for.

But you typically are not going to multicast 60MB files to many stations.

Link to post
Share on other sites

But you typically are not going to multicast 60MB files to many stations.

Oh, I don't know. Quite possible with Labview distributions :rolleyes:

I was talking more in context with IEC61850 rather than the OPs requirement (but I can see that I wasn't clear on that). It seems, from a casual perusal ,that it (IEC61850) is targeting similar requirements to Ethercat (the 4ms deterministic messaging - Fast Transfer of events requirement for example). In this scenario, I can see UDP multicast being very useful.and much easier for a device to "register" for messaging or "listening" for status events rather than the controller opening specific connections (and the overheads in the controller associated with copying messages to multiple clients).

Edited by ShaunR
  • Like 1
Link to post
Share on other sites

Oh, I don't know. Quite possible with Labview distributions :rolleyes:

I was talking more in context with IEC61850 rather than the OPs requirement (but I can see that I wasn't clear on that). It seems, from a casual perusal ,that it (IEC61850) is targeting similar requirements to Ethercat (the 4ms deterministic messaging - Fast Transfer of events requirement for example). In this scenario, I can see UDP multicast being very useful.and much easier for a device to "register" for messaging or "listening" for status events rather than the controller opening specific connections (and the overheads in the controller associated with copying messages to multiple clients).

ShaunR, once again you show how smart you are :) Actually in IEC61850 GOOSE messages, we have a publisher/subscriber structure, similar to UDP multicast, however the messages are directly mapped to the physical layer (ethernet) without going through any underlying protocol, it is some sort of peer to peer communication. For this reason, GOOSE messages are so fast, even faster than the UDP. For our purpose, i spent the two previous days with labview ,and i ended up with an application that generates a 6% error in the received data, however, it looks like i have some luck, our goal is to send a TDMS file with UDP, and we are mainly interested with the values palette. the weird fact is that the error never occures at the level of the values, it just occures with the graph and some of the properties!!

Link to post
Share on other sites

... For our purpose, i spent the two previous days with labview ,and i ended up with an application that generates a 6% error in the received data, however, it looks like i have some luck, our goal is to send a TDMS file with UDP, and we are mainly interested with the values palette. the weird fact is that the error never occures at the level of the values, it just occures with the graph and some of the properties!!

The last part I don't understand. What do you mean with "graph" in respect to the TDMS file structure?

From my understanding a error rate of >0% _will_ always result in some broken data. TDMS has no integrated consistency checks, as long as the TDMS segment headers survive, the whole file will look correct when opened. Nothing I would want in any form of production quality code/system.

Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Similar Content

    • By iayestaran
      Hi everyone,
      I am perceiving some strange behavior in my UDP connections. Every time I re-start my cRIO 9063, the UDP Write block outputs ERROR 66 (LabVIEW:  The network connection was closed by the peer. If you are using the Open VI Reference function on a remote VI Server connection, verify that the machine is allowed access by selecting Tools>>Options>>VI Server on the server side.) and it does not transmit. 
      The most bizarre thing is that if I simply stop the execution and re-launch it again (given that I am executing the code of the cRIO in Debug mode from a Windows laptop), the UDP connection behaves perfectly well. The only time it fails is the first time I launch the program after I reset the cRIO. For me, it looks like some service is not properly initialized when the cRIO is started.
      I attach a picture with the UDP connections of my LabVIEW RT program. The 'Comms RX' VI simply calls the 'UDP Read' function, and the 'Comms TX' calls the 'UDP Write'. The 'Open UDP Conn' also simply calls 'UDP Open'. 
      Can anyone help me?

    • By Calorified
      Hi there,

       

      I have RIO in a Windows VirtualBox inside a Ubunbtu Host OS.

       

      I am sending data from a C++ program in the Ubuntu Host system to labview within the Guest OS.

      I can receive the data on labview installed on the windows guest os. Below is the png of the Windows working program.

       



       

      But when I tried to send the data to myRIO, I was getting a udp read only error 42.

       

      Somewhere on the NI forums, someone suggested the net address of the "UDP Multicast Read-Only" vi  be wired to the address of the RIO which I have done below

       



      .

       

      Now, the code runs on myRIO but I can't receive any data on RIO. The multicast address I am sending to from Boost Asio C++ is 235.255.0.1 on port 30001.

       

      The RIO has a public ip of 172.22.11.2 and I set up a static ip address for it in NI MAX as the address of the UDP Multicast : "235.255.0.1.

       

       

       

      At this moment, I do not see what I am missing. All firewalls have been disabled and I have set the permissions for RIO through the Windows security page.

       

      Any help would be appreciated.

      Thank you!

    • By Calorified
      I have a UDP connection setup on a Windows 7 machine that runs an IMAQ code. The machine can send and receive udp connection so long as data being sent it is on the same machine. When I send the code from a different process (running on Windows 7), I am able to read the data on the RIO udp receiver VI (running on the Windows Host) as well.
      So I changed the sending program to a labview code and the udp receiver won't pick up the data.
      Could there be something I am missing?
      Attached are the PNG files (lava1 is the talker (same as the Kinect Code VI) while lava2 is the receiver).
      Any clues would be appreciated.


      Kinect Code.vi
    • By ShaunR
      Name: Transport.lvlib
      Submitter: ShaunR
      Submitted: 27 Aug 2011
      Category: Remote Control, Monitoring and the Internet
      LabVIEW Version: 2009License Type: Other (included with download)



      Transport.lvlib is a LabView API to simplify and accelerate networked communication development.
      It simplifies development by abstracting TCPIP, UDP and Bluetooth and TLS interfaces
      into a single polymorphic vi which is a thin wrapper around the conventional
      open, read, write, close and listener VIs for all the network interfaces.
       
      Features:
      Supports TCP/IP, Bluetooth, UDP (p2p, broadcast and multicast) and TLS.
      Supports symetric encryption (blowfish).
      Supports compression (zlib).
       
      INSTALLATION:
      Run the supplied installer and follow the instructions.

      Click here to download this file
    • By hilbert00
      Dear LabVIEW friends, 
       
        I'm running into a problem distribution my application. This is opening a UDP connection registering a service as explained in the documentation. It is perfectly working when running within the development system and also as a standalone application on the development computer. 
       
      The problem is with the installer and the distribution package. When I install the application of a different computer, it fails with error 63 at the Open UDP. Reading the available documentation it looks like it is a Service Locator fault. So I checked and indeed the NI Service Locator is not running on that machine and it is not even distributed with the installer. 
       
      I've tried to go through the list of additional installers in the Installer build properties but couldn't find anything useful. 
       
      I'm sure I'm doing badly something wrong, can you help me in sorting this out?
       
      Cheers,
      toto
×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.