Idea for Remote Updating of Files

[dblk22vball]

Hello,

I am trying to think of a way to remotely "tell" my test fixtures that they need to update their tables, because there was a change in the master table by someone. The files are stored on a shared network drive, and then fixtures would copy over the files (text documents) that need to be updated.

The "settings editor" can be run an any machine in the plant, since the master files are centrally located.

I read that a shared variable with datasocket might be easiest, but that seemed like the variable had to be polled (not a big issue), but what do I do about the shared variable URL, since it could change (as the "master" computer can change). Or did I miss something with this.

The networked drive is a company wide drive that is on a server run by IT, so I cant install any labview code on it.

I have not used the tcp vis before, so I am not sure if these would be better or not.

I could have the computers update nightly, but if we need to change a parameter during the day, this will not work.

Your ideas are appreciated.

[Mark Yedinak]

If I were you I would consider using network queues. Here is a thread that discusses them. Attached is the version we have been using in our applications.

[crelf]

QUOTE (dblk22vball @ Mar 5 2009, 10:36 AM)

I could have the computers update nightly, but if we need to change a parameter during the day, this will not work.

Can't you make them update more often than that? Or you caould have them read the table directly from the network.

[jdunham]

QUOTE (crelf @ Mar 5 2009, 08:01 AM)

Can't you make them update more often than that? Or you caould have them read the table directly from the network.

Unless it's super-slow, I think Chris is right, just get the files from the network. You can cache them locally so that if the network is down, you can still run a test.

If access is slow, you could have a version number in a single file (versioning your test specs is a good idea anyway). Each tester would remember its current version number, check the version number on the network, and then if they are not equal, get the updated files from the network drive.

At that point you could just keep the versioning for each specific file. But then you'd have a poor-man's implementation of Subversion [smell the irony?], so you should just convince your IT group to put the real Subversion on the server. In that case you'd just run a simple svn update before executing any test.

[dblk22vball]

I have 6 test stations, each running a test roughly every 1-2 minutes, so I was trying to limit the network usage.

As for updating more often, I think I would run into the same issue of network usage. Granted, there is not a lot of overhead in a couple mb of files. But since we are building items quickly, if we did an update and had to wait 10 min (assuming you are checking the tables every 10 min), it wouldnt be a huge deal, but still an inconvenience, since people are paid by the part and get nasty when the test station cant let them build.

I will take a look at the network queues, thanks.

[Mark Yedinak]

It would seem that you have several options. One would be for your applications to poll the status of the configuration file to see if it has been modified and update itself if it has. It could use the modification date of the file for this. This would also mean that your computer's clocks should be synchronized. A second option would be to have your applications update itself at some regular interval. The third, and most complex option is to have a central sever application which your applications would register with. If any instance of your application updated the configuration it would notify the update server which in turn would send a message to all registered applications that the update has occurred. A slight variation of this approach would be for each application have a process that listens for broadcast messages. You could use UDP for this. When an application updated the configuration settings it would broadcast a message indicating that the update has occurred and all of the other applications could update their settings accordingly. This option is probably the most flexible. It would require that your IT staff allow UDP broadcast packets on your network and may require them to open a specific port on the network. You could create a simple protocol for this where your broadcast messages are a message type and a variant. Based on the message type you could send additional information in the variant data and the message type would dictate how to interpret the data itself.

[dblk22vball]

ok, so going off of the thought of the fixtures accessing the master file on the server at all times, I asked IT about this.

They are actually going to give me my own drive and they said that I could poll away, since they can put it on a spare server....

based on previous experience, I was not expecting that....

i will also have the program local cache it as a back up.

thanks

[Mark Yedinak]

QUOTE (dblk22vball @ Mar 5 2009, 10:35 AM)

ok, so going off of the thought of the fixtures accessing the master file on the server at all times, I asked IT about this.

They are actually going to give me my own drive and they said that I could poll away, since they can put it on a spare server....

based on previous experience, I was not expecting that....

i will also have the program local cache it as a back up.

thanks

That's good. I would still consider using the message broadcast instead of polling. It will be much more efficient and it will require very little overhead in your applications.

[jdunham]

QUOTE (dblk22vball @ Mar 5 2009, 08:35 AM)

ok, so going off of the thought of the fixtures accessing the master file on the server at all times, I asked IT about this.

They are actually going to give me my own drive and they said that I could poll away, since they can put it on a spare server....

based on previous experience, I was not expecting that....

i will also have the program local cache it as a back up.

thanks

I still think you should look at Subversion. It was designed for source code revision control, but basically it's a system for maintaining a synchronized collection of files. It has a built-in local cache for each client (all useful SCC systems have this) and it only updates sections of a file which need updating.

[Mark Smith]

QUOTE (Mark Yedinak @ Mar 5 2009, 09:24 AM)

... You could create a simple protocol for this where your broadcast messages are a message type and a variant. Based on the message type you could send additional information in the variant data and the message type would dictate how to interpret the data itself.

I have done something similar but rather than use a variant I used a string (byte array) for the data and use the "flatten to string" to convert the data and also prepend the message type as a flattened string - this way, the message is platform/program agnostic. All the recipient needs to know is how LabVIEW represents the data. Could a variant work the same way? I don't know much about how LabVIEW packages variants.

QUOTE (jdunham @ Mar 5 2009, 10:49 AM)

At that point you could just keep the versioning for each specific file. But then you'd have a poor-man's implementation of Subversion [smell the irony?], so you should just convince your IT group to put the real Subversion on the server. In that case you'd just run a simple svn update before executing any test.

When you do something like this w/Subversion, do you use the command line and build a batch file to execute or do you use an API?

Thanks,

Mark

[jdunham]

QUOTE (mesmith @ Mar 5 2009, 10:05 AM)

When you do something like this w/Subversion, do you use the command line and build a batch file to execute or do you use an API?

I have some wrapper VIs which invoke the SVN command-line client via System Exec.vi. I guess I should open-source them, since they are pretty simple. I'm sure other heavy SVN users have something similar.

Jason

[Mark Yedinak]

QUOTE (mesmith @ Mar 5 2009, 12:05 PM)

I have done something similar but rather than use a variant I used a string (byte array) for the data and use the "flatten to string" to convert the data and also prepend the message type as a flattened string - this way, the message is platform/program agnostic. All the recipient needs to know is how LabVIEW represents the data. Could a variant work the same way? I don't know much about how LabVIEW packages variants.

I agree that a basic string is more portable. From what I have observed the variant type is fairly easy to decode however since this is an NI internal structure there is no guarantee that things won't change over time. Variants aren't much more than a flattened string with additional type and size information embedded in the string.

Personally I like the structure used in SNMP messages. (Actually the entire SNMP packet is constructed this way) Each item consists of three parts. The first part is the data type, the second part is the length of the following data and the third part is a variable length byte array which is the data itself. The type and length fields are fixed length. This structure is easy to decode and it is platform and language independent. Of course your message is not necessarily sized the most efficiently however this is generally not that big of an issue.

My suggestion of using variants is more of a LabVIEW only solution but it didn't sound like other applictions would be involved. Your approach is more universal.

QUOTE (jdunham @ Mar 5 2009, 12:35 PM)

I have some wrapper VIs which invoke the SVN command-line client via System Exec.vi. I guess I should open-source them, since they are pretty simple. I'm sure other heavy SVN users have something similar.

Jason

If you could package them up that would be great. I am working on an automated test system that has literally thousands of data files that we need to manage. We have discussed using CVS for this but I would prefer to go with SVN. This is a feature we will be adding in the near future and if we could use some existing code it would save us some development effort.

[Rolf Kalbermatter]

QUOTE (mesmith @ Mar 5 2009, 02:05 PM)

When you do something like this w/Subversion, do you use the command line and build a batch file to execute or do you use an API?

I use SVN in several applications for some of our customers to version control Lua test scripts that they write. If they do not have a fully blown Apache server available to run the SVN repository on, I just have them install the SVN service on a regularly backed up spare machine. Usually their IT dep does the maintenance of that.

This allows as many test stations as they want to be synchronized with the central repository. Each test station does a SVN Update on startup and an operator can easily force a manual Update from the application menu at any time. The application also allows to visualize the status of the local repository (modified, stale, conflict, up to date) and all individual scripts and also allows maintenance including committing changes, but most people seem to prefer SVN Tortoise to do that :-).

I first intended to write an intermediate DLL to access the SVN API directly from LabVIEW, assuming this to be a cleaner solution than the command line tool. But time constraints made me opt for a System Exec approach instead for a start and it is working so smoothly I guess I won't consider going back to get the DLL interface ready. The SVN DLL API while very powerful is also quite involved (and certainly in no way suited to be interfaced directly through the Call Library Node). So I have a hard time to justify the rather big time investment since the command line tool works so good.

Rolf Kalbermatter