Sharing configuration information between processes

[Neil Pate]

I am wondering how others implement the sharing of configuration information between processes.

For example, I have a camera with an acquisition time specified by a user. This value is editable by the user during run-time and persisted to disk so next time the application starts it is used. The process doing the image acquisition is free-running, and the GUI doing the editing of this parameter is in another process. The technique I use to solve this is a bit old and rusty, namely an LV2 (actually Action Engine) with all the system config in it with methods to read and write from/to disk as well as set accessors for each of the variables. The acquisition actor just reads out the value as it needs it, but does not actually store it internally in its own process. I currently have no mechanism in my framework  for informing a process that a configuration value has changed. This works fine, but I wonder if others have gone the extra step and implemented configuration data using User Events and removing the global data store?

In an actual system there will likely be significantly more configuration parameters.

 

[ShaunR]

9 minutes ago, Neil Pate said:

I am wondering how others implement the sharing of configuration information between processes.

For example, I have a camera with an acquisition time specified by a user. This value is editable by the user during run-time and persisted to disk so next time the application starts it is used. The process doing the image acquisition is free-running, and the GUI doing the editing of this parameter is in another process. The technique I use to solve this is a bit old and rusty, namely an LV2 (actually Action Engine) with all the system config in it with methods to read and write from/to disk as well as set accessors for each of the variables. The acquisition actor just reads out the value as it needs it, but does not actually store it internally in its own process. I currently have no mechanism in my framework  for informing a process that a configuration value has changed. This works fine, but I wonder if others have gone the extra step and implemented configuration data using User Events and removing the global data store?

In an actual system there will likely be significantly more configuration parameters.

 

If you can use a LV2Global to transfer information then it isn't a separate process-in the operating system sense. May seem pedantic, but it makes a huge difference to the solutions available.

Anyway.....

I think most people now use a messaging system of various flavours so it's usually a case of just registering for the message wherever you need it. Usually for me that just means re-read the database which can supply you with the sharing like a global but without dependencies..

Edited May 13, 2016 by ShaunR

[Neil Pate]

1 hour ago, ShaunR said:

If you can use a LV2Global to transfer information then it isn't a separate process-in the operating system sense. May seem pedantic, but it makes a huge difference to the solutions available.

Anyway.....

I think most people now use a messaging system of various flavours so it's usually a case of just registering for the message wherever you need it.

Yeah, when I say process I do not really mean OS process, rather "free-running loop within the LabVIEW application context".

I have messaging for a bunch of other stuff in my framework, just never got around to doing this for config type information.

[ShaunR]

1 hour ago, Neil Pate said:

Yeah, when I say process I do not really mean OS process, rather "free-running loop within the LabVIEW application context".

I have messaging for a bunch of other stuff in my framework, just never got around to doing this for config type information.

Well. Messages are ephemeral and config info is persistent so at some point it needs to go in a file. This means that messages are great for when the user changes something but a pain for bulk settings. Depending on the storage, your framework and your personal preferences, the emphasis will vary between the extremes of messaging every parameter and just messaging a change has occurred.

[drjdpowell]

1 hour ago, Neil Pate said:

I currently have no mechanism in my framework  for informing a process that a configuration value has changed. This works fine, but I wonder if others have gone the extra step and implemented configuration data using User Events and removing the global data store?

You could extend your Action Engine to have a “Register” action, which takes a User Event and saves it in an array internally, then posts to the Event whenever the config value changes.  Then processes loops can register if they need to be informed when something has changed.  Then you have a hybrid AE/messaging system.

[drjdpowell]

6 minutes ago, ShaunR said:

Well. Messages are ephemeral and config info is persistent so at some point it needs to go in a file. This means that messages are great for when the user changes something but a pain for bulk settings. Depending on the storage, your framework and your personal preferences, the emphasis will vary between the extremes of messaging every parameter and just messaging a change has occurred.

I use messages containing JSON for configuration, combined with a subVI for handling "Set Config” or "Get Config" messages that accepts config info as either a variant cluster or an array of control references.  Once set up, adding new configuration info is trivial.

[ShaunR]

3 hours ago, drjdpowell said:

I use messages containing JSON for configuration, combined with a subVI for handling "Set Config” or "Get Config" messages that accepts config info as either a variant cluster or an array of control references.  Once set up, adding new configuration info is trivial.

Yes. I use a different topology-I'm at the other end of the spectrum I talkedabout earlier. I only send UPDATE messages and whoever needs the configuration goes and queries the database for whatever they are interested in. You push; I pull

[Michael Aivaliotis]

@Neil Pate I think what you're doing is fine. I've used that strategy as well. I think you can enhance the design by adding a globally registrable message that can notify a process that the config has changed and that it can perform a certain action based on that.

The question is, why does the process care about the change. Because if it's an action-engine then won't the process get the new config whenever it happens to read\need the global data again? I think that's the thing you need to look at. For example, the process has a UI that displays Units and the units have changed in the config so that must be reflected on the screen. So then it might be registered for a global user event called: "Update UI". Or it could be called "Config Changed". Then the process must do the appropriate thing. Where the data comes from is secondary.

Another approach I use involves lvoop. I create a configuration class that contains one or more DVRs. Any other class that needs the configuration data contains the configuration class.

[Neil Pate]

So my current framework is very "actor" based (home-rolled solution, yup "yet another actor framework"), but one thing my actors do not do is take care of the serialising of their own configuration information, instead this is handled by the System Config action engine.

Now I think perhaps each actor should be responsible for the serialisation of its own config. However this could lead to trouble if they are all trying to write to the same file on disk at the same time, so I guess each actor needs its own config (ini type) file or other solution (a DB as proposed, which I am not that keen on as I am very much a fan of human readable config files).

My plan was to extend my config action engine to include a user event for data change, and try and remove the global access out of each actor so that my actors (just a class), store the config in their own private data. I definitely do not want a message per parameter though, just a single event saying "config changed" but then I think this leads to more pain as the user event would either need to include the entire system configuration as its data, or I need to have an event per parameter? hmmmm...

 

[ShaunR]

19 hours ago, Neil Pate said:

So my current framework is very "actor" based (home-rolled solution, yup "yet another actor framework"), but one thing my actors do not do is take care of the serialising of their own configuration information, instead this is handled by the System Config action engine.

Now I think perhaps each actor should be responsible for the serialisation of its own config. However this could lead to trouble if they are all trying to write to the same file on disk at the same time, so I guess each actor needs its own config (ini type) file or other solution (a DB as proposed, which I am not that keen on as I am very much a fan of human readable config files).

My plan was to extend my config action engine to include a user event for data change, and try and remove the global access out of each actor so that my actors (just a class), store the config in their own private data. I definitely do not want a message per parameter though, just a single event saying "config changed" but then I think this leads to more pain as the user event would either need to include the entire system configuration as its data, or I need to have an event per parameter? hmmmm...

 

OK. So you want to "pull" and like ini-files. Hell. there is a whole operating system based on ini files, eh Linux?  So why not?

Each "actor" (I hate that terminology) has its own ini-file, whos name is dependent on the actor, process, service or whatevers name Lets say you have an "Acquire_DC_volts" then you could have acquire_dc_volts.ini.

Your "System config Engine" broadcasts a "UPDATE" message (event or queue list-up to you and your architecture) that prompts each actor that wants config info to issue a query message to the "System Config Engine" that says "I want this file". The "System Config Engine" then loads the file and sends it to the requester.  Why do it this way instead of relying on each actor to load it? Because you can make an API and handle more than just config files, you can order the requests if there are inter-dependencies and you can swap out ini-files for a DB later once you have succumbed to the dark side ..

You can see this type of messaging in the VIM Hal Demo. There is a FILE.vi [service] that does the actual reading and if you look in MSG.vi you will see the requests (e.g. MSGPOPUP>FILE>READ>msg.txt" and "MSGPOPUP>FILE>READ>bonnie.txt" ), It doesn't happen  on config changes because there is no user changeable config, rather, when certain events happen - but it's the same thing.

[drjdpowell]

On May 14, 2016 at 7:37 PM, Neil Pate said:

Now I think perhaps each actor should be responsible for the serialisation of its own config.

Why?  Actors are tools.  One configures a tool for a specific task.   You know the task; the tool doesn’t.  Why would the tool be responsible for configuring itself?

[Neil Pate]

On 5/15/2016 at 10:33 PM, drjdpowell said:

Why?  Actors are tools.  One configures a tool for a specific task.   You know the task; the tool doesn’t.  Why would the tool be responsible for configuring itself?

You are right. I am overthinking this, I will keep my System Config action engine and just extend it a bit.