Leaderboard

I finally completed version 1.0 of my messaging system (or messaging architecture if you prefer that nomenclature) and I figured I would post it here and see what mistakes other can find in my approach. My basic goal was to build something that could replace the old school messaging in my pre-LVOOP applications. So, I wanted something that allowed for flat messaging (all processes are peers and can direct message each other) and had some type checking on the messages (I replaced the old typedef'd enum and variant with a message class). I also wanted it to be easy to extend with additional features, such as a message log or the ability to send messages between application instances. Along the way, I added a way of building hierarchical message systems like Actor Framework. I also added subscriptions like JAMA. And I tried out dynamic processes and well as static ones. I am attaching a series of example projects, each one contains the core message architecture components so if you unzip them all, it will overwrite those parts each time. The basic concept of the architecture is comprised of four components: The system class defines the overall functionality of the message system The message class that all messages inherit from The transport class that defines the generic interface for sending and receiving messages The process class that creates the message handler. (This is roughly equivalent to an Actor in AF.) The examples demonstrate remote messaging, dynamic processes, hierarchies of processes and message subscriptions. Most basic example: Message System Basic Example.zip Examples for demonstrating client-server network messaging: Message System Client Example.zip Message System Server Example.zip Example of dynamically loaded processes: Message System Dynamic Process Example.zip Example of a hierarchy of message systems (like AF): Message System Isolation Example.zip Example of message subscriptions (like JAMA): Message System Subscription Example.zip Please comment (good or bad) on the system overall and how it is organized. I am sure there is lots of room for improvement. Thanks for the feedback! -John ps. If you have any ideas for what to name it, let me know, YAMS? (yet another message system) TUMS? (the ultimate message system ) Any other ideas?

I'm currently adding a scripting support via LuaJIT to a program I'm working on. The script is working with potentially large arrays (1 million+ elements) of doubles and strings. I pass both types via Adapt to Type Handles by value and I am able to read both of them without issue. I can also modify values within the double array. Now my problem is I want to output arrays of doubles and strings from my script, but their size is unknown until I run the script (I do know a maximum size, but that will be far larger than needed in the majority of cases). I can think of a few approaches but since I can't find a good description of the semantics of the LabVIEW memory manager and especially it's interaction with DS* functions I'm unsure if what I do will produce leaks or deallocate possibly in use data. I'm just going to list some of the approaches I have come up with with string arrays since double's should be a much easier problem. The safe method: Run the script have it store the data separately then read the string array one element at a time into the LabVIEW array. While this is certainly safe it's also horrendously slow. The I think might be safe method: Have LabVIEW pass a handle to a string array that's been preallocated with empty strings, the length of the array is the maximum amount of strings I could possibly need. Now use DSSetHandleSize on all the string handles I'm writing to to make space and write the string data and string size. Next use DSDisposeHandle on all of the unused strings (I assume I'm responsible for deallocating them since I'm throwing them out them), then DSSetHandleSize on the array handle and set the array size. With this method I'm assuming LabVIEW is still responsible for cleaning up all the resized handles. The problem with this that the array is probably several orders of magnitude too large, so I'm putting extra pressure on the memory manager. The method I want to use: Have LabVIEW pass a handle to an empty string array. Use DSSetHandleSize on the array handle to make space for the strings I'll need and set the array size. Then use DSNewHandle to create new strings to fill up and set their sizes. Now typically the thing responsible for allocating is also responsible for deallocating, is this still true when I'm putting the string handles in the array that LabVIEW is already handling (in my previous approach I deallocted handles that LabVIEW made, would LabVIEW do the same when it's done with the ones I allocated). If I need to deallocate the strings myself do I need to be extra careful that LabVIEW isn't currently using that data already (IE use the always copy node on anything that comes out of the string array, before I pass the string array back in to the script to close it). There's a decent change I might end up using arrays of single element clusters of arrays of strings. If there's some reason the concepts from the above wouldn't apply to that. Matt W

Using DETT and a test VI LabVIEW did deallocate strings that I allocated and put into a LabVIEW allocated array. The deallocation happened before the vi terminated, since all DS* allocated data would have been cleaned up by then anyway. I'm forcing deallocation by reshaping the array. So it looks like LabVIEW will deallocate handles placed properly into aggregate data types. I'm not sure how uncontained handles are dealt with (my guess is that those would need to be removed by hand), but that isn't relevant for my issue. I don't suppose anyone knows where this stuff is documented (I assume the people who used to write CIN's would have experience with this). Testing LabVIEW this way would be good for double checking the documentation does what I think it says, but seems error prone to figuring out what's exactly going without documentation. I'm going to assume my third approach will work, unless someone knows of some issues with it.

Rolf's suggestion is not the best way to do this. You actually can directly pass an array of clusters to C code and treat them as C structs, and you can even modify and resize that array. Here's how: Configure your call library node parameter as "Adapt to Type", and make sure that you have "Handles by Value" selected for the Data Format. Once that's done, wire up your array of clusters (if you start with an empty array, just create an empty array constant with the right type and wire that). Now, right-click on the call library node and choose "Create .c file...". That will generate a file containing C code with the proper C types for your array of clusters and the right function prototype. It will look something like this: /* Call Library source file */ #include "extcode.h" /* Typedefs */ typedef struct { int32_t Element; } TD2; typedef struct { int32_t dimSize; TD2 Cluster[1]; } TD1; typedef TD1 **TD1Hdl; void funcName(TD1Hdl arg1); void funcName(TD1Hdl arg1) { /* Insert code here */ } Obviously we don't do a good job of naming these types, so you should rename them first (ex: replace TD1 with MyArray and TD2 with MyStruct). Important: On 32-bit Windows you need to do one more thing to this code to make it work right in every case. Modify it like so: /* Call Library source file */ #include "extcode.h" /* Typedefs */ #if MSWin && ProcessorType != kX64 #pragma pack(push,1) #endif typedef struct { int32_t Element; } MyCluster; typedef struct { int32_t dimSize; MyCluster Cluster[1]; } MyArray; typedef MyArray **MyArrayHdl; #if MSWin && ProcessorType != kX64 #pragma pack(pop) #endif void funcName(MyArrayHdl arg1); void funcName(MyArrayHdl arg1) { /* Insert code here */ } The #if/endif and #pragma lines are the ones you need to add. This fixes alignment on 32-bit Windows because LabVIEW on that platform does not use the default alignment. If you don't do this then the C code will not interpret the data correctly in some cases. With that done, you just have to implement your code. Note that the function takes a MyArrayHdl (aka, a MyArray**). This is a "handle" to a LabVIEW array. LabVIEW arrays internally are structures containing an int32 dimension size (for each dimension) followed by an inline array of the elements. So, for instance, to sum all the elements in the example above you would write code like this: int32_t sum = 0; if(arg1) // empty arrays will have a NULL handle { for(int32_t i = 0; i < (*arg1)->dimSize; ++i) { sum += (*arg1)->Cluster[i].Element; } } That's all it takes to just read or modify the existing elements of the array of clusters. What about resizing? That's a bit trickier, but it's still possible. To resize the array you need to resize the handle, update the dimSize, and initialize any new elements (if you grew the array). When you resize the handle you have to calculate the size of the whole array in bytes. Here's the correct way to grow the array above by one element: MgErr err = mgNoErr; if( mgNoErr == (err = DSSetHSzClr(arg1, Offset(MyArray, Cluster) + sizeof(MyCluster)*numElems)) ) { (*arg1)->dimSize = numElements; // Initialize new elements } else { // error (probably mFullErr) } If you allow for an empty array to be passed in then you might get a NULL handle, which you can't resize. To allow for that, change your call library node by setting the Data Format of that parameter to "Pointers to Handles". This will change the type from MyArrayHdl to MyArrayHdl* (aka MyArray***). You would then work with it like this: MgErr err = mgNoErr; size_t arraySizeInBytes = Offset(MyArray, Cluster) + sizeof(MyCluster)*numElems; if(NULL != *arg1) { err = DSSetHSzClr(arg1, arraySizeInBytes); } else // empty array, must allocate { if( NULL != ( *arg1 = (MyArrayHdl)DSNewHClr(arraySizeInBytes) ) ) err = mFullErr; } if(mNoErr == err) { (**arg1)->dimSize = numElems; // Initialize new elements } else { // handle error } The last thing you have to do is to link your DLL to labviewv.lib (in the cintools directory of your LabVIEW installation, along with extcode.h). This gives you access to the DS* functions (and all the other functions in extcode.h). Make sure you use the labviewv.lib version. That's the one that's smart enough to make sure that it uses the correct versions of those functions even if you have multiple LabVIEW runtimes loaded in the same process. Now, obviously a lot of this is a bit tedious (much harder than using a simple C-style array), but it's not actually very difficult once you know how to do it. Don't be afraid to try it. It's easier than it looks, and it can make your LabVIEW/C interactions much more flexible.