Jump to content

TDMS Read to DVR - Memory Usage


Recommended Posts

Hi

I am reading big TDMS files (~1GB) and I try to minimize memory usage.

I read a signal once and I want to keep it for later usage, so I tried to write it to a DVR in the hope that I have only one instance of the data in memory.

image.png.c141382bf34f0f62efa1c654500f6b31.png

However the task manager tells me that uses 2 times more memory than it should.

I think the problem is that the array is not deallocated although I don't need it after writing it to the DVR.

Is there a solution to make sure I don't have copies of data?

Link to post
Share on other sites

in theory i think you can use the 'swap values' primitive between the dvr read and dvr write to swap the specific handle over into the dvr, but that may be wrong.

tdms also has an e-dvr option: https://zone.ni.com/reference/en-XX/help/371361R-01/glang/tdms_advanced_functions_dataref/

that should have almost no memory overhead.

Link to post
Share on other sites

The swap value does not seem to get rid of the problem. The second data instance is still in memory. (When I call the VI several times in a row the number copies grows less with the swap, but that's still not good enough.)

If I use the Request Deallocation VI, that seems to get rid of the copies when the VI finishes, but I don't know if that's a good design. And the second copy is anyway in the memory during that the VI runs, which I would prefer to avoid.

I already looked at the TDMS with external DVR but I don't know if/how I can use that. I don't seem to be allowed to wire a normal  DVR to the VI.

Link to post
Share on other sites

A) LabVIEW will hold onto memory once allocated until the system says it is running low or until we have a large enough block to give back to the OS. Task Manager tells you nothing about how much memory LabVIEW is using, only how much it has available to use. 
 

B) Every wire can maintain an allocation of the data if the compiler determines that keeping the buffers inflated gives us the best performance. You can force a a subVI to release those allocations at the end of every execution by calling the Request Deallocation primitive, but I REALLY DO NOT recommend this. You’ll kill performance and possibly not save anything as people usually discover their app immediately re-requests the freed memory. Unless you are getting out of memory errors or are a real pro at data flow memory optimization, I recommend that you let LabVIEW do its job to manage memory. Beating LabVIEW and OS memory management generally requires bursty data flow with infrequent but predictable data spikes, where you can teach LabVIEW something about the spikes. 

  • Like 1
Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

  • Similar Content

    • By Ram Prakash
      Can anyone please tell what a DVR [ Data value reference ] is ? I want to know at what situation it will be used and what are the advantages we get by using DVR. I am really confused in this topic . If someone has any code in which they have worked with DVRs. kindly share it to me.
       
      Thank you.
    • By Zyl
      Hi everybody,
       
      I'm running into something I don't really understand. Maybe you can help me here !
      I've got a LVLIB that is used as an 'Interface': it exposes public VIs which wrap around public functions of a private class (see code attached) . The class is private because I want to force the users to use the 'interface' functions.
      In one of my interface VI, I create a DVR on the private class (Interface_init). The DVR is stored into a typedef (FClass_DVR.ctl) and this typedef is the 'reference' that link all the interface public functions.
      In TestCode.vi (which is not part of the lvlib and illustrates the standard code that a user can create to use my driver), I can call my public interface functions and link them without any problem.

      But as soon as I create an indicator on that reference (to create a state-machine-context-cluster for example), my TestCode VI breaks !

      The error returned is : This VI cannot use the LabVIEW class control because of library access scope. The LabVIEW class is a private library item and can only be accessed from inside the same library or libraries contained in that library.
      I understand that the class is private. But the DVR is contained into a public control. Using an In Place structure on that DVR into TestCode would not work, since the class is private. So why is the DVR control problematic at that point ? Creating it do not breaks any access protection...
      Am I missing something ?
      DVR Private POC.zip
    • By IpsoFacto
      I've got some weird stuff going on with a cRIO project I'm working on wanted to get some opinions on it. The basic architecture is a set of classes that do some process. That process registers with a server. The internal data of the process is held in a DVR and the server get's access to that DVR. Clients use TCP to ask the server to do something, the server makes a call against the classes DVR and returns a response to the client.
      To simplify the issues I'm seeing I created a class that internally just increments an integer every 500ms. The client asks the server what's the current count, the server asks the Counter class and returns the answer to the client. This works perfectly fine when running the VI in the IDE. When built it connects, will get the JSON message back, but always gets a default value from the DVR call (zero, in this case). As soon as I open a remote debug panel to the cRIO, everything is working. The count is correct, the client calls work, just like normal. As soon as I right-click, close debug, it goes back to zero. Open debug works, close debug, back to zero. I know the DVR isn't getting dropped because the count continues to increment while not in debug, the process is still running happily with no issues.
      Here's a few screenshots of the code;
      Count Class process (get the count, increment, write it back to the DVR) - Counter Class process
      You can see the DVR vi's are actually vim's using a cast. I can't imagine that's the issue.
      Server Side call - Server Side calls
      All this does is get the count from the DVR (same as above) and wraps it in JSON and passes it back to the client as a JSON string.
      I also implemented an Echo class that ignores the process and DVR's, it just takes whatever string you sent form the client to the server and passes it back with a prepended "@echo". This works when running as an executable with the debug turned off so I know the client, server, and the server/class calls are all working as expected.
      Any thoughts here would be welcome, thanks.
      edit: I added the any possible errors coming from the variant cast to the JSON reply. When the debug is open there are no errors, when the debugger is closed it throws error 91, but the in-place element structure reading the DVR does not throw any errors. How can a variant not exist until a debugger is opened and than it magically exists?
      edit: the internal data dictionary is a wrapper around a variant attribute, I wired out the "found?" terminal all the way out to the JSON reply and if the debugger is open the attribute is found, but not if the debugger is closed. Anyone have issues with Variant Attributes in Real-Time?
    • By the_mitten
      The introduction of parallel, read-only access for DVRs in LabVIEW 2017 adds a great deal of flexibility to using DVRs to monitor values in parallel executions of code. Fo\The downside of this, of course, is the necessity of using the In Place Element (IPE) throughout your code simply to read the value. Having IPEs throughout your code just to read a value both takes up block diagram real estate and also takes more clicks than desirable to insert.
      Similarly, though less frequently, there are times when you only need to update the value within a DVR without actually performing any logic inside of the IPE.  This situation is less frequent, at least for me, as I am usually using arrays or classes with DVRs such that I actually need to modify the existing data rather than simply replacing it.
      A more preferable solution to the above situations would be to have Read/Get and Write/Set VIs for the DVRs to simplify the process of working with them. This way, and IPE on the block diagram would only be needed when you were actually modifying the existing data within the DVR, rather than simply overwriting or returning the current value.
      Thanks to the power of malleable VIs and the type specialization structure that is now officially released in LabVIEW 2018, a better solution is now available. I’ve created two malleable VIs, Read  DVR Value (Parallel) and Write DVR Value that allow you to perform a write and a parallel read on any DVR data type.
       Now, you can use a single VI that you can insert via Quick Drop to read or to write DVR values.  
      Download the attached ZIP file to access the two malleable VIs and example code, and please let me know your thoughts in the comments!
       


      DVR Read and Write VIs 1.0.0.zip
    • By UnlikelyNomad
      I've been working a lot lately with by-reference architectures that still cooperate completely with LabVIEW's implementation of OOP and polymorphism. I've also recently taken an interest in trying to speed up development with secondary providers (similar to GOOP) to enable automatic creation of accessor VIs hidden behind the DVR, automatic creation of the private data type and constructor/destructor, etc. within the project window. I'm generally not a fan of the extra stuff that goop adds in to classes, I'd prefer to keep the source code looking as close to a normal class as possible.
      That said, I've started on my first ever XNode and it's a cross between an unbundle by name node and the -> operator in C. It functions just like a normal UBN, however it was also pull items out of DVRs. Having to plop down a UBN to pull out the reference from the class, an in place node to dereference, and then another UBN to pull the data out gets tiresome after about the 100th property VI gets written.
      So far I've gotten the node drawing completed (except for data type coloring of the labels), the type inferencing from the input wire, and the popup menu for selecting an item. Next up will be the menu selection code so that the names will finally show up in the terminals! Then I get the daunting task of scripting up the GenerateCode ability >_>
      Anyone interested in something like this? Following this will be a match to the Bundle by Name node that serves the same purpose except to write the items.



×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.