Jump to content
GregFreeman

Slow Index Array of Classes

Recommended Posts

I have an array of classes, let's call the object TestPass, of size 1 (but it is an array because it can scale out to multiple test passes). In this class, there is one other nested class which is not too complex, then various numeric and string fields to hold some private data. There is also an array of clusters. In this cluster there is a string, two XY pair clusters, and an integer. Not very confusing.

This array of clusters gets fairly large, however, upwards of 80-100k elements. What I am finding is when I index the array of pass classes it is crazy slow. On the order of 30 ms. Doesn't seem like much, but we are indexing the array in our method to "Get Current Pass" which is used in various places throughout our code. This is adding potentially hours to our test time over the 80k devices we are testing. 

So, I started digging. When I flatten the class to a string and get the length, it's 3 mb. But, when I run the function with the profiler is is allocating close to 20 mb of memory!

My gut feel was that the string is causing the issues. So I removed the string from the cluster and the index time went to 0 ms. 

Luckily we can normalize a bit and pull the strings out of the cluster since a lot of them are duplicates. But it makes our data model a bit uglier. 

Has anyone seen these kind of performance issues before? I saw them in 2013 and 2017.

Edited by GregFreeman

Share this post


Link to post
Share on other sites

I normalized my data but was still seeing awful use of memory. Upwards of 3 gb, and I would randomly get a copy that would give me an out of memory error.

So, i went into my project settings, unmarked everything that had compiled code separated. Cleared compiled object cache. Did a save all. My memory usage has dropped from 3gb with tons of seemingly unnecessary copies to 1 gb just by doing this. I have on and off seen some very bizarre issues with classes and separate source from compiled, and even with that setting I still get lots of dirty dots anyways which isn't buying me much. I think I'll be staying away from it in the future.

Edited by GregFreeman

Share this post


Link to post
Share on other sites

I definitely second the idea of nuking the mutation history (the most unwanted feature in LabVIEW I have ever come across). 

Share this post


Link to post
Share on other sites
7 hours ago, Neil Pate said:

the most unwanted feature in LabVIEW I have ever come across

Sorry I couldn't hear you over the sound of my ZOOOOOOM!

Okay maybe zoom wasn't my number 1 request, but it likely was the number one request by over all.  Maybe XNodes would be my most requested features.  VIMs gets 80% there.

  • Like 1

Share this post


Link to post
Share on other sites
7 hours ago, Neil Pate said:

I definitely second the idea of nuking the mutation history (the most unwanted feature in LabVIEW I have ever come across). 

It's not a feature. It is an internal class mechanic that you were never supposed to realise was there.

Share this post


Link to post
Share on other sites

When you are trying to hide something, making it smaller is generally preferable...  Making it gigantic almost ensures it will be found...

Share this post


Link to post
Share on other sites

Well...didn't fix it per se. But, we did a build WITHOUT normalizing the string array (i.e. no code changes) and it's using drastically less memory in the EXE than the dev environment....We're talking like 600 mb of memory usage instead of 2.5 gb.

My guess now is having debugging on in some of these VIs is causing issues in the dev environment. Probably copies everywhere. Either way, normalizing things was a much more memory efficient way of doing this and is a needed improvement. Rather than have 24 classes each with 80k strings, many of which are duplicates, we have 24 classes each with about 2k strings, and we have 80k integers that point to an index from that string array.

As much as I'd love to dig into the LabVIEW memory manager to truly understand what's happening in the dev environment (not), I am just going to put this in the "no longer a problem" column and move on. 

Share this post


Link to post
Share on other sites

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.


  • Similar Content

    • By Ryan Vallieu
      I have seemingly found an issue with the shipping example code for Nested Malleable VIs.  Another user has verified that he saw the same behavior in 2019.
       
      I am working through the examples and the presentation from NIWeek 2019.  In running the Lesson 2b code (C:\Program Files (x86)\National Instruments\LabVIEW 2019\examples\Malleable VIs\Nested Malleable VIs) I found the Equals.vi in the class was not being leveraged and the search failed.  When I went to my LabVIEW 2018 machine and ran the Lesson 2b.vi the code worked to find the element by correctly leveraging the in-class Equals.vi.
      One difference I see is that in the 2018 example the Equal.vi is in the example folder with the code, and in 2019 the Equal.vi has been moved to VI.lib - otherwise the code looks to be the same.  The Equals.vi code looks identical, and the calling VIM look identical.  I posted on the LabVIEW NI.com forum here: 
      https://forums.ni.com/t5/LabVIEW/LabVIEW-2019-Malleable-VIs-Shipping-Examples-Lesson-2b-Nested/m-p/3966044/highlight/false#M1129678
       
      I am trying to determine what may have broken or changed between the implementation in 2018 and 2019, visually the code looks the same.
    • By GregFreeman
      I currently have a project that I am refactoring. There is a lot of coupling that is not sitting well with me due to typedefs belonging to a class, then getting bundled into another class which is then fired off as event data.
      Effectively, I have class A with a public typedef, then class B contains ClassA.typedef and then class B gets fired off in an event to class C to be handled. Class C now has a dependency on class A which is causing a lot of coupling I don't want.
      For my real world example I query a bunch of data from our MES, which results in a bunch of typedef controls on the connector panes of those VIs. Those typedefs belong to the MES class. I then want to bundle all that data into a TestConfig class and send that via an event to our Tester class. But, now our tester has a dependency on the MES.
      I see a few ways to handle this. First is move the typedefs currently in the MES class, to the TestConfig class. The MES VIs will now have the typedefs from the TestConfig class on their connector panes, but at least the dependency is the correct "direction." Or, I can move the typedefs out of classes all together, but then I am not sure the best way to organize them. Looking for how others have handled these sorts of dependencies.
       
    • By Voklaif
      Hello all,
      I am programming with LabVIEW for around 2 years and was recently stumbled upon LVOOP.
      I am required to write a communication protocol to work with a micro-controller, which later will be also used for ATP and debug purposes.
      I want to build the program "correctly" from the beginning so it will be maintainable and flexible to additions and changes.
      My natural way of building a program would have been a queued state machine, with several loops, each loop is in charge of a different module (one for GUI obviously), but as I stated in the beginning, I want to use LVOOP.
      Does anyone have a LVOOP project I can use as reference? I've searched online and found some nice examples, but they are small and teach you the basic stuff.
      For me it's important to see the how to use the project tree wisely, where to place the classes, see the managing loop and to learn as much as possible before I create one of my own.
      Thanks in advance,
      Voklaif
    • By ted Francis
      I am new to LVOOP and have jsut started writing my first LVOOP program which I have attached.
      I would appreciate greatly help with the question I have
      Thank you in advance 
      Ted
      This vi will perform two tasks 
      1.Generating Report data sheet for metrology 
      2. updating the scales in a MAX .nce file
      1. Metrology will input calibration information into the tables on the tabs
      Metrology will then click "Update Tables" then "Create Report ( create report section of code is not yet written
      Update Tables will write all information entered in the tabs to class varaibles and will also delete current Max informatiomn
      2. Metrology will click "Load NCE Scale"
      vi will prompt for nce file to load and then once file is selected, display existing scales for two channels (Current Motor 1 and 
      Current Motor 2)
      Metrology will then click "Update Scales"  the program will replace the existing scales with those entered in Step 5.14 and 5.15
      from the tables on the tab
      Question 1.  Steps 5.14 and 5.15 are needed by both classes ( Table Variable and MAX) - what is the best way to share this information
       
      CAT0000032 Class Version.zip
    • By shoneill
      I was browing through the actor framework discussions on the NI site yesterday and I came across a statement by AQ.
      Never inherit a concrete class from another concrete class
      I had to think about that for a second.  The more I think about it, the more I realise that all of the LVOOP software I have been writing more or less adheres to this idea.  But I had never seen it stated so succinctly before.  Now that may just be down to me being a bit slow and all, but in the muddy and murky world of "correct" in OOP-land, this seems to be a pretty good rult to hold on to.
      Are there others which can help wannabe plebs like me grasp the correct notions a bit better?  How about only ever calling concrete methods from within the owning class, never from without?  I'm learning for a long time now, but somehow, my expectations of LVOOP and the reality always seem a little disconnected.  AQs statement above helped crystallise out some things which, up to that point, had been a bit nebulous in my mind.  Well, I say I'm learning..... I'm certainly using my brain to investigate the subject, whether or not I'm actually LEARNING is a matter for discussion... The older I get, the less sure I am that I've actually properly grasped something.  The old grey cells just seem to get more sceptical with time.  Maybe that in itself is learning...
×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.