Optimize table updates

[Grampa_of_Oliva_n_Eden]

I have run into this situation more than once and its time to gather up ideas! When attempting to transfer large data -set to a table, we are seeing delay times of from 3 to 6 seconds depending on what machine we run our benchmarks. The data set can be 250K lines and can be as large as 30Meg on disk! We have issolated the performance issue to ONLY the time required to update the table. Using the technique to defer FP updates, hide table, update table, show table, undefer updates has improved performance. So.... I am toying with writing my own Xcontrol but before I do, I'm looking for other ideas and suggestions. So what do YOU think? Ben Cross posted on NI at

http://forums.ni.com/ni/board/message?boar...2&jump=true

[Neville D]

My only thought is: who is the user? I would like to meet him/her since they can go through 250,000 lines of a table.. !

Seriously, display only the bits of interest, and store the rest in a shift-reg.

Neville.

[LAVA 1.0 Content]

QUOTE(Neville D @ Sep 7 2007, 01:03 PM)

I would like to meet him/her since they can go through 250,000 lines of a table.. !

Seriously, display only the bits of interest, and store the rest in a shift-reg.

I was going to say the same thing. Use Array subset and only show a table on the screen as big as necessary.

I recalled that MS Excel only allows upto 65536 rows but had to check; In Excel 2007 you can no have up to 1,048,576 rows; Ouch! Talk about repetitive stress injury and mouse miles!

As far as the XControl; that might be a nice idea. I'm gonna play with an idea and see what I get...

[Grampa_of_Oliva_n_Eden]

QUOTE(LV Punk @ Sep 7 2007, 01:24 PM)

I was going to say the same thing. Use Array subset and only show a table on the screen as big as necessary.

...

As far as the XControl; that might be a nice idea. I'm gonna play with an idea and see what I get...

My customers' customers have auditors to keep happy.

I have been putting off the Xcontrol learning curve until I needed it.

It just may be time.

Ben

[Ton Plomp]

QUOTE(Ben @ Sep 7 2007, 07:47 PM)

I have been putting off the Xcontrol learning curve until I needed it.

It just may be time.

YES IT IS,

now I'm gonna tell you why.

1. 8.5 is way much faster on XControls
   
2. 8.5 is still XControl Bugfree (one month after release )
   
3. XControl will force you to think event driven
   
4. I like them
   

Ton

[Grampa_of_Oliva_n_Eden]

QUOTE(tcplomp @ Sep 7 2007, 02:41 PM)

YES IT IS,

now I'm gonna tell you why.

1. 8.5 is way much faster on XControls
   
2. 8.5 is still XControl Bugfree (one month after release )
   
3. XControl will force you to think event driven
   
4. I like them
   

Ton

Damn it! Y'all are always giving me reasons to learn more.

Thank you!

Ben

[Ton Plomp]

QUOTE(Ben @ Sep 7 2007, 08:51 PM)

Damn it! Y'all are always giving me reasons to learn more.

Thank you!

Ben

And I missed number 5:

XControls prepare you on XNodes whenever they are released.

Now I think of that, has anyone ever tried to build Express VIs, I could assume that is like building an XNode?

Ton

[LAVA 1.0 Content]

We build Express VIs all the time here! When coupled w/ teststand, they are VERY handy!! We don't use them much if at all in our code though. What do you want to know about making them?

It's actually very easy and very unlike creating Xnodes

[ragglefrock]

QUOTE(LV Punk @ Sep 7 2007, 12:24 PM)

I was going to say the same thing. Use Array subset and only show a table on the screen as big as necessary.

I recalled that MS Excel only allows upto 65536 rows but had to check; In Excel 2007 you can no have up to 1,048,576 rows; Ouch! Talk about repetitive stress injury and mouse miles!

As far as the XControl; that might be a nice idea. I'm gonna play with an idea and see what I get...

I agree with only showing as much data as necessary. You could use custom scrollbar controls to emulate real scrollbars for the table. When their value changes, get the new bounds for the large table and replace the visible data. This way the table never really stores more than say 200 cells worth of data. Here's an 8.5 example (non-XControl, sorry!)

[LAVA 1.0 Content]

One thing to add on, is that if you only show a subset of a very large list sometimes you don't want to update the entire data set as you scroll.

Here is an alternative option that I have seen in some software packages.

Instead of updating the table as you scroll, have a hovering tip strip or something of the like, that is linked to a column and displays what would be the topmost item if you stopped scrolling at that point.

Sometimes row number is enough, sometimes something else makes more sense.

[Grampa_of_Oliva_n_Eden]

QUOTE(NormKirchner @ Sep 7 2007, 05:46 PM)

One thing to add on, is that if you only show a subset of a very large list sometimes you don't want to update the entire data set as you scroll.

Here is an alternative option that I have seen in some software packages.

Instead of updating the table as you scroll, have a hovering tip strip or something of the like, that is linked to a column and displays what would be the topmost item if you stopped scrolling at that point.

Sometimes row number is enough, sometimes something else makes more sense.

Now your talking!

Lets see what I can figure out this week-end.

Ben

[Ton Plomp]

QUOTE(NormKirchner @ Sep 7 2007, 09:38 PM)

We build Express VIs all the time here! When coupled w/ teststand, they are VERY handy!! We don't use them much if at all in our code though. What do you want to know about making them?

It's actually very easy and very unlike creating Xnodes

Allright, I was assuming this because you'd had to react to a drop reaction, and configuration reactions.

But it might be nice if some info about Express VIs was added to the WIKI

Ton