Dynamic UI Elements using HTML & CSS - Thumbnail Grid

[Ravi Beniwal]

We often need to let our users select an item from a list. When it comes to making UI elements to achieve this that are a bit more sophisticated than a table, listbox or tree, LabVIEW's current capabilities leave a lot to be desired.

Here's an alternative using HTML and CSS. The current example provides an interface for selecting an item from a thumbnail grid, similar to Windows Explorer.

Run "Thumbnail Grid .NET Browser.vi" or "Thumbnail Grid ActiveX Browser.vi" from the attached code folder. The .NET version provides a cleaner interface as it provides a bit more control over its behavior. You can select any number of images and any size of the pane (read Web Browser control). You can set any thumbnails as disabled, which will appear grayed out. I have included some images for demo from the Open Icon Library.

The code calculates the best grid pattern (the one with the smallest aspect ratio) and generates html code for displaying the images as a table and then generates CSS code for styling the elements. The web browser takes care of the visual effects and LabVIEW maps mouse cursor position to the calculated grid pattern for detecting the selected thumbnail. We could get fancier and use web services to trigger a selection, but that may be an overkill for this case.

This approach can be extended to displaying dynamically created tabs or list boxes with bigger symbols than the current 16px limit for list box symbols. I will create examples of these over the next few days.

The only issue I've noticed with this approach is that every time we change/refresh the browser, we hear the "Start Navigation" sound that is part of the Explorer shell. It may or may not be OK to disable that in all cases. I guess we can find a clean solution as we experiment more with this code.

Thumbnail Grid.zip

[Yair]

I once blindly implemented this algorithm which is supposed to find the optimal setup for a grid (although I didn't bother to clean it up or optimize it, since I never found any actual use for it) and running your VI through the test cases shows a couple where your results are different. You can see the attached code which includes the test cases.

Max squared.zip

[hooovahh]

Very nice, but I didn't like how panel resize wasn't handled, so I modified the .Net version to basically perform a refresh after a resize. I also wanted to see the disable function in action, so I added code to disable items randomly when a refresh takes place. One side effect is I see flickering of the images when resizing slowly. I added a defer front panel at one point but that didn't seem to fix it.

Regarding the "Click" noise, I'm wondering if you can query Windows and ask what the volume settings are (muted or not and volume level) then mute the speaker, then set the speaker back to the previous settings that it was before a refresh. There are examples on how to mute speakers using Windows DLL calls, but I don't know how to query the current settings.

Thumbnail Grid .NET Browser.vi

Edited November 26, 2012 by hooovahh

[ShaunR]

The LabVIEW UI has sucked for quite a long time now, but If your going to go for HTML and CSS you might as well take it completely out of LabVIEW and make it platform independent.

Unfortunately, When it comes to artistic talent, I wasn't even in the building when they were handing it out So as much as I complain about the LV UI, I can't really do any better. The guys/gals that make those interfaces for the coding challenges just blow me away.

Any UI in CSS, SVG and HTML that I do takes weeks and looks, at best, mediocre.

Edited November 26, 2012 by ShaunR

[Ravi Beniwal]

Hi hooovahh, thanks for adding the test cases.

I did test resize event handling and then noticed the same issues that you did. Since we are generating a new html document and loading it in the browser every time the size changes, it was really slow and the clicks were annoying.

For handling panel resize event, while still doing most of this stuff in LabVIEW, I would rather refresh in a separate loop and use lossy enqueue to a single element queue to pass on the panel resize message so that we don't refresh for every single resize event LabVIEW fires. Ideally, we should resize the thumbnails using JavaScript onresize event and calculate the thumbnail sizes within JavaScript, not in LabVIEW. Yair has lots of munitions on that. May be we can put together a quick example.

[Darin]

I once (briefly) looked into eliminating those clicks. Changing volume stopped the clicks, and whatever music was streaming at the time. The only effective method I found was reading/clearing/resetting the following registry key:

HKEY_CURRENT_USER\AppEvents\Schemes\Apps\Explorer\Navigating\.Current

You can read the current value, empty it, do your thing, then restore the original value.

[GregFreeman]

The LabVIEW UI has sucked for quite a long time now, but If your going to go for HTML and CSS you might as well take it completely out of LabVIEW and make it platform independent.

Unfortunately, When it comes to artistic talent, I wasn't even in the building when they were handing it out So as much as I complain about the LV UI, I can't really do any better. The guys/gals that make those interfaces for the coding challenges just blow me away.

Any UI in CSS, SVG and HTML that I do takes weeks and looks, at best, mediocre.

I feel your pain. I am no GIMP or photoshop wiz either, which makes certain things quite difficult. I am running into this more lately, where customers say "I want it to look like .NET does" but they want to use LabVIEW for the sake of quick development. I understand the NI "branding" that comes with a LabVIEW-looking UI, but I think it is important for NI to make some headway on this front to allow us more modern UIs. To be sure, the silver controls were a step in the right direction (though I still find them to take up way too much space). But, when I drop a .NET control and see the ridiculous amount of functionality available, I realize LabVIEW definitely is still lacking.

[hooovahh]

Hi hooovahh, thanks for adding the test cases.

I did test resize event handling and then noticed the same issues that you did. Since we are generating a new html document and loading it in the browser every time the size changes, it was really slow and the clicks were annoying.

For handling panel resize event, while still doing most of this stuff in LabVIEW, I would rather refresh in a separate loop and use lossy enqueue to a single element queue to pass on the panel resize message so that we don't refresh for every single resize event LabVIEW fires. Ideally, we should resize the thumbnails using JavaScript onresize event and calculate the thumbnail sizes within JavaScript, not in LabVIEW. Yair has lots of munitions on that. May be we can put together a quick example.

I'm not sure how close you looked at my resize code, but it is essentially a lossy queue. If you resize a windows by dragging you get lets say 100 events to resize, but I don't have the resize take place, all I do in those 100 events is set a boolean to TRUE, which is handled in the timeout case. So the resize only takes place if another resize hasn't been fired in the last 0ms. This isn't perfect but much better than performing 100 resizes on the UI.

[Ravi Beniwal]

Ah, I totally missed that. It is a neat trick!

[mje]

Re: clicking sound from navigation

Rather than rendering a file via a navigation event, in the past I've directly manipulated the document object model from LabVIEW. I'm not sure if any of the above code does that, just throwing it out there...

[asbo]

One side effect is I see flickering of the images when resizing slowly. I added a defer front panel at one point but that didn't seem to fix it.

This doesn't work because LabVIEW doesn't control the rendering anymore, it's moved into dotNET's domain.

I feel your pain. I am no GIMP or photoshop wiz either, which makes certain things quite difficult. I am running into this more lately, where customers say "I want it to look like .NET does" but they want to use LabVIEW for the sake of quick development. I understand the NI "branding" that comes with a LabVIEW-looking UI, but I think it is important for NI to make some headway on this front to allow us more modern UIs. To be sure, the silver controls were a step in the right direction (though I still find them to take up way too much space). But, when I drop a .NET control and see the ridiculous amount of functionality available, I realize LabVIEW definitely is still lacking.

In terms of aesthetics, the system controls get you 90% of the way there. WRT functionality adjustments, I think LV will always be lacking because it's such a low value-add.

[hooovahh]

This doesn't work because LabVIEW doesn't control the rendering anymore, it's moved into dotNET's domain.

Yeah I kinda figured that's why it behaves the way it does.

[hooovahh]

Okay so I promise I haven't been working on this since November, but I did come up with a solution that uses pure LabVIEW (mostly) and doesn't rely on ActiveX, .Net, or HTML.

 

It is very rough around the edges.  There is little error handling, there is at least one default subVI Icon, and I'm pretty sure there are some unclosed references.  But in any case, extract the zip and run Main Parent.  It will call a dynamic number of Children, each one being an item in the grid.  They are then put into the Parent using some Windows DLLs.  Then when there is a mouse down we can fire the event and determine what one was clicked.  It still supports captions, mouse over, and disabling items.  It also now supports resizing without the odd clicking sound.  The images look much more crappy because of how LabVIEW loads and then zooms the images.

 

You can also change the number of items while it is running.  For me it took about 700ms to load 8 images but I do have 8 logical CPU cores.  Alot of the Parent/Children code was taken from the DynaPanels found here.

Native Test.zip

[viSci]

Next Generation of User Interfaces...

 

http://zone.ni.com/wv/app/doc/p/id/wv-3567

 

I really love LV so it is especially disheartening to see what we are missing...

[hooovahh]

I promise I haven't been working on this for 5 years.  But I thought I'd come back with something new that I think is a bit better in some ways and a bit worst in others.  This method works a bit better when it comes to encapsulation.  I wrote a single Image Grid class that can be created, updated, and destroyed.  When creating it you specify the Subpanel that it should be inserted into.  This means that we can likely turn this into an XControl one day.  The previous method of having parent/child windows floating around that needed to be resized and moved, complicated that encapsulation and likely never could have been in an XControl.

Resizing in my opinion is done better here too.  Instead of generating a resize event, and then having to resize each window, we just let LabVIEW take care of it.  LabVIEW is pretty good at resizing when using panes and having a single object fit to that pane.  So when a resize occurs the only thing that is done is the optimal number of rows and columns are calculated and if there is no change then nothing else needs to happen in the G code.

Minimum pane sizes are used so that hopefully things don't get too small if the subpanel is set to fit to the pane.

Now the new limitations...this is limited to the number of subpanels I made and at the moment this is 20 rows, and 20 columns.  It is possible I could have gotten creative and generated all the prime number of subpanels, and then used dynamic subpanel subpanel insertion to get an unlimited number but wow that sounded difficult.  I figured how often will I need more than 20 rows or columns?

Another limitation is at the moment you cannot add or remove images without closing the session and opening a new one.  The properties of each image can be manipulated quickly by providing the index, and the new Caption, Disabled Status, Image Path, or Background color (alpha layers supported BTW).

Also no way to modify the caption other than the text.  I'd like to add a hide/show ability and a way to change font size.

Anyway source is 2015, uses OpenG and here is a video.  It is slightly smoother to resize than it looks in the video.

https://www.screencast.com/t/qdZMxBuzprH

Image Grid Subpanels Of Subpanels.zip