Reds

18 hours ago, GregSands said:

Reds, did you miss my earlier comment - while lvanlys.dll does not use multicore functions, the LabVIEW Multicore Analysis and Sparse Matrix Toolkit does. Despite X's comments on the lack of documentation, which are quite fair and accurate, I'm seeing huge speedups on my i7-12700F with 8P+4E cores (20 logical processors according to CPU Information).

Thanks Greg, appreciate it! I think I need to figure out how this MASM toolkit would be installed with our EXE on customer systems. If I can integrate the MASM installer into our LabVIEW installer package, this could work for us. I'll investigate...

You make a good point about single precision being all that's necessary also.

Interesting, thanks Rolf! So then....I wonder if lvanlys.dll is not written to call the multi-core version of the FFT in MKL?

Intel vTune Profiler also provides the hard evidence:

And if this is not a failing grade for lvanlys.dll, I don't know what is:

But if I use Resource Monitor to look at the DLL's my LabVIEW-built EXE is calling, none of the DLL's have "MKL" in the filename. It seems to me like LabVIEW is using our old friend "lvanlys.dll" to perform FFT calculations. 

Can anyone confirm my suspicion??

If anyone else is interested, here's some evidence from one of my machines, indicating that NI is installing various versions of the Intel MKL. They look reasonably up-to-date. So I'm still unsure why they're not taking advantage of multi-core...

"Documentation is aspirational" is a great line that I'm totally stealing. 😂

I laid down the big bucks for the top-of-the-line 18-core PXIe-8881. I feel like the engineering gods are mocking me as 17 of the 18 cores are idle when I run my FFT. I mean, what is the point of an 18-core PXI CPU if NI's default math library can only use one of them? Is there any other T&M application besides math/analysis that would actually benefit from 18 cores? Maybe I need to use MatLab to access all 18 cores? The whole thing is kind of crazy if you ask me.

Thanks Rolf.

i was actually looking at switching to the Intel MKL (instead of LabVIEW native) in a bid to improve multi core performance. But if LabVIEW is already using MKL, I wonder why it doesn’t seem to take advantage of multi core for FFT?

Does anyone know which math library LabVIEW uses to do the FFT and vector math operations? 

Has it been updated over the years to accomodate the latest Intel CPU extensions, or has it been static over time?

21 hours ago, GregSands said:

I'm guessing there must be more to your question, but based on your specs, I'd be asking whether it was worth spending time and effort deleting a relatively tiny part of a file. 100k out of tens of GB? I'd just leave it there and work around it!

Yeah, I wish that was possible.

The problem is that a third party analysis application can't understand the first 100kB of the file, and so that software incorrectly concludes that the entire remainder of the file must be corrupt.

Quote

Well. What is your immediate pain? Can you elaborate?

The jumbo file is recorded with a bunch of header data starting at file offset zero. This header data is not actually useful, and it actually causes a third party analysis application to think that the recorded data is corrupt. If I can manage to delete only the header data at the beginning of the file, then the third party analysis application can open and analyze the file without throwing any errors.

Yeah, I dug into the Microsoft docs on sparse files, and I don't think that technology is going to solve my problem after all. Cool stuff. Good to know. But doesn't seem like it's going to solve my immediate pain.

I guess what's really needed is a way to modify the NTFS Master File Table (MFT) to modify the starting offset of a given file. But, I didn't actually see any Win32 API's that could do that. I'm sure it must be possible to do that with some bit banging, but I'd probably be getting in way over my head if I tried to modify the MFT using a method that was not Microsoft endorsed.

9 hours ago, ShaunR said:

Indeed. However. Hole punching is much, much faster. If you are talking terabytes, it's the only way really.

Set the file to be Sparse. Write 100k zero's to the beginning. Job done (sort of).

Yes, we are indeed talking terabytes.

Reading the original file and writing a new one will take many minutes. It will also require the storage medium to have terabytes of free space available to perform the operation. Maybe even a whole separate partition would need to be set aside with free space. "Copy only the parts you want to save" is certainly the obvious solution, but it's not a good one for really big files.

Thanks for the Microsoft link to Sparse files. I"ll dig into that and learn more.

For sure, the biggest hazard of LabVIEW is that it permits you to easily blur the lines between "data acqusition" and "user interface" as ShaunR points out. So dangerous.

I guess a SQL database is one way to draw a hard line in the sand between these two components. But I actually prefer to do it with an API (implemented as a LabVIEW Packed Library). In my opinion, the healthiest architecture decision you can make up front is that your Graphical User Interface will only be allowed to call an API (which you define) in order to configure, read, update, and delete the acquired data. If you have *any* instrument driver or communications code "above" your API, then you've violated your architectural contract.

Thanks for the ideas fellas, I'll report back on my progress.

I guess I was hoping for some Win32 API that could tweak the NTFS tables to change the starting sector of a file (but I guess that would be too easy).

Let's say you have a really big binary file. A file so big that it won't fit into your PC RAM. Now let's say you wanted to delete the *first* 100kB in that file, and leave the rest of the file alone.

How would you do that? Can it be done quickly? Can it be done without creating a whole new file?

They also say this in the annual report: "We have empowered hundreds of thousands of loyal users of LabVIEW, a unique graphical software platform optimized for engineers, and numerous other application software tools".

But I don't see how that statement could possibly be supported by current facts, unless they're including everyone who has ever used LabVIEW at any time in the last 30 years.

On 8/21/2023 at 12:12 PM, X___ said:

$37M/quarter at $2.5K a license/year, that's  ~60,000 yearly licenses (assuming all are LabVIEW Pro only).

OK, I did some more digging in the investor's annual report. $37mm/quarter consists of:

• LabVIEW
• LabWindows
• Measurement Studio (is this still a thing?)
• TestStand
• VeriStand
• Flexlogger
• SystemLink
• Optimal Plus
• DIAdem

43% of their business in the Americas (incl. Mexico and South American presumably). So we could guess that's about $50mm/year in US license revenue for *all* of the above products.  And if we assumed all of those licenses averaged out to $2.5k/year each, that's 20,000 active US licenses for all NI software products. Now lots of people are going to have both LabVIEW *and* TestStand licenses, so the number of active LabVIEW developers is presumably much less than 20,000 in the US.

Thoughts on this analysis??

22 hours ago, X___ said:

Precisely.

The NI P&L statement has a revenue line item called "Software Maintenance" at about $37mm per quarter. It's trending downward. Does any know if LabVIEW revenue is included in this P&L line item? What other things might be in this bucket besides LabVIEW?

9 hours ago, Rolf Kalbermatter said:

That is partly NI's work. They were pretty aggressive about defending their idea by applying for quite a few patents and defending them too. Of course if you go to the trouble to apply for a patent you have to be willing to defend it, otherwise you eventually use the right to a patent anyways.

And they did buy up some companies that had something similar to LabVIEW, such as DasyLab for instance, although in my opinion DasyLab didn't quite go beyond the standard "wire some icons together" similar to what HPVee did, and what Node-Red is doing too. But they tried to use some structures that were darn close to LabVIEW loops and that was a prominent NI patent. So NI eventually approached them and made them the offer to either buy them or meet them in front of a judge. The rest is history.

100% this. Also, don't forget they famously sued MathWorks when MatLab started to roll out graphical programming tools that looked a little bit too much like LabVIEW. That cold war finally ended, but only very recently.

Y'all should read the PDF versions of the letters Emerson sent before getting your panties in a bunch about LabVIEW's future. Quoting from one of Emerson's letters:

We are very excited about the combination of our two firms and the potential we can achieve together. Emerson has long admired NI as a technology leader in the electronic test and measurement industry, a complementary adjacency to our Automation Solutions business with a similar technology stack of intelligent devices, controls, and software. We have been particularly impressed with NI’s portfolio including modular intelligent devices and the LabVIEW suite of offerings, as well as NI’s industry stewardship over many decades in this space. Combining NI with Emerson would lead to significant opportunities for both of our teams and further develop our position as a premier global automation company.

2 hours ago, ShaunR said:

Maybe. Although he's getting on a bit and has an interest in Alzheimer's. Could also be looking at passing an inheritance where a cash drop like this would be great. Difficult to understand he'd be happy about his life's work being swallowed by another corporation just for a few decimal points of his billions in cash.

Interesting he talks about "stakeholders" instead of shareholders but it does not bode well for NI.

The plot thickens!

I didn't quote the part where he explicitly states his disappointment with NI's "execution" and how it has lost focus on it's "stakeholders" including customers and partners.  Check out the article and decide for yourself. I mean, he's not wrong if you ask me.

Well, Dr. T came out in favor of the transaction. 
 

Thats a pretty significant commentary on how he feels about current management.

https://www.msn.com/en-us/money/companies/national-instruments-co-founder-backs-7-billion-emerson-offer/ar-AA16JEEA

For those interested in an announcement I mentioned at the start of this thread:

8 hours ago, Rolf Kalbermatter said:

It does probably mean that an installer only comes with one version of VI libraries (the supported minimum version) and that it will invoke a mass compile after installation. And there is of course the option that some of those VIs use features that may get depreciated in a future LabVIEW version and eventually get disabled so that some driver VIs might not load without broken error in a far out version of LabVIEW.

The issue I've found is not the LabVIEW vi's bundled with the driver installers. The *real* issue is the property nodes. Some drivers, like NI-RFSA, have tons of things that can only be set through a property node. You can recompile and move/copy vi's to your heart's content, you'll never get those property nodes to work in any version of LabVIEW that is not "supported". I'm certainly hoping that they find a way to allow property nodes from any version of NI-xxxx to work in any future version of LabVIEW. One can dream, right...

Has anyone noted anything particularly noteworthy from NI Connect this week?

One thing I noticed is that NI is going to be decoupling NI driver versions from LabVIEW version numbers. I think this means that you will be able to use any version of LabVIEW with any version on hardware driver in the future. 

Anyone notice any other announcements of interest?

4 hours ago, Rolf Kalbermatter said:

If you can understand this you also can guess where NI is heading. They won't die and their shares will likely not falter. But what they will be has little to do with what they used to be. If LabVIEW still has a place in this I do not know.

Wow. Thank you for a valuable post Rolf. All of that helped me answer and understand so many open questions I had. When I look at recent NI actions from these angles, it does all make sense. 

It doesn’t make me happy, but at least I can understand it.