Reds

3 hours ago, ShaunR said:

QUIC

In theory maybe....

But how would I get LabVIEW to create QUIC connections?

8 hours ago, djolivet said:

@Reds

How do you handle your office chores with this setup?  Do you have email/spreadsheet/wordprocessor/etc loaded to each of the various partitions/VHDs?

No - I've always maintained completely separate office and development machines on two separate physical Ethernet networks.

In my opinion, that will be unavoidable in the modern cyber threat environment. You can't give people (or even yourself) local Admin account access on any network that is used for browsing the Internet, reading email or doing normal office stuff.

Personally, if I'm doing Dev stuff on my dev machine, I'll pop open a RDP session into my office machine to do normal office stuff.

Devs needs to have Admin account access on their dev machines. Devs should never have Admin account access on their office machines. Full stop.

2 hours ago, Matt_AM1 said:

Reds,

 

NGL, I don't understand what you're saying, but I'm gunna take a stab at it.

 

It sounds like you have massive SSD with the various partitions for the specific versions of LabVIEW on it. 

Yup.

From the VERY brief googling I've done, it sounds like BCDBoot sets up a boot a virtual hard drive (VHD).

Yes, BCDBoot can definitely do that. But it can also just tell the PC to boot using a copy of Windows that is on a partition somewhere else on your drive. I guess you could say you're telling it to use a non-standard partition.

  The VHD can be booted from windows (I am not sure what side-by side means with respect to booting the VHD, but I am assuming it means I can run the "real OS" and the "LV OS" versions of windows at the same time). 

Nope. You're typically only running one copy of Windows at a time if you're using the Windows Boot manager (BCDBoot).

I think BCDEdit allows me to modify the boot of the VHD or it is the command line execution that creates the VHD's and BCDBoot is the one that boots the VHD.  

BCDEdit is how you tell your PC: "When you turn on, use the copy of Windows I have over here on this alternate partition...". You can also use BCDEdit to say "Don't boot from a partition at all, boot from this completely self-contained VHD file instead".

So, assuming I am on the rightish track, would the VHD only contain information related to LV specifically, such as LV/MAX/other NI software/git with its specific diff/merge for the LV version?  If so, then what would the "real OS" be in charge of with respect to LabVIEW?  

The VHD contains an entirely separate copy of Windows. There are no differentials. When you boot from a VHD you have a Windows environment that doesn't know anything at all about any other copies of Windows you have in different drive partitions. Native Boot from VHD is a completely and totally standalone environment. In fact, you could actually boot your PC directly from a single VHD file every day, and not even have a traditional Windows partition.

2 hours ago, Matt_AM1 said:

 

I guess what I'm not fully putting together is, why virtual hard drives?  From my very basic (mis)understanding of things, the VHD can be mounted/unmounted at will to swap between the versions of LV, so why couldn't you do that with multiple partitions on a SSD? It sounds like that was Tim_S's solution, with the various flash drives being the VHDs in this scenario.  Googling the difference, I see that VHDs are meant for virtual machines, so I am assuming that when you BCDBoot the VHD, you're booting a virtual machine's image to work in tandem with windows which contains the LV version and what not.

 

Matt

I have a system I've been using for years that works pretty well.

Windows has a very capable boot manager. Once setup, it will prompt you to select which Operating System you want to boot into at system startup. You can have a different copy of Windows installed on basically every partition on your GPT formatted hard disk. If you start googling around for the "bcedit" and "bcdboot" Windows commands, you'll quickly find yourself deep into this rabbit hole. Sadly, I have the options for those command line tools memorized because I do this so frequently, and for so many years

If you really want to get sophisticated, you can also use the Windows boot manager to boot *directly* into a VHD - without any host operating system running. This allows your VHD *direct* access to all the NI hardware without any translation layers in between. This is known as "Native VHD Boot" in the Microsoft documentation.

If you're booting directly from VHD's - these VHD's are portable to new machines. You're just going to have get very familiar with the bcdedit, bcdboot, and diskpart command line tools. It's all doable, and it all works, but these tools will start to make your feel like you've living in Linux world, and not Microsoft world . It just takes time to learn and get familiar with the tools.

I've been taking this approach for many years - I can't even imagine living without these capabilities.

I don't know why anyone would mess around with docker or any of that stuff. The capabilities built directly into Windows meet 100% of my needs. The only special thing you need is a generously sized SSD, formatted with many different partitions (one partition per Windows copy).

I have a LabVIEW application that produces a continuous stream of binary measurement data. Right now, this measurement data streams into a LabVIEW notifier, and multiple "listeners" can "subscribe" to that data and monitor what is happening. Both the producer (publisher) and consumers (subscribers) are all native to LabVIEW right now, so it  all works great with LabVIEW notifiers.

But, now I want to have a non-LabVIEW listener. What to do?

The semi-obvious choice would be to switch the code so that the data would be streamed on TCP/IP instead. But what higher-layer protocol would you want to use for that? It seems to make like the RESTFul and gRPC models are more client-server oriented. I want something this a very loosely couple producer-consumer model, where you have one producer and any number of listeners.

What are some good candidates?

18 hours ago, GregSands said:

Reds, did you miss my earlier comment - while lvanlys.dll does not use multicore functions, the LabVIEW Multicore Analysis and Sparse Matrix Toolkit does. Despite X's comments on the lack of documentation, which are quite fair and accurate, I'm seeing huge speedups on my i7-12700F with 8P+4E cores (20 logical processors according to CPU Information).

Thanks Greg, appreciate it! I think I need to figure out how this MASM toolkit would be installed with our EXE on customer systems. If I can integrate the MASM installer into our LabVIEW installer package, this could work for us. I'll investigate...

You make a good point about single precision being all that's necessary also.

Interesting, thanks Rolf! So then....I wonder if lvanlys.dll is not written to call the multi-core version of the FFT in MKL?

Intel vTune Profiler also provides the hard evidence:

And if this is not a failing grade for lvanlys.dll, I don't know what is:

But if I use Resource Monitor to look at the DLL's my LabVIEW-built EXE is calling, none of the DLL's have "MKL" in the filename. It seems to me like LabVIEW is using our old friend "lvanlys.dll" to perform FFT calculations. 

Can anyone confirm my suspicion??

If anyone else is interested, here's some evidence from one of my machines, indicating that NI is installing various versions of the Intel MKL. They look reasonably up-to-date. So I'm still unsure why they're not taking advantage of multi-core...

"Documentation is aspirational" is a great line that I'm totally stealing. 😂

I laid down the big bucks for the top-of-the-line 18-core PXIe-8881. I feel like the engineering gods are mocking me as 17 of the 18 cores are idle when I run my FFT. I mean, what is the point of an 18-core PXI CPU if NI's default math library can only use one of them? Is there any other T&M application besides math/analysis that would actually benefit from 18 cores? Maybe I need to use MatLab to access all 18 cores? The whole thing is kind of crazy if you ask me.

Thanks Rolf.

i was actually looking at switching to the Intel MKL (instead of LabVIEW native) in a bid to improve multi core performance. But if LabVIEW is already using MKL, I wonder why it doesn’t seem to take advantage of multi core for FFT?

Does anyone know which math library LabVIEW uses to do the FFT and vector math operations? 

Has it been updated over the years to accomodate the latest Intel CPU extensions, or has it been static over time?

21 hours ago, GregSands said:

I'm guessing there must be more to your question, but based on your specs, I'd be asking whether it was worth spending time and effort deleting a relatively tiny part of a file. 100k out of tens of GB? I'd just leave it there and work around it!

Yeah, I wish that was possible.

The problem is that a third party analysis application can't understand the first 100kB of the file, and so that software incorrectly concludes that the entire remainder of the file must be corrupt.

Quote

Well. What is your immediate pain? Can you elaborate?

The jumbo file is recorded with a bunch of header data starting at file offset zero. This header data is not actually useful, and it actually causes a third party analysis application to think that the recorded data is corrupt. If I can manage to delete only the header data at the beginning of the file, then the third party analysis application can open and analyze the file without throwing any errors.

Yeah, I dug into the Microsoft docs on sparse files, and I don't think that technology is going to solve my problem after all. Cool stuff. Good to know. But doesn't seem like it's going to solve my immediate pain.

I guess what's really needed is a way to modify the NTFS Master File Table (MFT) to modify the starting offset of a given file. But, I didn't actually see any Win32 API's that could do that. I'm sure it must be possible to do that with some bit banging, but I'd probably be getting in way over my head if I tried to modify the MFT using a method that was not Microsoft endorsed.

9 hours ago, ShaunR said:

Indeed. However. Hole punching is much, much faster. If you are talking terabytes, it's the only way really.

Set the file to be Sparse. Write 100k zero's to the beginning. Job done (sort of).

Yes, we are indeed talking terabytes.

Reading the original file and writing a new one will take many minutes. It will also require the storage medium to have terabytes of free space available to perform the operation. Maybe even a whole separate partition would need to be set aside with free space. "Copy only the parts you want to save" is certainly the obvious solution, but it's not a good one for really big files.

Thanks for the Microsoft link to Sparse files. I"ll dig into that and learn more.

For sure, the biggest hazard of LabVIEW is that it permits you to easily blur the lines between "data acqusition" and "user interface" as ShaunR points out. So dangerous.

I guess a SQL database is one way to draw a hard line in the sand between these two components. But I actually prefer to do it with an API (implemented as a LabVIEW Packed Library). In my opinion, the healthiest architecture decision you can make up front is that your Graphical User Interface will only be allowed to call an API (which you define) in order to configure, read, update, and delete the acquired data. If you have *any* instrument driver or communications code "above" your API, then you've violated your architectural contract.

Thanks for the ideas fellas, I'll report back on my progress.

I guess I was hoping for some Win32 API that could tweak the NTFS tables to change the starting sector of a file (but I guess that would be too easy).

Let's say you have a really big binary file. A file so big that it won't fit into your PC RAM. Now let's say you wanted to delete the *first* 100kB in that file, and leave the rest of the file alone.

How would you do that? Can it be done quickly? Can it be done without creating a whole new file?

They also say this in the annual report: "We have empowered hundreds of thousands of loyal users of LabVIEW, a unique graphical software platform optimized for engineers, and numerous other application software tools".

But I don't see how that statement could possibly be supported by current facts, unless they're including everyone who has ever used LabVIEW at any time in the last 30 years.

On 8/21/2023 at 12:12 PM, X___ said:

$37M/quarter at $2.5K a license/year, that's  ~60,000 yearly licenses (assuming all are LabVIEW Pro only).

OK, I did some more digging in the investor's annual report. $37mm/quarter consists of:

• LabVIEW
• LabWindows
• Measurement Studio (is this still a thing?)
• TestStand
• VeriStand
• Flexlogger
• SystemLink
• Optimal Plus
• DIAdem

43% of their business in the Americas (incl. Mexico and South American presumably). So we could guess that's about $50mm/year in US license revenue for *all* of the above products.  And if we assumed all of those licenses averaged out to $2.5k/year each, that's 20,000 active US licenses for all NI software products. Now lots of people are going to have both LabVIEW *and* TestStand licenses, so the number of active LabVIEW developers is presumably much less than 20,000 in the US.

Thoughts on this analysis??

22 hours ago, X___ said:

Precisely.

The NI P&L statement has a revenue line item called "Software Maintenance" at about $37mm per quarter. It's trending downward. Does any know if LabVIEW revenue is included in this P&L line item? What other things might be in this bucket besides LabVIEW?

9 hours ago, Rolf Kalbermatter said:

That is partly NI's work. They were pretty aggressive about defending their idea by applying for quite a few patents and defending them too. Of course if you go to the trouble to apply for a patent you have to be willing to defend it, otherwise you eventually use the right to a patent anyways.

And they did buy up some companies that had something similar to LabVIEW, such as DasyLab for instance, although in my opinion DasyLab didn't quite go beyond the standard "wire some icons together" similar to what HPVee did, and what Node-Red is doing too. But they tried to use some structures that were darn close to LabVIEW loops and that was a prominent NI patent. So NI eventually approached them and made them the offer to either buy them or meet them in front of a judge. The rest is history.

100% this. Also, don't forget they famously sued MathWorks when MatLab started to roll out graphical programming tools that looked a little bit too much like LabVIEW. That cold war finally ended, but only very recently.

Y'all should read the PDF versions of the letters Emerson sent before getting your panties in a bunch about LabVIEW's future. Quoting from one of Emerson's letters:

We are very excited about the combination of our two firms and the potential we can achieve together. Emerson has long admired NI as a technology leader in the electronic test and measurement industry, a complementary adjacency to our Automation Solutions business with a similar technology stack of intelligent devices, controls, and software. We have been particularly impressed with NI’s portfolio including modular intelligent devices and the LabVIEW suite of offerings, as well as NI’s industry stewardship over many decades in this space. Combining NI with Emerson would lead to significant opportunities for both of our teams and further develop our position as a premier global automation company.

2 hours ago, ShaunR said:

Maybe. Although he's getting on a bit and has an interest in Alzheimer's. Could also be looking at passing an inheritance where a cash drop like this would be great. Difficult to understand he'd be happy about his life's work being swallowed by another corporation just for a few decimal points of his billions in cash.

Interesting he talks about "stakeholders" instead of shareholders but it does not bode well for NI.

The plot thickens!

I didn't quote the part where he explicitly states his disappointment with NI's "execution" and how it has lost focus on it's "stakeholders" including customers and partners.  Check out the article and decide for yourself. I mean, he's not wrong if you ask me.