.NET and VI Profile Metrics

[mje]

Quick question with regards to profiling metrics and .NET interop for LV. The short of it is, where does time spent outside of the LV process show up when using the profile tools in the LV development environment?

I have a VI proof of concept I'm working on, this is the first time I've used .NET in LabVIEW. I'm serializing some pretty hefty XML data. When I profile the current concept, I get a run time on the order of 40 seconds for a moderate sized XML file (150 MB), yet the SubVI time only registered as 1.5 seconds, see below:

   Profile Data: all times in milliseconds, memory usage in kilobytes.		 Begin: Tue, Aug 19, 2008 11:19:16 AM.		 End: Tue, Aug 19, 2008 11:20:08 AM.		 														VI Time   Sub VIs Time   Total Time  # Runs		 Spectrum XML.vi								37125.0   1500.0		 38625.0	 1		 LE_DATA Parse and Append.vi					937.5	 250.0		  1187.5	  579		 Spectrum XML Read - LE_DATA.vi				 265.6	 1187.5		 1453.1	  1		 Trim Whitespace.vi							 218.8	 0.0			218.8	   87144		 LE_DATA Resize.vi							  31.2	  0.0			31.2		176		 Spectrum XML Read - APEX3D.vi				  15.6	  0.0			15.6		1		 Spectrum XML Read - FORMAT.vi				  15.6	  0.0			15.6		2		 Spectrum XML Read - PARAMS.vi				  15.6	  0.0			15.6		1		 Build Error Cluster__ogtk.vi				   0.0	   0.0			0.0		 0		 Spectrum XML Read - MASS_SPECTRUM.vi		   0.0	   0.0			0.0		 1		 Spectrum XML Read - PROCESSING_PARAMETERS.vi   0.0	   0.0			0.0		 1		 Wide String to String.vi					   0.0	   0.0			0.0		 579

The main VI (Spectrum XML.vi) is most definitely not using the processing time shown in the profile, it's job is simply to call a few SubVIs. I'm wondering if all the .NET time is put into the main VI's entry since it is the one that created the original .NET reference that is passed to all subVIs, and it's that object that creates any new instances of .NET objects? Anyone seen this type of behaviour before? I'm considering redoing the little project in C# just to compare performance, to see if the performance hit is coming from the .NET/LV communication.

I'm still working with 8.5, haven't gotten around to 8.5.1 let alone the 8.6 disks yet...

Regards,

-Michael

[LAVA 1.0 Content]

QUOTE (MJE @ Aug 19 2008, 07:07 PM)

Anyone seen this type of behaviour before? I'm considering redoing the little project in C# just to compare performance, to see if the performance hit is coming from the .NET/LV communication.

How do you pass around the data? Via disc or via memory. In the first case there shouldn't be a difference between C# and LabVIEW, however loading .NET items into memory sometimes takes time.

Ton

[mje]

QUOTE (Ton @ Aug 19 2008, 01:17 PM)

How do you pass around the data? Via disc or via memory. In the first case there shouldn't be a difference between C# and LabVIEW, however loading .NET items into memory sometimes takes time.

Well, the XML data is from disk. Save for a few kilobytes of header information, the entire XML file is read via a re-used string buffer allocated in LV (technically, a U16 array since .NET uses wide characters instead of multibyte characters). I've tracked the problem down by literally removing all processing:

http://lavag.org/old_files/monthly_08_2008/post-11742-1219171064.png' target="_blank">

All this VI does is iterate over every element in the XML file and literally does nothing with it. It takes around 37 seconds to execute on my system. The XML structure is pretty simple. Save for a kilobyte or two of header data contained in maybe a few dozen nodes, most of the file is "flat", just a whitespace delimited list of numbers contained within a "data" node (yes, I know it's a not a good use of XML and a terrible waste of space, but it's not my data format).

I'm going to whip up a .NET binary to see if it takes the same time to do "nothing". I should hope that .NET isn't pre-fetching the file, the whole point of using the XmlReader class is to use buffered reading methods such as ReadValueChunk to avoid using strings hundreds of megabytes in size.

Something's fishy. The attached C# code is functionally equivalent to the LV code posted above and executes in under a second.

   using System; using System.Xml;  namespace XmlTestReader {	 class Program	 {		 static void Main(string[] args)		 {			 if (args.Length < 1)			 {				 Console.WriteLine("Please provide an input file as a command line argument.");				 return;			 } 			 DateTime start_time = DateTime.Now; 			 XmlReaderSettings rs = new XmlReaderSettings();			 rs.IgnoreWhitespace = true;			 rs.IgnoreComments = true;			 rs.IgnoreProcessingInstructions = true; 			 XmlReader r = XmlReader.Create(args[0], rs);			 while (r.Read())			 {				 if (r.NodeType == XmlNodeType.Element)				 {					 String name = r.Name;				 }			 } 			 DateTime end_time = DateTime.Now; 			 Console.WriteLine(String.Format("Elapsed Time: {0}", end_time - start_time));		 }	 } }

Conclusion: it's not the .NET implementation.

[John Lokanis]

QUOTE (Ton @ Aug 19 2008, 10:17 AM)

In the first case there shouldn't be a difference between C# and LabVIEW

Unfortunatly, this is not true. LabVIEW is wildly more inefficient when calling .NET functions than C#. This has to do with the method NI choose to perform the calls to the .NET interfaces. I don't know all the technical details (Brian Tyler, formerly of NI, once explained it to me in detail but it was over my head) but I do know to never make a large number of calls to .NET functions in a loop in LabVIEW. Instead, build a simple c# assembly to do the loop processing and then pass the data back to LabVIEW inside and ArrayList. Check out Brian's old BLOG for the details. If you can't find it, or don't understand how to do this let me know and I will post an example.

-John

[mje]

QUOTE (jlokanis @ Aug 20 2008, 02:11 AM)

LabVIEW is wildly more inefficient when calling .NET functions than C#...I do know to never make a large number of calls to .NET functions in a loop in LabVIEW...

I'm beginning to realize this. Simply moving the XmlNodeType.Element read outside the loop reduces execution time by four seconds (!). That's a read of a simple enum constant.

My limited experience with .NET in LV (this single application) has left a sour taste in my mouth. Yes, LV can use .NET, but I'd hardly call it acceptable performance for anything but the most basic functionality. NI's implementation is rather disappointing, regardless of reasons. I'm going to hunt down that blog though.

-michael

[crelf]

QUOTE (jlokanis @ Aug 20 2008, 02:11 AM)

LabVIEW is wildly more inefficient when calling .NET functions than C#.

That might be true (don't know without any metrics), but that doesn't make it unuseable. I'm working on a project at the moment that does a *lot* of .net calls and it's easily acceptable for what I need. I know, I know: apples and oranges

QUOTE (MJE @ Aug 20 2008, 08:57 AM)

Yes,

LV

can use .NET, but I'd hardly call it acceptable performance for anything but the most basic functionality.

Have you talked to NI directly about your issue? There might be something they can help with, or point you in a direction that'll ease your pain.

[John Lokanis]

QUOTE (crelf @ Aug 20 2008, 07:51 AM)

That might be true (don't know without any metrics), but that doesn't necessarily make it unusable.

I agree with that. I also use .NET a lot in my code. It is great for accessing lots of powerful OS functionality. I also use it with XML files and databases. Like I said in my first post, the only cavet is to not do any large enumerations within a loop. Unfortunatly, since .NET was designed to work with Virtual Machine languages (VBScript, JavaScript, etc), most of it's data access is in the form of enumerations (get the count, then loop on the count and get each item) instead of passing blocking of memory directly.

The ArrayList 'trick' on Brian's BLOG shows how to get around this. I use this when retriving record sets from a database and it speeds things up by at least 20x.

The problem is the overhead LabVIEW experiences when calling out to .NET. I will see if Brian can provide a description of this and what, if anything, can be done to improve the situation.

-John

[lycangeek]

Hey guys, tis I, Brian - John contacted me and ask that I look at this thread.

There is cost involved in calling .NET that is just a fundamental issue w/ a non-.NET application calling .NET. We also have some additional issues based on the way LV is designed versus a text language. However, let me explain what happens when you call out to .NET...Note, this assumes you've called .NET before: I'm not going to go over the startup cost for the very first method to .NET.

1. The Invoke node takes each input parameter and converts it to .NET data.

a. If the type is a primative, such as an int, this is a very trival operation.

b. If the type is larger, such as a string, then we must make a copy of the string. The reason is that we can't pass a C character point to memory to a .NET routine expecting a .NET string. This can be very expensive depending on the types.

c. If the type is a .NET refnum, then the "copy" is just an integer of the refnum value.

2. We jump across to the .NET app domain. The app domain is a .NET concept - kind of like a process within a process. In earlier versions of LV, this was very, very expensive. One of the last things I worked on before I left NI was to make this much, much faster. I believe that would have shipped in LV 8.5, but can't say for certain...but I would be suprised if it didn't.

3. Any .NET refnum needs to be "unpacked". Basically whenever we return to LV w/ an object reference, we ask .NET to give us a "cookie" that represents the .NET object (a GCHandle for those that are curious). Now that we are coming back in, we have to ask for all such cookies to be converted back into the .NET objects themselves. This should be fairly fast.

4. We now need to invoke the method. We use reflection to do this - a fairly standard mechanism for a non-.NET language to call into .NET, but unfortunately it isn't very fast. If you have any experience w/ COM, it's equivalent to using IDispatch, like VB of old used to do.

5. When the method returns, we reverse direction and do steps 1-3 in reverse order for the return data.

Overall, the execution process is fairly good given all the translations required, but it's not free. This is especially painful for lots and lots of little calls where the overhead of calling the method really dwarfs the method you are calling. In the past, I've always recommended to treat calling .NET the same way you treat machine-machine programming - avoid chatty interfaces. Obviously this isn't always possible w/o writing extra .NET code, but there you go.

Hopefully this helps explain, although obviously you've already done the work to get the concrete data.

Brian

[mje]

Thanks, Brian, that's some great information.

Ultimately, I think what will doom this application from being coded in LV is my requirement to keep the memory footprint inline. Balooning out to several hundred megabytes to load a huge XML file isn't an option. Hence the XmlReader, which allows very frugal use of memory, but at the expense of many iterative calls. The overhead in those calls adds up I imagine, to the point where using LV just isn't practical.

This shouldn't be too much of a problem, I can easily work in C# to get the main serialization done. It's actually my preference, I find LV code with excessive amounts of property/invoke nodes rather "clunky" when looking at the block diagram.

Quick question though since many of you seem to have the .NET experience in LV that I lack. As long I keep my data in the form of arrays of primitives, data moving across the application boundaries shouldn't be too much of an issue, just a simple* copy, correct?

*And by "simple" I mean a "painful" copy of a few hundred megabytes...

Thanks everyone for the input!

-Michael

[Rolf Kalbermatter]

QUOTE (MJE @ Aug 20 2008, 05:28 PM)

Thanks, Brian, that's some great information.

Ultimately, I think what will doom this application from being coded in LV is my requirement to keep the memory footprint inline. Balooning out to several hundred megabytes to load a huge XML file isn't an option. Hence the XmlReader, which allows very frugal use of memory, but at the expense of many iterative calls. The overhead in those calls adds up I imagine, to the point where using LV just isn't practical.

This shouldn't be too much of a problem, I can easily work in C# to get the main serialization done. It's actually my preference, I find LV code with excessive amounts of property/invoke nodes rather "clunky" when looking at the block diagram.

Quick question though since many of you seem to have the .NET experience in LV that I lack. As long I keep my data in the form of arrays of primitives, data moving across the application boundaries shouldn't be too much of an issue, just a simple* copy, correct?

*And by "simple" I mean a "painful" copy of a few hundred megabytes...

Thanks everyone for the input!

-Michael

Well you have two options:

1) Write a small .net assembly that does all the nitty gritty enumeration work and returns a collection of data to LabVIEW.

2) Or use the LabVIEW XML library from the internet toolkit, a fairly full featured interface to the Xerces Open Source Library from the Apache project

2.1) Maybe you could use the EasyXML Toolkit from JKI software

Ok there is a third option that I would myself not consider an option:

3) Do everything in something else than LabVIEW.

Rolf Kalbermatter

[Yair]

Great to hear from you again, Brian. Good stuff. I take it your MS project is still secret?

Michael, I believe 8.6 should have a native DOM parser (possibly even the one Rolf mentioned), so you can also try looking in that direction. Of course, if parsing XML is easier in C#, then that's probably a better solution.

[mje]

Yeah, the LV XML parser actually uses the Xerces DOM implementation. I'm going for more of a SAX style, and I'm evaluating whether or not you can even use Xerces SAX in LabVIEW...I've only ever played with it in C++, but that was long ago and I abandoned it before completion in favor of something else. From what I remember, the Xerces SAX event model requires you to derive from an exported class to receive event notification (opposed to registering callback functions or delegates), though I could be wrong.

As for the JKI toolkit, we own it, but I don't think it has quite what we need.

Ultimately, I'm siding on still using the .NET XmlReader and doing the data serialization via a C# assembly, which will expose a few methods to allow me to get the data back in LabVIEW. The XmlReader is really quite elegant, and much easier to use than an old-school SAX implementation.

-michael

[lycangeek]

QUOTE

Quick question though since many of you seem to have the .NET experience in LabVIEW that I lack. As long I keep my data in the form of arrays of primitives, data moving across the application boundaries shouldn't be too much of an issue, just a simple* copy, correct?

*And by "simple" I mean a "painful" copy of a few hundred megabytes...

Everything is a copy - but primatives at least are on the stack. .NET is very fast at memory allocation, but nothing beats the stack The main problem here is that you are dealing with a lot of string data, which really gets expensive copying back and forth.

QUOTE

I take it your MS project is still secret?

Kinda yes, kinda no - I'm the tech lead for the project system in Visual Studio - at least for the upcoming (after VS2008) release, which isn't really announced yet. I'm going to roll out a new blog on MSDN when we can start talking publically about it (maybe mine, maybe a team blog), but it'll be of more interest to folks that want to create or extend project types in Visual Studio. I've thought about firing up detritis again, but I have to say I've got so little time it's not been top of the priority list. Partly that is due to work, and party it's due to the pacific NW - there is so much to do up here

[Yair]

QUOTE (lycangeek @ Aug 21 2008, 06:50 PM)

but it'll be of more interest to folks that want to create or extend project types in Visual Studio.

That kind of rules me out, then. I only code in LabVIEW, so I can't say I have any need for VS features. One thing I can say is that the graphical project design feature which a friend showed me in the VS2008 beta a few months ago (and which I assume made it to the final version) looks nice and is at least reminscient of LabVIEW.

Nice to hear you have interesting stuff to do.

[Neville D]

QUOTE (lycangeek @ Aug 21 2008, 08:50 AM)

Partly that is due to work, and party it's due to the pacific NW - there is so much to do up here

True.. beats Austin in summer anyday!

Neville.