Jump to content

XML File reader with Validation


Recommended Posts

Here is an example of how to read an XML file, extract attributes of elements and also validate it against a schema.

I use this to read user-editable configuration files for my project. By using XML with a schema, I can control the format of the file and verify that it adheres to this format when reading it. This allows me to abort application startup with a meaningful error message if someone makes an invalid edit to the configuration file.

This is also a good primer on how to read XML files using .NET calls. I hope you find this useful.

I also suggest you read some of the tutorials on the net about XML and XSD (schema) to understand how to create and edit your own files. In addition, I recommend XMLSpy by Altova for working with XML and XSD files.

-John

Download File:post-2411-1219705387.zip

Link to comment

Here is an example of how to read an XML file, extract attributes of elements and also validate it against a schema.

I use this to read user-editable configuration files for my project. By using XML with a schema, I can control the format of the file and verify that it adheres to this format when reading it. This allows me to abort application startup with a meaningful error message if someone makes an invalid edit to the configuration file.

This is also a good primer on how to read XML files using .NET calls. I hope you find this useful.

I also suggest you read some of the tutorials on the net about XML and XSD (schema) to understand how to create and edit your own files. In addition, I recommend XMLSpy by Altova for working with XML and XSD files.

-John

Download File:post-2411-1219705387.zip

Link to comment

Useful demo. My only suggestion would be to stay away from the XML file specifying the schema file. I'd specify it to the validator explicitly, otherwise validating against a possibly user modified/specified schema isn't of much use. Ideally in an application implementation, the schema would be an internal resource, or a checksumed external file so it's integrity can be verified, but that last step is a bit beyond the scope of a demo. A demo simply taking the path to the schema would suffice.

I *really* like what you did with the error queue during validation. That's bloody genius, I'm gonna have to remember that trick.

Link to comment

QUOTE (MJE @ Aug 26 2008, 05:37 AM)

My only suggestion would be to stay away from the XML file specifying the schema file.

Yes, that is an option, but then they would have to select the schema file or I would have to hard code it's name into the code. Also, if I choose to put the schema in another directory, I would have to hardcode that full path into the code. If I left the schema in the same directory and used the reletive path of the XML doc, then there is nothing that would stop them from seeing it and editing it as well.

Finally, by having the schema specified in the XML and having it local to the XML, other XML editors can use this to enforce the schema when editing the file. This helps avoid errors when making edits. My goal is to help the user not make a mistake when editing that would cause the file to be unreadable. If they wish to be malicious, then I really don't care.

But if someone sees value in having the schema specified externally, that can be done. As one of my old professors used to say: 'That exercise is left for the student'.

BTW: I'm glad you like the callback trick! I'l have to post a few more examples of interesting 'tricks' I have come across over the years...

-John

Link to comment

Hi there,

Thanks for the demo/tutorial I am an RF engineer who is now in a software role as a LabVIEW programmer and I have found learning to use XML really difficult as all the examples are are in code like javascript or c# etc. All my colleagues have never seen LabVIEW and I dont really know with any ability any other programming languages so there is a massive barrier to learning to use XML and .NET calls.

First I'll give a brief description of my task and then tell you my problems. I have to program a system for making some RF tests on a satellite payload. My system will be used for some of the smaller tests and is based on Off the shelf Agilent RF equipment. This will be part of a larger system that commands the payload configuration, stores data, graphically shows the payload configuration and monitors the payoad temp/volts/amps etc. These parts are done using teststand, SQL, in-house created software and webservices. This is the way it has to be done since the flexibilty to change supplier of the Main RF Measurement System is needed for future flexibility. What my specific problem is that I am subscribing to the webservice and from this service I am given a string XML which I have just about managed to read and query using similar methods from your demo.

Your demo uses single elements with lots of attributes for data this make it easy to iterate through and grab out an array and is ideal for your suggested gonfiguration use. However, the format of the parameters passed in the webservice conforms more strictly to the 'best form' w3c standard which moves away from attribute use and adds them as sub elements. Are you familiar with parsing this type of XML. As I am quite fresh to the XML scene do you consider it to be very/too difficult to iterate the elements instead. My biggest problem with XML something you may be able to help/make clearer for me is it seems like you can only really rip out data/info to a table or something structured if you know what it is you are going to receive. I am wondering if this is where the schema should be used or if there are better more generic element searches that can be used. I have had some success using the wildcard functions with in the xpath.selectnodes function to get the info but due to the varying attributes and elements sizes it is extremely tricky to align the data correctly in tables.

Here is an excerpt from the xml from altova:-

<Parameter xmlns="" Name="MaskSet1" Type="Mask">

<ARRAY>

<ARRAY xmlns="" Name="MaskXcoord" Type="FloatArray">

<VALUE Col="1">-2.000000000000000e+001</VALUE>

<VALUE Col="2">-1.500000000000000e+001</VALUE>

<VALUE Col="3">1.500000000000000e+001</VALUE>

<VALUE Col="4">2.000000000000000e+001</VALUE>

</ARRAY>

<ARRAY xmlns="" Name="MaskYcoord" Type="FloatArray">

<VALUE Col="1">-3.000000000000000e+001</VALUE>

<VALUE Col="4">-3.000000000000000e+001</VALUE>

<VALUE Col="2">-2.500000000000000e+001</VALUE>

<VALUE Col="3">-2.500000000000000e+001</VALUE>

</ARRAY>

<ARRAY xmlns="" Name="MaskId" Type="IntegerArray">

<VALUE Col="1">1</VALUE>

<VALUE Col="2">1</VALUE>

<VALUE Col="3">2</VALUE>

<VALUE Col="4">2</VALUE>

</ARRAY>

</ARRAY>

</Parameter>

Altova recognises the nesting correctly but the system xml amd xpath functions don't seem to recognise child and parent elements correctly.

Any help would be greatly appreciated and thanks in advance.

Link to comment

I'm not completely sure I understand all your questions, but if you are asking if this format can be read, the answer is yes. However, I have found that in LabVIEW, you cannot make a generic XML reader easily. In most cases, you need to know the format of the XML document when you write the LV code to parse it.

That does not mean that you cannot have variable length sections, like you excerpt shows. You can use the features in the MSXML.NET assembly to determine child count and iterate through the elements, building an array in LV. This is something I do all the time. I have a much more complex XML file that describes an entire test hierarchy that has N test plans with N test suites, each with N tests, each with N parameters and N measurements. I am able to read this into a complex array of clusters of arrays of clusters, etc...

So, it can be done, but you need to think carefully about the structure of the XML you will receive and then design you code to deal with the sections that are variable in length.

I also noted that your example stored the data in elements but still used attributes to specify information about each element. So, you will need to deal with extracting that part of the data and use it to organize the elements in their proper order, unless you can assume they will always be formatted in ascending numerical order (which is likely but nothing is ever guaranteed).

Anyways, I think the example code gives you a good start toward reading this but I suspect you will need to modify it to fit your format. My goal was not to make a generic XML reader but rather demo the basics of accessing XML data and protecting it with a schema.

You might also check out JKI's EasyXML tools. I think they might be useful to help parse your XML.

Link to comment

QUOTE (NeilA @ Aug 28 2008, 05:26 AM)

Hi there,

Thanks for the demo/tutorial I am an RF engineer who is now in a software role as a LabVIEW programmer and I have found learning to use XML really difficult as all the examples are are in code like javascript or c# etc. All my colleagues have never seen LabVIEW and I dont really know with any ability any other programming languages so there is a massive barrier to learning to use XML and .NET calls.

First I'll give a brief description of my task and then tell you my problems. I have to program a system for making some RF tests on a satellite payload. My system will be used for some of the smaller tests and is based on Off the shelf Agilent RF equipment. This will be part of a larger system that commands the payload configuration, stores data, graphically shows the payload configuration and monitors the payoad temp/volts/amps etc. These parts are done using teststand, SQL, in-house created software and webservices. This is the way it has to be done since the flexibilty to change supplier of the Main RF Measurement System is needed for future flexibility. What my specific problem is that I am subscribing to the webservice and from this service I am given a string XML which I have just about managed to read and query using similar methods from your demo.

Your demo uses single elements with lots of attributes for data this make it easy to iterate through and grab out an array and is ideal for your suggested gonfiguration use. However, the format of the parameters passed in the webservice conforms more strictly to the 'best form' w3c standard which moves away from attribute use and adds them as sub elements. Are you familiar with parsing this type of XML. As I am quite fresh to the XML scene do you consider it to be very/too difficult to iterate the elements instead. My biggest problem with XML something you may be able to help/make clearer for me is it seems like you can only really rip out data/info to a table or something structured if you know what it is you are going to receive. I am wondering if this is where the schema should be used or if there are better more generic element searches that can be used. I have had some success using the wildcard functions with in the xpath.selectnodes function to get the info but due to the varying attributes and elements sizes it is extremely tricky to align the data correctly in tables.

Here is an excerpt from the xml from altova:-

<Parameter xmlns="" Name="MaskSet1" Type="Mask">

<ARRAY>

<ARRAY xmlns="" Name="MaskXcoord" Type="FloatArray">

<VALUE Col="1">-2.000000000000000e+001</VALUE>

<VALUE Col="2">-1.500000000000000e+001</VALUE>

<VALUE Col="3">1.500000000000000e+001</VALUE>

<VALUE Col="4">2.000000000000000e+001</VALUE>

</ARRAY>

<ARRAY xmlns="" Name="MaskYcoord" Type="FloatArray">

<VALUE Col="1">-3.000000000000000e+001</VALUE>

<VALUE Col="4">-3.000000000000000e+001</VALUE>

<VALUE Col="2">-2.500000000000000e+001</VALUE>

<VALUE Col="3">-2.500000000000000e+001</VALUE>

</ARRAY>

<ARRAY xmlns="" Name="MaskId" Type="IntegerArray">

<VALUE Col="1">1</VALUE>

<VALUE Col="2">1</VALUE>

<VALUE Col="3">2</VALUE>

<VALUE Col="4">2</VALUE>

</ARRAY>

</ARRAY>

</Parameter>

Altova recognises the nesting correctly but the system xml amd xpath functions don't seem to recognise child and parent elements correctly.

Any help would be greatly appreciated and thanks in advance.

NeilA,

As John mentioned, you could try the EasyXML toolkit. I just spent a few minutes to create a LabVIEW Data structure to accommodate your XML data (to read it in LabVIEW) and it work fine (see screenshot below).

post-121-1219959741.png?width=400

PJM

Link to comment

QUOTE (jlokanis @ Aug 28 2008, 01:04 PM)

JKI has a really easy to use toolkit called the JKI EasyXML Toolkit. Its designed such that all you have to do is to define a cluster that matches your XML structure and you're basically good to go. You can find out more on www.jkisoft.com/easyxml.

Here are screenshots of how the data would be parsed using our tool, I've also attached the VI that was used in the screenshots.

[Front Panel]

post-5746-1219961297.png?width=400

[block Diagram]

post-5746-1219961306.png?width=400

In order for the attached VI to run, you'll need to install the evaluation (or professional) copy of EasyXML. You can get it from the jkisoft website here: http://www.jkisoft.com/easyxml/download/

Go to jkisoft.com if you have additional questions on our solution.

QUOTE (Omar Mussa @ Aug 28 2008, 03:12 PM)

.

Here are screenshots of how the data would be parsed using our tool, I've also attached the VI that was used in the screenshots.

[Front Panel]

post-5746-1219961297.png?width=400

[block Diagram]

post-5746-1219961306.png?width=400

In order for the attached VI to run, you'll need to install the evaluation (or professional) copy of EasyXML. You can get it from the jkisoft website here:

Go to
if you have additional questions on our solution.

Apparently Philippe posted at the same time I did, but luckily our solutions are the same ;)

Link to comment

QUOTE (Omar Mussa @ Aug 28 2008, 03:32 PM)

I'm not sure what 'this' means... I assume you mean the JKI EasyXML Toolkit because obviously the NI code does not depend on OpenG (and likely never will). The JKI EasyXML toolkit depends on the following OpenG packages:

oglib_string

oglib_lvdata

oglib_error

oglib_array

No, actually I meant the Code Repository example from the top of this thread: viz,

DEMO_XML_File_Read.zip ( 136.39K )

Does that "it" use an OpenG or is it only native LV?

Link to comment

QUOTE (Val Brown @ Aug 28 2008, 03:01 PM)

Does this make use of any OpenG code or is it all native LV?

This example is strictly standard LV code. No add-ons of any kind were used.

The example is written in LV8.5. I don't have older versions so I cannot down-rev it if you do not have 8.5. I would avoid .NET in 8.2 anyways since there were significant bugs.

That said, I use OpenG in many places in my apps and strongly recommend other use it too.

Link to comment

QUOTE (jlokanis @ Aug 28 2008, 06:25 PM)

This example is strictly standard LV code. No add-ons of any kind were used.

The example is written in LV8.5. I don't have older versions so I cannot down-rev it if you do not have 8.5. I would avoid .NET in 8.2 anyways since there were significant bugs.

That said, I use OpenG in many places in my apps and strongly recommend other use it too.

OK, thanks for the clarification. I'll look at it now. I don't use OpenG -- NOT because I don't think it's good. As I've said elsewhere, it's a wonderful set of tools which I have also recommended to others. It's just that, in my environment, I need to stay with as much native LV as possible.

val

Link to comment

QUOTE (Val Brown @ Aug 28 2008, 06:50 PM)

OK, thanks for the clarification. I'll look at it now. I don't use OpenG -- NOT because I don't think it's good. As I've said elsewhere, it's a wonderful set of tools which I have also recommended to others. It's just that, in my environment, I need to stay with as much native LV as possible.

Hi Val,

When you say that you "need to stay with as much native LV as possible", does that mean that you don't use VIs and only primitive functions? Nearly all the OpenG VIs are written in "native LV" (a.k.a., "Pure G"), with the exception of the zip library that makes a call into a DLL.

I'm not trying to make light of your requirements, but to understand your environment and use cases.

Thanks,

-Jim

Link to comment

QUOTE (Jim Kring @ Aug 28 2008, 07:34 PM)

Hi Val,

When you say that you "need to stay with as much native LV as possible", does that mean that you don't use VIs and only primitive functions? Nearly all the OpenG VIs are written in "native LV" (a.k.a., "Pure G"), with the exception of the zip library that makes a call into a DLL.

I'm not trying to make light of your requirements, but to understand your environment and use cases.

Thanks,

-Jim

Hi Jim,

Perhaps I should clarify a bit more what I mean by "native LV". Yes, I use VIs but I use VIs -- as far as possible -- that either I have created or that come for NI directly in their official releases. This way I know that, if there is some problem at some point with a VI no longer working as expected, NI will address it. So I use the Database Connectivity Toolkit instead of just using ADO/DAO or LabSQL, etc because I know, if the functionality isn't there with an update of LV or for whatever reason, it will be addressed. In the past I've made use of several "suites" that were made available by others -- some open source some not -- and then there was a problem when LV updates and the suite from a third party wasn't updated, etc. I'm still working with zlib and the blowfish implementation of encrypt/decrypt because, at that time, it was the most accessible tool available; however, now it's a bit of an albatross. I'd like to get rid of it but probably won't do much about it until after my current pending distro release.

val

I should also add that I've been a bit put-off in the past by all of the confusion around copyright and such as it applies to different third party and/or open source products. Since I distribute a built EXE to end users and I want to be consistent, clear and legal in my use of various tools, it's been a bit challenging to figure out how to include ALL of the various different, and sometimes conflicting, copyright notices needed if using 3rd party additions.

Link to comment

QUOTE (Val Brown @ Aug 29 2008, 10:55 AM)

This way I know that, if there is some problem at some point with a VI no longer working as expected, NI will address it. So I use the Database Connectivity Toolkit instead of just using ADO/DAO or LabSQL, etc because I know, if the functionality isn't there with an update of LV or for whatever reason, it will be addressed.

That's a good one. NI's release schdules are usually in at least six months cycles and even then there is no guarantee for your problem to be fixed in the next release. I'm not saying that NI doesn't resolve the issues or offers workarounds, just that it's not guaranteed to do it and relying on it is potentially dangerous. At least in open source (and some of NI's code also falls under that category) you have the option of fixing some problems yourself.

A couple of examples that spring to mind:

  1. You mentioned the DB toolkit. The DB toolkit relied on a certain behavior of variants when flattening them to be saved to a binary field. In LV 8.0, NI changed the internal structure of variants and this broke, because that use case wasn't tested. This might have been fixed in the new version of the DB toolkit which came out with 8.6, but I never ran into this personally, so I don't know.
  2. Before LV 7.1, you could control the properties of a caption without any problems. Since 7.1 you had to manually make the caption visible once before it would be created. If you didn't do that, the code which worked for years would now be broken. Did NI fix that? No. It was decided and you had to modify your own code.

Link to comment

QUOTE (Yair @ Aug 29 2008, 04:44 AM)

A couple of examples that spring to mind:

Did anyone else get hit a few years ago when the installation of VISA overwrote the serial setup VI that came with the base package? It killed a TON of my code...

Link to comment

QUOTE (Yair @ Aug 29 2008, 04:44 AM)

That's a good one. NI's release schdules are usually in at least six months cycles and even then there is no guarantee for your problem to be fixed in the next release. I'm not saying that NI doesn't resolve the issues or offers workarounds, just that it's not guaranteed to do it and relying on it is potentially dangerous. At least in open source (and some of NI's code also falls under that category) you have the option of fixing some problems yourself.

A couple of examples that spring to mind:

  1. You mentioned the DB toolkit. The DB toolkit relied on a certain behavior of variants when flattening them to be saved to a binary field. In LV 8.0, NI changed the internal structure of variants and this broke, because that use case wasn't tested. This might have been fixed in the new version of the DB toolkit which came out with 8.6, but I never ran into this personally, so I don't know.
  2. Before LV 7.1, you could control the properties of a caption without any problems. Since 7.1 you had to manually make the caption visible once before it would be created. If you didn't do that, the code which worked for years would now be broken. Did NI fix that? No. It was decided and you had to modify your own code.

M experience has been precisely the opposite of you, and certainly so in re: to the DB Toolkit. Working with an AE (I'm on platinum support) we resolved the issue. I addressed it again with the advent of 8.5 and issue with Vista. With some third party tools I was told by the developer, essentially "what you see is what you get". I don't use captions so I have no experience with them.

Yes, the release schedules NOW are pretty predictable -- I've been using LV since v4x so that has improved over the years -- but the support has been and continues to be good and, ultimately, definitive. My experience with open source groups -- of various products -- has been far, far less supportive. I know that's probably blasphemous here but that's how it is.

QUOTE (jed @ Aug 29 2008, 09:40 AM)

Did anyone else get hit a few years ago when the installation of VISA overwrote the serial setup VI that came with the base package? It killed a TON of my code...

Yes, I did AND I got the workaround from an AE, which has been working in my code since then. It STILL works with 8.6 even though I had thought it was "killed" with an earlier 8x release. I've now, finally, moved on to replace that code but not because it doesn't work -- it STILL works, just as it was written for LV 5x. I've replaced it because I know have some new drivers for some older devices (came from the manufacturer of those devices) so it was time to move on.

BTW, I believe that the fix to maintain the legacy serial i/o functions is described in a thread here on LAVA, if that info would be useful at this time.

Link to comment

QUOTE (Val Brown @ Aug 29 2008, 08:12 PM)

M experience has been precisely the opposite of you, and certainly so in re: to the DB Toolkit. Working with an AE (I'm on platinum support) we resolved the issue.

I didn't say anything about my experience, so you can't really have the opposite opinion. I've used 7.0 almost exclusively for the past 5 years and was refering to issues I saw others have or only ran across occasionally. Did the issue you resolve have anything to do with saving variants to a DB when upgrading from pre-8.0 to 8.x? If not, then it's not really relevant to the example I cited.

In any case, you should note that I'm not saying anything about the level of support you get from NI, just that in some cases they don't have a solution for you either and if you look through NI's known issues documents you will find some issues with no known workarounds.

Link to comment

QUOTE (Yair @ Aug 30 2008, 11:39 AM)

I didn't say anything about my experience, so you can't really have the opposite opinion. I've used 7.0 almost exclusively for the past 5 years and was refering to issues I saw others have or only ran across occasionally. Did the issue you resolve have anything to do with saving variants to a DB when upgrading from pre-8.0 to 8.x? If not, then it's not really relevant to the example I cited.

In any case, you should note that I'm not saying anything about the level of support you get from NI, just that in some cases they don't have a solution for you either and if you look through NI's known issues documents you will find some issues with no known workarounds.

I guess I'm still a bit confused about what you're saying. What I understood from you was that you prefer open source because you "can fix it yourself" as opposed to relying on NI and that you were reporting on experiences that you knew about where others -- or you?? -- had encountered problems that NI couldn't resolve and, therefore, you prefer open source. That doesn't make much sense to me as, unless it's a primitive (which is uneditable for everyone), you can always "fix it yourself" in LV or ask others in the community for what they've done to fix, work around, etc whatever you encounter. In any event, in the end, we're all dependent on what NI provides -- certainly in the case of primitives -- and the rest we can always "fix" or adapt ourselves if it's in a VI. If it's a primitive (not something that can be edited directly in LV) then we're all equally "stuck" with whatever NI does regardless of our commitments to open source. If it's anything else, we can modify it, again whether we're committed to open source or not. After all, there was a time when there wasn't a three button dialog and even now that there is one, there's a thread here using that for a coding challenge to improve it. Before the NI version came out I found it a real nightmare to use the third party workarounds that I found. Each was different and didn't play well with the others. At least with the NI version we now have a shared version that is "official" and standard.

Having been through the early Unix and C wars (never mind Pascal, etc), I really don't want to have to continue to play the "keep up with all the versions" game. I do understand that for others that's a perfectly wonderful way to go and the VIPM package in particular seems like a great platform for just that purpose. For me, it's overkill and itself another interface that I would have to learn and maintain. So my perspective may very well be unique here but I do think it's valid, at least for me and perhaps others in my situation.

Yes, I had the exact same problem with variants and the DCT, while moving some of my code through the transition to 8x.

Link to comment

I'm not saying that I prefer open source. I use both kinds (including the DBCT, since we've mentioned it).

What I'm saying is that even if NI are guaranteed to address your issue (to use your original term), they're not guaranteed to offer what you would consider a completely satisfactory solution.

I'm not saying that's wrong, either. They can't solve everything.

My point is that the assumption that paying for service necessarily means you'll always get what you want could be dangerous. How was the issue with saving variants resolved in your case?

Link to comment

QUOTE (Val Brown @ Aug 29 2008, 12:55 AM)

I use the Database Connectivity Toolkit instead of just using ADO/DAO or LabSQL

FWIW, the Database Connectivity Toolkit is just a wrapper around ADO.NET. And not a very good one at that. It is extremely slow when working with large record sets.

If you truely want to be native, then write your own DB VIs calling ADO.NET yourself. If you use some of the tricks from Brian Tyler's old blog, you can get a 10x+ speed improvement.

-John

Link to comment

QUOTE (jlokanis @ Aug 31 2008, 03:46 PM)

FWIW, the Database Connectivity Toolkit is just a wrapper around ADO.NET. And not a very good one at that. It is extremely slow when working with large record sets.

If you truely want to be native, then write your own DB VIs calling ADO.NET yourself. If you use some of the tricks from Brian Tyler's old blog, you can get a 10x+ speed improvement.

-John

Yes, it is but it's a wrapper that's being maintained by a company, not just "some folks" who might, or might not, get around to it when something happens, and goes wrong. I'm much more interested in reliability and maintainability than speed -- for THAT part of what I do -- as I've got those functions all arranged to happen "off line" in non-time critical ways.

I don't remember off the top how it was resolved. I really have to look back over my archived code to find out how because what I remember is that I submitted the SR, interacted with the AE, got workarounds/solutions and applied them. It was a while ago at this point and I just no longer keep all of those details in my head any longer. Whatever the workaround/solution was, I no longer had the problems with variants with the DCT.

Link to comment

Thanks for all your replies. Sorry I have not thanked you sooner I have been away on a stag weekend for a friend.

I think I may have to struggle on with a semi-generic self written XML parser. It feels like I am re-inventing the wheel but I think that I may run into copyright issues using the easyXML tool as it is not open source/ purchasing a copy for all the machines that will use it in the end is not really the best solution.

Thanks again.

Link to comment

Join the conversation

You can post now and register later. If you have an account, sign in now to post with your account.

Guest
Reply to this topic...

×   Pasted as rich text.   Paste as plain text instead

  Only 75 emoji are allowed.

×   Your link has been automatically embedded.   Display as a link instead

×   Your previous content has been restored.   Clear editor

×   You cannot paste images directly. Upload or insert images from URL.

×
×
  • Create New...

Important Information

By using this site, you agree to our Terms of Use.