Rolf Kalbermatter

5 hours ago, alvise said:

In this application, callback is used but PlayM4.dll is not used.

The callback itself should not leak memory but it does two handle allocations and deallocations each time around.

The first handle is allocated and deallocated in the callback function itself. The second handle is allocated in LVPostUserEvent() when it copies the incoming data and released in the callback event frame.

I never understood why there is not at least an option to tell LVPostUserEvent() to actually take ownership of the data. But it can't be changed by now.

The optimization would be to store the handle between callback invocations somewhere and only resize it when the new data needs a bigger buffer. However do not attempt to do that yourself! This is fraught with trouble! You can't just store it in a static global variable since the callback can be potentially called from more than one thread in parallel once you would operate two or more cameras at the same time. You would need to store that handle in TLS (Thread Local Space).

18 hours ago, alvise said:

It's misunderstood, but not because you don't explain it well, but because I don't know some terms well and it's like someone learning to swim in a pool trying to swim in the ocean.

One point that confused me was this.
-Run in UI Thread: how it works and how it should be set.
-Run in Any Thread: how it works and how it should be set.
What is enterant and Reanterant and how is it related to UI thread or Any Thread.

Not understanding these in all its depth causes confusion. I am trying to understand it again by analyzing the Application we have created so far and taking into account the work it has done.

I didn't see the edit you added. Because when I read your first post, that plugin was not there. I guess I was trying to understand what dadreamer was saying when you added it later.

The application you are editing is really neat and tidy.I tested the app you edited and it responded with an error like below.

erorr #define PLAYM4_INVALID_PORT 32 

As said many times before. I can't test the software without hardware so this is all a dry swim exercise for me.Hikvision-labview SDK -Test-v1.1.1.zip

But try with this version, it should fix this particular problem. I excluded the lib folder as it is huge and has no changes in respect to the earlier version. So you need to extract this into your previous sources folder (but make sure to delete all LabVIEW VIs in there to avoid potential conflicts).

5 hours ago, alvise said:

Thanks for your answer, actually, what I wrote in the previous message is not that I don't know completely about this subject, I meant some things that I don't understand deeply.
For example; Why are we talking about reentrant and non-reentrant when using the Call library function node?

Warning: It's possible that everything I've said here is wrong!

Possible answer: Because we want to have parallel loops and the same subVI (for the previous Play4M_GetJpeg) may need to be run several times. Each time it is called in the subVI we are calling the same function (CFLN) again, and in this case the non-reentrant function if called at the same time can cause queueing. It means it wouldn't make sense to choose non-reentrant.
So why is the VI we use chosen as non-reentrant?
etc. many questions.

But at the current stage, some of the contradictions I mentioned above have disappeared, and there is no need for this at the last point that Rolf shared and in general we reached. because since the same set of gates are used by all callers of the VI, the values from the last call of that VI are available to the next caller.

By the way, sorry for my bad english.From everyone participating in the discussion on this forum.  

 

The Call Library Node itself is NEVER switching threads on the fly. When LabVIEW executes VIs it can arbitrarily switch between the multiple threads it has allocated for a particular execution system. But when it executes a Call Library Node it selects a thread (usually the one it was already executing the VI in except that thread is already tied up for something else) and passes control to the shared library function. LabVIEW has no way to take control back from this until the function returns control to the caller. The setting really is called "Execute in any thread", not "reentrant", but a function needs to be reentrant  or multi-threading safe in order for this to be safe.

If a function is not reentrant safe (for instance because it uses global resources such as a global variable or accessing some hardware, or is simply badly programmed) then you should not use the "Execute in any thread" setting since there could be situation where the shared library gets called multiple times in parallel when doing so and then the multithreading unsafe functions could stomp on each others global resources and cause race conditions or worse.

Then you can choose to run the nodes in the "UI thread" this is the only execution system in LabVIEW that is guaranteed to only use a single thread. So multiple Call  Library Nodes set to run in the "UI thread" can NEVER execute in parallel and therefore can not stomp on each others global resources or similar at the same time. But they also have to compete with many other things in LabVIEW that are required to execute in a single thread, including all screen updates. This can lead to very sluggish operation as each Call Library Node set to run in the UI thread has to wait for that thread to be available before it can call the shared library function.

If a shared library (or particular functions in a shared library) are multithreading safe is only something that can be answered by the original developer. However many developers have just like you no good idea what the issues could be and if asked about this they will simply give you an empty stare, wondering what you are talking about, respectively either saying that is isn't multithreading safe if they are the cautious type or boldly claiming that it of course is, if they prefer to look cool and are not ashamed to lie flat out.

In the case of the PlayCtrl.dll it states explicitly that since version 6.1.1.17 the DLL is multithreading safe. It mentions a caveat that it may block if you try to call certain shared library APIs from within a callback. This is because they seem to have made the library multithreading safe by using mutexes. An API that you call from your application acquires the mutex for that resource, preventing other threads to use that resource for the duration of the call. Before returning from that function the mutex is released. As long as your application only calls APIs it can't really deadlock. The worst that can happen is that some calls will have to wait for the mutex to be available again before they can start doing their work.

But if such an API causes callback functions to be called synchronously then it may hand off an already acquired resource to the callback which then calls a library API that tries to acquire its mutex too, but it can't get it until the original API has released the mutex, which it can't do until the callback returns. => Fatal deadlock.

13 hours ago, alvise said:

I set it as you said. I think you said it's safe for us to choose this as the UI thread. But Only for PlayM4.

Sometimes I feel like I must be talking some obscure galactic slang or something!

Setting Call Library Nodes to run in UI thread is indeed safe but I never said this is required here.

Quote

If you use a PlayCtrl.dll version 6.1.1.17 or newer it SHOULD be safe to set the Call Library Nodes to run reentrant!

What about that English sentence is not clear? Have you looked at the last attachment I added to one of my last messages?

8 minutes ago, alvise said:

Then you say that choosing "UI thread" for "Play4M" and "HCNetSDK.dll" makes more sense for now.

 

No, the opposite if you have a DLL version 6.1.1.17 or newer. In that case it SHOULD be safe to set all Call Library Nodes for the PlayCtrl.dll functions to reentrant.

6 minutes ago, alvise said:

I have to use the method here, right?

No again! The byte stream coming from PlayM4_GetJPEG() is already a JPEG formatted byte stream. Dump into a binary file and it is a JPEG image. For Windows make sure to give it the filename extension .jpg or .jpeg as otherwise Windows Explorer will not understand that it is a JPEG image, but that is just a Windows thing. Microsoft in its infinite wisdom decided loooooooooong ago that the file ending is the only reliable way to determine what type a file is. Of course that is a very unreliable assumption but hey they have gotten away with it for about 40 years. So what!

2 hours ago, alvise said:

What I can briefly explain is that if I set it in the UI thread, I can run unsafe DLLs. But if I run it as "Any thread", I will run it in the unsuitable thread. This can cause problems

The documentation for the PlayM4 shared library states explicitly that the DLL should be since version 6.1.1.17 multithreading safe. Of course that statement can be as true or false as anything but it is probably safe to assume for now that it SHOULD work if we don't do massive multithreading. And since we run pretty much everything actually in our test VI event structure it is actually executing very sequentially so the multithreading aspect shouldn't even come into play for now. For a more generic library there would need to be a lot more testing needed however.

And I would actually pull the whole thing a bit apart. I have cleaned up a lot of stuff and put the PlayM4 functions in their own library. My idea was to push the image retrieval into the the timeout case as in included example. Adding that into the data callback may be a little too much taxing. Also Once we know that the callback data seems to be working I would actually remove the data array on the front panel. That does take quite a bit of performance as LabVIEW needs to create a copy of that array every time to display in the control.

Depending on how much the callback event taxes the system it may however never arrive in the timeout case so possibly there is never any update of the BMP data. If that happens you have anyhow already reached the limit of what your system can do and there is simply no way to try to make this still work in this way.

The main idea is that if the program does get into the timeout event case I try to retrieve the current played/decoded frame number and if that is different than the previous one, the stream should be able to provide a BMP image. Retrieve it and once we get there we can look further. I choose to use a BMP retrieval as that should be already in 32-bit RGBx format and that should then be fairly easy (but anything but performant) to pull into a LabVIEW pixmap.

184663212_Hikvision-labviewSDK-Test-v1.1.0.zip

2 hours ago, alvise said:

Returns 2 values.

But if I take a little long to get the stream it returns 34.

Well 2 means there is no data yet to work with, 34 means PLAYM4_NEED_LARGER_BUFFER. Seems obvious what's the problem here.

33 minutes ago, dadreamer said:

You could try to switch to PlayM4_GetBMP, but I assume it gives nothing new. This is odd that even the last error number is zero.

No it may not be. Last error values are notoriously tricky to handle in multithreading code and LabVIEW is highly multithreading. Even if they are stored in thread local storage as do the Microsoft APIs, they are difficult to handle in LabVIEW as your call to GetLastError() is potentially executed in a different thread than the previous call that caused an error and voila you are reading the last error from some completely different function call. And if you force all calls into the UI thread you are in fact calling the functions all in the same thread but between the call to the error causing function and the retrieval of the last error there could be zillion other calls that might have overwritten that last error value already. Same problem exists independent of how the functions are called when the library only maintains a global last error value. If this library has a thread safe last error value or not is not documented as far as I remember.

And are you sure that the PlayM4 APIs make use of the NET_DVR_GetLastError()? If so they are almost certainly thread local storage and you need to make sure that all the functions execute in the same thread. But with call backs going on in the background there could be all kinds of things going on in the time between calling the PlayM4_GetJPEG() function and the NET_DVR_GetLastError() one.

No it has its own PlayM4_GetLastError(long nPort) function! And since it takes the port (handle) as a parameter it is most likely thread context safe as the error is apparently stored in the port somehow.

20 minutes ago, dadreamer said:

Doesn't this work also? Do you ever see PlayM4_GetJPEG returning 1 sometimes?

I'm afraid that doesn't work. It was an idea based on the fact that this function does return a size in its last parameter. If it was a Microsoft API it would work like that, and there was a reasonable chance that they would have followed the model of Microsoft APIs but alas, it seems not the case. It would have been to simple. 😃

I think you have to use the PlayM4_GetPictureSize() to calculate a reasonably large buffer and allocate that and pass it directly to the PlayM4_GetJPEG() function to MAYBE make it work.

2 hours ago, alvise said:

-Where did "BITMAPFILEHEADER" and "BITMAPINFOHEADER" come from here?

What has happened to your Google?

https://letmegooglethat.com/?q=BITMAPINFOHEADER

On 6/8/2022 at 3:38 AM, JKSH said:

Try to develop a distributed SystemLink project where you have both both desktop and real-time targets. Every time you switch targets, you need to wait for all the ~100s of VIs to be recompiled.

Partly valid. This driver independence should have been there before SystemLink was a thing really. It might have made SystemLink a bit harder to make work more comfortable but still. As to SystemLink itself, when it came out it had this "SliverLight" feeling to it for me. It didn't quite go as bad so far, but still.  And on top of that it is very expensive.

I do often combine desktop and multiple real-time target projects. It works of course best with compiled code separated but I have worked that way before separate compiled code was a (reliably working) fact.

2 hours ago, alvise said:

Yes,but  it was causing trouble every time. It was crashing Labview so I didn't add it later because I thought it wasn't necessary.

It absolutely is needed. How should the decoder otherwise get the compressed data to actually get image data????

You have no idea what you are doing here and will never get anywhere in this way. You do not understand what data comes from where, where it should go and how that all relates to each other.

1) The NET_DVR functions are the functions that communicate with the camera. 

2) You can let them draw directly into a window handle. Fairly easy and works as proven already.

3) You can instead (or in addition) install a callback function which is called whenever there is a data package from the camera and that data is passed to that callback.

3a) This data is compressed in the camera in some way, in your case it seems to be MPEG4 data but it could be also another compression.

3b) When you have determined that it is the HKMI (apparently their HikVision Media data  Identifier) datastream in the SYSHEAD message, you need to pass this data to their decoder, this are your PlayM4 functions. If you do not pass those data packages to the MPEG4 decoder stream it has NOTHING to decode and therefore can NOT return any image data ever!!!!!

3c) This passing of the compressed data packages happens with the PlayM4_InputData() method. This function hands the received data do the decoder which then buffers it until it has a complete frame and then decodes that frame and probably, maybe, possibly, hopefully or something stores it in a different buffer in its port (handle) as a decoded frame buffer. 

3d) Only once a complete frame has been received and decoded, can you hope to get something back from the PlayM4_GetJPEG() or one of the other PlayM4_Get****() functions.

If you would not keep putting only images in your post, I might be tempted to give every now and then a fixed VI but I'm not feeling inclined to try to debug images and fix them.

Something like this MIGHT work. But there is no guarantee and I can not find any documentation for those functions, not even in bad English. The idea is that the function is called with an empty array buffer and a 0 nBufSize, then on return it MIGHT return a TRUE status if there is a full frame available or it might NOT. If it returns true then the last parameter is hopefully valid and can be used to determine what size of buffer needs to be allocated and then the function is called again with that buffer. If it does not return true you should not even try to interpret the returned last parameter.

Another possibility MIGHT be to use the PlayM4_GetPictureSize() function. It returns a height and width value. Multiply those and multiply the result with 4 which hopefully should be enough for an entire picture buffer. The code in the Client Demo application does a bit more elaborate calculation for this, naemly:

size = 3 * (4 * lWidth * lHeight + 100)

It then calls PlayM4_GetJPEG() with a buffer allocated to that size, and the size value. And if PlayM4_GetJPEG() does not return true it did not return an image, no matter what. Maybe because there is not yet a full data frame to decode, maybe because one of the maybes up to now in above text didn't hold true, or some other bad luck or whatever.

However even if this eventually works you have another problem. You have data in JPEG format, yet another compressed format that you still can't do much with. Sure you can decompress it with one of the earlier mentioned JPEG Stream to LabVIEW pixmap functions but it is getting now pretty complicated and multi-decoding and multi layer and therefore slooooooooooooooooow.

There seems to be another method which uses yet another callback!! Yupee!!

Looking at the Client Example JPEGPicDownload.cpp file you may have to actually use PlayM4_SetDecCallBack() to install a new callback function that is called when a full frame has been decoded. Fuuuuuuuuuun! This full frame seems to be fully decoded, so this COULD be converted into an IMAQ Vision image for instance. How? How many more weeks do you have for this available?

PS: The StandardCallback I tried to use in my original code only works if the camera supports RTP according to the SDK documentation, so yours may not support that.

11 minutes ago, alvise said:

First of all, there is something I don't really understand. Do I need to create something like the photo below?

That would work, but you also need to wire the nBufSize parameter. Can you guess with what value?

55 minutes ago, crossrulz said:

In my opinion, part of the issue was getting confidence in the "separate from compiled" feature.  I would have to dig into some history to figure out when NI started using it for vi.lib.  But shortly after that, it would make a lot of sense to push this in the drivers.  So if you really want to play the hindsight game here, it is something that should have been feasible around 5 years ago.  But NI was putting all of its marbles into NGX at that time.

Regardless, I am happy about this feature and I look forward to being able to avoid updating drivers purely for the "new LabVIEW version support" and no other changes.

I think the separate compiled code is only a by-product. It could have been done even without the VIs having been set to use separate compiled code. In the worst case the VIs would have been compiled the first time you open a project that uses them. So what?

Inconvenient? I don't think so, at least not nearly as much as having to always chase the right drivers.

9 minutes ago, hooovahh said:

This is a neat feature, but I feel like it would have been more beneficial had it came out 10 years ago or so.  We've struggled with multiple versions of LabVIEW and drivers on a single OS, so now many developers will use VMs or Docker to sandbox a project to only have one version of LabVIEW, and the right version of the tools it uses.  I feel like we've all gravitated to this due to the issues that come up with trying to manager version interoperability.

I do still have a few versions of LabVIEW installed on my main machine.  But if I'm ever building a binary I make sure and do it in the right VM for the project, so an older project doesn't now suddenly rely on the newest version of some dependent tool. 

Definitely agree that it should have been done like that many moons ago already. But it was in a way maybe easier to do it like that as there was limited testing needed and maybe there was also a bit of an intention behind it as if you wanted to use new hardware you had to install the new driver and that made also a new IDE version necessary if the previous one was to old. So a bit of a hidden upgrade nudge. Now with the new NI structure where the different software departments are very much decoupled and independent from each other it proved to be an unmanageable modus operandi. It's not feasible anymore to have to pester DAQmx (and cRIO and System Control and, and, and) folks to release a new version of their software simply because there is a new LabVIEW version.

48 minutes ago, alvise said:

Firstly, I sent 0 value to ''pJPEG'' parameter of ''PlayM4_GetJPEG'' function and 0 value to ''nBufSize'' parameter. Everything worked. ''PlayM4_InputData'' returned 1 value. But ''PlayM4_GetJPEG'' function returned 0 value.

I made some plugins like below, but I don't understand exactly what you want to say here.

Since I cannot directly create a constant U8 array and write it to its zero index (for example: 640*480*3/2=460800), I need to convert this result to U8 array, right?

Unless your data strream consists of very large packets in the beginning, there is absolutely no guarantee that your PlayM4 stream already has enough data the first time around to produce a valid JPG image. It is very well possible that you must feed it with a dozen or more compressed data packages before it is able to even create a single full frame image. And until then the GetJPEG function will likely return false as it has no data to return yet.

Once it has enough data stored internally it may return a valid image but not in the buffer size that you passed to the IntputData method. That image will typically require quite a bit more bytes.

What I did mean to do is to call GetJPEG() first with a null pointer for the data and 0 for nBufSize and then use the return value of pJPEGSize to allocate a big enough buffer to call the function again This may or may not work (again!) depending on the PlayM4 API functionality.

Currently you are anyhow creating a crasho-mobil (again!!) since you pass in an empty byte array and have not configured that parameter to have a Minumum Size that is nBufSize. So you tell the functon: Look I have here a buffer and it contains at least nBufSize bytes, but you really pass in a buffer that contains 0 bytes. Kabom! Crash! as soon as the function has something to write into that buffer and tries to do that, since you told it that there is a buffer of a certain size it can write into.

Quote

Since I cannot directly create a constant U8 array and write it to its zero index (for example: 640*480*3/2=460800), I need to convert this result to U8 array, right?

I have no idea what you try to say with this. If you mean to create an array constant with that size, you can but it is useless. Either you use Initialize Array to create an array of the necessary size or you configure in the Call Library Node the Minimum Size parameter of your array to be that of one of the other parameters, (why not use the nBufSize parameter here, as it very accidentally tells the function what size the byte array parameter has been allocated to). I mean it is just an idea, but it would seem logical to me. 😃

17 minutes ago, alvise said:

-I haven't tried, but I guess so.

 

Now I want to adapt it to ''PlayM4_InputData'' labVIEW, but your previous idea was to replace it with ''PlayM4_GetJPEG''. So I'm looking into this function, but there is ''pJpeg'' in the parameters as in the picture below, what values should be sent to it and I guess its output should be an array of images, right? But here only DWORD returns a number in its output.

 

You definitely will need the PlayM4_InputData(). Somehow you must provide the decoder with the continuous stream of data packages. Then it may or may not choke on the NULL HWND and it may or may not store the decoded data somewhere internal in the port and if it doesn't choke on the NULL HWND AND does store the decoded data somewhere in the port, you can call the PlayM4_GetJPEG() data. But of course you need to provide a large enough buffer to it, so MAYBE you can first call it with a NULL pJpeg and nBufSize = 0 and use the pJpegSize to call it again with a properly allocated buffer. Lots of may and may not and the only one who can find out about that is you!

1 hour ago, Antoine Chalons said:

hey... wait a minute, does this mean that when a class is loaded on host and on RT target, it will be locked, and we'll be stuck with this limitation untill the end of LabVIEW?

Not sure why you respond in this thread with this. But yes that has always been like this. Except that the locking goes away if you remove the reference to the class, or library from any but one of the targets.

What's the calling convention you set for these functions?

And no, you can calculate that multiplication by hand, your hand calculator, an abacus or explicitly in LabVIEW and wire that result to the Call Library Node. It does NOT matter.

But you will want to store that "port" value somewhere (maybe in a shift register, it's just a suggestion) to use when sending the actual data packages to the decoder.

49 minutes ago, alvise said:

I created it as you suggested. But there are 2 things I can't understand.
-For "BYTE *pBuffer", is it necessary to convert the "Handle" array to string and then convert it back to U32 number?
- For ''if (!PlayM4_OpenStream(lPort, pBuffer, dwBufSize, 1024*1024))'', is it necessary to transmit the result of multiplying the number here with ''1024*1024''?

NO!

Configure that parameter simply as Array, Element Type U8, Pass as Data Pointer! But seriously you should first learn more C before trying to go further on this!

And there is no need to convert the byte array into a string to determine its length to pass to the DLL function. There is also an Array Size function, you know?

11 hours ago, alvise said:

 

Are you saying it's unnecessary to send any value to "dwDataType" like in the photo?

I'm saying you send NOTHING there.  This is the definition of the user event datatype and what you "send" there into that cluster makes absolutely no difference. You could just as well leave the entire Bundle by Name away. What is important is the data type that the LVEventData element defines, nothing else. And this data type must at ALL times match the data type setup on the C side for the user event data structure or VERY BAD things will happen.

16 minutes ago, alvise said:

Problem solved. "play mpeg 4.h" no longer crashes, it can be compiled.
The cause of the error was the line here.It was only necessary to leave a space.

 #define PLAYM4_API  extern "C" __declspec(dllexport) 

Of course, and where did you get that header from? It can never compile in any C compiler that I know of. But it would help if you use generally accepted terms in your communication. A CRASH is something that occurs AFTER you have COMPILED and LINKED your application and start it. I usually means that your process dies with or without an according error message. What you encountered was a COMPILE ERROR since the compiler could not understand the code statement.

16 hours ago, alvise said:

Something caught my attention here.

-If I set 'dwDatatype' to 1 it returns output like this.

-If I set 'dwDatatype' to 2 it returns output like this.

As you change the 'dwDataType' from 1 to 2 and run it again, the 'SetCbState return' number value increases. But if I run it again without changing the 'dwDataType', the value of ''SetCbState return'' remains constant. Is there any relation between the number returned by ''SetCbState'' and ''DwDataType''? Why does it have such an effect?

That makes doubly no sense. Or you need to specify which dwDatatype you mean. First as dadreamer already explained and I tried to explain at least twice before.

Quote

If you want the function to return something you can't just configure the Call Library Node to return a value! You need to change the C code accordingly to actually return something. The last version you showed is declared to return void (or in other words nada, njet, nothing). What LabVIEW will return is simply the random value left behind in the EAX register by that function. That could be anything including your birthday!

Second,which dwDataType are you talking about? The dwDataType in the callback routine is not something you should change at all. It is what the SDK driver returns to you and that's it. That is the definition of a callback, it is called by the other software component with values that you should use, not manipulate unless the callback function documentation states explicitly that you are expected to return some data in one or more of the passed in buffers.

59 minutes ago, alvise said:

In a way that I can't understand now, the problem was solved and I ran the example.
I get an output like below. Returns the number ''5017'' when the data is not read.

I guess 5017= NET_DVR_PARAMETER_ERROR

And as you can see in the code, the error value is 5000 + whatever the NET_DVR_GetLastError() returned. Now you go into the SDK documentation and read what error 17 means.

So yes the function thinks that one of the parameters is not valid. The callback function pointer can't really be it, neither can it say anything about the callback data parameter, so that leaves only the session handle. I have no idea why it thinks that is invalid. 

52 minutes ago, alvise said:

I have compiled this review by adding additional addictions as in the picture below.
Probably causes this problem.

Nope, can't be!