alvise

Without buttons, I want the video to be displayed directly when the VI is run. I tried several methods to use the event structure without the button, but in a way that I can't understand, the video capture speed drops. But in general, in the method I currently use, the frame capture rate decreases over time.

I have assigned a Numeric indicator for each event frame and I assign values to the indicators with Value sign, this way I start the Frames one after the other. It's not much problem when it works first time, after a while I notice that the video flow starts to seem slow.

Does this occur due to the accumulation of undesirable events?

The example I'm using right now is the example here, only the flat sequence plugin in the picture was made.

Does anyone have an idea what is wrong with this code?

12 hours ago, Rolf Kalbermatter said:

Well a Windows bitmap file starts with a BITMAPFILEHEADER.

 

Thanks for your answer.
-I think you created something similar in the example you posted before. But it was just necessary to decode the extra compressed images, right?
-Currently, there are 2 kinds of image output formats in the camera's own settings, one is JPEG and the other is BMP, the desired output format can be selected.
And as an extra, H.264 or MPEG4 video output can be selected. I tested both separately and the video can be read. Of course, I don't know what is in the background here. Does it really decode H.264 data or does it still receive data in MPEG4 format even though I choose H.264 video output format? I don't know.I tested it with a different camera. It only works with a camera with an H.264 output format.
-To be honest, I don't know if I can work harder right now.I guess it will still require a lot of effort.Maybe there will be people who can add to it in the future

And I don't want to bother you any more. You and dadreamer have been very helpful so far, thanks for your help, this example we created will probably help other people as well. It was nice to know that precious people like you still exist.

-The last thing I want to do (using my last energy for this job) is to create an example instead of buttons that are displayed when the VI is run directly and can be turned off when the Windows shutdown button is pressed.

13 hours ago, Rolf Kalbermatter said:

There is another function in the PlayCtrl.dll called PlayM4_GetBMP()  There is a VI in my last archive that already should get the BMP data using this function. It is supposed to return a Windows bitmap and yes that one is fully decoded. But!

You retrieve currently a JPG from the stream which most likely isn't exactly the same format as what the camera delivers so this function already does some camera stream decoding and than JPEG encoding, only to then have the COM JPEG decoder pull the undecoded data out of the JPEG image anyhow!

Except that you do not know if the BMP decoder in PlayCtrl.dll is written at least as performant as the COM JPEG decoder, it is actually likely that the detour through the JPEG format requires more performance than directly going to BMP. And the BMP format only contains a small header of maybe 50 bytes or so that is prepended in front of the bitmap data. So not really more than what you get after you have decoded your JPEG image. 

Thanks for your answer.
Actually I was able to get the stream using GetBMP function and even view it with "Win32 Decoder Img Stream.vi" with NI IMAQ but I don't think that is the right way.

-What should be done to decode video stream information with ''GetBmp'' instead of ''GetJpeg'' function directly?

- In the example you shared, I couldn't get a video stream from the example that uses the "GetPlayedFrames" function. I tried to fix the problem but couldn't find the source.

3 hours ago, dadreamer said:

Remove this and try again.

I did what you said above, but I couldn't get any results. Therefore, I used it with the first state.
Finally, the sample project I created is as follows.

memory usage increases over time, but not that fast and not that much (0.1MB).
I tested this increased memory usage in the sample application shared with the SDK and it was also increasing the memory usage in that example.The sudden changing memory usage does not occur much in the method I use below. However, it increases by 3-5 MB from time to time and returns to normal.
This way If I read video in 1280x720 resolution it is using 13% CPU.

Hikvision-labviewSDK-Test-v1.1.4.rar

in an interesting way.When I run the "Preview Demo" example built with C# that comes with the SDK, it seems to increase memory usage, is this normal?

In the example we created in Labview, it is seen that the memory increase observed over time is the same as in the example created with C#. This seems to explain the reason for the increase in memory usage.

by the way,It seems harmless but I want to understand why

6 hours ago, alvise said:

When running -VI, memory usage sometimes goes up and down. For example: 178.3MB to 182.3MB then be 178.3MB again.

it still seems to have something to do with "CoUnitialize" because If I uninstall Win32 Decode Img Stream.vi this problem won't occur.

6 hours ago, Rolf Kalbermatter said:

I did check the callback function from earlier again and there is of course memory leak in there!

The NumericArrayResize() is called twice. This in itself would be just useless but not fatal if there was not also a new eventData structure declared inside the function.  Without that the second call would be simply useless, it would not really do much as the size is both times the same and the DSSetHandleSize() used internally in that function is basically a NO-OP if the new size is the same as what the handle already has.

But with the eventData structure declared again it creates in fact two handles on each call but only deallocates one of them. Get rid of the first two lines inside the if (!err) block!

I removed the lines that you said have a problem and compiled it again,

#define LibAPI(retval)       __declspec(dllexport) EXTERNC retval __cdecl
#define Callback(retval)     __declspec(dllexport) EXTERNC retval __stdcall
#include "lv_prolog.h"
typedef struct
{
    int32_t size;
    uint8_t elm[1];
}
LVByteArrayRec, * LVByteArrayPtr, ** LVByteArrayHdl;// ** LVByteArrayHdl;pointer to a pointer

typedef struct
{
    LONG realHandle;
    DWORD dataType;
    LVByteArrayHdl handle;
} LVEventData;
#include "lv_epilog.h"
extern "C" __declspec(dllexport) void __cdecl SetCbState(LVBoolean * state);
// where Standard Boolean type case is set: typedef uInt8 LVBoolean
LVBoolean cbState = LVBooleanFalse;
// always set to false on startup
extern "C" __declspec(dllexport) void __cdecl SetCbState(LVBoolean * state)
{
    cbState = *state;
}
extern "C" __declspec(dllexport) void __stdcall DataCallBack(LONG lRealHandle, DWORD dwDataType, BYTE * pBuffer, DWORD dwBufSize, DWORD dwUser)
{

    if (cbState == LVBooleanTrue || dwDataType == 1 )
    {
        if (dwDataType == 1) 
        {
            DbgPrintf("NET_DVR_SYSHEAD received-test 1");
        }
        LVEventData eventData = { 0 };
        MgErr err = NumericArrayResize(uB, 1, (UHandle*)&(eventData.handle), dwBufSize);
        if (!err)// send callback data if there is no error and the cbstatus is true.
        {
            LVUserEventRef userEvent = (LVUserEventRef)dwUser;
            MoveBlock(pBuffer, (*(eventData.handle))->elm, dwBufSize);
            (*(eventData.handle))->size = (int32_t)dwBufSize;
            eventData.realHandle = lRealHandle;
            eventData.dataType = dwDataType;
            PostLVUserEvent(userEvent, &eventData);
            DSDisposeHandle(eventData.handle);
        }
    }
}
//If the above if condition does not occur, the LVUserEventRef here does not take a value.
typedef BOOL(__stdcall* Type_SetStandardDataCallBack)(LONG lRealHandle, void(CALLBACK* fRealDataCallBack) (LONG lRealHandle, DWORD dwDataType, BYTE* pBuffer, DWORD dwBufSize, DWORD dwUser), DWORD dwUser);
extern "C" __declspec(dllexport) MgErr __cdecl InstallStandardCallback(LONG lRealHandle, LVUserEventRef * refnum)
{
    HMODULE hDLL = LoadLibraryW(L"HCNetSDK.dll");
    if (hDLL)
    {
        Type_SetStandardDataCallBack installFunc = (Type_SetStandardDataCallBack)GetProcAddress(hDLL, "NET_DVR_SetRealDataCallBack");
        if (installFunc)
        {
            BOOL retval;
            if (refnum && *refnum)
                retval = installFunc(lRealHandle, DataCallBack, (DWORD)(*refnum));
            else
                retval = installFunc(lRealHandle, NULL, 0);
            if (retval)
                return mgNoErr;
            else
                return 5000 + NET_DVR_GetLastError();
        }
        FreeLibrary(hDLL);
    }
    return mgNotSupported;
}

I think the problem is solved. Currently, memory usage has remained stable. Exiting VI releases used memory. It has a memory usage and CPU usage as follows.

But;

When running -VI, memory usage sometimes goes up and down. For example: 178.3MB to 182.3MB then be 178.3MB again.

-I still think it's a slight memory leak because memory usage is increasing albeit very slowly.

Example:

By the way, I tested again by removing PlayM4 and Win32 Decode Img Stream.vi. Again, there are small leaks as in the picture above.

35 minutes ago, dadreamer said:

Actually I would try to eliminate one handle allocation by passing a pointer with PostLVUserEvent and dereferencing it in LabVIEW (+ deallocate it manually, of course) (as I already mentioned on the first pages of this thread)

I guess I'll go step by step again.
-In the example we created in C++, we will export a pointer as an export function with PostLVUserEvent and re-reference it in lab view with CFLN in lab view.
Is it true so far?

actually I don't think the problem is in "Win32 Decode Img Stream.vi" because as you can see in the pictures I shared before, I completely removed it and observed the memory usage, nothing changed.memory usage continued to increase.

2 hours ago, Rolf Kalbermatter said:

There of course always is. But I have no idea where the memory leak would be. One thing that looks not only suspicious but is in fact possibly unnecessary is the call to CoInitializeEx(). LabVIEW has to do that early on during startup already in order to be ever able to access ActiveX functionality. Of course I'm not sure if LabVIEW does this on every possible thread that it initializes, most likely not. So you run into a potential problem here. The one thread it for sure will do CoInitialize(ex) on is the UI thread. So if you execute all your COM functions in that decoder VI in the UI thread you can forget about calling this function. However that has of course implications for your performance since the entire decoding is then done in the UI thread. If you want to do in any arbitrary thread for performance reason you may need to call the CoInitializeEx anyways just to be sure, BUT!!!!!! Go read the documentation for that function!

Your function does call CoInitializeEx() on every invocation but never the CoUnintialize(). That certainly has the potential of allocating new resources on every single invocation that are never ever released again. You will need to add a CoUnintialize() at the end of that function and not just in the SUCCESS (return value 0) case but also in the Done Nothing (return value 1) case when CoInitializeEx() returns.

Of course returning a BMP instead has likely the advantage to not only do away with that entire CoInitialize() and CoUninitialze() business but also avoids the potential need of extra resources to decode the MPEG (or with another camera maybe H264) frame, encode it into a JPEG image, and then decode it back into a bitmap. Instead you get immediately a decoded flat bitmap that you only have to index at the right location from interpreting some of the values in the BITMAPINFOHEADER in front of the pixel data.

By the way, you mentioned CoInitializE(), but when I remove Decode Image Stream VI, there is no change in memory usage, which increases over time, so it still increases.

-How do we eliminate all CoInitialize work by returning BMP? Why does returning BMP have such an advantage?

-Does  taking it as BMP also cause a drop in camera FPS? As far as I know, BMP is bigger in size.

By the way, I tried to find the source of the problem in the example you edited earlier (which takes the video as BMP), but I couldn't find the source of the problem.

Yes just the callback does not cause a memory leak per se it is not a memory leak that is happening right now.
So how can we solve this problem, is there a solution to it?
 

1 hour ago, Rolf Kalbermatter said:

The callback itself should not leak memory but it does two handle allocations and deallocations each time around.

The first handle is allocated and deallocated in the callback function itself. The second handle is allocated in LVPostUserEvent() when it copies the incoming data and released in the callback event frame.

I never understood why there is not at least an option to tell LVPostUserEvent() to actually take ownership of the data. But it can't be changed by now.

The optimization would be to store the handle between callback invocations somewhere and only resize it when the new data needs a bigger buffer. However do not attempt to do that yourself! This is fraught with trouble! You can't just store it in a static global variable since the callback can be potentially called from more than one thread in parallel once you would operate two or more cameras at the same time. You would need to store that handle in TLS (Thread Local Space).

Currently, when I run the VI, the memory consumption remains the same as when I close the VI. I guess the memory space used is not freed when the VI is shut down.

Currently I have a problem like this when trying to read images from a single camera and I was intending to run 2 cameras

That's what I think too, something I shouldn't do on my own because it can cause new memory leak issues, but it can also cause other issues (like RAM burnt out).
-So if you tell me what to do step by step to solve this problem, I think I can handle it.

9 hours ago, dadreamer said:

Does it get to 1GB after running for 1 hour?

The memory usage observed after 40 minute in the  photo below.

This app used PlayM4 decoder and the image conversion

In this application, the dll used for callback is not used, only the video is taken from the picturebox.

In this application, callback is used but PlayM4.dll is not used.

As you said, I removed everything used in the "LVEventData" framework, as in the picture below, but again, increasing memory usage is observed.

8 minutes ago, dadreamer said:

Yeah, something is not freed properly and eating your resources. You need to find what it is.

By the way, while the application is running, the memory usage is increasing. If I stop the application, the memory used by the labview remains the same as when I closed the application. If I run the application again, it continues to increase the memory usage from where it left off.

Is there any possibility that the callback dll is causing this?

We are not overwriting the labview data, are we, because if I close and open the labview completely, the memory usage returns to normal.

I guess it's a memory leak.

The memory usage used when the problem came out is as follows, which I viewed in the task manager.when I run the app again.

When I restart Labview, the memory usage is displayed in the task manager as follows.when I run the app again.

Meanwhile, the memory usage is increasing while the application is running.

36 minutes ago, dadreamer said:

Very hard to tell. You need to check the functions return values one by one, and not only in LabVIEW, but in the DLL as well. Use DbgPrintf window to ease the debugging. To me it seems more like a call sequence mistake of some sort. But it may be resource leaking too, as if for example you'd forget to close some handle or free some pointer.

This must be really hard to debug because I'll have to wait at least 1 hour after each fix to see if it's the problem again

By search order error do you mean "playM4.dll" or for any dll?

Something like the following catches my attention. I get this error if ''JpegSize'' reaches over 9000.

Of course, something else caught my attention.
Before adding the ones in the picture below, when I stopped the application and tried to close the labview project window after closing the VIs, the labview was crashing, the project window was closing by itself. If I opened the labview again, the recovery window would appear. Of course, when I added the functions in the picture below, I didn't encounter such a problem anymore.

I think there is a problem now.
Although everything seemed to be working, the video was being received without any problems, but I read the video information without turning it off for a long time(1 hour). When I stop the application and try to run it again, the video information is no longer read. If I close the labview completely and open it again, I can get the video again.

I've tested this several times, it happens like this, what could be the reason?

I made the correction as you suggested. I got an output like below.

Correction! It was even faster when I removed the arrays.

I guess this is how it's supposed to be here, right?

By the way, the first sample application we created consumes a lot of CPU as in the picture below.I think this application alone consumes 35-40% CPU. Is this normal? Or is there an error?

189071249_Hikvision-labviewSDK-Test-v1.1.2.rar

Thank you for your explanation.

After pressing the "Start" button, I get the following error without pressing the "Stream" button.
 

The "User ID" here is increasing by a number with each new run, I guess, which means that it is logged in and not logged out again.

It doesn't return true at all here.

3 hours ago, Gribo said:

The thread setting and execution mode (re-entrant vs non re-entrant) are set in the VI properties, under execution.

Re-entrant means that the OS (Or the LVRTE) can stop the VI's execution, switch to another thread, switch back, and the VI should continue as normal. Non-reentrant means the VI has to finish execution before such context switch happens, otherwise bad things (tm) happen.

Thanks for your answer, actually, what I wrote in the previous message is not that I don't know completely about this subject, I meant some things that I don't understand deeply.
For example; Why are we talking about reentrant and non-reentrant when using the Call library function node?

Warning: It's possible that everything I've said here is wrong!

Possible answer: Because we want to have parallel loops and the same subVI (for the previous Play4M_GetJpeg) may need to be run several times. Each time it is called in the subVI we are calling the same function (CFLN) again, and in this case the non-reentrant function if called at the same time can cause queueing. It means it wouldn't make sense to choose non-reentrant.
So why is the VI we use chosen as non-reentrant?
etc. many questions.

But at the current stage, some of the contradictions I mentioned above have disappeared, and there is no need for this at the last point that Rolf shared and in general we reached. because since the same set of gates are used by all callers of the VI, the values from the last call of that VI are available to the next caller.

By the way, sorry for my bad english.From everyone participating in the discussion on this forum.  

4 hours ago, Rolf Kalbermatter said:

Sometimes I feel like I must be talking some obscure galactic slang or something!

Setting Call Library Nodes to run in UI thread is indeed safe but I never said this is required here.

It's misunderstood, but not because you don't explain it well, but because I don't know some terms well and it's like someone learning to swim in a pool trying to swim in the ocean.

One point that confused me was this.
-Run in UI Thread: how it works and how it should be set.
-Run in Any Thread: how it works and how it should be set.
What is enterant and Reanterant and how is it related to UI thread or Any Thread.

Not understanding these in all its depth causes confusion. I am trying to understand it again by analyzing the Application we have created so far and taking into account the work it has done.

4 hours ago, Rolf Kalbermatter said:

What about that English sentence is not clear? Have you looked at the last attachment I added to one of my last messages?

I didn't see the edit you added. Because when I read your first post, that plugin was not there. I guess I was trying to understand what dadreamer was saying when you added it later.

The application you are editing is really neat and tidy.I tested the app you edited and it responded with an error like below.

erorr #define PLAYM4_INVALID_PORT 32 

27 minutes ago, dadreamer said:

Something like that, but...

- Why all your CLFNs are orange again?

- You did not introduce a minimum delay for the loop.

- Better to use some button, pressing which you would capture the image. So you would need a Case Structure or an Event Structure in that second loop to handle the button actions.

-Are you saying they should all be yellow (Run in Any Thread).
I never understood this. From what Rolf said;
 ''HCNetSDK.dll'' and ''hikLabview-0.0.01.dll'' should have CFLNs  yellow(Run in Any Thread).
  ''PlayCtrl.dll''' CFLNs should be orange(Run UI thread). as I understood.

-I add -10 milliseconds wait

-Actually, I need to add a button to start the stream, but I want to start the stream automatically as soon as the VI is run. I don't want the user to press any button to start the video, I will try to render it.

I think you mentioned something like this. Do you think there is something missing or wrong with this VI?

Preview CAM-3.vi

Currently I'm calling "PlayM4_GetJPEG" in a parallel while loop so that the data stream is faster and LabVIEW doesn't freeze, but for some reason "PlayM4_GetJPEG" is not returning any value, what could be the reason? The data seems to be passed to the second while loop without any problems looks.

What am I missing here?

Preview CAM-2.vi