Using the DLL files of an application compiled with C# with labview

[alvise]

36 minutes ago, dadreamer said:

Very hard to tell. You need to check the functions return values one by one, and not only in LabVIEW, but in the DLL as well. Use DbgPrintf window to ease the debugging. To me it seems more like a call sequence mistake of some sort. But it may be resource leaking too, as if for example you'd forget to close some handle or free some pointer.

 

This must be really hard to debug because I'll have to wait at least 1 hour after each fix to see if it's the problem again

By search order error do you mean "playM4.dll" or for any dll?

Something like the following catches my attention. I get this error if ''JpegSize'' reaches over 9000.

Of course, something else caught my attention.
Before adding the ones in the picture below, when I stopped the application and tried to close the labview project window after closing the VIs, the labview was crashing, the project window was closing by itself. If I opened the labview again, the recovery window would appear. Of course, when I added the functions in the picture below, I didn't encounter such a problem anymore.

 

 

 

Edited June 10, 2022 by alvise

[dadreamer]

12 minutes ago, alvise said:

By search order error do you mean "playM4.dll" or for any language?

I'm afraid you need to check and test everything. By the way. Do you see a memory leak in Windows Task Manager when running the app for a long time? Compare the occupied memory space, when you've just started the stream, with that one after a hour.

14 minutes ago, alvise said:

I get this error if ''JpegSize'' reaches over 9000.

You mean GetJPEG returning 0? If that occurs on the recent version of your VI, then it might be that when the image is very large, it takes too much time to process the data, and your LV User Event doesn't have a chance to handle it all in time. You need to view the Event Inspector and watch the amount of events in there.

 

[alvise]

I guess it's a memory leak.

The memory usage used when the problem came out is as follows, which I viewed in the task manager.when I run the app again.

When I restart Labview, the memory usage is displayed in the task manager as follows.when I run the app again.

Meanwhile, the memory usage is increasing while the application is running.

Edited June 10, 2022 by alvise

[dadreamer]

Yeah, something is not freed properly and eating your resources. You need to find what it is.

[alvise]

8 minutes ago, dadreamer said:

Yeah, something is not freed properly and eating your resources. You need to find what it is.

By the way, while the application is running, the memory usage is increasing. If I stop the application, the memory used by the labview remains the same as when I closed the application. If I run the application again, it continues to increase the memory usage from where it left off.

Is there any possibility that the callback dll is causing this?

We are not overwriting the labview data, are we, because if I close and open the labview completely, the memory usage returns to normal.

Edited June 10, 2022 by alvise

[dadreamer]

5 minutes ago, alvise said:

Is there any possibility that the callback dll is causing this?

You may relatively easy check this. Make a test VI and remove there everything related to the PlayM4 decoder and the image conversion. Leave the User Event completely empty. Get the memory space measurements during the run of that VI.

Edited June 10, 2022 by dadreamer

[alvise]

As you said, I removed everything used in the "LVEventData" framework, as in the picture below, but again, increasing memory usage is observed.

 

[dadreamer]

Does it get to 1GB after running for 1 hour?

Edited June 10, 2022 by dadreamer

[alvise]

9 hours ago, dadreamer said:

Does it get to 1GB after running for 1 hour?

The memory usage observed after 40 minute in the  photo below.

This app used PlayM4 decoder and the image conversion

 

In this application, the dll used for callback is not used, only the video is taken from the picturebox.

 

In this application, callback is used but PlayM4.dll is not used.

Edited June 11, 2022 by alvise

[Rolf Kalbermatter]

5 hours ago, alvise said:

In this application, callback is used but PlayM4.dll is not used.

The callback itself should not leak memory but it does two handle allocations and deallocations each time around.

The first handle is allocated and deallocated in the callback function itself. The second handle is allocated in LVPostUserEvent() when it copies the incoming data and released in the callback event frame.

I never understood why there is not at least an option to tell LVPostUserEvent() to actually take ownership of the data. But it can't be changed by now.

The optimization would be to store the handle between callback invocations somewhere and only resize it when the new data needs a bigger buffer. However do not attempt to do that yourself! This is fraught with trouble! You can't just store it in a static global variable since the callback can be potentially called from more than one thread in parallel once you would operate two or more cameras at the same time. You would need to store that handle in TLS (Thread Local Space).

[alvise]

1 hour ago, Rolf Kalbermatter said:

The callback itself should not leak memory but it does two handle allocations and deallocations each time around.

The first handle is allocated and deallocated in the callback function itself. The second handle is allocated in LVPostUserEvent() when it copies the incoming data and released in the callback event frame.

I never understood why there is not at least an option to tell LVPostUserEvent() to actually take ownership of the data. But it can't be changed by now.

The optimization would be to store the handle between callback invocations somewhere and only resize it when the new data needs a bigger buffer. However do not attempt to do that yourself! This is fraught with trouble! You can't just store it in a static global variable since the callback can be potentially called from more than one thread in parallel once you would operate two or more cameras at the same time. You would need to store that handle in TLS (Thread Local Space).

Currently, when I run the VI, the memory consumption remains the same as when I close the VI. I guess the memory space used is not freed when the VI is shut down.

Currently I have a problem like this when trying to read images from a single camera and I was intending to run 2 cameras

That's what I think too, something I shouldn't do on my own because it can cause new memory leak issues, but it can also cause other issues (like RAM burnt out).
-So if you tell me what to do step by step to solve this problem, I think I can handle it.

Edited June 11, 2022 by alvise

[Rolf Kalbermatter]

What I meant to say is that the callback alone should not leak memory. But the frequent allocation and deallocation certainly will fragment memory over time. That is not the same, although it can look similar. Due to fragmentation more and more blocks of memory are getting allocated and while LabVIEW (or whatever manages the memory for the LabVIEW memory manager functions) does know about these allocations and hasn't forgotten about them (which is the meaning of a memory leak) it can't reuse those blocks easily for new memory requests, leaving that memory reserved but unused. That way the memory footprint of an application can slowly increase. It's not a memory leak since that memory is still accounted for internally but it causes more and more memory to be allocated.

If you however allocate a memory pointer (or handle) and then consequently forget about it you have created a true memory leak. In the case of a handle there is the potential to hand it to LabVIEW for further use in some instances which will make LabVIEW responsible to release it, but these are fairly limited, usually only to parameters that are passed in from the diagram through Call Library Node parameters and then returned back.

[alvise]

Yes just the callback does not cause a memory leak per se it is not a memory leak that is happening right now.
So how can we solve this problem, is there a solution to it?
 

[Rolf Kalbermatter]

1 hour ago, alvise said:

Yes just the callback does not cause a memory leak per se it is not a memory leak that is happening right now.
So how can we solve this problem, is there a solution to it?

There of course always is. But I have no idea where the memory leak would be. One thing that looks not only suspicious but is in fact possibly unnecessary is the call to CoInitializeEx(). LabVIEW has to do that early on during startup already in order to be ever able to access ActiveX functionality. Of course I'm not sure if LabVIEW does this on every possible thread that it initializes, most likely not. So you run into a potential problem here. The one thread it for sure will do CoInitialize(ex) on is the UI thread. So if you execute all your COM functions in that decoder VI in the UI thread you can forget about calling this function. However that has of course implications for your performance since the entire decoding is then done in the UI thread. If you want to do in any arbitrary thread for performance reason you may need to call the CoInitializeEx anyways just to be sure, BUT!!!!!! Go read the documentation for that function!

Quote

To close the COM library gracefully on a thread, each successful call to CoInitialize or CoInitializeEx, including any call that returns S_FALSE, must be balanced by a corresponding call to CoUninitialize.

Your function does call CoInitializeEx() on every invocation but never the CoUnintialize(). That certainly has the potential of allocating new resources on every single invocation that are never ever released again. You will need to add a CoUnintialize() at the end of that function and not just in the SUCCESS (return value 0) case but also in the Done Nothing (return value 1) case when CoInitializeEx() returns.

Of course returning a BMP instead has likely the advantage to not only do away with that entire CoInitialize() and CoUninitialze() business but also avoids the potential need of extra resources to decode the MPEG (or with another camera maybe H264) frame, encode it into a JPEG image, and then decode it back into a bitmap. Instead you get immediately a decoded flat bitmap that you only have to index at the right location from interpreting some of the values in the BITMAPINFOHEADER in front of the pixel data.

Edited June 11, 2022 by Rolf Kalbermatter

[alvise]

2 hours ago, Rolf Kalbermatter said:

There of course always is. But I have no idea where the memory leak would be. One thing that looks not only suspicious but is in fact possibly unnecessary is the call to CoInitializeEx(). LabVIEW has to do that early on during startup already in order to be ever able to access ActiveX functionality. Of course I'm not sure if LabVIEW does this on every possible thread that it initializes, most likely not. So you run into a potential problem here. The one thread it for sure will do CoInitialize(ex) on is the UI thread. So if you execute all your COM functions in that decoder VI in the UI thread you can forget about calling this function. However that has of course implications for your performance since the entire decoding is then done in the UI thread. If you want to do in any arbitrary thread for performance reason you may need to call the CoInitializeEx anyways just to be sure, BUT!!!!!! Go read the documentation for that function!

Your function does call CoInitializeEx() on every invocation but never the CoUnintialize(). That certainly has the potential of allocating new resources on every single invocation that are never ever released again. You will need to add a CoUnintialize() at the end of that function and not just in the SUCCESS (return value 0) case but also in the Done Nothing (return value 1) case when CoInitializeEx() returns.

Of course returning a BMP instead has likely the advantage to not only do away with that entire CoInitialize() and CoUninitialze() business but also avoids the potential need of extra resources to decode the MPEG (or with another camera maybe H264) frame, encode it into a JPEG image, and then decode it back into a bitmap. Instead you get immediately a decoded flat bitmap that you only have to index at the right location from interpreting some of the values in the BITMAPINFOHEADER in front of the pixel data.

By the way, you mentioned CoInitializE(), but when I remove Decode Image Stream VI, there is no change in memory usage, which increases over time, so it still increases.

 

-How do we eliminate all CoInitialize work by returning BMP? Why does returning BMP have such an advantage?

-Does  taking it as BMP also cause a drop in camera FPS? As far as I know, BMP is bigger in size.

By the way, I tried to find the source of the problem in the example you edited earlier (which takes the video as BMP), but I couldn't find the source of the problem.

 

Edited June 11, 2022 by alvise

[dadreamer]

Win32 Decode Img Stream.vi does not produce memory leaks of any sort. I've been running it on a production for months. Never ever received errors from that VI. As to CoInitializeEx, it was implemented this way in the original thread on SO, I just borrowed the solution. But I checked now, CoInitializeEx always returns TRUE, no matter what. Extra resources are not allocated. I assume it's safe enough to call it multiple times from the same thread. But you may easily add CoUninitialize to there, if you're afraid it works improperly. I'm just thinking this might be not a good idea, given that description of the function:

Quote

Closes the COM library on the current thread, unloads all DLLs loaded by the thread, frees any other resources that the thread maintains, and forces all RPC connections on the thread to close.

A lot of work would be done on each call. Better to do this once on the app exit. Or leave it to the OS, when LabVIEW quits.

Edited June 11, 2022 by dadreamer

[alvise]

actually I don't think the problem is in "Win32 Decode Img Stream.vi" because as you can see in the pictures I shared before, I completely removed it and observed the memory usage, nothing changed.memory usage continued to increase.

Edited June 11, 2022 by alvise

[dadreamer]

Actually I would try to eliminate one handle allocation by passing a pointer with PostLVUserEvent and dereferencing it in LabVIEW (+ deallocate it manually, of course) (as I already mentioned on the first pages of this thread). It would be more of a test to see if the large memory consumption goes away. If not, then it's reasonable to allocate the space just one time (when the DLL is loaded, for example, or right on the diagram with DSNewPtr) and free it when the app ends (when the DLL is unloaded or with DSDisposePtr on the diagram).

Edited June 11, 2022 by dadreamer

[alvise]

35 minutes ago, dadreamer said:

Actually I would try to eliminate one handle allocation by passing a pointer with PostLVUserEvent and dereferencing it in LabVIEW (+ deallocate it manually, of course) (as I already mentioned on the first pages of this thread)

I guess I'll go step by step again.
-In the example we created in C++, we will export a pointer as an export function with PostLVUserEvent and re-reference it in lab view with CFLN in lab view.
Is it true so far?

[dadreamer]

For this you would need to:

- change the type of the user event in two locations: on the diagram and in the callback code;

- slightly change the callback logic, trying some things: first try to post the cluster with the data handle address instead of the array, second try to not allocate the array and copy the data into it, but post the original SDK pointer to LV directly. Plus in fact we even don't decode the data ourselves - the PlayM4 decoder does it for us! As we need the data type mostly and don't need the actual data contents, we could ease the callback logic a little.

But I neither want to say you to do this right now, nor I'm inclined to do this myself. I think Rolf is going to come with more advanced diagnostics or solutions.

Edited June 11, 2022 by dadreamer

[Rolf Kalbermatter]

6 hours ago, dadreamer said:

Win32 Decode Img Stream.vi does not produce memory leaks of any sort. I've been running it on a production for months. Never ever received errors from that VI. As to CoInitializeEx, it was implemented this way in the original thread on SO, I just borrowed the solution. But I checked now, CoInitializeEx always returns TRUE, no matter what. Extra resources are not allocated. I assume it's safe enough to call it multiple times from the same thread. But you may easily add CoUninitialize to there, if you're afraid it works improperly. I'm just thinking this might be not a good idea, given that description of the function:

A lot of work would be done on each call. Better to do this once on the app exit. Or leave it to the OS, when LabVIEW quits.

Well, how many times per second do you call that VI? In this camera application it was called ONLY about 20 to 50 times per second. And MSDN definitely and clearly states that you should balance every call to CoInitialize(Ex) with exactly one call to CoUnitialize. And that of course has to happen in the same thread!

If it would not allocate something, somehow, somewhere, that would not be necessary as the CoIntialize(Ex) when it returns TRUE (actually S_FALSE) simply indicates that it did not initialize the COM system for the current thread, but MSDN still says you need to call CoUninitialize even when CoInitialize(Ex) returns S_FALSE. That is definitely not for nothing! And if you do it in the LabVIEW VI you have actually a problem as you can not guarantee that CoUnitialze is called in the same thread as CoInitialize was, unless you make the entire VI subroutine priority. This guarantees that LabVIEW will NOT switch threads ever for the duration of the entire VI call. 

If it always returns TRUE (S_FALSE) even the first time you call it, it simply means that LabVIEW apparently already initialized the COM system for that thread.

CoUnitialize should not do much, except maybe deallocate that thread local storage that it somehow creates to maintain some management information unless it is the matching call to the first CoIntialize(Ex). In that case it gets indeed rather expensive as it would deinitialize the COM system for the current thread.

So to be fully proper without risking to allocate new resources with every call I think the safest would be:

To add a CoUnitialize call at the end of the VI if CoInitializeEx returned 1 (S_FALSE) to make sure nothing is accumulating but don't call it if CoInitializeEx returned 0 (S_OK) as it was the first initialization of COM for the current thread. And make that VI subroutine. It's a pitta for debugging but once it works you simply should guarantee that COM functions execute from the same thread in which an object was created. Most COM class implementations are not guaranteed to work reliable in full multithreading operation.

And please do NOT try to pass the pointer that the callback function receives directly through the LabVIEW event. That pointer ceases to be valid at the moment the callback function returns control to the caller, but that event goes through an event queue and then the event structure and by the time your event structure sees that event the pointer from the callback function has either been reused by the SDK already for something else, or even completely deallocated. You MUST create a copy of that pointer if you want to read the data outside of the callback function.

I did check the callback function from earlier again and there is of course memory leak in there!

The NumericArrayResize() is called twice. This in itself would be just useless but not fatal if there was not also a new eventData structure declared inside the function.  Without that the second call would be simply useless, it would not really do much as the size is both times the same and the DSSetHandleSize() used internally in that function is basically a NO-OP if the new size is the same as what the handle already has.

extern "C" __declspec(dllexport) void __stdcall DataCallBack(LONG lRealHandle, DWORD dwDataType, BYTE * pBuffer, DWORD dwBufSize, DWORD dwUser)
{
    if (cbState == LVBooleanTrue)
    {
        LVEventData eventData = { 0 };
        MgErr err = NumericArrayResize(uB, 1, (UHandle*)&(eventData.handle), dwBufSize);
        if (!err) // send callback data if there is no error and the cbstatus is true.
        {
            // LVEventData eventData = { 0 };
            // MgErr err = NumericArrayResize(uB, 1, (UHandle*)&(eventData.handle), dwBufSize); // Not useful
            LVUserEventRef userEvent = (LVUserEventRef)dwUser;
            MoveBlock(pBuffer, (*(eventData.handle))->elm, dwBufSize);
            (*(eventData.handle))->size = (int32_t)dwBufSize;
            eventData.realHandle = lRealHandle;
            eventData.dataType = dwDataType;
            PostLVUserEvent(userEvent, &eventData);
            DSDisposeHandle(eventData.handle);
        }
    }
}

But with the eventData structure declared again it creates in fact two handles on each call but only deallocates one of them. Get rid of the first two lines inside the if (!err) block!

Edited June 12, 2022 by Rolf Kalbermatter

[ShaunR]

11 hours ago, alvise said:

I completely removed it and observed the memory usage, nothing changed.memory usage continued to increase.

But not by the same amount.

That is indicative of multiple memory leaks. You must use the Diagram Disable structure to disable parts of the code until there is no memory leak and re-enable a couple of functions at a time to see when a leak returns. Then you can try and figure out why it is leaking. Until you know where it's occurring you are just flailing wildly and we can't really help you.

[alvise]

6 hours ago, Rolf Kalbermatter said:

I did check the callback function from earlier again and there is of course memory leak in there!

The NumericArrayResize() is called twice. This in itself would be just useless but not fatal if there was not also a new eventData structure declared inside the function.  Without that the second call would be simply useless, it would not really do much as the size is both times the same and the DSSetHandleSize() used internally in that function is basically a NO-OP if the new size is the same as what the handle already has.

But with the eventData structure declared again it creates in fact two handles on each call but only deallocates one of them. Get rid of the first two lines inside the if (!err) block!

 

 

I removed the lines that you said have a problem and compiled it again,

#define LibAPI(retval)       __declspec(dllexport) EXTERNC retval __cdecl
#define Callback(retval)     __declspec(dllexport) EXTERNC retval __stdcall
#include "lv_prolog.h"
typedef struct
{
    int32_t size;
    uint8_t elm[1];
}
LVByteArrayRec, * LVByteArrayPtr, ** LVByteArrayHdl;// ** LVByteArrayHdl;pointer to a pointer

typedef struct
{
    LONG realHandle;
    DWORD dataType;
    LVByteArrayHdl handle;
} LVEventData;
#include "lv_epilog.h"
extern "C" __declspec(dllexport) void __cdecl SetCbState(LVBoolean * state);
// where Standard Boolean type case is set: typedef uInt8 LVBoolean
LVBoolean cbState = LVBooleanFalse;
// always set to false on startup
extern "C" __declspec(dllexport) void __cdecl SetCbState(LVBoolean * state)
{
    cbState = *state;
}
extern "C" __declspec(dllexport) void __stdcall DataCallBack(LONG lRealHandle, DWORD dwDataType, BYTE * pBuffer, DWORD dwBufSize, DWORD dwUser)
{

    if (cbState == LVBooleanTrue || dwDataType == 1 )
    {
        if (dwDataType == 1) 
        {
            DbgPrintf("NET_DVR_SYSHEAD received-test 1");
        }
        LVEventData eventData = { 0 };
        MgErr err = NumericArrayResize(uB, 1, (UHandle*)&(eventData.handle), dwBufSize);
        if (!err)// send callback data if there is no error and the cbstatus is true.
        {
            LVUserEventRef userEvent = (LVUserEventRef)dwUser;
            MoveBlock(pBuffer, (*(eventData.handle))->elm, dwBufSize);
            (*(eventData.handle))->size = (int32_t)dwBufSize;
            eventData.realHandle = lRealHandle;
            eventData.dataType = dwDataType;
            PostLVUserEvent(userEvent, &eventData);
            DSDisposeHandle(eventData.handle);
        }
    }
}
//If the above if condition does not occur, the LVUserEventRef here does not take a value.
typedef BOOL(__stdcall* Type_SetStandardDataCallBack)(LONG lRealHandle, void(CALLBACK* fRealDataCallBack) (LONG lRealHandle, DWORD dwDataType, BYTE* pBuffer, DWORD dwBufSize, DWORD dwUser), DWORD dwUser);
extern "C" __declspec(dllexport) MgErr __cdecl InstallStandardCallback(LONG lRealHandle, LVUserEventRef * refnum)
{
    HMODULE hDLL = LoadLibraryW(L"HCNetSDK.dll");
    if (hDLL)
    {
        Type_SetStandardDataCallBack installFunc = (Type_SetStandardDataCallBack)GetProcAddress(hDLL, "NET_DVR_SetRealDataCallBack");
        if (installFunc)
        {
            BOOL retval;
            if (refnum && *refnum)
                retval = installFunc(lRealHandle, DataCallBack, (DWORD)(*refnum));
            else
                retval = installFunc(lRealHandle, NULL, 0);
            if (retval)
                return mgNoErr;
            else
                return 5000 + NET_DVR_GetLastError();
        }
        FreeLibrary(hDLL);
    }
    return mgNotSupported;
}

I think the problem is solved. Currently, memory usage has remained stable. Exiting VI releases used memory. It has a memory usage and CPU usage as follows.

But;

When running -VI, memory usage sometimes goes up and down. For example: 178.3MB to 182.3MB then be 178.3MB again.

-I still think it's a slight memory leak because memory usage is increasing albeit very slowly.

Example:

By the way, I tested again by removing PlayM4 and Win32 Decode Img Stream.vi. Again, there are small leaks as in the picture above.

Edited June 12, 2022 by alvise

[alvise]

in an interesting way.When I run the "Preview Demo" example built with C# that comes with the SDK, it seems to increase memory usage, is this normal?

In the example we created in Labview, it is seen that the memory increase observed over time is the same as in the example created with C#. This seems to explain the reason for the increase in memory usage.

 

by the way,It seems harmless but I want to understand why

6 hours ago, alvise said:

When running -VI, memory usage sometimes goes up and down. For example: 178.3MB to 182.3MB then be 178.3MB again.

it still seems to have something to do with "CoUnitialize" because If I uninstall Win32 Decode Img Stream.vi this problem won't occur.

Edited June 12, 2022 by alvise

[LogMAN]

20 hours ago, alvise said:

When I run the "Preview Demo" example built with C# that comes with the SDK, it seems to increase memory usage, is this normal?

22 hours ago, alvise said:

When running -VI, memory usage sometimes goes up and down. For example: 178.3MB to 182.3MB then be 178.3MB again.

In C#/.NET, memory usage can fluctuate because of garbage collection. Objects that aren't used anymore can stay in memory until the garbage collector releases them.

https://docs.microsoft.com/en-us/dotnet/standard/garbage-collection/

This could explain why your memory usage goes up and down.

There is a way to invoke the garbage collector explicitly by calling GC.Collect(). This will cause the garbage collector to release unused objects.