Allocating Host (PC) Memory to do a DMA Transfer

[PJM_labview]

Hi

We have an application where we need to have a custom PCIe board transfer data to the PC using DMA.

We are able to get this to work using NI-VISA Driver Wizard for PXI/PCI board.

The recommended approach is to have VISA allocate memory on the host (PC) and have the PCIe board write to it (as seen below).

While this approach works well, the memory allocated by VISA for this task is quite limited (~ around 1-2MB) and we would like to expand this to tens of MB.

Note: The documentation (and help available on the web) regarding these advanced VISA function (for instance "VISA Move out 32" and "VISA Move In 32") is parse. If someone has some deep knowledge regarding theses, please feel free to share how we could allocate more memory.

Since we are not able to allocate more memory using the VISA function at this time, we investigate doing the same operation using the LabVIEW Memory Manager Functions which allow us to allocate much larger memory block.

Below is the resulting code.

Unfortunately while we can validate that reading and writing to memory using this work well outside the context of DMA transfer, doing a DMA transfer do NOT work (although the board think it did and the computer is not crashing).

We are wondering why this is not working and would welcome any feedback.

Note: the DMA transfer implemented on the board requires contiguous memory for it to be successful. I believe that the LabVIEW Memory Manager Functions do allocate continuous memory, but correct me if I am wrong.

To troubleshoot this, I did allocate memory using the LabVIEW memory manager function and try to read it back using VISA and I got a "wrong offset" error (Note: This test may not be significant)

Another data point; while the documentation for DSNewPtr do not talk about DMA transfer, the one for DSNewAlignedHandle does. Experiment using LV memory manager Handles has not got us anywhere either.

We would welcome any feedback regarding either approach and about the LabVIEW Memory Manager Functions capabilities in that use case.

Thanks in advance.

PJM

Note: We are using LabVIEW 2014 if that matter at all.

 

Edited October 1, 2016 by PJM_labview

[Rolf Kalbermatter]

Well, I'm pretty sure that for DMA transfer you do need a physical memory address.The LabVIEW DSNewPtr() function allocates a chunk on the process heap, which is a virtual memory area for each process. Between this virtual adress space and the actual physical address is the CPU MMU (memory management unit) which translates every address from the virtual memory to the actual physical memory address (and in the case of already cached memory locations actually, this step is skipped and directly translated to the cache memory location). The DMA controller only can transfer between physical memory addresses, which means for DMA transfer all the cache lines currently caching any memory that belongs to the DMA transfer block need to be invalidated.

So you first need to lock the virtual address block (the DMA controller would get pretty panicky if the virtual memory suddenly was moved/paged out) which will also invalidate the cache for any area in that block that is currently cached and retrieve its real physical address. Then you do the DMA transfer on the physical address and afterwards you unlock the memory area again, which also invalidates the physical address you previously got. Incidentially the services you need to access to do physical address translation and locking are all kernel space APIs. VISA comes with its own internal kernel driver, which exports functions for the VISA layer to do these things but performance suffers considerably if you do it this way. The alternative however is to have to write a kernel driver for your specific hardware. Only in the kernel driver do you have direct access to physical memory translation and such things, since these APIs are NOT accessible from the ring 3 user space, normal Windows processes are running in.

And yes, Windows limits the size of blocks you can lock. A locked memory area is a considerable weight on the leg of every OS, and in order to limit the possibility for a kernel driver to drown the OS for good, this limitation is necessary. Otherwise any misbehaving driver could simply DOS the OS by requesting an unreasonable big block to be locked.

Edited October 3, 2016 by rolfk

[PJM_labview]

Thank RolfK.

I was actually hoping that you would chime in on that topic

We were suspecting something around the lines of what you described (Virtual memory address not appropriate for the DMA transfer). We will move forward and find out how to lock the memory.

Thanks again.