Sending image data from LabVIEW to Python via TCP

[atokad]

I'm trying to send images from a camera connected to my cRIO to a Python script running on my computer. I'm storing the image data in a JSON format, and all of the data is being written correctly, but when I go to send it through the TCP functions, my JSON is getting cut-off about a third of the way through. Here is what the LabVIEW code looks like:

and my Python script:

import socket

server = socket.socket(socket.AF_INET, socket.SOCK_STREAM)
server.bind(('', 3363))
server.listen(3)

while True:
    conn, addr = server.accept()
    cmnd = conn.recv(4)  # The default size of the command packet is 4 bytes
    print(cmnd)
    if 'CAPT' in str(cmnd):
        # Do the initialization action
        filename = input('Enter the file name: ' )
        filename = filename + '.json'
        with open(filename, 'wb') as f:  # open in binary mode
            data_len = int.from_bytes(conn.recv(4), 'big')  # find out how much data is waiting
            f.write(conn.recv(data_len))  # collect that much data
            f.close()
            print('File has been saved.')

server.close()

I think the error is coming from having to flatten the image to a string and then having it converted to bytes during the transmission control protocol, but I'm not sure how to fix that or work around the issue

output.json

Edited June 27, 2018 by atokad

[Tim_S]

I'm not terribly proficient in Python, but wondering if Python is reacting to a termination character.

[atokad]

Would I need a termination character if I am sending the byte size though? Shouldn't the data_len determine how much data needs to be written

[drjdpowell]

You seem to be sending CAPTURE... but your receiver is expecting only CAPT...

[atokad]

Even when I change that I am still getting the same output.

[ShaunR]

Add a delay between the last write and close and make sure you are not terminating (closing) the connection before all bytes have been transmitted.

[atokad]

1 hour ago, ShaunR said:

Add a delay between the last write and close and make sure you are not terminating (closing) the connection before all bytes have been transmitted.

I added a 4.5 second delay, and that solved my issue!! Thank you so much I've been struggling with this for the last two days

[smithd]

5 hours ago, ShaunR said:

Add a delay between the last write and close and make sure you are not terminating (closing) the connection before all bytes have been transmitted.

Today I learned that labview aborts connections immediately even if you dont wire in abort, because...

" abort is reserved for future use. " (help)

[ShaunR]

12 hours ago, atokad said:

I added a 4.5 second delay, and that solved my issue!! Thank you so much I've been struggling with this for the last two days

Yeah. It's not ideal since it depends on how much data is being sent. A more robust method is to send an ACK back so that a read function (before close) blocks until all bytes have been received.

[atokad]

4 hours ago, ShaunR said:

Yeah. It's not ideal since it depends on how much data is being sent. A more robust method is to send an ACK back so that a read function (before close) blocks until all bytes have been received.

Can you explain that a bit more?

[ShaunR]

1 hour ago, atokad said:

Can you explain that a bit more?

At a very simple level just add a

conn.send('ACK')

after

print('File has been saved.')

in your Python code. Then in your LabVIEW code insert a Read.vi before the close.

[atokad]

56 minutes ago, ShaunR said:

At a very simple level just add a

conn.send('ACK')

after

print('File has been saved.')

in your Python code. Then in your LabVIEW code insert a Read.vi before the close.

Oh I get it! Sending a string back ensures all bytes are written to Python before closing the server 

[atokad]

Now I am running into the issue of duplicate images being stored, any suggestions on how to get rid of my race conditions?

I think the two main issues are:

1. The loops are being ran at different time intervals, which causes the images to not be captured at the correct times (see: time intervals array)
2. The local variables might not be fast enough to keep up with how quickly the images are being captured

There may be some other issues, but those are the only ones I can think of. How would I go about solving/minimizingthese problems

Edited June 28, 2018 by atokad

[ShaunR]

Take a look at the producer/consumer. There is a project template when you create a "new" project.

[atokad]

18 hours ago, ShaunR said:

Take a look at the producer/consumer. There is a project template when you create a "new" project.

Which one would be better for my project figure one or figure two?

 

I guess what I'm asking, is what would be the best way to implement the Producer/Consumer pattern with my current VI. Should I be constantly queuing the images in the producer loop? Also, would my dequeue element be placed inside or outside of my for loop in the consumer loop?

Edited June 29, 2018 by atokad
Clarification on my confusion

[atokad]

Current code as a Producer/Consumer:

[ShaunR]

17 hours ago, atokad said:

Which one would be better for my project figure one or figure two?

Try both and see which one fits your project.

17 hours ago, atokad said:

I guess what I'm asking, is what would be the best way to implement the Producer/Consumer pattern with my current VI.

I think you will find your current VI will be insufficient for what you are trying to achieve. The thing to be aware of with the producer/consumer it that it always ends with the same quandary - what to do if the consumer cannot keep up with the producer?

If your current code can keep up, that's fine. If it can't; you need to optimise. If it still can't then you need to send pointers rather than data (kind of an optimisation). If even that doesn't work, you need to decide what data to lose. If you can't lose any, then........

17 hours ago, atokad said:

Should I be constantly queuing the images in the producer loop?

Yes.

17 hours ago, atokad said:

Also, would my dequeue element be placed inside or outside of my for loop in the consumer loop?

As it stands currently, inside.

Edited June 30, 2018 by ShaunR

[atokad]

On 6/30/2018 at 3:19 AM, ShaunR said:

I think you will find your current VI will be insufficient for what you are trying to achieve

Okay since that's the case I am working on a different method, by trying to combine the two examples of a low-level grab, and a low-level sequence.

I'm trying to do it with a case structure to determine if the images need to be captured or not, it's kind of rough, but the general idea is here. Would this be a better method for what I'm trying to accomplish?

True case:

False case:

Currently it is without the JSON formatting/TCP send, but that can be added once I get the acquisition correct.

[ShaunR]

 

Personally. I would start by looking at the TCP/IP connection. In your original code; you are opening and closing the connection with every image (or every couple of images).That is inherently slow. I would be looking to keep the connection open during acquisition and trying to stream the data to the receiver. You haven't stated your image size and frame rate but a 640x480 image at 30fps is about 70Mb/s which is achievable even on a 100Mb wired connection. This is what I was hinting at when talking about optimising and that it probably wasn't sufficient in its current form. Additionally, it would also solve your closing prematurely problem.

Edited July 3, 2018 by ShaunR

[smithd]

Can you explain your goal? Do you want a continuous stream of images to python, or just a sequence? Whats the purpose of the time interval array? Are those actually all different delays or fixed? Is there a reason your camera is plugged into a cRIO rather than your computer with the python script? What is the latency you can permit between acquiring some images and getting them on the python side?

If you can plug your camera into the computer I'd just use: https://pypi.org/project/pynivision/

If you need to stream them for whatever reason, and the time intervals are all constant, I'd suggest setting your camera to produce data at exactly that frame rate and stop timing the loop at all (which doesn't do much anyway since you are asking imaq for a buffer number rather than the latest frame). If the time intervals are not constant but can be made to be multipliers of some common factor, do the same thing but use buffer number*N to pull out the frames you care about.

If you can handle high latency, which it seems like you can, maybe just write the images to disk and then use webdav or ftp to pull the files off from your host computer? This would be a lot more straightforward to implement.

Edited July 4, 2018 by smithd