atokad Posted June 27, 2018 Report Share Posted June 27, 2018 (edited) I'm trying to send images from a camera connected to my cRIO to a Python script running on my computer. I'm storing the image data in a JSON format, and all of the data is being written correctly, but when I go to send it through the TCP functions, my JSON is getting cut-off about a third of the way through. Here is what the LabVIEW code looks like: and my Python script: import socket server = socket.socket(socket.AF_INET, socket.SOCK_STREAM) server.bind(('', 3363)) server.listen(3) while True: conn, addr = server.accept() cmnd = conn.recv(4) # The default size of the command packet is 4 bytes print(cmnd) if 'CAPT' in str(cmnd): # Do the initialization action filename = input('Enter the file name: ' ) filename = filename + '.json' with open(filename, 'wb') as f: # open in binary mode data_len = int.from_bytes(conn.recv(4), 'big') # find out how much data is waiting f.write(conn.recv(data_len)) # collect that much data f.close() print('File has been saved.') server.close() I think the error is coming from having to flatten the image to a string and then having it converted to bytes during the transmission control protocol, but I'm not sure how to fix that or work around the issue output.json Edited June 27, 2018 by atokad Quote Link to comment
Tim_S Posted June 27, 2018 Report Share Posted June 27, 2018 I'm not terribly proficient in Python, but wondering if Python is reacting to a termination character. Quote Link to comment
atokad Posted June 27, 2018 Author Report Share Posted June 27, 2018 Would I need a termination character if I am sending the byte size though? Shouldn't the data_len determine how much data needs to be written Quote Link to comment
drjdpowell Posted June 27, 2018 Report Share Posted June 27, 2018 You seem to be sending CAPTURE... but your receiver is expecting only CAPT... Quote Link to comment
atokad Posted June 27, 2018 Author Report Share Posted June 27, 2018 Even when I change that I am still getting the same output. Quote Link to comment
ShaunR Posted June 27, 2018 Report Share Posted June 27, 2018 Add a delay between the last write and close and make sure you are not terminating (closing) the connection before all bytes have been transmitted. 2 Quote Link to comment
atokad Posted June 27, 2018 Author Report Share Posted June 27, 2018 1 hour ago, ShaunR said: Add a delay between the last write and close and make sure you are not terminating (closing) the connection before all bytes have been transmitted. I added a 4.5 second delay, and that solved my issue!! Thank you so much I've been struggling with this for the last two days Quote Link to comment
smithd Posted June 28, 2018 Report Share Posted June 28, 2018 5 hours ago, ShaunR said: Add a delay between the last write and close and make sure you are not terminating (closing) the connection before all bytes have been transmitted. Today I learned that labview aborts connections immediately even if you dont wire in abort, because... " abort is reserved for future use. " (help) 1 Quote Link to comment
ShaunR Posted June 28, 2018 Report Share Posted June 28, 2018 12 hours ago, atokad said: I added a 4.5 second delay, and that solved my issue!! Thank you so much I've been struggling with this for the last two days Yeah. It's not ideal since it depends on how much data is being sent. A more robust method is to send an ACK back so that a read function (before close) blocks until all bytes have been received. Quote Link to comment
atokad Posted June 28, 2018 Author Report Share Posted June 28, 2018 4 hours ago, ShaunR said: Yeah. It's not ideal since it depends on how much data is being sent. A more robust method is to send an ACK back so that a read function (before close) blocks until all bytes have been received. Can you explain that a bit more? Quote Link to comment
ShaunR Posted June 28, 2018 Report Share Posted June 28, 2018 1 hour ago, atokad said: Can you explain that a bit more? At a very simple level just add a conn.send('ACK') after print('File has been saved.') in your Python code. Then in your LabVIEW code insert a Read.vi before the close. Quote Link to comment
atokad Posted June 28, 2018 Author Report Share Posted June 28, 2018 56 minutes ago, ShaunR said: At a very simple level just add a conn.send('ACK') after print('File has been saved.') in your Python code. Then in your LabVIEW code insert a Read.vi before the close. Oh I get it! Sending a string back ensures all bytes are written to Python before closing the server Quote Link to comment
atokad Posted June 28, 2018 Author Report Share Posted June 28, 2018 (edited) Now I am running into the issue of duplicate images being stored, any suggestions on how to get rid of my race conditions? I think the two main issues are: The loops are being ran at different time intervals, which causes the images to not be captured at the correct times (see: time intervals array) The local variables might not be fast enough to keep up with how quickly the images are being captured There may be some other issues, but those are the only ones I can think of. How would I go about solving/minimizingthese problems Edited June 28, 2018 by atokad Quote Link to comment
ShaunR Posted June 28, 2018 Report Share Posted June 28, 2018 Take a look at the producer/consumer. There is a project template when you create a "new" project. Quote Link to comment
atokad Posted June 29, 2018 Author Report Share Posted June 29, 2018 (edited) 18 hours ago, ShaunR said: Take a look at the producer/consumer. There is a project template when you create a "new" project. Which one would be better for my project figure one or figure two? I guess what I'm asking, is what would be the best way to implement the Producer/Consumer pattern with my current VI. Should I be constantly queuing the images in the producer loop? Also, would my dequeue element be placed inside or outside of my for loop in the consumer loop? Edited June 29, 2018 by atokad Clarification on my confusion Quote Link to comment
atokad Posted June 29, 2018 Author Report Share Posted June 29, 2018 Current code as a Producer/Consumer: Quote Link to comment
ShaunR Posted June 30, 2018 Report Share Posted June 30, 2018 (edited) 17 hours ago, atokad said: Which one would be better for my project figure one or figure two? Try both and see which one fits your project. 17 hours ago, atokad said: I guess what I'm asking, is what would be the best way to implement the Producer/Consumer pattern with my current VI. I think you will find your current VI will be insufficient for what you are trying to achieve. The thing to be aware of with the producer/consumer it that it always ends with the same quandary - what to do if the consumer cannot keep up with the producer? If your current code can keep up, that's fine. If it can't; you need to optimise. If it still can't then you need to send pointers rather than data (kind of an optimisation). If even that doesn't work, you need to decide what data to lose. If you can't lose any, then........ 17 hours ago, atokad said: Should I be constantly queuing the images in the producer loop? Yes. 17 hours ago, atokad said: Also, would my dequeue element be placed inside or outside of my for loop in the consumer loop? As it stands currently, inside. Edited June 30, 2018 by ShaunR Quote Link to comment
atokad Posted July 2, 2018 Author Report Share Posted July 2, 2018 On 6/30/2018 at 3:19 AM, ShaunR said: I think you will find your current VI will be insufficient for what you are trying to achieve Okay since that's the case I am working on a different method, by trying to combine the two examples of a low-level grab, and a low-level sequence. I'm trying to do it with a case structure to determine if the images need to be captured or not, it's kind of rough, but the general idea is here. Would this be a better method for what I'm trying to accomplish? True case: False case: Currently it is without the JSON formatting/TCP send, but that can be added once I get the acquisition correct. Quote Link to comment
ShaunR Posted July 3, 2018 Report Share Posted July 3, 2018 (edited) Personally. I would start by looking at the TCP/IP connection. In your original code; you are opening and closing the connection with every image (or every couple of images).That is inherently slow. I would be looking to keep the connection open during acquisition and trying to stream the data to the receiver. You haven't stated your image size and frame rate but a 640x480 image at 30fps is about 70Mb/s which is achievable even on a 100Mb wired connection. This is what I was hinting at when talking about optimising and that it probably wasn't sufficient in its current form. Additionally, it would also solve your closing prematurely problem. Edited July 3, 2018 by ShaunR Quote Link to comment
smithd Posted July 3, 2018 Report Share Posted July 3, 2018 (edited) Can you explain your goal? Do you want a continuous stream of images to python, or just a sequence? Whats the purpose of the time interval array? Are those actually all different delays or fixed? Is there a reason your camera is plugged into a cRIO rather than your computer with the python script? What is the latency you can permit between acquiring some images and getting them on the python side? If you can plug your camera into the computer I'd just use: https://pypi.org/project/pynivision/ If you need to stream them for whatever reason, and the time intervals are all constant, I'd suggest setting your camera to produce data at exactly that frame rate and stop timing the loop at all (which doesn't do much anyway since you are asking imaq for a buffer number rather than the latest frame). If the time intervals are not constant but can be made to be multipliers of some common factor, do the same thing but use buffer number*N to pull out the frames you care about. If you can handle high latency, which it seems like you can, maybe just write the images to disk and then use webdav or ftp to pull the files off from your host computer? This would be a lot more straightforward to implement. Edited July 4, 2018 by smithd Quote Link to comment
Recommended Posts
Join the conversation
You can post now and register later. If you have an account, sign in now to post with your account.