Simple HTTP Client

[LVBeginner]

Ok I have this simple HTTP server for testing purposes of the LabVIEW server and TCP/IP sockets. Now I can't get the TCP Open Connection function to open a connection. I get error 56 which is timeout error. I have turned TCP/IP on and I have turned the web server on in the LabVIEW options. I have also tried different ports but I can't establish a connection. Can somebody please help me out.

I'm using LabVIEW 8.0 and have attached a screenshot of the .vi.

[Mark Smith]

QUOTE (LVBeginner @ Mar 4 2009, 09:47 AM)

Ok I have this simple HTTP server for testing purposes of the LabVIEW server and TCP/IP sockets. Now I can't get the TCP Open Connection function to open a connection. I get error 56 which is timeout error. I have turned TCP/IP on and I have turned the web server on in the LabVIEW options. I have also tried different ports but I can't establish a connection. Can somebody please help me out.

I'm using LabVIEW 8.0 and have attached a screenshot of the .vi.

TCP Open Connection won't do a domain name lookup for you and resolve the address - put the IP address of the server in the address box and try again

[LVBeginner]

QUOTE (mesmith @ Mar 4 2009, 06:18 PM)

TCP Open Connection won't do a domain name lookup for you and resolve the address - put the IP address of the server in the address box and try again

I tried soing that but I get the same error 56 from LabVIEW. Is there something maybe I forgot to change in the options of LabVIEW?

Thanks for the quick reply

[jdunham]

QUOTE (LVBeginner @ Mar 4 2009, 08:47 AM)

I'm using LabVIEW 8.0 and have attached a screenshot of the .vi.

Well port 3363 is the VI Server port, which is specific to LabVIEW. It's highly unlikely that www.w3.org would be listening on that port. Try port 80.

QUOTE (mesmith @ Mar 4 2009, 09:18 AM)

TCP Open Connection won't do a domain name lookup for you and resolve the address - put the IP address of the server in the address box and try again

Totally untrue.

[LVBeginner]

QUOTE (jdunham @ Mar 4 2009, 06:46 PM)

Well port 3363 is the VI Server port, which is specific to LabVIEW. It's highly unlikely that www.w3.org would be listening on that port. Try port 80.

Totally untrue.

Well I tried that too, still no luck. Does it maybe have to do anything with the windows firewall maybe, even though labview is already an exception.

Thanks

[Mark Smith]

QUOTE (jdunham @ Mar 4 2009, 10:46 AM)

Totally untrue.

Really? I've never tried and couldn't see where a domain name server would be invoked - my bad

[jdunham]

QUOTE (LVBeginner @ Mar 4 2009, 09:55 AM)

Well I tried that too, still no luck. Does it maybe have to do anything with the windows firewall maybe, even though labview is already an exception.

Generally the firewall is for incoming connections, unless your IT department is super-restrictive. Nobody can get any work done if they block port 80, so that is usually not going to be filtered out. On the client side (that's you) there is no ability to filter specific programs out.

Since that's true, the next thing to do is to open a Windows Command Prompt (Start -> Run -> CMD)

1. ping the server to make sure you can resolve the address (of course sometimes servers disable ping, but www.w3.org does not)

2. telnet to the server and type the your message.

I entered "telnet www.w3.org 80" and then I typed in your GET statement, and it spit a bunch of HTML back at me.

3. try the same thing again in LabVIEW. Your program is basically doing the exact same thing as telnet. Be sure to note exactly which VI or function is throwing the error.

Note that you may have to install telnet, since that's part of Microsoft's policy of burying all the really useful stuff or not installing it by default.

QUOTE (LVBeginner @ Mar 4 2009, 08:47 AM)

Now I can't get the TCP Open Connection function to open a connection. I get error 56 which is timeout error.

How are you getting a timeout from TCP Open when you have the timeout = -1 (forever)? It seems more likely that you are connecting, but getting a timeout in TCP Read, since you are asking for 100,000 bytes in Standard mode. You may want to try Immediate mode. The help file has more information about TCP Read and its modes.

[LVBeginner]

QUOTE (jdunham @ Mar 4 2009, 07:28 PM)

Generally the firewall is for incoming connections, unless your IT department is super-restrictive. Nobody can get any work done if they block port 80, so that is usually not going to be filtered out. On the client side (that's you) there is no ability to filter specific programs out.

Since that's true, the next thing to do is to open a Windows Command Prompt (Start -> Run -> CMD)

1. ping the server to make sure you can resolve the address (of course sometimes servers disable ping, but www.w3.org does not)

2. telnet to the server and type the your message.

I entered "telnet www.w3.org 80" and then I typed in your GET statement, and it spit a bunch of HTML back at me.

3. try the same thing again in LabVIEW. Your program is basically doing the exact same thing as telnet. Be sure to note exactly which VI or function is throwing the error.

Note that you may have to install telnet, since that's part of Microsoft's policy of burying all the really useful stuff or not installing it by default.

How are you getting a timeout from TCP Open when you have the timeout = -1 (forever)? It seems more likely that you are connecting, but getting a timeout in TCP Read, since you are asking for 100,000 bytes in Standard mode. You may want to try Immediate mode. The help file has more information about TCP Read and its modes.

I typed in "telnet www.w3.org 80" then it said connecting to www.w3.org, but after a while it said that the connection to the host on port 80 could not be opened...connection failed

[Mark Smith]

OK, so I was wrong the first time , but I think this may be why I've never had any luck with getting the host names to resolve correctly and it could be the problem here

http://forums.ni.com/ni/board/message?boar...essage.id=90988

I can connect to my proxy server using it's host name but I can't connect to anything else because there's evidently no automatic (easy) way to use a proxy server with the LabVIEW TCP/IP tools

I learn something everyday (at least I hope so)

Mark

Edit: Try looking at the configuration of your web browser (assuming it can reach the requested host) and see if it's using a proxy server

[jdunham]

QUOTE (LVBeginner @ Mar 4 2009, 10:55 AM)

I typed in "telnet www.w3.org 80" then it said connecting to www.w3.org, but after a while it said that the connection to the host on port 80 could not be opened...connection failed

Well that's good, you've learned that this is not a LabVIEW problem. Is your web browser able to connect? In your web browser's advanced settings, is there a proxy configuration? Presumably if some of these are non-standard, then you have an IT department which can help you more than I can. Good luck.

[LVBeginner]

QUOTE (jdunham @ Mar 4 2009, 08:03 PM)

Well that's good, you've learned that this is not a LabVIEW problem. Is your web browser able to connect? In your web browser's advanced settings, is there a proxy configuration? Presumably if some of these are non-standard, then you have an IT department which can help you more than I can. Good luck.

Well that's good. Just found out that there is a proxy configuration. So now I will go to my IT department and see what they can do for me.

Thanks for all the help

[jdunham]

QUOTE (LVBeginner @ Mar 4 2009, 11:09 AM)

Well that's good. Just found out that there is a proxy configuration. So now I will go to my IT department and see what they can do for me.

Thanks for all the help

By the way, what are you really trying to do? It seems likely that your proxy config only affects port 80, and no one in their right mind writes a program to implement the HTTP protocol in LabVIEW when there are other free programs which do a far better job of browsing the web.

[LVBeginner]

QUOTE (jdunham @ Mar 4 2009, 08:13 PM)

By the way, what are you really trying to do? It seems likely that your proxy config only affects port 80, and no one in their right mind writes a program to implement the HTTP protocol in LabVIEW when there are other free programs which do a far better job of browsing the web.

I don't want to browse the web. I have to write a program that will allow a user to transfer some data to an off-site server. Eventually I'm hoping to make that all automatic so that it will do that every hour or so. Right now I wrote a program that will ask for users input and make a xml file to a specific format given by the off-site server company. Now I have to send the xml file via the internet to the off-site server. I wanted to test the TCP/IP in a simple http client program before I go and adjust my program.

[jdunham]

QUOTE (LVBeginner @ Mar 4 2009, 11:18 AM)

I don't want to browse the web. I have to write a program that will allow a user to transfer some data to an off-site server.

Yeah as soon as I asked, I thought of a half-dozen things I might want to do programmatically over HTTP. In our own app, we do some of this, and we use the ActiveX Object "WinHTTP", which is part of Windows. It handles proxy servers and lots of other stuff and makes a lot more sense than rolling your own. Unfortunately as a "LVBeginner" you probably didn't want to muck with ActiveX right away, but I think it would end up being a better use of your time.

There is also some .NET equivalent which is totally different but I don't know anything about that. It might be easier to use.

[LVBeginner]

QUOTE (jdunham @ Mar 4 2009, 08:25 PM)

Yeah as soon as I asked, I thought of a half-dozen things I might want to do programmatically over HTTP. In our own app, we do some of this, and we use the ActiveX Object "WinHTTP", which is part of Windows. It handles proxy servers and lots of other stuff and makes a lot more sense than rolling your own. Unfortunately as a "LVBeginner" you probably didn't want to muck with ActiveX right away, but I think it would end up being a better use of your time.

There is also some .NET equivalent which is totally different but I don't know anything about that. It might be easier to use.

You got it as soon as I looked a little bit into ActiveX, I thought "Ohhh hold your horses there LVbeginner". I figured that TCP/IP would be more straight forward, well at least it was while I was reading about it.

Thanks I gues I will look more into the ActiveX portion.

Okay one little question...should I use the WinHTTP or the WinINet feature.

[Rolf Kalbermatter]

QUOTE (mesmith @ Mar 4 2009, 03:02 PM)

OK, so I was wrong the first time , but I think this may be why I've never had any luck with getting the host names to resolve correctly and it could be the problem here

http://forums.ni.com/ni/board/message?boar...essage.id=90988

I can connect to my proxy server using it's host name but I can't connect to anything else because there's evidently no automatic (easy) way to use a proxy server with the LabVIEW TCP/IP tools

That is because a proxy is only meant for HTTP (and possible other similar protocols like FTP sometimes). The LabVIEW TCP/IP nodes access directly the Winsock library and the thing that does for IP what a proxy does for HTTP is called a network router .

So it is not a problem of the LabVIEW TCP functions but simply a question of proper setup of your network. Basically a proxy simply acts as a web server that does hand the HTTP requests through to the end server (that could again be another proxy and so on). The way to handle such a case in LabVIEW is by making a thin layer above the TCP/IP nodes. By the way the OpenG HTTP functions do some proxy server handling. But it seems they have not been packaged yet, however they are on the OpenG Toolkit CVS server repository on sourceforge.

Basically you open a connection to the proxy instead and then you have to parse every HTTP request and prepend the actual end address of the desired server to every relative URL in there.

Rolf Kalbermatter

[Mark Smith]

QUOTE (rolfk @ Mar 18 2009, 02:20 AM)

That is because a proxy is only meant for HTTP (and possible other similar protocols like FTP sometimes). The LabVIEW TCP/IP nodes access directly the Winsock library and the thing that does for IP what a proxy does for HTTP is called a network router .

So it is not a problem of the LabVIEW TCP functions but simply a question of proper setup of your network. Basically a proxy simply acts as a web server that does hand the HTTP requests through to the end server (that could again be another proxy and so on). The way to handle such a case in LabVIEW is by making a thin layer above the TCP/IP nodes. By the way the OpenG HTTP functions do some proxy server handling. But it seems they have not been packaged yet, however they are on the OpenG Toolkit CVS server repository on sourceforge.

Basically you open a connection to the proxy instead and then you have to parse every HTTP request and prepend the actual end address of the desired server to every relative URL in there.

Rolf Kalbermatter

Rolf,

Thanks for the information. Can you give a simple example of how one could create an address string that would get to a site like "www.google.com" through a proxy server called "myproxy.mysite.net"? Can you pass this composite address to the LabVIEW TCP Open Connection and expect it to work?

Thanks,

Mark

[Rolf Kalbermatter]

QUOTE (mesmith @ Mar 18 2009, 08:50 AM)

Rolf,

Thanks for the information. Can you give a simple example of how one could create an address string that would get to a site like "www.google.com" through a proxy server called "myproxy.mysite.net"? Can you pass this composite address to the LabVIEW TCP Open Connection and expect it to work?

Thanks,

Mark

No it is not exactly as you seem to think. Lets take your example:

The proxy address is usually in the form of "myproxy.mysite.net:8080" since it seldom uses the standard HTTP port of 80.

So you parse that address into the DNS name (everything up to the colon) and the port number, using 80 if there is no port number. Now pass this information to TCP Open.

Then you have a HTTP Request of the form "GET /index.html HTTP/1.1". Since the Proxy must know to what server to forward the HTTP request you have to modify this by prepending the server address to the absolute path of the resource. So the command to send to the proxy will be "GET http:/www.google.com/index.html HTTP/1.1".

That's all.

Rolf Kalbermatter

[Mark Smith]

QUOTE (rolfk @ Mar 18 2009, 06:03 AM)

No it is not exactly as you seem to think. Lets take your example:

The proxy address is usually in the form of "myproxy.mysite.net:8080" since it seldom uses the standard HTTP port of 80.

So you parse that address into the DNS name (everything up to the colon) and the port number, using 80 if there is no port number. Now pass this information to TCP Open.

Then you have a HTTP Request of the form "GET /index.html HTTP/1.1". Since the Proxy must know to what server to forward the HTTP request you have to modify this by prepending the server address to the absolute path of the resource. So the command to send to the proxy will be "GET http:/www.google.com/index.html HTTP/1.1".

That's all.

Rolf Kalbermatter

Thanks - I thought I was missing something obvious BTW, I certainly appreciate all of your very knowledgeable advice on all the threads I follow!

Mark

[EHM]

QUOTE (rolfk @ Mar 18 2009, 09:03 PM)

No it is not exactly as you seem to think. Lets take your example:

The proxy address is usually in the form of "myproxy.mysite.net:8080" since it seldom uses the standard HTTP port of 80.

So you parse that address into the DNS name (everything up to the colon) and the port number, using 80 if there is no port number. Now pass this information to TCP Open.

Then you have a HTTP Request of the form "GET /index.html HTTP/1.1". Since the Proxy must know to what server to forward the HTTP request you have to modify this by prepending the server address to the absolute path of the resource. So the command to send to the proxy will be "GET http:/www.google.com/index.html HTTP/1.1".

That's all.

Rolf Kalbermatter

Hi Rolf,

Thank you for your explaination.

However, there is another question which I would like to ask.

I'm doing a HTTP GET Request as follow:

GET http://www.example.com/index.html HTTP/1.1

Host: www.example.com

The proxy server IP is xxx.xxx.xxx.xxx:80.

I pass the Proxy IP as input to "TCP Open Connection" with Port "80".

Whereas the HTTP Get Request as mentioned is passed as input to "TCP Write".

I received error code 404, indicating that "index.html" which I'd requested was not found.

It seems like the request was not directed to the destinated host.

In a normal internet browser, we do not access to this proxy server directly. Instead, we are using a Javascript Auto-Proxy Configuration.

Does it really matter if I'm doing a direct connection to the proxy server in this case?

Thanks,

Mean

[Rolf Kalbermatter]

QUOTE (EHM @ Mar 26 2009, 10:37 PM)

Hi Rolf,

Thank you for your explaination.

However, there is another question which I would like to ask.

I'm doing a HTTP GET Request as follow:

GET http://www.example.com/index.html HTTP/1.1

Host: www.example.com

The proxy server IP is xxx.xxx.xxx.xxx:80.

I pass the Proxy IP as input to "TCP Open Connection" with Port "80".

Whereas the HTTP Get Request as mentioned is passed as input to "TCP Write".

I received error code 404, indicating that "index.html" which I'd requested was not found.

It seems like the request was not directed to the destinated host.

Not sure about your proxy but I did not add the Host: xxxxxxxxxxxx in the past to my proxy requests and that worked. The Host can be added according to the RFC for forward compatibility but I don't think it is required. The absolute URI however seems to be at least for older HTTP versions a requirement. I did however use HTTP/1.0 as version indication so maybe that is why.

And last but not least: are you sure there is an index.html document on the server. It could be index.htm or something entirely different so as a start I would try just the server address http://www.example.com/ without any document path.

Rolf Kalbermatter

[EHM]

QUOTE (rolfk @ Mar 27 2009, 03:34 PM)

Not sure about your proxy but I did not add the Host: xxxxxxxxxxxx in the past to my proxy requests and that worked. The Host can be added according to the RFC for forward compatibility but I don't think it is required. The absolute URI however seems to be at least for older HTTP versions a requirement. I did however use HTTP/1.0 as version indication so maybe that is why.

And last but not least: are you sure there is an index.html document on the server. It could be index.htm or something entirely different so as a start I would try just the server address http://www.example.com/ without any document path.

Rolf Kalbermatter

Thanks for your information.

I've tried to make a HTTP/1.0 request to http://www.example.com/ through the proxy server by accessing directly to the IP address, but it is still not working.

However, "Host" is a necessary field if it is HTTP/1.1 request

Now, I'm trying "DataSocket Read" function by passing in the full URL of the web page.

It seems like it is working..

[EHM]

QUOTE (EHM @ Apr 1 2009, 03:46 PM)

Thanks for your information.

I've tried to make a HTTP/1.0 request to http://www.example.com/ through the proxy server by accessing directly to the IP address, but it is still not working.

However, "Host" is a necessary field if it is HTTP/1.1 request

Now, I'm trying "DataSocket Read" function by passing in the full URL of the web page.

It seems like it is working..

Ok. After testing by using "DataSocket Read" function, all general web pages can be accessed without problem.

However, there is another question here.

What I'm trying to do here is to make a HTTP request to Yahoo! Maps to query for the geocodes of a particular location.

This is an example of url which includes an application id (appid), street, city and state information.

http://local.yahooapis.com/MapsService/V1/...le&state=CA

This link works properly when access using a normal internet browser.

When I passed this url into the input of "DataSocket Read" function, error 42 was returned.

One thing that puzzled me is when I passed in a normal google search URL, it works.

for eg: http://www.google.com.sg/search?hl=en&...mp;aq=f&oq=

The only difference between the two URLs is the existence of the Application ID in the query string.

Appreciate for any advice and comments given.

Thanks.

Mean

[Rolf Kalbermatter]

QUOTE (EHM @ Apr 3 2009, 02:57 AM)

Ok. After testing by using "DataSocket Read" function, all general web pages can be accessed without problem.

However, there is another question here.

What I'm trying to do here is to make a HTTP request to Yahoo! Maps to query for the geocodes of a particular location.

This is an example of url which includes an application id (appid), street, city and state information.

http://local.yahooapis.com/MapsService/V1/...le&state=CA

No problem here with that URL when I use the OpenG HTTP VIs. I'm not using a proxy but this shouldn't really be a problem.

Go get those VIs (there is no HTTP OpenG package yet so you have to get them from the sourceforge CVS repository) and look at them.

Rolf Kalbermatter

[BobHamburger]

QUOTE (EHM @ Apr 3 2009, 02:57 AM)

No problem here with that URL when I use the OpenG HTTP VIs. I'm not using a proxy but this shouldn't really be a problem.

Go get those VIs (there is no HTTP OpenG package yet so you have to get them from the sourceforge CVS repository) and look at them.

Rolf Kalbermatter

Rolf, could you be a little more specific on the location of the HTTP OpenG VI's? I can't find what you mean by the "CVS repository" on SourceForge.