Distributed "SETI" type application

[Phil Duncan]

Hi All,

I am working on a pet project that involves a MASSIVE number of calculations (in the order of 1175445251780800!). I have streamlined the application to prevent running out of memory but the calculation time is still far too excessive. So I am considering some sort of technique to share the computation load across multiple PC's on a network, akin to the SETI or human genome projects.

Has anybody achieved something along those lines with LV and could provide some usefule links?

Much appreciated.

Cheers & Beers

:thumbup: :beer:

[Rolf Kalbermatter]

QUOTE (Phil Duncan @ Mar 28 2008, 01:21 AM)

Hi All,

I am working on a pet project that involves a MASSIVE number of calculations (in the order of 1175445251780800!). I have streamlined the application to prevent running out of memory but the calculation time is still far too excessive. So I am considering some sort of technique to share the computation load across multiple PC's on a network, akin to the SETI or human genome projects.

Has anybody achieved something along those lines with LV and could provide some usefule links?

Much appreciated.

Cheers & Beers

:thumbup: :beer:

I think the distributed architecture is fairly easily sovable with things like VI server or your own little TCP/IP protocol. What will be a challenge is to make your calculation algortihme distributable and decide on a manager component architecture that can handle this properly, and here everything will be very specific to the problem you are trying to solve. So even if someone has done this already it's not likley that you would gain much by looking at it. The main work will be in the part that decides how to distribute the subsets to possible clients and how to integrate the actual results back together.

The actual data transfer to and from the clients could be as simple as one or two VI Server calls and the client itself won't be much more work. And a little TCP/IP server in the manager for the clients to connect to and advertise their availibity can be easily adapted from one of the TCP/IP examples that come with LabVIEW or if you want to do robust TCP/IP go with the template that you can get from www.labuseful.com.

Rolf Kalbermatter

[jbrohan]

This is a very interesting question suggesting perhaps the possibility of writing LabVIEW programs to leave on the 'net for people to execute.

One aspect that might be a step towards a general solution is the ability of LV (7.0 at least) to be run from a memory stick. http://forums.lavag.org/Memory-Stick-Insta...Soft-t8472.html . It may be possible to adjust matters to run it directly from the web, though there are some security issues with this apporach.

The point I am making rather clumsily is that it is not necessary to have the install of the LV runtime as a separate step, leading to a simpler appearing download from teh web and execute.

Hope this helps.

John

[Phil Duncan]

Thank you folks for your input. I have since looked at different approaches but a distributed system is still the best way to tackle this problem. I think I will try rolfk's ideas.

Fortunately (and surprisingly) the algorithms have been configured in such a way that the distribution of processing segments and mashing of results is straight forward.

Thanks again.

Cheers & Beers

:thumbup: :beer: