Renting a Supercomputer

Quote from aphexcoil:

To the "Computer Cluster" crew and other CS majors:

Is there a rough equation that will tell me (assuming all of the computers have equal processing power) how much faster than X (X being the processing power of one computer) a collection of Y computers would give me?

Example

1X (one computer) = 1Y

2X (two computers) = 1.6Y?

I know you cannot double your power with two computers since their is overhead in negotiation between the two, but perhaps there is a rough formula involved.

I am interested in knowing how much power I will get from a cluster of (10) 3.0 Ghz Pentium IV.
More...

Hmm, look at it this way:

386 < 486 < pentium < pentium II < etc.

Think of this as your base hardware logic. Rather than determining what POWER LEVEL you'll achieve, determine an outline of the tasks at hand. The beauty of this cluster concept is that you can assign tasks to each node/computer thereby increasing the speed and abilities of the different aspects of your task. Learn to think, multi-dimensional and multi-layered for your simultaneous multi-tasking environment. :)
 
Quote from aphexcoil:

To the "Computer Cluster" crew and other CS majors:

Is there a rough equation that will tell me (assuming all of the computers have equal processing power) how much faster than X (X being the processing power of one computer) a collection of Y computers would give me?

Example

1X (one computer) = 1Y

2X (two computers) = 1.6Y?

I know you cannot double your power with two computers since their is overhead in negotiation between the two, but perhaps there is a rough formula involved.

I am interested in knowing how much power I will get from a cluster of (10) 3.0 Ghz Pentium IV.
More...

Depends on the effiency of your code.

That is more important than the hardware with the exception of the network speed/latency ...
 
Quote from canyonman00:



Hmm, look at it this way:

386 < 486 < pentium < pentium II < etc.

Think of this as your base hardware logic. Rather than determining what POWER LEVEL you'll achieve, determine an outline of the tasks at hand. The beauty of this cluster concept is that you can assign tasks to each node/computer thereby increasing the speed and abilities of the different aspects of your task. Learn to think, multi-dimensional and multi-layered for your simultaneous multi-tasking environment. :)
More...


That is interesting. Well, let's say I ask a computer to find a list of all prime numbers. So I come up with a POS algorithm. I want to distribute this task to 10 computers. Since I have no way of splitting up this algoritm into multiple ones, the best I can come up with is to send a group of numbers to a computer and have each computer do a batch and send them back to the main computer.

The server computer acts as the batch giver and when .. let's say computer 8 has just finished finding out its batch of prime numbers, the server sends computer 8 a new batch to check out. All computers are sending back their batches to the server, which is keeping track of distribution of batches and keeping a database of all calculated information.

I am an Econometrics major with a minor in mathematics -- I am not a CS major but program on the side, so I don't know the deepest details.
 
Yeah.... performance gains against a farm of number
crunchers is going to be tightly coupled to
the serial/lack of serial nature of the computation at hand.

For example... if what your crunching can be done
within the context of a SINGLE day of ES data, then
your system will scale almost infinitely.

1000 computers (all identical), will perform this task
nearly 1000 times faster than a single computer.

HOWEVER... if your calculations are dependent on
ALL previous data (the worst case), your seriously
screwed and will have to be far more creative in how
you break up the processing so it can be done in parallel.

Alphe, you better write a cool ass screen saver like
the Seti one, and get 10,000 home PC's to crunch the
data for you :D It would be FREE this way.

Or you could write a virus, and mail it to 1,000,000 idiots,
which then downloads batches of data, crunches it, and
returns it to you. All without being caught :D

peace

axeman
 
As I understand clusters are great for batch program number-crunching. Are they also great for realtime calculations, e.g. analysing the financial markets or would the latency be a problem here?
I mean a 'do-it-yourself-at home-cluster', based on Pentium computers, linux , and ethernet.
 
Not an issue unless your calculations are so intensive
they take so long that the entry opportunity has already
passed.

I at times have my computer watch about 500 stocks
in parallel and doing computations on the fly.

Its a 2.0 ghz P4, and bandwidth, not CPU is my bottle neck.

Assuming I had the bandwidth, I could easily feed this
to a cluster and watch every stock, tick by tick.


peace

axeman


Quote from Ninja:

As I understand clusters are great for batch program number-crunching. Are they also great for realtime calculations, e.g. analysing the financial markets or would the latency be a problem here?
More...
 
Quote from axeman:

Not an issue unless your calculations are so intensive
they take so long that the entry opportunity has already
passed.

I at times have my computer watch about 500 stocks
in parallel and doing computations on the fly.

Its a 2.0 ghz P4, and bandwidth, not CPU is my bottle neck.

Assuming I had the bandwidth, I could easily feed this
to a cluster and watch every stock, tick by tick.


peace

axeman
More...

Yes thats what I mean. T1 line and tick by tick data for ALL stocks and options and then analyse it on a small cluster (3-5 computers). Could this system be fast enough to give me the results in a timeframe of tens of seconds?
 
Well it would seem that if everyone goes from human trading to computer trading, then it really just matters who has:

a) The fastest bandwidth
b) Access to the most current quotes
c) The better algorythm / program
d) The CPU power to run it


Let the games begin.

Axeman,

Great ideas!!!

Also, there might be a way to tap into the routers upstream and intercept the data-transmissions and see who is buying and selling before the data got to the exchanges.

 
You must log in or register to reply here.
Back
Top