has anybody tried distributed computing for backtesting? I am thinking of NVIDIA GPU

[Corey]

From what I have heard, Jacket and GP-You are pretty easy to work with if you like Matlab.

If you are doing things like Montecarlo, CUDA has a couple examples. I guess the finance world has been dabbling in GPU work for a while now. From what I have heard, the language itself isn't hard to work with, but your program needs to translates well to matrices and GPU related functions. Otherwise, it just won't work well. But I don't have much experience past a couple demos here and there.

On the other hand, distributed can be a bit easier to get up and running with -- especially in 'embarrassingly parallel' applications, like Montecarlo.

Do you have a preferred language you want to work with? I might have a couple resources on distributed computing for you to work with -- even with EC2 (which I have used extensively in my back testing...). Unlike maxdama, I found EC2 to be an excellent utility. Got 15 computers up and running pretty quickly and was able to get an application that was supposed to take months to finish on my PC only a day or two on the cloud...

 

[maxdama]

Corey,

I'd like to know more about how you use EC2. My only problem was that Matlab required some ridiculous licensing to run on multiple computers in the cloud. I'm sure it would be good using a different language. And I just so happen to have been credited $2100 toward AWS by Amazon.

Regards,
Max

 

[Corey]

Quote from maxdama:

Corey,

I'd like to know more about how you use EC2. My only problem was that Matlab required some ridiculous licensing to run on multiple computers in the cloud. I'm sure it would be good using a different language. And I just so happen to have been credited $2100 toward AWS by Amazon.

Regards,
Max

It was one of those 'embarrassingly easy' parallelized problems. Backtesting hundreds of equities over twenty years with about forty parameters. Basically, it was looking at each equity on its own, so I didn't need to do any portfolio management stuff -- which made things easier to distribute.

I used a custom Ruby solution. Pretty much booted the 10-20 EC2 boxes by hand (though, now, I could probably use PoolParty) and downloaded my custom software from my website. Ran the software and using Distributed Ruby (DRb), it was pretty easy to manage. Just wrote a few wrappers here and there to make things easier.

On a side note, how are you enjoying UC Berkeley? Just graduated from Cornell myself ... wishing I were back in college already. How did you get $2100 credit for AWS? I would love to get that...