Revolution Computing ParallelR, parallelizing R

G

garchbrooks

Has anyone experimented with their stuff? As an example, say I'm fitting an ARMA model and I'm trying to assess which fit works better than others, but I want to do a bruteforce n^2 (i.e., p from 1 to n, q from 1 to n) fit until I get a decent model and I want to parallelize this, ... how would I do this without spending a fortune?

If I could do this over multiple processors, even, or across multiple machines. Does anyone out there have a free solution? The current solution I have involves running commands over ssh in a python script which invokes R and has it run and write the results somewhere where I then reap the results, but I'd like something more integrated that doesn't require me to launch a full-blown R process across the entire cluster to do one calculation.

FWIW, I'm using Yellow Dog Linux and Ubuntu.
 
Quote from garchbrooks:

Has anyone experimented with their stuff? As an example, say I'm fitting an ARMA model and I'm trying to assess which fit works better than others, but I want to do a bruteforce n^2 (i.e., p from 1 to n, q from 1 to n) fit until I get a decent model and I want to parallelize this, ... how would I do this without spending a fortune?

If I could do this over multiple processors, even, or across multiple machines. Does anyone out there have a free solution? The current solution I have involves running commands over ssh in a python script which invokes R and has it run and write the results somewhere where I then reap the results, but I'd like something more integrated that doesn't require me to launch a full-blown R process across the entire cluster to do one calculation.

FWIW, I'm using Yellow Dog Linux and Ubuntu.
More...

libgretl - http://gretl.sourceforge.net/API/
OpenMP - http://en.wikipedia.org/wiki/OpenMP
MPI - http://en.wikipedia.org/wiki/Message_Passing_Interface

I'd just go the C/C++ route, use libgretl, and then OpenMP or MPI. There's also Python MPI bindings, so you'd just have to write a wrapper for libgretl, I think there's someone working on a SWIG wrapper for it.
 
Quote from loltrader:

libgretl - http://gretl.sourceforge.net/API/
OpenMP - http://en.wikipedia.org/wiki/OpenMP
MPI - http://en.wikipedia.org/wiki/Message_Passing_Interface

I'd just go the C/C++ route, use libgretl, and then OpenMP or MPI. There's also Python MPI bindings, so you'd just have to write a wrapper for libgretl, I think there's someone working on a SWIG wrapper for it.
More...

Holy crap, I didn't even know about libgretl! That is freaking amazing. I'd hug you if you were here next to me (in friendly manner non-suggestive of any underlying sexual motives, of course.) I've been lurking on this forum for some time, and when someone asked a similar question a while back, no one answered -- aside from some horrible templatized class for regression that happened to exist in quantlib. You sir, have truly made my day.

Still though, assuming there is a huge learning curve for both packages (gretl+openMP or MPI), I would like to know the answer to the original question. I am familiar with R and invested a lot of time into learning it, writing my own packages, etc. It's just that the drawbacks in R are so cleanly addressed by Revolution Computing's package, although their costs are too prohibitive for me at this point in time.
 
You must log in or register to reply here.
Back
Top