Thousand+ core cluster of Raspberry Pi

nitro

nitro

I am seriously considering putting together a large number of Raspberry Pi computers together in a cluster. At $5 a piece, this seems like a no brainer.

http://www.engadget.com/2015/11/26/raspberry-pi-zero/

I am not sure if the RPi supports network boot, PXE-booting, but sites hint that you can, e.g.,

https://www.reddit.com/r/raspberry_...it_possible_to_have_a_raspberry_pi_boot_over/

To me this is imperative, since you don't want to install a thousands OS on sd cards. Not only is it more expensive, it is much more error prone to say nothing that you woud have to update each OS seperately. PXE-booting is the single most important thing the inexpensive Pis' have to support in order to make them cluster-friendly. The other "must" have is IMPI so as to be able to remotely access the machine at the console. I don't know if those two "must have" would drive up the cost of the Pi-Zero too much.

Then, install something like http://lubuntu.net/

Finally, it would be awesome if someone sold a 1U case that simply housed say 64 or 128 Pi-Zero per 1U (36 or 42 inch depth). I don't know of anyone that has done this, probably because there is no market for it.

There are other low cost low power options on the horizon, but the Pi at $5 is intriguing.
 
Last edited:
nitro said:
I am not sure if the RPi supports network boot
More...

I'm sure they do ....... otherwise that would be a serious limitation. I think your project is a popular one for the Raspberry and there must be YouTube videos on it.



:)
 
You can get a free Pi-Zero if you get a magazine

freepi.jpg

https://www.raspberrypi.org/magpi/issues/40/
 
nitro said:
I am seriously considering putting together a large number of Raspberry Pi computers together in a cluster. At $5 a piece, this seems like a no brainer.

http://www.engadget.com/2015/11/26/raspberry-pi-zero/

I am not sure if the RPi supports network boot, PXE-booting, but sites hint that you can, e.g.,

https://www.reddit.com/r/raspberry_...it_possible_to_have_a_raspberry_pi_boot_over/

To me this is imperative, since you don't want to install a thousands OS on sd cards. Not only is it more expensive, it is much more error prone to say nothing that you woud have to update each OS seperately. PXE-booting is the single most important thing the inexpensive Pis' have to support in order to make them cluster-friendly. The other "must" have is IMPI so as to be able to remotely access the machine at the console. I don't know if those two "must have" would drive up the cost of the Pi-Zero too much.

Then, install something like http://lubuntu.net/

Finally, it would be awesome if someone sold a 1U case that simply housed say 64 or 128 Pi-Zero per 1U (36 or 42 inch depth). I don't know of anyone that has done this, probably because there is no market for it.

There are other low cost low power options on the horizon, but the Pi at $5 is intriguing.
More...


I am all for creative case ideas. I actually like to surf the net for these sites. The best so far:

a) Hollowed out AT case that doubles as a PC and coffee brewer
b) A bitcoin mining farm with milk crates.

Imagine an in-wall post office mailbox system with individual locks.
Or a fancy toolbox tray system/matchbox collection briefcase for a portable cluster.
Also, think about retail display cases behind the register.

Please tell us more about the potential applications.
 
Last edited:
It doesn't support networking or standard USB ports... With only a 1 GHz ARM and 512 MB RAM and no networking or USB, I'd be hard-pressed to find any practical use for 1, let alone 1000 of them.
 
This is fun:
http://www.southampton.ac.uk/~sjc/raspberrypi/

I do wonder if this is really a cost effective thing to do (though fun no doubt). Think back to the early days of google, I wonder if a better approach might be to just source loads of old computers (you can pick them up for peanuts). Rip them out of the cases and rack them up in an old filing cabinet (great for distributing heat). Then install linux and blah blah blah. I reckon you could get much more raw processing power for your money.

If you have the money http://cluster.engineering/ubuntu-orange-box

GAT
 
Even with adding the USB to ethernet dongle, you aren't even taking into account the overhead a USB based ethernet solution would have on that under-powered board. The performance hit would be massive in most high-bandwidth cases... USB can be very dependent on the CPU and an under-powered device like the Pi, combined with latency sensitive and high throughput traffic ethernet usage, would make this very noticeable.

Also, you can get better performance out of a single entry-level desktop than you would out of multiple Pi Zeros. Take a moment to understand what workloads could benefit from multi-node architectures.. and even then, consider the performance limits of a Pi Zero. And NO, just adding more Zeros to the cluster won't make up for it.

As a lab or tool to learn how to build clusters, a bunch of Pi's might fit.. but outside of a learning tool there's nearly no practical reason to use Pi's for something trading related (if that's what you wanted to eventually use the cluster for,) over regular commodity hardware.
 
You must log in or register to reply here.
Back
Top