Sector/Sphere is what the Sloan Digital Sky Survey uses.
Instead of supplying map and reduce routines, you implement generic "user defined functions". This gives you some more flexibility about how the work is handled, though if you want to just implement map and reduce UDFs, it supposedly gets better performance than Hadoop.
It's also designed to support distributing work over WANs. I think Hadoop really wants every compute node to be on the same LAN.
>I think Hadoop really wants every compute node to be on the same LAN.
Fucking a-right it does. You should see the labyrinthine depths people descend to in order to scale Hadoop. Sub-clusters of sub-clusters, rack-local clusters, Zookeeper nodes all over the place.
Instead of supplying map and reduce routines, you implement generic "user defined functions". This gives you some more flexibility about how the work is handled, though if you want to just implement map and reduce UDFs, it supposedly gets better performance than Hadoop.
It's also designed to support distributing work over WANs. I think Hadoop really wants every compute node to be on the same LAN.