Illumina/Solexa pipeline w/ Grid Engine
From the trenches @ BioTeam …
Illumina (www.illumina.com) is one of a few companies involved in the “next generation” DNA Sequencing race. Each company has has technology that wildly decreases the cost and time involved in large scale genome analysis. Everyone in this space wants to be the first company to produce a box capable of cranking out a human genome for $10K or less.
These lab instruments produce terabyte volumes of data per experimental run, and thus often need to get hooked into complex IT infrastructures (this is what pays my bills …)
This screenshot shows the end result of integrating the instrument data analysis pipline with Grid Engine software running on a midsized linux cluster.
In this test I have the software using 32 cores on 6 servers for the run and I’m timing it against the same analysis done manually on a single server.
Popularity: 28% [?]
Instead of spending 20 minutes to format an ext3 volume every time you need one, you could just snapshot a ...
Let me add a few remarks about Ext3 as well.
The reason we usually recommend Ext3 instead of XFS is because ...
I'm using these slides to tune SGE 6.2 @ European Space Agency, thanks a lot! 
Great post.
Can you also try our native scale out NFS protocol? Next 3.1 release of Gluster will include it ...
So far my mental architecture is shaping up to be something like this:
Use the ephemeral disks for scratch space and ...
This shows that on-demand HPC still has a ways to go, it is very unfortunate that only ext3 is supported ...
We have had sge62u3 working fine in a 10.5 cluster using cron to startup the sge_execd process on clients. Trying ...
Instead of increasing your disk image size, you could have decreased the memory by writing a very short C program ...