CSIRO bets on Beowulf cluster

Life science researchers at CSIRO will soon get access to a powerful new computing cluster as the nation's largest scientific organisation prepares for the February launch of a dedicated bioinformatics supercomputer.

Supported by a CSIRO investment of more than $1 million upfront and an estimated $300,000 annually thereafter, the new system will utilise 'Beowulf' technology -- a method for linking large numbers of standard PCs into a single computing cluster capable of processing large volumes of data at once.

CSIRO's cluster, which will be housed within the organisation's Plant Industry division, will be made available to all of the approximately 900 life sciences researchers within CSIRO's Life Sciences division (of which Plant Industry is a subdivision) and will be managed by a committee with members taken from each subdivision.

Specifications of the systems to be used aren't clear yet; CSIRO is currently evaluating Intel against AMD processors and will soon issue an RFP for suppliers interested in supplying what will eventually be a cluster of 70 dual-processor PCs, with each processor probably running at 2GHz or faster.

In line with Beowulf's design, the systems will run the Linux operating system, a flexible and customisable Unix derivative that's been popular in similar projects. Beowulf is an ongoing, collaborative research effort promoted worldwide by The Beowulf Project, which has successfully documented the use of Linux to build powerful computing clusters, primarily in academia.

The cluster's aggregate computing power will be able to be put to virtually any computational task, points out Dr Liz Dennis, program leader for genomics and plant development with CSIRO Plant Industry. "If you divide up your tasks, you can get the boxes to work independently and put the results together," she explains. "It lends itself particularly well to homology searches [for example, analysing microarray data produced when running a DNA sequence against a database of known DNA patterns]. And it is lower cost."

Dennis says the cluster will be applied to a broad range of uses. From her perspective within Plant Industry, she envisions it will be invaluable in facilitating DNA analysis that can improve understanding of plant characteristics such as drought and pest resistance. CSIRO Plant Industry works with all types of plants including rice, cotton, grapes, eucalyptus, wheat, and others.

CSIRO's commercially led decision to invest in Beowulf technology is a change from the technology's normal role as a novelty for the interest of researchers. Celera, the high-profile company behind the recent human gene sequencing effort, is one of the best-known users of Beowulf clusters, but within Australia most Beowulf work has been limited to academia.

Investment in a Beowulf cluster, rather than spending far more on a conventional niche-market supercomputer, will deliver much better price/performance than proprietary systems might. This reflects the global push towards commodity computing platforms, which are being extended to high-end applications through innovative use of technologies like Beowulf.

But Dennis asserts that CSIRO isn't buying into Beowulf just to be a pioneer. "It's not a bioinformatics centre per se," she says. "It's going to be a facility for us to be able to do our work. It will mean that we can really make sense of our results, and work towards finding genes that are going to be useful in agriculture."