In 1996 the Scientific Visualization and Image Analysis Facility was founded to support research in radiological imaging reconstruction, analysis, and visualization. The main components were anSGIOnyx 10000 computer equipped with 8 MIPS R10000 CPUs, 2.5GB of memory and an Infinite Reality Graphics Engine with 64MB of texture memory, and aFakeSpace Systems Immersadesk:
Over the next few years the cost of 3D graphics hardware dropped drastically. In 1996 texture memory for the SGI Infinite Reality Engine was around $1000 per MB; today, graphics accelerators with 1GB of memory can be purchased for under $150. As a result 3D medical imaging can be done on commodity workstations and desktops and there is less of a need for a central visualization facility. When the time came to update the facility a decision was made to augment the computation and storage systems, retire the visualization components, and rename it as the Scientific Image Reconstruction and Analysis Facility.
The SIRAF in its current form is a high performance computation (HPC) cluster optimized for the processing of 2- and 3-D medical images from a variety of modalities. The cluster serves primarily as a load balancing cluster in which many independent applications run concurrently rather than acting as a single parallel computer. Thus the computation nodes (CNs) are equipped with multi-core CPUs and large amounts of local memory, while the interconnect between CNs is Gigabit Ethernet instead of a high speed, low latency network such as Infiniband or Myrinet.
There are two types of CNs, "small" nodes with 8 CPU cores and "big" nodes with 32. The 8-core nodes contain two Intel Xeon quad-core CPUs with 16GB of memory, while the 32-core nodes contain eight AMD Opteron quad-core CPUs with 64GB of memory. The 8-core nodes have better single core performance but less memory bandwidth and thus are best suited for computationally intensive, highly parallel problems. The 32-core nodes have greater memory bandwidth to support problems which are better suited to a large shared memory programming model. In addition, the 8-core nodes contain one nVidia 32-bit graphical processing unit (GPU) with 512MB of memory and the 32-core nodes contain two 32-bit GPUs with 1GB of memory each. GPUs can accelerate many integer and floating point computations by a factor of 5-100x.
The CNs share a common cluster filesystem for user accounts and data via the Redhat Global Filesystem (GFS), so there is no need to move or copy files between nodes. The cluster filesystem is physically located on redundant hard disk arrays which connect to each node over Gigabit Ethernet, avoiding the need for a separate storage interconnect. Read speeds approach 200MBps and write speeds 100MBps. The cluster will eventually have over 20TB of storage available.
85% of the computational resouces are reserved for use by the eight projects of the principal investigators who participated in the shared instrument grant. The projects, followed by their project code, are as follows:
- Real-time Computer-Aided Dagnosis for Diagnostic Mammography (CAD-DM)
- CAD for Lung Cancer Screening Using Computed Tomography (CAD-LCT)
- CAD for Breast Tomosynthesis (CAD-BT)
- MR Imaging of Breast and Prostate with High Spectral and Spatial Resolution (MRI-3D)
- Targeted Imaging in Helical Cone-Beam CT (CT-IR)
- Development and Evaluation of Receiver Operator Characteristic Software (ROC)
- Multi-modality CAD in Breast Imaging (CAD-BU)
- Real-time CAD for Diagnosis of Lung Nodules (CAD-LND)
At least 15% of resources are available for use by the general research community. In the event that a project is not using all of its resource shares, those shares are available for use by other projects.
The SIRAF cluster acquisition was funded primarily via an NIH Shared Instrument Grant (1 S10RR021039-01) and is supported by the UCCRC through a grant from the NIH/NCI (P30 CA14599) and by the University of Chicago Dept of. Radiology. In order to maintain support for the facility, investigators who make use of its resources are encouraged to include the following in their publications:
"Partially funding for this work was provided by the NIH S10 RR021039 and P30 CA14599 grants. The contents of this paper are solely the responsibility of the authors and do not necessarily represent the official views of any of the supporting organizations"
Citations to such talks, papers, posters, proceedings, or abstracts would also be most welcomed.
For information about how to request an account on SIRAF, to provide citations, or to ask general questions about the facility please e-mail to:
siraf-admins_at_siraf-login.bsd.uchicago.edu