High performance computing for research
The University of Newcastle has a new high performance computing (HPC) system available to staff and research higher degree students. This Research Compute Grid (RCG) is a University funded facility focused on providing computing capacity for Research. It can be used to undertake computer modelling, data analysis or be used as a platform to test and develop computing code prior to running it on National facilities such as the National Computational Infrastructure (NCI) based out of Canberra.
The RCG is based on DELL Servers running Redhat Linux that are combined using the PBS Professional Cluster Management system. It has 648 CPU cores with over 3.5 Terabytes of RAM along with specialised GPU capacity consisting of NVIDIA and Intel Xeon Phi (MIC) processors. This provides approximately 20 Teraflops of processing capacity to University of Newcastle Researchers.
Compilers for Fortran, C, C++ are provided along with scripting languages such as python and pearl. A plethora of Grid aware applications are also available (Refer Table 2 - RCG Applications)
The Linux based grid is best suited to non-interactive 'batch' jobs where the data is read in from a file and results are written to an output file. Interactive jobs are usually run one at a time and may have to wait until a free grid node is available.
The current grid configuration (Refer Table 1 – RCG hardware configuration) consists of a grid head node that maintains the queues, a login node on which jobs are compiled/test run/submitted to the grid and a number of execute nodes on which your program is executed. The execute nodes are configured identically and there is no way to tell which node will be selected by the grid. Additional nodes with specialist hardware also form part of the grid.
Table 1: RCG hardware configuration
|Grid Head||Grid Storage System. Provides storage for all compute nodes within the grid.|
DELL R720XD Rack Server with:
|Login Node (Compile and Submit)|
Provides a system to compile code using one of the available compliers (Fortran, C or C++) and the ability to submit compute jobs to the Research Compute Grid(Refer to Table 2 for a list of applications available on the RCG)
DELL R715 Rack Server with:
|Traditional CPU compute Nodes (Total of 8 Servers)||Provides traditional CPU processing of GRID jobs.|
DELL Chassis with 8 x M915 Blade Servers. Each server has:
Nvidia Kepler GPU Execute Nodes(Total of 2 Servers)
|Provides GPU processing using CUDA code.|
Two DELL R720 rack Servers. Each server has:
Intel Xeon Phi GPU (MIC) Execute Nodes(Total of 2 servers)
|Provides GPU processing using the MIC Intel GPU cards.|
Two DELL R720 rack Servers. Each server has:
Large Memory Execute Node(Total of 1 Server)
|Provides traditional CPU processing of GRID jobs which require a larger memory footprint in a single server (512 GB RAM)|
DELL R815 Rack Server with:
Further expansion of the RCG is underway which will provide an additional 512 Cores, 2 terabytes of RAM and a Microsoft Windows based High Performance computing environment.
The RCG is managed by the Academic & Research Computing Support team (ARCS) within IT Services. If you would like to run jobs on the University's RCG please contact the 17triplezero IT Service Desk on 4921 7000 and request access to the Research Compute Grid (RCG).
Table 2: Research Compute Grid Applications
|Abyss||Pair Sequence Assembler||Yes|
|ANSYS||ANSYS CFX Solver||Upon request|
|Autodock||Molecule Docking Tool||Yes|
|Bedtools||Chromosome Overlap Tool||Yes|
|Boost||C++ Scientific Libraries||Yes|
|Blast||Basic Local Alignment Search Tool||Yes|
|Bowtie||DNA Sequence Indexer||Yes|
|Cplex||Linear Equation Solver||Yes|
|Cufflinks||RNA Sequence Analysis Tool||Yes|
|Fluent||Fluid Dynamics||Upon request|
|Gromacs||Molecular dynamics package||Yes|
|Gsnap||Genome Mapping Alignment||Yes|
|LAMMPS||Molecular Dynamics Simulator||Yes|
|MPICH||MPI programming library||Yes|
|NAMD||Scalable Molecular Dynamics||Yes|
|NetCDF||Network Common Data Forum Libraries||Yes|
|Numpy||Numerical Python Tools||Yes|
|Plink||Genotype Phenotype Analysis||Yes|
|PennCNV||Copy Number Variation Detection||Yes|
|PyCUDA||Python bindings for CUDA||Yes|
|OpenMPI||MPI Programming library||Yes|
|Samtools||Nucleotide Sequence Toolbox||Yes|
|Scipy||Scientific Python Tools||Yes|
|Tophat||RNA Sequence Mapper||Yes|
|Velvet||DNA Sequence Assembler||Yes|
The University of Newcastle provides a Microsoft compute cluster that is available for staff/students wishing to run jobs.
Other software may be installed after consultation with ARCS staff.
|Hardware:||Intel Xeon 2.33GHz, 8GB RAM, Total 300GB storage of which 200GB is available as shared space for users.|
|Operating System:||Windows Server 2008 – HPC Edition|
|Software Installed:||MS HPC Pack, MS Visual Studio.NET 2005|
|Compute Nodes (8 nodes)|
|Hardware:||Intel Xeon 2.00GHz, 16GB RAM, 160GB hard drives|
|Software Installed:||2008b, R, Magma|
Accessing the Cluster
SCIT xGrid Clusters
The School of Design, Communication and IT manages two separate adhoc clustering systems, meaning that clients join the cluster when they are free or otherwise instructed to. One cluster runs Apple's xGrid Technology, the other a proprietary animation rendering cluster technology.
Although the xGrid cluster can compute a range of programmed mathematical computations/requests, the School uses it for image processing when rendering and compressing video footage into required formats for editing and output. The most power the cluster has registered is 313 processors totalling 751GHz of processing power, and 592GB RAM.
The second cluster works only with the School's chosen animation software, Cinema 4D, and renders animation output into a number of different formats. The most nodes we have had attached to this cluster to date is 121. To give a real world example from our own use, this cluster has output one project in less than 6 hours which, by comparison, took 16 days to process on one machine.
If you have any questions about the above information, please contact ARCS.