May Gritty Award Winners: The High Performance Computing Team!

The HPC team, down two staff, performed a large upgrade of the High Performance Computing (HPC) environment. Over $1.7 million in new hardware was purchased, including new HPC CPU nodes, new GPU nodes, and over 2.5 Petabytes of storage. The project has significantly enhanced UMBC’s capacity to support high-performance computing (HPC), providing a more secure and robust environment for interdisciplinary and inter-institutional collaborations.

Outstanding Contributions

This project would not have been possible without the DoIT Research Computing and Unix teams! Thank you to the following folks for their contributions:

Roy Prouty, Assistant Director for Research Computing. Provided background support for all installation, configuration, and support tasks.

Greg Ballantine, Research Computing Systems Administrator. Greg was hired a few months into the project, but quickly got up-to-speed on HPC administration and development. Greg also took over the storage cluster deployment and has worked to integrate the storage cluster with the computing cluster.

Andy Leeds, Coordinator of Research Computing. Helped to layout the computing cluster internal networks and deploy the first layer of the storage cluster.

A special, wonderful thank you to the DoIT Student Employees from the Unix Group, who did much of the heavy (server) lifting and spent many shifts learning the ins and outs of HPC system administration.

Unix Students, left to right: Danielle Esposito, Max Breitmeyer, Phil Henry, Beamlak Bekele

Danielle Esposito, Information Systems. DoIT Student Staff. Helped with physical and software installation and worked to establish baseline usage statistics.

Max Breitmeyer, Computer Science. DoIT GA. Helped with physical and software installation, worked to configure the computing cluster management systems, and developed tutorials and documentation for new users.

Phil Henry, Computer Science. DoIT Student Staff. Helped with physical installation and worked to field tickets relating to the new cluster.

Beamlak Bekele, Computer Science. DoIT Student Staff. Helped with physical and software installation as well as serving as the lead user support student staff member.

Deployment

DoIT’s Research Computing team has successfully deployed over 60 new CPU and GPU nodes, along with integrating the high-performance computing environment with the Retriever Research Storage system (RRStor). The infrastructure enhancements support research activities ranging from artificial intelligence development to atmospheric science analysis. 

This investment, totaling over $2 million, modernizes our research computing infrastructure by replacing aging hardware and introducing major updates to the management and security to the cluster. These updates will aid researchers in meeting security requirements in research grants and improve the capabilities and access to high-performance computing needs. 

Key Achievements

• 51 new CPU nodes on chip: 13 high-memory and 38 regular-memory nodes, providing a total of 3264 CPU cores and 32TB of RAM. 

• 12 new GPU nodes on chip: 10 LS40S nodes and two H100 nodes, adding 320 CPU cores and 44 GPU Cards.

• Retriever Research Storage System (RRStor): Added 2.5 PetaBytes of storage, leveraging the Ceph distributed file system to enhance data mobility for research workflows, as part of the NSF Grant awarded to DoIT last year.

• Modernized software environment: Transition to Red Hat Enterprise Linux 9 (RHEL9), improving system reliability, security, and compatibility with modern software stacks.

• Redundant head nodes: Implemented across clusters to eliminate single points of failure, ensuring a more reliable and secure system.

• First HPC Bootcamp: Delivered on November 12, 2024, as part of the NSF SCIPE grant, providing hands-on GPU training for over 20 participants from iHARP, IMET, and UMCES.

The project has significantly enhanced UMBC’s capacity to support high-performance computing (HPC), providing a more secure and robust environment for interdisciplinary and inter-institutional collaborations.

Future Plans

• The DoIT Research Computing team will continue to monitor and support the new systems post-deployment.

• Address optimization needs as researchers test and rebuild software stacks for the new hardware environment.

• Expand programming opportunities with additional HPC Bootcamps to train new and experienced users in maximizing the HPC environment.

A special thanks to Vandana Janeja, iHARP Project Lead, and Sai Vikas Amaraneni, iHARP Graduate Student, for their support throughout the process. UMBC Faculty Contributors have played a significant role in the initial specifications, testing, and providing insight along the way, as did the NSF SCIPE Grant Team and the entire UMBC Research Community, including iHARP, IMET, andUMCES.

We would like to extend our gratitude to everyone who contributed to this project's success!

For all of your exceptional efforts and commitment to excellence, we are proud to present you all with the May DoIT Gritty Award. Congratulations! 

Thank you, Roy, Greg, and Andy, for your dedication to research computing!

The DoIT Community & The DoIT Gritty Committee

Erica D’Eramo

Ally Hepp

Carlos McKinney

David Toothe

Dondre Hatef

Kashka Donaldson

Mariann Hawken 

Nick Beech

Tony Finneran

Roy Prouty (recused himself from the meeting and voting)