In yet another example of HPC, cloud and big data convergence, Adaptive Computing announced today that its cluster and cloud management software Moab is part of a pilot IT infrastructure built to address the needs of Canadian healthcare organizations. The project, called High Performance Computing for Health Sciences, or HPC4Health for short, was formed to help researchers and clinicians leverage HPC and big data to unlock the potential of personalized medicine.
Founding partners include two of the biggest hospitals in Canada, The Hospital for Sick Children (SickKids) and the University Health Network’s (UHN) Princess Margaret Cancer Center. By working together, the collaborators have built a model for a computing infrastructure that balances resources efficiently but also has sufficient processing and storage power to handle complex molecular analyses and medical imaging workloads. The project is starting in Toronto, but the partners share a broader vision is to eventually expand access to the service across Canada.
Launched in July and located in the SickKids’ datacenter, the HPC4Health infrastructure is comprised of 340 SGI compute nodes, 13,024 compute threads, 52.7 terabytes of RAM, 306 terabytes of total local disk space and 4 PB of storage. Each organization has their own dedicated resources that they control plus access to a common shared pool. Moab HPC Suite, Enterprise Edition, version 8.1 was selected for its elastic computing, advanced policies and accounting capabilities.
As workloads increase, Moab automates these requirements and dynamically allocates and then relinquishes resources back to the communal pool. Moab’s accounting capabilities track and record how the resources are used. Moab also provides the ability to burst to a third-party, for example a public cloud, although this feature has yet to be deployed in this instance.
Marty Smuin, CEO of Adaptive Computing, said that his company was very happy to be part of such a ground-breaking project. He underlined the fact that by helping the partners bring HPC capabilities into a cloud environment, diseases will be cured and lives will be saved.
“We’ve been working for Hospital for Sick Kids for a while,” Smuin told us. “And not so long ago a lot of the things we’re doing right now weren’t possible yet, but as we continued to build our relationship, we learned they wanted to do something pretty substantial by mixing HPC into a cloud environment.”
Smuin related that the objective of building this shared high-availability infrastructure with HPC components is to enable the processing of massive amounts of data in a very short amount of time in a highly secure way. A primary defining characteristic of “cloud” is the principle of economy of scale. Costs are amortized across bigger datacenters. With Adaptive and the other vendor partners — Bright Computing, OpenStack, SGI, Mellanox, EMC, and Scalar (the integrator) – the partners at SickKids and UHN’s Princess Margaret Cancer Center were able to consolidate their computing resources within a shared, secure datacenter.
These partners have massive amounts of job load, Smuin told HPCwire. They’ve been able to bring the cost of DNA sequencing down considerably, and now they want to be able to do DNA sequencing for as many people as possible so they can build a secure database. With the ability to share resources more efficiently and to boost to a third party, information can be collected, processed, and pulled in order to make the decisions that will guide treatment, ultimately saving lives.
This ability to manage resource expansion through bursting, whether to a shared cloud or another datacenter resource, is another cloud hallmark: elasticity. Elastic computing was originally part of Moab’s Cloud Suite, but was introduced to Moab as part of the 8.1 rollout. Adaptive explains that elastic computing is triggered when a threshold set in Moab is exceeded. Working in tandem with Openstack, Moab completely wipes each resource after use in compliance with Canadian privacy regulations.
HPC4Health cloud stack
Other components of Adaptive Computing software being used by HPC4Health partners include scheduling, workload management, accounting and advanced policies, including auto enforcement of Service Level Agreements (SLAs), dynamic provision of virtual resources, and job arrays.
As explained by company literature:
- Auto SLA enforcement schedules and adjusts workloads to consistently meet service guarantees and business priorities so the right workloads are completed at the optimal times.
- Dynamic Provisioning discovers that the current level of resources will not meet a given SLA, then reaches out to a provisioning tool that has access to the communal pool of virtual resources. The resources are allocated and then provisioned to match the needed environment. When the workload is complete the added resources are returned to the communal pool (de-provisioned and removed from the workload manager).
- Job Arrays support the submission of many sub-jobs that perform the same work using the same script, but operate on different sets of data.
The post Moab Helps Canada Hospitals Build HPC4Health Cloud appeared first on HPCwire.
