Scientists at the Department of Environment and Science (DES) have upgraded their High Performance Computing (HPC) platform, replacing at-risk and end-of-life components, and increasing storage capacity and computing power.
The HPC platform allows them to map, analyse, model and predict patterns and scenarios related to climate, the physical features of an area (remote sensing) and the movement, distribution, and quality of water (hydrology).
It provides critical information to the Queensland Government and helps us manage priority environmental issues including the Great Barrier Reef catchment’s water quality, vegetation management (including land clearing), seasonal rangeland growth, drought and climate change effects.
To ensure the upgrade delivered a faster, more capable HPC platform, the Accelerating Science Delivery Innovation program identified they needed to:
- fix the at-risk and unsupported HPC platform components
- modernise the HPC platform to support increasing data storage and computing demands, including the uptake of machine learning
- create a roadmap for a dynamic future-state HPC platform that takes advantage of cloud computing and other technological advancements.
- consulted with HPC users and experts to understand what they needed from the current and future HPC platform
- engaged specialists to develop the HPC and Hybrid Cloud Strategy 2017-2022 which would inform the HPC upgrade
- consulted with stakeholders across government to ensure that the strategy aligned with government priorities including DIGITAL1ST.
The new mass storage tape library provides up to 24 petabytes of data storage (1 petabyte = 1,000,000 gigabytes). The increased capacity and faster data retrieval means that scientists can better address big data challenges. They can collect, manage and use data more effectively to support managing Queensland’s environment. These improvements also safeguard our scientific data.
The upgrade increased computing power by approximately 8 times over the previous capacity. The upgrade also introduced the latest technologies, including graphics processing units and flash storage.
Graphics processing units have more processing power with tasks that need multiple parallel processes, such as machine learning and processing high-resolution satellite imagery.
A proof-of-concept successfully used machine learning to identify a variety of land use types, taking advantage of the enhanced HPC compute capability.
“Science Division’s HPC is critical computation infrastructure for undertaking 21st century science. For the first time, next generation machine learning hardware allows us to deploy artificial intelligence solutions for enhanced insights into our most complex environmental problems.”
A new high-speed clustered storage system provides a shared parallel data processing environment. This means scientists can more effectively process scientific data and deliver remote sensing and climate science.