High-profile ocean models accelerated by custom software

This figure shows surface currents simulated by MPAS-Ocean.Credit: Los Alamos National Laboratory, E3SM, U.S. Department of Energy

On the beach, ocean waves provide soothing white noise. However, in scientific laboratories, they play an important role in weather forecasting and climate research. The ocean, along with the atmosphere, is typically one of the largest and most computationally intensive components of Earth system models, such as the Department of Energy’s Energy Exascale Earth System Model (E3SM).

A breakthrough in ocean modeling

Most modern ocean models focus on two categories of waves: barotropic systems, where the wave propagation speed is fast, and baroclinic systems, where the wave propagation speed is slow. To address the challenge of simulating these two modes simultaneously, a team from DOE’s Oak Ridge National Laboratory, Los Alamos National Laboratory, and Sandia National Laboratories has We have developed a new solver algorithm to shorten it. -Ocean, E3SM ocean circulation model, increased by 45%.

The researchers tested the software on the Summit supercomputer at ORNL’s Oak Ridge Leadership Computing Facility, a DOE Office of Science user facility, and the Compy supercomputer at Pacific Northwest National Laboratory. They ran the main simulations on the Cori and Perlmutter supercomputers at the National Energy Research Scientific Computing Center at Lawrence Berkeley National Laboratory, and their results were International Journal of High Performance Computing Applicationss.

Computing innovations for climate modeling

because TrilinosBecause open source software databases ideal for solving scientific problems on supercomputers are written in the C++ programming language, and Earth system models like E3SM are typically written in Fortran, the team took advantage of the advantages of For Trilinois an associated software library that incorporates Fortran interfaces into existing C++ packages to design and customize new solvers focused on barotropic waves.

“A nice feature of this interface is that you can use all the components of the C++ package in the Fortran language, so you don’t have to translate anything, which is very convenient,” said lead author Hyun, a computational earth systems scientist. Kang said. ORNL.

Improvements to MPAS-Ocean

This work is built on Research results announced before Journal of Advances in Earth System Modeling In this paper, researchers at ORNL and Los Alamos National Laboratory handcrafted code to improve MPAS-Ocean. This time, the ForTrilinos-enabled solver overcomes the remaining shortcomings of the solver obtained in previous studies, especially when the user runs his MPAS-Ocean using a small number of computing cores for a given problem size. Did.

MPAS-Ocean’s default solver is an explicit sub-solver, a technique that uses a large number of small time intervals or time steps to compute barotropic wave properties in conjunction with baroclinic calculations without destabilizing the model. Cycle dependent. If the barotropic and barotropic waves can be advanced with time step sizes of 300 and 15 seconds, respectively, then to maintain the same speed the barotropic calculation would need to complete over 20 times more iterations, a huge amount requires computational power.

In contrast, the new solver for barotropic systems is semi-implicit. That is, it is unconditionally stable, allowing researchers to use the same number of large time steps without sacrificing accuracy, saving significant time and computational power.

The community of software developers has spent years optimizing Trillinos and Fort Lilinos’ various climate applications. As such, a modern MPAS-Ocean solver that leverages this resource will outperform hand-crafted solvers and enable other scientists to accelerate their climate research efforts.

“If we had to code every algorithm individually, it would require much more effort and expertise,” Kang said. “But with this software, you can run simulations quickly and quickly by incorporating optimized algorithms into your programs.”

Future enhancements and impact

Current solvers still have scalability limitations for high-performance computing systems, but they perform very well up to a certain number of processors. This drawback exists because the semi-implicit method requires all processors to communicate with each other at least 10 times per time step, which can reduce model performance. To overcome this obstacle, researchers are currently optimizing processor communication and porting solvers to GPUs.

In addition, the team updated the time-stepping method of the pressure clinic system to further improve the efficiency of MPAS-Ocean. Through these advances, researchers are making climate predictions faster and more reliable, an essential upgrade to ensure climate security and enable timely decision-making and high-resolution forecasting, aims to be more accurate.

“This barotropic mode solver enables faster calculations and more stable integration of models, especially for MPAS-Ocean,” said Kang. “Extensive use of computational resources requires enormous amounts of power and energy, but by accelerating this model we can reduce energy usage, improve simulations, and improve performance over decades and even beyond.” It will be easier to predict the effects of climate change thousands of years into the future.”

Reference: “MPAS-ocean implicit pressure mode solver using a modern Fortran solver interface” by Hyun-Gyu Kang, Raymond S Tuminaro, Andrey Prokopenko, Seth R Johnson, Andrew G Salinger, Katherine J Evans, 2023. November 17th, International Journal of High Performance Computing Applications.
DOI: 10.1177/10943420231205601

This research was supported by E3SM and the Exascale Computing Project (ECP). E3SM is sponsored by the DOE Office of Science’s Biological and Environmental Research Program, and ECP is managed by DOE and the National Nuclear Security Administration. The DOE Office of Science’s Advanced Scientific Computing Research Program funds OLCF and NERSC.