Code Generation and Performance Engineering for Matrix-Free Finite Element Methods on Hybrid Tetrahedral Grids

Böhm F, Bauer D, Kohl N, Alappat C, Thönnes D, Mohr M, Köstler H, Rüde U (2025)

Publication Language: English

Publication Type: Journal article

Publication year: 2025

Journal

Book Volume: 47

Pages Range: B131-B159

Journal Issue: 1

DOI: 10.1137/24M1653756

Abstract

This paper introduces a code generator designed for extreme-scalable, matrix-free finite element operators on hybrid tetrahedral grids. It optimizes the local evaluation of bilinear forms through various techniques including tabulation, relocation of loop invariants, and interelement vectorization-implemented as transformations of an abstract syntax tree. A key contribution is the generator's ability to optimize not only the local assembly, but also the loop over the underlying grid. To that end, the developed "cubes" loop pattern leverages local structure resulting from grid refinement and significantly enhances arithmetic intensity. Its effectiveness is established during a memory and layer condition analysis. The paper demonstrates the generator's capabilities through a comprehensive, educational cycle of performance analysis, bottleneck identification, and emission of dedicated optimizations. For three differential operators (-Δ, -∇ ∙ (k(x) ∇), α (x) curl curl +β (x)), the set of most effective optimizations is determined from a larger pool. They result in matrix-free operators with a kernel throughput of 1.3 to 2.1 GDoF/s, achieving up to 62% peak performance on a 36-core Intel Ice Lake socket. We make performance comparisons with a sparse matrix-vector multiplication and operators generated by the Firedrake project at increasing levels of refinement. Finally, the solution of the curl-curl problem with more than a trillion (10¹²) degrees of freedom on 21,504 processes in less than 50 s showcases the generated operators' performance and extreme scalability as part of a full multigrid solver.

Authors with CRIS profile

Involved external institutions

Ludwig-Maximilians-Universität (LMU) Germany (DE)

How to cite

APA:

Böhm, F., Bauer, D., Kohl, N., Alappat, C., Thönnes, D., Mohr, M.,... Rüde, U. (2025). Code Generation and Performance Engineering for Matrix-Free Finite Element Methods on Hybrid Tetrahedral Grids. SIAM Journal on Scientific Computing, 47(1), B131-B159. https://doi.org/10.1137/24M1653756

MLA:

Böhm, Fabian, et al. "Code Generation and Performance Engineering for Matrix-Free Finite Element Methods on Hybrid Tetrahedral Grids." SIAM Journal on Scientific Computing 47.1 (2025): B131-B159.

BibTeX: Download