High-Order Discretization Mini‑App and Mathematical Framework

1. Overview

This document describes our effort to develop a portable mini‑app for high‑order discretizations using Kokkos (targeting both CPUs and GPUs) and a mathematical framework that unifies a wide range of methods, drawing inspiration from the Feel++ project. In our work we combine two complementary ideas:

  • A mini‑app that encapsulates the core high‑order discretization kernels.

  • A proxy‑app that serves as a simplified version of a full‑scale application, emphasizing performance‑critical operations.

Furthermore, we take inspiration from the ECP’s development of [libCEED concepts](libceed.org/en/latest/libCEEDapi/), which guide our low‑level operator design.

2. Mini‑App

A mini‑app is a lightweight application designed to capture the essential computational patterns of a complex code base without the full implementation overhead. Our mini‑app:

  • Implements high‑order discretizations for partial differential equations.

  • Leverages Kokkos to ensure performance portability across different hardware architectures (CPUs and GPUs).

  • Focuses on key computational kernels (e.g. basis function evaluation, operator assembly, and matrix–vector products).

  • Serves as a testbed to validate both the numerical methods and the performance optimizations.

\[\frac{\partial^2 u}{\partial t^2} + \frac{\partial u}{\partial t} -\nabla \cdot ( -c \nabla u + \beta u + \gamma ) + a u = f\]
\[\frac{1}{\rho c^2} \frac{\partial^2 u}{\partial t^2} -\nabla \cdot ( \frac{1}{\rho} \nabla u ) = f\]

3. Proxy‑App

A proxy‑app is a simplified, stand‑alone application that replicates the performance‑critical aspects of a full‑scale production code. In our context the proxy‑app:

  • Emulates the discretization workflow by assembling the key operators.

  • Is used to benchmark and tune performance on various hardware configurations.

  • Provides an environment for exploring

    • various finite element libraries

    • optimization strategies (such as tuning Kokkos execution policies) without the complexity of a complete application.

  • Acts as a bridge between the mini‑app’s numerical kernels and a full‑scale solver.

4. LibCEED and Its Concepts

The Exascale Computing Project (ECP) developed libCEED to provide a lightweight, efficient, and portable API for high‑order finite element discretizations. The libCEED approach is based on a few key concepts:

  • Element Restriction: Maps global degrees of freedom into element‑local data, enabling efficient local operator evaluations.

  • Basis Evaluation: Offers routines to evaluate finite element basis functions (and their derivatives) at quadrature points.

  • QFunctions: Encapsulates the action of local operators as small user‑defined functions that are independent of the global assembly process.

  • Operator Assembly: Provides a modular framework to compose element restrictions, basis evaluations, and QFunctions into a complete discretized operator.

These design ideas allow libCEED to separate the mathematical formulation from the hardware‑specific optimizations. In our work, we follow similar principles to create a mathematical framework that supports a broad range of methods (as seen in Feel++) while keeping the user API as stable as possible.

Questions:

we should agree on which c++20 concepts/features/components (idem for the lib) we want to use in our project.

5. FEEL++: A Mathematical Framework for Finite Element Methods

The FEEL++ framework is designed to provide a flexible and generic platform for finite element methods. At its core, the framework relies on a set of polynomials that form a primal basis. This primal basis is then used to construct the finite element spaces following the classical finite element theory.

In FEEL++, various geometric entities such as segments, triangles, quadrangles, tetrahedrons, hexahedrons, prisms, and pyramids are defined on a reference element. From each reference element, a primal (or primary) basis is built. Finally, using the mathematical principles of the finite element method, a final (polynomial) basis is constructed for the discretization of partial differential equations.

5.1. Geometric Entities and the Reference Element

Each finite element in FEEL++ is defined on a reference element. For example, a triangle in 2D or a tetrahedron in 3D is described in a canonical domain where the coordinates are standardized. This reference element serves as the foundation upon which the basis functions are constructed.

/* Example of a reference triangle definition */
ReferenceElement triangleReference( "triangle", { {0,0}, {1,0}, {0,1} } );

5.2. The Primal Basis

The primal basis is a set of polynomials defined on the reference element. These polynomials (for example, Dubiner, Legendre, or canonical monomials) provide the building blocks for the finite element space.

For a given reference element, let \(\{ p_0, p_1, \dots, p_n \}\) be the set of polynomial basis functions. In

\[p_i(x) \quad \text{for} \quad i = 0,\dots,n\]

the coefficients \(c_{ij}\) that relate the polynomial representation to the final finite element basis are determined from the mapping between the reference and physical elements.

/* Instantiation of a primal basis (e.g., using the Dubiner basis on a triangle) */
DubinerStaticBasis<double, 2, 2, 3> dubinerBasis;

5.3. Building the Finite Element Basis

Once the primal basis on the reference element is constructed, the final finite element basis is obtained by mapping the reference basis to the physical element. This mapping is carried out through a geometric transformation that takes into account the shape and size of the physical element.

For example, if \(\phi_i\) denotes a finite element basis function on the physical element, it can be expressed as:

\[\phi_i(x) = p_i(F^{-1}(x))\]

where \(F\) is the mapping from the reference element to the physical element.

The construction process involves:

  1. Evaluating the primal basis functions on the reference element.

  2. Applying the geometric mapping \(F\) to transfer these evaluations to the physical element.

  3. Assembling the local element matrices and vectors accordingly.

5.4. Portable High-Order Discretization with Kokkos

To achieve performance portability across CPUs and GPUs, FEEL++ leverages Kokkos. Kokkos provides:

  • Portable Data Structures: For example, using Kokkos::View for storing the polynomial coefficients.

  • Parallel Execution: Parallel loops for evaluating basis functions and assembling element operators.

  • Memory Abstraction: The same code can run on different architectures without major modifications.

By integrating Kokkos into the FEEL++ framework, the user API remains as close as possible to the original design. In many cases, the only difference is in the instantiation of classes (e.g., using Kokkos-enabled storage classes) while the overall usage—evaluation, differentiation, and assembly—remains unchanged.

in GPU recomputing can be critical. see Geos proxy app. @kfelix @stefano ?

6. References

  • [libCEED API Documentation](libceed.org/en/latest/libCEEDapi/) – An example of how low‑level operator abstractions are defined for high‑order methods.

  • FEEL++ documentation and source code for further details on the implementation.