Developer Productivity Engineering Blog

Accelerate your Quarkus Maven builds with Develocity Build Cache

By Jerome Prinet
August 20, 2024

Introduction

What is Quarkus

Quarkus is a Kubernetes-native Java framework designed to optimize Java for containers. Its main goals are to make Java a leading platform in Kubernetes and serverless environments while providing a developer-friendly experience.

What is the Quarkus build goal

The Quarkus build goal is brought by the quarkus-maven-plugin. This goal builds the Quarkus application, which can have several forms:

  • A jar file
  • An uber-jar
  • A native executable

What is Develocity Build Cache

Develocity Build Cache speeds up software builds by reusing unchanged outputs from previously successful builds. When a goal is run, its results are stored, and if the same goal with the same inputs is run again, the stored outputs are used instead of rebuilding everything.

Both local and remote caches are supported: local caches store outputs on the developer’s machine, while remote caches share outputs across multiple machines and are usually populated by CI jobs. This reduces build times by avoiding work that has already been done.

Why should we make the Quarkus build goal cacheable

The build time is usually dominated by the Quarkus build goal, and this is especially true when creating native executables. The data shows native compilation time taking minutes and representing 90% of the overall build time.

The key here is to use the Maven quarkus-build-caching-extension. Let’s see what that means in practice.

In practice

Requirements

Configuration steps

Reference the extension in .mvn/extensions.xml (this extension requires the develocity-maven-extension):

Enable Quarkus config tracking in pom.xml:

Add the track-prod-config-changes execution to the quarkus-maven-plugin configuration:

Scope

The current optimization can be applied on any build (CI and local), and jar, uber-jar and native executable construction can be cached, although the latter would benefit the most from caching due to the processing time required and the avoidance savings implied.

Improvements in action

The Quarkus build itself caches the native Quarkus builds of some integration tests.

Looking below at the Develocity Build Scan® from the Quarkus Develocity instance we can see the cache hits at a glance by looking at the overall build time, reduced from 5+ minutes to less than a minute on such builds:

The screenshot below illustrates the savings obtained with a cache hit on the Quarkus build goal. By fetching the goal outputs from the Develocity cache, 4 minutes and 52 seconds are saved by not executing the quarkus:build goal unnecessarily:

Optimizations

Building the Quarkus application is usually the last step in a build and many changes can affect the Quarkus build goal outputs. This reduces the chances of a cache hit, although some builds may run without any changes in between. Some configuration tweaks can help to maximize the cache hit rate.

Cross-os cache entries

The CI builds should populate cache entries that can be consumed by local builds. This can be achieved with two distinct approaches:

  • Using the in-container build strategy with a fixed build image. This enforces that the produced applications are compatible.
  • Running the CI build on a host system similar to the local system (ie. identical os.name, os.version. os.arch, java.version). Some specific cache seeding CI jobs can be created to cover this scenario.

Checked-in Quarkus configuration dump

A key component of the caching mechanism is the Quarkus configuration dump file .quarkus/quarkus-prod-config-dump, which contains all the Quarkus properties used during the Quarkus build process.

This file is generated by the Quarkus build goal when the Maven property quarkus.config-tracking.enabled is true. The Quarkus build goal is cacheable only if the Quarkus configuration dump is present.

This file should be committed in the source code to allow cache hits on a fresh clone of the project repository. This is especially relevant for CI running builds in ephemeral containers.

The way to get started is to run a local build and a CI build and compare both generated configuration dumps:

  • If no differences are encountered, the file can directly be added to the Git repository
  • If some safe-to-be-ignored properties are encountered (meaning changing the property does not impact the Quarkus build goal output), then you can exclude them from the configuration tracking process by configuring the quarkus.config-tracking.exclude property, here is an example with quarkus.native.graalvm-home: <quarkus.config-tracking.exclude>quarkus.native.graalvm-home</quarkus.config-tracking.exclude>
  • If some properties can’t be ignored, then CI and local will have a specific Quarkus configuration dump both added to the Git repository but with a different name. Its suffix will be configured in a Maven profile accordingly (here file would be named quarkus-prod-config-check-local and quarkus-prod-config-check-ci):

Solution overview and methodology

Classic approach

Making a custom goal cacheable usually requires identifying all the inputs and outputs and declaring them through the Develocity caching API.

The problem here is that identifying all the inputs of the Quarkus build goal is a complex operation.

Several sources can be used to define Quarkus properties with overriding options.

Some properties can be declared without having a real impact on the produced artifacts, which confirms that the traditional approach is not applicable here.

Collaborative approach

In order to identify all the Quarkus configuration properties participating in the build process, we have been partnering with the Quarkus team which implemented a way to record all the relevant Quarkus properties involved in the build process in a file.

However, this mechanism is deeply interweaved with the build process, and identifying the relevant properties takes almost as much time as running the build.

Thus, using the traditional way of collecting inputs to compute the cache key on the build goal does not work here. To address this, the Quarkus team created the track-config-changes goal which can query the actual value of the properties recorded in a previous build.

This goal is fast enough to be executed before each build and allows to confirm that the current Quarkus configuration has not changed since recording the properties, meaning the current context is eligible for a cache hit.

Implementation details

The track-config-changes goal creates a file target/quarkus-prod-config-check containing all the properties from the .quarkus/quarkus-prod-config-dump with their actual value.

If property values are identical in the two files, it means that the Quarkus configuration was not changed since recording the Quarkus properties, and therefore the Quarkus build goal can be marked cacheable.

When the Quarkus build goal is marked cacheable, the regular caching process kicks in.

Illustrated sequence of operations

Initialization build (one-off step):

  • track-config-changes does nothing as .quarkus/quarkus-prod-config-dump is absent
  • Quarkus configuration from current and previous build differ

=> The build goal is not cacheable:

  • build executes and creates .quarkus/quarkus-prod-config-dump

First (post-initialization) build:

  • track-config-changes creates target/quarkus-prod-config-check
  • Quarkus configuration from current and previous build are identical (assuming Quarkus configuration was unchanged)

=> The build goal is cacheable

  • Cache lookup happens: CACHE MISS
  • build executes and creates .quarkus/quarkus-prod-config-dump
  • output is stored into the cache

Next builds:

  • track-config-changes creates target/quarkus-prod-config-check
  • Quarkus configuration from current and previous build are identical (assuming Quarkus configuration was unchanged)

=> The build goal is cacheable

  • Cache lookup happens: CACHE HIT
  • build is not executed

Conclusion

We have demonstrated that the Quarkus build goal can be made cacheable with minimal modifications to the Maven build configuration. This means that by implementing a few straightforward changes, you can enable caching for Quarkus builds, potentially reusing outputs from previous builds to save time.

Although not every build would benefit from caching, the time savings for those that do can be substantial. Given the often lengthy process involved in running a Quarkus build, experimenting with build caching in your development environment is highly recommended. By doing so, you can assess the potential acceleration and efficiency improvements specific to your projects, leading to more streamlined and productive build processes.

Related links