Develocity is an InfoWorld 2024 award winner! Try it out today.
Develocity 2024.3
In the 2024.3 release of Develocity, we’ve introduced several key enhancements aimed to improve reporting, resource observability, and cross-platform integration:
- Develocity Reporting and Visualization Enhancements: This release presents enhancements to the Reporting and Visualization toolchain, particularly emphasizing test acceleration and dependency management.
- Develocity Edge (beta): This release introduces Develocity Edge, a new feature that reduces latency and improves build performance by allowing users to deploy Develocity services closer to build agents and developer workstations.
- Build Scan® and Develocity API Support for npm & Python (beta): This release introduces preliminary Build Scan functionality for two new platforms: npm (a leading package manager and build platform for the JavaScript ecosystem) and Python.
Read on for more details of these and other new features.
For optimal usage of Develocity, use version 1.23 of the Develocity Maven extension, version 3.19 of the Develocity Gradle plugin, version 1.1.2 of the Develocity sbt plugin, version 3.2 of the Develocity Test Distribution agent, version 20.0 of the Gradle Build Cache node, version 0.9.0 of the Develocity npm agent (beta) and version 0.9 of the Develocity Python agent (beta).
For optimal integration of your CI provider with Develocity, use the latest versions of the Gradle Jenkins plugin, Build Scan plugin for TeamCity, Develocity Bamboo plugin, Develocity GitLab templates, GitHub Actions for Gradle, and GitHub Actions for Develocity.
Highlights
Observe the adoption of Develocity's test acceleration features
The Develocity Reporting and Visualization Kit now includes visualizations for test acceleration time-saving realization and potential for Develocity’s Predictive Test Selection and Test Distribution features for Gradle and Maven builds.
Realized test acceleration savings dashboard
This new dashboard facilitates assessing the adoption of Predictive Test Selection and Test Distribution across projects in an organization and observing their effectiveness.
Predictive Test Selection savings potential dashboard
This new dashboard facilitates assessing the test acceleration that could be achieved from Predictive Test Selection. The measurements are based on simulating the effect of applying Predictive Test Selection to test executions that did not enable it.
Along with projected time savings, the visualization provides several critical metrics to assess prediction accuracy, such as the percentage of predicted task/goal failures and the amount of avoided test class executions.
The visualization will adjust based on the chosen selection profile, helping to find the right balance between time savings and prediction accuracy.
Improved configuration cache effectiveness reporting
The configuration cache is a Gradle feature that greatly enhances build performance by caching the outcome of the configuration phase and utilizing it for future builds.
A Build Scan now displays the dependency resolution time separately from the configuration cache store time. This helps you better understand the configuration cache’s effectiveness, as dependency resolution typically occurs during execution and isn’t part of the configuration cache overhead.
The /api/builds/{id}/gradle-configuration-cache endpoint of the Develocity API has been extended to include dependency resolution time during storage.
Additionally, Develocity Reporting and Visualization now includes a dashboard to help users analyze the effectiveness of Gradle’s configuration cache, including total configuration cache savings and reasons for configuration misses.
This allows you to identify the most frequent causes of misses, prioritize fixes, and verify that your hit rate does not regress.
Identification of dependencies used across projects
Develocity Reporting and Visualization includes a new dependency dashboard that offers reporting and search functions for dependency information related to libraries utilized in Gradle and Maven builds.
For example, this feature can be instrumental in identifying projects that use dependencies with known vulnerabilities. The dependency build models can also be accessed through the Develocity API.
Identification of build plugins used across projects
Develocity Reporting and Visualization includes an updated dashboard that facilitates understanding Gradle and Maven plugin usage within your builds.
The filter features enable you to search according to plugin utilization. The dependency build models can also be accessed through the Develocity API.
Improved insights into build resource usage observability
This release of Develocity introduces several enhancements to build environment resource usage observability, initially released in Develocity 2024.2.
To help identify which processes within a Build Scan of Gradle, Maven, and sbt consume the most resources during a build, the individual CPU and memory usage of each of the top 5 processes for each metric is now displayed.
The tooltip now also shows the absolute time of the resource usage snapshot for Gradle, Maven, Bazel, and sbt build tools. This helps users to correlate resource usage with other information, such as start times and logs from CI servers.
Furthermore, users can now share specific points on the resource usage graph by clicking a point and sharing the URL, facilitating collaboration.
Performance improvements for Develocity Reporting and Visualization with AWS Athena
This release delivers significant query performance improvements for users of Develocity Reporting and Visualization backed by AWS Athena. This is achieved via a disabled-by-default option to export data to S3 as compacted ORC files instead of a single JSON file per build. The Athena query engine can query over this data format much more efficiently, significantly reducing query execution times and costs.
Existing configurations using the JSON-based data export are unaffected by this change and will continue to function without interruption upon upgrade.
To use this new capability, please contact your Customer Success manager to discuss the technical requirements for adopting this feature.
Reduction to the Maven extension input fingerprinting footprint
Develocity Maven extension 1.22.2 introduces a significant optimization that can reduce time spent on input fingerprinting by up to 40% compared to earlier versions.
The Develocity Maven extension now attempts to reuse checksums from the Maven repository when it is safe to avoid logically redundant operations to determine input identity.
Builds running on ephemeral CI agents are expected to see the greatest performance improvements from this optimization.
Simplified geographic distribution of Develocity services
Develocity Edge is a new component introduced in this release that allows you to establish Develocity points of presence (or “Edges”) across geographic boundaries. This makes it easier to provide Develocity services closer to where builds are run and makes builds faster by reducing network latency.
In this release, the Develocity Edge can be used as a Build Cache server for Gradle, Maven, Sbt, and Bazel builds. Each Edge automatically manages data sharing with and from the Develocity server and other Edges.
Develocity Edges are assigned to a location. Build users can set their location by logging in to the Develocity application and accessing “My Settings” from the user menu.
The Gradle and Maven Develocity plugins provide a new opt-in “Edge discovery” option, which uses the user’s location setting to automatically assign an Edge to use as a remote build cache. Using an Edge in a nearby location improves build time by reducing network latency and improving throughput.
As each Edge automatically manages data sharing with and from the Develocity server and other Edges, any Edge can be used without sacrificing hit rates.
Previously, using the Develocity build cache node in a distributed fashion required per-user local build configuration changes. The Develocity Edge with “Edge discovery” removes the need for extra local build configuration and allows Edge locations and instances to be updated dynamically as the Develocity server manages all configuration.
The Edge component is available as a Docker image or standalone fat JAR and is easy to deploy and operate via Kubernetes or any modern computing platform.
Edge discovery is available for:
- Gradle builds with Develocity Gradle plugin 3.19+
- Maven builds with Develocity Maven extension 1.23+
Support for sbt builds with the Develocity sbt plugin is planned for a future release.
Build cache storage and usage observability enhancements
The Build Cache administration page now visualizes the following indicators that provide insights into build cache usage patterns and whether the build cache size is undersized, affecting build performance due to a too-short retention period:
- The rate of cache misses for recently evicted entries
- The incoming data storage rate
- For least-recently-used items, how long since the items were last used
- The storage size required to achieve a seven-day retention
Please note that these metrics apply to the logical build cache served by the Develocity server and Develocity Edges. They do not provide insight into Develocity Build Cache nodes.
Build Scan and Develocity API support for npm
Develocity now integrates with npm, the standard package manager for Node.js.
This first release provides basic Build Scan functionality and several build metrics that help you understand, debug, and observe your use of npm.
Build Scan gives you access to this information and more, allowing you to investigate the console output of individual npm invocations.
The Develocity API now includes a new `npm-attributes` endpoint, which helps you:
- Track how often npm is used
- Identify the machine on which it ran
- Measure how long it took
- Discover the npm and Node.js versions in use across various projects
Build Scan and Develocity API support for Python
Develocity now integrates with the Python ecosystem.
The first release includes basic Build Scan functionality and several build metrics that help you understand, debug, and observe the usage of various Python tools.
Build Scan gives you access to this information and more, allowing you to investigate the console output of individual usages of supported tools.
The Develocity API now includes a new `python-attributes` endpoint, which helps you:
- Track how often Python is used
- Identify the machine on which it ran
- Measure how long it took
- Discover the Python version in use across various projects
Support for PostgreSQL 17
Develocity 2024.3 is compatible with PostgreSQL databases up to and including version 17. Previously, version 14 was the maximum supported version.
The User-managed database migration guide describes the prerequisites for upgrading your Develocity instance’s user-managed database to the latest version of PostgreSQL.
To learn more about running Develocity using a user-managed database, you can read about the trade-offs in either the Kubernetes Helm chart configuration guide or the Standalone Helm chart configuration guide.
The embedded database that Develocity uses, if no user-managed database is configured, continues to run PostgreSQL 14.
Upgrade notes
Using the latest versions of all components ensures that you benefit from security updates, bug fixes, and feature enhancements. Please check the version compatibility matrix for further reference.
It is recommended that you update the configuration of the Gradle Jenkins plugin, Build Scan plugin for TeamCity, and Develocity Bamboo plugin to reference the latest version of the Develocity Gradle plugin.
Temporarily degraded performance due to data reindexing
Upon upgrading, a data reindexing process will be initiated in the background.
Develocity will remain usable during this process.
- CPU usage will increase, and performance may be slightly degraded.
- Depending on the size of the build history, reindexing can take several hours to days.
- Some builds may be omitted from cross-build data visualizations during this time.
- Recent builds are prioritized, ensuring their data becomes available sooner.
Predictive Test Selection is temporarily unavailable after upgrading
The update to the prediction data processing in this release requires re-analyzing the build history, which is initiated automatically after upgrading.
Attempting to use Predictive Test Selection during this process will result in all tests being selected for execution.
The duration of this process depends on the amount of build history stored by your installation and the availability of CPU resources. An installation with a build history of 6 months, 60,000 builds per week, and eight processor cores is estimated to take approximately 24 hours.
Therefore, it is recommended that installations using Predictive Test Selection be upgraded at the end of the week to allow this process to be completed over the weekend.
A banner will be displayed on the Predictive Test Selection dashboard while this re-analyzing occurs.
Predictive Test Selection with Develocity 2023.4 requires Develocity Gradle plugin 3.16+ or Develocity Maven extension v1.20+.
Attempting to use Predictive Test Selection with older versions will result in all tests being selected for execution.
The Predictive Test Selection Simulator is unaffected by this requirement.
Build data of Predictive Test Selection users is temporarily unavailable in the Develocity Visualization and Reporting Kit after upgrading
The out-of-the-box visualizations of the Predictive Test Selection data introduced in this release rely on the prediction data available for the builds in the selected time frame.
Therefore, the data synchronization components of the Develocity Visualization and Reporting Kit will wait for the prediction data to be collected before displaying any data for a given build.
This may result in the build data appearing with a slight delay in the provided visualizations and if queried with the Develocity API.
Supported Kubernetes and Helm versions
- This release’s minimum supported Kubernetes version is 1.29.0 (in Develocity 2024.2, it was 1.28.0)
- This release’s minimum supported Helm version is 3.14 (in Develocity 2024.2, it was 3.13)
Please check the version compatibility matrix for further reference.
Develocity Helm Chart upgrade notes
Please consult the upgrade documentation for detailed information about how to upgrade and to ensure optimal continuity after upgrading.
Develocity Reporting Kit Helm Chart upgrade notes
If you deploy the Develocity Reporting Kit chart using tools that depend on deterministic output from template rendering, you must set additional Helm values when upgrading to version 1.2.0.
These are outlined in the relevant appendix of the installation manual for the newly released version.
If you deploy the Develocity Reporting Kit chart into an environment that uses a service mesh, such as Istio, please read and follow the relevant instructions in the documentation about HTTPS communication between components of the Reporting Kit.
Notice of discontinuation
Build cache access control with username and password
Develocity 2024.1 announced the deprecation of username and password access control for the Build Cache. With Develocity 2024.3, this form of access control has been discontinued.
This change will affect both standalone Build Cache nodes and the Build Cache integrated with the Develocity server.
If you are using username and password access control, please consult the Develocity Access Control for Build Cache migration guide for guidance on transitioning to Develocity account-based access control.
Failure to transition before upgrading to Develocity 2024.3 will disrupt service to build cache users.
If you have any questions or concerns about the migration or need assistance, please contact the Develocity support team.
Build cache node versions older than 14.0
With the release of Develocity 2024.3, support for Build Cache node versions older than 14.0 has officially ended.
As a result, these older versions will no longer be able to connect to Develocity 2024.3 and future releases. Critical security vulnerability patches will not be backported to these unsupported versions.
From December 18, 2024, these versions will no longer be available for download from Docker Hub or the Build Cache Node User Manual.
If you have any questions or need assistance to upgrade to a more recent version of the build cache node, please contact the Develocity support team.
Notice of deprecation
Alpine-based database and database-upgrade images
With the release of 2024.3, the Alpine-based database and database upgrade images have been deprecated.
If you currently use these images, please contact the Develocity support team.
Removals
The 2024.3 release removes several deprecated configuration properties related to Object Storage.
Key changes include:
- Configuration of Object Storage in unattended configuration: Removed after being deprecated in 2023.3, including the UI configuration page.
- Embedded Storage Capacity: The property objectStorage.embedded.storage.data.capacity has been replaced with objectStorage.embedded.storage.buildCache.capacity.
- AWS S3: Several keys in objectStorage.s3.credentials (e.g., accessKey, secretKey) are removed, and users must use new credentials.type values like keys, irsa, or instanceProfile.
- Google Cloud Storage: The objectStorage.type: googleCloud and related configuration are replaced by googleCloudStorage, which supports more secure authentication options.
- Microsoft Azure Blob Storage: The objectStorage.type: azure and related configuration are replaced by azureBlobStorage, which supports more secure authentication options.
Users are encouraged to follow the 2024.3 upgrade documentation for further details and migration steps.
If you have any questions or need assistance, don’t hesitate to contact the Develocity support team.
Additional notes
Renaming the Gradle Enterprise Build Validation Scripts repository
The repository of the Develocity Build Validation Scripts, formerly known as the Gradle Enterprise Build Validation Scripts, will be renamed from https://github.com/gradle/gradle-enterprise-build-validation-scripts to https://github.com/gradle/develocity-build-validation-scripts during the third week of December 2024.
If you run the Build Validation Scripts in GitHub Actions using the included actions, these actions will automatically redirect to the renamed repository in most cases.
However, we strongly encourage you to update your workflows to use the new repository name. GitHub Enterprise Cloud users must update the actions to use the new repository name, as redirects will not be performed.
If you have any questions or need assistance, don’t hesitate to contact the Develocity support team.
Resource adjustments
The CPU requirements for the gradle-enterprise-operator have been increased by 250 mCPU.
Changes
- [NEW] Support ANSI escape codes in the Gradle console log
- [NEW] Toggle to synchronize only the most recently used entries from the canonical cache catalog to connected nodes
- [FIX] Develocity contains a high rated security vulnerability
- [NEW] Build Scan and Develocity API support have been added for npm
- [NEW] Build Scan and Develocity API support have been added for Python
- [NEW] Build Scan will now inform users of the maximum configured build cache artifact size in instances where the remote store rejects an item due to size constraints
- [NEW] Build Scan now displays CPU load for the top 5 CPU-intensive processes
- [NEW] Build Scan now displays the top five processes with the highest memory usage, detailing the specific usage for each process
- [NEW] The resource usage graph tooltip in Build Scan now includes absolute timestamps
- [NEW] The Bazel Invocation ID is displayed in a Build Scan
- [NEW] The performance configuration page in Build Scan now supports Gradle's Isolated Projects feature
- [NEW] Users can now download cached artifacts from the Bazel Build Scan targets section
- [NEW] Users can now pin and share the resource usage graph tooltip via the URL
- [NEW] Develocity now reports the dependency resolution time separately from the configuration cache store time in both the user interface and the API
- [NEW] Gradle builds can now be filtered by configuration cache outcome in advanced search
- [NEW] Dependency information for Maven and Gradle builds can now be accessed through Develocity API build models
- [NEW] Dependency tracking functionality is now included in Develocity Reporting and Visualization
- [FIX] Permissions errors were preventing the start-up of new Develocity installations using an Azure PostgreSQL database