Develocity 2023.4 caps a strong year of product innovation for the engineering team at Gradle along three key dimensions. First, this year we expanded the market surface area for Develocity with GA support for Bazel and sbt (in beta).
We also expanded our partner ecosystem to integrate Develocity with key enterprise software development and delivery environment infrastructure technologies. Finally, we deepened our ability to support mission-critical levels of application security, and availability at hyper-scale and delivered many high-value customer-driven enhancements.
Specifically, Develocity 2023.4 provides an additional detection mechanism for flaky tests, improved test selection predictions, insight into artifact transform executions for Gradle builds, new API enhancements, and powerful new build filtering options to spot trends and identify issues. More details about the highlights described below and additional enhancements not covered here can be found in the Release Notes.
Note! Develocity 2023.4 is the first release where the new name “Develocity” (formerly Gradle Enterprise) is used in the product UI and documentation.
Detection of flaky tests across builds
Dealing with flaky tests is a significant challenge in software development, even for the most advanced development teams. These indeterminant and unpredictable tests can pass or fail without any differences in code, casting doubt on the reliability of your toolchain and ultimately on the application itself.
As part of its Test Failure Analytics capabilities, Develocity detects flakiness by re-running failed tests as part of the same build execution. In addition to tests that are retried in the same build with different outcomes, Develocity 2023.4 now identifies tests that are flaky across different build executions.
This makes the impact of a flaky test more observable. As a result, holistic information on frequency of occurrence, recency, and impact for a customer set of builds can be used to better prioritize flaky tests for remediation.
Increased predictive accuracy of Predictive Test Selection
Develocity Predictive Test Selection (PTS) saves testing time by identifying, prioritizing, and running only tests that are likely to provide useful feedback during test runs by using a machine-learning model. Develocity automatically develops a test selection strategy by learning from historical code changes and test outcomes from your Build Scan® data to predict a subset of relevant tests, which are then executed by your build.
Improvements to the machine learning model in this release increase the likelihood of predicting test failures, with little to no loss in time savings. When using the standard selection profile, testing on several enterprise and open-source projects showed an average of 49% fewer missed test failures compared to the previous version of Develocity.
This was done by increasing the size and variety of the training dataset as well as updating the model’s input features to include more relevant information. Now users of PTS can get the same fast test feedback cycle improvements—typically up to 70%—with increased confidence in the model’s effective test coverage.
Insight into Gradle Build Tool artifact transform executions
Build Scan® for Gradle Build Tool now provides build cache effectiveness and other finer-grained performance insights for artifact transforms executions, This expands the observability capability of Build Scan to an important component of overall build performance for performance troubleshooting and optimization purposes.
Artifact transforms are a concept from Gradle Build Tool’s dependency management. When Gradle Build Tool resolves artifacts for a configuration and finds that they are not in the required format, Gradle Build Tool can convert them to the requested format by executing artifact transforms. Users do not directly request these transforms if a task requires artifacts as input, Gradle Build Tool executes the appropriate transforms before the task runs or as part of task execution. Artifact transforms can be cacheable, and benefit from the same avoidance optimizations as tasks.
Insights into transform executions are present on a new tab on the Performance page. A breakdown of the executions allows you to spot possible performance problems in the build at first glance.
From the breakdown, users can drill down into the different categories to investigate certain categories or individual executions. Additional search criteria are available to restrict the data to what is necessary for the current investigation. To find the most expensive transform executions, users can sort the list by longest duration, or group by transform action type. Similarly, for an individual task on the timeline, the transform requests tab allows navigating back to an individual execution requested by the task.
Develocity API enhancements
Three new API enhancements are now available as part of Develocity 2023.4. First, new API endpoints that allow retrieving aggregate test information for Gradle, Maven and Bazel builds have been added to this release. They provide aggregated information about tests for a set of builds such as which tests were the flakiest or slowest.
These endpoints can be used, for example, to automate monitoring for test classes or cases that have recently been flaky or generating a periodic report on the slowest tests. The Develocity API sample application has been updated to demonstrate usage of the new API capability.
Second, you can now retrieve build network request information via the Develocity API. New API endpoints for Gradle and Maven provide information about dependency downloads and expose the impact of network requests on build time.
Finally, enhancements to the Develocity API make it easier to retrieve build information. 2023.3 added the ability to filter builds returned by the /api/builds API endpoint using a concise and powerful query language. This made it easier to work with a subset of builds, but still required one or more further API calls per matched build to access detailed information about each build. In Develocity 2023.4 API consumers can now retrieve the detailed information as part of the build list, simplifying API clients.
See the reference documentation for the /api/builds/{id} and /api/builds endpoints in the Develocity API User Manual for more information.
New Build Filtering Options
Users can now filter builds by build options and the build environment via the advanced Build Scan query and when querying the /api/builds API endpoint. This allows for powerful queries to identify trends, compare different sets of builds and identify issues. For example, you can view performance trends of builds with specific build options enabled such as the build cache, view test failures restricted to a subset of build hosts, based on hostname or resources like CPU and JVM memory, and verify that no builds are running without the expected set of build options. Users can now also filter builds to find those that had build cache errors. This is useful in identifying configuration issues in different environments and verifying a healthy build environment.
There are of course many other goodies in 2023.4 (like support for Cucumber tests in Test Distribution and Predictive Test Selection to name just one). You can learn a whole lot more of course in our Release Notes and from there you will see links to the next level of documentation.