Gradle Enterprise 2019.5

Gradle Enterprise 2019.5 helps deal with the problem of flaky tests by identifying and analyzing flaky tests in individual builds and across many builds, and processes incoming build scans much more efficiently.

Read on for more details.

Use version 1.3.5 of the Gradle Enterprise Maven extension, and version 3.1.1 of the Gradle Enterprise Gradle plugin for optimal usage of Gradle Enterprise.

Highlights

Flaky test identification and analysis

Gradle and Maven build scans now show a test as FLAKY when it was executed multiple times and reported at least 1 FAILED and 1 PASSED result, such as when using the new Test Retry Gradle Plugin or specifying rerunFailingTestsCount when using the Maven Surefire or Maven Failsafe plugin.

Build Scan with flaky test

In addition, the top tests report in the Tests Dashboard now shows the tests that were the most flaky for your selected set of builds, along with an overall flaky tests trend chart.

Tests Dashboard with top flaky tests

The Tests Dashboard is available in 2019.5 as a partial preview of a larger set of testing-oriented functionality that will be available as an add-on package in upcoming Gradle Enterprise versions. Depending on your usage license, this new functionality may not be available to your installation when it is no longer in feature preview. If you have questions regarding this matter, please contact Gradle Enterprise support.

As you’d expect, you can drill down to see which methods are flakiest and list recent builds where a single test was flaky or failed.

Visualize most flaky tests

Learn more in this blog post.

More efficient build scan collection and processing

When a Gradle Enterprise enabled build is run, it publishes a record of what happened to the Gradle Enterprise server. The server then processes the raw data and transforms it into a form suitable for long term storage. This process involves aggressive deduplication of data and other transformations to minimize the storage space requirements.

Organizations that perform many builds each day and/or have large builds that do a lot of work can generate significant amounts of data to be processed, which can strain the Gradle Enterprise server and lead to delays in the availability of build scans after upload.

This data ingestion process has been improved to be more efficient in Gradle Enterprise 2019.5. This reduces the processing latency and reduces the CPU and memory usage of the server when processing large volumes of data.

Upgrade notes

If upgrading from a version of Gradle Enterprise prior to 2019.4, be sure to also consult the release notes for all interim versions.

Temporarily degraded performance due to data reindexing

Upon upgrading, a data reindexing process will be initiated in the background with Gradle Enterprise being usable for its duration. CPU usage will be increased and performance may be slightly degraded. For large installations storing many build scans, the reindexing process may take several hours. During this time some builds may be omitted from the Tests dashboard.

Changes

Jan 13, 2020
  • [FIX] - Test duration in build scans are inaccurate
  • [FIX] - Test names containing underscores cannot be searched for in the tests dashboard
  • [FIX] - Builds occurring in an unknown timezone may fail to be indexed
  • [FIX] - Composite builds where an included build has a buildSrc project may fail to load
Dec 19, 2019
  • [FEATURE] - Tests dashboard visualizes flaky tests
  • [FEATURE] - Flaky tests are shown in build scans
  • [FIX] - Build scan ingest processing can be more efficient
  • [FIX] - Tests dashboard help does not convey that build count is constrained to builds that ran tests
  • [FIX] - Tests dashboard allows analyzing not only failed builds
  • [FIX] - Tests dashboard does not forward outcome criterion to data endpoints
  • [FIX] - Tests dashboard database indexes can require less storage space
  • [FIX] - Failures dashboard corrupts state of the outcome search filter
  • [FIX] - After encountering an error in build scans, going back doesn't restore a working state
  • [FIX] - Misleading zero console log line count is shown when console log is too big
  • [FIX] - Font rendering performance can be improved
  • [FIX] - Error is thrown when pressing enter on timeline search
  • [FIX] - Gradle artifact transforms create excessing logging in build scans