Virtual Training: Gradle Build Cache Deep Dive – November 29, 2022. Register now.
OSS Projects Revved Up by Gradle Enterprise
Gradle is proud to support key open-source projects with FREE instances of Gradle Enterprise.
Gradle Enterprise is quickly emerging as a de facto standard tool for build and test data analytics and as a source of acceleration technology for many of the most important open source projects. Here is a roundup of OSS projects that rely on Gradle Enterprise to improve build and test feedback cycle times and make troubleshooting more efficient by combining root cause analysis data with failure analytics.
Spring is the world’s most popular Java framework for making Java programming quicker, easier, and safer for everybody. Gradle Enterprise is being used by the Spring Boot, Spring Framework, and Spring Security projects. After the Spring Boot project migrated from Maven to the Gradle Build Tool and further optimized the Build Cache effectiveness with Gradle Enterprise, CI builds now take roughly 20 minutes on average, 3-4 times faster than before. Local builds are taking an average of 2.5 minutes, which is 20-30 times faster than before. The team has also started looking at Gradle Enterprise Failure Analytics to address flaky tests and other avoidable failures.
JetBrains Kotlin is a general-purpose, free, open source programming language initially designed for the JVM and Android and is the most used programming language for building JVM applications after Java. The JetBrains Kotlin Compiler open source project is built with Gradle Enterprise about 60,000 times every month. Gradle Enterprise was used to reduce median build time from about 10 minutes to about 3.3 minutes. JetBrains made their custom tasks cacheable and also optimized the configuration phase of the build with the help of Build Scan insights.
JUnit is the most widely used testing framework on the JVM. Given its widespread usage, JUnit needs to be thoroughly tested on all current JDKs and early access versions. Since enabling the Gradle Enterprise remote Build Cache, the average build time decreased by about 50%. Build Scans allow the JUnit team to collaborate on build failures and point out problems in pull requests. In addition, the JUnit team takes advantage of the Gradle Enterprise Test Failure analytics to stabilize flaky tests. More recently, the JUnit build has adopted Gradle Enterprise Test Distribution for CI and local builds to speed up slow integration tests.
Spock is a testing and specification framework for Java and Groovy applications. Spock’s beautiful and highly expressive specification language differentiates it from alternative solutions. Its JUnit runner makes Spock compatible with most IDEs, build tools, and continuous integration servers. Gradle Enterprise surfaced several performance issues in Spock’s Gradle build. Build Scans showed that tasks were not up-to-date with no code changes because of changing snapshot dependencies. The Gradle Enterprise task inputs comparison identified tasks that were not being pulled from the cache. After solving those problems, as well as a few others, Spock’s Gradle build could take full advantage of the Gradle Enterprise remote build cache. This brought CI build times for the Spock project from a median of 3 minutes per build to about 1 minute per build.
JHipster is a development platform to quickly generate, develop, and deploy modern web applications and microservice architectures. JHipster helps developers create deployments across a range of both frontend and backend languages and frameworks. JHipster’s project generator creates backend Java applications which in turn have their own individual build and test cycles. Gradle Enterprise’s build caching and scan features are now seamlessly available in generated JHipster projects, saving time for JHipster developers and their users. JHipster also makes use of the Build Scan functionality to more easily identify previously undetected areas for build time improvement as well as analyze build failures caused by differences in infrastructure.
Ratpack is a simple, capable, toolkit for creating high performance and scalable web applications built on Java and the Netty event-driven networking engine. It consists of a set of Java libraries for building scalable HTTP applications. Optional Gradle build time support makes building and testing applications a breeze. The Ratpack team relies on Gradle Enterprise Build Scans to efficiently debug build failures and on the remote build cache to accelerate building locally and in their CI pipeline.
Micrometer is a vendor-neutral instrumentation facade for JVM-based applications which provides multiple interfaces that developers can use to instrument their code without being locked into a monitoring solution. The dimensional data model, in combination with the timers, gauges, counters, distribution summaries, and long-task timers provided by the interfaces allows for searching and drilling down on metrics. Before optimizing the build with Gradle Enterprise the average build time for the project was 3 minutes; after optimizations the build time was less than 1 minute. Analysis of test-related task inputs found multiple opportunities for avoidance savings during test cycles, leading to further reduction in feedback cycle times. The insights gained from the Build Scan service have helped the team uncover multiple inefficient dependency repository lookup calls alongside improvements to the project’s overall cacheability. Build Scan dependency info, plugin insights, and deprecation warnings help the team ensure that build systems stay up to date with the latest version of the Gradle Build Tool and the related plugins that they use.
The OpenRewrite project is an ecosystem of utilities used to search and transform source code at a large scale. OpenRewrite’s platform contains numerous recipes which can be used to refactor code automatically. The platform can be used to enforce code best practices, upgrade code from legacy standards, automate the remediation of style violations, and perform many other tasks involving code transformation. The OpenRewrite team uses Gradle Enterprise to speed up their build and test cycles both locally and on CI, and to gain deep insights into the state of their builds and toolchain. The remote Build Cache has reduced their average build times on CI from 30-80% depending on the job run. Gradle Enterprise’s Trends & Insights capability has presented new opportunities to improve performance across their codebase and test performance analytics has allowed them to identify slow tests and take steps to speed them up. Additionally, the centralized and shareable approach that the Build Scan offers has reduced the amount of guesswork typically involved in remediation efforts and made troubleshooting far more data-driven than before.
Testcontainers assists developers in running complex tests by providing disposable instances of commonly used databases, web browser environments, and other components commonly used to support automated testing efforts. The Testcontainers community has adopted several Gradle Enterprise features to accelerate their build times and assist in collaboration. First, URL linking to parts of a Build Scan has been particularly useful for the distributed Testcontainers team, providing an easy way to share issue details instead of scrolling through Github action logs. Second, Task Input Comparison was used to maximize efficiency by discovering and removing impediments to caching such as unnecessary time stamps in build files. Finally, remote Build Cache, which replaced the existing caching mechanism, greatly simplified build logic by removing bespoke test caching logic and resulted in more accurate test caching.
The AndroidX project produces Android Jetpack, a suite of libraries that helps Android developers follow best practices, reduce boilerplate code, and write code that works consistently across Android versions. The AndroidX team uses Gradle Enterprise to speed up builds with enhanced caching, as well as gain deep insights into build performance. In one instance, the Gradle Enterprise Build Scan service helped the team identify and fix a pathological dependency resolution issue, reducing dependency download times on CI by almost 30 minutes. Gradle Enterprise also helped to identify and fix a misconfigured task that was causing cache misses.
The Micronaut project is a modern, JVM-based, full-stack framework for building modular, easily testable microservice and serverless applications. It runs smoothly on GraalVM and compiles to a native executable format that offers low startup times and a small memory footprint. Gradle Enterprise is a Tool and Infrastructure Partner to Micronaut on all 80 current projects. As a result of using Gradle Enterprise to optimize their build cache performance, CI build times have been reduced 50% on average. Further, Build Scan was used to identify and fix cache misses during that optimization process, and continues to help the team collaborate on issues. Finally, the team leveraged Test Failure Analytics to identify and reduce test flakiness.
OpenTelemetry is a collection of tools, APIs, and SDKs used to instrument, generate, collect, and export software telemetry data. Organizations use OpenTelemetry to analyze and better understand software performance and behavior. The OpenTelemetry team uses Gradle Enterprise to improve the developer experience for their community of contributors. Test Insights uncovered a number of flaky tests which have been fixed, and is being used extensively to keep test flakiness under control. OpenTelemetry experiences failures that have root causes that are often difficult to determine, such as those that originate from transitive dependencies. The team uses Task Input comparison to help optimize their builds and trace failures back to their root cause. OpenTelemetry reports that investigating cache misses has been much easier with Gradle Enterprise. Additional enhancements such as the removal of deprecation warnings and an improved shadowjar configuration have paid off in terms of build performance and consistency. Collectively, these improvements have taken OpenTelemetry’s median CI build time from 10 minutes down to 2 minutes, and Trends and Insights ensure that builds will remain as fast as possible.
XWiki is a free wiki software platform written in Java with a design emphasis on extensibility. XWiki is an enterprise wiki and includes WYSIWYG editing, OpenDocument-based document import/export, semantic annotations and tagging, and advanced permissions management. Gradle Enterprise’s local and remote cache capabilities have sped up XWiki Common Maven builds by 25%. In addition, Build Scan™ has helped bring visibility to builds with heavy downloading of dependencies, and the data provided by Failure Analytics has allowed the team to make data-driven decisions about which failures to prioritize and fix.
Grails is a powerful Groovy-based web application framework for the JVM built on top of Spring Boot. The Grails team, an early adopter and long-time user of the Gradle Build Tool, utilizes Gradle Enterprise to gain deep insights into their builds. This has helped the team keep Gradle up to date by providing observability during upgrades. Additionally, Grails is realizing significant benefits from build caching, achieving a 50% improvement on the project’s CI build time. Grails has used the observability features of Gradle Enterprise to optimize cache usage by identifying cache misses and remediating them, including misses caused by inconsistent dependency versions pulled from Maven local repositories.
Hibernate ORM is an ubiquitous and powerful object-relational mapping tool for Java. Hibernate maps Java classes to database tables, and Java data types to SQL data types. It also provides data query and retrieval capabilities. The Hibernate team used Gradle’s incremental build system to improve build performance by more than 70% on average. Gradle Enterprise build insights have helped the Hibernate team significantly optimize their build. Issues such as missing cacheable task annotations, timestamps in JAXB-generated headers, and tasks not taking advantage of Gradle’s incremental UP-TO-DATE capabilities have been revealed and resolved using Gradle Enterprise build observation features.
The Nokee project provides open source plugins for developers which improve support for native builds in Gradle. Nokee is using Gradle Enterprise to speed up their builds on average 66% by taking advantage of incremental building as well as local and remote caching. The insights provided by Gradle Enterprise allowed the Nokee team to identify several opportunities for improvement in their build process, including cache utilization, normalization of path declarations, OS-bound tasks with improper inputs, and classpath ordering issues.
Gradle has emerged as the build tool of choice for projects within the JVM ecosystem, including Kotlin. Gradle Build Tool has a comprehensive automated test suite that covers different operating systems and Java versions. The team runs about 70,000 builds a week. The build relies heavily on the remote Build Cache for build acceleration, Build Scans for finding performance regressions, flaky test management to identify and prioritize fixing the most flaky tests, and Test Distribution to further reduce build times.