Performance Continuity

Build Performance Continuity Solution Overview

With very few exceptions, organizations have no relevant and actionable insight into the performance and reliability of the toolchain their developers are using. The tech industry that has enabled other industries to become so efficient by providing data visibility and analytics has so far not applied productivity practices to itself. Develocity Performance Continuity performance profiling data and analytic techniques to optimize and sustain build performance levels over time, given code architecture, technology stack, and other constraints. It is now easy to (1) continuously monitor build and test performance metrics and trends and (2) observe and effectively determine the root cause of performance pathologies, bottlenecks, and regressions.

Performance Continuity works hand in hand with the other Develocity Performance Acceleration technologies—which include Build Cache, Test Distribution, and Predictive Test Selection—to deliver an integrated performance acceleration strategy. This means these technologies can be used to raise the theoretical maximum achievable performance and performance continuity techniques can then be used to sustain and minimize the variance of actual gains over time.

A critical mistake is to view these performance acceleration technologies as “fire and forget” solutions. Unmanaged, performance optimization opportunities will be squandered, and performance will regress overtime as the codebase grows in size and complexity, leading to potentially negative savings. Performance Continuity practices and the enabling technology provided by Develocity aim to prevent that.

Key Build Performance Continuity Solution Benefits

Build Performance Continuity Features

Develocity Performance Continuity provides comprehensive build and test performance profiling that is easily accessed and organized in a well-structured UI that facilitates root cause analysis for collaborative performance troubleshooting and continuous proactive improvement. 

This results in increasing (1) the motivation of developers to report problems since they are more likely to get addressed than reports with limited context and data, (2) the rate at which problems are addressed successfully since it is easier to detect the root cause, and (3) the ease with which performance problems can be prioritized and determined to be worth fixing since it is easier to determine the overall impact on the organization.

Individually and collectively, detailed build performance data described below can help to qualify decisions about where to invest in further build optimization and where specifically to focus your experiments.

Performance Trends Dashboard	Build Timeline Profile	Build Performance Details Profile
This is an interactive and multidimensional visualization of build performance metrics over day, week, and month time intervals and across builds that match your search criteria and filters. This allows easier discovery and analysis of builds with specific performance characteristics or anomalies related to build time, task parallelization, incremental build and build cache effectiveness.	The Build Scan build timeline provides an intuitive way to visualize the order in which build actions are executed, the critical path and concurrency of execution, execution duration, and details like time spent “snapshotting” inputs, compiling code, downloading dependencies, and running annotation processors. This makes it easier to assess where best to invest in optimization.	Build Scan provides a simple UI and record of the performance details for every build including total build time, initialization and config time versus execution time, total garbage collection time, and peak heap memory usage.
Configuration Time Profile	Dependency Resolution Profile	Build Action Execution Profile
Build Scan configuration time profiling provides total configuration time and breaks configuration execution time between the time spent compiling build scripts versus time spent applying plug-ins. This helps prioritize investments in minimizing configuration time and determine which bottlenecks to focus on like lifecycle callbacks.	Build Scan provides details on the time spent resolving dependencies for individual builds and Performance and Trend Dashboard visualizations show how much build time is spent obtaining dependencies. This is very useful for identifying the build time cost of ephemeral CI builds, poor network performance, or frequent dependency changes.	This provides an aggregated view of build execution details within Build Scan, like work avoidance statistics with supporting data (e.g., cache hit/miss metrics and incremental build savings). A ledger of Gradle Tasks, Maven Goals, or Bazel Targets is combined with these aggregate statistics so that the execution performance of each step of the build is understood.
Build Cache & Test Distribution Profile	Predictive Test Selection Profile	Network Activity Profile
Build Scan tracks cache hit/miss rates for each build. This data can be used to determine whether individual builds are taking full advantage of available remote and local caching resources, and view how much overhead is associated with caching. Test Distribution profiling information includes details on time saved resulting from parallel execution of tests.	The Predictive Test Selection Dashboard provides data accuracy and explainability metrics and details for the test sets it optimizes. Users can view the percentage of test failures that were correctly predicted and the overall work avoidance savings. Also, information on why Predictive Test Selection chose to run or skip a test can be viewed for each evaluated test.	Easily accessible network activity profile information includes names of all files downloaded, the size of all files downloaded and the download times. Knowing how much time is spent during a build waiting on network resources helps guide decisions about an organization’s artifact repository infrastructure strategy.
Switches & Infrastructure Profile	Test Performance (Slow Test Analysis) Analysis	Settings and Suggestions
Available at-a-glance on/off switch settings and execution environment information about the JDK used to execute the build, the machine’s host name, operating system, number of cores, memory allocation and more, is relevant to understanding build and test performance outcomes.	Test performance can be viewed on an individual-test and test-class level. Slowest test views identify the longest running tests within a single build and across builds to determine which tests are worth making faster. Filters can identify the slowest tests within a subset of a single task (e.g., within a particular test task or goal) while the Tests Dashboard makes identifying the slowest tests across builds easy.	Settings and suggestions provide further actionable insights like why a task was not cacheable, if parallel execution is enabled, or whether the build was run via the daemon which makes builds much faster.

Performance Continuity Engineering Framework

Here is a simple framework which demonstrates how the analytic data, insights, and performance profiling information available in Develocity can be used to achieve high levels of performance continuity. Without the Performance Continuity data provided by Develocity, it is almost impossible to find performance related issues that occur irregularly (like so-called flaky performance issues) because they are extremely difficult to reproduce. But with the data of an instrumented toolchain at your fingertips, this is now practical to pursue.

Performance Continuity In Action

Tests Dashboard showing the build search criteria and (mean execution time) filter used to identify and prioritize slowest tests for remediation.

Use the Trends Dasbhoard to observe when your Maven or Gradle build times are regressing and the Performance Dashboard to understand why.

Combine Test Distribution with other Performance Acceleration Technologies for a Force-Multiplier Effect

While Performance Continuity does not depend on other Performance Acceleration technologies available in Develocity to be useful in observing and tuning your build performance achieved through existing processes and tools, combining Performance Continuity with Build Cache, Predictive Test Selection and Test Distribution can provide a force multiplier effect on feedback cycle times as illustrated below.