Flaky tests, be gone!
Flaky Test Detection captures data during testing to provide you with a variety of information on test performance. By analyzing test retries and comparing executions of tests across builds, it surfaces insights about test flakiness. With this data at your fingertips, you can find and mitigate erratically failing tests before they become a drain on your productivity.
Restore confidence in tests
When you trust your test suite, you can confidently modify the codebase without fearing you'll miss bugs during the testing phase.
Make better use of resources
Developers re-run builds they know have flaky tests since the build will often pass on the second or third try. Eliminating flakiness means you're re-running fewer builds, saving developer time and compute resources.
Improve software quality
A flaky test can be caused by a bad test or it can be caused by bad code. Proactively addressing flaky tests allows you to focus on the real issues in the application—and as a result, ship a better, more stable product.
Boost developer morale
When you don't know whether a test failed because it's flaky or because there's a real problem in the code—well, that's downright frustrating. Identifying and eliminating flaky tests means a smoother development process and happier developers.
Identify
Develocity automatically monitors a build's test results and identifies “flaky” tests—tests that have both passed and failed with no change in the inputs.
Investigate
Find out why an erratic test is failing using data from Build Scan®. Get a detailed examination of each build and test run so you can determine in what context the test passed or failed, and identify why that test was flagged as flaky.
Resolve
Use the Tests API to fetch a list of all flaky tests from a particular time period and assign them out for resolution.
Monitor
Ensure flaky tests don't reappear with on-demand monitoring. You can keep an eye on the health of your tests over time, and so be alert to any potential regressions.
For this particular problem, data is key. Investing time in keeping track of the previous and historic runs helps you understand where tests are flaky. This allows you then to perhaps prioritize those tests on subsequent runs so you can get faster feedback cycles and fail earlier.”
Robert Keith, JP Morgan Chase
Read moreLearn more about Flaky Test Detection
Get three key elements for your flaky test remediation strategy
This blog post walks through a multi-faceted approach to flaky tests that integrates strategy, process, and resource alignment.
Take a tour of Develocity's troubleshooting features
Learn more about how Develocity helps troubleshoot build and test failures with Build Scan®, Flaky Test Management, and Failure Analytics.
Learn five ways to use Develocity to manage flaky tests
Find out how Develocity can help identify and manage test failures and flakiness in this blog post.
