Dependency issues are frustrating. The rules are complex and without the right data, determining exactly what caused the issue can take time and energy. The new dependency visualization feature in build scans can help.
Snapshot and Other Dynamic Versions
Say I’m working on a project built with Maven, in this case Apache Maven itself. I want to try out the latest snapshot version of Apache Commons CLI to see if it’s compatible. I update the pom file to use the 1.5-SNAPSHOT. Depending on the repository’s update policy, Maven may use the snapshot from the local repository or check for a new one in the remote repository. If the resolved version is not the same as the latest version in the remote repository, it may explain why a certain feature or fix is missing. To figure out which snapshot version was added to the Apache Maven Distribution runtime classpath, I look at the dependencies section of the build scan (Figure 1).
Figure 1. Snapshot versions
All of the dependencies for each project are shown in the graph under their respective scope. Each dependency can be expanded to show any transitive dependencies. The highlighted line in Figure 1 shows that for the Apache Commons CLI 1.5 snapshot version I requested (left of the arrow), the resolved version was 1.5-20200111.144009-53 (right of the arrow). The entire list is rather long so I want to see just the versions of Apache Commons CLI. I type the dependency name into the search bar above for a filtered list and confirm that the same version is used for all of the projects.
So far so good. That was the version number I was expecting. Out of curiosity, I click on the details view for the Apache Maven Distribution project to look at what repositories were configured.
I see Maven central, Apache snapshots, and Sonatype Nexus Snapshots. To dig into what settings the build uses for the Apache snapshots repository, I go to the repositories details (the blue link seen at the top-center of Figure 1).
Looking at the settings for the Apache snapshots repository, I see that it’s set to update daily. If the Apache Commons CLI (or an internal library) is under active development and pushing new snapshots with changes I want to incorporate, I would not have the latest changes. Of course, this is also true of any other dynamic dependency declaration like ranges. While I’m experimenting with dynamic versions, I will set the snapshots update policy to always.
Conflict Resolution
However, using substituted dependency versions like a snapshot or range comes with its own set of risks. Changing the dependency version requires that we test that the new version will work as expected in every subproject and every classpath. The upgrade may work fine for compilation but not runtime or similarly will work for the core app but not the distribution. It could also be that we know a particular version of a dependency is faulty but we aren’t aware that it is pulled in transitively.
Continuing on with our Apache Maven example, I’m not ready to upgrade to the snapshot on Apache Maven Distribution. I want to rollback to version 1.4 so I declare version 1.4 in the project’s pom.
Figure 2. Conflict Resolution
Figure 2 shows how Maven’s dependency resolution engine enforces the version I specified in the Apache Maven Distribution pom, 1.4, in the transitive dependencies. The list of all the dependencies is rather long, so to find the versions more easily, I set the resolution type filter to ‘Selected different from requested’. I can also see that one of the slf4j dependencies used a different version than I expected and I could investigate that further if I wanted.
Dependency Comparison
Dependency issues are seldom that straightforward. Frequently, we don’t know exactly which dependency is responsible for a given problem. I’m sure we’ve all seen situations where the version of a dependency suddenly changed and broke the build.
Since my experiment upgrading Apache Commons CLI worked just fine, I figure it’s safe to upgrade to the latest versions of all the dependencies and incorporate the latest security patches and fixes. I run mvn versions:use-latest-versions.
Now the Maven Embedder project is failing to compile because there is an issue with slf4j-simple. To see all the dependencies that were updated by versions:use-latest-versions goal, I click the ‘Compare build scan’ button (bottom left side of build scan) and find the build from Figure 1.
Figure 3. Dependency Comparison
Figure 3 highlights the places where the dependencies vary. The blue indicates the values from my new failing build and the green values come from the build in Figure 1. Since I ran an update of all the dependencies in the entire project the list is quite long so I filter the list by the Maven Embedder project and the compile scope. There, I see the difference in the slf4j-simple library.
Since it’s an update to the alpha of a major version update, I suspect I can get away with just changing that dependency back to the last stable version so I update the pom. Alas, the Maven Embedder project is now working but other projects are failing. After using these steps to wade through a few more dependency issues, I finally have a passing build again with all the latest security patches and fixes!
Conclusion
Build scans are powerful debugging tools that display actionable insights for both local and CI builds all in one report. The Dependencies tab available with Maven Develocity extension 1.4+ and Develocity 2020.1 provides Maven users with a graphical representation of all dependencies including transitive dependencies. The searchable list shows version conflicts, dependency types such as optional, pom, and effective pom, repository details, and other information you don’t get when running mvn dependency:tree from the command line. For changes between builds, the build comparisons show exactly which dependencies changed.
The Apache Maven project is open source so you can recreate the scans from examples 1 and 2 yourself by following these steps:
- Check out the Apache Maven project from GitHub
- Add the Develocity Maven Extension
- Run
mvn verify -Drat.skip=trueand follow the prompts - Go to the link at the end of the build output
If you aren’t already using Develocity, you can try this out with the free-to-use scans.gradle.com service which provides some of the features of Develocity. To enable dependency comparison, have complete on-premise data privacy, and get other features like build caching, the Trends and Performance Dashboards, Test Failures Dashboard and Flaky Test Analysis sign up for a trial of Develocity 2020.1. During your self-hosted free trial, you’ll also get in-house consulting from Maven experts on the Develocity team to help improve the speed and reliability of your Maven builds.