Maven Plugin Testing - In a Modern way - Part IV

Aug 30, 2020 · 5 min read ·

In the prevous part of the series - Maven Plugin Testing - In a Modern way - Part III we have seen how to define command line options. In this part we will take a deeper look which goals will run for each test case and how we can change that.

Let us start with simple example test case like the following:

1@MavenJupiterExtension
2class BaseIT {
3
4  @MavenTest
5  void the_first_test_case(MavenExecutionResult result) {
6     ...
7  }
8}

If we ran that integration test Maven will be called like the following:

1mvn -Dmaven.repo.local=<Directory> --batch-mode --show-version --errors package

We will concentrate on the part package in the above example. This is a life cycle phase of Maven. So what can we do if we like to call something like mvn .. verify instead? This can simply being achieved by using the @MavenGoal annotation like this:

1@MavenJupiterExtension
2class BaseIT {
3
4  @MavenTest
5  @MavenGoal("verify")
6  void the_first_test_case(MavenExecutionResult result) {
7     ...
8  }
9}

So let us take a deeper look onto the following example:

 1@MavenJupiterExtension
 2class BaseIT {
 3
 4  @MavenTest
 5  @MavenGoal("verify")
 6  void first(MavenExecutionResult result) {
 7     ...
 8  }
 9
10  @MavenTest
11  void second(MavenExecutionResult result) {
12     ...
13  }
14
15  @MavenTest
16  void third(MavenExecutionResult result) {
17     ...
18  }
19}

The test case first will be called with the phase verify whereas second and third will be called with the package. So this means you can overwrite the default behaviour for each test case separately.

Sometimes you want to execute Maven during an integration test like this: mvn clean verify or in general with multiple life cycle phase. This can be achieved by using multiple MavenGoal annotations as in the following example:

 1@MavenJupiterExtension
 2class BaseIT {
 3
 4  @MavenTest
 5  @MavenGoal("clean")
 6  @MavenGoal("verify")
 7  void first(MavenExecutionResult result) {
 8     ...
 9  }
10  ...
11}

Of course there are situations where you have a bunch of integration tests which needed to be executing the previously defined goals. This can handled by defining the @MavenGoal annotation on a class level instead like this:

 1@MavenJupiterExtension
 2@MavenGoal("clean")
 3@MavenGoal("verify")
 4class BaseIT {
 5
 6  @MavenTest
 7  void first(MavenExecutionResult result) {
 8     ...
 9  }
10
11  @MavenTest
12  void second(MavenExecutionResult result) {
13     ...
14  }
15
16  @MavenTest
17  void third(MavenExecutionResult result) {
18     ...
19  }
20}

This also gives the opportunity to let run a single test (or more than one) case within a test class with different goals depending on what you like to achieve.

Another example on how to define the @MavenGoal annotation on a class level which looks like this:

 1@MavenJupiterExtension
 2@MavenGoal("clean")
 3class GoalsOnClassIT {
 4
 5  @MavenTest
 6  @DisplayName("This will check the goal which is defined on the class.")
 7  void goal_clean(MavenExecutionResult result) {
 8    assertThat(result)
 9        .isSuccessful()
10        .out()
11        .info()
12        .containsSubsequence(
13            "Scanning for projects...",
14            "-------------------< com.soebes.katas:kata-fraction >-------------------",
15            "Building kata-fraction 1.0-SNAPSHOT",
16            "--------------------------------[ jar ]---------------------------------",
17            "--- maven-clean-plugin:3.1.0:clean (default-clean) @ kata-fraction ---"
18        );
19    assertThat(result)
20        .isSuccessful()
21        .out()
22        .warn().isEmpty();
23  }
24}

The next logical step is to create a meta annotation to make life easier. We would like to combine clean and verify within a single annotation @GoalsCleanVerify which can be done like this:

1@Target({ElementType.METHOD, ElementType.TYPE})
2@Retention(RUNTIME)
3@Inherited
4@MavenGoal({"clean", "verify"})
5public @interface GoalsCleanVerify {
6}

Such kind of meta annotation can be used on class level (defined by the annotation itself) as well as on method level like this:

1@MavenJupiterExtension
2@GoalsCleanVerify
3class MetaAnnotationGoalIT {
4 ...
5}

So now lets think about Part III where we defined this meta annotation:

1@Target({ElementType.METHOD, ElementType.TYPE})
2@Retention(RUNTIME)
3@Inherited
4@MavenOption(MavenCLIOptions.FAIL_AT_END)
5@MavenOption(MavenCLIOptions.NON_RECURSIVE)
6@MavenOption(MavenCLIOptions.ERRORS)
7@MavenOption(MavenCLIOptions.DEBUG)
8public @interface MavenTestOptions {
9}

This meta annotation can now being enhanced by the needed @MavenGoal annotations.

 1@Target({ElementType.METHOD, ElementType.TYPE})
 2@Retention(RUNTIME)
 3@Inherited
 4@MavenOption(MavenCLIOptions.FAIL_AT_END)
 5@MavenOption(MavenCLIOptions.NON_RECURSIVE)
 6@MavenOption(MavenCLIOptions.ERRORS)
 7@MavenOption(MavenCLIOptions.DEBUG)
 8@MavenGoal("clean")
 9@MavenGoal("verify")
10public @interface MavenTestOptions {
11}

This means you can define easily your set of annotation or combination of command line options and goals or more sophisticated combine it with @MavenJupiterExtension like this:

 1@Target({ElementType.METHOD, ElementType.TYPE})
 2@Retention(RUNTIME)
 3@Inherited
 4@MavenJupiterExtension
 5@MavenOption(MavenCLIOptions.FAIL_AT_END)
 6@MavenOption(MavenCLIOptions.NON_RECURSIVE)
 7@MavenOption(MavenCLIOptions.ERRORS)
 8@MavenOption(MavenCLIOptions.DEBUG)
 9@MavenGoal("clean")
10@MavenGoal("verify")
11public @interface MavenTestOptions {
12}

This will give us the option to use it like this:

 1@MavenTestOptions
 2class FailureIT {
 3
 4  @MavenTest
 5  void case_one(MavenExecutionResult project) {
 6    ..
 7  }
 8
 9  @MavenTest
10  void case_two(MavenExecutionResult result) {
11    ..
12  }
13
14  @MavenTest
15  void case_three(MavenExecutionResult result) {
16    ..
17  }
18
19  @MavenTest
20  @MavenOption(MavenCLIOptions.DEBUG)
21  void case_four(MavenExecutionResult result) {
22    ..
23  }
24
25}

This combines the given options with the defined goals in one single annotation. If you need to change something you have to fix only a single point.

So did we miss something? Yes we did. Sometimes you want to call your plugin with a separate goal like this:

1mvn org.test.maven.plugin:maven-x-plugin:goal

This can be achieved by using the @MavenGoal annotation like this:

 1@MavenJupiterExtension
 2class BaseIT {
 3
 4  @MavenTest
 5  @MavenGoal("org.test.maven.plugin:maven-x-plugin:goal")
 6  void first(MavenExecutionResult result) {
 7     ...
 8  }
 9  ...
10}

Now let us assume the given plugin is the one which should be tested via the given integration test. Then you need to define the correct groupId, artifactId, version and the correct goal. Unfortunately with each release of your plugin the version changes etc.

 1@MavenJupiterExtension
 2class BaseIT {
 3
 4  @MavenTest
 5  @MavenGoal("${project.groupId}:${project.artifactId}:${project.version}:goal-to-test")
 6  void first(MavenExecutionResult result) {
 7     ...
 8  }
 9  ...
10}

The placeholders ${project.groupId}, ${project.artifactId} and ${project.version} are exactly the information from your project pom in which you define the coordinates of the plugin your are developing. This makes sure only this plugin version is being used and not any other version is being tried to download from central or other repositories during your integration tests.

Good examples can be found in maven-invoker-plugin based integration tests.

So this it is for Part IV. If you like to learn more about the Integration Testing Framework you can consult the users guide. If you like to know the state of the release you can take a look into the release notes.

If you have ideas, suggestions or found bugs please file in an issue on github.

An example project which shows the previous example can be found on GitHub.