Ploblem Solved: Cucumber-JVM running actions before and after tests execution with JUnit

Background

It is frequent case when we need to do some actions before and/or after entire test suite execution. Mainly, such actions are needed for global initialization/cleanup or some additional reporting or any other kind of pre/post-processing. There may be many different reasons for that and some test engines provide such ability, e.g. TestNG has BeforeSuite and AfterSuite annotations, the JUnit has test fixtures which may run before/after test class (it's not really the same but when we use Cucumber-JVM it's very close to what we need).

Problem

The problem appears when you want to add some actions at the very latest or very earlies stages of tests execution and you use Cucumber-JVM with JUnit. In my case I wanted to add some reports post-processing to make an advanced Cucumber report generation. In this case JUnit fixtures didn't help as AfterClass-annotated method runs before Cucumber generates final reports.

At the same time adding @BeforeAll and @AfterAll hooks question raised on Cucumber side as well. And there was even some solution proposed. Unfortunately, authors decided to revert those changes as there were some cases when it does not work.

So, the problem is that I need something to run after entire Cucumber-JVM suite is done but neither Cucumber nor JUnit gives me built-in capability for doing this.

Solution

JUnit itself gives an ability to customize test runner classes. Actually, Cucumber runs JUnit tests via Cucumber JUnit runner which is already customized. But we can extend Cucumber capabilities by wrapping Cucumber runner inside our custom runner. That doesn't seem to be the smoothest solution as ideally this functionality should be provided by Cucumber itself. But if the solution looks ugly while without it things are getting even worth then this solution is no longer ugly. So, the problem is solved in three major steps:

1. Create BeforeSuite and AfterSuite annotations
2. Create wrapper on Cucumber JUnit runner
3. Use it

So, let's bring some more details on that.

Create BeforeSuite and AfterSuite annotations

The @BeforeSuite and @AfterSuite annotations are simply annotations without any parameter which are assigned to specific method. So, their implementation may look like:

package org.sample.cucumber.annotations;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.METHOD })
public @interface AfterSuite {

}

and the same for @BeforeSuite:

package org.sample.cucumber.annotations;

import java.lang.annotation.ElementType;
import java.lang.annotation.Retention;
import java.lang.annotation.RetentionPolicy;
import java.lang.annotation.Target;

@Retention(RetentionPolicy.RUNTIME)
@Target({ElementType.METHOD })
public @interface BeforeSuite {

}

So, now we can annotate some methods. Let's go to the next step.

Create wrapper on Cucumber JUnit runner

Custom runner implementation has several ideas on the back of it. They are:

• Extended Cucumber wrapper eventually runs Cucumber runner itself
• In addition to Cucumber runner invoke we look for methods annotated with @BeforeSuite and @AfterSuite annotations. If there are such methods we run them before/after invoking actual Cucumber runner. The sample implementation looks like:

package org.sample.cucumber;

import java.lang.annotation.Annotation;
import java.lang.reflect.Method;

import org.junit.runner.Description;
import org.junit.runner.Runner;
import org.junit.runner.notification.RunNotifier;

import org.sample.cucumber.annotations.AfterSuite;
import org.sample.cucumber.annotations.BeforeSuite;

import cucumber.api.junit.Cucumber;

public class ExtendedCucumberRunner extends Runner {

    private Class clazz;
    private Cucumber cucumber;

    public ExtendedCucumberRunner(Class clazzValue) throws Exception {
        clazz = clazzValue;
        cucumber = new Cucumber(clazzValue);
    }

    @Override
    public Description getDescription() {
        return cucumber.getDescription();
    }

    private void runPredefinedMethods(Class annotation) throws Exception {
        if (!annotation.isAnnotation()) {
            return;
        }
        Method[] methodList = this.clazz.getMethods();
        for (Method method : methodList) {
            Annotation[] annotations = method.getAnnotations();
            for (Annotation item : annotations) {
                if (item.annotationType().equals(annotation)) {
                    method.invoke(null);
                    break;
                }
            }
        }
    }

    @Override
    public void run(RunNotifier notifier) {
        try {
            runPredefinedMethods(BeforeSuite.class);
        } catch (Exception e) {
            e.printStackTrace();
        }
        cucumber.run(notifier);
        try {
            runPredefinedMethods(AfterSuite.class);
        } catch (Exception e) {
            e.printStackTrace();
        }
    }

}

Additional attention should be paid to the highlighted code. It doesn't use any object instance which means that @BeforeSuite and @AfterSuite annotations should be applied to static methods in order to make things working. OK, we're almost there. Here it goes the last step.

Use it

Now we can define our Cucumber test suite with custom runner. In JUnit/Cucumber-JVM each test suite corresponds to single JUnit class. So, we can declare something like:

package org.sample.cucumber;

import org.junit.runner.RunWith;

import org.sample.cucumber.annotations.AfterSuite;
import org.sample.cucumber.annotations.BeforeSuite;
import org.sample.cucumber.ExtendedCucumberRunner;

import cucumber.api.CucumberOptions;

@CucumberOptions(
        plugin = {"html:target/cucumber-html-report",
                  "json:target/cucumber.json",
                  "pretty:target/cucumber-pretty.txt",
                  "usage:target/cucumber-usage.json"
                 },
        features = {"output/" },
        glue = {"org/sample/cucumber/glue" },
        tags = { }
)
@RunWith(ExtendedCucumberRunner.class)
public class SampleTestClass {
    @BeforeSuite
    public static void setUp() {
        // TODO: Add your pre-processing
    }
    @AfterSuite
    public static void tearDown() {
        // TODO: Add your post-processing
    }
}

Highlighted parts are the places where we use our custom objects.

Generally, that's it! Now you have your own way to define pre/post conditions which are performed at the very beginning/at the end of entire test suite which works for JUnit in combination with Cucumber-JVM.