Cucumber JVM: Advanced Reporting 3. Handling FileNotFound or Invalid JSON Errors

Introduction

Since Advanced Cucumber JVM reporting was initially introduced and then enhancements were provided the most frequent question was related to the error when input file was not found or it was improperly formatted. Normally such error was accompanied with the output like:

java.io.IOException: Input is invalid JSON; does not start with '{' or '[', c=-1
 at com.cedarsoftware.util.io.JsonReader.readJsonObject(JsonReader.java:1494)
 at com.cedarsoftware.util.io.JsonReader.readObject(JsonReader.java:707)
 at com.github.mkolisnyk.cucumber.reporting.CucumberResultsOverview.readFileContent(CucumberResultsOverview.java:81)
 ...

or

java.io.FileNotFoundException
 at com.github.mkolisnyk.cucumber.reporting.CucumberResultsOverview.readFileContent(CucumberResultsOverview.java:76)
 at com.github.mkolisnyk.cucumber.reporting.CucumberResultsOverview.executeFeaturesOverviewReport(CucumberResultsOverview.java:189)
 ...

As it is one of the most frequent questions I decided to create separate post explaining reasons of it and the way to fix.

Major reasons of problem

There are a few reasons why such errors appear. Mainly they are:

• Report generation is called before the source report is generated
• Report generation is targeted to wrong report location

Let's take a closer look at all of those items.

Report generation is called before the source report is generated

Some people try to add report generation as the part of hooks or other post-actions which are mainly annotated with @After, @AfterClass or similar annotations. If you use any form of hooks or built-in post-actions you should know 2 major things:

1. Cucumber runner doesn't call default JUnit Before- and After- methods - that's why there are some pre-defined hooks with custom annotations for that.
2. Cucumber Reporting solution uses reports generated after entire Cucumber suite run completes including post-condition events - it means that if you add report generation code into @After method the required input file may even appear but it wouldn't be populated with the data yet. Data will appear when Cucumber completes it's execution entirely.

So, in order to generate Advanced Cucumber reports properly

DO NOT add report generation instructions into hooks or any other standard after-methods.

Report generation is targeted to wrong report location

In some cases we may use improper references for files. E.g. we have our Cucumber class like:

@RunWith(Cucumber.class)
@CucumberOptions(plugin = { "html:target/cucumber-html-report",
        "json:target/cucumber.json", "pretty:target/cucumber-pretty.txt",
        "usage:target/cucumber-usage.json",
        "junit:target/cucumber-results.xml"
        },
        features = { "./src/test/java/com/github/mkolisnyk/cucumber/features" },
        glue = { "com/github/mkolisnyk/cucumber/steps" },
        tags = {"@consistent"})
public class SampleCucumberTest {
}

and then somewhere we generate detailed report like:

        CucumberDetailedResults results = new CucumberDetailedResults();
        results.setOutputDirectory("target/");
        results.setOutputName("cucumber-results-width");
        results.setSourceFile("./src/test/resources/cucumber.json");
        results.setScreenShotLocation("../src/test/resources/");
        results.setScreenShotWidth("200px");
        results.executeDetailedResultsReport(true, false);

The highlighted fragments of code should point to the same location. Otherwise we either have FileNotFoundException or wrong report generated at all.

Major Checkpoints and Resolution

So, what do we need to overcome all those problems? There are several steps you should go through to make sure that you include reporting in proper way.

Make sure you use proper reporting library version

New features are always to come and versions are updated as soon as some certain number of implemented features is implemented. At the moment when this post is being written the current version is 0.0.11 and it can be included as Maven dependency using the following construction:

<dependency>
    <groupId>com.github.mkolisnyk</groupId>
    <artifactId>cucumber-reports</artifactId>
    <version>0.0.11</version>
</dependency>

or Gradle dependency:

'com.github.mkolisnyk:cucumber-reports:0.0.11'

Also, you can find the latest version of the reporting library in the Maven Central repository via any available web viewers, e.g. at search.maven.org or at mvnrepository.org.

Use extended Cucumber reporting functionality

The library itself includes Cucumber runner extension which includes reports generation at proper time. Here is the example of Extended Cucumber runner use:

import org.junit.runner.RunWith;

import com.github.mkolisnyk.cucumber.runner.ExtendedCucumber;
import com.github.mkolisnyk.cucumber.runner.ExtendedCucumberOptions;

import cucumber.api.CucumberOptions;

@RunWith(ExtendedCucumber.class)
@ExtendedCucumberOptions(jsonReport = "target/cucumber.json",
        retryCount = 3,
        detailedReport = true,
        detailedAggregatedReport = true,
        overviewReport = false,
        toPDF = false,
        outputFolder = "target")
@CucumberOptions(plugin = { "html:target/cucumber-html-report",
        "json:target/cucumber.json", "pretty:target/cucumber-pretty.txt",
        "usage:target/cucumber-usage.json",
        "junit:target/cucumber-results.xml"
        },
        features = { "./src/test/java/com/github/mkolisnyk/cucumber/features" },
        glue = { "com/github/mkolisnyk/cucumber/steps" },
        tags = {"@consistent"})
public class SampleCucumberTest {
}

So, in case of such extension use we make sure we have our reports processing done at the moment when standard Cucumber reports have already been generated.

Make sure both CucumberOptions and ExtendedCucumberOptions refer to the same report files

Advanced reporting is mainly about Cucumber standard reports post-processing so we should make consistent reference between values in CucumberOptions and ExtendedCucumberOptions annotations. Mainly it's about JSON file paths as for CucumberOptions it is output file while ExtendedCucumberOptions annotation utilizes this file as an input file.

Additionally, make sure that

Usage report operates with the output file different from the one required by other reports. Also, usage report is not generated for dry runs.

So, pay attention which files you refer to.

Summary

By following the above rules and getting basic understanding of how this solution works we can mitigate a lot of silly problems. Also, the Cucumber Reporting Documentation pages are available and they will be populated as soon as new features arrive. So, you can refer to it as well.