Search

Saturday 13 June 2015

Cucumber JVM: Advanced Reporting 2. Detailed Report (HTML, PDF)

Cucumber JVM: Advanced Reporting 2. Detailed Report (HTML, PDF)

Introduction

Previously I've described basic report samples based on Cucumber-JVM. Since I'm using this reporting solution on practice quite frequently the number of requirements and enhancements grows so since last time I had to add some more additional features which can be in use.

Some of new features are:

  • Cucumber extension to support failed tests re-run
  • Detailed results report which supports the following:
    • Detailed results report generation in HTML format
    • Detailed results report generation in PDF format
    • Screen shot are included into both the above reports
All those new features are included into 0.0.5 version, so we can add Maven dependency:
<dependency>
 <groupId>com.github.mkolisnyk</groupId>
 <artifactId>cucumber-reports</artifactId>
 <version>0.0.5</version>
</dependency>
or the same thing for Gradle:
'com.github.mkolisnyk:cucumber-reports:0.0.5'

Since Cucumber failed tests re-run functionality was described before in this post I'll concentrate more on detailed results report features.

Why do we need the detailed results report?

First question which may appear here is: why do we need such report? Most of the details can be retrieved from standard Cucumber HTML report. Well, main reason is that existing HTML results report isn't enough and on practice we need some additional features. Some of them are:

  • Report itself should have minimal external dependencies (e.g. styles, scripts) so that it is easier to publish it as build artifact
  • It is very useful to have report showing screen shots on every error encountered
  • The entire report should also have some portable and self-contained format so that it can be sent via e-mail. Thus, PDF format can be quite useful for that
Also, standard HTML report looks pretty "standard" and empty. So, I've added some features to make new detailed report more convenient to read and navigate through the list of test results.

Detailed Report Structure

The image below shows how detailed report looks like:

It contains 3 major sections:
  • Overview
  • Table of Contents
  • Detailed Results Report

Overview section

Overview section contains aggregated data on entire run results. In particular it shows the number of passed/failed/skipped features/scenarios and steps with the % of passed items in comparison to entire number of items. Here is how typical overview section looks like:

PassedFailedUndefined%Passed
Features43057.14
Scenarios476088.68
Steps40662892.27

Table of Contents section

Table of contents is the hyper-linked list of features and scenarios highlighting the execution status. It is convenient addition to the report which provides possibility to navigate to each specific scenario result. Thus we have fast way to get to the test we're interested in.

Detailed Results Report section

Detailed results report looks similar to standard HTML report and simply contains the description of steps with the status and detailed stack trace is case of error. Also, every scenario result contains hyperlink to the table of contents. Thus we can easily navigate to scenario we need and then switch to any other scenarios we're interested in.

How to Generate Detailed Report

The report generation is based on standard Cucumber JSON results report post-processing. So, before generating detailed results report we should specify which results report file we should use for report generation. So, typical Java code for report producing looks like:

        CucumberDetailedResults results = new CucumberDetailedResults();
        results.setOutputDirectory("target/");
        results.setOutputName("cucumber-results");
        results.setSourceFile("./src/test/resources/cucumber.json");
        results.executeDetailedResultsReport(false);
This code produces detailed report stored at the following path: target/cucumber-results-test-results.html.

Adding Screen Shots

Often it's very useful to have the screen shot on error to see what was the real problem which caused error. It's not the silver bullet but it's definitely helpful for analysis. Since currently described report is produced as the result of post-processing but not as the part of hooks the way screen shots are generated are out of report generator scope. We simply should assume that we already have some folder containing screen shot files which fit the following rules:

  • Image files are of PNG format
  • Each failed test contains only one screen shot as we normally have only one error on failed test. Maybe in the future I'll add some additional options to handle multiple files but currently it is the way we do things now.
  • Screen shot file names are taken from the names of corresponding scenario IDs where special characters like ; or spaces are replaced with underscore.
Once we meet such requirements all we need is to define such screen shot locations. Thus, the Java code producing detailed report now looks like:
        CucumberDetailedResults results = new CucumberDetailedResults();
        results.setOutputDirectory("target/");
        results.setOutputName("cucumber-results");
        results.setSourceFile("./src/test/resources/cucumber.json");
        results.setScreenShotLocation("../src/test/resources/");
        results.executeDetailedResultsReport(false);
The highlighted part defines where to look for screen shots.

PDF Generation

And the last feature is the ability to generate file in PDF format. Actually, when we generate HTML report with all the above features it is still hard to transport as it may contain references to image files. In order to make the report more portable (e.g. if we need to send it then via e-mail). In order to reach this we simply need to convert generated detailed report HTML to PDF file. This flow is handled by the parameter of the executeDetailedResultsReport method. If it is set to true the PDF report is generated in addition to HTML one. If it is false only HTML report is generated. So, in order to produce PDF output we should modify previous code sample this way:

        CucumberDetailedResults results = new CucumberDetailedResults();
        results.setOutputDirectory("target/");
        results.setOutputName("cucumber-results");
        results.setSourceFile("./src/test/resources/cucumber.json");
        results.setScreenShotLocation("../src/test/resources/");
        results.executeDetailedResultsReport(true);
The above example will produce PDF file located by the path: target/cucumber-results-test-results.pdf.

Summary

That were additional Cucumber reporting features introduced recently. So, currently we have reports which can be set as e-mail message body + some additional reports which can be provided as attachments. This is good foundation for quite convenient notification system where we don't need to keep all our reports as build artifacts as well as now we have one major report where we can get both overview and detailed information. The list of enhancements will grow as long as more additional requirements appears. But the stuff we have right now is already good.

89 comments:

  1. I am not sure where I should add these code. I have tried with the hook. Its not working. Is there a github project that i can look at

    Thanks

    ReplyDelete
    Replies
    1. This particular sample works as the post-processor for the Cucumber results. So, it should be executed after Cucumber completely finishes and generates the report. So, as a minimum you should extend Cucumber runner somehow like this: http://mkolisnyk.blogspot.co.uk/2015/05/ploblem-solved-cucumber-jvm-running.html
      Currently this is Unit-Only solution. You need to run this sample as the part of AfterAll action.
      The project itself is available on GitHub: https://github.com/mkolisnyk/cucumber-reports
      It has an extended Cucumber runner class with extended Cucumber options annotation included. With this help we can define all necessary reports in static way

      Delete
  2. Thank you so much for clarification

    ReplyDelete
  3. hi Kolesnik,
    I have implemented your code version "Cucumber Reports » 0.0.10" and able to generate .html detailed report where as .pdf is generated with aggregate details similar to "cucumber-results-agg-test-results.html".
    Please help how to solve this.

    Thanks,
    Anite

    ReplyDelete
    Replies
    1. Hi Anite,
      Yes, aggregated report is the final report which is also targeted to be converted into PDF so that it can be e-mailed. In order to help you solve you problem, please, specify what actual problem you have because looks like this part is missing

      Delete
    2. Hi Kolesnik,

      I am not able to generate detailed report in .pdf format using the following code
      CucumberDetailedResults results = new CucumberDetailedResults();
      results.setOutputDirectory("target/");
      results.setOutputName("cucumber-results");
      results.setSourceFile("target/cucumber1.json");
      results.executeDetailedResultsReport(true,false);

      CucumberResultsOverview results1 = new CucumberResultsOverview();
      results1.setOutputDirectory("target");
      results1.setOutputName("cucumber-results");
      results1.setSourceFile("target/cucumber1.json");
      results1.executeFeaturesOverviewReport();

      CucumberUsageReporting report = new CucumberUsageReporting();
      report.setOutputDirectory("target");
      report.setJsonUsageFile("target/cucumber-usage1.json");
      report.executeReport();

      Delete
    3. OK. First of all, I need to mention that aggregated report is mainly needed if we use failed tests re-run functionality. In this case detailed report definitely may contain repetitive entries for tests which were re-run. Aggregation simply collapses duplicating test results into one. So, the only case when aggregated report is different from detailed report without aggregation is when you use failed tests re-run functionality. In all other cases this statement:

      results.executeDetailedResultsReport(true,false);

      produces identical report to this one:

      results.executeDetailedResultsReport(true,true);

      So, it looks like your situation (if I understand it correctly).

      There may be another case if you have several expressions generating both aggregated and non-aggregated report. E.g. something like this:

      CucumberDetailedResults results = new CucumberDetailedResults();
      results.setOutputDirectory("target/");
      results.setOutputName("cucumber-results");
      results.setSourceFile("target/cucumber1.json");
      results.executeDetailedResultsReport(true,false);
      results.executeDetailedResultsReport(true,true);

      Note that in this sample we generate 2 HTML reports but with different aggregation switch (see second parameter values). In such instruction the names of HTML reports are different but PDF file name would be the same, so second executeDetailedResultsReport call would overwrite PDF generated by previous call. So, eventually you'll get only one version of the PDF.

      If PDF fails to generate at all it's very likely there would be a stack trace in console output. If this is the case it would be great if you paste it.

      Delete
    4. Hi Kolesnik,

      Thank you very much for your prompt reply!!!

      I understood the difference between aggregated and non aggregated reports and i am using the following code to generated detailed non aggregated report

      results.executeDetailedResultsReport(true,false);

      pdf is generated with headings alone with empty data as shown below, whereas html report is good.
      Please help me to solve this issue

      Overview
      Passed Failed Undefined %Passed
      Features 0 0 0 NaN
      Scenarios 0 0 0 NaN
      Steps 0 0 0 NaN
      Overall Duration: 0h 00m 00s
      Table of Contents
      Detailed Results Report

      No errors in console output:
      ---------------------------------------

      Running Scenario:
      US ---- ACP Add Network Started
      INFO [main] (Log.java:48)- US ---- ACP Add Network Started
      key ----> acp.account
      INFO [main] (Log.java:48)- key ----> acp.account
      Data----->Automation-Client
      INFO [main] (Log.java:48)- Data----->Automation-Client
      Time taken to load element : ACPOrganizationLookup.Client is : 9.41700 seconds
      Error - Required: Please fill out this field before submitting

      1 Scenarios ( [32m1 passed [0m)
      6 Steps ( [32m6 passed [0m)
      2m24.544s

      0 Scenarios
      0 Steps
      0m0.000s

      Delete
    5. Hi Kolesnik,

      Thanks for your prompt and explanatory reply!!!
      I understand aggregated and non-aggregated reports generation, and was able to generated in html format. I am trying to generated non aggregated detailed report .pdf using the following code, but the pdf is generated with only heading and no data/details is present whereas html file contains all details.

      CucumberDetailedResults results = new CucumberDetailedResults();
      results.setOutputDirectory("target/");
      results.setOutputName("cucumber-results");
      results.setSourceFile("target/cucumber1.json");
      results.executeDetailedResultsReport(true,false);

      There are no errors in console output. Please help me to solve this issue.
      Is there a way to add extra column in the detailed report to show error details for failed steps/scenarios.

      Thanks in advance,
      Anite

      Delete
    6. Hi Anite,
      This is now looks like good issue description and comments here are not the most appropriate place to process it. I suggest you to raise an issue on related GitHub project. Link to issues is here:

      https://github.com/mkolisnyk/cucumber-reports/issues

      I will also need that target/cucumber1.json file to test. If it's not so big, you can drop file content as the comment for the issue. I'm already working on some other fixes so new version of library is about to come soon. So, you problem can be resolved as the part of current version activities.

      Key point is that JSON file. When PDF is generated it simply takes generated HTML and converts it to PDF format. The major problem why something goes wrong in this process is various special characters. That's why it is important to obtain the JSON file which causes you problem.

      Please, let me know if it works for you

      Delete
  4. Hi Kolesnik,

    I tried implementing cucumber-report, however I get below error while i execute code.

    com.asite.adoddle.testrunner.RunCukesTest Time elapsed: 0.03 sec <<< ERROR!
    java.io.IOException: Input is invalid JSON; does not start with '{' or '[', c=-1
    at com.cedarsoftware.util.io.JsonReader.readJsonObject(JsonReader.java:1494)
    at com.cedarsoftware.util.io.JsonReader.readObject(JsonReader.java:707)
    at com.github.mkolisnyk.cucumber.reporting.CucumberResultsOverview.readFileContent(CucumberResultsOverview.java:81)


    Below is my Runner class,

    public class RunCukesTest{

    @AfterClass
    public static void generateReport() throws Throwable
    {
    CucumberResultsOverview results = new CucumberResultsOverview();
    results.setOutputDirectory(".//target");
    results.setOutputName(".//target//cucumber-results");
    results.setSourceFile(".//target//cucumber.json");
    results.executeFeaturesOverviewReport();
    }
    }

    I verified that my JSON is not empty and validated as well on http://jsonlint.com/.

    ReplyDelete
    Replies
    1. This error appears because at the moment this code was invoked the actual JSON file wasn't populated yet as all data is filled after all tests and all after-actions are done. To overcome this problem you can use Cucumber extension from the same library. The test code looks like:

      import org.junit.runner.RunWith;

      import cucumber.api.CucumberOptions;

      import com.github.mkolisnyk.cucumber.runner.ExtendedCucumberOptions;
      import com.github.mkolisnyk.cucumber.runner.ExtendedCucumber;

      @RunWith(ExtendedCucumberRunner.class)
      @ExtendedCucumberOptions(
      jsonReport = "build/cucumber.json",
      jsonUsageReport = "build/cucumber-usage.json",
      outputFolder = "build/",
      detailedReport = true,
      detailedAggregatedReport = true,
      overviewReport = true,
      usageReport = true,
      coverageReport = false,
      retryCount = 0,
      screenShotLocation = "screenshots/",
      screenShotSize = "300px",
      toPDF = true
      )
      @CucumberOptions(
      plugin = {"html:build/cucumber-html-report",
      "junit:build/cucumber-junit.xml",
      "json:build/cucumber.json",
      "pretty:build/cucumber-pretty.txt",
      "usage:build/cucumber-usage.json"
      },
      features = {"src/test/java/com/mkolisnyk/common/features"},
      glue = {"com/mkolisnyk/common/steps"},
      tags = {"@test"}
      )
      public class AcceptanceTest {
      }

      Of course, you should set proper references to paths and files. Nota that I'm using mix of CucumberOptions and ExtendedCucumberOptions

      Delete
    2. Hi Kolesnik,

      I am also getting similar error as mentioned by you above. I figured out that the issue was JSON file not getting generated and we trying to read from it. I just wanted to know where should i place this piece of code so that it is executed after json report is generated. Right now i have added it to @AfterClass method in my runner class but it still gives me error that file is empty. Kindly let me know what steps to take to resolve this. I am not using the extended class as stated above because it was giving me error.

      Delete
    3. See all details, explanations and possible solutions in the dedicated post

      Delete
  5. Hi,

    Thanks for your efforts in creating these wonderful reports. I have a query, could you please help me to clarify.
    In detailed report once the tests have finished and the report is opened, Overall Duration says :0h 00m 00s all the time the tests are run.

    ReplyDelete
    Replies
    1. Hi,

      You need to switch to newer version of the cucumber-reports library. Current version is: 1.0.0
      This problem should have been fixed there.

      Delete
    2. Hi, I have tried adding the new version of the report both manually and through maven but all to know avail .Kindly please help to resolve this issue.

      Thanks

      Delete
    3. I didn't get your last question. Do you mean the problem still persists? This issue was definitely fixed recently and it's no longer present on 1.0.0 version. If you use Maven make sure you have such dependency entry:

      <dependency>
      <groupId>com.github.mkolisnyk</groupId>
      <artifactId>cucumber-reports</artifactId>
      <version>1.0.0</version>
      </dependency>

      And then make sure you refreshed everything (you can run mvn clean command to remove old binaries)

      Delete
  6. Apologies for such a basic question but as I am a newbie I think I can get away with it.

    I'm currently running Cucumber-JVM with selenium and all is well. I also use TestRail (web based test management software) to manually add my test results.
    I wish to automatically update testrail with my cucumber results however I cannot see where my cucumber test results are stored, nor how I can access them programmatically. All of the example for plugins suggest my cucumber output is" /src/test/resources/" but when I check this there it nothing stored in there exceopt for my feature files . Any ideas feedback would be appreciated!

    ReplyDelete
    Replies
    1. Reports to produce are defined as the part of run options. Standard Cucumber JVM reports are defined as the part of plugins used. If you run tests as JUnit tests you'll probably have to define @CucumberOptions annotation. Here is an example of it:

      @CucumberOptions(
      plugin = {"html:build/cucumber-html-report",
      "junit:build/cucumber-junit.xml",
      "json:build/cucumber.json",
      "pretty:build/cucumber-pretty.txt",
      "usage:build/cucumber-usage.json"
      },
      features = {"src/test/java/com/mkolisnyk/common/features"},
      glue = {"com/mkolisnyk/common/steps"},
      tags = {"@test"}
      )

      Note that the plugin field defines the reports to generate as well as the output location relative to the root folder. If you don't define any report plugin you will not get any report (except some console output)

      If you use Extended Cucumber runner you also need to populate @ExtendedCucumberOptions annotation. It has the outputFolder field where you define the path where extended reports are dropped into. This path is also relative to the project root.

      Delete
  7. Hi Nickolay Kolesnik,

    Thanks for your API in generate detail report for cucumber. I had try to use it but got issue bellow:

    Caused by: java.lang.ClassCastException: [Ljava.lang.Object; cannot be cast to com.cedarsoftware.util.io.JsonObject
    at com.github.mkolisnyk.cucumber.reporting.CucumberResultsCommon.readFileContent(CucumberResultsCommon.java:90)
    at com.github.mkolisnyk.cucumber.reporting.CucumberDetailedResults.executeDetailedResultsReport(CucumberDetailedResults.java:417)
    at com.mh.runner.TestSuite.tearDown(TestSuite.java:46)
    ... 12 more

    ReplyDelete
    Replies
    1. my sample code bellow:
      @RunWith(ExtendCucumber.class)
      @CucumberOptions
      (
      features = "Feature"
      ,glue={"com.mh.stepsdefinition"}
      ,tags={"@Login"}
      ,plugin =
      {
      "html:build_report/HTM_Report",
      "junit:build_report/cucumber-junit.xml",
      "json:build_report/cucumber.json",
      "pretty:build_report/cucumber-pretty.txt",
      "usage:build_report/cucumber-usage.json"
      }
      )

      public class TestSuite {
      @After
      public static void tearDown() throws Exception {
      //notificationUser.sendEmail();
      CucumberResultsOverview results = new CucumberResultsOverview();
      //CucumberDetailedResults results = new CucumberDetailedResults();
      results.setOutputDirectory("build_report");
      results.setOutputName("build_report/cucumber-results");
      results.setSourceFile("build_report/cucumber.json");
      results.executeFeaturesOverviewReport();
      }

      Delete
    2. This error appears because you have somewhere explicit dependency on the com.cedarsoftware:json-io library of one of the recent versions. The reporting library depends on version 2.2.31. If you have latter version you'll have such exceptions. So, try to remove unnecessary dependencies on son-io library.

      Delete
    3. Hi Kolesnik,

      Thanks for your reply, now it's working perfectly.
      Thanks

      Delete
  8. Hi Kolesnik,
    Could you please tell me how to include screenshot in the reports. Although I am getting the reports fine and screenshots in the "/screenshot" folder but I can't get them included in the html/pdf report.
    Here is my "@After" hook
    ------------After hook--------------
    @After
    public void afterTest(Scenario scenario){
    LOG.info("Taking screenshot if test failed");
    if (!webConnect.getDriverType().equalsIgnoreCase("browserStack")) {
    try {

    if (scenario.isFailed()) {
    String scenarioID=scenario.getId();
    scenarioID=scenarioID.replaceAll(";","_");
    scenario.write(getDriver().getCurrentUrl());

    byte[] screenShot = ((TakesScreenshot) getDriver()).getScreenshotAs(OutputType.BYTES);
    scenario.embed(screenShot, "image/png");
    File scrFile = ((TakesScreenshot)getDriver()).getScreenshotAs(OutputType.FILE);
    FileUtils.copyFile(scrFile, new File(System.getProperty("user.dir")+"\\screenshot\\"+scenarioID+".png"));
    }
    } catch (WebDriverException | IOException e) {
    LOG.error(e.getMessage());

    }
    }
    --------------End After hook----------------
    And following is my Runner file
    --------------RunCukesTest.java---------
    @RunWith(ExtendedCucumber.class)
    @ExtendedCucumberOptions(
    jsonReport = "target/cucumber.json",
    jsonUsageReport = "target/cucumber.json",
    outputFolder = "target1/",
    detailedReport = true,
    detailedAggregatedReport = true,
    overviewReport = true,
    usageReport = true,
    coverageReport = false,
    retryCount = 0,
    screenShotLocation = "screenshot/",
    // screenShotSize = "300px",
    toPDF = true
    )

    @CucumberOptions(
    plugin = {"html:target/cucumber-html-report","junit:target/cucumber-junit.xml",
    "json:target/cucumber.json","pretty:target/cucumber-pretty.txt","usage:target/cucumber-usage.json"},
    tags = {"@firstProg"}

    )
    public class RunCukeTest {
    }
    -----------------End RunCuckesTest.java---------------------

    ReplyDelete
    Replies
    1. You need to do 2 things:
      1) make more replacements to get scenario ID. The actual replacement instruction should be like:
      scenarioID=scenarioID.replaceAll("[; !@#$%^&()+=]", "_");

      2) The screenshot location is supposed to be relative to the output directory. So, you need to output screenshot files using the following instruction:

      FileUtils.copyFile(scrFile, new File(System.getProperty("user.dir")+"\\target1\\screenshot\\"+scenarioID+".png"));

      Delete
    2. Hi Kolesnik,

      I am trying to add the failed scenarios' screenshots in PDF file. I did as you mentioned above. I see .png files getting created but not getting inserted into PDF file.

      Should I do anything else to insert the screenshot into PDF?

      Thanks,
      Srini

      Delete
    3. Hi,

      Mainly it's because of naming conventions. Library looks for files with specific name. But there is an alternative. You can embed screenshots into standard Cucumber report. The library will pick that up. All you need to do is to create Cucumber After hook like this:

      @After
      public void processStatus(Scenario scenario) {
      if (scenario.getStatus().equalsIgnoreCase("failed")) {
      try {
      WebDriver augmentedDriver = new Augmenter().augment(Driver.current());
      File scrFile = ((TakesScreenshot) augmentedDriver).getScreenshotAs(OutputType.FILE);
      byte[] data = FileUtils.readFileToByteArray(scrFile);
      scenario.embed(data, "image/png");
      } catch (Exception e) {
      e.printStackTrace();
      }
      }
      }

      That would make screenshot appear in standard JSON report. And the library handles it pretty well.

      Delete
    4. Hi,

      Thanks for your reply. I have used your code snippet. But getting the below mentioned exception while generating the report

      Exception 1

      java.lang.Exception: Error occured while generating Cucumber usage report
      at com.github.mkolisnyk.cucumber.reporting.CucumberUsageReporting.executeReport(CucumberUsageReporting.java:702)
      at com.github.mkolisnyk.cucumber.runner.ReportRunner.runUsageReport(ReportRunner.java:29)
      at com.github.mkolisnyk.cucumber.runner.ReportRunner.run(ReportRunner.java:152)
      at com.github.mkolisnyk.cucumber.runner.ExtendedCucumber.run(ExtendedCucumber.java:164)
      at org.junit.runner.JUnitCore.run(JUnitCore.java:137)

      Exception 2

      Caused by: javax.xml.transform.TransformerException: org.xml.sax.SAXParseException; lineNumber: 199; columnNumber: 224; Element type "img" must be followed by either attribute specifications, ">" or "/>".
      at org.apache.xalan.transformer.TransformerIdentityImpl.transform(TransformerIdentityImpl.java:502)
      at org.xhtmlrenderer.resource.XMLResource$XMLResourceBuilder.createXMLResource(XMLResource.java:189)
      ... 33 more
      Caused by: org.xml.sax.SAXParseException; lineNumber: 199; columnNumber: 224; Element type "img" must be followed by either attribute specifications, ">" or "/>".
      at org.apache.xerces.parsers.AbstractSAXParser.parse(Unknown Source)
      at org.apache.xalan.transformer.TransformerIdentityImpl.transform(TransformerIdentityImpl.java:485)
      ... 34 more

      Please help to resolve this issue.

      Thanks,
      Srini

      Delete
    5. Exception 1 is related to the fact that you try to generate usage report without usage report source defined. Make sure you set usageReport = false in the ExtendedCucumberOptions annotation.

      Exception 2 happens during PDF conversion of detailed report. Firstly, make sure that HTML detailed report is generated and rendered properly. If so, find the line 199 and column 224 of it and mainly check the path. It may happen that some special characters are used. For more details I need to see this part of HTML and IMG node in particular. Also, make sure you are using the latest version of the library which is now 1.0.7

      Delete
    6. Exception 1 is no more after setting usageReport = false and updated the library version to 1.0.7. I see this exception

      java.io.FileNotFoundException: target\screenshot\43-id-13:-recovery_id-477:-password-is-successfully-reset1481832541814.png (The filename, directory name, or volume label syntax is incorrect)
      at java.io.FileOutputStream.open0(Native Method)
      at java.io.FileOutputStream.open(FileOutputStream.java:270)
      at java.io.FileOutputStream.(FileOutputStream.java:213)
      at org.apache.commons.io.FileUtils.openOutputStream(FileUtils.java:360)

      And this is my configuration

      @RunWith(ExtendedCucumber.class)
      @ExtendedCucumberOptions(
      jsonReport = "target/cucumber.json",
      jsonUsageReport = "target/cucumber.json",
      outputFolder = "target/",
      detailedReport = true,
      detailedAggregatedReport = true,
      overviewReport = true,
      usageReport = false,
      coverageReport = false,
      retryCount = 0,
      screenShotLocation = "screenshot/",
      screenShotSize = "700px",
      toPDF = true
      )
      @CucumberOptions(plugin = { "html:target/cucumber-html-report",
      "json:target/cucumber.json", "pretty:target/cucumber-pretty.txt",
      "usage:target/cucumber-usage.json", "junit:target/cucumber-results.xml",}, tags = {"@Srini"}, features="classpath:features/04 ForgotPassword.feature")

      Note:
      HTML report shows the screenshot.

      - Thanks

      Delete
    7. OK. The problem is in the path like this: target\screenshot\43-id-13:-recovery_id-477:-password-is-successfully-reset1481832541814.png

      It contains special characters like colon or ":". The file name is formed based on Feature and Scenario names. Try to remove colon character from them. For HTML report it's OK (thus you can see it) but for PDF processing it causes the problem, especially if you use Windows as the colon character is quite specific for this system. So, remove colons from feature and scenario names. That should help.

      Delete
    8. Thanks a lot. PDF report is now getting generated successfully. But we have a scenario outline where we have outlined 'Scenario Names' like

      Scenario Outline:
      Given that I am on the RESET PASSWORD screen
      ..........
      .........
      .........

      Examples:
      |scenarioText|step1|errorMsg|
      |ID-000 - error |I enter a password that is the same as my current password|you need to pick a new password|

      Actual scenario names are not getting replaced from Examples table. So the test scripts are failing with FileNotFoundException saying the screenshot /target/screenshot/ID-000_error_.......png not valid.

      Is there any way to replace the actual scenario name before cucumber looks for the screenshot?

      -Thanks

      Delete
    9. This comment has been removed by the author.

      Delete
    10. Scenario Outline: """" // added quotes here to show scenario names outlined
      Given that I am on the RESET PASSWORD screen
      ..........
      .........
      .........

      Delete
    11. Just give the scenario outline an informative name. All those test groups definitely have some purpose, so reflect it in the name.

      Delete
    12. Thanks for your suggestion. I have managed to construct the scenario id by appending feature and scenario name. I do not see FileNotFound exception. Getting the below exception now

      Caused by: javax.xml.transform.TransformerException: org.xml.sax.SAXParseException; lineNumber: 82; columnNumber: 521; The value of attribute "src" associated with an element type "img" must not contain the '<' character.
      at org.apache.xalan.transformer.TransformerIdentityImpl.transform(TransformerIdentityImpl.java:502)
      at org.xhtmlrenderer.resource.XMLResource$XMLResourceBuilder.createXMLResource(XMLResource.java:189)
      ... 34 more
      Caused by: org.xml.sax.SAXParseException; lineNumber: 82; columnNumber: 521; The value of attribute "src" associated with an element type "img" must not contain the '<' character.

      As you asked in one of your previous reply, Which file do you want to check to resolve this issue?

      -Thanks

      Delete
  9. Hi Kolesnick,

    I have been researching a nice report for my cucumer-jvm projects and found your solution. The report is really nice. Unfortunately, when I add its dependency:







    my test using Spring framework fail. All the @Autowire ussage fail when running the test. Do you have experience on that?

    Kind regards,
    -- Trong

    ReplyDelete
    Replies
    1. here is the dependency:


      com.github.mkolisnyk
      cucumber-reports
      1.0.1

      Delete
    2. You should check the list of dependencies included by both Spring and cucumber-reports libraries. You should pay attention to all of them. Such problem may appear due to some dependent library conflict. E.g. I've seen errors related to conflicting version of xml-apis library when I had to specify proper dependency explicitly. Also, there may be something wrong with even log4j library. Anyway, the actual cause can be found by analysing dependencies.

      Delete
  10. Hi Kolesnik,

    I am using your dependency for reports, everything is fine, i could generate reports well

    But i have one small issue with retryCount

    I have given retryCount=1, so my test got failed and re instated again, now this time its passed

    Question is, if test is passed on 2nd run, then test should mark as Fail or pass ???

    BR,
    Rakesh

    ReplyDelete
    Replies
    1. when tests passed on 2nd run then build should be successful , is that possible ??

      Delete
    2. You can check that on your own as I don't know how you run it. But if you run it as regular JUnit test suite it's very likely that the entire run would be marked as failed. The thing is that re-run functionality is applied to the Cucumber runner, not the JUnit reporter. The JUnit reporter just logs the output and it cannot overwrite previous statements which may contain errors.

      Delete
  11. Thanks for another great post, Nickolay!

    Just a question: How would you need to alter the reports so that they read from more than one feature file.
    e.g. Website A has "Latest videos" and "Breaking news" sections. One feature would test all scenarios for the "Latest videos" section and another feature file would test all scenarios for the "Breaking news" section.

    Would it possible to show the aggregated details in the Overview/Coverage/Usage/etc reports?

    Warm regards,
    Brent

    ReplyDelete
    Replies
    1. Actually, report isn't bound to each specific feature. It is bound to test suite run which may include multiple features (in CucumberOptions annotation for JUnit class you can define the path to features which can be the folder). Technically, the reporting solution is just the post-processor for Cucumber JSON report, so as soon as you can produce such standard output for any combination of features you can generate any supported advanced report produced by cucumber-reports library.

      Delete
  12. Thanks for the quick response. I have adapted this change and the reports are all consolidated and much more informative.

    These simple tests set up successfully run using JUnit (my main test runner), however I am now getting an issue when running this using the "Maven test" protocol.
    The issue I receive is "Type com.github.mkolisnyk.cucumber.runner.ExtendedCucumber not present". Have you perhaps had anything similar occur?

    Error output:
    Running feature.steps._FeatureTestRunner
    Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.013 sec <<< FAILURE! - in feature.steps._FeatureTestRunner
    initializationError(feature.steps._FeatureTestRunner) Time elapsed: 0.007 sec <<< ERROR!
    java.lang.TypeNotPresentException: Type com.github.mkolisnyk.cucumber.runner.ExtendedCucumber not present
    at sun.reflect.annotation.TypeNotPresentExceptionProxy.generateException(TypeNotPresentExceptionProxy.java:46)
    at sun.reflect.annotation.AnnotationInvocationHandler.invoke(AnnotationInvocationHandler.java:75)
    at com.sun.proxy.$Proxy2.value(Unknown Source)
    at org.junit.internal.builders.AnnotatedBuilder.runnerForClass(AnnotatedBuilder.java:86)
    at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
    at org.junit.internal.builders.AllDefaultPossibilitiesBuilder.runnerForClass(AllDefaultPossibilitiesBuilder.java:26)
    at org.junit.runners.model.RunnerBuilder.safeRunnerForClass(RunnerBuilder.java:59)
    at org.junit.internal.requests.ClassRequest.getRunner(ClassRequest.java:33)
    at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:262)
    at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
    at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
    at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)
    at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
    at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)

    ReplyDelete
    Replies
    1. Caused by: java.lang.ClassNotFoundException: com.github.mkolisnyk.cucumber.runner.ExtendedCucumber
      at java.net.URLClassLoader$1.run(URLClassLoader.java:372)
      at java.net.URLClassLoader$1.run(URLClassLoader.java:361)
      at java.security.AccessController.doPrivileged(Native Method)
      at java.net.URLClassLoader.findClass(URLClassLoader.java:360)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
      at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      at java.lang.Class.forName0(Native Method)
      at java.lang.Class.forName(Class.java:340)
      at sun.reflect.generics.factory.CoreReflectionFactory.makeNamedType(CoreReflectionFactory.java:114)
      at sun.reflect.generics.visitor.Reifier.visitClassTypeSignature(Reifier.java:125)
      at sun.reflect.generics.tree.ClassTypeSignature.accept(ClassTypeSignature.java:49)
      at sun.reflect.annotation.AnnotationParser.parseSig(AnnotationParser.java:439)
      at sun.reflect.annotation.AnnotationParser.parseClassValue(AnnotationParser.java:420)
      at sun.reflect.annotation.AnnotationParser.parseMemberValue(AnnotationParser.java:349)
      at sun.reflect.annotation.AnnotationParser.parseAnnotation2(AnnotationParser.java:286)
      at sun.reflect.annotation.AnnotationParser.parseAnnotations2(AnnotationParser.java:120)
      at sun.reflect.annotation.AnnotationParser.parseAnnotations(AnnotationParser.java:72)
      at java.lang.Class.createAnnotationData(Class.java:3508)
      at java.lang.Class.annotationData(Class.java:3497)
      at java.lang.Class.getAnnotation(Class.java:3402)
      at org.apache.maven.surefire.common.junit4.JUnit4TestChecker.isValidJUnit4Test(JUnit4TestChecker.java:65)
      at org.apache.maven.surefire.common.junit4.JUnit4TestChecker.accept(JUnit4TestChecker.java:52)
      at org.apache.maven.surefire.util.DefaultScanResult.applyFilter(DefaultScanResult.java:97)
      at org.apache.maven.surefire.junit4.JUnit4Provider.scanClassPath(JUnit4Provider.java:206)
      at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:103)
      at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)
      at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
      at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)

      Delete
    2. ClassNotFoundException mainly indicates that the dependency wasn't retrieved. You should make sure that your pom.xml contains entry for required version (the latest cucumber-reports library version now is 1.0.1) and then you have to make sure that this dependency is downloaded.

      Also, you can clean your output directory before running tests. You can do it by running command like:

      mvn clean test

      This can guarantee that your code is properly refreshed and re-compiled before running tests

      Delete
  13. Thanks so much.
    I opted to use your project via the pom file. The 'mvn clean test' works beautifully there. However this throws an exception when using the SureFire plugin on Jenkins. Do you have any suggestions for this?

    ReplyDelete
    Replies
    1. I could have any suggestions if you could provide the information about the error. The only surefire related issue I encountered was when I tried to run JUnit tests while there were some TestNG tests. There is dedicated post describing problem and solution.

      Delete
  14. Hi Nickolay,

    First of all I would like to thank you for the utility that you have created for advanced reporting. Can you please let me know the combination of compatible version for cucumber reports and json-io.jar with java 1.7.

    At present i am using 0.0.3 for cucumber-reports and json-io-2.2.31 but with these the graph for scenario status is incorrect in my case.I have one feature file with three scenarios. while execution 2 failed and 1 pass. Report is showing 33.3 % failed and 66.7% pass but this is not correct. Can you please revert back when you have time with compatible versions and correct reporting. Thank you again for your help.

    ReplyDelete
    Replies
    1. First thing I suggest you is to use more recent version of the library. The latest version compatible with java 1.7 is 1.0.1.
      Also, pay attention to what exactly is shown as passed and failed because some of the reports show pass/fail statistics not just for scenarios but also for features and steps. And all those measures may show different ratio

      Delete
    2. Thank you for the reply. I tried the below combination:
      Java 1.7
      cucumber-report 1.0.1
      java-util-1.22.0
      json-io-4.4.0

      and I am getting the below error: Exception in thread "main" java.lang.ClassCastException: [Ljava.lang.Object; cannot be cast to com.cedarsoftware.util.io.JsonObject
      at com.github.mkolisnyk.cucumber.reporting.CucumberResultsCommon.readFileContent(CucumberResultsCommon.java:90)
      at com.github.mkolisnyk.cucumber.reporting.CucumberResultsOverview.executeOverviewReport(CucumberResultsOverview.java:121)
      at com.github.mkolisnyk.cucumber.reporting.CucumberResultsOverview.executeFeaturesOverviewReport(CucumberResultsOverview.java:142)
      at Technow.report.main(report.java:13)


      If possible can you please recommend the working combination of the jars and other dependency.

      Delete
    3. Remove json-io-4.4.0 dependency. The reporting library uses older version of this library and already has the dependency included.

      Delete
    4. Now the project has:
      Java 1.7
      cucumber-report 1.0.1
      java-util-1.22.0

      No luck....Now I am getting the below error:

      Exception in thread "main" java.lang.NoClassDefFoundError: com/cedarsoftware/util/io/JsonReader
      at com.github.mkolisnyk.cucumber.reporting.CucumberResultsCommon.readFileContent(CucumberResultsCommon.java:89)
      at com.github.mkolisnyk.cucumber.reporting.CucumberResultsOverview.executeOverviewReport(CucumberResultsOverview.java:121)
      at com.github.mkolisnyk.cucumber.reporting.CucumberResultsOverview.executeFeaturesOverviewReport(CucumberResultsOverview.java:142)
      at Technow.report.main(report.java:13)
      Caused by: java.lang.ClassNotFoundException: com.cedarsoftware.util.io.JsonReader
      at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
      at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
      at java.security.AccessController.doPrivileged(Native Method)
      at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
      at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
      ... 4 more

      Delete
    5. There can be 2 options:
      1) Explicitly add dependency for com.cedarsoftware json-io version 2.2.31
      2) Remove any dependency on com.cedarsoftware json-io
      Also, don't forget to refresh your dependencies. If you're running it from IDE such changes may be updated only after some manual trigger

      Delete
    6. Jars:
      cucumber-report 1.0.1.jar
      json-io version 2.2.31.jar

      Getting the below exception:


      Exception in thread "main" java.lang.NoClassDefFoundError: org/apache/commons/lang/ArrayUtils
      at com.github.mkolisnyk.cucumber.reporting.types.result.CucumberScenarioResult.getAllTags(CucumberScenarioResult.java:302)
      at com.github.mkolisnyk.cucumber.reporting.types.result.CucumberScenarioResult.isInTagSet(CucumberScenarioResult.java:307)
      at com.github.mkolisnyk.cucumber.reporting.types.result.CucumberScenarioResult.valuate(CucumberScenarioResult.java:104)
      at com.github.mkolisnyk.cucumber.reporting.types.result.CucumberFeatureResult.valuate(CucumberFeatureResult.java:64)
      at com.github.mkolisnyk.cucumber.reporting.types.result.CucumberFeatureResult.getStatus(CucumberFeatureResult.java:88)
      at com.github.mkolisnyk.cucumber.reporting.CucumberResultsOverview.generateFeatureOverview(CucumberResultsOverview.java:56)
      at com.github.mkolisnyk.cucumber.reporting.CucumberResultsOverview.executeOverviewReport(CucumberResultsOverview.java:125)
      at com.github.mkolisnyk.cucumber.reporting.CucumberResultsOverview.executeFeaturesOverviewReport(CucumberResultsOverview.java:142)
      at Technow.report.main(report.java:13)
      Caused by: java.lang.ClassNotFoundException: org.apache.commons.lang.ArrayUtils
      at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
      at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
      at java.security.AccessController.doPrivileged(Native Method)
      at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:425)
      at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:358)
      ... 9 more


      Thanks,
      Shrikant

      Delete
  15. Do you mean you just create empty java project and include jars? You should use something like Maven or Gradle to retrieve dependencies. Otherwise, you have to download all required dependencies on your own (and the number is essential).

    ReplyDelete
  16. Hi...can i convert the json file into html/pdf report after execution seperately.

    ReplyDelete
    Replies
    1. This conversion is actually done separately. After tests completion the HTML report is generated. If PDF generation flag is set the report is then transformed into PDF.

      Delete
  17. Hi.. screen shots are not visible in the report. but I see an image icon added at the end of failed scenario

    ReplyDelete
    Replies
    1. OK. It means that the image reference was created but no actual image found. Are you using embedded screen shots?

      Delete
    2. No. I am not using embedded screen shots.

      Delete
    3. Then you may look at the report source code and find that those image entries refer to the non-existing image file. Are you generating screen shots at all? The library itself doesn't do it as screenshot generation is very technology specific thing. So, just make sure that you generate images based on proper convention

      Delete
  18. Screen shots are generated and I can open them. The issue I see is, if I open the source code the image path is like img src= "\failed_scenario.png" where it should just be img src = "failed_scenario.png"

    If I edit source the image is displayed.

    When I look at the code, file path is added to img src as is. In my case the output directory is target/cucumber and the image is saved here aswell. according to the code below, to check if the shot is available I should give screenshot location only as "/" but if I give "/" then it is added to file path and the html is referencing unknown location.


    CucumberDetailedResults.class
    generateScreenShot()
    .
    .
    .
    String filePath = this.getScreenShotLocation()
    + this.generateNameFromId(scenarioId) + ".png";
    File shot = new File(this.outputDirectory + filePath);
    if (shot.exists()) {
    String widthString = "";
    if (StringUtils.isNotBlank(this.getScreenShotWidth())) {
    widthString = String.format("width=\"%s\"", this.getScreenShotWidth());
    }
    reportContent += String.format(
    "tr class=\"%s\">td colspan=\"2\">img src=\"%s\" %s /</td</tr", //removed tags
    step.getResult().getStatus(),
    filePath,
    widthString
    );

    Please help resolve this.

    ReplyDelete
    Replies
    1. Try to play with screenshot location parameter. E.g. you can set it as "./" or as "" (empty string if your images are in the same directory as the HTML report)

      Delete
  19. Hi Nickolay Kolesnik,

    The PDF report was awesome. But in "cucumber-results-test-results.pdf" generated, it includes only 3 status(Passed, Failed and Undefined). Other status like Skipped, Pending are not seen in the pdf report. In the blog its mentioned that skipped cases are also included. But unfortunately I am not seeing this status. Instead the count of skipped and pending scenarios get added to the count of Undefined scenarios in the report.

    ReplyDelete
    Replies
    1. Hi Gopinath,

      For results report both undefined and skipped steps are indicated as undefined. This is done intentionally as both types indicate that current step wasn't really performed. So, for results report it's the same. Undefined steps separately are detectable by coverage report.

      Pending steps are normally indicated by generated exception and in the final report such steps are not different from failed steps.

      Delete
  20. Hi Nickolay Kolesnik,

    I need to get the count of Scenario status(passed, failed and undefined)separately so that I can send it to an email id for quick review in addition to the pdf report. Is there any function available in java to get the total number of Scenarios passed, failed and undefined?

    ReplyDelete
    Replies
    1. There are many ways to do this. The fastest one I can see is to process test results dump file. It is XML file generated together with overview report and it usually ends with -dump.xml it contains stats (passed/failed/undefined) for features, scenarios and steps respectively. So, if you implement some post-processing you can grab those results.

      Delete
  21. Thanks for the quick reply. The -dump.xml(cucumber-results-feature-overview-dump.xml) contains all the required status. But the XML generated is not in correct format. I am pasting the xml file below for reference



    Here in the XML file root element is given as "int[][]" which is not an acceptable format since it contains "[][]". I am getting the following error when I try to extract the data from XML.

    [Fatal Error] cucumber-results-feature-overview-dump.xml:2:5: Element type "int" must be followed by either attribute specifications, ">" or "/>".
    org.xml.sax.SAXParseException; systemId: file:/home/user/Desktop/cucumber-results-feature-overview-dump.xml; lineNumber: 2; columnNumber: 5; Element type "int" must be followed by either attribute specifications, ">" or "/>".

    ReplyDelete
  22. Sorry I was not able to paste the xml content in the above post. Could you please look into the file that I have mentioned.

    ReplyDelete
    Replies
    1. The problem appears because you are trying to parse this XML via SAX parser. But you can use different processor. In particular, here is the line of code which converts data in 2-dimensional integer array into XML file. This is the JAXB.marshal(stats, outFile); where JAXB is taken from the javax.xml.bind.JAXB package. As I remember this is a part of standard Java library. You can find this class and try to use unmarshall method to convert this XML into int[][].

      Delete
  23. Even if I try to use the unmarshall method to convert the XML to int[][] I am getting the same error. I have used the code given in the link below to convert the XML. Please have a look

    http://www.javatpoint.com/jaxb-tutorial

    Also I tried to validate "cucumber-results-feature-overview-dump.xml", using an XML Validator and it shows "error on line 2 at column 5: error parsing attribute name".

    Used the following link to validate the xml
    http://www.w3schools.com/xml/xml_validator.asp

    Is it possible to parse the data from XML using any other processor if the XML is not a valid one?
    Please help me to solve this issue

    ReplyDelete
    Replies
    1. OK. Looks like the problem is that JAXB converts 2-dimensional integer array into XML like that. And such XML doesn't really conform standards due to those [][] characters. I'll raise an issue to avoid that. Meantime, you can use the quick fix like this:

      File dump = new File("./target/cucumber-results-feature-overview-dump.xml");
      String content = FileUtils.readFileToString(dump);
      content = content.replaceAll("\\[]\\[]", "");
      FileUtils.writeStringToFile(dump, content);
      int[][] result = JAXB.unmarshal(dump, int[][].class);

      The idea is that as soon as we read file, remove problematic [][] string and overwrite the file with new content, all the processing goes fine and you easily get 2-dimentional array of integers populated. I've checked this sample locally. It works.

      You may additionally need to include org.apache.commons.io.FileUtils class which is available in Apache commons library

      Delete
    2. Thanks a lot! That one fixed my issue. But the XML is generated only if I run the script as java application. If I try to run the script using the build tool 'ant' the XML is not generated. It's throwing the following error.

      Please note that the error occurs only if the script is run as Ant Build and 'overviewReport = true' is given in 'ExtendedCucumberOptions'. It works as expected without any error for the following cases

      1. remove 'overviewReport = true' in 'ExtendedCucumberOptions' and run as Ant Build. (XML not generated and no errors are thrown)
      2. overviewReport = true' is given in 'ExtendedCucumberOptions' and run as java application. (XML generated and no errors are thrown)
      3. remove 'overviewReport = true' in 'ExtendedCucumberOptions' and run as java application. (XML not generated and no errors are thrown)

      [java] javax.xml.bind.DataBindingException: javax.xml.bind.JAXBException
      [java] - with linked exception:
      [java] [java.lang.ClassNotFoundException: com.sun.xml.internal.bind.v2.ContextFactory]
      [java] at javax.xml.bind.JAXB._marshal(JAXB.java:574)
      [java] at javax.xml.bind.JAXB.marshal(JAXB.java:332)
      [java] at com.github.mkolisnyk.cucumber.reporting.interfaces.CucumberResultsCommon.dumpOverviewStats(CucumberResultsCommon.java:179)
      [java] at com.github.mkolisnyk.cucumber.reporting.CucumberResultsOverview.executeOverviewReport(CucumberResultsOverview.java:153)
      [java] at com.github.mkolisnyk.cucumber.reporting.CucumberResultsOverview.execute(CucumberResultsOverview.java:197)
      [java] at com.github.mkolisnyk.cucumber.runner.ReportRunner.runOverviewReport(ReportRunner.java:41)
      [java] at com.github.mkolisnyk.cucumber.runner.ReportRunner.run(ReportRunner.java:153)
      [java] at com.github.mkolisnyk.cucumber.runner.ExtendedCucumber.run(ExtendedCucumber.java:164)
      [java] at org.junit.runners.Suite.runChild(Suite.java:128)
      [java] at org.junit.runners.Suite.runChild(Suite.java:27)
      [java] at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
      [java] at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
      [java] at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
      [java] at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
      [java] at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
      [java] at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
      [java] at org.junit.runner.JUnitCore.run(JUnitCore.java:137)
      [java] at org.junit.runner.JUnitCore.run(JUnitCore.java:115)
      [java] at org.junit.runner.JUnitCore.run(JUnitCore.java:105)
      [java] at org.junit.runner.JUnitCore.run(JUnitCore.java:94)
      [java] at com.settings.Runjunit.main(Unknown Source)
      [java] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      [java] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      [java] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      [java] at java.lang.reflect.Method.invoke(Method.java:497)
      [java] at org.apache.tools.ant.taskdefs.ExecuteJava.run(ExecuteJava.java:217)
      [java] at org.apache.tools.ant.taskdefs.ExecuteJava.execute(ExecuteJava.java:152)
      [java] at org.apache.tools.ant.taskdefs.Java.run(Java.java:771)
      [java] at org.apache.tools.ant.taskdefs.Java.executeJava(Java.java:221)
      [java] at org.apache.tools.ant.taskdefs.Java.executeJava(Java.java:135)
      [java] at org.apache.tools.ant.taskdefs.Java.execute(Java.java:108)
      [java] at org.apache.tools.ant.UnknownElement.execute(UnknownElement.java:292)
      [java] at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      [java] at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      [java] at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      [java] at java.lang.reflect.Method.invoke(Method.java:497)

      Delete
    3. Use option 2 then. The dump file is generated as the part of overview report. So, naturally, you need overviewReport = true flag set

      Delete
    4. But the problem is I need to run the entire suite using ANT for continuous integration. In that case error will be thrown as I mentioned above, when 'overviewReport = true' flag is set.

      Delete
    5. The key error is this: java.lang.ClassNotFoundException: com.sun.xml.internal.bind.v2.ContextFactory

      There can be many reasons for that. But very likely you have some additional dependencies which may be in conflict. Some of the possible solutions can be found here. Main idea is to replace jamb-api library with jab-impl

      Delete
    6. Thanks a ton! Solved the issue that has been disturbing me for days. When the jar file jaxb-impl is added it worked fine.

      Delete
  24. In order to prepend the ProjectName in the pdf generated, I am using the option reportPrefix="ProjectName" in ExtendedCucumberOptions. I want to get the ProjectName from a property file since I have to use this name in many other classes. But unfortunately I cant assign the name to reportPrefix by reading from a property file. Is there any other option to overcome this?

    For CucumberOptions we have an option something like this
    System.setProperty("cucumber.options", "--glue com/stepdefs Features/login.feature --monochrome");
    JUnitCore junitRunner = new JUnitCore();
    junitRunner.run(com.settings.RunFeatures.class);

    Can we give something like this in ExtendedCucumberOptions?If not is there any other solution to assign the name to reportPrefix by reading from a property file?

    ReplyDelete
    Replies
    1. Since 1.0.6 version you can override extended cucumber option values from system properties. In your case you should pass the value to the cucumber.reports.reportPrefix system property.

      Delete
  25. Hi ,
    How we can browser information in reports.
    In @Before i am writing
    scenario.write("Browser Information - Chrome")

    I am getting Null pointer exception in JSON formatter

    ReplyDelete
    Replies
    1. I am using nfo.cukes:cucumber-java:1.2.5 which uses ghenkin 2.12.2

      Delete
    2. This is Cucumber-JVM thing. I suspect that the scenario.write writes text on the step level. Since hooks are triggered before any step was initialised, we are getting NullPointerException. There's no steps at that time yet.

      As an alternative, you can use scenario.write in After method. It's a bit inconvenient but this information will be placed in the output.

      Delete
    3. Is there any way to pass information in @Before method. I will check in After and confirm

      Delete
    4. Moreover , scenario.write putting data in Cucumber.json. I believe not able to render in reports

      Delete
    5. When you try to write something in the Before method the JSON reporter fails. It's internal Cucumber-JVM problem and reporting library isn't related to that. You can check that by running standard Cucumber-JVM tests with write call in the Before method. The result is the same.

      Delete