Karate Reports | InditexTech Karate Tools

Karate HTML Report - karate-reports:
- Automatically generated by Karate.
Karate Operations Usage Statistics Report - karate-reports/karate-operations-json.txt:
- Automatically generated by Karate-Tools.
Cucumber Results JSON - cucumber_result.json:
- Automatically generated by Karate Tools runner
- It can be used to report results to Test Management System supporting cucumber, for example: JIRA-Xray.
Surefire report - surefire-reports:
- Surefire XML Reports.
  - Automatically generated by surefire execution.
- Surefire HTML Reports, configured in the pom by Karate Tools Archetype.
  - Executed on demand.

The next sections provide additional information on the more relevant reports.

Karate HTML Report

This report is karate’s default and included in the karatetools-starter:

<!-- Karate Tools -->
<dependency>
  <groupId>dev.inditex.karate</groupId>
  <artifactId>karatetools-starter</artifactId>
  <version>${karatetools.version}</version>
  <scope>test</scope>
</dependency>

The command to generate the karate report is:
- Automatically generated by Karate.
The report is generated in:
- target/karate-reports
  - karate-summary.html

Karate Operations Usage Statistics Report

This json report is automatically generated by Karate Tools.

This report includes, for each API operation the number of smoke and functional scenarios which called each operation, as well as the number of total calls.

The API operation is identified as <API_PACKAGE>/<OPERATION_TAG>/<OPERATION_FOLDER>/<OPERATION>

For example:

{
  "com/.../xxx-api-rest-stable/BasicApi/createItems/createItems" : {
    "smoke" : 3,
    "functional" : 3,
    "calls" : 7
  },
  "com/.../api/xxx-api-rest-stable/BasicApi/listItems/listItems" : {
    "smoke" : 3,
    "functional" : 1,
    "calls" : 5
  },
  "com/.../api/xxx-api-rest-stable/BasicApi/showItemById/showItemById" : {
    "smoke" : 3,
    "functional" : 1,
    "calls" : 5
  }
}

The number of total calls can be higher than the aggregation of smoke and functional, since one operation can be used multiple times in a scenario, for example in end-to-end flows.

The report is generated in:
- target/karate-reports/karate-operations-json.txt

Cucumber Results JSON

This json report is automatically generated by Karate Tools runner by aggregating the karate cucumber results json files generated by karate for each feature file.

This report is used to import results to Test Management Systems supporting cucumber, for example: JIRA-Xray.

The report is generated in:
- target/cucumber_result.json

Surefire report

The configuration for maven-surefire-report-plugin and maven-site-plugin is included in the pom by Karate Tools Archetype as follows:

<properties>
  ...
  <!-- Test Execution & Reporting -->
  <maven-surefire-plugin.version>X.X.X</maven-surefire-plugin.version>
  <maven-surefire-report-plugin.version>X.X.X</maven-surefire-report-plugin.version>
  <maven-site-plugin.version>X.X.X</maven-site-plugin.version>
  ...
</properties>

<build>
  <plugins>
    <!-- Test Execution & Reporting -->
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-surefire-plugin</artifactId>
      <version>${maven-surefire-plugin.version}</version>
    </plugin>
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-surefire-report-plugin</artifactId>
      <version>${maven-surefire-report-plugin.version}</version>
      <configuration>
        <outputDirectory>${project.build.directory}/surefire-html-report</outputDirectory>
        <!-- Enable this line if using Karate v1.0.0 -->
        <xrefLocation>${project.build.directory}/karate-reports</xrefLocation>
      </configuration>
    </plugin>
    <plugin>
      <groupId>org.apache.maven.plugins</groupId>
      <artifactId>maven-site-plugin</artifactId>
      <version>${maven-site-plugin.version}</version>
      <configuration>
        <outputDirectory>${project.build.directory}/surefire-html-report</outputDirectory>
      </configuration>
    </plugin>
    ...
  </plugins>
</build>

The command to generate the surefire report is:

mvn surefire-report:report-only site -DgenerateReports=false

The report is generated in:
- target/surefire-html-report
  - surefire-report.html