Table of Content

Objective

Code coverage, also called test coverage, is a measure to what extent the source code of a program is executed when a test suite is run. It is often used to gauge the quality of the test suite, as such, to provide an assurance of software quality. There are a number of tools that can be used to measure code coverage. In this tutorial, we experiment two tools. JaCoCo, a Code Coverage library and tool for Java, and Visual Studio Code’s Coverage Tool. Thus, the objectives are three,

  1. To be able to use code coverage to guide the test case develoment;
  2. To be able to run JaCoCo and interpret the reports for Java projects;
  3. To be able to obtain code coverage using Visual Studio Code and interpret the reports for unmanaged Java projects.

Using JaCoCo for Unmanaged Projects

For small Java projects, it might not be necessary to use any IDE’s or to use any build tools, such as Maven or Gradle.

  1. Download and unzip JaCoCo from https://www.eclemma.org/jacoco/. From the command line, If you have curl, you can download using:

    curl https://search.maven.org/remotecontent?filepath=org/jacoco/jacoco/0.8.13/jacoco-0.8.13.zip \
  -o jacoco-0.8.13.zip

    Suppose we want to unzip to the lib directory,

    mkdir -p lib
cd lib
unzip ../jacoco-0.8.13.zip

    You will find a few jar files, e.g.,

    $ ls jacoco-0.8.13/lib
jacocoagent.jar
jacocoant.jar
jacococli.jar
$

  2. Assuming that we are writing Junit 5 test cases, we will need the Jar file for Junit 5. In the lib directory, download it using curl or other methods of your choice,

    curl \
  https://repo1.maven.org/maven2/org/junit/platform/junit-platform-console-standalone/1.13.0-M3/junit-platform-console-standalone-1.13.0-M3.jar \
  -o junit-platform-console-standalone-1.13.0-M3.jar

  3. Write a Java program, for instance, we create a simple Java class in the src/main directory,

    package main;
public class Client {
     public void checkBool(boolean x, boolean y, boolean  z) {
         if (x) {
             if (y && z) {
                 showFlag(0);
             } else {
                 showFlag(1);
             }
         } else {
             System.out.println("x is " + x);
         }
     }

     public void showFlag(int flag) {
         System.out.printf("Flag is %d", flag);
     }
}

  4. Write test case in the src\test\TestClient.java. We write one test case at a time:

    package test;

import java.io.ByteArrayOutputStream;
import java.io.PrintStream;

import org.junit.jupiter.api.AfterEach;
import org.junit.jupiter.api.Assertions;
import org.junit.jupiter.api.BeforeEach;
import org.junit.jupiter.api.Test;

import main.Client;

public class TestClient {
     private final PrintStream standardOut = System.out;
     private final ByteArrayOutputStream outputStreamCaptor = new ByteArrayOutputStream();

     @BeforeEach
     public void setUp() {
         System.setOut(new PrintStream(outputStreamCaptor));
     }

     @Test
     public void testXTrue() {
         Client cli = new Client();
         cli.checkBool(true, false, false);
         Assertions.assertEquals("Flag is 1", outputStreamCaptor.toString().trim());
     }

     @AfterEach
     public void tearDown() {
         System.setOut(standardOut);
     }
}

  5. Measure code coverage. This consists of compile the program and test code, generate coverage report, and examine the coverage report

    javac -cp lib/junit-platform-console-standalone-1.13.0-M3.jar:bin \
  -d bin src/main/*.java src/test/*.java
   
mkdir report

java -javaagent:lib/jacoco-0.8.12/lib/jacocoagent.jar=destfile=report/jacoco.exec ^
  -jar lib/junit-platform-console-standalone-1.13.0-M3.jar ^
  --class-path bin/ ^
  --scan-class-path

java -jar lib/jacoco-0.8.13/lib/jacococli.jar report report/jacoco.exec \
  --classfiles bin \
  --sourcefiles src \
  --html report/html \
  --xml report/report.xml \
  --csv report/report.csv

    In the above, the reports are generated in three formats, html, xml, and csv. Let’s examine them, for instance, open the report/html/index.html file using a Web browser.

  6. Write more test case, and report the above step.

Using an IDE for Code Coverage

Modern IDE’s often integrates code coverage tool. For instance, JaCoCo can be integrated with a number of build tools, and can thus be integrated with the IDE’s that support these build tools. In this part, we take a simple approach, i.e., to use Visual Studio Code’s test coverage support.

  1. Download and install Visual Studio Code from https://code.visualstudio.com/download if you haven’t done so.

  2. Set up Visual Studio Code for Java projects by following https://code.visualstudio.com/docs/java/java-project

  3. Let’s begin with an unmanaged Java project. Create directories, e.g.,

    mkdir myproject1
cd myproject1
mkdir -p lib bin src/main src/test

  4. Download the Junit 5 Jar file to the lib directory as before.

  5. Create the .vscode/settings.json file, follow the instruction at https://code.visualstudio.com/docs/java/java-project, set java.project.referencedLibraries to reference the path of the Junit 5 Java file, e.g.,

    {
 "java.project.sourcePaths": ["src"],
 "java.project.outputPath": "bin",
 "java.project.referencedLibraries": [
     "lib/**/*.jar"
 ]
}

    The settings file also sets the source and output paths.

  6. Write the program file, e.g., src/main/Client.java

  7. Write the test file, e.g., src/test/TestClient.java

  8. Follow the instruction https://code.visualstudio.com/docs/debugtest/testing#_test-coverage to compute and observe code coverage.