Jacoco Code coverage with Robotframework

转自：http://dmeio.blogspot.fr/2014/02/code-coverage-with-robotframework.html

If you are here, you probably know what code coverage is, and what Robotframework is either. This article is about measuring of Java code coverage during Maven build. As a tool for this, JaCoCo (Java Code Coverage library) is used. Configuration is based on a requirements of a project where I was implementing this. If you will find some part sub-optimal, please let me know.

Requirements and preconditions:

coverage is measured during the (Maven) build
coverage is aggregated for both Unit and Robot tests
application run in the Robot tests themselves

Maven and JaCoCo

JaCoCo itself has pretty good support for Maven builds, and you can find example configurations on JaCoCo website. You can get coverage for your Unit tests almost for free. As you can see, there is not any configuration for this in example POM file. Internally JaCoCo Maven plugin sets argLine variable during prepare-agent goal. This variable is used by surefire and failsafe plugin as JVM arguments. Robotframework doesn't support this argument for standard tests launch, but it can be used with Robot's external runner.

Unit and Robot tests result aggregation

This one is easy too, JaCoCo by default appends its data files. So you can run JaCoCo several times, and only after that generate the report. Here you only need assure that the same file is used by different runs and report generation is done only after last analysis.

Application running in tests

On the given project, application is started during first Robot test setup stage. This actually means that in-memory database, mock components and static data are initialized. This has advantage of simple "mvn install" to run all the tests without any setup and configuration. From coverage point of view, you only need configure coverage agent for tests, but not for separately running application. Disadvantage is, that it complicates initial setup of the tests.

Example

You can checkout example Maven project from this github url:

https://github.com/DeziderMesko/RobotframeworkCodeCoverage

This projects simulates use case, where you have some library, which you are testing from both Unit and Robot tests, and you want measure aggregated coverage of these two.

POM file

Jacoco configuration:

 <executions>  
  <execution>  
   <id>prepare-agent</id>  
   <goals>  
    <goal>prepare-agent</goal>  
   </goals>  
  </execution>  
  <execution>  
   <id>report</id>  
   <phase>verify</phase>  
   <goals>  
    <goal>report</goal>  
   </goals>  
  </execution>  
 </executions>

As I already mentioned, execution prepare-agent initializes variable argLine. If you need separate Unit tests and Robot tests coverage, add preprare-agent-integration execution, set its propertyName and use this name instead of argLine. Prepare-agent-integration execution is available only from JaCoCo version 0.6.4.

Robot configuration:

 <executions>  
  <execution>  
   <goals>  
    <goal>acceptance-test</goal>  
    <goal>verify</goal>  
   </goals>  
   <configuration>  
    <skip>${skipRobotTests}</skip>  
   <testCasesDirectory>.../src/test/robot/</testCasesDirectory>  
    <outputDirectory>.../target/robot-reports/</outputDirectory>  
    <externalRunner>  
     <excludeDependencies>false</excludeDependencies>  
     <jvmArgs>  
      <jvmArg>${argLine}</jvmArg>  
     </jvmArgs>  
    </externalRunner>  
   </configuration>  
  </execution>  
 </executions>

Here the magic comes. You need to use external runner to launch your Robot tests, and set jvmArg to appropriate value. In our case it is argLine variable, however you can hardcode string like this:

 <jvmArg>-javaagent:.../jacocoagent-0.6.2.jar=destfile=.../jacoco.exec</jvmArg>

Java and Robot files

These files aren't special in any way. Test library is just simple add function. Null checks of this function is tested by Unit tests, adding itself is tested by the Robot test. When you run just simple mvn clean install, resulting coverage reports should be 100%. If you run same command with -DskipRobotTests=true or -DskipUnitTests=true, coverage should change accordingly.

Conclusion

I hope that this article helped you with code coverage measurement of your project. I want just remind you, that 100% coverage is not needed and its probably even waste of time (and money) and at the same time, you probably should have right reasons to measure coverage of integration/acceptance tests.