Incremental Code Coverage using Emma

 

We are working on a web based project. Due to the wide spread (and various other reason) we don’t have a complete automation for testing end-end functionality of the project. Recently, we have started using Emma to analyze the amount of code that is really being exercised. This is when we hit with a hard problem (at least from my perspective)

                How do we do incremental code coverage of a product during development – testing phase?

To explain the problem, let’s say we have class JustForEmma.java as shown below

package test;
/**
  * @author veechand
 */
public class JustForEmma {
    public static void main(String[] args) {
        JustForEmma a = new JustForEmma();
        if (args[0].equals("True")) {
               System.out.println("True");
         } else {
             System.out.println("False");
         }
      System.out.println("Hey I am sleeping");
    }
}

 

 

Now this code is released for testing. All parts of the code are tested, i.e. Code coverage of 100% is attained (in this example case it is very easy, but statistics says industrial standard for code coverage is only 80%)  

After this, let’s assume due to requirement changes our JustForEmma class has undergone a change, more expected in a project of any size. New class file is as below

package test;
/**
 * @author veechand
 */
public class JustForEmma {
  public static void main(String[] args) {
        JustForEmma a = new JustForEmma();
        if(args[0].equals("Changed")){
                 System.out.println("Changed");
                 return;
         }
        if (args[0].equals("True")) {
            System.out.println("True");
        } else {
             System.out.println("False");
        }
     System.out.println("Hey I am sleeping");
  }
}

 

             Now do we need to repeat the complete set of test cases again (i.e. executing the program with True, False and then Changed) or just executing the test cases needed for exercising the new piece of code. When visualized in a project of larger size with millions of LOC, the second choice will be an ideal one. I just started to find a solution for this, keeping my limited experience in Emma. 

         At first I was confused with the incremental instrumentation in emma document, that is quite different from the item we are discussing here.  I tried by having more than one metadata file, one for the class file without change and one with change.  This doesn’t work out as the old coverage files are not compatible with the new metadata file and vice-versa. After some tries in vain, I just thought about combining session file. When I did merging of session file first time I was not able to get the expected result. But by luck I found the solution. The trick here is we need to give the newer session files followed by the older session files. Once this is done I was able to achieve the needed result.  Following are the series of commands that I used to achieve the result

 

1.       Get new Java file

2.       Instrument the class file : java -classpath <path to emma.jar>  emma instr -ip “<path to class files to instrument>” -m fullcopy -d <temporary location to copy instrumented files>. At the end of this step you will be getting a metadata file,  by default coverage.em

3.       Execute the test cases. You will get the coverage file, by default coverage.ec when JVM exists.

4.       Create a session file : java -classpath “<path to emma.jar>” emma merge -in <metadata file generated in step 2>,<coverage file generated in step 3> -out coverage.es

5.       Change the java file and repeat steps 2, 3, 4. Make sure that you change the session file name generated in step 4.

6.       Generate report (when ever needed) java -classpath <path to emma.jar> emma report -in <session file, newest first> -r html -Dreport.html.out.file=<path to html file> -sp <path to source file>

Note:

1.       All the options specified in the above commands are not mandatory. But I found these options useful

2.       The same can be achieved by ANT task too. 

Take a minute to comment, if you find this useful or if you have more efficient way 🙂

 
Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

%d bloggers like this: