µJava (muJava) is a mutation system for Java programs. It automatically generates mutants for both traditional mutation testing and class-level mutation testing. µJava can test individual classes and packages of multiple classes. Tests are supplied by the users as sequences of method calls to the classes under test encapsulated in methods in separate classes.

µJava is the result of a collaboration between two universities, Korea Advanced Institute of Science and Technology (KAIST) in S. Korea and George Mason University in the USA. The research collaborators are Yu Seung Ma, PhD candidate at KAIST in Korea, Dr. Yong Rae Kwon, Professor at KAIST in Korea, and Dr. Jeff Offutt, Professor at George Mason University in the USA. Most of the software development was done by YuSeung. Source files are available on a limited basis to researchers in mutation analysis; please contact Offutt for more information

We offer µJava on an "as-is" basis as a service to the community. We welcome comments and feedback, but do not guarantee support in the form of question answering, bug fixing, or improvements (in other words, we don't have money for support, just good intentions). This page contains the following:

1. An overview of object-oriented Java and µJava.
2. References to published papers.
3. Links to downloadable files.
4. Detailed description of how to install, set up, and run µJava.

March 2007: Laurie Williams and Ben Smith of NC State have modified µJava to be an Eclipse plugin. Their version is available from sourcefourge.

I. Overview

µJava was built by Ma, Offutt and Kwon [1]. µJava uses two types of mutation operators, class level and method level. The class level mutation operators were designed for Java classes by Ma, Kwon and Offutt [2], and were in turn designed from a categorization of object-oriented faults by Offutt, Alexander et al. [3]. µJava creates object-oriented mutants for Java classes according to 24 operators that are specialized to object-oriented faults. Method level (traditional) mutants are based on the selective operator set by Offutt et al. [4]. After creating mutants, µJava allows the tester to enter and run tests, and evaluates the mutation coverage of the tests.

In µJava, tests for the class under test are encoded in separate classes that make calls to methods in the class under test. Mutants are created and executed automatically. Equivalent mutants must identified by hand.

We have separate descriptions for the class-level mutation operators and the method-level mutation operators used by µJava in PDF.

History

• 2003. First released as JMutation (Java Mutation System).
• 2004. The name was changed to MuJava (Mutation System for Java).
• 2005. Software Copyright Registration, ALL RIGHTS RESERVED.
• 2005. Version 2 released with several bug fixes and modified mutation operators.
• 2008. Version 3 released for Java 1.5 and 1.6.

II. Published papers

For awhile we planned to list all papers published using µJava. This became impractical very quickly and is probably impossible. Google scholar lists hundreds of citations to the various µJava papers (241 to reference [1] below in December 2011, 311 in December 2012). We are glad that this tool has been of so much help!

1. MuJava : An Automated Class Mutation System, Yu-Seung Ma, Jeff Offutt and Yong-Rae Kwon. Journal of Software Testing, Verification and Reliability, 15(2):97-133, June 2005.
2. The Class-Level Mutants of MuJava, Jeff Offutt, Yu-SeungMa and Yong-Rae Kwon. Workshop on Automation of Software Test (AST 2006), pages 78-84, May 2006, Shanghai, China.
3. Inter-Class Mutation Operators for Java, Yu-Seung Ma, Yong-Rae Kwon and Jeff Offutt. Proceedings of the 13th International Symposium on Software Reliability Engineering, IEEE Computer Society Press, Annapolis MD, November 2002, pp. 352-363.
4. An Experimental Mutation System for Java, Jeff Offutt and Yu-Seung Ma and Yong-Rae Kwon. ACM SIGSOFT Software Engineering Notes, Workshop on Empirical Research in Software Testing, 29(5):1-4, September 2004.
5. A Fault Model for Subtype Inheritance and Polymorphism, Jeff Offutt, Roger Alexander, Ye Wu, Quansheng Xiao and Chuck Hutchinson. Proceedings of the 12th International Symposium on Software Reliability Engineering, IEEE Computer Society Press, Hong Kong China, November 2001, 84-93.
6. An Experimental Determination of Sufficient Mutation Operators, Jeff Offutt, A. Lee, G. Rothermel and R. Untch and C. Zapf, ACM Transactions on Software Engineering Methodology April 1996 5(2):99-118.
7. Rostra: A Framework for Detecting Redundant Object-Oriented Unit Tests, Tao Xie, Darko Marinov, and David Notkin. Proceedings of the 19th IEEE International Conference on Automated Software Engineering (ASE 2004), Linz, Austria, September 2004, 196-205.
   http://www.cs.washington.edu/homes/taoxie/publications/ase04.pdf.
   Used MuJava in an experimental study.
8. A Passive Test Oracle Using a Component's API, Rakesh Shukla, David Carrington and Paul Strooper. Proceedings of the 12th Asia-Pacific Software Engineering Conference (APSEC'05), Taipei, Taiwan, pp. 561-567, 2005.
   Used MuJava to seed faults into classes for an experiment.
9. Integrating techniques and tools for testing automation, Macario Polo, Sergio Tendero and Mario Piattini. Journal of Software Testing, Verification and Reliability, 2006 (early view posting 11 May, 2006).
   Used MuJava to seed faults for experimentation.
10. On Random Testing of Image Processing Applications, Johannes Mayer, Ralph Guderlei. accepted by QSIC 2006
    Used MuJava to seed faults into classes for an experiment.
11. An Empirical Study on the Selection of Good Metamorphic Relations, Johannes Mayer, Ralph Guderlei. accepted by COMPSAC 2006
    Used MuJava to seed faults into classes for an experiment.
12. Efficient and Effective Random Testing based on Partitioning and Neighborhood, Johannes Mayer. Proceedings of the 18th International Conference on Software Engineering and Knowledge Engineering (SEKE 2006), Knowledge Systems Institute Graduate School, Skokie, USA, 2006, pp. 479-484.
    Used MuJava to seed faults into classes for an experiment.
13. An Empirical Analysis and Comparison of Random Testing Techniques, Johannes Mayer and Christoph Schneckenburger. Proceedings of the 2006 International Symposium on Empirical Software Engineering (ISESE).
    Used MuJava to seed faults into classes for an experiment.
14. Simulation-Based Testing of Distributed Systems, Matthew J. Rutherford, Antonio Carzaniga, and Alexander L. Wolf. Proceedings of 14th ACM SIGSOFT Symposium on Foundations of Software Engineering, Portland, OR, November, 2006.
    http://serl.cs.colorado.edu/~rutherfo/pubs/RutherfordCarzanigaWolfFSE06.pdf.
    Used MuJava to seed faults into classes for an experiment.
15. Integrating techniques and tools for testing automation, Macario Polo, Sergio Tendero and Mario Piattini. Journal of Software Testing, Verification and Reliability, 17(1):3-39, January 2007.
16. A Framework for Statistical Testing of Software Components, Rakesh Shukla, Paul Strooper, and David Carrington. International Journal of Software Engineering and Knowledge Engineering, 17(3):379-405, June 2007.
    Used MuJava to seed faults into classes for an experiment.
17. An Evaluation of Mutation Operators for Equivalent Mutants, Maryam Umar. MS Project, Kings College, London, Supervisor: Mark Harman.
    Investigated the frequency of equivalent mutants created by MuJava.
18. State-based Testing is Functional Testing, F. Ipate and R. Lefticaru. Proceedings of the IEEE Testing: Academic and industrial Conference Practice and Research Techniques - MUTATION, pp. 55-66, September 2007.
19. Automatic State-Based Test Generation Using Genetic Algorithms, R. Lefticaru and F. Ipate. Proceedings of the Ninth IEEE international Symposium on Symbolic and Numeric Algorithms For Scientific Computing (SYNASC), September, 2007, pp. 188-195.

III. Downloading µJava

Although installing µJava is not technically difficult or complicated, it will require a little more effort than installing many commercial packages. µJava can run on Unix, Linux, and Windows platforms. You will need to download and deploy three files; two Java "jar" files and a configuration file.

The two jar files should be placed in a directory on disk that all users who need to use µJava can access, for example, in C:\mujava\.

Issues with version 3:

• µJava cannot compile classes that use Java generics. This is clearly a problem that we hope to correct.
• Fault fix, 5-December-2011: The program did not properly initialize the time-out counter, so time-out only worked if users explicitly set it. This caused µJava to freeze during execution sometimes.
• Fault fix, 6-December-2011: muJava implemented the ROR operator incorrectly.

µJava was updated in December 2008 to work with Java 1.5 and 1.6. The old versions are still available if you need it.

Version 2

mujava-v2.jar	µJava system library (Updates and bug fixes, May 2006)
adaptedOJ.jar	OpenJava library that is adapted for µJava (OpenJava WWW page)
mujava.config	A file to specifies the µJava system home directory

The mujava.jar file was updated in May 2006. The old version, mujava-v1.jar, is still available if you need it.

Although we have no license agreement, if you use µJava, we would like to hear about it. Particularly, if you publish papers that rely on µJava, please send an email to "offutt [at] gmu.edu" and we will add the reference to the µJava web page.

Version 1

A new version of µJava was made available in summer 2005. Some of the operators were modified and numerous bug fixes were made. For completeness, the previous version is still available, but we recommend that new users use the current version.

IV. Using µJava to Test Classes

The muJava system requires that the Java CLASSPATH be modified to include the muJava jar and the Java tools.jar files. Also, the general PATH variable must include the java bin directory; this is usually set automatically when java was installed, but not always. Then one GUI (Java applet) is used to generate mutants, the tester must create tests, and another GUI is used to run mutants.

1. Environment Settings for the muJava System

There are three steps to setting up the environment for muJava, (1) CLASSPATH, (2) setting the config file, and (3) creating subdirectories.

1. The Java CLASSPATH must include two µJava jar files and one standard Java jar file. tools.jar is standard with Java compilers and is probably located in the "lib/" directory. The two µJava files are mujava.jar and openjava2005.jar, which are downloaded from this site. One slightly awkward requirement is that the CLASSPATH must include the location of the classes under test when generating mutants, but NOT when running mutants. (If it does, no mutants can be killed.) This location is the "classes/" directory under where the mujava.config is stored.

   In a DOS window, use the following command (assuming that classes is under C:\mujava):

   set CLASSPATH=%CLASSPATH%;C:\mujava\mujava.jar;C:\mujava\openjava2005.jar;C:\j2sdk1.4.0_01\lib\tools.jar;C:\mujava\classes

   In a Cygwin window, use the following command:

   CLASSPATH="$CLASSPATH;C:\mujava\mujava.jar;C:\mujava\openjava2005.jar;C:\j2sdk1.4.0_01\lib\tools.jar;C:\mujava\classes" ; export CLASSPATH

   To change your CLASSPATH permanently in Win2000 and WinXP, go to Start-settings-Control Panel. Double-click System, go to the Advanced tab, and choose Environment Variables. Edit the CLASSPATH variable or create a new variable if there is none. Add the full path to mujava.jar and openjava2005.jar to the CLASSPATH.

   In Unix, set the CLASSPATH environment variable. Assuming the jar files are stored in the home directory of user gpd:
   CLASSPATH=$CLASSPATH:/home/gpd/mujava.jar:/home/gpd/openjava2005.jar:/java1.4/j2sdk1.4.0_01/lib/tools.jar ; export CLASSPATH
   Note that the syntax will vary under different shells, and it will be more convenient to put the command in your setup file such as .login, or .bashrc.

2. Next, modify the mujava.config file to point to a directory where you wish to store source Java files and muJava temporary files. The directory must be the complete path (either Windows or Unix). For example, the config file may contain the line: MuJava_HOME=C:\home\gpd\exp.

   IMPORTANT: It is necessary to copy the config file to the directory you run the muJava system in.

3. Finally, create a directory structure for the muJava system in the $MuJava_HOME directory. Assuming your MuJava_HOME directory is called MuJava, the subdirectories should look like:

   

   There should be four subdirectories, used in the following ways. MuJava_HOME\src

   directory for Java files to be tested
   MuJava_HOME\classes	directory for compiled classes of Java files from MuJava_HOME\src
   MuJava_HOME\testset	directory for test sets
   MuJava_HOME\result	directory for generated mutants

   You can create these subdirectories by hand or by using the muJava class "mujava.makeMuJavaStructure".

   java mujava.makeMuJavaStructure

Potential problems: We have identified several potential problems with installing µJava.

• It is important that the MuJava_HOME variable NOT have a trailing slash. This will confuse µJava.
• If you have a different version of the java compiler and the JVM, µJava may get confused. This happens sometimes when a new application on your computer updates the JVM. If you have problems compiling or killing mutants, we suggest deleting all Java components and reinstalling the latest version.
• If your tools.jar file is out of date (pre Java 1.4, we think), parts of µJava may not work.

2. Generating Mutants with muJava

Important note: You should run all commands in a directory that contains "mujava.config"

1. Put the source files to test to MuJava_HOME\src directory. muJava does not check for compilation errors, so all Java files should compile correctly. If the Java file to test needs other Java files or class files, they should also be placed in MuJava_HOME\src. For example, suppose you want to test B, which is a child class of A. Then, you should put both A.java and B.java into MuJava_HOME\src. If the file has a package structure, you should store the entire package underneath MuJava_HOME\src.
2. Compile all the Java files in MuJava_HOME\src and copy the .class files into the MuJava_HOME\classes\ directory.
3. Start the GUI from the command line. Use it to generate mutants:

   java mujava.gui.GenMutantsMain

   This command should bring a up a screen similar to the following:

   

   Select the files you want to mutate by clicking in the boxes on the left. Select the mutation operators you want to use by selecting their boxes. Then push RUN.

   Note: The class mutation operators produce far fewer mutants. Also note that a number of status messages go to the command window, but not the GUI.

   After mutants are generated, you can view the mutants in the "Class Mutants Viewer" and "Traditional Mutants Viewer" tabs, as shown in the following two figures.

   
   

   You may be interested in knowing where the mutant versions of the classes are stored. They are underneath the MuJava_HOME\result\ directory. The following example shows the directory Stack underneath result, with object-oriented mutants in class_mutants and traditional mutants in a separate directory.

3. Making a test set

A testset in muJava is a Java file that contains executable test scripts. Each test is a method that contains a sequence of calls to methods in the class under test. Each test method returns a string result that is used to compare outputs of mutants with outputs of the original class. Each test method should start with the string "test". The test methods and the test class should have public access.

Below is an example of a testset class for the class Stack. Its name is StackTest. StackTest contains two test case: test1() and test2(). The testset .class file should be in the directory MuJava_HOME\testset\. (By the way, although these test scripts are reminiscient of JUnit tests, µJava does not accept JUnit tests. The first version of µJava predated JUnit so we implemented a general test driver facility.)

4. Running mutants.

Run the mutants from another GUI. Start it with the following command:

java mujava.gui.RunTestMain

Note: Your CLASSPATH must not include MuJava_HOME\classes\. If it does, no mutants can be killed.

You should see the following GUI. You can select which collection of mutants to run, and which testset to use. The "Class Mutants Viewer" and "Traditional Mutants Viewer" tabs will show the source view of the mutants. You can design tests to kill mutants by finding a live mutant, then analyzing the program to decide what input will kill it. Remember that between 5% to 20% of the mutants are typically equivalent.

Last update: December 2011, Jeff Offutt and Nan Li

Assistance from the U.S. National Science Foundation
under grant CCR-98-04111 to George Mason University
was greatly appreciated.