Migrating to TestNG

August 2005
Discussion
TestNG is a promising new testing framework that addresses many of the shortfalls of JUnit in an elegant and extendable manner, without sacrificing any of the convenience.
This article will not focus on the merits of TestNG over JUnit (or any other testing framework), there are plenty of presentation slides and articles published covering the benefits and advantages.
However, one key blocker to TestNG‘s adoption is the huge mass of existing legacy JUnit tests. Most projects have hundreds, if not thousands of JUnit tests already written. Another is that most IDE‘s include JUnit support built-in, where it is possible to run any collection tests in methods, classes, and packages.
In terms of IDE support, the recently released IDEA and Eclipse plugins should be more than sufficient. Both adhere closely to their JUnit counterparts, and integrate well within the IDE environment. In IDEA‘s case, the plugin will also perform JUnit test migration for you, and the Eclipse plugin will likely gain that functionality soon as well. Integration with third party tools can likewise be leveraged by adding a JUnit reporter that will spit out xml files identical to those JUnit emits.
As for migration, it is often difficult to put forth a business case for investing more than minimal time and effort in achieving this. So clearly, for TestNG to achieve wider adoption, there must be a clearly documented and easy path for migration.
This guide will illustrate a number of tools that greatly ease the pain of that migration. The entire process should not take more than 10 minutes.
There are a number of options available for migration, and which you choose largely depends on how brave you’re feeling. The two main options are outlined below:
Option 1: No source changes...yet
This option is to get TestNG in the door, without actually using any of its features. The main advantage of this approach is that your source code is completely unmodified. It is run as-is. The only difference is that instead of JUnit being the invoker, TestNG is.
What does this gain you? It allows you to mix and match JUnit and TestNG style tests. You can for example keep using your legacy JUnit tests and use TestNG for all new tests, while incrementally migrating old tests to the new syntax on an ad-hoc basis.
Option 2: Source changes
Conveniently, TestNG comes with a simple utility that goes through your JUnit test source files and converts them to use TestNG annotations. The utility allows for generatin sources in-place or in a new location.
The advantages of this approach is that you can immediately harness the power of TestNG and start annotating your tests and cleaning up all the workarounds that are common in order to achieve state across tests in JUnit (such as statics). The starting point is the standard JUnit behaviour expressed in TestNG syntax, but it becomes very easy to move forward from this with either new tests or the existing ones. A caveat though is that any tests you have that create dynamic test suites or rely on obscure JUnit features will likely need to be hand-edited.
Regardless of whether you go with source changes or not, the steps involved are roughly similar:
1) Create testng.xml: The testng.xml file defines the test suite, and consists of one or more collections of test classes. This file is optional and it is possible to instead use a class pattern in build.xml.
2) Modify your ant task: The invocation for testng has fewer options than the junit task, so this will mostly involve deleting things.
The first step varies on whether you want to go with option 1 or 2. For the first option, assuming you decide to specify a testng.xml, you will need a utility class that will walk through your source tree to generate the file. TestNG has a helper class that will take in a list of class names, and output a testng.xml file. Here is a sample implementation that will walk a directory tree and create the desired file:
public class JUnitSuiteGenerator { public static void addClasses(File base, File dir, List classes) { File[] files = dir.listFiles(); for (int i = 0; i < files.length; i++) { File file = files[i]; if (!file.isDirectory()) { if (file.getName().endsWith(".java")) { String path = file.getAbsolutePath().substring(base.getAbsolutePath().length()); int startPos = 0; if (path.charAt(0) == File.separatorChar) { startPos = 1; } path = path.replace(File.separatorChar, ‘.‘); path = path.substring(startPos, path.length() - 5); classes.add(path); } } else { addClasses(base, file, classes); } } } public static void main(String[] args) { File root = new File(args[0]); List classes = new ArrayList(); addClasses(root, root, classes); //See javadocs for SuiteGenerator at http://www.testng.org/javadocs/ File file = SuiteGenerator.createCustomizedSuite ("MyProject", classes, null, null, null, null, 0).save(new File(args[1])); System.out.println("Generated suite file at " + file); }
Running this class will produce a file called something like:
MyProject_Custom_suite_c9d431a9.xml
The file is a good starting point, but will likely need some tweaking for your usage; picking more appropriate names and excluding helper/abstract classes that do not contain tests.
Note also that while the sample code above produces an entry for every class, it is possible to tweak it to produce a list of packages, since testng.xml can include any mixture of classes and packages.
The second approach is to use the bundled converter utility that will modify your tests and generate a testng.xml file for you. While an ant task is available for this, configuring it is somewhat annoying as it requires that you put some things in the ant classpath. Easier therefore to simply invoke it via the command line. Do not forget to back up your sources first!
An example invocation is:
java -classpath lib/testng-2.4.3-jdk15.jar org.testng.JUnitConverter -overwrite -restore -annotation -srcdir src/test
A whirlwind tour of the parameters for the above command, in order, is run the converter, overwrite existing source files, restore the same package structure as the original sources, use 1.5 annotations (instead of javadoc annotations), and convert all source files under the src/test directory.
The utility also supports an -outputdir parameter that will emit the modified annotated classes to a different destination.
Note that the command will likely spit out a whole bunch of errors about classes that aren’t found; these can be safely ignored since they are emitted by javadoc which is used internally by the utility, and so cannot be suppressed. Your sources will now be annotated, and you will have a testng.xml in the src/test directory.
Having modified the test sources, the next step is to compile them to verify the source changes. Your regular test compile process might need to be modified to specify -source 1.5 -target 1.5, since the new sources contain annotations which are a 1.5 specific feature. Of course, you don‘t need to do this if you use 1.4 style javadoc annotations which TestNG fully supports.
One potential pitfall is that the TestNG annotations used are ‘Test‘ and ‘Configuration‘, so if your tests use classes with these names, then you‘ll likely have compile errors due to the naming conflict. The resolution is the same as it‘d be for any different-package-same-classname issue; fully qualify either the class usage in your source or the annotation.
Once testng.xml is created and configured, the next and final step is to convert the junit invocation from ant.
Typically, an ant task in build.xml will define a batchtest, some formatters, and a variety of attributes to the junit task itself.
The testng task is much simpler. Consult the TestNG documentation for all the options available. A basic example is below :

You can now run ant test (assuming you have a compile target elsewhere and all your resources have been copied to the appropriate locations and so on).
At this point, your tests have been converted to using TestNG and can proceed with the rest of its features. However, there is still a dependency on junit.jar since the testcases still extend it, and likely call the asserts inherited from the Assert superclass.
So, if you really want to purge all traces of junit, two final steps remain:
- Change your asserts to use JDK 1.4 asserts, or use TestNG‘s Assert class, which is JUnit‘s equivelant with the same methods.
- Do not extend TestCase. Once you‘re no longer using the assert methods inherited, it should be a trivial matter to simply remove all references to TestCase. In some cases, you might be extending a third library addon to JUnit that provides its own testcase class (for example, XmlTestCase). For such usages, there‘s no harm in simply leaving the existing class in place.
The most interesting effect of the migration is how readily some of JUnit‘s weaknesses are revealed. I will attempt to list some of the fragile test code that was present in JUnit, and how it was refactored for TestNG.
1) Fragile setUp()/tearDown(): If you have a test hierarchy whereby most of your tests extend a common base class which performs some work in setUp(), a common problem is a subclass forgetting to call super.setUp() if it has to do its own init. This is a design flaw as the baseclass is forced to know some of what is going on in the parent. TestNG allows you to easily fix this, since the superclass setUp() method can now be renamed or something else and annotated, without the subclasses knowing anything about it. You can have multiple pre/post test methods too.
2) Per test/suite setup: This has to be one of the biggest gains. No more statics! TestNG allows you to annotate any method to be run before any suite, class, or test. While this might be frowned on by the purists, it is incredibly useful to be able to define a baseline state for all tests to go against.
3) Unconstrained method names: The reliance on annotations instead of magic names also highlights a design smell we‘re all very used to by now in all JUnit code. No longer are you restricted to naming conventions that might or might not make sense. You can free to define your own, depending on what makes most sense in your case. No common "test" prefix is mandated for tests.
In conclusion, a migration from even a large codebase of JUnit tests to TestNG need be neither painful nor time-consuming. While there are definitely corner cases that will need to be manually addressed, the process can, to a large degree, be automated. Having made the conversion, the pay off will be immediately apparent, and the overall functionality possible for tests greatly enhanced. Happy testing!
Biography
Hani Suleiman is the CTO of Formicary, a company that focuses on financial consulting services and provides portal solutions (http://www.go-epix.net). Hani is also the author of the bileblog, a controversial blog discussing all things Java.
_xyz