JUnit 4 vs. TestNG

From NeoWiki

Revision as of 07:39, 3 March 2007

Why TestNG is still the better framework for large-scale testing

Andrew Glover, President, Stelligent Incorporated

29 Aug 2006

With its new, annotations-based framework, JUnit 4 has embraced some of the best features of TestNG, but does that mean it's rendered TestNG obsolete? Andrew Glover considers what's unique about each framework and reveals three high-level testing features you'll still find only in TestNG.

JUnit 4.0 was released early this year, following a long hiatus from active development. Some of the most interesting changes to the JUnit framework -- especially for readers of this column -- are enabled by the clever use of annotations. In addition to a radically updated look and feel, the new framework features dramatically relaxed structural rules for test case authoring. The previously rigid fixture model has also been relaxed in favor of a more configurable approach. As a result, JUnit no longer requires that you define a test as a method whose name starts with test, and you can now run fixtures just once as opposed to for each test.

These changes are most welcome, but JUnit 4 isn't the first Java™ test framework to offer a flexible model based on annotations. TestNG established itself as an annotations-based framework long before the modifications to JUnit were in progress.

In fact, TestNG pioneered testing with annotations in Java programming, which made it a formidable alternative to JUnit. Since the release of JUnit 4, however, many developers are asking if there's still any difference between the two frameworks. In this month's column, I'll discuss some of the features that set TestNG apart from JUnit 4 and suggest the ways in which the two frameworks continue to be more complementary than competitive.

Tip: Running JUnit 4 tests in Ant has turned out to be more of a challenge than anticipated. In fact, some teams have found that the only solution is to upgrade to Ant 1.7.

Similar on the surface

JUnit 4 and TestNG have some important attributes in common. Both frameworks facilitate testing by making it amazingly simple (and fun), and they both have vibrant communities that support active development while generating copious documentation.

Where the frameworks differ is in their core design. JUnit has always been a unit-testing framework, meaning that it was built to facilitate testing single objects, and it does so quite effectively. TestNG, on the other hand, was built to address testing at higher levels, and consequently, has some features not available in JUnit.

A simple test case

At first glance, tests implemented in JUnit 4 and TestNG look remarkably similar. To see what I mean, take a look at the code in Listing 1, a JUnit 4 test that has a macro-fixture (a fixture that is called just once before any tests are run), which is denoted by the @BeforeClass attribute:

Listing 1. A simple JUnit 4 test case

package test.com.acme.dona.dep;

import static org.junit.Assert.assertEquals;
import static org.junit.Assert.assertNotNull;
import org.junit.BeforeClass;
import org.junit.Test;

public class DependencyFinderTest {
  private static DependencyFinder finder;

  @BeforeClass
  public static void init() throws Exception {
    finder = new DependencyFinder();
  }

  @Test
  public void verifyDependencies() throws Exception {
    String targetClss = "test.com.acme.dona.dep.DependencyFind";
    Filter[] filtr = new Filter[] { 
      new RegexPackageFilter("java|junit|org")};

    Dependency[] deps = 
      finder.findDependencies(targetClss, filtr);

    assertNotNull("deps was null", deps);
    assertEquals("should be 5 large", 5, deps.length);	
  }
}

JUnit users will immediately note that this class lacks much of the syntactic sugar required by previous versions of JUnit. There isn't a setUp() method, the class doesn't extend TestCase, and it doesn't even have any methods that start with test. This class also makes use of Java 5 features like static imports and, obviously, annotations.

Even more flexibility

In Listing 2, you see the same test, but this time it's implemented using TestNG. There is one subtle difference between this code and the test in Listing 1. Do you see it?

Listing 2. A TestNG test case