Resolve To Get FIT

From NeoWiki

Latest revision as of 16:20, 5 March 2007

Try FIT and JUnit for a requirements testing workout!

Andrew Glover, President, Stelligent Incorporated

28 Feb 2006

Whereas JUnit assumes that every aspect of testing is the domain of developers, the Framework for Integrated Tests (FIT) makes testing a collaboration between the business clients who write requirements and the developers who implement them. Does this mean that FIT and JUnit are competitors? Absolutely not! Code quality perfectionist Andrew Glover shows you how to combine the best of FIT and JUnit for better teamwork and effective end-to-end testing.

In the software development life-cycle, everyone owns quality. Ideally, developers start ensuring quality early in the development cycle with testing tools like Junit and TestNG, and QA teams follow up with functional system tests at the end of the cycle, using tools like Selenium. But even with excellent quality assurance, some applications are deemed low-quality upon delivery. Why? Because they don't do what they were intended to do.

Communication errors between the client or the business department that authors application requirements and the development team that implements them are a frequent cause of friction, and sometimes the cause of downright failure in development projects. Luckily, there are ways to facilitate the communication between requirements authors and implementors early on.

Tip: Download FIT The Framework for Integrated Tests, or FIT, was originally created by Ward Cunningham, who is best known as the inventor of the wiki. Visit Cunningham's Web site to learn more about FIT and download it for free.

A FITting solution

The Framework for Integrated Tests (FIT) is a testing platform that facilitates communication between those who write requirements and those who turn them into executable code. With FIT, requirements are fashioned into tabular models that serve as the data model for tests written by developers. The tables themselves serve as the input and expected output for the tests.

Figure 1 shows a structured model created using FIT. The first row is the test name and the next row's three columns are headers relating to inputs (value1 and value2) and the expected results (trend()).

Figure 1. A structured model created using FIT

The nice thing is that someone who hasn't a clue how to program can write this table. FIT was designed to enable customers or business teams to collaborate earlier in the development cycle with the developers who implement their ideas. Creating simple tabular models of the application's requirements lets everyone see clearly whether code and requirements are on the same page.

Listing 1 is the FIT code that correlates to the data model in Figure 1. Don't worry too much about the details -- just note how simple the code is and that it doesn't include validation logic (i.e., assertions, etc.). You may even notice some matching variable and method names from what you saw in Table 1; more on that later.

Listing 1. Code written from the FIT model

package test.com.acme.fit.impl;

import com.acme.sedlp.trend.Trender;
import fit.ColumnFixture;

public class TrendIndicator extends ColumnFixture {
  public double value1;
  public double value2;

  public String trend(){
    return Trender.determineTrend(value1, value2).getName();
  }
}

The code you see in Listing 1 was written by a developer who studied the table and plugged in the appropriate code. Finally, pulling everything together, the FIT framework reads the data in Table 1, calls the corresponding code, and determines the results.

Tip: Ensure your code quality with FIT To get the answers to your questions related writing FIT tables, coding FIT fixtures, or running FIT, visit the Code Quality discussion forum, moderated by Andrew Glover.

FIT and JUnit

The beauty of FIT is that it enables the customer or business side of an organization to get involved in the testing process early (i.e., during development). Whereas JUnit's strength lies in unit testing during the coding process, FIT is a higher level testing tool used to determine the validity of a proposed requirements' implementation.

For example, while JUnit is adept at validating that the sum of two Money objects is the same as the sum of their two values, FIT shines in validating that the total order price is the sum of its line-item's prices minus any associated discounts. It's a subtle difference, but really important! In the JUnit example, you're dealing with specific objects (or implementations of requirements), but with FIT, you're dealing with a high-level business process.

This is significant because, usually, the people who write requirements couldn't care less about Money objects -- in fact, they may not even know such things exists! They do care, however, that when line items are added to an order, the total order price is the sum of its line items with any discounts applied.

Far from being competitive, FIT and JUnit make a great match for ensuring code quality, as you'll see in the case study further down.

Tables FIT for testing

Tables are at the heart of FIT. There are a few different types of tables (for different business scenarios), and FIT users can author tables using a variety of formats. It's possible to write tables using HTML and even Microsoft Excel, as shown in Figure 2:

Figure 2. A table written using Microsoft Excel

It's also possible to author a table using a tool like Microsoft Word and then save it in HTML format, as shown in Figure 3:

Figure 3. A table written using Microsoft Word

The code a developer writes to execute a table's data is called a fixture. To create a fixture type, you must extend the corresponding FIT fixture, which maps to the intended table. As previously mentioned, different types of tables map to different business scenarios.

Fix it with fixtures

The simplest table and fixture combination, which is most commonly utilized in FIT, is a straightforward column table, where columns map to the input and output of a desired process. The corresponding fixture type is ColumnFixture.

If you look again at Listing 1, you'll note that the TrendIndicator class extends ColumnFixture and also corresponds with Figure 3. Notice how in Figure 3, the first row's name matches the fully qualified class name (test.com.acme.fit.impl.TrendIndicator). The next row has three columns. The first two cell's values match the public instance members of the TrendIndicator class (value1 & value2) and the last cell's value matches the only method found in TrendIndicator (trend).

Now look at the trend method in Listing 1. It returns a String value. As you may have guessed by now, for each row left in the table, FIT substitutes values and compares results. In this case, there are three "data" rows, so FIT runs the TrendIndicator fixture three times. The first time, value1 is set to 84.0 and value2 is 71.2. FIT then calls the trend method and compares the value obtained from the method to that found in the table, which is "decreasing."

In this way, FIT uses the fixture code to test the Trender class, whose determineTrend method is executed each time FIT executes the trend method. When it's done testing the code, FIT generates a report like the one shown in Figure 4:

Figure 4. FIT reports the trend-test results

The green coloring of the trend column cells indicates the tests passed (i.e., FIT set value1 to 84.0 and value2 to 71.2 and received back a value of "decreasing" when trend was invoked).

See FIT run

You can invoke FIT through the command line using an Ant task, and through Maven, making it easy to plug FIT tests into a build process. Because FIT tests are automated, just like JUnit's, you can also run them at regular intervals, such as in a continuous integration system.

The simplest command-line runner, shown in Listing 2, is FIT's FolderRunner, which takes two parameters -- the location of the FIT tables and where the results should be written. Don't forget to configure your classpath!

Listing 2. FIT for the command line

%>java fit.runner.FolderRunner ./test/fit ./target/

FIT also works with Maven quite nicely with the addition of a plug-in, as shown in Listing 3. Simply download the plug-in, run the fit:fit command, and you're good to go! (See Resources for the Maven plug-in.)

Listing 3. Maven gets FIT

C:\dev\proj\edoa>maven fit:fit
  _  __
|  \/  |__ _Apache__ ___
| |\/| / _` \ V / -_) ' \  ~ intelligent projects ~
|_|  |_\__,_|\_/\___|_||_|  v. 1.0.2

build:start:

java:prepare-filesystem:

java:compile:
    [echo] Compiling to C:\dev\proj\edoa/target/classes

java:jar-resources:

test:prepare-filesystem:

test:test-resources:

test:compile:

fit:fit:
    [java] 2 right, 0 wrong, 0 ignored, 0 exceptions
BUILD SUCCESSFUL
Total time: 4 seconds
Finished at: Thu Feb 02 17:19:30 EST 2006

FIT to be tried: a case study

Now that you have the basics of FIT under your belt, let's try an exercise. If you haven't downloaded FIT yet, now is the time to do it! As previously mentioned, this case study shows how easy it is to combine testing with FIT and JUnit for multitiered quality assurance.

Imagine that you've been asked to build an order-processing system for a brewery. The brewery sells various types of drinks, but they can all be grouped into two categories: seasonal and year-round. Because the brewery operates as a wholesaler, all beverages are sold by the case. There are discount incentives for retail outlets to buy multiple cases, and the discount structure varies based on the number of cases and whether the brew is seasonal or year-round.

The tricky bit is managing these requirements. For example, if a retail store buys 50 cases of a seasonal brew, no discount is applied; but if the 50 cases are not seasonal a 12% discount is applied. If a store buys 100 cases of a seasonal brew, a discount is applied, but it's only 5%. A 100-case order of a non-seasonal drink is discounted at 17%. There are similar rules for buying in quantities of 200.

To a developer, a requirement set like this could be kind of confusing. But watch how easily our beer-brewing business analyst describes the requirements using a FIT table, in Figure 5:

Figure 5. My business requirements make perfect sense!

Table semantics

That table makes sense from a business perspective, and it does map out the requirements nicely. But as a developer, you'll need to know a little more about its semantics to get value from it. First and foremost, the initial row found in the table states the table's name, which incidentally corresponds to a matching class (org.acme.store.discount.DiscountStructureFIT). Naming requires some collaboration between the table's author and you, the developer. At minimum, you need to specify a fully qualified table name (that is, you have to include the package name because FIT dynamically loads the corresponding class).

Notice how the table's name ends with FIT. Your first inclination may be to end it with Test, but doing so could cause some clashing with JUnit if you run FIT tests and JUnit tests in an automated environment. JUnit classes are usually found through a naming pattern, so you're best off to avoid ending or beginning your FIT table name with Test.

The next row contains five columns. The strings found in each cell are intentionally formatted using italics, which is a FIT requirement. As you learned earlier, the cell names match instance members and methods of fixtures. To be more precise, FIT assumes any cell whose value ends in parentheses is a method and any value that doesn't end in parentheses is an instance member.

Special intelligence

FIT uses intelligent parsing when it comes to cell values for matching to a corresponding fixture class. As you can see in Figure 5, the second row's cell values are written in plain English, such as "number of cases." FIT attempts to concatenate a string like this through camel casing; for example, "number of cases" becomes "numberOfCases," which FIT then attempts to locate in the corresponding fixture class. This principle also works for methods -- as you can see in Figure 5, where "discount price()" becomes "discountPrice()."

FIT also makes intelligent guesses as to a particular cell value's type. For example, in the eight remaining rows of Figure 5, each column has a corresponding type that is either guessed accurately by FIT or requires some custom programming. In this case, Figure 5 has three different types. The column associated with "number of cases" is matched to an int, and the column values associated with "is seasonal" is matched to a boolean.

The three remaining columns, "list price per case," "discount price()," and "discount amount()" obviously represent currency values. These require a custom type, which I'll call Money. As it turns out, the application requires an object to represent money, so I'll be able to utilize this object in my FIT fixture by just obeying a few semantics!

Summary of FIT semantics

Table 1 summarizes the relationship between named cells and a corresponding fixture's instance members:

Table 1. Cell-to-fixture relationship: instance members

Table 2 summarizes the relationship between FIT-named cells and a corresponding fixture's methods:

Table 2. Cell-to-fixture relationship: methods

Time to build!

The order-processing system you're building for the brewery has three main objects: a PricingEngine, which embodies the business rules for obtaining discounts, a WholeSaleOrder to represent an order, and a Money type to represent money.

One for the Money ...

The first class to be coded is the Money class, which has methods for adding, multiplying, and subtracting values. You can use JUnit to test the newly created class, as shown in Listing 4:

Listing 4. JUnit's MoneyTest class

package org.acme.store;

import junit.framework.TestCase;

public class MoneyTest extends TestCase {

  public void testToString() throws Exception{
    Money money = new Money(10.00);
    Money total = money.mpy(10);
    assertEquals("$100.00", total.toString());
  }

  public void testEquals() throws Exception{
    Money money = Money.parse("$10.00");
    Money control = new Money(10.00);
    assertEquals(control, money);
  }

  public void testMultiply() throws Exception{
    Money money = new Money(10.00);
    Money total = money.mpy(10);

    Money discountAmount = total.mpy(0.05);
    assertEquals("$5.00", discountAmount.toString());
  }

  public void testSubtract() throws Exception{
    Money money = new Money(10.00);
    Money total = money.mpy(10);

    Money discountAmount = total.mpy(0.05);
    Money discountedPrice = total.sub(discountAmount);
    assertEquals("$95.00", discountedPrice.toString());
  }
}

The WholeSaleOrder class

Next, the WholeSaleOrder type is defined. This new object is central to the application: If a WholeSaleOrder type is configured with the number of cases, the price per case, and the product type (seasonal or year 'round), it can be given to the PricingEngine, which determines the corresponding discount and configure it accordingly in a WholeSaleOrder instance.

The WholesaleOrder class is defined in Listing 5:

Listing 5. The WholesaleOrder class

package org.acme.store.discount.engine;

import org.acme.store.Money;

public class WholesaleOrder {

  private int numberOfCases;
  private ProductType productType;
  private Money pricePerCase;
  private double discount;

  public double getDiscount() {
    return discount;
  }

  public void setDiscount(double discount) {
    this.discount = discount;
  }

  public Money getCalculatedPrice() {
    Money totalPrice = this.pricePerCase.mpy(this.numberOfCases);
    Money tmpPrice = totalPrice.mpy(this.discount);
    return totalPrice.sub(tmpPrice);
  }

  public Money getDiscountedDifference() {
    Money totalPrice = this.pricePerCase.mpy(this.numberOfCases);
    return totalPrice.sub(this.getCalculatedPrice());
  }

  public int getNumberOfCases() {
    return numberOfCases;
  }

  public void setNumberOfCases(int numberOfCases) {
    this.numberOfCases = numberOfCases;
  }

  public void setProductType(ProductType productType) {
    this.productType = productType;
  }

  public String getProductType() {
    return productType.getName();
  }

  public void setPricePerCase(Money pricePerCase) {
    this.pricePerCase = pricePerCase;
  }

  public Money getPricePerCase() {
    return pricePerCase;
  }
}

As you can see in Listing 5, once the discount is set in a WholeSaleOrder instance, the discounted price and savings can be obtained by calling the getCalculatedPrice and getDiscountedDifference methods, respectively.

Better test those methods (with JUnit)!

With the Money and WholesaleOrder classes defined, you'll want to write a JUnit test to verify the functionality of the getCalculatedPrice and getDiscountedDifference methods. The test is shown in Listing 6:

Listing 6. JUnit's WholesaleOrderTest class

package org.acme.store.discount.engine.junit;

import junit.framework.TestCase;
import org.acme.store.Money;
import org.acme.store.discount.engine.WholesaleOrder;

public class WholesaleOrderTest extends TestCase {

  /*
   * Test method for 'WholesaleOrder.getCalculatedPrice()'
   */
  public void testGetCalculatedPrice() {
    WholesaleOrder order = new WholesaleOrder();
    order.setDiscount(0.05);
    order.setNumberOfCases(10);
    order.setPricePerCase(new Money(10.00));

    assertEquals("$95.00", order.getCalculatedPrice().toString());
  }

  /*
   * Test method for 'WholesaleOrder.getDiscountedDifference()'
   */
  public void testGetDiscountedDifference() {
    WholesaleOrder order = new WholesaleOrder();
    order.setDiscount(0.05);
    order.setNumberOfCases(10);
    order.setPricePerCase(new Money(10.00));

    assertEquals("$5.00", order.getDiscountedDifference().toString());
  }
}

The PricingEngine class

The PricingEngine class utilizes a business rules engine, which in this case is Drools (see "About Drools"). PricingEngine is extremely simple with just one public method, applyDiscount. Simply pass in a WholeSaleOrder instance and the engine asks Drools to apply the discount, as shown in Listing 7:

Listing 7. The PricingEngine class

package org.acme.store.discount.engine;

import org.drools.RuleBase;
import org.drools.WorkingMemory;
import org.drools.io.RuleBaseLoader;

public class PricingEngine {

  private static final String RULES="BusinessRules.drl";
  private static RuleBase businessRules;

  private static void loadRules() throws Exception{
    if (businessRules==null){
      businessRules = RuleBaseLoader.loadFromUrl(PricingEngine.class.getResource(RULES));
    }
  }

  public static void applyDiscount(WholesaleOrder order) throws Exception{
    loadRules();
    WorkingMemory workingMemory = businessRules.newWorkingMemory();
    workingMemory.assertObject(order);
    workingMemory.fireAllRules();
  }
}

Tip: About Drools Drools is a rules engine implementation tailored for the Java™ language. It offers a pluggable language implementation, and currently, rules can be written in Java, Python, and Groovy. For more information, or to download Drools, see the Drools homepage.

Rules by Drools

You'll have to define the business rules for calculating discounts in an XML file that is specific to Drools. For example, the snippet in Listing 8 is a rule that applies a 5% discount to an order if the number of cases is greater than 9 and less than 50 and it isn't a seasonal product.

You'll have to define the business rules for calculating discounts in an XML file that is specific to Drools. For example, the snippet in Listing 8 is a rule that applies a 5% discount to an order if the number of cases is greater than 9 and less than 50 and it isn't a seasonal product.

Listing 8. A sample rule from the BusinessRules.drl file

<rule-set name="BusinessRulesSample"
          xmlns="http://drools.org/rules"
          xmlns:java="http://drools.org/semantics/java"
          xmlns:xs="http://www.w3.org/2001/XMLSchema-instance"
          xs:schemaLocation="http://drools.org/rules rules.xsd
                             http://drools.org/semantics/java java.xsd">
  <rule name="1st Tier Discount">
    <parameter identifier="order">
      <class>WholesaleOrder</class>
    </parameter>

    <java:condition>order.getNumberOfCases() > 9 </java:condition>
    <java:condition>order.getNumberOfCases() < 50 </java:condition>
    <java:condition>order.getProductType() == "year-round"</java:condition>

    <java:consequence>
      order.setDiscount(0.05);
    </java:consequence>
  </rule>
</rule-set>

Tag-team testing

With the PricingEngine in place and the application rules defined, you're probably itching to verify things are working correctly. The question becomes, do you use JUnit or FIT? Why not both? Testing all the combinations through JUnit is possible, but it'll take a lot of code. Better to test a few values with JUnit to quickly verify things are working, and then rely on the power of FIT to run the desired combinations. See what happens when I give it a try, starting with Listing 9:

Listing 9. JUnit quickly verifies that things are working

package org.acme.store.discount.engine.junit;

import junit.framework.TestCase;
import org.acme.store.Money;
import org.acme.store.discount.engine.PricingEngine;
import org.acme.store.discount.engine.ProductType;
import org.acme.store.discount.engine.WholesaleOrder;

public class DiscountEngineTest extends TestCase {

  public void testCalculateDiscount() throws Exception{
    WholesaleOrder order = new WholesaleOrder();
    order.setNumberOfCases(20);
    order.setPricePerCase(new Money(10.00));
    order.setProductType(ProductType.YEAR_ROUND);

    PricingEngine.applyDiscount(order);

    assertEquals(0.05, order.getDiscount(), 0.0);
  }

  public void testCalculateDiscountNone() throws Exception{
    WholesaleOrder order = new WholesaleOrder();
    order.setNumberOfCases(20);
    order.setPricePerCase(new Money(10.00));
    order.setProductType(ProductType.SEASONAL);

    PricingEngine.applyDiscount(order);

    assertEquals(0.0, order.getDiscount(), 0.0);
  }
}

Doesn't FIT? Make it FIT!

There are eight rows of data values in the FIT table in Figure 5. You might have managed to code the first two with JUnit in Listing 7, but do you really want to write the rest? It will take a lot of patience to write the entire eight or to add the new ones when the client adds new rules. Good thing there's an easier way. Nope, it's not ignoring the tests -- it's FIT!

FIT is beautiful for testing business rules or anything involving combinatorial values. What's even better is that someone else has done the work of defining those combinations in a table. Before you can create a FIT fixture for a table, though, you'll need to add a special method to the Money class. Because you need to represent currency values in the FIT table (i.e., values like $100.00), you need a way for FIT to recognize instances of Money. You can do this in a two-step process. First, you must add a static parse method to the custom data type, as shown in Listing 10:

Listing 10. Adding a parse method to the Money class

public static Money parse(String value){
  return new Money(Double.parseDouble(StringUtils.remove(value, '$')));
}

The parse method in the Money class takes a String value (i.e., what FIT pulls out of a table) and returns a properly configured instance of Money. In this case, the $ character is removed and the remaining String is converted into a double, which matches a constructor already found in Money.

Don't forget to add some test cases to the MoneyTest class to verify the newly added parse method works as expected. Two new tests are shown in Listing 11:

Listing 11. Testing the parse method in the Money class

public void testParse() throws Exception{
  Money money = Money.parse("$10.00");
  assertEquals("$10.00", money.toString());
}

public void testEquals() throws Exception{
  Money money = Money.parse("$10.00");
  Money control = new Money(10.00);
  assertEquals(control, money);
}

Writing a FIT fixture

Now you're ready to code your first FIT fixture. Your instance members and methods are already sorted out in Tables 1 and 2, so you just need to wire things together and add one more method to handle the custom type: Money. For handling specific types in your fixture, you'll also need to add another parse method. This one has a slightly different signature from the last one though: the method is an instance method that you are overriding from the Fixture class, which is the parent of ColumnFixture.

Note in Listing 12 how the parse method in DiscountStructureFIT does a compare on the class type. If there is a match, the custom parse method from Money is invoked; otherwise, the super class's (Fixture) version of parse is invoked.

The rest of the code in Listing 12 is plug and play! For each data row in the FIT table shown in Figure 5, values are set, methods are invoked, and FIT verifies the results! For example, on the first run of the FIT test, DiscountStructureFIT's listPricePerCase is set to $10.00, numberOfCases is set to 10, and isSeasonal to true. Then DiscountStructureFIT's discountPrice is executed and the returned value is compared to $100.00 followed by the execution of discountAmount. Its return value is compared to $0.00.

Listing 12. Discount testing with FIT

package org.acme.store.discount;

import org.acme.store.Money;
import org.acme.store.discount.engine.PricingEngine;
import org.acme.store.discount.engine.ProductType;
import org.acme.store.discount.engine.WholesaleOrder;
import fit.ColumnFixture;

public class DiscountStructureFIT extends ColumnFixture {

  public Money listPricePerCase;
  public int numberOfCases;
  public boolean isSeasonal;

  public Money discountPrice() throws Exception {
    WholesaleOrder order = this.doOrderCalculation();
    return order.getCalculatedPrice();
  }

  public Money discountAmount() throws Exception {
    WholesaleOrder order = this.doOrderCalculation();
    return order.getDiscountedDifference();
  }

 /**
  * required by FIT for specific types
  */
  public Object parse(String value, Class type) throws Exception {
    if (type == Money.class) {
      return Money.parse(value);
    } else {
      return super.parse(value, type);
    }
  }

  private WholesaleOrder doOrderCalculation() throws Exception {
    WholesaleOrder order = new WholesaleOrder();
    order.setNumberOfCases(numberOfCases);
    order.setPricePerCase(listPricePerCase);

    if (isSeasonal) {
      order.setProductType(ProductType.SEASONAL);
    } else {
      order.setProductType(ProductType.YEAR_ROUND);
    }

    PricingEngine.applyDiscount(order);
    return order;
  }
}

Now, compare the JUnit test cases from Listing 9 to the code in Listing 12. Isn't Listing 12 more efficient? You certainly could code all the required tests with JUnit, but FIT makes your job so much easier! If you're satisfied (which you should be!), you can run a build and call the FIT runner to produce the lovely results shown in Figure 6:

Figure 6. These results sure are FIT!

In conclusion

FIT shines by helping organizations avoid the miscommunications, misunderstandings, and misinterpretations that often occur between business clients and developers. Bringing the people who write requirements into the testing process early is an obvious way to catch problems and fix them before they become the stuff of development nightmares. What's more, FIT is completely compatible with already entrenched technologies like JUnit. In fact, as I've shown here, JUnit and FIT complement each other beautifully. Make this a stellar year in your pursuit of code quality -- by resolving to get FIT!

Resources

Learn

• "In pursuit of code quality: Don't be fooled by the coverage report" (Andrew Glover, developerWorks, January 2006): Find out how test coverage measurements can lead you astray.
• "An early look at JUnit 4" (Elliotte Harold, developerWorks, September 2005): A guided tour of upcoming changes to this invaluable testing framework.
• "Continuously ensuring quality: A case study" (The Rational Edge, Laura Rose, December 2004): Learn about the tools and techniques IBM Rational uses to develop its own software products.
• "Ending requirements chaos" (The Rational Edge, Susan August, August 2002): Summarizes the challenges of writing and implementing application requirements.
• The Java technology zone: Hundreds of articles about every aspect of Java programming.

Get products and technologies

• Get FIT: Download the Framework for Integrated Tests and try it out for yourself.
• Get the FIT Maven plug-in: Download the plug-in and start automating your FIT tests in Maven.
• JUnit homepage: Download JUnit and get the latest news on JUnit 4.
• Drools homepage: Download the Drools rules engine used in this article.

About the author

Andrew Glover is president of Stelligent Incorporated, a JNetDirect company. Stelligent Incorporated helps companies address software quality with effective developer testing strategies and continuous integration techniques that enable teams to monitor code quality early and often. He is the coauthor of Java Testing Patterns (Wiley, September 2004).