Automate GUI Testing With TestNG-Abbot

From NeoWiki

Easily validate GUI components using fixture objects

Andrew Glover, President, Stelligent Incorporated

27 Feb 2007

TestNG-Abbot is a testing framework that breathes new life into testing GUI components. This month, Andrew Glover walks you through the hardest part of GUI testing with TestNG-Abbot, which is understanding how a user scenario will play out. Once you've got that down, you'll find it surprisingly easy to isolate GUI components and then verify them using the framework's handy fixture objects.

User interfaces built with Swing, AWT, and the like have traditionally presented a challenge to developer testing because of the following factors:

• The complexity of the underlying graphics framework
• The coupling of presentation and business logic within a GUI
• The lack of intuitive automated testing frameworks

Of course, the first two factors are nothing new -- graphical frameworks are complex by nature and adding business functionality to a GUI application has always posed a barrier to testing. On the other hand, a number of handy frameworks have popped up over the last few years that actually facilitate testing of GUIs.

This month, I introduce a neophyte framework that makes great strides toward easing the pain of testing GUIs.

Contents 1 Introducing TestNG-Abbot 1.1 Intuitive fixture classes 2 GUI validation isn't normal! 2.1 The Word Finder GUI 3 Testing with TestNG-Abbot 4 Resources 5 About the author

Introducing TestNG-Abbot

TestNG-Abbot is the result of the marriage of two successful developer testing frameworks: Abbot and TestNG. Abbot is a JUnit extension framework specifically intended to enable programmatic isolation of GUI components, and it provides an easy way to validate GUI behavior. For example, you could use it to obtain a reference to a button component, programmatically click it, and then verify the action that followed. Abbot also comes with a script recorder, which you can use to lay out a test scenario in XML format and run it programmatically.

I've written about TestNG quite a bit in this series, so I'll keep my comments here limited. Basically, TestNG is an alternative to JUnit. In addition to all the expected features, it enables a few extra ones. As I have written elsewhere, TestNG is especially well-suited for higher level testing where it can be used to test with dependencies and rerun only failed tests -- in short, the types of tests that come in handy when testing GUIs. (See Resources for more about TestNG.)

With progenitors this impressive, it's no wonder that TestNG-Abbot is such a whiz kid. Like Abbot, TestNG-Abbot enables programmatic isolation of GUI components. At the same time, it uses TestNG's assertions to abstract the details of GUI manipulation into simple fixtures that expose validation methods. Used properly, TestNG-Abbot's intuitive fixture classes make GUI testing almost as easy as stealing candy from a baby. (Not that you would ever want to do that!)

Intuitive fixture classes

The current release of TestNG-Abbot supports seven fixture types, including ones for manipulating buttons, menus, labels, and text-entry components, like text fields. What's more, these fixture types are logically linked to the code under test (that is, the components of the GUI) by name only. This makes for loose coupling between the GUI and its tests, which is beneficial for at least two reasons:

• The tests do not rely on GUI components being specifically located -- thus they can be moved without breaking your tests.
• Tests can be authored early and withstand layout and aesthetic changes during the development phase.

While currently only seven fixture types are supported, others are coming soon. More fixture types will only increase TestNG-Abbot's effectiveness at programmatically validating GUIs.

GUI validation isn't normal!

While TestNG-Abbot makes the process of validating GUIs easier, that doesn't mean it's easy. You have to approach GUI testing differently from unit or component testing. The process of validating business rules within GUIs becomes a process of validating user scenarios; or, to put it differently, GUI testing involves validating visual state changes.

For instance, if a save button on an order entry GUI is pushed, a business rule could stipulate that the contents of the order are persisted to a database. A user scenario, however, might stipulate that a successful status message be inserted below the button -- and that's the particular test you could write with TestNG-Abbot. In fact, if the GUI has been designed well, you can test that the order contents are persisted to a database without testing the GUI. It follows that you could also write these two particular tests simultaneously, and early.

The Word Finder GUI

So that you can see how TestNG-Abbot works, I've created a simple GUI that does one thing -- it looks up a given word in an underlying dictionary (which is a database) and displays its definition. Regardless of how the application is actually coded, testing a user scenario with this GUI involves three steps:

1. Enter a word in a text box.
2. Click the Find Word button.
3. Verify that the definition is present.

Of course, there are edge cases as well, such as if a user presses the Find Word button without having entered a word, or if the user enters an invalid word. I'll show you how to handle those scenarios with a few additional test cases.

Let's see the GUI ...

Figure 1 illustrates the Word Finder GUI upon startup. Keep in mind that this GUI is simple for a reason: It demonstrates three of TestNG-Abbot's fixture classes and a few test cases to boot!

Figure 1. The Word Finder GUI

When testing with TestNG-Abbot, you should begin by examining a GUI in terms of its parts. The Word Finder GUI is made up of the three components shown in Figure 2:

Figure 2. Components of the Word Finder GUI

As you can see, the Word Finder GUI consists of a JTextField, where the desired word is typed in; a JButton, which causes the GUI to obtain a definition from the dictionary database; and a JEditorPane, where the actual definition is displayed.

In a sunny-day scenario, if I typed in pugnacious and then clicked the Find Word button, the JEditorPane would display "Combative in nature; belligerent," as shown in Figure 3:

Figure 3. Sunny-day scenario -- everything works!

Testing with TestNG-Abbot

To get started with TestNG-Abbot, you need to create a normal test fixture that creates an instance of your GUI using TestNG's BeforeMethod and AfterMethod annotations. The TestNG-Abbot framework comes with a handy AbbotFixture object that facilitates working with GUI components and essentially bootstraps the entire testing process. To employ this object in a test fixture, you need to pass in an instance of your GUI to the fixture object's showWindow() method before a test and then have the fixture clean things up with the aptly named cleanUp() method.

In Listing 1, I've created a TestNG test (that doesn't actually have any tests in it) that uses TestNG-Abbot's AbbotFixture object in a fixture to hold an instance of my Word Finder GUI.

Listing 1. WordFindGUITest is defined using the AbbotFixture object

Resources

About the author

Andrew Glover is president of Stelligent Incorporated, which helps companies address software quality with effective developer testing strategies and continuous integration techniques that enable teams to monitor code quality early and often. Check out Andy's blog for a list of his publications.

"When you have learned to snatch the error code from the trap frame, it will be time for you to leave.", thus spake the master programmer.