Click here to monitor SSC
  • Av rating:
  • Total votes: 5
  • Total comments: 0
Michael Sorens

Acceptance Testing with FitNesse: Documentation and Infrastructure

23 August 2013

FitNesse is a popular general-purpose wiki-based framework for writing acceptance tests for software systems, including databases. It is intended to be easy for the tester to learn and use. In this article, Michael Sorens gives a 'view from the trenches' of Fitnesse's documentation and infrastructure.

Contents

    1. Documentation
    2. Infrastructure
      1. Differentiating Java from .NET
      2. Include .NET-Equivalent References
      3. Include .NET-Equivalent Using Statements
      4. Let FitNesse Dynamically Index Your Sub-Pages
      5. Allow Concurrent Test Access
      6. Avoid Duplicating Setup or Teardown Code
      7. Avoid Duplicating Any Other Code, Too
      8. Moving, Renaming, or Deleting Tests
    3. More to Come

FitNesse is a wiki-based framework for writing acceptance tests for software systems. If you are not familiar with FitNesse,  then you’ll find Part 1 of this series, 'Acceptance Testing With FitNesse, The Overview', useful  because it walks through a complete .NET example from writing the test in your browser to writing the C# code-behind. Although FitNesse provides a rather nifty and user-friendly way to write acceptance tests in general there are, in practice, plenty of quirks and glitches to watch out for. This article, and the subsequent parts of this series, provides “tips from the trenches”, by which I mean  an accumulation of tips collected from intensive daily  use of FitNesse  to alleviate or avoid many of those pain-points.

 

Part 1: FitNesse Introduction  and Walkthrough

Part 2: Documentation and Infrastructure

 

Part 3: Naming and Layout

 

Part 4: Debugging, Control Flow, and Tracing

 

Part 5: Symbols, Variables, and Code-Behind Style

 

Part 6: Multiplicities and Comparisons

 

Part 7: Database Fixtures, Project Overview

This article covers various issues with the documentation and infrastructure; subsequent parts will cover tips about such things as  naming, debugging, control flow, layout, variables, comparisons, tracing and database.

Documentation

FitNesse was originally designed for testing Java code, but it has since evolved to handle .NET code with the separate fitSharp  component  and does so pretty well. Database support came  later with the DbFit component. Those two components, together with the base FitNesse component, all provide their own documentation with different styles, organization and comprehensiveness. As so  often with software documentation, it can be a challenge to find the information  you are looking for. Here then, is a collection of the most useful links within the documentation of FitNesse, fitSharp, and DbFit:

The first part of this series was just intended  to help get you up to speed if you had never been exposed to FitNesse, so it is more tutorial than reference. This and subsequent parts, however, lean more towards reference than tutorial. The main FitNesse site provides some additional tutorial material—notably their One-Minute Description and Two-Minute Example, among other good learning material. Use the User Guide link above to see those.

Infrastructure

Differentiating Java from .NET

By default, FitNesse works for the Java environment. To work in a .NET environment you must download the fitSharp DLL and tell FitNesse to use that instead. Typically you do this on your root page—accessible from a link at the bottom of every FitNesse page (the address is http://localhost:port/root). Use the !define directive as follows:

!define COMMAND_PATTERN {%m -r fitnesse.fitserver.FitServer,fitsharp\fit.dll %p}

!define TEST_RUNNER {fitsharp\Runner.exe}

!define PATH_SEPARATOR {;}

variable defined: COMMAND_PATTERN=%m -r fitnesse.fitserver.FitServer,fitsharp\fit.dll %p
variable defined: TEST_RUNNER=fitsharp\Runner.exe
variable defined: PATH_SEPARATOR=;

References: Defining Common Actions, Customizing Test Execution, Using FitNesse

Include .NET-Equivalent References

You must include references in FitNessse with the !path directive to access the namespaces and classes in a DLL.This is In much the same way as a Visual Studio project. You do not need to include either  FitNesse, as this is handled by the JAR file you launched, or fitSharp, handled by the TEST_RUNNER definition described  in the previous section. You would, however, include DbFit if you are using database fixtures as well as your custom DLLs:

!path fitsharp\dbfit.dll

!path fitsharp\dbfit.sqlserver.dll

!path Fixtures\CleanCodeFixtures.dll

classpath: fitsharp\dbfit.dll

classpath: fitsharp\dbfit.sqlserver.dll

classpath: Fixtures\CleanCodeFixtures.dll

Just as with the COMMAND_PATTERN and TEST_RUNNER in the previous section, you typically specify these on your root page, which is accessible from a link at the bottom of every FitNesse page (the address is http://localhost:port/root). You may optionally specify these from a suite configuration file instead. If you do, you do not need the %p in the COMMAND_PATTERN.

Reference: Defining Common Actions, Using FitNesse,  Suite Configuration File

Include .NET-Equivalent Using Statements

In much the same way that you optionally import namespaces with using statements in C# to obviate the need to use fully-qualified paths when referring to classes, you may do the same in FitNessse with the !import directive. Typically these go in a SuiteSetUp  or SetUp page so they are inherited as needed, but you may use them directly on individual test pages as well.

A FitNesse import is analogous to a .NET using statement: you can then write unqualified class names in your code, making it more readable. At the left side of the table below, you see the use of an !import directive allowing the Echo fixture to be referenced without a fully-qualified namespace. At right, the namespace is required when the !import directive is not used.

 

With  !import

Without  !import

!|import                 |

|CleanCodeFixtures.Common|

 

!|Echo        |

|Value|Result?|

|25   |       |

|-99  |       |

!|CleanCodeFixtures.Common.Echo|

|Value         |Result?        |

|25            |               |

|-99           |               |

import

CleanCodeFixtures.Common

 

Echo

Value

Result?

25

 

-99

 

CleanCodeFixtures.Common.Echo

Value

Result?

25

 

-99

 

Reference: Import Fixture

Let FitNesse Dynamically Index Your Sub-Pages

You can explicitly list a set of links to subpages, as is done on the FitNesse top-level page:

[[A One-Minute Description][FitNesse.UserGuide.OneMinuteDescription]]

[[A Two-Minute Example][FitNesse.UserGuide.TwoMinuteExample]]

[[User Guide][FitNesse.UserGuide]]

[[Acceptance Tests][FitNesse.SuiteAcceptanceTests]]

[[Release Notes][FitNesse.ReleaseNotes]]

A One-Minute Description

A Two-Minute Example

User Guide

Acceptance Tests

Release Notes

That is fine if your list of links is hand-picked,  but in most cases you will be setting up suites of suites of suites. You want each suite page to contain links to all of its children and, should you add or delete children, you want the page to update automatically. To do this, rather than enumerate your pages or suites explicitly, just use the !contents directive. The –R flag indicates to recursively show all child pages, as shown in the example. Omit the –R flag to show only immediate children. Limit the recursion to a set depth by adding a numeric argument, e.g. –R2.

!contents –R

Sub Suite A

  o Page A

  o Page B

Sub Suite B

  o Page A

  o Page B

  o Page C

By default, creating a new page gives you that flag and more:  !contents -R2 -g -p -f –h

See the reference for details on all the other flags.

References: MarkupContents

Allow Concurrent Test Access

Where a test may access a common resource such as a database, you can allow several people to run the same test simultaneously by generating unique values per session. This assumes that each user runs their own FitNesse server, so each will thereby have a unique session.

First, setup a command file to launch FitNesse (e.g. LaunchFitNesse.cmd) defining an environment variable containing a random seed just before you launch the FitNesse server:

set SEED=%RANDOM%

java -jar fitnesse.jar -p port

Windows environment variables such as SEED are accessible within FitNesse tests just like variables you explicitly define. You can thus use this session-specific seed when creating one or more values to insert in your database table:

!define TestClient (MyTestClient_${SEED})

 

!|Echo                |

|Value        |Result?|

|${TestClient}|       |

variable defined: TestClient=MyTestClient_${SEED}

Echo

Value

Result?

MyTestClient_22861

 

From the variable definition,  it does not look like the value of SEED has been substituted within TestClient, but when it is used in the Echo table you can see it has.

Avoid Duplicating Setup or Teardown Code

Use appropriate setup and teardown pages. These will function exactly as do setup and cleanup pages in unit test frameworks. The table shows the corresponding elements between the two major .NET test frameworks and FitNesse:

 

Setup test

Cleanup test

Setup suite

Cleanup suite

NUnit

[SetUp]

[TearDown]

[TestFixtureSetUp]

[TestFixtureTearDown]

MSTest

[TestInitialize]

[TestCleanUp]

[ClassInitialize]

[ClassCleanup]

FitNesse

SetUp

TearDown

SuiteSetUp

SuiteTearDown

 Pages that you name SetUp or TearDown are executed automatically at the start and end of every test. Pages that you name SuiteSetUp or SuiteTearDown are executed automatically at the start and end of a suite. Note that a suite may be explicit—a page containing one or more child tests—but it may also be implicit: executing just a single test constitutes a suite, too.

These special pages are inherited in child pages, so often you just need to  define  these once at the top-level. However, you can override inheritance by redefining one further down your tree. This is because FitNesse pages are always arranged hierarchically in a standard tree structure. The root may typically contains suites, which may contain suites of their own, etc. Eventually, you will have leaf nodes—individual tests.

You can also inherit and override at the same time. Assume you have created a SetUp page at your root. Somewhere lower in your tree you also create a SetUp page. Its presence cancels inheritance. But if your goal is to add to the parent SetUp rather than replace it, simply start your lower SetUp page with an !include directive. My root page is CleanCode (in a browser, this is  http://localhost:port/CleanCode) so referencing my root SetUp page is done like this:

!include -setup .CleanCode.SetUp

Included page: .CleanCode.Setup (edit)

Reference: Special Pages, Defining Common Actions

Avoid Duplicating Any Other Code, Too

You are not restricted from using the !include directive from the last section just in SetUp or TearDown pages. Bring in other needed code fragments with the !include directive whenever you need:

!include path

A path may be relative or absolute. A path address is formed just like a file system path but you must use a period between component names instead of virgules or backslashes. A leading period makes a path absolute. If your path is incorrect, FitNesse will tell you immediately upon saving the page; you do not have to wait until you execute the test.

By default, the included block of text is expanded—you see the contents of the included file inline. Compare that to the example in the previous section; the –setup option collapsed the block so you only saw the file name. The !include directive provides several options:

  • Collapse it with the –c flag.
  • Expand it and make it look just as if it was part of the page with the –seamless flag.
  • Delegate control over whether it is expanded or collapsed to the global variable COLLAPSE_SETUP or COLLAPSE_TEARDOWN with the –setup or –teardown flags, respectively.

Reference: Includes & Informational, Global Variables

Moving, Renaming, or Deleting Tests

For your conventional code—e.g. C# code that you work with in Visual Studio—there are source control plugins available (notably AnkhSVN or VisualSVN) so that, when you move or rename a file in Visual Studio, it actually mirrors the operation correctly in source control, maintaining the history of the item in question. In FitNesse, you can similarly move or rename test pages, but there is no plugin to mirror this in your source control back-end! Thus, a moved item will appear as unversioned and its former named item will appear as missing when you review your changes to commit.  Because of this, it is easier to do file reorganizations outside of FitNesse, just using Windows Explorer or equivalent. Assuming you have followed the guideline above to automatically generate your suite pages with the !contents directive, the only thing you need do is restart your FitNesse server.  If you don’t restart,  you will pick up the items in the new locations or new names, but you will also still see the old locations/names.

Of course, the above applies to files that you have already committed to source control. If you are developing new test pages that are not yet committed, you can freely move, rename, or delete as needed within the confines of FitNesse.

More to Come…

There are many more aspects to review and issues to alleviate—stay tuned for part three!
Michael Sorens

Author profile:

Michael Sorens is passionate about software to be more productive, evidenced by his open source libraries in several languages (see his API bookshelf) as well as SqlDiffFramework (a DB comparison tool for heterogeneous systems including SQL Server, Oracle, and MySql). With degrees in computer science and engineering he has worked the gamut of companies from Fortune 500 firms to Silicon Valley startups over the last 25 years or so. Current passions include PowerShell, .NET, SQL, and XML technologies (see his full brand page). Spreading the seeds of good design wherever possible, he enjoys sharing knowledge via writing (see his full list of articles), teaching, and StackOverflow. Like what you have read? Connect with Michael on LinkedIn and Google +

Search for other articles by Michael Sorens

Rate this article:   Avg rating: from a total of 5 votes.


Poor

OK

Good

Great

Must read
Have Your Say
Do you have an opinion on this article? Then add your comment below:
You must be logged in to post to this forum

Click here to log in.
 

Top Rated

What is DNX?
 In the past, working in .NET for non-Windows platforms has been dependent on third-party frameworks... Read more...

Building a Customised ALM Platform with TFS and VSO
 The latest versions of Team Foundation Server are not only sophisticated, but extensible. Continue... Read more...

Rethinking the Practicalities of Recursion
 We all love recursion right up to the point of actually using it in production code. Why? Recursion... Read more...

Acceptance Testing with FitNesse: Multiplicities and Comparisons
 FitNesse is one of the most popular tools for unit testing since it is designed with a Wiki-style... Read more...

Acceptance Testing with FitNesse: Documentation and Infrastructure
 FitNesse is a popular general-purpose wiki-based framework for writing acceptance tests for software... Read more...

Most Viewed

A Complete URL Rewriting Solution for ASP.NET 2.0
 Ever wondered whether it's possible to create neater URLS, free of bulky Query String parameters?... Read more...

Visual Studio Setup - projects and custom actions
 This article describes the kinds of custom actions that can be used in your Visual Studio setup project. Read more...

.NET Application Architecture: the Data Access Layer
 Find out how to design a robust data access layer for your .NET applications. Read more...

Calling Cross Domain Web Services in AJAX
 The latest craze for mashups involves making cross-domain calls to Web Services from APIs made publicly... Read more...

10 Reasons Why Visual Basic is Better Than C#
 After having converted a whole lot of training materials based on VB.NET into C#, Andy ‘Wise Owl’ Brown... Read more...

Why Join

Over 400,000 Microsoft professionals subscribe to the Simple-Talk technical journal. Join today, it's fast, simple, free and secure.