Click here to monitor SSC
  • Av rating:
  • Total votes: 5
  • Total comments: 0
Michael Sorens

A TDD Journey: 4-Tests as Documentation; False Positive Results; Component Isolation

01 August 2014

In Test-Driven Development (TDD) , The writing of a unit test is done more to design and to document  than to verifiy. By writing a unit test you close a  number of feedback loops, and verifying the functionality of the code is just a minor one. everything you need to know about your class under test is embodied in a simple list of the names of the tests.  Michael Sorens continues his introduction to TDD that is more of a journey in six parts, by discussing Tests as Documentation, False Positive Results and Component Isolation.

 

Part 1: Trials and Tribulations of TDD

 

Part 2: Naming Tests; Mocking Frameworks; Dependency Injection

 

Part 3: Mocks vs. Stubs; Test Frameworks; Assertions; ReSharper Accelerators

Part 4: Tests as Documentation; False Positive Results; Component Isolation

 

Part 5: Tests vs. Code; Refactor Friendliness; Test Parameterization

 

Part 6: Mini-Factory Pattern; Don’t Care Terms

This is part 4 of our exploration into practicing hands-on TDD. Unlike most of my multi-part series, it is not advisable to join this one in the middle. So if you are arriving fresh, please go back and review part 1 for an overview of TDD and subsequent parts that both built out our tests & code and introduced ancillary concepts and techniques crucial to the TDD approach.

Here is a condensed version of the code so far:

namespace TddDemoTests

{

    public class WidgetActivatorTest

    {

        public void WidgetActivator_constructor_accepts_an_IWidgetLoader()

        public void Execute_delegates_to_IWidgetLoader_to_load_widget_details()

        public void WidgetActivator_constructor_accepts_an_IWidgetPublisher()

        public void Execute_delegates_to_IWidgetPublisher_to_publish_widget()

        public void Execute_returns_false_if_no_details_to_load()

    }

    public interface IWidgetLoader

    {

        void Load()

    }

    public interface IWidgetPublisher

    {

        void Publish()

    }

 

    public class WidgetActivator

    {

     public WidgetActivator(IWidgetLoader widgetLoader, IWidgetPublisher widgetPublisher)

     public bool Execute()

    }

}

Tests as Documentation

This code has been distilled to show just what you need to understand the system as it now stands and no more. That is, if you had never seen the code before, everything you need to know is there; the missing chunks of code would only slow down your process of understanding. Consider:

Premise (from reading the tests)

Conclusion (confirmed  by reading the code)

The test class is called WidgetActivatorTest.

Hence there should be a WidgetActivator class.

WidgetActivator needs both an WidgetLoader and an IWidgetPublisher

There is an WidgetLoader interface and an IWidgetPublisher interface, and these are both injected dependencies to the WidgetActivator constructor.

We are exercising an Execute method to do loading and publishing.

The WidgetActivator has an Execute method which invokes some WidgetLoader method to load and some IWidgetPublisher method to publish.

The final test says Execute needs to return a status.

The Execute method signature shows it returns a Boolean.

A careful reading of the above will reveal that the code has only and exactly what the tests demand. The only reason WidgetActivator.Execute returns a Boolean is because a test needs it to. Contrast that with WidgetLoader.Load and IWidgetPublisher.Publish, which both return void. Later, there will be tests that require these methods return something so, at that time, they will adapt to do so.

Now watch closely as the quality of your ignorance takes a leap right here (see part 1 if you think this is an insult :-)! In the table above, discard the right-hand column: you can immediately infer all of that from the left-hand column. But that means that, correspondingly, you do not have to look at the code, only the tests! Therefore, the code I showed above is too verbose; in actuality, all you need to understand the system as it now stands is the list of test names:

WidgetActivatorTest:

        WidgetActivator_constructor_accepts_an_IWidgetLoader

        Execute_delegates_to_IWidgetLoader_to_load_widget_details

        WidgetActivator_constructor_accepts_an_IWidgetPublisher

        Execute_delegates_to_IWidgetPublisher_to_publish_widget

        Execute_returns_false_if_no_details_to_load

From the test names you can generate the premises in the table above, and from the premises you can derive the conclusions. QED.

Recall in the last test (Execute_returns_false_if_no_details_to_load) that we simply hard-coded a false return value to make that test pass.

TEST: Now we make it more realistic by adding the opposite in a new test: if the loader did its work then Execute should return true.

[Test]

public void Execute_returns_true_if_details_are_loaded()

{

    var mockWidgetLoader = new Mock<IWidgetLoader>();

    mockWidgetLoader

        .Setup(x => x.Load())

        .Returns(() => true);

    var stubWidgetPublisher = Mock.Of<IWidgetPublisher>();

    var activator = new WidgetActivator(mockWidgetLoader.Object, stubWidgetPublisher);

 

    var result = activator.Execute();

 

    Assert.That(result, Is.True);

}

 

This test finally reveals what you need mock objects for. After declaring the mockWidgetLoader above, the Setup/Returns methods together provide some emulated behavior for this test double, to wit: whenever the Load method is called, return true. By passing that mock object in to the WidgetActivator constructor, that mock object will be used inside the Execute method, triggering the Setup/Returns action specified. But before you can actually see that, there is a compilation error to address.

CODE: The above test fails to compile because the Load method does not return anything. Let’s change it to return a Boolean to get it to compile.

public interface IWidgetLoader

{

    bool Load();

}

CODE: Now it compiles but the test reports failure in the Assert, because we had hard-coded the Execute method to return false. Revise the code to pass the return value of the Load method as the return value of Execute.

public  class WidgetActivator

  {

      private readonly IWidgetLoader _widgetLoader;

      private readonly IWidgetPublisher _widgetPublisher;

 

      public WidgetActivator(IWidgetLoader widgetLoader, IWidgetPublisher widgetPublisher)

      {

          _widgetLoader = widgetLoader;

          _widgetPublisher = widgetPublisher;

      }

 

      public bool Execute()

      {

          var result = _widgetLoader.Load();

          _widgetPublisher.Publish();

          return result;

      }

  }

Because we “pre-loaded” the Load method to return true, Execute will now return true in this test case, so the test passes. Hooray! But this change should have repercussions in the previous test which, as you recall, was testing just the opposite: if Load returns false then Execute should return false. If you rerun all the tests you will see that the test still passes—even though we did not prepare the mocked Load method to take any specific action! Here is the prior test (from part 3)—notice just two stubs, no method actions.

public void Execute_returns_false_if_no_details_to_load()

{

    var stubWidgetLoader = Mock.Of<IWidgetLoader>();

    var stubWidgetPublisher = Mock.Of<IWidgetPublisher>();

    var activator = new WidgetActivator(stubWidgetLoader, stubWidgetPublisher);

 

    var result = activator.Execute();

 

    Assert.That(result, Is.False);

}

So why does that test still pass? Moq is smart enough so that, in the absence of specific instructions, it will return false for a Boolean method. Thus when Load is called, it does in fact return false, and therefore Execute returns false, and the test passes. Neat!

TEST: Analogous to how the loader works, the publisher needs to return a status and the result of the Execute method should reflect the results of the publisher as well as the loader. Add a test to reflect the status of the publisher.

[Test]

public void Execute_returns_false_if_publishing_failed()

{

    var mockWidgetPublisher = new Mock<IWidgetPublisher>();

    mockWidgetPublisher

        .Setup(x => x.Publish())

        .Returns(() => false);

    var stubWidgetLoader = Mock.Of<IWidgetLoader>();

    var activator = new WidgetActivator(stubWidgetLoader, mockWidgetPublisher.Object);

 

    var result = activator.Execute();

 

    Assert.That(result, Is.False);

}

CODE: That fails to compile because the Publish method does not return anything, just like we saw with the previous test. Let’s change it to return a Boolean to get it to compile.

 

public  interface IWidgetPublisher

{

    bool Publish();

}

Beware Passing Tests

Now the test passes so the code must be doing just what it needs to satisfy this test. A reasonable statement… but completely wrong! We have not even hooked up the output of this Publish method yet! So why does it pass?  Here is the body of the Execute method again, line by line:

Statement

Interpretation

var result = _widgetLoader.Load();

Set the result to be the output of Load. Because we have not directed moq to take any particular action for this method, it returns false as you just learned.

_widgetPublisher.Publish();

Now we call Publish… and completely ignore its output; thus completely ignoring the forced return value of false we directed moq to provide in the test.

return result;

So merely by coincidence, the value we return happens to reflect the return value of the Publish method.

Coincidence like this can happen more often than you might think or expect (i.e. getting a test result that happens to agree with your expectation due to incorrect code). Or to state that another way:

Watch out for tests that pass for the wrong reason!

When you have a failing test, you know a priori the code is not (yet) correct. But when you have a passing test, you must confirm that it is passing for the reason you expect. As you continue to add tests, you may expose the flaw in an earlier test that falsely passes, but this is not always the case. The onus is on you to understand your test and your code well enough to be able to say that, yes, the test is passing because what the test is expecting is being delivered for the reasons specified in the test.

So going back to our test at hand, we want to force the mock publisher’s Publish method to return false and we want that return value to be reflected in the return value of the Execute method. As you just saw above, we are ignoring the output of the Publish method, so the test is returning a false positive result.

CODE: Let’s now actually use the output of the Publish method and see if it still works.

public bool Execute()

{

    var result = _widgetLoader.Load();

    result &= _widgetPublisher.Publish();

    return result;

}

Here we have adjusted the Execute method’s return value to be a conjunction, reflecting both the result of the Load and the Publish. The test still passes but now it passes for the right reason: we set up the mock object so that Publish will return false. The conjunction of that value with the Load result is still false, so Execute returns false, just as we wanted. So that test is complete. Unfortunately, we have an issue elsewhere…

Isolating Components

Rerunning the whole test suite reveals that the previous test now fails. Why? Because it was assuming the return value of Execute solely depended on the WidgetLoader.Load method and we just invalidated that assumption by making it also depend on IWidgetPublisher .Publish. Therefore, when you have more than one component that can affect the result of your class under test, you must isolate the component you wish to test. And that is the true purpose of mock objects.

In this case it is straightforward. Execute is supposed to reflect the return values of both IWidgetLoader.Load and IWidgetPublisher .Publish. If both are successful, Execute should return true but if either or both fails, Execute should reflect that failure. Thus Execute depends on a conjunction of the two results. To isolate the result of a conjunction to depend exclusively on one of those items, just force the other item to be true. Consider the truth table for a conjunction of our sub-results:

#

Publish

Load

Load ^ Publish

1

true

true

true

2

true

false

false

3

false

true

false

4

false

false

false

First, examine rows 3 and 4 where Publish returns false. As you can see, whatever the result of Load is, the conjunction returns false. But now look at rows 1 and 2 which says that if Publish always returns true, then the result of the conjunction exactly matches the result of Load, just what we need. And we can force Publish to always return true by specifying that in the mock.

TEST: To isolate the Load method we just need to force Publish to always return true. Here we are using Linq-to-Mocks syntax to always return true whenever Publish is called.

[Test]

public void Execute_returns_true_if_details_are_loaded()

{

    var mockWidgetLoader = new Mock<IWidgetLoader>();

    mockWidgetLoader

        .Setup(x => x.Load())

        .Returns(() => true);

    var stubWidgetPublisher = Mock.Of<IWidgetPublisher>(

        x => x.Publish() == true

        );

    var activator = new WidgetActivator(mockWidgetLoader.Object, stubWidgetPublisher);

 

    var result = activator.Execute();

 

    Assert.That(result, Is.True);

}

Refer back to part 2 where we introduced Linq-to-Mocks syntax: here we are directing the stubWidgetPublisher to return true whenever the Publish method is invoked, isolating the result of Execute to depend solely on the outcome of the Load method.

That test is concerned with seeing what happens when the Load method returns true. Similarly, we want to review the prior test that examined what happens when Load returns false. Close examination reveals this test passed coincidentally. That is, now that we have hooked up the Publish method into the results, we are now getting a false result because the Publish method returns false (the default return result) regardless of what the Load method returns. So again, we isolate the Load method by forcing the IWidgetPublisher stub to return true.

public void Execute_returns_false_if_no_details_to_load()

{

    var stubWidgetLoader = Mock.Of<IWidgetLoader>();

    var stubWidgetPublisher = Mock.Of<IWidgetPublisher>(

        x => x.Publish() == true

        );

    var activator = new WidgetActivator(stubWidgetLoader, stubWidgetPublisher);

 

    var result = activator.Execute();

 

    Assert.That(result, Is.False);

}

By now you should be starting to have a grasp on the workflow of TDD: write some test code then write some production code to make the test pass. That is necessary but not sufficient—you must also make sure the test and the code together are doing what you think! In the next installment you will add yet more tests and pickup more practical tips to help your TDD journey.

Michael Sorens

Author profile:

Michael Sorens is passionate about software to be more productive, evidenced by his open source libraries in several languages (see his API bookshelf) as well as SqlDiffFramework (a DB comparison tool for heterogeneous systems including SQL Server, Oracle, and MySql). With degrees in computer science and engineering he has worked the gamut of companies from Fortune 500 firms to Silicon Valley startups over the last 25 years or so. Current passions include PowerShell, .NET, SQL, and XML technologies (see his full brand page). Spreading the seeds of good design wherever possible, he enjoys sharing knowledge via writing (see his full list of articles), teaching, and StackOverflow. Like what you have read? Connect with Michael on LinkedIn and Google +

Search for other articles by Michael Sorens

Rate this article:   Avg rating: from a total of 5 votes.


Poor

OK

Good

Great

Must read
Have Your Say
Do you have an opinion on this article? Then add your comment below:
You must be logged in to post to this forum

Click here to log in.
 

Top Rated

Building a Customised ALM Platform with TFS and VSO
 The latest versions of Team Foundation Server are not only sophisticated, but extensible. Continue... Read more...

ASP.NET SignalR: Old-fashioned Polling, Just Done Better
 A website often needs to update a page as the underlying data changes. You can, of course, just poll... Read more...

Rethinking the Practicalities of Recursion
 We all love recursion right up to the point of actually using it in production code. Why? Recursion... Read more...

Acceptance Testing with FitNesse: Multiplicities and Comparisons
 FitNesse is one of the most popular tools for unit testing since it is designed with a Wiki-style... Read more...

Acceptance Testing with FitNesse: Documentation and Infrastructure
 FitNesse is a popular general-purpose wiki-based framework for writing acceptance tests for software... Read more...

Most Viewed

A Complete URL Rewriting Solution for ASP.NET 2.0
 Ever wondered whether it's possible to create neater URLS, free of bulky Query String parameters?... Read more...

Visual Studio Setup - projects and custom actions
 This article describes the kinds of custom actions that can be used in your Visual Studio setup project. Read more...

.NET Application Architecture: the Data Access Layer
 Find out how to design a robust data access layer for your .NET applications. Read more...

Calling Cross Domain Web Services in AJAX
 The latest craze for mashups involves making cross-domain calls to Web Services from APIs made publicly... Read more...

10 Reasons Why Visual Basic is Better Than C#
 After having converted a whole lot of training materials based on VB.NET into C#, Andy ‘Wise Owl’ Brown... Read more...

Why Join

Over 400,000 Microsoft professionals subscribe to the Simple-Talk technical journal. Join today, it's fast, simple, free and secure.