8th Light Blog

BOC is a testing pattern that Unclebob briefly mentioned on fitnesse.org shortly after it was published. It describes the typical steps taken in an automated test.

Experienced test writer use BOC whether they know it or not. For new-commers to automated testing, BOC is lesson #1.

If you think about it, BOC is very logical. Every test is testing something; some Operation that the system under test (SUT) performs. That’s where the Operate step comes from.

In almost all cases, you can’t just invoke the operation. The automated test Builds a testable environment first. Maybe data needs to be put in a database, or a service layer needs to be started. Either way, the Build step comes first.

After invoking the Operation, a test needs to Check that it behaved as expected. Clearly, the Check step has to come last.

Example:

Consider a test for the Withdraw feature of an ATM machine.

Build:

• You can’t withdraw money from an account unless there’s money in it. For the build step, add $500.00 to account 123.

Operate:

• Perform a withdraw of $250.00 from account 123.

Check:

• Check that the balance of account 123 is $250.00.

Next time you’re writing an automated test, think about it. More likely than not, you’re using BOC.

The self-shunt method of testing has been conflicting for me. Self-shunt is a good testing pattern to test observers/views. A colleague of mine has started to convince me that self-shunt has some serious drawbacks. I want to explore the different ways to address the same pattern and how they fit into the test process and the test content. Are certain patterns easier to write and certain ones have a better final form?

Lets look at some code from Michael Feathers self shunt paper (see bottom for link). I extended it to do the mocking examples. The user story we are working with is the name of an item must be displayed on the led when the item is scanned.

Writing all of the tests did not require a concrete window (Display). We didn’t need to create the stub of the file to use; we could just use the interface. All the test patterns let the test be isolated in the test class/interface/production class triad. As a developer, it is easier for me to produce faster without quality loss if my context of thought is limited as much to the scope of the test as possible. Each extra window I switch is more contexts to deal with.

Here are three different tests:

public class ScannerTest extends TestCase implements Display
{
    private Item lastItem;

    public void testScan()
    {
        Item item = new Item("corn flakes");
        Scanner scanner = new Scanner(this);
        scanner.scan(item);
        assertEquals(item,lastItem);
    }

    public void displayItem(Item item)
    {
        lastItem = item;
    }
}

The first test is making user of self-shunt test. Self-shunt was a quick way to start writing a test. The implementation on the interface in the test class will give you a good idea on what the implementation of the concrete class should look like when you get there. Also, the test is simple and the syntax is expressive. However, the scalability is questionable. If you had more than one self-shunted interface in a single test file, it would be hard to start differentiating between which variables are used in which. Also, moving around the file to get a good sense of what a test is doing is misdirectional and distracting. Ideally, I want to see the entire test on one screen, so I can read it. If there were many implemented methods in a test file that were not tests or helpers, there is the potential to be confused.

public class ScannerTest   extends TestCase
{
    public void testScan()
    {
        Item item = new Item("corn flakes");
        Display mockDisplay = new MockDisplay();
        Scanner scanner = new Scanner(mockDisplay);
        scanner.scan(item);
        assertEquals(item,((MockDisplay)mockDisplay).lastItem);
    }
}

public class MockDisplay implements Display
{
    public Item lastItem;
    public void displayItem(Item item)
    {
        lastItem = item;
    }
}

Then I wrote the hand mock test. This test got rid of the clutter of implementing interfaces in the test and having local variables. Also, there are no external libraries needed, like a dynamic mocking library. The test is simple and easy to read. The biggest drawback is dealing with the hand mock test file. It is untested code that I have seen a tendency for behavior to slip into, which is disastrous. It can cause a bug in your tests that take too long to track down. Ideally this doesn’t happen, but I have seen it many times.

public class ScannerTest extends TestCase
{
   public void testScan()
   {
       MockControl control = MockControl.createControl(Display.class);
       Display mockDisplay = (Display) control.getMock();
       Item item = new Item("");
       mockDisplay.displayItem(item);
       control.replay();
       Scanner scanner = new Scanner(mockDisplay);
       scanner.scan(item);
       control.verify();
   }
}

Finally the dynamic mock test was written. This test was the simplest to write as a matter of process also the fewest lines of code. I never needed to leave the scope of the test function. It was just setting up my expectations and letting the test run. As Micah Martin pointed out to me, the test execution seems backwards and unreadable with dynamic mocks. The verification statements are at the beginning of the method and the execution is at the end. This is different than all the other tests in the system that follows an intuitive flow cycle of build, operate, and check.

With a little investigation I found Michael Feathers paper on Self Shunt (www.objectmentor.com/resources/articles/SelfShunPtrn.pdf).

In a project I have been working on, we recently decided to allow wrapped assertions to customize our unit tests. Our purpose was to make the unit tests be sentential enough to explain the meaning of the assertion in the execution of the assertion.

We can’t get rid of all developer comments (variable and method names) because, if we did, we would lose the ability to easily read and, consequently, easily maintain the code. In fact, writing easily readable code is a primary goal of good practices, since it is directly correlated with making code easily maintainable. This is important because maintenance is the most expensive part of the development process. However, we must be cautious about writting comments, since they can often have the opposite effect‚ less maintainable code‚ when they are misrepresentations or plain lies. There is a balance that results in easily read and understood tests. Investing in writing tests to tell a story to the developers who may have to look at the tests in the future, makes maintenance and TDD easier and with a better flow.

We chose to achieve this balance by merging the assertions with the comments of the execution of the test, resulting in a pseudo-sentence. The statement is an explanation for the next developer to read. The form we chose is

Verify.That(positiveAssertionString).ProvedBy(subject).verb(predicate);

An example of one of these assertions is

public void test_PressingBrakesShouldDecelerateVehicle()
{

 int originalSpeed = vehicle.currentSpeed;
 vehicle.brake();

 Verify.That(‚"Brakes decelerate vehicle.").ProvedBy(vehicle.currentSpeed).IsLessThan(originalSpeed);<br>

}

When a developer comes across this story, along with PressingEmergencyBrakeLocksTires(), SlamOnBrakesShouldSkid(), and BrakesShouldAntiLockAtIntervals(), the developer is told a story of what the class is doing. Why do we use custom asserts when good test names will solve the problem?

Changing tests should be cheap. As new designs and behavior evolve in the system, tests are changed to conform to the new structures. When implementing a new design to an existing system (such as breaking up an inheritance in favor of composition), I change the tests to fit the new structure on a macro level. However, this often implies that I am changing the behavior (if only slightly) to please the new design. Yet, test names, like comments, are not executable, so you can‚’t trust them to tell you what the tests test. On a system with heavy refactorings, a test might have its details changed five times without the developer thinking to verify the integrity of the names. An answer is to write sentential explanation of the test as the assert statement since the assert statement executes. I will trust execution over comments every time.

There is a project in its early stages called NSpec(.NET), which is a framework for assertions of a more sentential form. The project is based on the idea of Behavior Driven Development (BDD). RSpec, the same project for ruby has a form of BDD assertions which take full advantage of the features of the dynamic language. It allows you to add the assertion to the object from the test.

For example:
def pressingbrakesshoulddeceleratevehicle
vehicle.brake
vehicle.decelerating?.should_be_true
end

This is really cool! It defines the test as behavior of the object, which it is. However, when the code isn’t ruby, there needs to be other means in place to bind the description of the assertion to something which executes.

Fri, 20 Jan 2006 05:37:12, nraynaud, Tests in a changing world You’re right, we are slowly going to agile world, and for now we have lots of meaningless test taken randomly from existing code (does it makes sense putting a multi megabytes source file in the compiler you test and expecting exactly 2008 errors? without even covering 10% of the compiler). We work on a language, yesterday, I added a restriction to the langugage, it took me 10 minutes to implement and test the change but the rest of the day to change all the others tests.

Moreover, I thank Intellij Idea that renames not only classes but also varialbes named after that class and even comments, this is on the way you takes.

Fri, 20 Jan 2006 13:13:22, David Chelimsky, should_be_decelerating I’ve just submitted a contribution to rspec that will make this even cooler. If vehicle can respond to any of these messages with a boolean value: vehicle.decelerating
vehicle.decelerating?
vehicle.is_decelerating
vehicle.is_decelerating?
then you can write this in your test:
def pressingbrakesshoulddeceleratevehicle
vehicle.brake
vehicle.should_be_decelerating
end
Admittedly, this may be confusing to a programmer who is looking for a “should_be_decelerating” method on vehicle, but the messaging is clear it fails (” should be true”) and it definitely improves the communication of intent. If you’re doing BDD (i.e. spec-first), you’ll get a !-NoMethodError-! on “decelerating”. Then it’s up to you to implement a method responding to any of the forms described above.

Fri, 20 Jan 2006 13:57:32, , How about using Higher Order Messaging style and let the user write:
vehicle.should_be.decelerating?
The test framework could mix should_be and similar higher-order messages into the Kernel module.

Fri, 20 Jan 2006 14:40:41, David Chelimsky, should_be That’s kind of how rspec works now. shouldequal, shouldcontain, shouldbe_true, etc are all mixed in to Object, so you can do things like someobject.someattribute.shouldbetrue. Your suggestion of tying it to a “shouldbe” method (rather than parsing the shouldbe_xxx message) might be more clear. I think, though, that “shouldbe.decelerating?” is weird to read (because of the question mark). Programatically, however, it does make the statement look more like the implementation. I’ll think on it some more…