Legacy Coderetreat: Part 14 – Document possible Defects with tests

Document possible Defects with Tests

Blog post series

This blog post is part of a series about legacy coderetreat and legacy code techniques you can apply during your work. Please click to see more sessions about legacy code.

Purpose

When writing unit tests on legacy code, we often discover behaviours of the system that seem wrong. The main rule is: DO NOT change the production code without being absolutely sure that the change does not introduce defects. Instead we need to mark all the tests that characterize a possible defective behaviour. Let’s look at some ways to document possible defects with tests.

Document Possible Defects with Tests

Concept

In the previous blog post we discussed how to write unit tests on an extracted class. We discovered some strange behaviour, and we wanted to mark it as a possible defect. I will show three different ways to categorise tests that show potential defects.

Solution 1: Use custom annotation

In the last blog post we used a custom annotation @PossibleBug to document possible defects with tests. Here are two tests that have this annotation documenting them as possible bugs.

@Test
@PossibleBug
public void generatesPlayerMessageWhenPlayerNameIsEmpty(){
    String emptyPlayerName = "";
    String message = PlayerMessage.createWhenNewPlayerAdded(emptyPlayerName);

    assertEquals(" was added", message);
}

@Test
@PossibleBug
public void generatesPlayerMessageWhenPlayerNameIs255Characters(){
    String playerNameWith255Characters = "";
    for(int letterIterator=0; letterIterator<255; letterIterator++){
        playerNameWith255Characters += "a";
    }
    String message = PlayerMessage.createWhenNewPlayerAdded(playerNameWith255Characters);

    assertEquals(playerNameWith255Characters + " was added", message);
}

We created the custom annotation to make it easy to search. Most IDEs will let you press Find Usages shortcut on the annotation, and then find a list of possible bugs, like in the picture below.

Possible Bugs Usages with Annotation

This solution works very well in static languages and it can work in some dynamic languages as well.

Solution 2: Prefix test name

We can prefix the test name with a something like POSSIBLE_BUG_, and the test would look like you can see below.

@Test
public void POSSIBLE_BUG_generatesPlayerMessageWhenPlayerNameIsEmpty(){
    String emptyPlayerName = "";
    String message = PlayerMessage.createWhenNewPlayerAdded(emptyPlayerName);

    assertEquals(" was added", message);
}

We can then search the text in all the files to create a list of possible bugs.

This solution is well suited for scripting languages. It works for other languages as well.

Solution 3: Create separate test class for defects

We can have a class PlayerMessageTests where we hold all the tests we are sure of.

public class PlayerMessageTests {
    @Test
    public void generatesCorrectPlayerAddedMessage(){
        String message = PlayerMessage.createWhenNewPlayerAdded("some player name");

        assertEquals("some player name was added", message);
    }

    @Test
    public void generatesCorrectPlayerAddedMessageForRomanianDiacritics(){
        String playerNameWithDiacritics = "șțăâîȘȚĂÂÎ";
        String message = PlayerMessage.createWhenNewPlayerAdded(playerNameWithDiacritics);

        assertEquals("șțăâîȘȚĂÂÎ was added", message);
    }

    @Test
    public void generatesCorrectPlayerNumberMessage(){
        String message = PlayerMessage.createWithNumber(2);

        assertEquals("They are player number 2", message);
    }

    @Test
    public void generatesSentToPenaltyBoxMessage(){
        String message = PlayerMessage.createWhenSentToPenaltyBox("some player");

        assertEquals("some player was sent to the penalty box", message);
    }
}

And we can have another class PossibleBugsForPlayerMessageTests where we hold all the tests we need to discuss with business people.

public class PossibleBugsForPlayerMessageTests {
    @Test
    public void generatesPlayerMessageWhenPlayerNameIsEmpty(){
        String emptyPlayerName = "";
        String message = PlayerMessage.createWhenNewPlayerAdded(emptyPlayerName);

        assertEquals(" was added", message);
    }

    @Test
    public void generatesPlayerMessageWhenPlayerNameIs255Characters(){
        String playerNameWith255Characters = "";
        for(int i=0; i<255; i++){
            playerNameWith255Characters += "a";
        }

        String message = PlayerMessage.createWhenNewPlayerAdded(playerNameWith255Characters);

        assertEquals(playerNameWith255Characters + " was added", message);
    }
}

When working in this way, we take the files one by one and discuss all the situations with the business people. It is very easy to use the IDE to move the test from the class PossibleBugsForPlayerMessageTests to PlayerMessageTests, in the case that situation is not a defect. No annotation removal and no renaming is needed.

The same as in the previous cases, it is simple to have a list of classes where we have bugs, because all of them will start with PossibleBugsFor.

After we document possible defects with tests

No matter which solution you choose, the discussion with a business person is essential. It is very important to be able to consult the list of possible bugs fast and simple. Any solution you choose, make sure it is well suited for the language you use and for the IDE you use.

After the discussion with the business persons we either find out that those are in fact features, and we just delete the note or we need to fix the system. Fixing bugs in legacy code is described in Part 4 – Fix bugs on Legacy Code and in the code cast Episode 3 – Fix bugs on Legacy Code.

Outcomes

We started with discovering some possible defects. We understood that it is dangerous to fix these possible defects, because they could be features. So we decided to document possible defects with tests and use them as a starting point of conversation with the business persons.

After the discussion we can understand how and what to change. We then have a backlog of bugs to fix. We need to be very careful when fixing these defects, as some other systems might stop functioning well. So whenever dealing with legacy defects, be sure to test more than just your system, and communicate the change to external system that your system integrates with.

Remarks

When we document defects with tests it is a good idea to group all the tests documenting possible bugs one after the other. The discussions with the business persons are easier and you do not need to search the next test again and again.

Always discuss with business persons before changing the code. Often one might think they understand what the code does, but the situation if often very different in practice. Think to verify every detail with business analysts before changing the existing code.

History

I started using this method to document possible defects with tests because I was not sure which was the correct behaviour of the system when working with existing code, without having any tests and not enough documentation.

It proved to be very useful as often I found defects that were solved easily, but some times several discussions were needed with the business analysts to clarify if that case was a bug or a feature.

Code Cast

Please find here a code cast in Java about this session

Acknowledgements

Many thanks to Thomas Sundberg for proofreading this post.

The post Legacy Coderetreat: Part 14 – Document possible Defects with tests appeared first on Adrian Bolboaca.