Good unit test – One Assert

In previous post, I defined a good unit test as one that:

  • tests single unit of work
  • contains only one assert
  • is self contained

Presented sample of code with one unit test, unfortunately had two asserts. Clear violation of the second rule. One of the fellow bloggers MJ pointed out this mistake. It is time to “fix” it and talk about “Only one assert per test” rule.

Why only One Assert

There are couple of problems with multiple asserts.

Unit Test Misinformation

Frameworks like NUnit do notify about failing unit test when one of the asserts is not met. You get the message that some condition failed, but only this one condition. Code, behind the failing assert, is not executed. In a scenario with multiple asserts, when the first one fails, test procedure is lost and potential problem looks like related to the first assertion check. This is a misinformation.

Presented example tests GenerateDocumentNumber function. This function creates simple string based on provided ID.

We want to check if :

  • document number contains ‘id’ value
  • document number contains keyword ‘new’

If procedure fails on newString assertion we know that one part of the algorithm doesnt work. The problem is that check for ID wasn’t done at all. We don’t know if ID is set correctly. Test like this, with multiple asserts, can’t be trusted.

Complicated Tests

There is another problem with the presented unit test sample. It is too complicated. Unit test has to be focused only on one part of the algorithm. First assertion, checks if there is a ‘new’ keyword. Second, looks for ‘ID’. Both of these scenarios are great candidates for separate test.

Also, part of the code responsible for the ‘ID’, is probably more general and doesnt apply only to ‘new’ documents scenario. It is misleading, based solely on the unit test code and the name, we can assume that ‘ID’ is only added to new documents code.

We can split this test in two:

  • GenerateDocumentNumber_when_new_document_then_contains_string_new
  • GenerateDocumentNumber_contains_id_of_the_document

With one assert per unit test mindset there is better chance of creating good unit test, concentrated on one particular problem.

Going back to previous post example.

There are three behaviours that we have to test.

  • output ‘*’ char for each char in provided password
  • return password
  • ConsoleKey.Enter breaks the procedure

‘Output char’ and ‘return password’ logic should be tested separaterly. The Enter key functionality is unfortunately more complicated beacuse it is mandatory in each test scenario. However still we can test some edge scenarios like “What happens when there is no Enter key ?”

One test per concept

There are of course exceptions and unit tests that could use multiple asserts.

Quote by Roy Osherove from Yeah it’s a blog

My guideline is usually that you test one logical CONCEPT per test. you can have multiple asserts on the same *object*. they will usually be the same concept being tested.

This rule allows multiple asserts but they have to be related and linked with simillar CONCEPT. For instance, this is not bad if multiple asserts are testing something like this.

We have three asserts but they are basically testing the same thing. If one of them fails then the rest can’t be ulfilled. Fail on the first one automaticlly means that whole test failed and something bad happened. Nothing is hidden.

Good Unit Test – Summary

  • try to use one assert per test
  • if there is a need for multiple asserts, remember one test per concept rule

Links

Roy Osherove Site

Post with solution to “hide” asserts

Share on Facebook+1Share on Twitter
  • http://profiles.google.com/grauenwolf Jonathan Allen

    I don’t understand why people lose their ability to write code when it comes to testing. Here is a couple good examples…

    > If procedure fails on newString assertion we know that one part of the algorithm doesnt work. The problem is that check for ID wasn’t done at all. We don’t know if ID is set correctly.

    That problem can be solved in less than a day. It is easy to write a test framework that allows for multiple asserts to be aggregated.

    > For instance, this is not bad if multiple asserts are testing something like this.

    What a waste of time. You could write a function called Assert.IsNotNullOrEmpty and reduce your line count by two-thirds.

    And the asserts are wrong anyways. Since you forgot the message parameter it is impossible to tell if documents or documents[0] is null.

    • http://www.mfranc.com/ Michal Franc

      > That problem can be solved in less than a day. It is easy to write a test framework that allows for multiple asserts to be aggregated.

      Yep like this one here :
      http://rauchy.net/oapt/

      >And the asserts are wrong anyways. Since you forgot the message parameter it is impossible to tell if documents or documents[0] is null.

      Ouch, that is a big mistake from my side here, and propably this example is not the best one.

  • Robert Pająk