16

To give a slightly contrived example, let's say I want to test that a function returns two numbers, and that the first one is smaller than the second one:

def test_length():
    result = my_function()
    assert len(result) == 2

def test_order()
    a, b = my_function()
    assert a < b

Here, if test_length fails, then test_order will fail too. Is it a best practice to write test_length, or to skip it?

EDIT: note that in this situation, both tests are mostly independent from each other, each one can be run in isolation, or they could be run in reverse order, this does not matter. So none of these former questions

is a duplicate of the one above.

Community
  • 1
Mihai
  • 271

4 Answers4

28

There can be value, but this is a bit of a smell. Either your tests aren't well isolated (since test_order really tests two things) or you're being too dogmatic in your testing (making two tests testing the same logical thing).

In the example, I would merge the two tests together. Yes, it means you have multiple asserts. Too bad. You're still testing a single thing - the result of the function. Sometimes in the real world that means doing two checks.

Telastyn
  • 110,259
5

Your tests should be explicit. It's not completely inferred that if text_length fails test_order fails.

I'm not sure how it goes in Python which you've posted, but if len(result) is 3 then the first will fail but the second may pass (and if not in Python, then in languages like JavaScript for sure).

Like you said, you want to test that the function return two numbers and that they are in order. Two tests it is.

Madara's Ghost
  • 8,997
2

The only value of test_length here is that, if everything passes, its very presence indicates that "length" has been tested.

So it's really unnecessary to have both these tests. Keep only test_order but consider renaming it test_length_and_order.

Incidentally, I find the use of names that begin with test a bit clumsy. I'm a strong advocate of test names that actually describe the condition that you're asserting.

Dawood ibn Kareem
  • 1,851
1

I'd leave these tests separate if that is how they've evolved to be. Yes test_order is likely to fail whenever test_length does, but you definitely know why.

I'd also agree that if the test_order appeared first and you found a testable failure was that the result was possibly not two values, adding that check as an assert and renaming the test to test_length_and_order makes sense.

If you had to also check the types of the values returned were integers, I'd include that in this "omnibus" results test.

But note now you have a battery of tests for the result of my_function(). If there are multiple contexts (or, more likely, multiple parameters) to test this can now be a subroutine to test all results from my_function().

However, when I write unit tests, I normally test the edge and bad cases separately from good input and normal output (partly because most of my "unit" tests are often mini integration tests), and often with multiple asserts, which are only broken into separate tests if they fail in ways where I find I want more information without debugging.

So I would probably start with your separate tests and expand test_length to test_length_and_types and leave test_order separate, assuming the latter is seen to be "normal processing".

Mark Hurd
  • 343