TDD Mock call verification - is it an anti-pattern?

Question

I've been doing TDD for a while now, I feel pretty good about it, I love my test suites and all. However I've noticed that lately I've been doing a lot of mock call verification. For example I'd have a Service that will have a Repository injected - in my unit test I'd pass a mock of the Repository and verify that it was called within the method that I'm testing. I'd then check if the results returned are correct (in another test). This definitely "feels" wrong, since my unit tests are now tightly coupled to the implementation details. I've heard that you should test "behavior", however in a lot of the situations that's ... emm - not possible? If you have a void method for example, you usually test side-effects. I mean it's easy to go ahead and show some simple code-kata's where this can be demonstrated, but IMHO it doesn't reflect very well to the real world programs that we write. Is what I'm doing wrong? Is this type of testing sort of an anti-pattern? I'd appreciate your opinion on this.

Answer 1

Well, you should be trying to test inputs and outputs. You should be verifying externally visible behavior. The "promises" or "contract" that your class makes.

At the same time sometimes there's no better way to test a method than to do what you said.

I do think that it makes your test more brittle, so you should avoid tests that rely on implementation details if you can, but it's not an all-or-nothing deal. It's OK sometimes, the worst thing that happens is you change the implementation and have to update the test.

Answer 2

The purpose of a test is to restrict the possible productive implementations. Make sure that you only put restrictions on the implementation that you actually need. Typically this is what your program should do, and not how it does it.

So if for example your service adds something to the repository, you should test that the new entry is contained in the repository afterwards, and not that the add action is triggered.

For this to work, you need to be able to use the repository implementation (tested elsewhere) in the test of the service. I found that using the real implementation of a collaborator is generally a good approach – because it is really the best implementation around.

---

"So but what if using the real implementations in the test is expensive (e.g. because they require resources which are complicated to set up)? I need to use mocks in this case, right?"

In any case you'd probably want one integration test which tests that the real implementations work together. Make sure that this one integration test is all that is needed to test your service. Or in other words: If a service plugs together a lot of collaborators (and is hence potentially hard to test), make sure that it doesn't contain any logic. If it does, and you'd need multiple (integration) tests, you need to change the structure of your code, e.g. by isolating the logic and hence making it more testable.

Using mocks in this case eases the pain of testing a piece of badly isolated logic, and hence hides an architectual problem. So don't use mocks to test badly structured code, but fix the structure instead.

Answer 3

My thoughts re: 'aggregate services'.

Call verification will do this, but won't provide much value. You're just checking your wiring.

There are 3, non-exclusive, other ways to this:

1. Extend the tests you have for each individual service so that it checks the higher level behaviour. For instance, if your hitting an in-memory database in your unit tests of the service, take it up a level so you're testing the service against an actual db. The service layer is higher up the abstraction tree, and so should your test.

2. Use code generation to create the service directly from the aggregated services.

3. Use some sort of reflection or dynamic language to do the same thing. For example, in Java, it may be possible to use a groovy interface, which passes the call on directly.

There are probably other ways to do this, but just checking wiring has very low pay back, and will hard wire you in to this implementation.

Answer 4

Call verification testing is a waste of time.

Why call a method? There are two reasons:

• The method returns a value.
• The method has a side effect.

That's it. A method might do both of those or only one, but if a method doesn't do either of those, then it doesn't do anything at all.

Now consider a line like this:

mockContext.Verify(x => x.SaveChanges(), Times.Once());

What does this test?

• It doesn't test any value the method might return.
• It doesn't test any side effect the method might produce.

No, the only thing it tests is whether the method was called, but it says absolutely nothing about why the method was called, or whether the method even did anything when it was called. The method could have done the wrong thing, or even nothing, and this test has nothing to say about it. The entire SaveChanges method body could be commented out or deleted and this "test" would still pass!

Call verification "testing" adds pointless, tightly-coupled "tests" that don't test anything.

Answer 5

Basically all the big answers here are correct. I've been throughout the same questions and, unfortunately haven't found something really clear. But then I realized that the answers where always there. I've read Roy Osherove in his wonderful The Art Of Unit Testing book, and also Kent Beck's TDD book, along with some others and lots and lots of reading and streaming about the subject. It's been a while since I saw my first test. So, in order to answer and enforce what was already said here: test the behavior, not the way you write the code. So validate that GetAll returns to a list of objects and that GetById returns the object when found or null (object.empty?). This way you'll mock whatever is inside your implementation and then test how your method behaves (does it respect its contract?).