What are the reasons not to use random values in unittests?

Question

I've read the questions regarding the use of random values in unit-tests and, well, I still don't quite understand what the argument against random values is.

I'm trying to understand because I've had an entire course dedicated to testing while I was doing my Masters in SE. The course focused heavily on random value generation, set theory and Hoare Logic. Maybe those ideas were without merit but I don't see why to be quite frank.

Why using random values is a good idea

The reasoning during the course was as follows.

If I have a Unit Under Test (UUT) which has a string parameter in its signature then, the UUT's post-condition predicate should remain true when I pass any element from the set of all strings.

Hence, I should be able to pass the UUT a random string value. I.e. a randomly selected element from the set of all strings.

If the UUT has a precondition predicate for the string parameter then, the UUT's post-condition predicate should remain true when I pass any element from the set of all strings for which the precondition predicate is true.

Hence, I should be able to pass the UUT a random string value for which the precondition predicate is true. I.e. a randomly selected element from the set of all strings for which the precondition predicate is true.

So what is the argument against random values?

Following this logic, I do not understand what the argument is against using random values in unit-tests. If my UUT has a parameter then, its value should be expected to vary. Parameters are variables after all.

If the parameter value should not be expected to vary then, the parameter can be removed because it's a constant.

If I only test a UUT with a constant then, what am I really testing except for a single value from an ocean of valid values?

A unit-test should be written in such a way that it can never fail because you've used a random value. If it does fail then, it does not do so because of the random value. Unless, the random value generator no longer generates values that match the precondition predicate or, it never has.

The only argument I could make against using random values is that people do not understand how the values are being generated. Hence, they find it "hard to debug". However, this is not an argument against random values in unit-tests. It is a reason to improve people's understanding.

Note that I am disregarding the "repeatability" of unit-tests on purpose. It is a given that a proper seed is to be used.

Thanks for any responses in advance.

Answer 1

It is a given that a proper seed is to be used.

The real world is very different from your masters course. People in the real world very often don't do this, and then it's a problem for the reasons you know about.

Answer 2

Unit tests are meant to be reproducible, i.e. unless the code changed somehow, their results must not change.

For unit tests, it's much more helpful to devise test cases for border conditions (for example, does the code handle long strings correctly, possibly by throwing an exception?). If your code needs to handle special cases, for example attempted division by zero, you better check these cases reliably every time you run your test suite instead of randomly whenever your RNG feels like outputting a zero for that parameter.

There's a use for random values in fuzz testing where you call code with weird parameter values trying to break things. Once you found such parameter combinations you've probably got some more border conditions to handle or off-by-one errors to fix.