How does one reliably test core Linux software (like package managers, UNIX utils, desktop environments, kernel modules, etc.)?

Question

If I'm not mistaken, a major part of testing software is deploying and using it on your platform of choice. For critical system software such as UNIX utils, package managers, desktop environments/window managers, kernel modules, etc., how would a developer reliably test the software without bricking their development environment?

Answer 1

You typically use a test environment that can easily be reset or rebuilt to a good state. There are different possibilities: Dedicated real machines (normally required for testing drivers), virtual machines, containers, emulators, separate filesystem paths, etc.

If you're adventurous and fully trust your backup process, you may of course test your developed software in your development environment, and restore it from backup if it breaks beyond repair. However, this is most often not the most efficient approach, as you would need to perform backups frequently so you don't lose too much.

Answer 2

The answer to the question is not really OS-dependent, but at the same time, it is. For example, on Linux, you can test the window manager, and if it breaks, you don't have to reboot, whereas on Windows, there is much less separation.

That said, you test the tools much the same way as any software. Let's say you need to test a cp command line tool. You don't need to test that you actually copy a file. If you can mock things so that you are testing what fopen is called etc., that is generally "good enough" for general testing.

For lower-level things like drivers, there is nothing for it but to have a real machine with real hardware and test against that.

If you are looking at something much higher level, like "packaging tools," you still do the same thing. General testing can be done by mocking things (one way or another). For example, if you are writing apt, You might create a repo with only one package and a small manifest. Then run your tests against that.

In the end, while the toolchain will change, the concept is the same.

• Test the most atomic thing you can.
• Trust the libraries that are not yours (or gain trust) (no need to test fopen that's not part of your project)
• Don't test 1s and 0s written to disk, test that the input produced the desired output (can save and read a file, not 101100110100 was written to sector 3)
• Fake the rest. (VM, different directory, mock objects, etc)