How does conditional compilation impact product quality, security and code complexity?

Question

Software libraries targetting resource constrained environments like embedded systems use conditional compilation to allow consumers to shave space by removing unused features from the final binaries distributed in production.

What are the tradeoffs to building a unique binary? Surely there are the obvious space savings on disk, memory and cpu caches; and the performance benefits brought by space-time tradeoffs. But at least initially it seems that would increase the possibilities to test and design against.

It seems that the implications differ from those of traditional runtime path complexity, not just because of the consideration of 2 different types of branching, but because compiler condition syntax is far unsafer than its runtime counterpart (at least in C).

The specific example that sparked this question is Busybox heavy use of conditionally compiled feature flags. https://git.busybox.net/busybox/tree/networking/httpd.c

Answer 1

Generally speaking, the use of conditional compilation increases complexity which makes it harder to achieve security and other desirable qualities. A part of this complexity is essential if you want to be able to introduce a certain amount of variability. The other part is accidental ans relates to the technical details of conditional compilation and available tool support.

The use of annotation-based variability mechanisms in which the code is annotated with variability information such as conditional compilation affects understandability and testability:

Reduced Understandability How do you know the context of a certain part of the code so you can understand it? How does the compiler know, so you can at least be sure there are no type errors etc.? The problem is that code related to configurable feature is scatteref across the code base. A certain discipline is required to cope with this problem as mentioned in another answer, but does not eliminate the underlying problem.

Reduced Testability How do you know the software works for all combinations of options?

The answer to these questions is that it is not possible for you, the compiler or mainstream testing tools to sufficiently reason about all possible variants before actually building them. The problem is you cannot build all combinations because of the resulting combinatorial explosion. There are concepts for variability-aware analyses but they are not widely adopted, yet.

While as a human you can to a certain extend try to consider a limited amount of variability when understanding code and there exist certain technical concepts to deal with the problem, it is still an #ifdef hell out there in many projects.

As you already mentioned there is no widely available tool support that helps you deal with conditional compilation.

The good news is that people in research are working on solving these problems and investigating alternative variability mechanisms. For instance composition-based variability mechanisms help with local reasoning by modularizing features ( think AOP ). There are also tools that help avoid errors when using annotation-based mechanisms by checking annotations against an explicit variability model for valid combinations.

To deal with testing in the presence of variability im general, there are several sampling approached that help select representative combinations based on different criteria.

There are also variability-aware analysis tools, but those still lack mature implementations and industry acceptance. For instance, many standards require product-bases analysis e.g. for safety-critical domains. Currently, there is a good bunch of concepts that may find their way into mature tools one day.

Answer 2

What are the tradeoffs to building a unique binary? Surely there are the obvious space savings on disk, memory and cpu caches; and the performance benefits brought by space-time tradeoffs. But at least initially it seems that would increase the possibilities to test and design against.

Assuming you're only talking about embedded systems, targeted for resource-constrained environments, like you said:

• For projects that need to be cross-platform, probably having an unique binary is impossible due to compilation errors yielded by different compilers on unknown structures or symbols.
• If dynamically linking to other things is not possible during runtime (this will depend on your project and environment targeted), using conditional compilation is a possibility that would solve this.

IMO, if you properly implement your project having separation of concerns and best coding practices in mind, no matter the technique you choose (conditional compilation or not) your project will be easily and safely implemented and maintained.

Example: if you have code for environment A, separate it into a specific file that will be compiled when you target environment A. Wrap/group stuff that belongs together into specific files and folders for that scenario, instead of putting #ifdef SOME_CONDITION spread throughout all of your code.

So, trying to answer your question -- How does conditional compilation impact:

1. Product quality: if done right (like the example above) it might not impact product quality, since the code does not get polluted or difficult to read (quite the opposite, because you moved code to separated files, and configured those files to be compiled based on some condition); additionally, for testing I don't see how much difference would it take to test several scenarios by: (1) testing different executables generated or (2) testing the same binary, but changing a configuration file; the performance possibly will increase, because a runtime approach will check for all possible options of your program, while a conditionally-compiled program will have only the necessary binary in there;
2. Security: again, if done right, you can be sure that your program does not contain a specific code that should not be there (for whatever reason of your project), or you can build a version of your application to contain only the capabilities/permissions necessary for a environment (e.g.: to avoid things like privilege escalation); also, you can use conditional compilation if you want to secure the usage of open-source code inside your program (example: you generate a specific version of your program that, for whatever reason, cannot contain GPL code inside of it)
3. Code complexity: if you spread ifdefs throughout all of your code, it obviously gets more and more complex and eventually unfeasible to maintain; if you can separate them into feature- or platform-dependent files, the complexity for writing code or reviewing will not be impacted that much.