0

In this question on the Aviation site, one of the reasons for the long development time of the F-35 was quoted as being Software. 8 million lines of code. Development from contract to in-service took 20 years, the last 10 years of which were mainly for systems since the first production aircraft was delivered in 2006.

It seems to me though that military software is like consumer software, in that it is built incrementally on top of existing functionality, and that development can be divided over many developers. Are there any limits to this, is there a sound barrier for software projects? I don't know if 8 million is an inordinate number of lines, but would this size be a reason per se to justify 10 years of development time?

What methods are there to limit development time in large software projects? Has there been a Henry Ford of software yet?

Koyovis
  • 109

3 Answers3

8

tl;dr summary: Software Engineering is a young discipline that is still figuring out how to build complex projects, and 8 MLOC is a very large and complex project.

Software Engineering is young. Very young. Like, two orders-of-magnitude younger than other engineering disciplines. Software Engineering started with the First NATO Software Engineering Conference in 1968, whereas e.g. Civil Engineering, Structural Engineering, and Mechanical Engineering started more or less with the building of the Great Pyramid … that's 60 years vs 4600 years of experience.

Or, to put it bluntly: we haven't figured it out yet.

There are a couple of things that make Software Engineering different from other engineering disciplines, most importantly the fact that software can be copied and re-used at no cost, and that software can be abstracted, parameterized, and generalized. What this means is that usually we only build things that nobody has ever built before, because if someone had built it before, we could duplicate that solution at no cost. This is different from e.g. structural engineering. You cannot just snap your fingers and clone a bridge, but you can do that with software.

So, other engineering disciplines have decades, centuries, even millenia of experience perfecting building (more or less) the same thing over and over again, whereas SE has only 6 decades of experience building completely new things every time.

Oh, and make no mistake: if you think comparing the F-35's software system to one of the Great Wonders Of The Ancient World is unfair to the herculean efforts by the ancient Egyptians, I would argue that the complexity of building a software system as sophisticated and advanced as the F-35's with its avionics, sensor fusion, and virtual reality environment (to name but a few) is comparable to the complexity of building something like the Great Pyramid.

The F-35 is actually not one plane but three. The F-35C has larger wings, so its aerodynamics are different. The F-35B is actually two completely different aircraft in one, one which generates lift with its wings and one which generates lift with its fan and engine nozzle. While this project saves cost and reduces complexity (at least that's the idea, there are people claiming otherwise) compared to developing and maintaining three different airplanes, it obviously adds complexity compared to each individual plane that would hypothetically have been developed instead, had they not chosen the "Joint" concept.

Add to that the fact this is a government project, where you have many potentially conflicting interests influencing the project that actually have nothing to do with the intrinsic functional or even non-functional requirements of the project.

8 MLOC is a large project. Think about the Linux kernel, its total size is about the same (~10+ MLOC at the moment, I think). But, roughly 90% of that codebase are device drivers (for example for dozens of graphics cards, dozens of SCSI controllers, obscure industrial input and output devices you have never heard of, …), so the actual Linux kernel is only about 1 MLOC. Of that, 90% is architecture-specific support code (for about 20 system architectures from no less than 3 PC architectures (x86 (32 bit), amd64 (64 bit), and x32 (amd64 in long mode, but with 32 bit pointers to save space)) to multiple different ARM, PowerPC, MIPS architectures to CPUs you have never even heard of). So, the real OS is only about 100 KLOC. Now, an OS is generally considered a complex system, and Linux is considered to be a large-ish OS. Well, the F-35's software system is about 100 times as large!

What methods are there to limit development time in large software projects?

That is essentially what Software Engineering is.

Jörg W Mittag
  • 104,619
1

One thing to consider about this particular example is that it's not simply the size of the code base or the complexity of that code. A big factor here is the risk involved. Consider 20 meter beam about the width of a balance beam set up less than a meter above the ground. Most people who can walk would consider walking across that beam to be fairly easy. Now consider that same exact beam placed 20 meters above the ground (in a controlled environment i.e. no wind.) Now most people would consider this a much more difficult task and would take more care in walking across it.

The act of walking across the beam is no different in either case yet it's clearly not the same to ask someone to do one over the other. The difference is the consequences of error. The case you have chosen here is one where the risks are immense. 1. As I understand it, a fighter jet is unstable and the computer systems are required to keep it flying. 2. The radar, navigation, stealth, and targeting systems are crucial to making it adequate for battle e.g. it could be destroyed easily by enemies if the computer systems do not function perfectly.

Most software doesn't have this level of cost. We usually are walking across the beam that is low to the ground. If we fall off, it's of limited consequence. This project is one where people could die and wars could be lost if it's goofed. The level of rigor and review is pretty much maximal.

Overall I think the short answer is that software does today get developed much faster than it was in the past. I don't have data, but I've been around long enough to notice changes. There are various reasons for this including but not limited to: better programming languages, better development processes, better standards, and better project management. One of the major factors has been the industry abandoning management models that were not appropriate for software development.

JimmyJames supports Canada
  • 30,578
  • 3
  • 59
  • 108
1

F-35's software size is not that large when compared with other systems.

The development process can make a huge difference.

Simply switching from waterfall and feature+integration branches to trunk based development (with a lot of test automation and a good CI system backing it up) can significantly reduce development time for very large/complex software development projects.

Such transition in a telecom equipment software (several times larger than F-35's) cut development time almost in half. Also reduced the overall number of checkins to about 30% - saving lots of resources previously wasted on the integration hell and on trying to stabilize those branches which weren't really what was shipped in the end.