Is it possible to write software that doesn't need to be continually modified?

Question

I have written a lot of software in many different languages, and I've also "written" hardware for use with FPGAs using Verilog and VHDL.

I tend to enjoy writing hardware more than software, and I think one of the primary reasons is that it is possible to write hardware that is "done" and never needs to be modified: you define the interfaces and the functionality, write a test bench, implement the hardware module, then test the heck out of it using a simulator. Then you can rely on that hardware module as a building block to create something bigger and better: if you need to add a feature to that module, you create a second module and add the functionality there. You never throw away the original module since it is working just fine and can still be useful.

One of my main frustrations with software is that it is never "done". There is always another feature to add. Often when adding a feature it introduces a bug somewhere else that was working just fine before. This doesn't happen in hardware as long as the interfaces are not violated.

To be clear, I'm not advocating building one version of something with a list of features and that's it forever: I'm in favor of iterations and multiple releases over time to add new features. I just don't want to poke the code on the left and find a bug on the right, and this seems to happen after adding a new feature.

Is it possible to write software in a similar way in which hardware is "written"? Is there a good software development methodology that allows forward progress to always be made and allows new functionality to be added without needing to rewrite existing code and introduce new bugs?

Answer 1

Maybe it has something to do with well-defined interfaces and testing, like hardware?

Exactly my thoughts!

Well-designed modules with clear interfaces tend to be essentially perfect. Think of something like String class of Java. It is a computer program, but it has a crystal-clear interface. There are no known bugs in it. It does what it is supposed to do, perfectly. Sure, it has been extensively tested during the last 15 years, and since virtually all programs use Strings as basic building blocks, any bug in it would be quickly noticed. Any "quirks" - not strictly bugs, but design details worth being aware of - such as those described here http://www.jwz.org/doc/java.html are well-known by now, and thus can be taken into account.

Buggy software is partly a cultural issue: people are used to buggy software, and unlike hardware, software can usually be easily fixed afterwards, so it doesn't have to perfect initially (or ever, because hey, we must ship now, let's fix it in the next version). But for a large part, it is a real complexity issue: software complexity has been steadily increasing for the last 50 years or so, but the human brain is the same. When the growing difficulty of achieving perfection and the growing ease of fixing things later on (fast, automatic builds, internet distribution) combine with schedule pressure and lack of discipline, the result is what it is.

Is there a good software development methodology that allows forward progress to always be made and allows new functionality to be added without needing to rewrite existing code and introduce new bugs?

No silver bullet probably, but a good combination of best practices, including but not limited to:

• Simple, autonomous modules. In other words, low coupling and high cohesion.
• Immutability. Particularly important with growing concurrency.

It's worth noting that both points aim at reducing complexity. That's the key point. The entropy always tends to increase, and unless we fight that, we'll soon drown in complexity. It's also interesting to see that during the last few years, programming languages have been evolving toward encouraging or even enforcing the above mentioned practices. Particularly, the rise of functional languages is just that: pure functions always return the same value for the same input, there's no state in them. Then you just compose pure functions that take and return immutable values, and limit the inevitable mutability to small well-defined places instead of spreading it all around. Check this out: http://clojure.org/state

Answer 2

The difference is just how much easier and cheaper it is to modify software compared to hardware. If hardware was as easy and cheap to modify and ship to customers, they would.

I think if I could sum up my poorly-expressed question it would be something like, "how can I have more fun writing software by not introducing bugs into working code and always making forward progress?" Maybe it has something to do with well-defined interfaces and testing, like hardware?

You should definitely check test-driven development.

Answer 3

I will comment on some of your remarks, hoping you get the answer from these comments.

One of my main frustrations with software is that it is never "done".

This is because the solution specification is incomplete or because the plan to deliver enhancements is not accurate. This can happen to any project software, hardware or any other.

Is there a good software development methodology that allows forward progress to always be made and allows new functionality to be added without needing to rewrite existing code and introduce new bugs?

Of course, creating independent modules should greatly reduce dependency. This has to be considered when you are designing the software. You need to consider, separation of concerns, layers, tiers, controller objects, interfaces, etc.

"how can I have more fun writing software by not introducing bugs into working code and always making forward progress?"

1-Understand requirements carefully. May be you need to consider closing the requirements before design. If you do iterative development, there is no chance of doing this. Some methodologies encourage this, but personally, I think this is not good for all project types. Building software on solid requirements allows for better design.

2-Make your user understand this long-tern philosophy.

3-Plan implementation carefully

4-Design before code.

5-Use generic design when appropriate.

6-Use prototypes as a design confirmation tool.

Answer 4

As people are usually very quick to point out, one of the benefits of software is that it is supposed to be easy and relatively cheap to change compared to hardware. This is especially important when your realize late that you got something fundamentally wrong. Do the same with hardware, and you lose a million dollars, so as you've said, you use your simulators etc, and you test the bazinga out of it. This I think is where the paradigm fails somewhat when you shift to software.

Get into the average software developer's head, and what you have is a very busy person with an incredibly tight deadline. His manager is saying that it's ok to leave a few bugs because you can always fix it later. Tests are often an afterthought, but even in a test-driven scenario, the tests are kept minimal and the code written to the minimum of the tests, and often shortcuts are taken so that many of the boundary cases might be missed. The system may be thoroughly unit tested yet is rarely tested rigorously as a whole, and just as rarely stress tested to any great degree. Add to this that you write software from scratch, and there is little opportunity to simulate the software before you commit to writing it, primarily because we rarely write software from the same sort of fine-grained building blocks that you would find in hardware. Taken from this perspective - which I don't believe is too far fetched an example - the software is not tested to the same high degree that hardware would be, because realistically it can't be, and to attempt to do so would not be seent to be very cost effective.

Back to the OP's question. Could you define a system of building blocks from which to derive all of your software? Possibly. Would it be very cost effective? Probably not, because by the time you get around to developing a robust enough system of components, tests, and other paraphernalia to support this ideal system of programming, you would find that your competition would have already beaten you to the market, and even worse, from the average programmer's perspective you would probably find a "cookie-cutter" style of programming system to be very limiting and more likely very boring. I personally work on an API, where the bulk of the module code has been refined and standardized so completely, that all I do now is generate a code template and fill in the blanks. Most of my time can be spent writing simple connector code, and getting the modules out the door as fast as possible. It's seriously mind-numbing. There is very little opportunity to do more than just code the same sorts of things over and over, so when another project opportunity comes along, I jump at the chance to be able to do ANYTHING else.

So how can you get to deliver high quality and well factored software, and yet actually enjoy yourself doing it? I believe this comes down to your choice of tools and methodology. For me the answer has been to employ the use of a good BDD API, because it has allowed me to create very easy to read, yet highly granular code. I can build a suite of tests out of a minimal number of reusable methods, and describe my tests in the language of the specifications. In this way, I come close to a more component-ized development approach, except for the fact that I am responsible for designing and checking the building blocks. Additionally, the test output pinpoints the exact part of the test where the failure occurs, so that I don't have to guess whether the failures are in the setup or the assertion. This means I spend more of my time focused on solving problems instead of chasing them, my time is used more efficiently, I get more work done more quickly, and I can move on to other interesting tasks more quickly.

Tuning your methodology also helps. I'm a big advocate for applying lean development principals, and combining them with many of the other techniques and principals that the Agile movement has been banging-on about for many years now. Having eliminated most of the wasteful practices that I used to find so frustrating has helped a great deal to make development a more enjoyable activity. I'm still left with the issue that sometimes - but hopefully not too often - bugs will appear in my code, however I now find myself with more time, and even more incentive to spend more time writing more robust tests, and aiming for 100% test coverage. Even better, it feels really great to see all of those green lights appear at the end of my day, and be able to provide a complete - automatically generated - test report to give to my boss at the end of the day.

Answer 5

One of my main frustrations with software is that it is never "done". There is always another feature to add.

If that frustrates you, consider a different career. Seriously.

The point of software is to be able to add features. The entire reason for inventing "software" in the first place was so that we could add features.

Often when adding a feature it introduces a bug somewhere else that was working just fine before.

That's a QA problem.

This doesn't happen in hardware as long as the interfaces are not violated.

That's also true in software.

Is there a good software development methodology that allows forward progress to always be made and allows new functionality to be added without needing to rewrite existing code and introduce new bugs?

Yes. You have to actually practice Quality Assurance.

Answer 6

it is possible to write hardware that is "done" and never needs to be modified: you define the interfaces and the functionality, write a test bench, implement the hardware module, then test the heck out of it using a simulator. Then you can rely on that hardware module as a building block to create something bigger and better: if you need to add a feature to that module, you create a second module and add the functionality there. You never throw away the original module since it is working just fine and can still be useful.

In the software world, we call that "module" a library, and we use it just the same way. Many libraries are built to the point that they function well, and then sit contentedly doing their jobs with no change until Something Important leads to the next revision. Think of them as software that's been potted with epoxy :-)

One of my main frustrations with software is that it is never "done". There is always another feature to add. Often when adding a feature it introduces a bug somewhere else that was working just fine before. This doesn't happen in hardware as long as the interfaces are not violated.

Hogwash. Perhaps you're personally better than many other non-soldering-iron "hardware" people, but I've seen any number of bad circuit designs, failing chips (e.g., the famous Intel "f00f" problem), but that doesn't speak to the field as a whole. And as faux-hard ware becomes "softer", the problems become more difficult to prevent.

Is it possible to write software in a similar way in which hardware is "written"? Is there a good software development methodology that allows forward progress to always be made and allows new functionality to be added without needing to rewrite existing code and introduce new bugs?

Yes. We just don't use those methodologies much. They tend to be extremely expensive to operate, and most programmers don't enjoy working within their restrictions. But when human life is involved, for example, well, yeah, we try not to kill the users.

One last point: Software has a different financial model from hardware, even programmed hardware. Most non-consumer software, and some consumer software also, is sold in a manner that encourages change. When you can tell a business "Pay us $10,000 now plus 18% a year", you can essentially re-sell the product every few years. But to justify that fee, you need to give the customer the changes they want. Hmm ... thinking of Apple's hardware-obsolescence curve, perhaps this isn't a difference after all - hardware just makes you really re-buy it!

Answer 7

Is it possible to write software in a similar way in which hardware is "written"? Is there a good software development methodology that allows forward progress to always be made and allows new functionality to be added without needing to rewrite existing code and introduce new bugs?

I recommend you look into "formal methods" for verifying the correctness of designs and software. The simulator tools you use for hardware design are trying to do something close. I do not believe that tools for formal methods are anywhere close to being useful in industry at this time, and the only industries that have strong incentives to be defect-free are avionics and medicine (interestingly enough, the FDA clearly says "software is different from hardware" on that link). Furthermore, if you are developing with Verilog/VHDL, then you are sticking with binary logic. This drastically reduces complexity. There isn't going to be a hardware equivalent to a Y2K issue.

The big problem is that things are complex. And you cannot eliminate complexity, you can only move it around.

Answer 8

I would also point out that software bugs in hardware can often kill people. So there is more care taken to thoroughly determine the requirements and extensively test them up front. And those requirements dont need to change until the hardware does. And since new hardware may require a rewrite, I suspect the cruft doesn't build up so much either.

Business requirements on the other hand change constantly, sometimes you can hardly get one requirement into production before a change is asked for. Sometimes, I've had the requirement change multiple times before it does get to production. This is a result of several things. First the project stakeholder on the business side is often less interested in spending the time to thoroughly define what he wants because he is "busy" and "important" and people won't die and the families sue him or get him thrown in jail if he blows off his part of the process. Second, the project stakeholders tend to actually have a better idea of what they want the hardware to do as it is less abstract to them. They really don't know what they want until they see it. Which is less of aproblem with Hardware.

Answer 9

how can I have more fun writing software by not introducing bugs into working code and always making forward progress?

I would love to find a final answer to your question. But the reality is that there is no easy way to do it, thats why extreme programming and TDD technics are becoming so popular. You need to embrace change, because is going to happen. I dont know if its funnier this way, but a lot less stressful for sure ;-)

http://en.wikipedia.org/wiki/Extreme_Programming

When you interact with hardware, the hardware needs x value and thats all (in theory), but when you interact with people today they need x, and tomorrow they can need y, etc. It is how it is, bussines and people requirements change. Because Persons != machines, so code that NEVER changes most of the times is not possible.

Also as i said on my previous/deleted answer, try to avoid changes that are not important by making people think before start coding. Involve users more on decision making, etc. Clarify costs of changes, plan more, etc. Those are not "ways of coding", are ways of "not coding" because with more though about requirements there will be less changes and more fun.

Answer 10

Is it possible to write software in a similar way?

Yes, it is. Just be as careful as if you are developing hardware, test everything you can, and your software will be of similar quality.

by the way, haven't you heard of HW bugs? Much nastier then any SW bug and harder to fix (not just upgrading the software)

Answer 11

There are high level tools with lots finished 'bricks', as you call them, that you can combine into applications. The bricks are finished pieces for you to use, you just have to combine them together. Maybe you think that's easier... until your customer asks you for some weird and unexpected changes.