How come my Waterfall projects were largely more satisfying than all those new "agile" projects?

Question

Disclaimer: I think about the agile values a lot, and happily support them.

After almost 20 years of "non-agile" software development projects, and a couple of years in Scrum and alleged "XP" projects, I'm sorry to say that:

• the non-agile projects worked out pretty well (they were <= 1000 man-days, small-ish teams, ~3 milestones in each project)
• in general it was really satisfying to have rather large assignments and to organize work and communication quite freely and cooperatively
• and yes, I learned a lot despite not regularly (and not for endless hours) sitting next to others in order to do pair programming
• nowadays, everyone around me hates being suffocated by "agile" processes, tiny tasks and massive control machinery around
• people change jobs a lot more than before, become quickly frustrated due to constantly being regulated, and actually the productivity goes down tremendously

Of course I read "so obviously you guys are doing [insert-cool-method-here] wrong!" a lot. Well, then maybe it's quite difficult to do it correctly, and so far it has only frustrated everyone I know, and not brought any benefit - except that we learned about ticket-based development and how it is making us really unhappy. (I may add that I know of a couple of rare cases where Scrum worked out OK for rather low-profile work. Also the junior-senior ratio seems to be much higher today.)

I understand that requirements change, communication is important, and we need ways to keep track of the project. Actually I've also been a quite decent "rather traditional" project manager, too. Well, we ... inspected and adapted, actually! And we didn't even give it any special name.

One thing I don't get at all is why more process, and more control, should ever make software development better? To developers (and probably many other jobs), I sometimes feel some sort of Heisenberg effect applies.

If any recent management method is supposed to be the solution to any evil-Waterfall-related problem, then I decidedly want that problem back.

But then, a question remains: how come my Waterfall projects were largely more satisfying - and also more successful in a business sense - than all those new "agile" projects? Is there a practical, yet not-so-much advertised approach I may dare to suggest to my managers?

Answer 1

Disclaimer: what follows are merely my own opinions, with my limited experience of 4 workplaces.

First, I think the most important is the people you work with. I kind of like the term "peopleware". Because in the end we do the software, not some methodology. And the results highly depends of the people involved and they work together.

Regarding the methodology, I think applying either in their "pure" form sucks badly. The ideal IMHO is something in between that fits the team/project.

Waterfall

"Plan first, then do!"

How it can be done well

It makes sense to clarify what should be done and how before writing all the code, especially if it's a fairly big project. You first analyze what should be done, then split the work, then do it. It's quite effective actually. Waterfall doesn't mean that it's written in stone either. Of course you re-iterate and adjust it as you go. It's just that you plan what to do before you do it instead of the other way round.

How it can go wrong

Of course it's possible to make disasters out of it. Distance between the "architect" and the developers, over engineered stuff, with lots of intertia, and when change is requested, people raise their arms saying "that was not in the specs". Although it doesn't have to be, it has a tendency to it.

Agile

"Let's start and adapt as we go on!"

How it can go well

I think ideal for smaller projects or when the end result is quite blurry, which is actually fairly common in my experience. Moreover, if clients have a prototype they can toy with, it can be very useful for feedback or clarifying the details. It's a very adaptive process, and the progress is more visible.

How it can go wrong

First, it can be a pain to rewrite everything every month because of constant change or whims. Also, often, temporary code becomes permanent and "should be done later" stays forever, leading to technical dept that just piles up as the software grows. IMHO there's a bit too much micro-managament involved in the SCRUM stuff. Instead of empowering the devs, it treats them like robots.

Side story, just as an anecdote. In one project I was in, we had a daily standup meeting with a team of 15 guys. It was simply a silly waste of time, everyone talked about specific details nobody else understood or cared about.

Current workplace

At my current workplace, we are fairly "process-less" and I think it works quite well. It's actually really simple. We meet up once a week in a small team, have a small chat about progress, events, and what should be done next. That's it. Everything else is up to the dev. No time estimates, no schedules, no reporting, no management per se, just: "ok, the next thing we should do is XYZ ...and if you still have time, there's also ABC. Do it as you please.". Usually tasks taking days to weeks. Separately, there are bug reports that should be resolved with priority, but these aren't too many.

---

Conclusion

IMHO strictly following either methodology won't write better software. Having friendly contact, competent co-workers, clear objectives and being heard will do.

my 2 cents

Answer 2

I have worked in successful and non-successful projects, some of which used agile and some of which didn't. Overall, I would say what differentiates satisfying projects from those that aren't is the presence or lack of good project management. So, the successful non-agile projects had regular team meetings (not necessarily every day) and also good project management where tasks were intelligently assigned to those who had the most expertise.

I would categorize "agile" as a buzzword. Everyone is talking about it these days. Yet for very large software development projects (10+ years, 7-70 persons), it makes sense to consider people not as identical but as individuals with differing skills and differing knowledge of various parts of the codebase. Exactly the opposite of what "agile" means. In one company where I worked, we were officially using Scrum, but the process was adapted to the special nature of the large (10+ years, 70 persons) software development project. So not very "agile" after all... And perhaps that's why it worked.

For small projects where all people are competent and there is so little so trivial code that special and differing expertise in some parts of the codebase doesn't develop, I would actually say agile may be the best method. But for long-term large projects, it certainly isn't, unless so heavily modified you cannot anymore call it "agile".

Answer 3

How come my Waterfall projects were largely more satisfying than all those new “agile” projects?

If I had to guess, nostalgia. It is more satisfying to get this big 6 months project done than this stupid little 2 weeks of work - that doesn't mean it was done better (on time, on budget). And of course there's also the problem that most places are only sorta-Agile. They say they're agile, they have the same ceremonies, but have none of the mindset.

For example:

nowadays, everyone around me hates being suffocated by "agile" processes, tiny tasks and massive control machinery around

So change. You are a self-organizing team, right? A cornerstone of many Agile approaches?

Even if you're not, you should definitely abide by "Individuals and interactions over processes and tools". It's literally line #1 of the agile manifesto!

One thing I don't get at all is why more process, and more control, should ever make software development better?

There's situations where it may, but odds are that they won't. Hence manifesto item #1.

I hate to repeat things you've heard, and I wish things were better for you, but your companies are doing it wrong. You are right that it's difficult to do. It's difficult for people to trust one another. Trust that people can make good decisions without process. Trust that if we make choices towards the goal we'll eventually get there. Trust that individual interaction is happening and effective rather than having meetings and oversight and onerous bug tracking micromanagement.

Answer 4

I think 'Agile' works well, but when I say it I mean:

• 5 to a team
• Week long sprints
• Daily Standup : you point at the task you did yesterday and are working on today and the only time you have anything more than "I did this, now im doing this" to say is if its going wrong. Basically its just roll call.
• no retros, no planning, no estimates. Just pull from the backlog and slap new tasks into it when you think of them.
• Deploy/Demo at the end of the week.

It works when you have people who know what they are doing "write me an app/website/business workflow/whatever" and have the necessary tools to do it to hand (language chosen, dev env, production env)

It solves 2 problems

• Big Company problem : We have to do months of requirement gathering and document writing before we can even start programming and it all gets ignored anyway.

• Small Company problem : The devs say they are working but i have no idea on what or when they will be finished or whether it will work at the end of it.

My guesses as to why developers often don't like it are:

• Developers don't see the Big Company problem. We get the well defined specs and write some code that satisfies it. Easy.
• Developers LIKE the small company problem, we get to be all silicone valley startup rock stars! yeee haa!
• Scrum has a billion meetings and meetings are shit.
• Estimation is a hard and thankless job.
• Developers HATE sales people and 'agile coaches'/consultants/scrum masters/Martin 'so called' Fowler if that even is his real name! etc are basically selling agile.