12

Here is a quote from a training manual at work regarding SLIM and software estimation:

Notice also, there is a correlation between Effort and Defects. This means, the more people there are assigned to a project of a given size, the more Defects there will be.

Effort is person-time (person-years, person-months) for the project. Defects is the count of defects detected at any point in the lifecycle. Size is defined as the use cases, function points, or SLOC that compose the project.

This seems counterintuitive, assuming a good process and capable engineers. For example, having more people means more eyes on all of the artifacts (requirements specs, designs, code, tests). Aside from having more eyes, my intuition suggests that there is little relationship between effort and defects on a project that utilizes appropriate quality techniques.

I haven't been able to find any documents, aside from those about the Putnam Model (which is used by SLIM), that suggest any kind of known relationship between defects and effort or defects and number of people on a project. Is this a known relationship, and is the assertion that "more people = more defects" valid?

Thomas Owens
  • 85,641
  • 18
  • 207
  • 307

7 Answers7

14

My intuition goes like this:

The more people assigned to a project of given size, the bigger the communication overhead
=> the higher the chances of misunderstandings and all sorts of communication problems
=> the higher the number of resulting defects.

And

Good developers are rarer, thus harder to find and hire, than mediocre / bad ones
=> the more people assigned to a project of given size, the lower their average level of competence
=> the higher the number of resulting defects.

But these may be just my reasoning, I have no supporting evidence.

As a side note, your assumptions emphasized below are IMHO (sadly) quite strong, taking into account the state of our profession:

This seems counterintuitive, assuming a good process and capable engineers. [...] my intuition suggests that there is little relationship between effort and defects on a project that utilizes appropriate quality techniques.

Péter Török
  • 46,547
5

It could just be a correlation. Management tends to assign more people to help on projects they deem will be more complex, or projects that are falling behind due to lots of intransigent bugs, or projects that require a lot of coupling between various components. If you could take management decisions out of the mix, I suspect that correlation would at least diminish, if not reverse.

Karl Bielefeldt
  • 148,830
3

Given the recently updated definitions of size and effort, I would suggest that perhaps the results are due to the fact that Effort (total man-hours) is actually a better estimator of true project size than the measures the source is using (Effort would be a perfect measure if all teams and team's resources were equivalent).

Therefore what is really happening is that larger projects have more defects, which is not surprising at all.

psr
  • 12,866
2

This seems counterintuitive, assuming a good process and capable engineers.

I do not think you can assume either of these in the real world. The more people on a project, the more likely you are to have bad apples who cannot keep up and will slow down the best developers. Even if you go with the assumptions there are a few other things that slow down projects and cause more defects as you increase the number of people:

  • communication overhead
  • code reading overhead (by this I mean that even the best devs take more time to read and consume other people's code than their own)
  • testing must be more thorough (We all shoot for 100% test coverage, but that rarely happens in the real world. The more people you put on a project, the scarier refactoring gets without extremely thorough automated testing, since you are just hoping your changes will not have side effects. Not all tests can even be automated in any reasonable manner, which takes even more time.)

In my experience the negative effects of projects loaded up with developers goes way down when the project is very modular and you have just 1 or 2 people per tier. I don't care what version control system you happen to be using, having 4 developers all auto-merging checkins to the same file at once will probably be a bad idea.

2

There's a difference between correlation and causation; the quote seems to be saying that the total number of defects tends to be higher for projects where larger numbers of engineers are allocated. This makes perfect sense to me, I am sure MS Windows has more defects than the applications that I create, but that doesn't mean that my applications are superior quality.

To give another, more abstract example - if we took the total number of deaths per country and correlated that with the total number of doctors in the country, I am sure we could say "countries with more doctors had more deaths". This would not be because the doctors caused the deaths, but rather that large numbers of doctors are indicative of a large population.

Daniel B
  • 6,204
  • 1
  • 24
  • 30
1

Let's put aside the assertion about the number of people for a moment.

Looking at the number of defects injected as a function of effort might make sense as long as you assume that increased effort necessarily requires increased size, as there is a strong correlation between the number of defects and software size.

So if you assume that the more effort that is put into a project, the more lines of code are written, then you could probably use effort as a proxy for size to predict the number of defects. Correlation between size (e.g. LOC) and the number of defects has been shown time and again in work by Watts Humphrey, Capers Jones, and others.

I don't see how the number of people fits, other than more people implies more effort.

As a side note, don't confuse correlation with causality. While there is a correlation between size and the number of injected defects, size is not the cause. The cause usually comes from, as you've pointed out, people and process issues. That said, defects as a function of size is a great metric for understanding if there is a problem and for gauging the effectiveness of change.

Michael
  • 2,142
0

I'm assuming this is limited to members of the core programming team and not a situation where there are specialists like: UI, UX, DBA, etc.

I think it needs to be managed well, but that's not easy. Small teams made up of quality developers can manage themselves. It's more likely to avoid large sections of code created someone with less talent.

Having more team members can make it easier to divide duties. Put the better or more experienced devlopers on the difficult areas. Take away some of the mundane or non-programming tasks away from your better developers and let the junior devs handle the interuptions. This is going to take more planning and thought, but provides an opportunity to leverage your talent.

There's the notion that it's better to have a great developer who needs to pick up on a new skill than a below-average dev who already knows it. This is great if you have the time. Probably the reason more develpers are being assigned to a project is because of the amount of work required and the time limts. You may have someone who can focus on a specific area and become more skilled. It's great to have that well-rounded knowledge, but sometimes with a little direction, a lesser dev can take some instruction and run with it.

The reality is, there aren't a lot of people who have ever managed a large team on a successful project. Even if they are all talented, it's difficult for large teams to self-manage. Do egos get in the way?

JeffO
  • 36,956