The upper management at our company has laid out a goal for our software team to be “15% more productive” over the next year. Measuring productivity in a software development environment is very subjective, but we are still required to come up with a set of metrics.
What sorts of data can we capture that would measure our team’s productivity?
I try very hard not to write non-answers on this site but I do believe that, in this case, I have to. It's the only right answer. But I'll try to help you out with more than a quip and a "you can't."
In all seriousness, there is no valid measure of developer productivity. I know this is hard for managers to cope with, but it's a fact. Refer them to a few links from people very experienced in the field. For a couple of examples:
So not just is business value hard to measure, there's a time lag too. So maybe you can't measure the productivity of a team until a few years after a release of the software they were building.
I can see why measuring productivity is so seductive. If we could do it we could assess software much more easily and objectively than we can now. But false measures only make things worse. This is somewhere I think we have to admit to our ignorance.
Let's start with plain old productivity. It's rather hard to measure programmer productivity; almost any metric you can come up with (lines of debugged code, function points, number of command-line arguments) is trivial to game, and it's very hard to get concrete data on large projects because it's very rare for two programmers to be told to do the same thing.
Also ask them who is responsible for that increase. What measures are they allowed to take?
It is my experience that managers set these goals because they have zero clue what goals to set with respect to development teams. Maybe you can help them out with that.
Explain to them that you (or the team) want to take your targets seriously, but they have to be SMART or they're meaningless. Suggest to them some targets which are SMART. do you have a build/CI server? If not, setting one up is a SMART goal. If so, do you have some way of displaying quality statistics? If not, setting that up is also a SMART goal.
If so, then you have something that's very measurable: code quality. Maybe bringing your technical-debt rating down is a SMART goal, which will in turn improve productivity, unless they're assuming that people are slacking off, in which case you have an entirely different problem to solve: visibility.
Help them to give you targets you can actually achieve. There's no satisfaction in having goals that cannot be proven or disproven a year from now, or where you'll be wasting time gaming the system rather than improving it.
One of my most productive days was throwing away 1000 lines of code.
— Ken Thompson, designer of the original Unix operating system
Measuring software productivity isn't really that difficult, although it's somewhat imprecise. Survey your programmers and ask what percentage of their working time is wasted on non-productive tasks, and what those are. Have them focus not on personal productivity issues, but on non-productive assigned work tasks. Some examples are:
However, due to an observer effect, it's impossible to tie those survey results to any sort of reward or consequence to the programmer, including having management angry that productivity didn't increase 15%. They will game the system even if they suspect that's how the numbers are used, and you end up with a completely useless metric.
What you can do is use the responses to move productivity in the right direction. Pick the worst two or three and put a lot of effort into it. Improvement can easily be measured numerically for many of the individual complaints, and hopefully when you do the survey again next year, some of the complaints will no longer be listed, or at least be reported by fewer of your programmers.
In other words, you can observe movement in the right direction, even if you can't quantify it.
Instead of trying to focus on metrics for programmers which as others have suggested are incredibly imprecise, try suggesting non-code things like the following:
Explain in detail why each of these things will actually make a programmer more productive.
These are the types of things that can drive programmer productivity, not trying to set some unachievable targets around LOC, or bug count.
I agree with pdr's answer. That's the way to go. However, here's another take.
Most process improvement efforts need to start, as you have, with deciding what to measure and then baselining current operations.
What to use as a baseline?
How about Joel's twelve questions?
If you're not already doing 11 or 12, then you only need to do another 2 and you get your 15% improvement.
In my opinion, for programmers there's only 1 metric that makes sense: the number of developers who leave to do similar work for another company.
If one programmer is unhappy enough to find a job elsewhere, it's very likely that a lot of other programmers are also unhappy and performing badly.
I think it is difficult to do, but possible, if you use good case management software like FogBugz and keep track of old data. Obviously you can't say that we will improve performance by an exact number like 15%, but you can definitely do something about it. So let's say you have:
5 developers
1 month release cycle
These have to be pretty consistent from release to release, otherwise accounting for those can get messy. Here are some things you can measure and improve.
-- Estimates vs actual dev time.
Similar to Agile velocity. If developers are planning to work whole month let's say 20 days or 100 days for 5 devs this month. They estimated all their tasks and told management that they can fit all the features. But it actually took 120 days instead of 100 days.
the 20 days (4 extra days for 5 devs), developers where under a lot of pressure and wrote a lot of spaghetti code.
So to improve performance - increase estimates for next month by 20% and see how much closer it becomes to actual time. Keep updating it for future releases if it is still way off the mark. The performance improvement will not be 20%, but should be substantial, since there is going to be a lot less stress, bugs, and working weekends.
-- Amount of time spent by developers for fixing bugs found by QA
Let's say for this rollout, it took 5 man/days for QA to test.
You can try to improve this number by having less bugs:
Keep measuring bug fixin time every release and you should see improvements if you introduce the things above.
-- Do some research on bottlenecks to development
You may find that there are other things, like communication between Business Side and developers, that are having a huge negative effect on performance. Many times a project can be delayed because of scope creep or feature requirements not clearly communicated. This usually happens over and over again.
This one is a bit hard to measure, but use can use measurements from 1. for this.
-- Type Faster...
Words per minute is a proven way to measure performance not only for developers but also anyone else using computers.
-- Measuring dev performance ain't easy
As other people already mentioned it is not easy thing to do, so how you communicate it to "upper management" is really important. They need to understand that measuring development is not like measuring sales. There is no hard numbers, but there are definitely some things you can improve. So you can give them a plan with some of the improvements and use measures from step 1. and 2. to demonstrate if there is improvement.
