Is it good that testers are competing to see who opens more bugs?

Question

I'm a software developer. There is a team of testers who follow and run test cases written by the analyst, but also perform exploratory testing. It seems like the testers have been competing to see who opens more bugs, and I've noticed that the quality of bug reports has decreased. Instead of testing functionality and reporting bugs related to the operation of the software, the testers have been submitting bugs about screen enhancements, usability, or stupid bugs.

Is this good for the project? If not, how can I (as a software developer), try to change the thinking and attitudes of the team of testers?

Another problem is because the deadline is estimated and cannot change, so as the deadline nears, the testers will be scrambling to finish their test cases, and this will cause the quality of the tests decrease. This will cause legitimate bugs to be in the final product received by the client.

OBS: This competition is not a practice of the company! It is a competition between only the testers organized by them, and without any prizes.

Answer 1

I do not think it's good that they make a contest out of finding the most bugs. While it is true that their job is to find bugs, their job is not "find the most bugs". Their goal isn't to find the most, their goal is to help improve the quality of the software. Rewarding them for finding more bugs is about the same as rewarding a programmer for writing the most lines of code, rather than the highest quality code.

Turning it into a game gives them an incentive to focus on finding many shallow bugs, rather than finding the most critical bugs. As you mention in your edit, this is exactly what is happening in your organization.

One could argue that any bug they find is fair game, and that all bugs need to be discovered. However, given that your team likely has limited resources, would you rather have a tester focus several hours or days probing deeply into your system trying to find really big bugs, or spend several hours or days skipping through the app looking for typographical errors and small errors in the alignment of objects on a page?

If the company really wants to make a game out of it, give the developers the power to add points to a bug. "stupid bugs" get negative points, hard to find bugs with well written reports get multiple points. This then moves the incentive from "find the most" to "be the best at doing your job". However, this isn't recommended either, because a programmer and QA analyst could work together to artificially pad their numbers.

Bottom line: don't make a game out of finding bugs. Find ways in your organization to reward good work and leave it at that. Gamification rewards people for reaching a goal. You don't want a QA analyst to have the goal of "find the most bugs", you want their goal to be "improve the quality of the software". Those two goals are not the same.

Answer 2

I am going to disagree a bit with the other answers. "Finding bugs" for a tester is a bit like "writing code" is for a developer. The raw amount is meaningless. The job of the tester is to find as many of the bugs that exist that they can, not to find the most bugs. If tester A finds 5 of the 10 bugs in a high quality component and tester B finds 58 of the 263 bugs in a low quality component, then tester A is the better tester.

You want developers writing the minimum amount of code to solve a particular problem, and you want a tester writing up the minimum number of reports that correctly describe broken behavior. Competing to find the most defects is like competing to write the most lines of code. It is far too easy to slip into gaming the system to be useful.

If you want to have testers to compete, it should be more directly based on what they are there to do, which is to validate that the software works as described. So perhaps have people compete to see who can write the most accepted test cases, or even better, write the the set of test cases that cover the most code.

The better measure of developer productivity is the number of tasks complete times task complexity. The better measure of tester productivity is the number of test cases executed times test case complexity. You want to maximize that, not bugs found.

Answer 3

Based on my personal experiences, this is not a good thing. It almost always leads to developers filing bugs that are duplicates, ridiculous, or completely invalid. You'll typically see a lot of these appearing suddenly at the end of a month/quarter as testers rush to meet quotas. About the only thing worse than this is when you also penalize developers based on the number of bugs found in their code. Your test and development teams are working against each other at that point, and one can't succeed without making the other look bad.

You need to keep your focus on the user here. A user has no idea how many bugs were filed during testing, all they see is the one that got through. Users ultimately don't care if you file 20 bug reports or 20,000, as long as the software works when they get it. A better metric for evaluating testers would be the number of bugs that were reported by users but that should have reasonably been caught by testers.

This is a lot harder to keep track of, though. It's fairly easy to run a database query to see how many bug reports were filed by a specific person, which I suspect is the main reason why the "bugs filed" metric is used by so many people.

Answer 4

There is nothing wrong with making a game out of finding bugs. You have found a way to motivate people. This is good. It's also revealed a failure to communicate priorities. Ending the contest would be a waste. You need to correct the priorities.

Few real games have a simple scoring system. Why should the bug hunt?

Rather than score the game simply by number of bugs you need to provide a measure of bug report quality. Then the contest is less about number of bugs. It'll be more like a fishing contest. Everyone will be looking to find the big bug that'll get a high priority score. Make the quality of the bug report part of the score. Have the developers provide the testers with feedback on the quality of the bug report.

Fine tuning game balance is not a simple task so be prepared to spend some time getting this right. It should communicate your goals clearly and it should be fun. It'll also be something you can adjust as business needs change.

Answer 5

Finding bugs is their job. As long as they aren't making things less efficient (for instance, by opening a bug for ech of 10 typos instead of one to cover several of them) this is encouraging them to do exactly what they're supposed to be doing, so I can't see much of a downside.

Answer 6

This is an expansion on @CandiedOrange's answer.

To get started on shifting the attention to more useful objectives, consider something very informal and unofficial. For example, the developers could buy some small tokens and trophies.

Each day that at least one significant bug was reported, leave a "Bug of the Day" token on the tester's desk. Once a week, hold a ceremony with a procession of developers delivering a bigger and better "Bug of the Week" token or trophy. Make the "Bug of the Month" trophy delivery even more dramatic, perhaps with cake. Each token or trophy should be accompanied by a citation saying why the developers thought it was a good thing that bug was found in testing. Copies of the citations should be put somewhere where the testers can all read them.

The hope is that the testers would switch their attention from finding the most bugs to collecting the most trophies and tokens. Their best strategy for doing that would be to read the citations and think about what approaches to testing are likely to bring out bugs the developers will consider important.

Simply ignore unimportant bug reports. Since it would all be very unofficial and informal, it could be shut down or changed at any time.

Answer 7

Is this good for the project?

No. You have pointed out, yourself, that you have observed that it results in low-quality reports that are not targeted at required functionality, and that the testers end up, to compound the problem, scrambling to complete the work that they are actually "supposed" to be doing.

If not, how can I (as a software developer), try to change the thinking and attitudes of the team of testers?

Raise the issue with your Project Manager. They should consider this sort of thing to be part of their job. If your PM is unwilling or incapable of dealing with it, you are sort of stuck developing your own coping strategies. (which would be a different question)

Answer 8

I think how it will be(or how it already is) if it goes on like this, you wont necessarily get lower quality. Al though I think it will decrease the quantity to quality ratio. It depends if this is a bad thing or not. It depends if

reporting bugs about screen enhancements, usability, or stupid bugs.

is something you really dont want. If this is clear with the testers, I would just tell them not to do the things you dont want reported but be clear about it. Do it when one of those reports show up again.

The reason they have a competition is probably to have fun while working, so they are probably not intending to do bad work(if this is considered bad).