18

I'm writing a game and the accompanying graphics engine on top of OpenGL in C++. Im also a fan of good coding processes and automated testing. Graphics code + testing seems pretty immiscible, since the output is often visual only, or very heavily visual-oriented.

For example, imagine analyzing the raw image-stream that is rendered to the screen byte-by-byte - you need test-data to compare with, which is hard to create/obtain, and often the rendered images aren't identical on a byte level when running at different times - small changes in algorithms will wreck this approach completely.

I'm thinking of creating a visual unit-test suite, in which I can basically render different test-scenes, showing stuff like shadow mapping, animation, etc etc etc. As part of CI, these scenes would then be rendered to a video file (or possibly leave it as a executable) with different metrics. This would still require manual inspection of the video file, but atleast it would be somewhat automated and standardised.

What do you think? I'm hoping there are better ways?

maple_shaft
  • 26,570
Max
  • 2,039

3 Answers3

8

The opencv image processing library does it by saving the image and comparing it to a reference image - it has a bunch of c++ test functions and macros to handle approximate image matching etc.

Martin Beckett
  • 15,846
4

Your test framework can render its test image into a buffer, retrieve the rendered image, and compare it to a "golden" reference image which has been previously generated for that purpose.

This will not work as well for cases in which the results of your test are not expected to remain exactly the same. However, you can compute the squared difference of the test and reference images, and compare it against a threshold.

You may also want to provide for logging and checking performance data, since a major regression in performance is another possible failure mode.

comingstorm
  • 2,737
4

Even if you can't compare the output images, at least you'll be able to test that your renders complete appropriately (no crashes, long waits, etc, etc). Finding a way to check the images is better, of course, but even without it you'll have gained something from the tests.

Michael Kohne
  • 10,146