If you ask several software test leads or managers “What makes a great tester?” Each will most likely give you many varied answers but the following will be common among all:
Communication skills; In particular the ability to clearly and accurately describe any and all steps leading up to defective behavior in the software.
However, no matter how good your observation or note taking skills are it is impossible to observe and note everything.
What if you had a fairly easy, inexpensive and non-intrusive method that would allow all your testers to inherit the benefits of the very best observational and note taking skills? Well, there is such a thing. We call it the system test “black box”.
Using off the shelf technology, we have put together a set up that records all user activity leading up to a defect or crash/hang in the software. Being able to replay user interaction allows you to notice things you didn’t spot the first time, as well as freeing up more of your capacity as you’re not distracted by trying to take notes on your actions. (Humans have limitations, both in attention and memory – let machines do what they’re best at and free your testers to exercise their creativity and analysis skills.)
Let me itemise the easy, inexpensive and non-intrusive aspects of this method:
Easy: about 15 minutes one time setup
Inexpensive: about $75 per test station
Non-intrusive: No additional software or drivers running on the test station to interfere with the software being tested.
Throughout my many years in the software-testing field, one of the more frustrating experiences testers come across is a hanging or crashing bug that they are unable to reproduce. Many times this is dismissed as intermittent. Unable to reproduce the bug, the tester moves on and the opportunity to trap and fix the problem is lost. This opportunity does not have to be lost. You can easily capture and review the last few minutes of activity prior to a defective event using a miniature security DVR connected to your PCs VGA output via a video interface box.
 An inexpensive small continuous recoding security DVR: Typical cost is approximately $50. This device captures an RS170 video signal continuously over a period of time onto a micro SD card. The duration of video recording is determined by the size of the SD card. Typically 15 to 30 minutes per Gigabyte. Note this device can be set up to record continuously 24/7, thus if you are using say a 2GB SD card that holds 1 hour of video, when the recording is stopped you will always have the last hour of activity for review.
The DVR shown [right - click for a better view] is representative of such a device. It is approximately 3” x 3” x 1” and weighs only 3.5 oz. This particular device will operate in continuous capture mode and accommodates up to a 32 Gigabyte micro SD card that translates to several hours of video.
 The DVR accepts an RS170 video input signal but the output of your PC to your monitor is typically VGA. No problem, you can purchase an inexpensive converter (about $15) that will split your VGA output signal from your PC into two signals. One VGA output to the monitor and a second RS170 video output to the DVR. The split signals are available simultaneously.
 You will also need a micro SD card for the DVR. Depending on the size you choose, this will range in cost from about $5 to $25.
The basic interconnection [right] of the DVR and interface box is quite simple:
The VGA video from the PC is fed to the video converter box which converts the VGA signal to RS170. The RS170 signal is then input to the DVR. Thus everything that is displayed on the PC monitor is being recorded by the DVR on to an SD card. In addition to converting the VGA signal to RS170, the video converter box also passes the original VGA signal thru to an output that can be connected directly to the monitor. Thus you have essentially tapped into and are recording the PCs monitor output. And this is being accomplished without having to run any recording software on the PC or adding any additional load to the CPU.
There is one limitation worth noting. RS170 video is not known for it’s high resolution/video quality. If your monitor is set for a resolution much higher than 1024×768, the resulting recorded video may be of inadequate quality to be viewed comfortably. We have found that setting the monitor resolution to 1024×768 is a good compromise between reasonable monitor resolution and adequate recorded video quality.
The DVR and video interface box are small and light and can be tucked away and forgotten until a crash/hang or defect occurs. At that time the recording can be stopped and reviewed to determine the user interaction leading up to the event.
The size of the SD card determines the duration of video that is recorded. You may want to use a smaller SD card to loop on recording just a few minutes of video. On the other hand, you may choose a larger SD card to record several hours of video. We have found a smaller SD card useful for interactive testing while a larger SD card has proven more useful for extended automated testing. For example, if the software crashes/hangs during the night, you can extract data from the video recording as to what time the crash occurred and what functionality was being exercised at the time of the crash/hang.
When a crash occurs it’s always good to have a “black box” recording of the events to help determine root cause. An example recording can be viewed below. It’s a bit long, but watch out for the crash at 3:32:39 (by the timestamp).
About the Author
Ian Adams is a Senior System Test Engineer at Microscan Systems Northeast Technology Center based in Nashua NH. Microscan is a global leader in machine vision and barcode verification tools headquartered in Renton, Washington. Ian has over 30 years of software and hardware test experience in the automation and industrial system control fields. His experience is currently focused on machine vision and previously focused on industrial robotics. Ian holds a BS degree in Engineering from Wentworth Institute of Technology. He is also a licensed master electrician.