“Designing an experiment is like gambling with the devil: Only a random strategy can defeat all his betting systems.” (R. A. Fisher)
These days just about everyone does some form of A/B testing to optimize pages. But as Ronald Fisher knew, A/B testing is loaded with traps, and the only way to avoid them is through careful use of randomization and statistics.
There are plenty of free tools out there that make A/B testing easy and accessible, but not all tools are created equal. Google’s Website Optimizer is one of the most complete and polished, designed to gather events directly from your site and display results in a nice report. GWO is a great way to get started with A/B testing and is how we ran our first few tests at Thumbtack.
Pretty soon, however, we started to outgrow to limited event gathering and reporting features of GWO. As data-driven decision making runs strong in our DNA, we decided to develop our own event tracking and reporting system. One component of this system was an A/B test report inspired by GWO.
So give Abba a spin, check out the source, and let us know what you think! The demo page has a full FAQ where you can find plenty of details about how to interpret the report and how everything works under the hood.