How a disease gave birth to A/B testing
A/B testing is at the center of the start-up ecosystem. Managers don’t want to make decisions unless they have data-backed evidence. That’s a great practice, of course, and fortunately, there are lots of ways to get information without having to rely on one’s intuition. One of the most common methods is A/B testing.
This modern notion dates back to a completely distinct and non-digital age, and it is almost 300 years old. The origins of A/B testing can be traced back to James Lind’s 1753 A Treatise of the Scurvy. Scurvy was the leading cause of disease and death among sailors in the 18th century.
James Lind’s well-planned clinical trials provided supplements to the otherwise identical diets of 6 pairs of men with symptoms of scurvy. The 6 pairs were kept in the same sickbay and given the same care. 4 pairs showed no improvement, while the pair that received a quart of cider each day improved slightly. However, the two sailors that were given 1 lemon and 2 oranges daily made astonishing recoveries.
James Lind’s trial proved that citrus was beneficial against scurvy, whereas other remedies had little effect. Sadly, it took the Royal Navy 40 years to issue citrus juices to seamen. Today we know that scurvy is caused by a lack of vitamin C.
Much like James Lind’s 18th-century search for the cause of scurvy and its remedy, A/B testing is a controlled experiment. You set up two scenarios that are identical, except for one feature. This allows you to test which of the two options works better at catching the attention of your audience and engaging them to take action.
Google’s famous case of 41 shades of blue
Google famously tested 41 different shades of blue to decide which colour to use for advertisement links in Gmail. The company showed each shade of blue to 1% of users. A slightly purple shade of blue maximized the likelihood of users clicking on the advertised links. Some reports suggest that optimisation resulted in 200Mn$ more ad revenue.
In 2009, Doug Bowman, then the company’s top designer, cited it as part of the reason for his departure. “It’s true that a team at Google couldn’t decide between two blues,” he wrote in his goodbye post, “so they’re testing 41 shades between each blue to see which one performs better. I had a recent debate over whether a border should be 3, 4 or 5 pixels wide, and was asked to prove my case. I can’t operate in an environment like that. I’ve grown tired of debating such miniscule design decisions. There are more exciting design problems in this world to tackle.”
