What is Multivariate Testing?
Life Before Website Testing
Before Website Testing, we had the Website Dark Ages. The Dark Ages went like this:
- We just rolled out cool ideas onto our sites.
- We just rolled out cool ideas that our bosses liked.
- We measured, changed the page, and measured again. Before and after, you might say.
Now we have Website Testing
“Before and after” isn’t all bad. But it doesn’t take into account changes in the season, the competition, and the economy. Today, we have tools like Google Website Optimizer (and others) that can test our pages scientifically.
These tests come in two flavors: Split testing, and multivariate. So, what is multivariate testing?
Let’s say that you want to test a page of your site. You are considering two headlines, a new picture, three buttons, and a new block of text. That means that you have 48 variations (counting the original text/copy/button/headline). Now we will see the beauty of multivariate testing. All you need to do is create the three new buttons, the alternative picture, the one new block of text and the two alternative headlines. Then, work with a software package like Website Optimizer.
The software serves up variations of the tests each time. One person may get Headline 2 with the original picture and Button 3 and the original text. All the combinations get tested, and after you have enough data, the software will tell you which title works best with which picture and which test and which button. “Works” will have been defined by you at the beginning of the test. Often, it is defined as “which combination increases my conversion rate the most?” (The software packages give you a place to define success.)
