“What is the best page to start optimizing?”
This question comes up quite often in conversion rate optimization.
It’s an excellent question that really deserves a detailed (and lengthy) answer, but today I want to tackle a specific misconception about choosing a page that I’ve seen many people make (and even some books).
The misconception is that “It’s always better to test pages that have more traffic“.
Here’s a simple example with some numbers to explain my point.
Lets say you have an ecommerce site with these stats:
- 10,000 visitors a month (which land on the homepage)
- 3,000 visitors a month who reach your product detail page (the one with the add to cart button)
- 600 visitors a month buy (convert)
This means you have a 6% conversion rate for homepage visitors and a 20% conversion rate for visitors who reach your product detail page.
All things being equal, do you start testing on your homepage or the product page?
Lets run the numbers and see.
Assuming you run a simple test with just two variations (the control/original and a single test version) and that you’ll be able to get a 15% (relative) increase in conversions from your changes, here are the results:
The control group
5,000 Homepage visitors
1,500 product page visitors
300 conversions
Testing the Homepage
Your change to the Homepage causes 15% more visitors to reach the product page which leads to 15% more conversions:
5,000 Homepage visitors
1,500 * 1.15 = 1,725 product page visitors
300 * 1.15 = 345 conversions
Using my simple Conversion Confidence Calculator it’s showing a 93% confidence level (screen shot below).
Testing the product page
Your change to the product page causes 15% more visitors to “add to cart” which leads to 15% more conversions:
5,000 Homepage visitors
1,500 product page visitors
300 * 1.15 = 345 conversions
Using my simple Conversion Confidence Calculator it’s now showing a 95% confidence level (screen shot below).
In terms of statistical significance, I’ll actually get results a bit faster testing the product page which has far less traffic than the homepage.
So, am I getting something wrong or is this a general misconception?
– Ophir
September 10, 2008 at 12:25 pm
Very interesting. So it seems it would be best to first test the product page to increase conversion of 15% on there and once you have done that test the homepage to get more people to the product page increasing the amount of conversions from the product page again.
I’m not a big numbers guy but since the homepage is the page funneling to the product page the number of final conversions are the same so of course the percentage is going to be higher on the page with less visitors and the same amount of conversions.
Not sure I did this right but if you just compare the two pages themselves by running the homepage with the following
version a 5000 visitors 1500 conversions
version b 5000 visitors 1725 conversions
The confidence level was 99%
September 10, 2008 at 1:30 pm
I wouldn’t go as far as saying “it would be best to first test the product page” without knowing more specific data about the pages in question. I purposely oversimplified the picture where in real life there are many additional factors to look at.
The sole point of my posting was to debunk the mindset that it’s always better to start with high traffic pages.
Ophir
May 18, 2009 at 11:22 pm
To amplify your point here is another example where traffic is secondary. As one who does a lot of testing on many websites I would say that the state of the pages can also be a big factor. For example, you may see things on the home page that you can easily change and cause a big difference whereas the product page is in good shape. In that case you may get better results by starting with the home page.