Prusak.com

Online Since 1996

Google Website Optimizer Video Tutorial

Posted by ophir on May 12th, 2008

Google Website Optimizer Tutorial
A few months ago a created a Google Website Optimizer Video Tutorial

I really didn’t do much with it, and since then they’ve slightly updated the interface.

The good thing is that it provides a step-by-step guide explaining exactly how to setup a test, as well as some interesting background information and common mistakes to avoid.

If you find it helpful, please let me know.

Check it out Here.

Ophir

Posted in Google Website Optimizer | 1 Comment »

Conversion Confidence Level Calculator

Posted by ophir on May 7th, 2008

One of the most common questions when setting up an experiment in Google Website Optimizer is how long will the experiment take?

In other words, how long will it take to have “enough” data.

The official Google Website Optimizer docs say:
https://www.google.com/analytics/siteopt/siteopt/help/techoverview.html

If one combination emerges as a clear winner, in the future it will exceed the performance of any of the other combinations with a 95% confidence or better.

But what do you do if you want to estimate the experiment length in advance?

Just use my conversion confidence level calculator

Here is a simple example:

Your site gets 2,000 visitors a week.
If you’re doing a simple A/B split with just two versions (original and test A) then each one gets 1,000 visitors a week.

Your current conversion rate is 2% * 1,000 visitors = 20 conversions a week for the original version.
You are hoping to get a 50% improvement for a 3% conversion rate * 1,000 visitors = 30 conversions a week for the test version.

Here’s what it will look like. The fields in yellow are the fields you enter.

The confidence calculator shows an 84% confidence level.
Not bad, but your boss says she wants at least a 95% confidence before declaring a winner.

You’ll see the confidence level for twice the traffic (two weeks worth of data) is 95%.
This means that if you get the same conversion rates and double the traffic, you’ll have a 95% confidence level.

- Ophir

Posted in Google Website Optimizer | 1 Comment »

Revenue Tracking with Google Website Optimizer

Posted by ophir on February 28th, 2008

I came across this question on the Google Website Optimizer group:

Hi, I run a website where I sell 7 different products each at a different price level.
I am wondering whether there’s a way so that I can track the changes I make in terms of their effect on my overall revenue?

From a marketing perspective, this raises an interesting question.
Will the test version with the best conversion rate always bring the most revenue?

If what you are testing has no relationship to what you are selling, then probably yes.

On the other hand, lets say you want to test what product you should be highlighting.
In such a case, measuring overall revenue makes sense.

So here is how I would do it…

Read the rest of this entry »

Posted in Google Website Optimizer | 2 Comments »

Get Total Control of Google Website Optimizer

Posted by ophir on February 6th, 2008

One of the limitations of Google Website Optimizer is that you have no control over which combination will be displayed to a user.

Wouldn’t it be great to have total control over which variation is displayed in each section?

For example, lets say you want to test a new navigation scheme that required testing of the page header and footer, but they had to be in sync with each other. In other words, header #1 must be displayed with footer#1, header #2 must be displayed with footer#2 and so forth.

This “hack” gives you the ability to read and write which variation is displayed in each section.

Read the rest of this entry »

Posted in Google Website Optimizer | 3 Comments »

Good Online Marketing Books

Posted by ophir on October 26th, 2007

Here’s a short list of some of the online marketing books I’ve read over the past few years (in no particular order).

Influence: The Psychology of Persuasion
You need to read this book. Even if your not in marketing, this is one of the better books I have ever read that explains the why and not just the what of human behaviour, specifically, in terms persuading someone to do something.

Waiting for Your Cat to Bark?: Persuading Customers When They Ignore Marketing
The Eisenberg brothers do a great job explaining why you need to look at your customers in multiple dimensions. Implementing their suggestions isn’t something is easy, but for anyone who’s reached the “lowest common denominator wall”, this is your best way to improve your site.

Read the rest of this entry »

Posted in Internet Marketing | No Comments »

Google Optimizer A/B Test With a Private Conversion Page

Posted by ophir on October 9th, 2007

I just received an email with this question:

I’m trying to set up an GWO A/B test, but I’m having a problem
completing the set-up of the test because the confirmation page
requires a login.  It’s not a public page because there’s sensitive
order info on there.   So we’ve got the CWO code posted on the
confirmation page but GWO can’t validate it because it’s not
a public page.   

However, when a real order comes through, the Javascript will
fire and it should work okay.
Any idea how I can complete the set-up
of the test without going through successful validation?  Right now I’m
stuck because I can’t hit the continue button in GWO to get past the
“Validate pages” step.

Jonathan

So I checked and indeed Jonathan is right.
You can’t continue without a “live” conversion page that is publicly accessible.

The solution
Create a dummy conversion page that is publicly accessible. Enter the dummy page URL during the experiment setup and simply copy the conversion page javascript code to both the dummy page and the real page.

Once an experiment is live, Google Website Optimizer doesn’t care what the URL of the conversion page is. The conversion event will be triggered when the javascript code is executed, regardless of the page URL (if it’s not on the same domain - you need to add some code).

When creating a Multivariate experiment, this is a non-issue because you can upload the conversion page from your local file-system, bypassing the need for the file to be publicly accessible.

Hope that helps
- Ophir

Posted in Google Website Optimizer | 3 Comments »

Why you NEED to be using Google Analytics with Google Website Optimzer

Posted by ophir on September 7th, 2007

I’ve written in past posts about how to get all of the great GA data for each of your combinations in GWO.

A couple of months ago I ran an experiment that showed me why doing so isn’t just a “nice thing to have”, but a necessity.

I was comparing two significantly different versions of a promotion page (whole page split). You can see them here:
Old (control) Version
New Version

After a couple of weeks of testing, GWO was reporting that the new version was performing 7.03% worse than the old version. I was a bit surprised because of the radical difference between the two pages.

At that point I started looking at numbers GA provided as well. At first I didn’t see anything out of the ordinary. Then I took a look at the page performance segmented by visitor type (New / Return visitors) and my eyes opened wide.

For new visitors, the new version of the page was performing much better (over 40% improved conversion rate) and for return visitors the old version of the page was performing much better (over 20% better conversion rate).

Going forward, we now show new users the new version of the page and return users the old version. I have a few theories on why we’re seeing this behavior, but that’s a bit beyond the scope of this post. I’d love to hear your ideas in the comments.

What this shows us is that while GWO is an excellent tool, it can be a bit blunt when used on it’s own. Very often the most valuable information is in the details.

- Ophir

Posted in Google Website Optimizer | 4 Comments »

Improved Whole Page Testing with Google Website Optimizer

Posted by ophir on July 5th, 2007

UPDATE: (Sep 6, 2007) Google just redid the entire A/B split testing process. See:
http://adwords.google.com/support/bin/answer.py?answer=71362

While GWO’s was originally created to test specific elements within a page, sometimes you want to test totally different versions of a page.

Lets say you want to test three totally different versions of page1.html. You create three new pages: page1a.html, page1b.html and page1c.html. You and then add some custom code to page1.html which redirects visitors to one of the new pages.

GWO added this feature a while back. You can read the official page at Comparing entire pages (A/B testing)

Recently, one of the posts on the Official GWO Forum mentioned a shortcoming of Google’s solution - the fact that it doesn’t carry forward the query string.

The Problem:
Lets say the URL of your main test page is landing.html and one of your test pages is landing_1.html.

If a user is sent to landing.html?code=123 they will be forwarded to landing_1.html and not landing_1.html?code=123 which is what you probably wanted.

The Solution:
This is easier then you might think.
Look for this line in Google’s original code for your test page:

document.location = test_url;

and change it to:

document.location = test_url + self.location.search;

That’s it!

Posted in Google Website Optimizer | 2 Comments »

Google Website Optimizer and Analytics Bugfix

Posted by ophir on June 13th, 2007

A few weeks ago Shawn Purtell from ROI Revolution wrote an excellent article on combining GWO with GA. If you haven’t read his article - please do so now. This posting describes a bug fix for his code, but you really should first read his article to fully understand how to use it.

I just found a bug in his code and am providing a fix for it. Under some circumstances, his code will provide the wrong combination number for the combination that is being displayed. Different combinations can be given the same combination number.

Read the rest of this entry »

Posted in Google Website Optimizer | 3 Comments »

New Version of Google Analytics!

Posted by ophir on May 8th, 2007

 Google just announced a new and improved version of their analytics software:

http://analytics.blogspot.com/2007/05/new-version-of-google-analytics.html

- Ophir

Posted in Google Website Optimizer | No Comments »