Prusak.com

Online Since 1996

Archive for the 'Google Website Optimizer' Category

Postings about Google’s new “Website Optimizer” tool. I haven’t found anyone thats posting their experiences with it - so I thought I’d start.

Google Website Optimizer with forms using POST

Posted by ophir on 11th June 2009

One of the limitations of using a standard A/B split test is that it won’t work out of the box if the page you are testing is the “action” page for a form that uses method=POST.

For example, lets say you have a simple form page - form.html that looks something like this:

<form action=’process_form_1.php’ method=’POST’>
Address <input type=text name=’address’><br>
<input type=submit>
</form>
 

You’ve created process_form_2.php to split test against process_form_1.php.

The problem is that when process_form_1.php redirects to process_form_2.php, you will have lost all of your form data since your form uses POST.

There are two solutions for this:

1. Switch from method=POST to method=GET.
If this is an option, it’s probably the easiest solution.

2. Setup the experiment as a multivariate experiment and change the form action line in the HTML.
For example:

<script>utm_section(’form action’)</script>
<form action=’process_form_1.php’ method=’POST’>
</noscript>
 

and then in the experiment setup, add a variation like this:

<form action=’process_form_2.php’ method=’POST’>

Hope this helps.
Ophir

Posted in Google Website Optimizer | No Comments »

How expected improvement affects expertiment duration

Posted by ophir on 10th June 2009

Someone recently asked on the GWO form:

Why would 10% expected improvement take a lot longer than a 20%?
I would think it is the other way around.

The truth is - the smaller the expected improvement, the longer it takes.
Or, in other words, the bigger the difference in conversion between two variations, the shorter time it will take to see the difference.

Here is an analogy that will hopefully provide an intuitive asnwer.

Lets say I want to test two basketball teams against each other, seeing who’s the better team.
Team A currently wins 20% of the time.

Scenario 1
I want to test if team B is a 100% improvement over team A, which would mean team B wins 40% of the time.
The question is, how many games do they need to play to be 95% sure than group B is indeed better. (95% is the confidence level GWO uses).

Without going into the math, if team B indeed is twice as good as team A, I should know fairly quickly.

Big differences are obvious in a short time.

On the other hand …

Scenario 2
I want to test if team B is just 10% better than team A, which would mean team B wins 22% of the time.
How many games do they need to play to be 95% sure than group B is indeed better.

Again, without going into the math, if team B indeed is just 10% better than team A, they’ll need to play quite a lot of games to be 95% certain that they are indeed 10% better.

Small differences take longer to notice than big differences.

Posted in Google Website Optimizer | No Comments »

Happy New Year!

Posted by ophir on 1st January 2009

Just a quick post wishing everyone a Happy New Year.

My first new year’s resolution is to start posting once a week. See you all next week!

- Ophir

Posted in Google Website Optimizer | No Comments »

Google Analytics Tracking in Flash

Posted by ophir on 17th November 2008

Google just announced an official ActionScript 3 library for tracking flash events in Google Analytics.

You can read the whole post here:
http://analytics.blogspot.com/2008/11/want-to-track-adobe-flash-now-you-can.html

I’ve implemented Google Analytics tracking from withing Flash in the past, but it was simply having the Flash call some JavaScript code on the page.

This solution is much more elegant and opens up many new possibilities.

This is also very exciting for Google Website Optimizer users, as it should now be very easy to use an event within a Flash as the conversion event.

Posted in Google Website Optimizer | 1 Comment »

Google Analytics Initial Referrer Update

Posted by ophir on 4th November 2008

I just posted a solution for capturing the initial referrer on landing pages that use JavaScript to redirect to a different page, and already I’ve found another (and maybe better) solution.

I just read a post by ShoreTel who suggests using initData without the trackPageview call. This will indeed set the GA cookies with the correct initial referrer value and not mess up pageview numbers.

On the landing page that does the JavaScript redirect, copy the original Google Analytics tracking code (something like this):

<script type="text/javascript">
var pageTracker = _gat._getTracker("UA-xxxxxx-11");
pageTracker._trackPageview();
</script>
 

and change it to this:

<script type="text/javascript">
var pageTracker = _gat._getTracker("UA-xxxxxx-11");
pageTracker._initData();
</script>
 

Both solutions should work fine!

Posted in Google Website Optimizer | 8 Comments »

Google Analytics Referrer Override

Posted by ophir on 4th November 2008

A couple of weeks ago I posted about a known issue using A/B experiments in Google Website Optimizer.
http://www.prusak.com/google-website-optimizer-and-referer-data/

Basically, when conducting an A/B experiment in Website Optimizer, the original referrer data is lost because of the JavaScript redirect to the test versions. This means that the traffic source will be wrong (it will say direct instead of the true traffic source).

This issue isn’t just a problem with website optimizer. Any landing page that uses JavaScript to redirect will loose the original referrer data. For example if your site provides easy to remember URLs like www.mysite.com/freeshipping and then uses JavaScript to redirect users to a different URL.

I wrote in my previous blog post that I wished Google Analytics provided an easy way to over ride the referrer value (and not use the document.referrer value).

It seems I spoke to soon.

Google Analytics DOES have a way to overwrite the referrer data.

It just doesn’t seem to be documented anywhere :)

I planned on writing my own hack to override the referrer value in Google Analytics. I was doing some reverse engineering of the ga.js code and found this interesting tidbit:

a._setReferrerOverride=function(b){a.yb=b}

I’m still running some tests, but based on my initial results, the _setReferrerOverride function provides a simple way to manually set the referrer value.

So now, we can actually capture the original referrer even when the visitor is redirected. Here’s how:
1 - Store the original referrer data in a cookie named realreferrer right before we redirect
2 - On the test pages, look for the realreferrer cookie. If it exists use it to over ride the referrer value in Google Analytics and then delete the cookie (so it won’t mess up page views that weren’t a result of a redirect from the original version).

Here’s the code I’m currently using in my tests.
Add this on the original page before the redirect (the GWO control script):

<script type="text/javascript">
document.cookie = "realreferrer="+encodeURIComponent(document.referrer)+"; path=/";
</script>
 

On the test page, add this code before any GA or GWO calls:

<script type="text/javascript">
function readCookie(name) {
    var nameEQ = name + "=";
    var ca = document.cookie.split(";");
    for(var i=0;i < ca.length;i++) {
        var c = ca[i];
        while (c.charAt(0)==" ") c = c.substring(1,c.length);
        if (c.indexOf(nameEQ) == 0) return c.substring(nameEQ.length,c.length);
    }
    return "";
}

var realreferrer = decodeURIComponent(readCookie("realreferrer"));
document.cookie = "realreferrer=; path=/";
</script>
 

Then change your GA code from this:

<script type="text/javascript">
var pageTracker = _gat._getTracker("UA-XXXXXX-Y");
pageTracker._trackPageview();
</script>
 

to this:

<script type="text/javascript">
var pageTracker = _gat._getTracker("UA-XXXXXX-Y");
if (realreferrer.length > 0) {
    pageTracker._setReferrerOverride(realreferrer);
}
pageTracker._trackPageview();
</script>
 

Please let me know if this works for you !

Update:
I just found another (and possibly better) solution. See:
http://www.prusak.com/google-ananlytics-initial-referrer-update/

Posted in Google Website Optimizer | 2 Comments »

Does Website Optimizer Append or Merge?

Posted by ophir on 30th October 2008

When running an A/B experiment Google Website Optimizer appends any parameters from the original page when redirecting to one of the test pages.

For example, the original page is index.php and the test page is index2.php.
If a user clicks on index.php?name=fred they will be redirected to index2.php?name=fred

So what happens if the test page URL already has a query parameter in it?

For example, the original page is index.php and the test page URL is index2.php?section=b
What will happen if the user clicks on index.php?section=d ?

About seven months ago I had a client with this exact scenario so I tested it.
The user was directed from index.php?section=d to index2.php?section=b&section=d
I found a work-around and didn’t think about it after that.

A few weeks ago I found out that Google was indeed aware of this “issue” and fixed the GWO code :)
Now GWO will merge the parameters from the two URLs, not just append them. The parameters defined during setup for the test page URL will get precedence over the parameters from the URL the user clicked on.

For example, the original page is index.php and the test page URL is index2.php?section=b
What will now happen if the user clicks on index.php?section=d&cat=22 ?

They’ll be redirected to index.php?section=b&cat=22

This doesn’t effect 99.99% of all A/B tests, but if it does effect you, Google’s fix is a life saver :)

Posted in Google Website Optimizer | 2 Comments »

Google Help Pages Feature Request

Posted by ophir on 24th October 2008

I’ve been doing a lot of reading on the Google Website Optimizer help pages:
http://www.google.com/support/websiteoptimizer/

For the most part, the help pages are kept up to date, but every now and then something falls between the cracks and I get the feeling the page does not reflect the latest updates.

I was just thinking how every Google help page should really have a time stamp, letting readers know when the page was first created and when it was last updated.

Something like:
This page was created on Sep 20th, 2007 and last updated Oct 24th, 2008.

Any thoughts?

Posted in Google Website Optimizer | 2 Comments »

Awesome New Features on Google Analytics

Posted by ophir on 22nd October 2008

Google just announced some awesome new features on the official Google Analytics blog:

http://analytics.blogspot.com/2008/10/more-enterprise-class-features-added-to.html

Most of the features are “being added to all accounts in the coming weeks”.
This includes advanced segmentation, custom reports, motion charts, and a new Account Management Dashboard.

They’re also going to eventually be releasing a data API, but if you really wanted to, you can already grab the Google Analytics data manually.

I’m impressed.

Avinash also has a great posting on the advanced segmentation function.

Posted in Google Website Optimizer, Internet Marketing | No Comments »

Google Website Optimizer and Referer Data

Posted by ophir on 19th October 2008

Google Website Optimizer A/B experiments use JavaScript to redirect the visitor to a test pages.

When a visitor is redirected with JavaScript, you loose the original HTTP referer data (the page that did the redirecting now becomes the HTTP referer)

Here’s an example:
Lets say the original page is page.html and we have a test page page_1.html

Before the test, page.html got 1,000 visitors a day coming from Google searches (organic traffic).

So we have Google -> page.html

Now that the test is running you’ll have 500 visitors to page.html:
Google -> page.html

and 500 to page_1.html
Google -> page.html (JavaScript redirect) -> page_1.html

If the web analytics code is after the redirect on page.html (which is what Google tells you to do) then data based on the HTTP referer will be incorrect (traffic source, keywords, etc).

If you REALLY want to capture the HTTP referer related data you can put the analytics code before the redirect. You’ll now be able to capture the original HTTP referer data, but you’ve also just inflated the page views.

I’ve solved this issue for some in-house code that uses the referer data by simply storing the HTTP referer value in a cookie right before the redirect, and then using the cookie value on the test page.

Unfortunately Google Analytics doesn’t do this.

Here’s a feature request for the Google Analytics team:

Add a way pass in the HTTP refer data manually.

I’m sure Website Optimizer isn’t the only tool using JavaScript redirects from a landing page.

Posted in Google Website Optimizer | 2 Comments »