UPDATE (May 28th, 2007): It seems some of my suggestions have already been implemented. I based my wish list on the functionality that was available when I started the beta (Nov 2006). I wrote a followup article with some Google Optimzer tips and tricks. Kudos to Google!

As I said in a previous post, we’ve been using Google Website Optimizer (I’ll call it GWO) for a while, and overall I’m pretty impressed.

Still, there are a few comments and suggestions I have to make it even better:

– Be able to have two experiments with the same test page, but different conversion events.

I’m running an experiment on my homepage with a total of 12 different combinations. Ultimately, it’s all about sales, so I need to run the test until I can see statistically significant results based on sales. Based on experience from previous experiments I’ve run, I can use the conversion event of “reached the product details page” as a rough estimate to see how many sales I’ll eventually have. I can’t use it as my only conversion event, because we’ve seen there isn’t always a direct correlation to sales.

So, Ideally, I’d have two different experiments running with the same test page but two different conversion pages – the product details page and the product sale page. If any of the combinations perform significantly worse than the others, I can find out earlier with the product details page, delete it, and not waste my time testing it.

– Be able to have different conversion URLs for the same experiment Feature “added” 

Lets say I want to measure how the message on my homepage effects overall newsletter signups. My problem is that I have a few different places you can signup for the newsletter and each of them has a different “thank you page” URL.

Another reason I’d need this is to test exit rates. I could put the conversion code on all of the pages linked from my homepage.

– Connect Google Website Optimizer with Google Analytics

This would solve 99% of my problems with not being able to measure stuff. GWO is great at what it does, but it only really looks at a single question – did the conversion even occur or not. What I’d really like is to be able to do segment all of the data I already have with Google Analytics and break it down by experiment combinations from Google Website Optimizer.

The breakdown would need to be added to the “cross segment performance” option. I was thinking add Optimizer as an option and the breakdown would be by experiment / combination. Since both GWO and GA use the same javascript include, I’m thinking it couldn’t be that hard. Worst case, I’ll try to do it myself with visitor segments.

– Keep data on both same session and longer term events.

Currently, Google Website Optimizer only measure events that happened in the same session. While this is a “good enough” indicator, I’d like to be able to get data on conversion events that happened at a later date. Of course this means they need to modify the cookie code, and try to make sure you saw the same version across sessions, but I think Google can handle it.

– Have “multi page” test pages with the same element. Feature “Added”

For example I want to test the wording for the “continue reading” button which appears on 5 different pages. I need the wording to be consistent across the pages.

– Be able to read / write what combination a user gets.

For the most part, Google Website Optimizer is a “closed box”. I have no idea what combo a user was shown nor can I control control what combo is shown to a user.

Actually, I do know that I can append #utmxpreview=X (where X is the combo number) to the end of a test page URL and manually override what version a user is shown. I don’t know if that causes the visit to be in a “test” mode where clicks wouldn’t count.

– Rename the “Original” version

This is a small gripe, but as long as I’m at it …
Each page section variation has a name. The name of the original version is always “original”. Why can’t I change it to something more meaningful like I can for all of the other versions?

– Be able to “delete” experiments.

Of course Google never really deletes anything, but it would be nice if I could only view experiments that aren’t deleted.