UPDATE (May 28th, 2007): It seems some of my suggestions have already been implemented. I based my wish list on the functionality that was available when I started the beta (Nov 2006). I wrote a followup article with some Google Optimzer tips and tricks. Kudos to Google!
As I said in a previous post, we’ve been using Google Website Optimizer (I’ll call it GWO) for a while, and overall I’m pretty impressed.
Still, there are a few comments and suggestions I have to make it even better:
– Be able to have two experiments with the same test page, but different conversion events.
I’m running an experiment on my homepage with a total of 12 different combinations. Ultimately, it’s all about sales, so I need to run the test until I can see statistically significant results based on sales. Based on experience from previous experiments I’ve run, I can use the conversion event of “reached the product details page” as a rough estimate to see how many sales I’ll eventually have. I can’t use it as my only conversion event, because we’ve seen there isn’t always a direct correlation to sales.
So, Ideally, I’d have two different experiments running with the same test page but two different conversion pages – the product details page and the product sale page. If any of the combinations perform significantly worse than the others, I can find out earlier with the product details page, delete it, and not waste my time testing it.
Be able to have different conversion URLs for the same experiment Feature “added”
Lets say I want to measure how the message on my homepage effects overall newsletter signups. My problem is that I have a few different places you can signup for the newsletter and each of them has a different “thank you page” URL.
Another reason I’d need this is to test exit rates. I could put the conversion code on all of the pages linked from my homepage.
– Connect Google Website Optimizer with Google Analytics
This would solve 99% of my problems with not being able to measure stuff. GWO is great at what it does, but it only really looks at a single question – did the conversion even occur or not. What I’d really like is to be able to do segment all of the data I already have with Google Analytics and break it down by experiment combinations from Google Website Optimizer.
– Keep data on both same session and longer term events.
Currently, Google Website Optimizer only measure events that happened in the same session. While this is a “good enough” indicator, I’d like to be able to get data on conversion events that happened at a later date. Of course this means they need to modify the cookie code, and try to make sure you saw the same version across sessions, but I think Google can handle it.
Have “multi page” test pages with the same element. Feature “Added”
For example I want to test the wording for the “continue reading” button which appears on 5 different pages. I need the wording to be consistent across the pages.
– Be able to read / write what combination a user gets.
For the most part, Google Website Optimizer is a “closed box”. I have no idea what combo a user was shown nor can I control control what combo is shown to a user.
Actually, I do know that I can append #utmxpreview=X (where X is the combo number) to the end of a test page URL and manually override what version a user is shown. I don’t know if that causes the visit to be in a “test” mode where clicks wouldn’t count.
– Rename the “Original” version
This is a small gripe, but as long as I’m at it …
Each page section variation has a name. The name of the original version is always “original”. Why can’t I change it to something more meaningful like I can for all of the other versions?
– Be able to “delete” experiments.
Of course Google never really deletes anything, but it would be nice if I could only view experiments that aren’t deleted.