Ever have one of those days where it just seems impossible to debug the code? GWO is not your friend or mine when it comes to debugging.

You can easily set up the experiment to look at your staging server and test like that but you are not going to get too many visits and/or conversions since it is a staging environment with only developers and a few others testing on it. When you launch a project on a site with decent traffic, that is when you can start having some issues. GWO and GA usually work together fairly well but I have run a couple of experiments that jogged the GAs numbers as soon as you implemented GWO.

The main problem I had recently was on a very complex issue, that ended up being a simple fix. Here is the flow of the experiment, an AB experiment testing 2 registration funnels

[image]

Everything was looking good except on side (Control) was getting a few more visits on staging. Not a huge deal with only around 30 visits accrued. We decided to move forward and launch. That is when the discrepancy between the 2 numbers became quite clear. After one day the numbers were
1344/75 (visits/conversions)
Control has 37/127
Variation had 38/47

There was a problem with the tracking script on Funnel B. We realized there was a problem and thus began me trying to figured out what was happening.

The way the experiment was set up was a little hacky, the first page the user hit has the GWO control code and sets the _utmx and _utmxx cookie to determine which version A or B it will display, if it is A it continues to display the first page, then it hit a META redirect to pass the user into funnel A. if the GWO determines the user should see B, the user was passed to that funnel.

I thought that maybe the redirect was taking a hold of the page before it got the response from google. That was not the case.

Funnel B also had ClickTale running on it, so the entire page was wrapped in a ClickTale JS. That was my first warning flag, I thought for sure the scripts were conflicting and not allowing the pageTracker tracking script on funnel B page to fire. Removing CT did nothing.

Back to the drawing board and a lot of testing both funnels using Firebug and Charles to track requests and cookies.
Everything looked good, the AB GWO decision was definitely working correctly. The tracking for the control page Funnel A was working correctly. And conversions for both pages were working as well. It just looked like Funnel B tracking wasn’t working.

But how was the conversions working? Well the cookie set from the control script chooses what page to display A or B, that is a permanent cookie so that users get the same page or variation of a page every time they visit. The conversion goal is determined by the pageTracker call and by determining what cookie the user has set on their browser. So it reads the cookie, determines the user in on Variation X, and then sends the tracker to GWO.

So the last thing I tried (of course it was the solution) was to check out the domains. The tracking script on Funnel B was on a secure server AND a sub-domain. Well you don’t have to look far to see how many times that question is asked in the GWO support. The solution is to modify the control and tracking scripts and add in your domain so it will track all the sub-domains.

Before the Control script
[

<script>
_udn = ".example.com";
</script>


Within the Tracking script

<script type="text/javascript">
var gaJsHost = (("https:" == document.location.protocol) ? "https://ssl." : "http://www.");
document.write(unescape("%3Cscript src='" + gaJsHost + "google-analytics.com/ga.js' type='text/javascript'%3E%3C/script%3E"));
</script>
<script type="text/javascript">
try{
var pageTracker = _gat._getTracker("UA-xxxxxx-x");
pageTracker._setDomainName(".example.com");
pageTracker._trackPageview("/xxxxxxxxxx/yyyy");
} catch(err) {}
</script>

2 hours after I implemented this (about how long GWO needs to collect some data), along with doing some subtractions from the bogus numbers, we started seeing the correct numbers come in.

Of course I should have spotted the sub-domain issue earlier, but all of this did allow me to get my hands dirty and learn a lot about the behind the scenes of the GWO AB test.

~ Snah