Help With Data Analysis (5)
I'm hoping I can get a few members help me sift through some of my data to determine my next steps.
Background:
I have some data that was collected from an Exoclick campaign in the mobile dating niche. The data was collected over a period of about 7 days. The campaign was initially setup with 7 banners, 1 offer and 3 landers. About 4 days in, I added 2 additional offers to the rotation to see if there were any other offers that would produce a better ROI, since it was in the -60% range at that time. About 5-6 days in I dropped the banners to 6 as one of them was running a consistent 0.15% CTR, whereas the others were 0.19-0.26%.
I've spent the last couple of days looking through Caurmen's guide, which has certainly helped. After reading the guide, I've realized that I was basically throwing stuff against the wall to see what sticks. I can see now that there needs to be more thought put into the campaign than what I did initially.
That said, when looking through my data in iMobiTrax, it looks like there are a couple of variants that would be worth pulling out into a new campaign and beginning another round of split testing, but I wanted to get you guys' thoughts before I begin to dump additional cash into this project.
Possible Winners:
This shot of my stats shows data from one of the three offers measured against one of the three landers. While there have been a few clicks, 418, I don't know if it's really enough to be statistically significant. Given that my overall CTR on this campaign is about 0.2%, that would mean there was ~209K impressions to generate this.
The other thing that's throwing me just a bit is that in this particular view, the LP CTR is showing only just over 3%, whereas the LP CTR for the campaign as a whole is running ~24%. So I'm not sure if this lower CTR is a result of the way iMobiTrax is pulling the data apart when taking a deep dive, and possibly not correct, or if the LP CTR is truly correct for this deep dive view.
At this point, I'm a bit concerned that the data I've collected may not be useful to gauge the next step and is potentially useless due to adding the additional offers.
Any input would be greatly appreciated. I'm not looking for any sugar coated comments, I have a thick skin, so lay it all on the line guys. If I totally screwed up, lemme know and I'll go back to the drawing board. 
The LP CTR for the offer specifically (3%) isn't what you want to look at - if you're judging how your lander is performing as a whole. It's basically saying there was a 3% CTR when it decided to use that one offer in the rotation (meaning all the other clicks on the lander when it was using the other offers are added into the CTR formula, but considered not clicked since it wasn't showing that offer -- and yeah, I don't think I explained it that well but hopefully you get what I mean). I don't really see how that number is helpful - unless you're somehow changing the lander when it shows that particular offer. I think 24% LP CTR is not bad as a whole. Doesn't mean you couldn't improve it though.
And yeah - when you add offers or landers to the same campaign, it makes it very difficult to judge what is working better or worse. It skews the data when looking at the whole lifetime of the campaign. You can see how it's performing after you added them - only by looking from that date forward.
If making additions like that, it might be better to just create another campaign.
Thanks for the info on the CTR. That actually makes perfect sense. As I added this particular offer later on in the test cycle, it would have received considerable fewer impressions versus the initial offer, which would skew the CTR down.
It also makes sense that I would need to be looking at the date ranges from when the new offers were added to get a more apples to apples comparison.
So the other thing I think I picked up from Caurmen which seems to be echoed by Larceny is that when making changes with items such as banners, offers, landers, etc., it would be best to create a new campaign with the traffic source along with a creating new tracking links in the data capture tool (iMobiTrax in my case) to ensure proper correlation of captured data from those changes that were made. Is this correct?
Yes I think so. When you are comparing variables/data - if you really want to see correct data correlations, you should make sure they were all in use during that timeframe. If you make changes or add or take away certain variables within the same timeframe - it really messes up the data.
Not saying I haven't done it myself dozens of times, but when I do - I make sure I basically forget about the days prior to me adding new banners or whatever.. Basically I just look at the date I added them - forward.. which basically is like creating a new campaign.. the data prior to me adding whatever, is useless then.
Home > Questions and Answers > General Questions