I'm looking to run some higher payouts offers on mobile traffic (about $45 / sale).
What I'm thinking of doing, is run 4 campaigns for wifi+app, wifi+site, mobile+app, mobile+site with 16 banners each.
Just curious how I should optimize / test, when does statistical significance occur for conversions that are higher in value? Should I run 1-2x payout (lets say $50-$100 per banner) before cutting anything?
And should I determine profitable campaigns based on each campaign + creative, or should I look into further more detailed tracking options?
Haven't done much mobile before, but figure its always a good time to try something new.
If you have minimal mobile experience, I would highly recommend steering clear of such a high payout offer to start - simply because you will not easily come to grips with the many facets of optimisation required in mobile if you have few conversions.
The words "statistical significance" are often misused - if only one could go back in time and change the wording choice made here.
To quote mathematician Jordan Ellenberg:
"Statistically significant" is one of those phrases scientists would love to have a chance to take back and rename. "Significant" suggests importance; but the test of statistical significance, developed by the British statistician R.A. Fisher, doesn't measure the importance or size of an effect; only whether we are able to distinguish it, using our keenest statistical tools, from zero. "Statistically noticeable" or "Statistically discernible" would be much better.
Statistical significance doesn't 'occur', rather if you compare to tests e.g. impressions to conversions for a specific banner versus another, you can compute if the difference in the data is large enough to justify some decision.
If something has 10x the payout, all else being equal, you will have to spend the exact same amount to get to a similar level of certainty in your data comparisons - because payout isn't often factored into the calculations you do.
However, it's more likely that the 10x higher payout will also bring a 10x lower conversion rate. So, you would spend 10x as much to get to the same number of conversions. In this situation, you have a larger number of trials (clicks or imps), lower CVR and identical number of conversions.
When this happens, the level of certainty in your comparisons decreases.
For example, say you have 100/10 vs 100/15 click/conversion data for two banners. Compare to 1000/10 and 1000/15 from a different campaign.
If you compare these two data sets, the confidence that the 2nd beats the first is slightly lower for the example with 1000 trials, and the predicted conversion rate has a slightly larger range.
I know this is dragging on but it's important to understand that 'statistical significance' is not something achieved by spending 2x the offer payout on a banner, and thinking like this with a $45 payout offer will likely lead to you dumping half a grand on a poorly planned approach.
TL;DR: learn mobile with low payout offers that have higher conversion rates and let you learn the caveats, nuances and mechanics of the traffic sources and advertising landscape in general, before trying to push high ticket offers.
Thanks for taking the time to write that long reply!
I do understand what you mean, but I'm looking to get more traffic for specific offers and figured mobile is a huge market that I can tap into. I have a mobile optimized funnel setup, and actually ran some traffic earlier. Wasn't much, but spent just under $400 at about -50% ROI. I stopped it though as I didn't have my tracking setup correctly.
But anyways, obviously I'm not profitable, but considering I did absolutely 0 optimization I believe I may have a good starting point?
I may be a noob to mobile but not to IM, and I'm willing to pay the price for data...unless my approach is still flawed. Once I get to about 50 conversions, which at -50% ROI would cost me around $4-5k, would I not have at least some decent data to cut extreme under performers, focus on the ones getting the most sales, and close that gap while trying to take it to profitability?
I understand.
However, even if you are experienced with IM and have the budget, are willing to pay and so on, I still would recommend that you invest some funds running other, lower payout campaigns simply to learn how the platform(s) work.
There are so many variables that come with mobile traffic - specific carriers, specific placements, app vs site placement, wifi vs carrier traffic, OS and models, exchanges used and of course the traffic source used.
That being the case, it would be worthwile to just throw a few hundred $$ at something like a $1 app install, just for the optimisation practice, and to get hands on experience with the way the platform works.
My intentions aren't to imply that you won't be able to get the $45 payout offer profitable, or that it would be silly to push mobile traffic to it, but rather that it would not be sensible to do so as a babe in the woods in mobile.
If you're happy spending 4-5k at -50% ROI just to get data, then spending just 10% of that running a few case studies/test runs with app installs is a no-brainer IMO!
^^ And that's why Zeno has a PhD. Well, that and a hell of a lot of hard work.
I could not agree more with his points - all of them!
In particular, definitely start out by learning mobile with a low-cost offer. It's a much, much cheaper way to get over the initial humps and hurdles - no sense in spending $180 or so just to learn you placed a targeting parameter a bit wrong...