Good evening guys,
I'm new here and having some real trouble with statistical significance. So right now I'm testing 4 ads, 2 landing page variations, and 4 offers. The problem is if I do not have enough traffic to each element then I will not get sufficient data to determine which ads/landing pages/offers I should cut and which I should keep.
Obviously, if I only have 50 visitors to my offer then its not enough data to determine what ads to keep/cut. However, if I wait until I have 10,000 visitors to my offer, then I will have more data than necessary to make decisions on what to keep/cut and I'll be spending money on unnecessary data.
I've realized in affiliate marketing there are no clear cut answers because every campaign is different. Also CPM must be taken into account because it doesn't make sense to cut a banner if I'm spending $.10 CPM and have only received 10,000 impressions for a banner ($1 for testing that banner).
At the end of the day, only profitability matters. So how do you guys balance meeting statistically significant data with not overpaying for data?
Yeah, statistical significance is one of the tricky bits all right!
I strongly recommend using a Bayesian split-testing tool and a binomial distribution calculator. See the one linked in my signature below, or have a read of http://stmforum.com/forum/showthread...ats-Calculator and http://stmforum.com/forum/showthread...-Landing-Pages
Actually, the price you're paying for your impressions isn't relevant to how many impressions you need, except in that it affects your minimum required conversion rate for your offer. The simplest thing you can do to reduce the amount you need to spend on testing is to test lower payout offers: lower payouts mean more conversions per dollar for a break-even campaign, and that means more data, faster.
Hope that helps! Let me know if you need more detail.