How much money do you spend to properly test an angle? (7)
How much money do you spend to properly test an angle?
Hello,
I'm trying to figure out how much money I'd need to spend to properly test an angle.
How do you guys do it?
Do you choose a solid amount of money to just spend on the angle? Like say $25 or $50?
or
Do you spend only how much you need to test each variable in the campaign? Like say you have 5 banners / 2 landers / 2 offer URL's... Would you spend - 5 x $6.50 x 2 x 2 = $130 (testing each banner @ 2x payout offer: $3.25)
Also, how do you decide which angle to keep on running after you're done testing it?
Say the angle test resulted in -50% ROI, would you keep on testing that angle using new banners/landers? Or do you just scrap it and move onto the next angle?
What's the max negative ROI an angle test can have before you decide to get rid of it?
Thanks all for the help.
Found this post by @iAmAttila where he mentions he re-tests angles that have anything lower than -30% ROI.
Is -30% ROI a good indicator of the effectiveness of the angle?
Can I use that percentage as a base line when deciding on which angles to continue testing?
Look forward to any replies.
Thanks all for the help.
Basically, you want to reach statistical significance : http://stmforum.com/forum/showthread...nificance-quot
In reality, you need to adjust your tests to reflect current situation that you are facing. Sometimes you will see that an angle just sucks and there is no point in pushing it further. Sometimes, the differences will be so small that you won't be able to tell which one is the clear winner and you will have to continue for a longer period of time.
It's hard to say how much negative % is too much as it depends on other factors too. For example, the value will be way different when testing a new source VS a source where you already have paused the worst placements. Generally speaking, 50% is considered as something that could be optimized into profit. The more south it goes, the harder it becomes.
It also depends on the data - how many placements it is - imagine that there is one large but really bad spot responsible for 80% of the traffic. The ROI will change dramatically when you cut just that one.
There are sadly no definitive answers for questions like these.
If you want a solid rule to go by, as Matuloo says, you can't use just one.
However, you can develop a "waterfall" approach to testing. It won't cover every situation, but it'll cover most.
So, for example, if I'm looking at a mobile pop campaign, and it was -70% ROI after first run:
First question: assuming this is a traffic source where I don't already know all the bad placements, were there any placements that ate over 20% of the spend? If so, was that placement particularly low-ROI? If so, and if in aggregate they made up more than 50% of the spend, cut them and test again.
Second question: what was the variance in the lander performance? If all the landers performed evenly, move on to the next step. If one or more performed radically better, keep that one, drop the others, test again.
Third question: If neither of the first two questions were relevant, and I was testing multiple offers, did any of those offers significantly outperform the others? If so, drop other offers, test again.
Fourth question: Were there any "pockets of profit" in the campaign? Did a particular device, carrier, browser, placement achieve profit? If so is the volume sufficient to be worth my while? If so set that up as a separate whitelist campaign, test that.
Fifth question: If I haven't tested this offer before, do I have any other offers that would work with this angle? If so, and lander CTR performance wasn't terrible, test again with a different offer.
Sixth question: does none of the above apply? Then the campaign quite possibly sucks. Drop it.
You may notice this is what I use in the Cookbook series.
It's all about eliminating possibilities for what went wrong in a systematic fashion.
Hope that helps!
I would add another suggestion: Test multiple angles at the same time instead of just one.
That way, your chances of hitting on a profitable angle is greatly improved. Plus it's also easier to identify the winning angles when you're comparing multiple angles.
It would also help to create more than just one banner for each angle - by doing it that way, the "winning angle" may just be due to a good image and not the actual angle. Testing 3 banners with different headlines and images for example, would be a better approach. That way you can judge an angle based on the overall performance across the 3 banners.
Amy
You guys are great, thanks so much for your help. 
Much appreciated for the advice.
Home > >