Home > Questions and Answers > General Questions

Testing Budget (10)

11-21-2013 04:57 AM #1 freeflowexp (Member)
Testing Budget

Hello all,

Just wanting to confirm that when people state 3 to 5 times payout (or whichever number it is) that they are referring to a single day's budget to acquire statistical significance.

If there is very little or no conversion then the offer is dumped. However, if there is some promise in the conversion then you begin the next split test on the same spend limit (3 to 5 times payout or whichever it is).

Regards,

Freeflowexp

11-21-2013 03:01 PM #2 caurmen (Administrator)

It's a somewhat imprecise rule, but yes. Doesn't have to be over a single day - just a total of spend that's equal to 3-5x the campaign payout.

No conversions? Kill it. Some conversions? Keep testing, and break out the statistical significance calculators.

11-21-2013 08:15 PM #3 freeflowexp (Member)

Great thanks for that Caurmen.

And just one more follow-up query: Alex made a suggestion with mobile that if you aren't getting a single conversion that you check to see if there is one or two sites which are giving an outright majority of your clicks, that you block those sources and retry the test.

Would it therefore be okay to split your, say, 5x payout budget, between an initial test 50% budget test to weed out these budget suckers then dedicate the remaining 50% to a test without the budget suckers?

I'm guessing it's probably best just to run an entire test through to 5x payout (or whatever your statistical significance) then a further test without the budget suckers given that it's an imprecise rule anyway and mobile would usually be expected to require a larger budget.

11-22-2013 04:52 PM #4 caurmen (Administrator)

I'm afraid you're right. Splitting the budget just gives you bad data

11-22-2013 07:43 PM #5 freeflowexp (Member)

Thanks for the confirmation.

Nothing to be disappointed about as I always knew mobile would require a higher budget.

11-22-2013 09:46 PM #6 stackman (Administrator)

Quote Originally Posted by caurmen View Post
It's a somewhat imprecise rule, but yes. Doesn't have to be over a single day - just a total of spend that's equal to 3-5x the campaign payout.

No conversions? Kill it. Some conversions? Keep testing, and break out the statistical significance calculators.
This is spot on!

The only time it's different is when you're running 30 banners, 4 landings pages and rotating 4 offers. Then you'll need to spend a lot more then 5x the payout of the offer. With so many variables you'll want to spend 10-20x. This way you have enough stats to figure out the best combos and build on that.

11-23-2013 11:13 AM #7 caurmen (Administrator)

Yeah, that's a very good point! The more variables, the more spend you need.

11-23-2013 07:29 PM #8 freeflowexp (Member)

Great! Thanks for the info guys!

11-23-2013 09:06 PM #9 kaiser (Member)

stupid question time;

No conversions? Kill it.
Kill what ??

.. this might be obvious to some .. but one of my bigger mental hangups, my choices here are

kill:

placement
banners
offer
landing page

??

11-25-2013 10:33 AM #10 caurmen (Administrator)

In this case, I'm referring to the offer. However, that assumes you're split-testing a couple of landers and a couple of ads, and there's nothing obviously wrong with the CTR you're getting from either of those.

If you're just running one lander, I'd split-test another very different lander before killing the offer. And if your CTR's 0.0001%, obviously you can't blame the offer for not converting just yet

Home > Questions and Answers > General Questions