Home > Paid Traffic Sources > Mobile

---

Optimizing Mobile Pops (11)

---

Hey guys,

A few questions here on optimizing mobile pops.

1. With ZeroPark, specifically - is there a way to select specific carriers? I didn't notice that there was.
2. Anyone have a good solid approach to optimizing a pop/ppv campaign? i.e. How many pops before you cut a placement/carrier.. (realize there's a lot of variables but do you have a general guideline you work from..)
3. How do you decide on when to cut the campaign all together, how many pops/spend? (Are you using the 15X payout rule from mobile cookbook) to give up on the campaign..

I realize there might not be a perfect answer, just looking for open thoughts, nobody's writing a thesis on this. I've been looking at costs and cutting placements once they hit my payout but it's still expensive given how many placements there are and slow..

Thanks!

---

I'll take a stab at those!

1)Here's what zeropark said in their email from yesterday

2)The general rule of thumb is to cut a placement when it hasn't converted after spending 2-3 times the payout. For placements that ARE converting but at a negative ROI, it can be difficult to decide when/whether to cut them. This is where statistical math comes in. What you basically need to find out, is how likely the placement will become profitable for you in the long run (based on data collected up to the present). This thread would be a good place to start:

http://stmforum.com/forum/showthread...l+significance

I've also programmed an excel spreadsheet to do the same calculations:

https://www.dropbox.com/sh/yy4opi9q1..._r58AEPra?dl=0

Everything should be self-explanatory (feel free to PM me if it isn't). Simply paste/specify all the input data, and copy the formulas for as many rows as you have data.

IMHO, setting the confidence level to 80% would be plenty. If you set it higher you'd reduce the risk of cutting good placements, but the trade-off is you'd need to spend more before you can kill a placement - not recommended if there are lots and lots of placements *ahem zeropark ahem*. On the other hand, decreasing this value will allow you to cull placements faster and cheaper at the risk of cutting out good placements. You'll need to decide for yourself how to find that balance.

For "Min. ROI" you can put 0% if it's early in your campaign optimization and you feel that after optimizing other aspects of your campaign, you can probably take break-even placements to positive ROI (e.g. after optimizing your lander, cutting carriers/devices, doing dayparting....). For a fairly optimized campaign you'll want to set Min. ROI to something positive like 30%+.

3)As always, everybody has different criteria on how to do this. It would also depend on what you'd consider to be a single "campaign". Is it one lander and one offer? Multiple landers and one offer? Multiple landers and multiple offers?

For me personally, I don't set up a campaign and decide whether to cut it or not based on ROI alone. I pick a vertical that I KNOW is making bank for a lot of people, ask my AMs for 1-3 best offers in the vertical, and just use those to test landers up the yin-yang (after the initial day or two I'd just use the best-converting offer to continue testing, pausing the rest, to make my testing as cheap as possible). Every lander I set up to test is for a reason. For example I'd set up 2 landers that are same in every way except one has audio and the other one doesn't. Or 2 same landers, one with entry popup and the other one without. This kind of testing will give you valuable insight on what elements work, and creates a more efficient optimization strategy than killing a campaign based on initial ROI alone.

Once you have a few good landers, you can test a variety of offers in that vertical, for different geos.

I completely feel you there! It's hard to cull placements when you're getting a couple of impressions from each of thousands. Try to set up a "Target" campaign and whitelist placements that have the most traffic (or at least enough of it to be worthy of testing!) Alternatively, set up a RON campaign and keep pausing placements that are sending say under 5/10/100 views per day (your preference).

Good luck!


Amy

---

@vortex, nice calculator.

Is not my area of expertise to give advice about which offer to choose, but I know about the landers. I assumed, but I had no idea how many affiliates make the decision to declare the lander a winner/loser just by "gut" and not by any statistical relevance. I've seen this mistake so many times on my tool that I need to add a feature to guide them to take better decisions.

I'm curious how many affiliates from STM use statistical significance calculators to make decisions. Especially on pops, when you get alot of traffic, that's a must. There are so many free sample size and statistical calculators out there...use them guys, you need them the most.

---

Amy, thanks. When to cut landers and placements had been bugging me for a long time.

Could this also be applied to landers?

I'm using Bayesian Calculator as a basis to cut the landers. But it doesn't take account of the impressions and spending per lander.

So let's say I have Lander A and Lander B. Both landers has the same ad spent.
Lander A has 20,000 impressions, 100 clicks with 10 conversions. Low CTR of 0.5%.
Lander B has 20,000 impressions, 300 clicks with 15 conversions. Higher CTR of 1.5%.

With Bayesian, Lander A wins with 95% approx chance for it to be the best. But if we look at it, Lander B netted more conversions, and with higher CTR theoretically speaking more chances for it to nett more conversions.

Of course I know all of us will continue to run traffic to both landers. But what if one needs to be cut, what will it be?

---

1st: Why do you care about CTR? On landers, CTR doesn't mean much. A better CTR doesn't always result in better conversions. You can create some kind of "click bait" , your CTR goes up but no extra conversions, or even lower. The lander is to pre-sell for better conversions not CTR.

2nd: With most statistical significance calculators you won't get Lander B as winner. There are multiple testing algorithms: Z-Test, Welch T-Test, chi-squared,etc. On Z-test that LPOptimizer uses, Lander B is not a winner.

3nd: At your sample size, no lander is the winner. In fact, you need A LOT MORE impressions to determine a winner at that low conversion rate. Unfortunately(and affiliates don't like this), you need to keep buying impressions to get real statistical data. All A/B testing platforms have this glitch when they declare the winner without enough sample size. Even if the testing algorithm declares a winner, you need to wait until enough sample size is reached. The sample size depends on the difference in conversion rates between the tested landers, the lower the difference the more sample is needed to see which is the winner.

For example, at your conversion rates(if it keeps to be at the same proportions which I assume they will be) you need to have about 150,000 impression/branch to see which is the winner. To determine sample size, you can use calculators like this http://www.evanmiller.org/ab-testing/sample-size.html

---

Thanks Crysper, that was the answer I was looking for - Sample size. Mind explaining how do I plug in the numbers for that?

Taking the case above, baseline conversion rate should be 5%. What about Minimum Detectable Effect though? The difference of CR between lander A and lander B?

And yes, more impressions is needed for the above case, I just plucked the numbers out of thin air for an illustration of what I meant about A/Bs.

In my case though.. For an offer I ran, I ripped like 30 unique landers over 2 weeks and putting it into the rotation continuously. But I pretty much cut a lot of landers based on Bayesian + Binominal + Gut feeling. But I'm not convinced though after looking back at the stats.

Oh and, CTRs. Why, because all my 30 landers averagely performed the same with a very low CTRs - about 0.5% to be exact. Meh.

---

The minimum detectable effect is the difference between lander A and B(in your case). You have 2 options there, absolute and relative. Relative is the difference in % of the 2 conversion rates, absolute is just the difference (CR for lander A - CR for lander B).

Normally the calculator is to detect the sample size required for a test, but you can use it to determine if the sample size is enough for the 2 landers, if you are not sure which is the winner.

Bayesian approach may be good for low sample size, where there is no way to get more data, but this gives you just and estimation of which variation MAY be better. This is not used to get real statistical significance(as far as I know).

If you use an algorithm like Welch T-Test or Z-Test(as most platforms use) and a good sample size(use that calculator I showed you), you'll get valid results that you can be sure of.

---

Very interesting discussion crysper and letsgo!

Crysper has made many very good and knowledgeable points. The question "when should I cut [something or other]?" comes up quite a lot. I think I'll start a separate thread and share what I know (which is JUST enough). Will link to that over here when I'm done! Stay tuned over the next 24-48 hours!


Amy

---

Love the dialogue, thanks for all the replies. Amy thanks for taking a stab, appreciate the detailed thoughts, super helpful.

---

Hey guys! As promised I wrote-up some posts on how to cull things. You can find them here:

Part 1 Part 2

Enjoy!

Amy

---

@crysper - Bayesian approaches do work as a way to get statistically significant information: they're significantly used in the medical industry for drug trials, where obviously they're quite interested in getting the right result.

However, there's a long-running argument between Bayesian and Frequentist statisticians as to which is better For ads, personally I think Bayesian has some pretty significant advantages, but using ANY statistical significance calculator that's based on a solid mathematical approach is about 2000% better than using none.

---

Home > Paid Traffic Sources > Mobile