How to Cut Placements, Landers and Banners - Part 2 (28)
Is the formula Bayesian?
Outstanding, thank you for taking the time to put this together.
Going to apply it a few places and I will make sure to report back 
Great stuff Amy!
For those of you who aren't yet doing much statistical analysis of your results, start using this now.
It has never been handed to you on a shinier, more silver platter. It's a platinum platter.
If I remember correctly this kind of formula is also used to cut off placement early when the max CR < required to break even CR. please correct me if I'm wrong
For those of you who aren't yet doing much statistical analysis of your results, start using this now.
It has never been handed to you on a shinier, more silver platter. It's a platinum platter.
(EDIT: Need to add - the "Minimum Viable CR" is actually NOT always the break even CR. It's the minimum CR required to reach the ROI you specify for "Minimum ROI". Sorry for missing that earlier!)
Amy
How would this be different from cutting based on placement spend in relation to offer payout? 2x 3x etc.
Great explanation. I've been looking into this lately as well, so I'm glad you started this discussion
This does seem like a good way of cutting placements. I'm concerned whether it might make it too early to cut as there might be more profitable times of day. But if your exposure is so short, you may not have fully tested the placement.
In terms of the ROI you are looking for when plugging in the numbers. What would you look for during the initial test before a few rounds of LP split testing?
However, the spreadsheet actually doesn't determine how long the testing is - it only needs to collect enough impressions etc. for it to make a decision. You could be collecting the data all in one day or over several days - the tool wouldn't know the difference.
Let's face it, any way you cut things, you run the risk of cutting good placements. It's like a yin-yang situation where if you want more accuracy, you need to spend more time and money and vice versa. The trick is to find that sweet spot of balance, which will vary for different people/situations.
The 80/20 rule applies here as well: One can choose to settle for "good enough", gather data quickly and scale quickly; or strive for perfection, and need to spend several times the time and money before being ready to scale.
The choice would be for each individual to make.
-Say I'm promoting a free mobile app. It's a special type of app that, after the offer ends I probably won't be able to find a similar one to take its place. In this case I'd set the Minimum ROI at 0% to cut placements while testing banners. Once I find a winning banner, I'd pause all other banners, move the ROI to 30%+, and continue cutting placements. In other words, I need to cut placements rigorously and quickly, because for this type of offer speed is key - I want to scale it as fast as possible before it gets pulled.
-On the other hand, say I want to break into the adult vertical. I know that there will always be adult dating offers around, so that once I find winning creatives I'd be able to bank forever (in reality I'd need to continue to test, but I'm simplifying the situation). Adult tends to be competitive - my competitors have optimized everything because they've been playing longer, so to compete I'd need to spend money to test as many creatives as I can. Therefore my main focus would be to get as much traffic as possible from the same traffic source to test stuff. In this case I would only cut the very worst placements by setting minimum ROI to something low - probably a negative percentage in the beginning - because my temporary aim is to test, not to profit. When the testing yields winning creatives, I may set the Minimum ROI to break even (0%) or positive. In this case, I don't want to limit the traffic I need for testing, and because I'm not in a hurry to get profitable, I don't need to cut half as rigorously.
My age must be catching up with me - getting really long-winded! In the end, the calculator is an inanimate tool - everyone will use it a bit differently. Thanks for your questions - they've inspired me to do more thinking.
Holy crap Amy! This is amazing! Thank you.
Originally Posted by Mr Green
Amy
Amazing, knew the how and not the WHY, thanks for that.
Should be a sticky thread I can see how it can help many affiliates!
Awesome stuff Amy!
VERY good stuff indeed. Thanks for posting this.
Interestingly, the "4x payout" rule is based on some statistical calculations, or at least breaks down to some reasonably sane assumptions (around +50% ROI floor, IIRC) - but as you say, it's very inflexible and doesn't take into account your particular goals or the nature of each campaign.
@grassr00ts, omrikos, EpicTrends, caurmen - Thanks for the kind words you guys! Just glad this is helping some people.
@Caurmen: It was inspired by your earlier threads so having you come bless the thread means something special! Also, it sounds like the 4x rule isn't as bad as I had originally thought! Maybe we should just all go back to using that instead. 
Amy
quantum27's question regarding how to set the Min. ROI, got me to think where my thoughts have never gone before.
After some sleep, I gave the matter more thought - here are some suggestions on how to set the Min. ROI:
Set a Higher Min. ROI For When...
-You're getting traffic from tons of placements and would rather cut faster and sacrifice some placements that still have positive ROI (but lower than the ROI you set)
-You're getting lots of traffic volume so you don't need to be cautious when cutting, knowing there'll be lots left over when you're done
-You're scaling so you want to limit your spend on cutting each placement to free up cashflow, and also speed up the testing
-Your creatives are already fairly optimized, and/or you've cut out low-converting devices/models/carriers/etc., so you can be more critical of placements (i.e. your ship is already tight so-to-speak, so if traffic isn't converting for a placement, it HAS to be because the placement's crappy!)
-The offer you're promoting is time-sensitive and/or could be pulled at any time, so speed is of essence
Analogy: War has just broken out, you have tons of people sign up for enlistment, you only have 3 months to train them. So what do you do? You drill them real hard - so hard that many of them either drop-out or pass-out. You may lose some potentially capable candidates, but the ones you have left over are the best of the best, and you still have enough to send to the battlefront.
Set a Lower Min. ROI For When...
-The traffic source being used only has a few big placements so you better be damn sure before cutting any (i.e. test them longer)!
-You're in the offer/vertical for the long-haul so you'd be willing to spend more time and money in initial testing
-In addition to testing placements, you're also testing banners and/or landers (more on this below)
-The particular traffic source already isn't giving you that much traffic, and you want to squeeze whatever profits from it you could (i.e. every drop counts)
Another VERY IMPORTANT thing to consider:
Sometimes, placements are not the only thing you're testing in a campaign. You may be testing banners and landers at the same time. Where placements aren't the only test variable, you'd want to set the Min. ROI to a lower value when cutting placements, in order to be "fair" to them.
However, when you've already optimized other aspects of your campaign - meaning your banners are landers are optimized and proven to convert well, and you've cut out low-converting devices/models/carriers/etc. - then placements are basically your only variable. In this case, you'd want to set the Min. ROI to a higher value.
Analogy: You're the head of a constructions company. You send out 2 new guys you just hired, to a site to do a job. At the end of the day, the job isn't finished. Who do you fire? Answer: You can't fire either one without being unfair.
Another scenario. Instead of 2 new guys, you send out a new guy with another employee who has a proven track record. At the end of the day, the job doesn't get done. Who do you fire? The new guy of course!
(I believe in NOT testing banners/landers + placements at the same time in one campaign. Will probably share my test approach in a separate thread later on.)
Once again, I've talked too much!
Amy
Originally Posted by vortex
Eg.
Placement: XYZ Targeting WIFI
Placement: XYZ Targeting Cellular
They would be in their own separate campaigns, and I would run a full test on each one of them separately.
Do you think there would be a way to just decide Placement: XYZ is no good, and no further testing is required?
Originally Posted by quantum27
As for deciding whether placement XYZ is good or not - that's an interesting question!
I can only tell you how I do things, which may or may not be useful.
I know that some placements will almost always convert badly - for example Grindr (folks there looking to hookup) and flashlight apps (people probably have their eyes on what they're shining the light at) So I guess there IS such a thing as intrinsically bad placements.
On the other hand, there are "specialty" or "niche" placements that may or may not convert for your particular offer. But this will be reflected in the conversion rates so we can just let the calculator spreadsheet decide for us.
I would 1)blacklist Grindr and flashlight apps right off the bat, 2)blacklist all the app/sites from the Decisive blacklist found here, then 3)use the calculator spreadsheet to continue cutting from remaining placements once traffic starts.
As for a test strategy that includes banners/landers: I should start a discussion about that in a separate thread to see what everybody's approaches are. For myself, I like to test banners first, during which I would only cut the worst placements even without using the spreadsheet (remember the 2 new workers analogy?) Once I find an OK banner, I would use the spreadsheet to cut placements at Min. ROI = 0% (breakeven). I may make a new campaign with some of the better-converting placements whitelisted (note: not necessarily the ones identified by the spreadsheet - just the ones with best CR) and use this new camp to test banners. Once I have a winning banner I'd let it replace that "OK" banner in the original camp, then move Min. ROI up to say 30% and continue cutting placements.
The gist of it is I try to use good banners to test and cut placements, and good placements to test and cut banners. That would be two campaigns each serving a different purpose. Once the spreadsheet starts to identify "WHITELIST!" placements, I'd start a 3rd campaign for those, which will be my "profits" campaign.
And if we add landers into the mix - I think I would try to use my best banner(s) and placement(s) to test landers. Thus far for mobile app installs I've never used landers.
Nice discussion!
Amy
This is awesome! Not sure I get the stats behind it, but can it work for pops too? Or would anything change?
Originally Posted by thedav
(It will also help identify placements that are likely to stay profitable.)I do pops too. On networks that have a ton of placements (e.g. zeropark), this tool will tell you which ones to cut without your having to manually look over the stats for every placement.
Hope you have fun with it!
Amy
@vortex I'm wondering what's the biggest difference between this method and the part 1 method . Is part 2 focused on cutting out proven losers instead of waiting for a winner ?
Can part 2 technique be applied to cutting out underperforming landing page too?
Thank you
Originally Posted by ryanfuse
Whereas the method in Part 2 actually evaluates EACH ITEM and tell you which ones are unlikely to ever be profitable (so you can cut them) and which ones are likely to remain profitable (so you can whitelist them if you wish), in the long run. This is especially useful for cutting placements.
You've asked a really interesting question - I suppose you could use method 2 on landers to see which ones are unlikely to be profitable and cut them that way. But using the split-tester as described in part 1 would work better for most purposes, because in most cases you'd want to pause all landers except the best. Unlike with placements, where you want as much traffic from as many placements as possible as long as it's profitable for you.
So to recap: part 1 is about picking out the best of a group, whereas part 2 is about picking out all profitable and unprofitable ones in a group. Part 1 is relative, and part 2 is absolute.
(Note that for the part 1 method, that being the best in a group doesn't necessarily means it's good! You can evaluate a group of crappy landers where even the best of them will lose you money. However, if you keep doing more and more rounds of testing, and keep beating the "winner" from your last round of testing, your ROI will be sure to go up. Unlike the spreadsheet calculator in part 2 which will tell you which placements are profitable or unprofitable and are likely to remain so over time.)
Hope I've made things clearer instead of even more confusing!
Amy
This is interesting thank you. I suppose method 2 can be repurposed to find the 80/20 landers . Say u want to focus on only the big results , the lander that converts huge . So u cut off every lander that's normal and keep on churning until you find the big hit out of the park .
Originally Posted by ryanfuse
So a better approach, even for finding the "80/20" landers, would STILL be method 1, except you won't keep cutting until only 1 is left. You'd just cut until you feel that all of the ones left over have potential to be winners, then you'd stop cutting, and start making versions of each of those to optimize them further. In fact this is what I'm doing exactly on my current campaigns.
Note: The ones to reach stat-sig the earliest and get cut, tend to be the worst ones of the batch, which is what we want. As to exactly WHEN to stop cutting - that would be a subjective decision I'm afraid - and would be up to the individual.
Another comment I want to make: Unlike placements, which either works or doesn't for the most part, a lander can always be optimized further. So sometimes the "big hit out of the park" will only come after you've tweaked a lot of elements on the same lander (which method 1 will help to do). Of course, once in a while you'll come up with a truly innovative lander that blows all previous landers out of the water even before you optimize it further. But in that case, you wouldn't need a tool to tell you how kick-ass it is - and method 1 will let you know that it's the best of the batch anyway.
I'm probably complicating the discussion unnecessarily. In the end, it really only comes down to one thing: continuously improving your lander will result in profits eventually, and the only way to compare is to use method 1.
Thanks for the discussion!
Amy
This is great Amy, really useful tool! Thanks
Originally Posted by OJay
I believe that everything that happens, does so at the right time and place. The fact that you had to "reinvent the wheel" was probably because it was meant for you to learn more on the topic of statistics. Your spreadsheet reflects the knowledge and understanding you've achieved through doing your own research and thinking. I'm 100% certain that this knowledge you've gleaned from your little exercise will stand you in good stead when analyzing your own stats in the future - so great job mate! *high five*
Like I've said time and again - I only know just enough about statistics to analyze my own stats. So if you have any additional insights on this topic you can share - please feel free to do so!
Thanks so much for your post!
Amy
Thanks Amy, this is Great tool to work with.
Now things like this makes STM the best Affiliate marketing community
Originally Posted by leadmarketing
https://stmforum.com/forum/showthrea...anceampampquot
The main kill/whitelist calculator does the exact same thing as what @caurmen described in his formulas, except it can do mass calculators in bulk.
Quick review: The main kill/whitelist calculator will tell you whether a placement is ready to be whitelisted or blacklisted or not, based on the following inputs - confidence level, offer payout amount, minimum ROI you want to achieve, the CPM you're bidding, the total impressions received so far, and the number of conversions received so far.
If you click on the second tab "Tweak Bid CPM", what it will do is tell you the maximum CPM you can bid, in order for a placement to have a "WHITELIST" verdict in the first tab.
For example: Let's say your offer payout is $5, you want your minimum ROI to be 30%, this one specific placement has received 3833 impressions and got 2 conversions, and you're currently bidding $0.30 CPM.
So you put these into the 1st tab (the main kill/whitelist calculator) and find that the placement is ready to be whitelisted:
So you think to yourself: I want to bid higher to get more traffic and potentially more conversions! But how much higher? Because bidding too high may make this placement lose money.
So you take your numbers and plug them into the second tab:
And the calculator will tell you that the maximum CPM you can bid, and STILL expect the placement to continue to be profitable, is $0.533669.
Of course this is all PREDICTIVE based on statistical trends - there's no guarantee the placement will remain in profits whatever amount you bid. The calculator is just a tool that can help you to decide what bids to test.
Important: It does NOT mean that this bid will maximize your profits for that placement - you'll need to experiment to find that number. But you know that the ideal bid that will maximize your profits will probably be less than that maximum CPM.
Hope that makes sense!
Amy
Amazing explanation @vortex. Thanks
Home > General > Affiliate Marketing Forum