Home > Paid Traffic Sources > Mobile

---

How many LPs you test at the same time ? (7)

---

Hello STM,

I was wondering how many landers you test at the same time ?

For example, in my case there's always 3 landers:
A. The best performer
B. Best performer variation
C. New layout

But i'm always tempted to add more, but it takes too much time to get statistical signifiance.
On the other hand, sometimes, i wish i only keep the best one and maximize ROI (short term).

Do you test more/less than 3 landers at the same time ? What's your strategy ?

---

It depends on how much traffic you have and how much of an improvement do you want.

Remember that you need to have statistical significance for your test to have any meaning. Otherwise it's just noise.

Use this calculator to see how many days your test needs to run based on your traffic and variations:
https://vwo.com/ab-split-test-duration/
https://vwo.com/bayesian-ab-testing/

---

If you want to test correctly, you need traffic, that's the first thing.

If you don't have the budget, stick with 3-5 landers. Once you have the budget and you find a good and steady campaign, you'll naturally optimize the landers more and more, you'll look after multi-variate testing and so on.

I've found out that most affiliates know pretty well when to increase the number of landers tested, either by experience or necessity. When you need to test more or when you need to use an optimization tool you'll know...

---

First of all, I would suggest that you find enough traffic to test your landers. It's just a pain in the arse when you have to wait days for a round of 3 landers to finish testing. Look for a traffic source that has more traffic for your geo and other targeting parameters. Bid more to increase traffic volume. Try not to cut placements except ones that are really draining your budget without converting. When you're testing, place less importance on ROI and more importance on traffic volume.

Another thing you can do is seek out offers that have lower payout. The assumption here is that it can still do similar ROI as your current offer, by having a higher conversion rate to make up for the lower payout. Low payout offers will take less traffic and smaller test budget to reach statistical significance. This may or may not be possible for you depending on your niche and geo, but may still be worth looking into.

Your approach is interesting! Here's what I usually do:

1)Test landers that look and function very differently. Rip and mod and perhaps even ones made from scratch. I don't eliminate all the way down to just the last best lander. I want to identify the best few in the group unless there's a single one that blows all the others out of the water.
2)Take the promising landers from round 1 and make variations and test those. Use the same text with different templates, same template with different text, etc. That way I can see which template and which text do well.
3)Come up with original angles, write original ad text based on them, and plug them into the best templates found in rounds 1&2.

In other words, I try to cast a wide net in the beginning, then zoom in on the stuff that has potential to test it further. A common newbie mistake is to take a lander they don't even know will convert well, and start testing stuff like colors and CTA. That would be the equivalent of choosing a random fishing spot and trying to vary the depth of your hook and wondering why you're not getting bites - not very efficient use of test budget or time at all.


Amy

---

Thanks for your help.

I forgot to say i was not in the beginning of my test.
The offer is converting well and i have 1-2 really good landers after of course a couple rounds of optimization (tested 20+ landers).

I'm about to scale the campaign, but in my mind i always need to keep testing new angles/landers etc.

So when you scale, i guess you keep testing new things, right ?

But you always give up a bit on ROI in order to test new things: in my case if campaign was running 100% on my best lander => best ROI
So if i add 1-2 challengers to this lander, my ROI will be lower unless both challengers perform better than my best lander.

In other words, once you are ready to scale, do you restrict the quantity of elements you test (better ROI) or you keep on testing lots of thing (lower ROI) ?

Btw i'm running pops

---

When you're already at a point where your offer + lander(s) is good enough to scale, by all means decrease the amount of testing you do.

I like to assign less weight to new offers and landers I'm testing (for example 20-30%, with 70-80% going to the best offer + lander). Either that or start a new camp just to continue testing and assign a low daily budget to it or just target the most profitable few hours of the day (just need to watch that doing so won't pull alot of traffic away from your main camp - depends on the traffic source).

When you're scaling, you'll inevitably encounter sources where your current creatives aren't "good enough" to profit. With continual testing, every time you find better creatives, you can retry those sources again, and some of them may be profitable then.

Continual testing will also keep you one step ahead of your competition. Unless you have a really good way of shielding your lander from spying eyes/tools, it would just be a matter of time before your lander gets ripped by more and more people. Keeping up with testing will let you stay ahead.

So yes - testing should never stop, unless you have another camp that is more profitable that will yield more profits when you spend time on that instead of the current camp.


Amy

---

I recommend three LPs during testing and 1 (the winner) full blast during upscaling. If performance goes down and you're getting close to break even point, repeat!

---

Home > Paid Traffic Sources > Mobile