Home > Paid Traffic Sources > Mobile

---

Landing Page Split Test Help (10)

---

Hi guys

I've run a split test on 5 landing pages 3 out of 5 are very different, two are very similar ( 3 and 5)

Here's the result so far, I've got 5 conversions so thought I could use the AB split test calc, it shows around 97% certainty that lander 5 is the winner. And without a doubt that lander 3 is the worst even though it reached 1 conversion. My problem now is the best performing lander for conversions gets the least amount of click through, and the one that perform the best with click through performs the worst on conversions. Whats even weirder is they are pretty much identical, both have timers and both won't allow the back button, but for some reason 1 is performing 10 times better on CT, and the other is performing 3 times better on CR.

There are some subtle differences in the wording but not much else, any ideas greatly appreciated.



Thanks

---

With only 5 conversions I'm surprised a tool would be picking a winner right now.

I'd recommend waiting.

---

Yeah I wasn't too sure I only took the recommendation from Caurmen in another thread

It's just ticked over to 7, how long would you recommend to wait conversion/traffic wise ?

I dropped Lander 2 and left the four remaining for now.

---

Hey Jasona,
I'm curious about how similar 3&5 are, is it just tiny copy changes? Is the logical consistency between ad-lander-offer the same for all?

But look, the ROI kinda speaks volumes here although it's not a lot of data. Has this been traffic of just today?
Perhaps slow it down a little and spread it out over a few days, depending on what you're doing, offer, traffic source etc. wise.

Why not take 4 & 5 and make variations on those 2 to split test? Keep testing

---

I crunched your numbers in the peakconversion calculator and found that only landers 1 and 2 are ready to be cut - lander 3 and 4 aren't ready yet:






However, I do agree that 90% probability of being best may be too low. Matuloo pointed this out a while back, and I've since changed my approach - I now wait until I have 95% or even 100% probability of being best before cutting offers and landers.

Having said that - even cutting at 100% won't give you 100% accuracy in reality, because there are too many uncertainties and volatility in campaign conditions. I would recommend to just do your reasonable best and leave it at that.

A thought: Have you checked the load times of the 2 landers that you said are similar? Although really, a difference of 2 conversions isn't that big.



Amy

---

Hey thanks for replying

They are around 95% the same, lander 5 has an extra window appear over the lander to make it appear more genuine (Android system type message) - the text is pretty similar, especially the instructions to follow once they click on the CTA button, both have auto refresh after around 30 seconds to the offer page, Images and colours are the same too.

The split test is just for today, I have been running lander 5 for the last 4 days and decided to test it against other landers, it's coming out on top which is what I expected, what's thrown me though is the CTR of lander 3 compared to 5 when they are so similar, if I could get a similar CTR on lander 5 as I currently have on 3 keeping the right variables I'd hope to see the conversions follow the same path.

---

Another question: Those stats in your first screenshot - were they collected during the same time period, where ALL 5 landers were in rotation?

Please note that you can't combine past and current data or the results would be off.



Amy

---

Thanks Amy will check that next - the CTR is so far apart I must be missing something on them.

---

Yes all 5 landers ran for about 15 hours solid today in rotation.

---

Check out this tool as well:

https://www.convert.com/tools/ab-tes...on-calculator/

It will give you a true idea of how much time you will need to get true statistical significance.

The thing that you'll notice right away is that almost no one is going to test long enough to get accurate data.

As affiliate marketers, there is a trade off you have to make between getting some sort of improvement vs waiting to fully validate that improvement.

Because

1. You don't have the budget to do a real test
2. You need to move quickly
3. There are just too many moving parts to do a real test.
4. Multiple other reasons

So yes with the current data, you can make some semi-intelligent decisions about winners and losers.

You may have already done what I'm about to say, if so this is for the other people

The general best practice tips from the pros is that, you want to be testing very different ideas, concepts and treatments when you are starting out. So changing a word or button color is not the kind of test you should be running.

Think more about testing angles. Once that's been locked down you can fine turn by testing elements such as the headlines.

-The following a slightly modified snippet from a newsletter I sent out. With fancy hand made images and all lol



What this does is allow you to explore more of the spectrum and improve your chances of finding the global maximum (The best converting combination ever) instead of the local maximum (The best of a less optimal combination).

As you can see in this image, you could be wasting your efforts on A/B tests that are not effective.
Imagine you are split testing a landing page. Think of every point on the image as a unique combination of elements. By making little tiny tweaks that are closely related to the initial page you have, you will end up optimizing to potentially reach a peak, but the peak you optimize to reach could be the one I highlighted as the best of the worst. Spend time finding the winning themes and ideas (angles), then you can tweak to find the peaks with the best performing combinations..

---

Home > Paid Traffic Sources > Mobile