Home > Paid Traffic Sources > Mobile

Should I wait until I reach statistical significance? (8)

06-13-2016 08:52 PM #1 success1 (Member)
Should I wait until I reach statistical significance?

Hello,

I have been testing a campaign for a week. The campaign consist of three sweeps win an iphone offers that have the same payout of $1.60. The offers actually have landing pages built in, so I am direct linking the campaign.

I am considering each landing page as a different angle. I am running this on PopCash as a run of network.

So far, all three offers have consistent conversions. (Please see attached results for each offer). But based on the AB split test graphical Bayesian calculator, I can't cut any of the offers because they haven't reached statistical significance.

I am at a negative -69 ROI at this point. I have found 7 profitable placements and three that aren't profitable yet have conversions.

Based on this data, can you please let me know if it's worth it to keep testing until I reach statistical significance?

Regards,

Success1

06-13-2016 11:41 PM #2 matuloo (Legendary Moderator)

The offers perform pretty much the same. With numbers like this, significance will not really help you, you would need larger volume to see if one of them starts to stand out a bit or not. Imagine the one with 5 conversions gets 1 more, there you have it, all pretty much the same.

The good thing is, that all of them do make conversions, so the offers are alive and converting. What you need to do is to improve the funnel, you really need to introduce some landing pages. There is some pre-selling needed, warm the users up, make them want to finish the registration process. Get the CVR up a bit and then you can start looking at traffic optimization possibilities.

06-14-2016 12:11 AM #3 success1 (Member)

Quote Originally Posted by matuloo View Post
The offers perform pretty much the same. With numbers like this, significance will not really help you, you would need larger volume to see if one of them starts to stand out a bit or not. Imagine the one with 5 conversions gets 1 more, there you have it, all pretty much the same.

The good thing is, that all of them do make conversions, so the offers are alive and converting. What you need to do is to improve the funnel, you really need to introduce some landing pages. There is some pre-selling needed, warm the users up, make them want to finish the registration process. Get the CVR up a bit and then you can start looking at traffic optimization possibilities.
Thanks matuloo! Appreciate your insight. My gut said the same thing, but I wanted to double check, as I read some of the posts on statistical significance that suggest to wait until reaching that milestone.

As I mentioned, the three offers actually already have proven landing pages with the typical sweeps themes such as Facebook, WhatsApp and Spin The Wheel.

I am unsure what my options are in terms of adding my own landing page. Do you have any suggestions?

Also, since I posted this, I spoke to my AM and she said one of the offers is doing the best across the network. Do you think that I should pause the other two and go with the one she said is best?

06-14-2016 12:32 AM #4 simon_89 (Member)

Trust affiliate managers with a grain of salt. What one affiliate manager tells you could be different from what another manager tells you.

What I would do is take the offer with 7 conversions and start testing them against 5-10 ripped landers. Once you have narrowed down to 1 lander. You can improve that lander by putting your own spin on the ad copy, optimizing the speed of the lander, hire a developer to revamp the whole site not to look buggy.

06-14-2016 06:07 AM #5 thuglife (Member)

No. You should work on things that can really impact your EPC.

e.g.
1. Going direct and getting twice the payout.
2. Getting an exclusive on a GEO.
3. Getting an exclusive on a traffic source with the biggest affiliate networks for your vertical.
4. Kicking other affiliates off of the offer because you can do volume.
5. A new angel that hasn't been "run to death" by other affiliates.
6. etc

because, sheeps get slaughtered.

06-14-2016 09:47 AM #6 caurmen (Administrator)

"All of them are performing the same" is the absolute classic pain in the ass scenario for split-testing.

Given the numbers you're seeing right now, I'd run them a bit longer up to about 10 conversions, and if one of them doesn't start pulling ahead, just pick one and focus on that so you can start building up volume.

06-14-2016 11:33 AM #7 matuloo (Legendary Moderator)

Quote Originally Posted by thuglife View Post
No. You should work on things that can really impact your EPC.

e.g.
1. Going direct and getting twice the payout.
2. Getting an exclusive on a GEO.
3. Getting an exclusive on a traffic source with the biggest affiliate networks for your vertical.
4. Kicking other affiliates off of the offer because you can do volume.
5. A new angel that hasn't been "run to death" by other affiliates.
6. etc

because, sheeps get slaughtered.
You realize the OP is a new affiliate trying to learn the ropes, right Advice like this won't do him any good. Stuff like this is for the top 1% maybe, having exclusivity with a traffic source for the whole vertical for the whole GEO? How many exclusive like this do you have running at the moment? I'd like to see them

06-14-2016 11:39 AM #8 matuloo (Legendary Moderator)

Quote Originally Posted by success1 View Post
Thanks matuloo! Appreciate your insight. My gut said the same thing, but I wanted to double check, as I read some of the posts on statistical significance that suggest to wait until reaching that milestone.

As I mentioned, the three offers actually already have proven landing pages with the typical sweeps themes such as Facebook, WhatsApp and Spin The Wheel.

I am unsure what my options are in terms of adding my own landing page. Do you have any suggestions?

Also, since I posted this, I spoke to my AM and she said one of the offers is doing the best across the network. Do you think that I should pause the other two and go with the one she said is best?
Even tho the offers have somewhat good looking lander, you can still improve the performance further with a proper LP. You need to call out the user and make him believe its a special offer just for him for example, that's what you can do with the LP. Definitely try to get some and test them.

AMs can be of great use but Simon is right, you need to take their advice with a grain of salt sometimes. The numbers they work with are based on mixed traffic and sometimes its not easy for them to separate the sources, so if they see an offer doing good generally, it can still work well just on banner traffic and do poor on pop traffic and they never really know this with 100% accuracy.

Home > Paid Traffic Sources > Mobile