Home > Mobile >

Double Meta Refresh - Does it affect your ROI or not - Let's find out! (39)

12-02-2015 11:14 AM #1 matuloo (Legendary Moderator)
Double Meta Refresh - Does it affect your ROI or not - Let's find out!

There has been a lot of talking about DMR (Double Meta Refresh) lately on the forum, whether it decreases ROI or not, so let me take the unpopular burden of loosing money on the testing and find the definitive answer for you guys

I will use one of my larger campaigns for this, already optimized to make profit.

It's targeting Android and iOs phones, tablets are excluded.
Majority of the traffic is 3G - Im targeting 3G only but some wifi always slips through.
The GEO is a rather large Asian country.
Im using a landing page for this, running on rackspace CDN.
The campaign is tracked by Voluum.
The offer is a PIN submit.
All the traffic is from banner clicks.

So much for the test details, I think the most important information is there.

Let's start with a screenshot of the performance without the DMR. I've set the timespan big enough, so there is enough clicks and conversions. I blurred out the sensitive parts obviously, but all the info we need for the purpose of this test is there.

The important values we are gonna watch are LP CTR, CR and CV.

In case the theory of the DMR slowdown is true, we should see a decrease mostly in the CV column, which is conversion rate from the overall traffic sent to this campaign, the slower redirect should cause some "loss" of the traffic so it should be the most visible here. The LP CTR should remain more or less the same as the DMR happens only once someone clicks through the LP. The CR value should also go down.

So much for the theory, let's see what is gonna happen. The campaign is already running for some time, the plan is to let it run until it sees pretty much the same amount of traffic (15 000 clicks), unless I see a significant difference sooner.

Stay tuned for the results

EDIT : after some replies and tips, I decided to run this as a proper split test, with the help of paths in Voluum. Unfortunatelly, voluum support confirmed for me that its not possible to have 2 flows for one campaign and it's also not possible to check stats by path, so I will be using the trick suggested by "olorin" - I made 2 new offers (duplicates from the original one) in voluum and assigned one of them to each of the paths. One path is configured to 302 redirect, the other one to DMR so the results should be truly representative

12-02-2015 11:47 AM #2 manu_adefy (Veteran Member)

Why do you show previous results? You should split test and have the same traffic running 50/50 DMR and no-DMR. I didn't understand that you do that from what you explained. Why does previous data have any relevance whatsoever?

12-02-2015 12:26 PM #3 matuloo (Legendary Moderator)

Originally Posted by manusit

Because you cannot split test DMR and no DMR in Voluum in one campaign, you either run it one way or another. So first I collected data without DMR and now Im running with DMR. Once I have comparable size of data, we will see what it looks like. In this first stage, I want to find out if the difference is significant or just small. If the numbers will look similar, it will need more testing and more detailed setup.

I'm very well aware that different hours/days perform differently, that's why I picked a well oiled and optimized campaign with stable results that dont go up and down a lot.

12-02-2015 12:42 PM #4 archieaff (Member)

Originally Posted by matuloo

Because you cannot split test DMR and no DMR in Voluum in one campaign, you either run it one way or another.

Isn't it possible by using flow instead of path in campaign settings?

12-02-2015 12:45 PM #5 matuloo (Legendary Moderator)

Originally Posted by archieaff

Isn't it possible by using flow instead of path in campaign settings?

Didnt think about this, let me take a look.

12-02-2015 12:48 PM #6 thuglife (Member)

Originally Posted by matuloo

Didnt think about this, let me take a look.

You can set two separate flows for one campaign, one uses DMR, the other doesn't.

Subscribed! Been waiting for a case study on this.

12-02-2015 12:59 PM #7 matuloo (Legendary Moderator)

Originally Posted by thuglife

You can set two separate flows for one campaign, one uses DMR, the other doesn't.

Subscribed! Been waiting for a case study on this.

Im not sure it's doable this way. I made two flows, but dont see an option to use 2 flows for the same campaign at the same time.
So I edited one of the flows to have 2 paths, one with and one without DRM, but now I can't figure out how to check the stats for the paths separately.

Anyone has experience with this?

EDIT : I searched the Voluum support forums and looks like its not doable, unless they made an update recently. I emailed them just in case, so we will see what they reply.

12-02-2015 01:50 PM #8 mykeyfocus (Member)

Sweet, subscribed

You could send traffic to a PHP file and randomly redirect between two identical Voluum campaigns (one DMR - the other 302 redirect)

The code to do so is below, just copy into whateveryouwant.php. Being PHP you'll need a VPS, not a CDN,

Code:

<?php

	$a = htmlspecialchars($_GET['a']); // Pass any variables you want by using this line, copy paste as needed
	$b = htmlspecialchars($_GET['b']); // Pass any variables you want by using this line, copy paste as needed

	$lp1 = 'http://voluumDMR/1?a='.$a.'&b='.$b; // to add another variable just plop a .'&c='.$c on the end (before semi colon)  :)
	$lp2 = 'http://voluum302/2?a='.$a.'&b='.$b;
	
	$links = array($lp1,$lp2);
	
	srand(time());
	$random = mt_rand(0,count($links)-1);
	
	header('Location: '.$links[$random]);
	exit;

?>

12-02-2015 01:54 PM #9 matuloo (Legendary Moderator)

Originally Posted by mykeyfocus

Sweet, subscribed

You could send traffic to a PHP file and randomly redirect between two identical Voluum campaigns (one DMR - the other 302 redirect)

The code to do so is below, just copy into whateveryouwant.php. Being PHP you'll need a VPS, not a CDN,

Code:

<?php

	$a = htmlspecialchars($_GET['a']); // Pass any variables you want by using this line, copy paste as needed
	$b = htmlspecialchars($_GET['b']); // Pass any variables you want by using this line, copy paste as needed

	$lp1 = 'http://voluumDMR/1?a='.$a.'&b='.$b; // to add another variable just plop a .'&c='.$c on the end (before semi colon)  :)
	$lp2 = 'http://voluum302/2?a='.$a.'&b='.$b;
	
	$links = array($lp1,$lp2);
	
	srand(time());
	$random = mt_rand(0,count($links)-1);
	
	header('Location: '.$links[$random]);
	exit;

?>

Looks like this is actually the only way to do it with Voluum, but dont want to put any extra redirect into the chain now. I will let it run for now as it is and analyze the results once I have enough data. So far it looks like the difference will be there and bigger than I expected.

12-02-2015 01:55 PM #10 olorin (AMC Alumnus)

Originally Posted by matuloo

So I edited one of the flows to have 2 paths, one with and one without DRM, but now I can't figure out how to check the stats for the paths separately.

Anyone has experience with this?
.

The only way i can think of, is to duplicate your lander and use landerA in path A (DMR) and landerB in path B (no DMR).
That way you will get some kind of "path stats" while looking at the landers stats

12-02-2015 02:12 PM #11 matuloo (Legendary Moderator)

Originally Posted by olorin

Bingo, this should do the trick I believe

But I will use offers for this, gonna make two "new" offers in Voluum for this and rotate them via paths. Thanks for the suggestion !

12-02-2015 11:52 PM #12 pronewbie (Member)

Originally Posted by matuloo

Bingo, this should do the trick I believe

But I will use offers for this, gonna make two "new" offers in Voluum for this and rotate them via paths. Thanks for the suggestion !

Why making new duplicate offers? Just send the lander with dmr and the other without dmr to the same offers because you will see the stats in Campaign - Lander. I'm doing this now and the lander with dmr goes thru a custom domain to the offer and the other lander to the same offer without the dmr

12-03-2015 08:47 AM #13 matuloo (Legendary Moderator)

Originally Posted by pronewbie

Doesnt matter what you use as a "separator tool", the lander or the offer, the results will be the same. Its still the same offer, just a duplicate of it in Voluum. And I wanted to be to compare the data on the offer level with the old performance so that's why I've set it that way.

12-03-2015 09:09 AM #14 matuloo (Legendary Moderator)

Ok, so here are the results and I have to say Im a bit surprised by them

The numbers took some time to even up, so I waited a bit longer than I planned to. At the begining, the DMR version performed way worse, but with more and more traffic the numbers evened up and ended up like you can see in the screenshot below :

As you can see, the DMR path had a bit lower CTR, but the conversions itself remained almost 100% the same. The 302 path did only 5 more conversions and since we have a sample size of almost 800 conversions I think I can safely assume that DMR had NO effect on the conversions during my test. The lower CTR is probably a result of bot/fake traffic that couldnt be redirected by the double meta.

When you look at the first screenshot, you will notice that the overal CV went down from 2.33 to 1.93 which is a rather significant hit. It could have been a bad day tho. I disabled the flow now and running through my original path again, so we will see what the numbers will look like later today.

12-03-2015 09:56 AM #15 reacheffect (Member)

Thank you for sharing this info Matuloo, very interesting.

12-03-2015 10:55 AM #16 rockstar john (Member)

Nice to know!! Thanks for sharing your results!!!

12-03-2015 11:40 AM #17 thuglife (Member)

Thanks for posting the results!

Would you mind sharing what type of traffic this was?

(desktop, mobile, pop, 3g/wifi)

12-03-2015 12:00 PM #18 matuloo (Legendary Moderator)

Originally Posted by thuglife

Thanks for posting the results!

Would you mind sharing what type of traffic this was?

(desktop, mobile, pop, 3g/wifi)

Everything you asked for, and more, was covered in the first post

mobile, banner clicks, mostly 3G

12-03-2015 12:09 PM #19 thuglife (Member)

Originally Posted by matuloo

Everything you asked for, and more, was covered in the first post

mobile, banner clicks, mostly 3G

Great, one more question, do you know if there was any leaks on Voluum's DMR?

12-03-2015 12:38 PM #20 matuloo (Legendary Moderator)

Originally Posted by thuglife

Great, one more question, do you know if there was any leaks on Voluum's DMR?

Leaks? Be more specific please

12-03-2015 12:40 PM #21 thuglife (Member)

Like a small percentage still passing referrers.

12-03-2015 12:44 PM #22 matuloo (Legendary Moderator)

Originally Posted by thuglife

Like a small percentage still passing referrers.

Ah, you mean if the refferer still got passed further in some cases? I didnt do any tests regarding this, my goal was to test how it affects conversions only. I might ask the aff network that this traffic was sent to, but as I already changed the settings I think the data will be mixed and they wont be able to tell.

12-04-2015 04:36 AM #23 beckslash (Member)

Myth busted!

Since 3g is usually the slowest connection, it's probably safe to assume this conclusion applies to wifi and desktop as well. Even so, the geo you've chosen might have very good internet speeds so there still might be some discrepancies in others. But if you got to DMR you got to DMR and your profitability shouldn't relay on such low margins this could cause anyway.

12-04-2015 06:25 AM #24 acjhonrosya (Member)

Thanks for clarification that there are not much difference in 302 and DMR.

so you just selected path and selected DMR option in Voluum? are there are other setting?

Thanks again.

12-04-2015 07:49 AM #25 beemarkt (Member)

@matuloo .....
Great sharing, thank you !!

Is it okay to cloak the mobile lps with single meta refresh ? Or DMR is necessary in order to cloak the lps properly.

Thank you

12-04-2015 08:10 AM #26 thuglife (Member)

One more question, where is your site hosted? Are you on a CDN? Where is your traffic from? same GEO as your server/CDN?

12-04-2015 10:04 AM #27 matuloo (Legendary Moderator)

To make things even more interesting, I looked at the data again today and some delayed conversions came in, this happens with mobile all the time. People forget about sites opened in their browsers and discover them later on etc ...

So here is the new screenshot :

As you can see, the DMR path actually made a few more conversions than the 302 path. Its only a few tho, so it doesnt by any means prove that DMR is better to use. It just confirms, DMR vs 302 doesnt really have a winner in this test. From what I can see in the data collected, it doesnt matter which of them you use in terms of conversions or ROI, the only thing that was different, was the LP CTR %.

12-04-2015 10:06 AM #28 matuloo (Legendary Moderator)

Originally Posted by beckslash

Actually, this was a GEO with not so good mobile internet infrastructure. I chose it just because of this, as I expected the difference would be significant here.

12-04-2015 10:14 AM #29 matuloo (Legendary Moderator)

Originally Posted by acjhonrosya

Thanks for clarification that there are not much difference in 302 and DMR.

so you just selected path and selected DMR option in Voluum? are there are other setting?

Thanks again.

Im not sure what other settings you are asking about. I used the FLOW option, with 2 paths, 1 with DMR and 1 with 302.

Originally Posted by beemarkt

@matuloo .....
Great sharing, thank you !!

Is it okay to cloak the mobile lps with single meta refresh ? Or DMR is necessary in order to cloak the lps properly.

Thank you

To cloack properly, you need a good cloacker, DMR alone wont cut it. It just hides the referer, but Im not even sure its 100% good at this.

Originally Posted by thuglife

One more question, where is your site hosted? Are you on a CDN? Where is your traffic from? same GEO as your server/CDN?

This is irelevant for this test. Both the DMR and 302 paths had the same setup/conditions. So it doesnt matter where it was hosted etc ... But yes, it was on a CDN (rackspace) - to my knowledge, they have a network of datacenters across the world and the data is copied to all of them, so they can serve it from a "server" closest to the surfer. Im no techie so I might be wrong, but I dont think it matters much what GEO is the CDN server that you upload the file to originally.

12-05-2015 10:14 AM #30 thuglife (Member)

Originally Posted by matuloo

Im not sure what other settings you are asking about. I used the FLOW option, with 2 paths, 1 with DMR and 1 with 302.

I'm going to test this on desktop traffic where the redirect is more prominent and see if it affects conversion.
I'll create my own thread for this.

Edit: Another thing I forgot to ask you, did you use a custom redirect domain or Voluum's default?

12-05-2015 01:12 PM #31 matuloo (Legendary Moderator)

Originally Posted by thuglife

I'd be surprised if the difference would be bigger with desktop traffic, but everything is possible

I used the Voluum's redirect settings, didnt use any custom redirects.

12-05-2015 01:14 PM #32 thuglife (Member)

Originally Posted by matuloo

I'd be surprised if the difference would be bigger with desktop traffic, but everything is possible

I used the Voluum's redirect settings, didnt use any custom redirects.

I did it. 40% less revenue/conversion with 90% statistical significance.

12-05-2015 01:16 PM #33 matuloo (Legendary Moderator)

Originally Posted by thuglife

I did it. 40% less revenue/conversion with 90% statistical significance.

Thats too much of a difference, I'm having a hard time to believe it after the numbers I got with my test. Can't wait to see the numbers once you post them.

12-14-2015 04:22 PM #34 andrea1987 (Member)

Thanks man, cool stuff

12-14-2015 08:21 PM #35 matuloo (Legendary Moderator)

Originally Posted by andrea1987

Thanks man, cool stuff

You're welcome, thats what Im here for

12-15-2015 12:09 AM #36 kenny_digital (Member)

Monster ctr!!!

12-15-2015 12:40 PM #37 matuloo (Legendary Moderator)

Originally Posted by kenny_digital

Monster ctr!!!

I was surprised nobody else noticed

With banner traffic and the right targeting, this is doable indeed

03-11-2017 10:39 AM #38 aloeveraa1491 (Member)

HI Matuloo! This is a great and very interesting thread, combined with the volume sent and the results obtained, it's incredible!
Thanks for conducting the test

Since this was done from mobile display banner clicks,
What about for mobile pops?

have you (or is there any) conducted a test with purely mobile pop traffic and see if it made any difference?

03-11-2017 10:50 AM #39 matuloo (Legendary Moderator)

Originally Posted by aloeveraa1491

Loading speeds definitely have bigger impact on POPs, since those load without user intent. When someone clicks a banner, they will certainly wait a moment or two for something to load, but that's not the case with an intrusive format such as POPs. I'm not aware of large test with POPs thou that someone would post on STM - I'm pretty sure the effect will be larger thou, that's absolutely inevitable.

Home > Mobile >