Caurmen Vs Adult Mobile: The Tech Follow-Along (53)

Yes, it has finally happened!

I've been promising a tech-focused follow-along for a while, and I finally have the break in other campaigns and work to fit it in.

So, here we go.

The Plan, Man

Traffic Source: Mobile. Specifically, I'm going for ZeroPark to start with, because the PPV-like nature of the traffic favours speedy, well-set-up servers. That means I'll be doing more optimisation (good for this follow-along) and also means I can hopefully outperform my less-techie competition (good for my bank balance).

Vertical: Adult Dating. I've not tried it yet, I know that ZeroPark has good adult inventory, and I'm informed by Maynzie on these very forums that it's a "choo-choo money train.". (As I'm British, I assume that means it's always at least 2 hours late and stops working whenever leaves fall on the line.) This is a totally new vertical for me, so please do make suggestions if - when - I'm being a noob!

Region: USA I believe that the US is a tough nut to crack in adult, but I also know it's where most of ZeroPark's traffic comes from, and I'm hoping the newer traffic source will mean it won't be too saturated yet.

Offers: I'm being guided by my knowledgeable AMs from Adsimilis, PKM and Neverblue here - current plan is to split-test Adult Friend Finder, CheekyLovers and WildBuddies. Suggestions for alternate offers to test very welcome!

And with that, we're off to the races!

Day 1: Server Setup

First things first, I'm going to need a new VPS. I'll be installing new tracking software (the new STM tracker from bbrock - I'm very excited about this), and my plan is to test a lot of cutting-edge software on this follow-along, so I don't want to be fucking around with my existing server setups.

Job 1: choose a hosting provider. I'm not going to go straight for a Beyond server because I'm wanting to build everything from the bare operating system up. (Tyler - do you offer unmanaged OS-only VPSes at all?)

Normally, I'd go straight for a Linode server because of their excellent peering and generally high-quality support, but I want something with SSDs, and Linode aren't there yet. They've got badass CPUs, but unless I start doing something crazy (always possible), I'm not going to be stressing the CPU anyway.

Instead, I want to check out a new contender in the hosting arena: Digital Ocean. They're very cheap for what they offer - for the price of a 512Mb Linode box, I can get a 2Gb VPS with 30Gb of SSD. But that means nothing if they're slow as hell.

Given they have a really cheap trial (like, free), I fire up one of their servers, install an out-of-the-box Nginx install on it (apt-get nginx. service start nginx. Done.), and test it against Linode's public speed test locations. I want to know how the speed of connection is across the whole USA, so Pingdom won't do the job. Instead, I fire up five instances of Uptrends' wonderful worldwide speed tester, dump my new server and the Linode speed test locations in there, and let it roll.

Results? It appears that Digital Ocean's datacenter is, on average, slightly faster than Linode's setup. The slowest US connection is from Seattle, taking 0.082 sec to connect. Most connections are happening within 0.05sec or less. That's perfectly acceptable for just about any use, so I go ahead and set up MySQL, PHP-FPM and my virtual host, aquire a domain name from Namecheap, and get everything running.

Tips if you're doing the same:

• .info domains are waaaay cheaper than .coms. I tend to always go for a .info on any URL I'm expecting to be throwing away at some point, like an LP server.
• PHP-FPM connects to Nginx via TCP by default. That's a big slowdown. It's very easy to uncomment the line making it connect via a unix socket instead.
• DO uncomment "server_tokens off" in the default nginx setup. It makes your system harder to hack by obscuring version information.
• I never install a stack from memory. There are lots and lots of studies showing that checklists are a superb way to reduce problems. In this case, I used this excellent tutorial to avoid screwing up or missing anything.

• BrowserStack is a great way to load up landers to view their Javascript without worrying about infecting your computer with something lasty.
• To easily research your mobile competition, use Chrome's developer tools to change your user agent string. This tricks sites and tracking software into thinking you're on a phone.
• One landing page I viewed was loading all of JQuery plus another 50k of code - and then not using any of it. On a mobile. Don't do this.
• CSS gradients are a great way to make a page look more sophisticated without adding much in the way of filesize. Just add one on a couple of your divs, and bingo - it looks like you've hired a graphic designer, or at least sat in the same room as one for a while.

Thsank you for doing this Caurmen.

I don't get the cloud hosting solutions and have some questions.
They are not managed and require set ups.

How does one add the OS, etc? Is this it? http://www.howtoforge.com/installing...untu-12.04-lts

Does it need Cpanle?

Can one add and change its IP's

Can one load windows and create a remote Desktop for use as an IP?

No worries!

All of what I'm doing I'm doing using the Linux command line - a text-based interface. On Windows, you'd use Putty, which is an SSH client, to connect to the server, then type in commands.

A VPS will come with an operating system - usually Linux - already installed, and a single IP assigned, although you can ask for more IPs from the hosts.

Once you've logged in through SSH, you can then add a lot of software with simple text commands. Ubuntu and Debian are particularly good for this - you can install most common software by typing "apt-get install NAME_OF_SOFTWARE". Makes Windows installers look like very hard work

You don't need CPanel, although if you want it you can install it. And yes, you can also install a window manager (not Windows - but something that turns it into the equivalent Linux desktop) and use remote desktop software from there. Here's a tutorial for that.

Wow you're on fire buddy

Best of luck for this follow long!

Been trying to install cpanel through ssh half of the day already. Just keeps failing.. broken pipe here, hang there. it pisses me off badly.

@h0mp - personally I tend to just administrate everything through SSH. I've never found CPanel to be terribly useful - half the time it's more annoying than just typing "nano blah.conf".

Originally Posted by caurmen

@h0mp - personally I tend to just administrate everything through SSH. I've never found CPanel to be terribly useful - half the time it's more annoying than just typing "nano blah.conf".

Caurmen,
Does the use of a .info domain impact ad approval or the perceived quality of the ad by FB in any way?

@caurmen: yes you are right. I got used to GUI and clicking icons because cpanel has been pre-installed on almost every server so far. I'm sorry

Also, I never imagined cpanel to be so huge, big and bulky.

@sandyone - I don't actually know the answer to that - I'm not particularly expert with Facebook. I'd be interested to know, though, if anyone else does know?

@h0mp - Hey, whatever works. No "Oh, I only use CLI" epeen here I use PHPMyAdmin, for example, and I'm not afraid to admit it.

@timtetra - Yeah, depending on your setup that can be unfun. If you want something that's insanely easy to add domains to, check out the Symbiosis OS, written by Bytemark - it's a repackage of Debian designed from the ground up to do web hosting. Adding a new domain's a matter of creating a new directory and sticking some files in it! Sadly, it runs Apache, or I'd be using it for all my hosting.

Aaaaand it's time for...

Day 2: Setting Up Tracking

I'm lucky enough to be testing out the new Version 3 of the STM Mobile Tracker. For those of you who don't know about it, the STM Mobile Tracker is a heavily modified version of Prosper 202 that has been optimised to within an inch of its life to work specifically for mobile. I've not used the previous versions, but this one's really impressing me - there are a lot of new features and simplifications.

As someone who gets very annoyed by having to spam redirect scripts all over my LP folders, for example, the new single-URL, no-Javascript, one-piece-of-easy-code tracking setup is a major speed increase:



Particularly given you can copy-paste that code between landers and just change a single line, I'm liking the change!

Of course, the install process still hit a few problems - all with my setup, not the tracker, I might add. For anyone going through a similar process:

• The default Ubuntu installation of PHP appears to be a bit screwed up - it doesn't set a timezone by default, which can cause all kinds of problems. To fix that, find the timezone line in php.ini, uncomment it, and add the appropriate timezone from this list in quotation marks.
• The latest version of MySQL gets hella confused if you don't specify a location for a new user. So make sure to GRANT ALL to "Bobsuser@localhost", not just "Bobsuser", unless you enjoy swearing at MySQL documentation...

i love this follow along already! subscribed and thanks!

how would you say nginx is compared to litespeed?

@nyc - The major thing I notice about nginx compared to Litespeed is that nginx is free They're both excellent web servers. I'm not as familiar with Litespeed because Nginx will, with some prodding, do everything that I want it to, and it's absurdly memory- and CPU- efficient.

Nginx has a reputation on the Web for being under-documented, but I don't find that's the case these days. So many people use it that it's simple to get an answer to most questions. And it's adding capabilities fast - notably, it has a Rewrite Engine equivalent now.

I'd be very interested to hear a counter-argument on this point, actually, as the more I think about it the more I can't see why you'd not use Nginx. Anyone want to explain why they use LiteSpeed rather than Nginx?

Quick update because it's Saturday and I want to go spend some time outside with my gf

Initial very small test (rather smaller than expected because I forgot to refill my account) yesterday went well. Cheeky Lovers isn't converting yet, but WildBuddies is doing OK. I've added the Adult Friend Finder mobile offer into the mix, and we'll see how that pans out. Sadly, their terms mean I have to remove my Adult Friend Finder - related keywords from the pile, but they weren't getting much traffic anyway.

I've also pushed my bids up a bit to get more traffic - up to 0.012 now - and I've added a bunch of porn-related keywords.

More on Monday, when I'm going to start developing hypotheses for my split-tests. Stand back, I'm going to try science.

Spend: $8
Revenue: $4.48
ROI: -55%

OK, I forgot it was a public holiday on Monday, so this update's Tuesday instead. Still, that means more dataz, and more dataz is good.

Now it's time to develop and test some hypotheses toward my goal of, y'know, money.

Since start of campaign:

Spend: $48
Income: $37
ROI: -22%

Not at all bad for a control!

All three offers are performing tolerably well, with CheekyLovers leading the pack with a 0.0096 EPC (sounds pretty terrible, but remember I'm buying clicks at around $0.011 here) and AdultFriendFinder trailing with a 0.007 EPC. However, we're not at statistical significance there and there's no huge variation, so I'm going to leave all of them running for now.

The lander's performing at a 20-22% CTR and an average .68% signup rate (somewhere between .49% and .92% at a 90% confidence interval). Hmm.

Obvious first step is to ask my AMs for a bump on those offers, so of course I'll do that. But what else?


Analysis of data

It's very easy to jump straight to solutions without first considering the problem. I know this because I do it all the bloody time. But with this Follow-Along, I'm going to put serious effort into doing it properly

So, first let's consider the funnel we're using to get people to the offer, and then we can identify the leaks.

1. Person browsing on their phone mistypes a porn URL.
2. They end up on the Rules Lander I'm using. Chances are the picture catches their eye first, then they either click the "join" button straight away, or read the text.
3. They click through to the offer page.
4. For WildBuddies and CheekyLovers, they complete a 3-4 field form, including a couple of text fields.
5. For AFF, they click through another "Search" LP, then complete a longer registration form, then have to confirm their email/phone number.

• We're losing more than 3/4 of all traffic when they first arrive on the page. Not surprising, because what they're seeing isn't what they wanted. However, this implies the page isn't super-"grabby" yet.
• Even on the short-form submits, we're losing 19 out of 20 people who DO click through. That implies they're nowhere near sold enough on the offer.
• Even the shortest of the forms require several fields' worth of typing, plus some personal information. That's a bit of a killer particularly on a phone. We either need to give them more trust (to get them to submit personal info) or make them want the offer more (to get them to get through the text submit).
• We lose 4 times as many people on the AFF signup process - again, not surprising. It's longer and needs a reconfirm. We may be either losing people to the long form, or the reconfirm at the end.

• We don't know which signup button people are clicking on - it could be the button over the image (in which case they may not be reading the text) or it could be the button at the bottom.
• We don't know where we're leaking on AFF - could be email, could be the long form.
• We don't know why people are clicking away from the other signup forms - lack of interest, "aargh text input", or lack of trust.

Awesome follow along caurmen! I always love reading more on the tech side of things since that is how I got in to AM in the first place.

Quick question: Is the STM mobile tracker recommended for regular web based campaigns? In terms of speed and convenience?

Check to see which button results in clicks / conversions. would be fairly easy to do and might result in tossing the rules lander overboard and go with something completely different.

@h0mp - I completely agree Indeed, that's one of the tests I've got set up. Looking at the lander design, my initial hunch is that the over-the-image button might be a big leak.

@rafael - The STM Mobile Tracker is a heavily modified version of Prosper, and has a lot of Prosper's strengths and weaknesses, although the tracking method is different. If you're using Prosper anyway, and you're hosting your tracking software in the same country as your landing pages, I'd say it's probably a better option than vanilla Prosper.

So, here's what I actually decided to test:

Test 1 and Test 2: Up Volume!

The first thing I'm addressing is the stickiness of the lander. Although given some of the creatives I've seen, I'm not sure I want to use the word "stickiness" in connection with adult marketing any more.

Why? Because if I can boost the volume I'm getting here, it makes all my subsequent conversion tests much faster and cheaper to run.

I'm running two simultaneous split-tests, of headline and image. This is a relatively new technique for me - but people with more math knowledge than I have tell me that it's OK to run multiple split-tests side by side provided that they're unlikely to interact.

If I was changing the *angle* with either of these tests, that'd be a problem, because the success of the lander is then affected by how well the image works with the headline. But I'm still testing a relatively default headline and image to get an idea of what language and pictures work well.

I've got 4 images and 4 headlines running. Because that's a rather excessive number of landing pages to create, and to simplify future tests, I've spent some time editing the STM Mobile Tracker to hack in automatic cycling (and tracking) of two variables that I can use to rotate my image and headline. (More on how I did that later, if people are interested.)

I'm testing more amateur images vs pro images, and GIF vs JPG, as well as "adult" vs "very very adult". What I'm trying to do here is test my assumptions: that an animation will work better than a static image, that amateur images will convert better. I'm also testing what level of XXX-ness works best.

Oh god, my eyes.

Test 3: Different Advertiser LPs!

I've also added two more advertiser LPs into the mix, from AFF and CheekyLovers.

The AFF LP is a single-field form - email address - followed by the rest of the form fields on a separate page. My hypothesis here is that this'll tell me if the problem with AFF is "oh god, huge form" or "no way are you getting my email addy".

If it's a trust problem, my theory is that we'll see conversions go down, as visitors hit the email page, say "fuck that", and leave. If it's a tons-of-fields problem, they have less fields to fill in on page 2 and are now more committed, so I theorise conversions will go up.

CheekyLovers is a crazy one - it's just "short form" vs "longer form". I'm testing this to see what the drop-off is between short and long - and indeed if there is one.

One of my assumptions is that the shorter the form, the better the conversions. Using the two different forms controls for all other variables, so I should be able to get a solid test.

Test 4: Over-image button vs bottom button

I want to track whether clicks come from my over-image button or my button at the bottom of the page, as h0mp suggested.

My initial hypothesis here is that clicks on the over-image button are less committed and so drop out of the conversion funnel.

However, my initial coding for this was FUBAR, and I ran out of time rewriting the tracker for my LP rotation, so fixing this will have to wait until tomorrow.

And there we go! Any predictions? I'm going to let the test run for a decent period to pick up data, then review.

you have a double test 3 up there.


Anyway, i'm particularly interested in seeing how your LP testing turns out.
My prediction is that you'll be trying to catch and keep the visitor's attention sooner or later. I mean.. given the nature of your traffic, you'll have to produce a captivating transition from expecting to see pr0n to sign up for dating.

I have been working with redirect traffic from Adultmoda for quite some time now and if you just throw up a rules lander, you will lose a lot of your visitors just because they land on a page that they weren't looking for and I think it pisses them off mostly. Like.. they wanna rub one out but instead they are presented with rules and asked about STDs and stuff.

The easy way is to redirect them to a live cam site or one of the many paid tube sites or better even to your own white label.

I think your lander will only reveal it's true potential once you've been able to catch and hold your visitors and make them interact at least once with your page.

A 2 page setup will do the trick. On page 1 present xxx sluts images for visual lock followed by short copy below the fold that will 301 their minds from jacking off to fuck local sluts by tapping the screen wide button that presents them page 2.. which is your dating lander.

I guess you can eventually end up doing well with just a lander, keywords and targets... but ok, I think you get my point.

*Nods* That all makes a lot of sense. In fact, one of the next tests I've already been thinking about is an interstitial page doing just what you describe there - giving a slightly less jarring transition from "I want teh boobiez" to "sign up here". I think I'll pencil that in for the round of testing right after this.

Thanks!

(Also, I fixed the Test 3 thing. Derp.)

Shit this is shaping up nicely, good stuff Caurmen!

I would think that especially for mobile the shorter and least complex the signup process the better.

What's a Rules page?

What does AFF mean? ( Adult Friend Finders?)

1. Person browsing on their phone mistypes a porn URL.
2. They end up on the Rules Lander I'm using.

How is this done?

Mobile Tracker to hack in automatic cycling (and tracking) of two variables that I can use to rotate my image and headline. (More on how I did that later, if people are interested.)

Teach me! Teach me!

@MrGreen - Thanks!

@Sandyone - Rules Lander is the classic "Before you sign up to our site, you must agree to these rules: 1) No matter how many gorgeous women are begging for your bits..." lander. Poke around on any adult website and you'll find one behind an ad in a matter of moments.

AFF is indeed Adult Friend Finder. Thanks for reminding me not to go acronym-crazy there!

The flow of the offer, ending up at my lander, is how ZeroPark, the traffic source I'm using, works. It's nice - means that you don't have to produce a bunch of banners and can get right to the LP. On the other hand, it also means your LP needs to kick ass.

Mobile tracker hacking - coming soon!

Small problem with today's stats - none of the clicks went through to the offer pages.

Why? Well, obviously, because of a very technical problem. All to do with gigabytes and advanced coding practices and...

OK, I cannot lie. I was in a hurry yesterday and uploaded an old development version of the lander by mistake. So all through yesterday, confused pornhounds were being redirected to herp.com rather than my affiliate links...

That's now fixed, and I've actually, y'know, checked my lander.

ALWAYS CHECK YOUR LPS AFTER YOU CHANGE THEM.

And that's today's lesson - for me, most of all.

At least I'm still running on a testing budget, so I didn't lose too much money. I shall update with stats tomorrow, once they actually mean what they should mean!

Hey Caurmen, thanks for this follow along! I would like to know how you've tweaked nginx's performance? Or have you left it like it was out of the box? and what about php?
How fast are your LPs being served now?
Are you using a CDN or is everything on the box?

@refresh - Heya! I've mostly left nginx, MySQL and PHP at their default settings for now. That's mostly because I'm going to talk everyone through my optimisation process later in the follow-along once I have data, but also because, given I'm running a 2Gb box, I'm not very worried about either of them being catastrophically slow. 90% of the time in my experience you can leave nginx more or less as it is, apart from a couple of bits of security optimisation. PHP-FPM is a bit more likely to Eat All The Memory, but in this case I've got memory to spare and it's only running 10 processes.

The LP's loading in around half a second. Not bad, but there's optimisation room there for sure.

Everything's on the box. That was the point of all the ping testing I did before choosing a service provider - I've got about the best-peered service for the mainland USA I could find, so there's basically no point adding a CDN to that. This also fits with my usual "nuts to CDNs, just locate a server in the country" attitude

One day's testing and straight to statistical significance. I do like ZeroPark for that.

So, the text options ended up giving me CTRs ranging from 11.6 to 15.5. Three of the text options aren't sufficiently differentiated to be eliminated yet, but the bottom one is statistically significant against the top one (using this handy calculator), so I'm dropping it out of the rotation.

(The FAILHeadline, for those running similar tests, was "WARNING: Someone You Know Is Far Dirtier Than You Think - And This Site Has Pics")

Meanwhile, in the images, the GIFs are roundly trouncing the JPEGs. I've added the two GIFs and the two JPEGs' results together, compared them, and they're statistically significant, so goodbye to the JPEGs and hello to two new GIFs!

Interestingly, the larger GIF, at 91k, is competing fairly evenly with the smaller one at 50-odd k.

I'm changing the tests here in a fairly simple way: in order to not confuse my tracker data, I'm restarting my image tests with image numbers 5 through 8 (all gifs, including the two I tested before), whilst I'm just dropping the losing headline and compressing the headline test down to 1-3.

Next is to test which button is getting clicked, before I move on to testing a two-page setup.

Button Testing

Simple enough - I've added a new page called "buttonlp", which consists of

<? 

$newHandle = @fopen('button/buttonclicks.txt', 'a+');
@fwrite($newHandle, $_GET['subid'].',');
@fclose($newHandle);

require_once('lp.php');

?>

Monday morning update: loads of interesting data flowing in.

We have an answer on trust vs pain-in-the-ass-to-type: the AFF landing page which started with an email address got no conversions at all. Zero, none, nada. In addition, in a shocking development, the CheekyLovers long-form page actually converted better, albeit over a small sample size, than the shorter form.

I wouldn't have called that. So, it seems that trust is the issue, not typing.

In other news, the most disgusting, fluid-heavy GIF is massively outperforming the others (although oddly the least shocking GIF is doing second-best), to the point that after the weekend it's statistically significant against every other image. And a winner's emerging in the headlines too, with my slightly modified headline beating everything else.

(Incidentally, this is another good reason not to just rip-and-run with landers. In every case here, modifications of the absolutely standard lander are beating the standard one. )

Now, to the buttons.

Taking my comma-separated list of subids, the first interesting - and surprising - feature is that most of the clickthroughs are actually not coming from the button hovering over my image - about 1/6 of them are, in fact, with the rest presumably being from the button at the bottom. That's not at all what I expected - hurrah!

Now, to figure out which of my Subids converted. I opened a spreadsheet, stuck the subids that had clicked the button in one column, the subids that had converted over that time period in another column, and used

=ISNUMBER(FIND(CONVERTED SUBID;RANGE OF SUBIDS))

So, I've had a bit of a pause in this follow-along whilst I worked on other business stuff (launching films ate most of my week), but I'm back now.

After a bit of thought, I've decided to go with the top image and headline from my previous test. The image was statistically significant, the headline wasn't but was close. And by settling on these, I can split-test other stuff.

First thing - I'm messing with the "rules" text. I'm expecting some texts to produce a better CVR than others, as the "rules" are of course disguised sales points.

Because "rules" or body text in general are a big pile of assorted characters, I'm using a slightly obscure feature in PHP to make them easier to add and update.


Big Chunks Of Text As PHP Variables

Normally, when you're declaring some text in PHP, you'd use

$sometext = "blah blah wibble";

$sometext = 'blah blah "blah" wibble';

$rules = <<<EOT

<p>blah blah</p>
<p><a href="a thing">it's full of single quotes syntax</p>

EOT

<? echo $rules; ?>

<script>
function pageone() {

javascript:document.getElementById('firstone').style.display = 'none'; document.getElementById('secondone').style.display = 'inline';

}
</script>

<a href="javascript:pageone()">

No predictions.. but do you already know what type of devices make up for the majority of your traffic?
There are still a lot of phones that don't handle JS too well or support for it is turned off.. or something like that.

I came across this issue during an email capture campaign. The capture form was JS and I lost a substantial amount because of it. I ended up displaying a "Can't see the form? >click here<" link which redirected them to a PHP capture form. In the end, the PHP form made for 25% of the emails..well worth the effort.

That's a good point. I have been tracking devices, and I believe that most of the common ones have Javascript enabled, but I should check for sure. Thanks!

IMO If device can't render JS properly, chances are it will not convert to sales -> hurt quality.
Predictions: text+vanilla image + short rules win! donno why. Lets see what happens!

Hey caurmen. Friendly advice - i'm not sure how has it all go for you but you gotta have some sort of a priority in testing them because it looks to me like you have ALOT of variables on hand.. you were even testing buttons.. when buttons should be tested when you are optimizing your profit.

Not saying i'm 100% correct, i'm still learning it myself, but what i would do when i'm not yet in profit, i will test 2 things, lander template (could be same content), and offer. Once i find the killer offer, i start working on getting in my content (long rules short rules) etc, once i get into profit then you test buttons, headlines etc.

Other pros might have better insights.

But ultimately, i will look into a source that i can REALLY spend first, like maybe 100-300 bucks a day etc to speed up all the process..

@joejoechen - Thanks for the detailed comment! I'm doing this deliberately, don't worry.

If I was doing standard A/B testing I'd agree with you, absolutely - indeed, I've given people on STM the "no, seriously, test buttons LATER" advice more than once

I'm intentionally running multiple non-interacting A/B tests at once here (not a true multivariate). Because I'm doing that, I get to test a lot of different variables with the same spend, which means I get more data for the same money. (This is the same approach that I believe Landing Page Genius uses.) Technically I could actually test more at once, but the more I test the more I'd be in danger of ignoring hidden interactions between them, and I'm not ready to fire up the hideously complicated math to do full Taguichi method multivariate testing on this quite yet (although that'd be an interesting future experiment).

I'm still testing them in strict order of "what's likely to make the most difference". Headline and image first, then body copy, pre-pop and so on. I'm not actually convinced, from hundreds of split tests I've run, that design always makes a vast amount of difference, although it's on the list - after all, this is a new vertical for me and design may be more key. (Anyone familiar with adult care to comment?)

I've not actually been split-testing buttons - I may not have been clear there, and I agree with you, they're lower priority. I've been testing whether clicks from one part of the lander (before my sales message, essentially) convert differently to clicks from another part.

I know this isn't the usual way you'll have seen a lot of affiliates work, or that you probably work yourself. As this is a tech follow-along, I'm partially doing this to show everyone reading another way of doing things - and whether or not it works However, the math is sound and a lot of people outside the affiliate sphere use these methods to make a lot of money. Generally in the past I've run campaigns a more conventional way, so this is an interesting experiment for me! I'm also being very methodical in my testing because this is a new vertical for me, and so I'm gathering information on what seems to work not just for this campaign, but for future campaigns.

As for why I'm not spending more money - I'm spending enough to get to statistical significance for each test. I know this seems like a small spend, and certainly if I was using a classic banners->lander traffic source I would be spending more, but the beauty of ZeroPark is that the clicks go straight to your lander, dramatically reducing the spend you need for decent quantities of data. Spending more wouldn't actually be a very good idea - if I'm not making a profit and have already reached statistical significance on a test, I'm just throwing money away rather than gathering more data.

I'm reasonably confident that once I have a working funnel I can scale, but obviously I don't want to start scaling before the funnel works.

I'm very interested to hear comments on the approach I'm taking here, btw! It's a bit off the beaten track, and so feedback and suggestions are always useful!

@nefig - Interesting! Personally, whilst I know that my predictions are right about as often as if I made them with a coin-toss, I reckon filthy image + long rules will win. No idea which copy variant - I kind of want it to be my original one for ego reasons, but I suspect that the "standard" copy is standard because it kicks everything else's ass

Let's see which one of us is right! Anyone else?

Wednesday update - unfortunately payout on several of the offers I'm testing has dropped, or the campaign would already be comfortably in profit. As it is, it's about breaking even. Good news, particularly on a Tuesday. Next step is almost certainly to choose a single offer and pile conversions into it so that I can get a quality review.

Initial results aren't anywhere near conclusive for the various rules copy options. However, the pre-pop lander is already showing some very interesting results (I wasn't expecting them this early). It would appear, initially, that the non-pre-pop option (single-page) is absolutely kicking the ass of ALL the pre-pop versions!

That might be because of the Javascript issue, as mentioned above. I'm going to alter it so that if the phone doesn't work with Javascript, it doesn't matter.

Originally Posted by caurmen

[*]PHP-FPM connects to Nginx via TCP by default. That's a big slowdown. It's very easy to uncomment the line making it connect via a unix socket instead.

Friday Update

First and biggest news: the "front page" idea definitely doesn't work, at least not as I have it currently implemented.

The best "front page" variant I was testing has a clickthrough of just above 6%, and a signup ratio of 0.61%.

By contrast, the "no front page" option has a clickthrough of 23% and a signup of 1.25%.

That's after 1600 page views each.

I must confess, I'm really surprised by this - particularly after I checked the Javascript issue. But no, the front page doesn't work at all.

I may revisit it with a dramatically different design at some point, but at the moment - no dice.

In other news, my various "Rules" copies are still trundling away, with no statistical significance yet. Currently my home-made copy is in last place, with the short two-rule copy and the "standard" rules page copy competing for first place.

But the best news: this process works. Even with the front pages dragging the CTR and S/U down, Wednesday saw the campaign's first significantly profitable day, with a +30% ROI.

What test next? Well, it's time to test to see if all the tweaks have left me with a profitable campaign, or if more testing is needed. I've eliminated the pre-pop front page options, and I'm going to run the campaign over the weekend with just the rules lander text options changing. Let's see how things are going!

Thinking about the front page option some more, one thing stands out - whilst CTR and signup were both lower on the "front page" versions, the ratio between CTR and Signup was nearly double on the best front page option compared to the no-front-page option. In other words, people who I didn't lose at the lander converted a lot better after clicking through. That might imply that if I can find a much, much more "grabby" front page, it could significantly increase my signup ratio, just as h0mp predicted.

I think that might be something to work on after the weekend...

Hey caurmen. I think, it's also possible that some of the visitors were either bots or people with buggy/outdated browsers, so they failed to click on js lander.

Is the STM Mobile Tracker V 3-4 out yet? If so where is it? Thank you.

Hey dude could you please give a general data summary on how it's all going, I can't keep up with all your updates!!

Mr Green - No worries, I'm starting to get a little lost myself!

As a matter of fact, that comment prompted me to spend some quality time with my stats, which was a very good thing.

Currently, my best-performing variant has:

CTR of 25.5% (up from a base after "new banner syndrome" wore off of about 16%)
Signup Ratio of 1.53% (up from a base of 0.32%)
ROI of approximately 77% (up from an initial ROI of -22% - however, some of that data was before some of my payouts dropped)

My conclusion is that at this point, I need to send some more volume to the campaign to get a good picture of how it's going. My best-performing offer currently appears to be Adult Friend Finder mobile, but it's also the one with the highest payout, so to get statistically signficant ROI numbers I need more volume.

Hence, my next move is easy - turn off split-testing for the moment, and run a significantly higher budget on the campaign for a few days to establish a new baseline. After that, I'll have some really solid stats to share.

Notepad Post

Whilst I'm waiting for my stats to come in from the "more volume" test, I'm going to use this post to make a list of the stuff I should test on the lander next.

• Callouts - carrier, model, geolocation
• Everything from Maynzie's great adult post over here - color match, multiple images, older women

great follow along, I wanted to ask how do you setup multiple offers to a single lander. Do you set it up under a single campaign? Or do you have multiple campaigns in STM tracker in a single lander?

I use the offer rotation feature in the STM Mobile Tracker - it's very robust and does everything I need. I use a single campaign and rotate offers.

Here's a tutorial I wrote on the best way to set it up - section #3, under "With Rotation". I basically do exactly that!

Larger Volume Results

So, I've run the resulting campaign from all my testing for a few days with close to $100 spend a day - what were the results?

SPEND: $343
REVENUE: $429
PROFIT: $86
ROI: 25%
ROI Confidence Range at 95% confidence: 20% to 30%

So, from a completely standard "seen it all before" lander being run in the US market, the optimisation I've done so far has us clearly in profit over a statistically significant sample size. Hurrah!

The news here is actually even better than it looks. I had to increase my bids nearly 50% from my initial bid to get volume - but the new ROI calculation includes that huge increase. There's nothing to stop me from scaling this campaign a lot more.

Except... on the last day I ran this campaign, my AM asked me to pause the campaign on the primary offer. It turns out that the quality of leads wasn't high enough for the advertiser.

So the next question is whether the problem was the offer, the angle, some leak in the LP (although I've avoided obvious traps, like "FREE!") or even (although I think this is unlikely) the traffic source. I promptly moved to my second-best offer and fired a few hundred leads at that, and I'm currently waiting on a quality review.

So what's the next step?

It's going to depend on the quality report I get back, but my suspicion at the moment is that the angle itself, whilst it converts, doesn't lend itself to good quality. (It's the absolutely bog-standard "There are sexually explicit pictures of people you know on this site!" angle - any comments on likely quality from anyone else who runs it?)

Probably back toward the drawing board and testing a different angle, at this point - and most likely also testing in other countries too. All the tweaks I have listed in my "notepad" post above are still relevant, so those will be rolled in, and I'll probably test a battery of new images and headlines to boot.

But most of this is going to be, tech-wise, much the same as what I've done already, so I'm going to conclude this Follow-Along here! I certainly won't be leaving this campaign here, though - it's been a fascinating and profitable introduction to a new niche, and you can't beat the combo of learning and getting paid to do it.

Things I've Learned

• Running multiple A/B tests simultaneously is the dog's bollocks. Had I been doing things the old-fashioned way the optimisations here would have taken at least three times as long and cost three times as much. I'm actually going to go back to my established campaigns and set this technology up on as many of them as have the volume to give useful results. I strongly suspect I'll be able to resurrect a LOT of old campaigns that only barely broke even before. I may even test out using this approach on ads rather than LPs.
• Why having a single-jump traffic source like PPV or ZeroPark rocks. Rather than having two stages to your optimisation (ads and then LP) and rather than having a very slow drip to the second of those, these sources blast a crapload of test data straight at you, and if you're being clever about your spend and intelligent about your testing, the path to money suddenly becomes a lot shorter. I'm DEFINITELY going to run more ZeroPark campaigns in the future, and I might even stick my nose back into PPV (which I cordially hated last time I tried to learn it).
• Not so much something I learned, as something I had reinforced - neither total originality nor outright ripping are the way to go. Starting out from a (presumably) proven base - the rules lander in my case - and then methodically testing innovation is something that has worked for me before, and it worked very well here, again.
• Spending time on the tech pays off. I jumped into the most competitive adult market - the US - with no knowledge of the adult vertical beyond reading a lot of STM threads. Using a bog-standard lander I was immediately only 25% from profitability. That could have been for a number of reasons, but looking at my competition, I'm fairly sure that having thoroughly tested server response speeds, optimised my lander to hell and back, and generally followed best practise made a lot of difference. For anyone looking to benchmark - my lander loads in its entirety in under 150ms on wifi. I think for single-step sources like ZeroPark, that's a good speed to aim for.
• Work on sources with massive potential scale. Up until now I've mostly been a niche guy - smaller niche, less competition, right? And it's done me pretty well. But affiliate marketing works out as a simple equation: cash = potential scale / time to optimise. This campaign hasn't taken me a lot more time to optimise than a POF niche campaign, and it's pretty obvious that once I work out the quality problem, there's a potential upside in the hundreds or thousands of dollars.
• Assuming that the quality problem I had isn't universal, ZeroPark's a solid traffic source. It's got volume, it's reasonably priced, it's easy to use - it probably has rhythm and music too. Who could ask for anything more?
• I need to learn more about quality on adult. I'm pretty darn good at quality in mainstream dating, or at least so I'm told by my AMs, but adult's a different beast. Job #1 - review everything ever written on STM about adult and lead quality.
• Track which link clicks from lander to offer come from. I'm personally going to try very hard never to run an LP with multiple click-out options - like the button over the image and the button at the bottom - again without thoroughly tracking them. Different click locations, it turns out, can lead to significantly different signup ratios. Indeed, I'm probably going to try to craft my LPs to push people through the sales text more, even if it reduces my CTR.
• "Science: It Works, Bitches". This is the first time I've tried such a formally scientific approach to my optimisation: define problem, form hypotheses, develop test, run to statistical significance, analyse results, repeat. It works - progress might have been a bit slower, but the testing process never felt like it got stuck. Indeed, the entire thing felt like a tank - rumbling forward slowly and carefully crushing every obstacle in its path.
• Finally - doing a Follow-Along is great. I ran this Follow-Along primarily to demonstrate a technical approach to a campaign and give people a bunch of useful tech titbits to take away, but actually I feel I've gotten far more out of it than I put in. The exercise of sitting down, writing up results, and considering what they mean is incredibly valuable by itself, and having a community of hundreds of very knowledgable people then read and make suggestions, challenge assumptions, and encourage is absolutely fantastic. If you haven't run a follow-along yet, I strongly recommend it, and I also strongly recommend being as detailed as you can be without risking your valuable data - you'll get a lot more out of them. Expect more techie follow-alongs from me in the future!

Originally Posted by caurmen

I'm running two simultaneous split-tests, of headline and image. This is a relatively new technique for me - but people with more math knowledge than I have tell me that it's OK to run multiple split-tests side by side provided that they're unlikely to interact.

If I was changing the *angle* with either of these tests, that'd be a problem, because the success of the lander is then affected by how well the image works with the headline. But I'm still testing a relatively default headline and image to get an idea of what language and pictures work well.

I've got 4 images and 4 headlines running. Because that's a rather excessive number of landing pages to create, and to simplify future tests, I've spent some time editing the STM Mobile Tracker to hack in automatic cycling (and tracking) of two variables that I can use to rotate my image and headline. (More on how I did that later, if people are interested.)

He Amy, he is doing multivariate testing, where you can test multiple elements at the same time.

I know Caurmen is using some advanced math to find the best combos. He can chime here and give more details

Yep, bbrock's correct - what I was doing here was an extremely naive form of Multivariate Testing.

So, I was testing multiple headlines at the same time as I was also testing multiple images and multiple body texts.

That essentially gave me 3 split tests for the price of one set of traffic.

The way I was working here is not the best way of doing a multivariate test, but it's still surprisingly useful. Essentially, the weakness with the approach I took here is that I was ignoring all possible interactions - so I was assuming that Headline A would work equally well with Image A, B, C or D. That's not always the case.

Nonetheless, even this very simple approach worked pretty well, as you can see in the follow-along. I've used it subsequently in other campaigns and it's always proved to be pretty powerful.

There are much more advanced ways of doing similar testing. The next level of complexity is what's referred to as "Full Factorial Multivariate", where you test all possible combinations as separate landing pages: so Headline A / Image A / Body Text A is treated as one variant, Headline A / Image A / Body Text B is a second variant, and so on.

That approach allows you to detect all interactions between elements and sometimes to come up with very powerful combinations of elements, but the downside is that it needs a huge amount of data.

There are more sophisticated techniques beyond that, which allow you to combine the advantages of 0-interaction-assumed tests (the first type) and multivariate (the second type) - allowing you to determine the importance of elements as well as how each individual element performs without the damaging assumptions of 0-interaction-assumed. They're quite heavy on the maths, which is why I'm currently spending a lot of time talking to a programmer with a PhD in complicated mathematics.

(If you want to hear more about that work as it develops, you may see the link to "ZEN.ag" in my signature - that's where I'll be sharing that work and its results.)

Hope that makes sense! This is a big, hairy topic, but one with absolutely massive potential dividends for affiliates - definitely worth checking out.

Thanks bbrock and caurmen for your responses!

Caurmen - Your post has given me the motivation to learn more about MVT. So you were doing what's called "Fractional Factorial MVT", which if I understand correctly, can give more meaningful data than A/B testing (which doesn't take into account the dependencies between variables) but require less testing/cost than "Full Factorial MVT" (which tests all possible combos of the variates).

And what a life-saver this information is! I was getting pretty desperate in my search for an efficient test method when I came across this thread. Right now I'm struggling with my first campaign, in that sometimes I'm not sure what I'm testing or how I should interpret the data. Although I'm still managing to reach statistical significance here and there, and weeding out creatives and offers somewhat systematically, I'm aware that I'm probably spending several times more money than I'm needing to. I'm also having to spend a lot more time pouring over all the data to figure out what the implications are. (This is nothing special of course - just something noobs have to go through en route to becoming pro!)

My mission for the weekend will be to design a systematic test method based on the fractional factorial MVT approach, to come up with a system that I can continue to use and refine for future campaigns. I just signed up at your ZEN.ag site. Looking forward to what you're secretly cooking up over there!

Many thanks caurmen for pointing me in the right direction!

Amy