Click Loss and Networks - What You Should Know! (22)
Let’s talk click loss.
Most of us know what it means, some of us understand where it comes from, and it affects all of us.
Click loss, to put it simply, is where some clicks don’t make it from A to B.
Now, some traffic sources over-report clicks, some under-report and this is a different problem – but nevertheless, some clicks are truly 'lost in transit'.
There can be several reasons for this and the major contributor is generally network-level:
Long loading delays (network latencies), multiplied by many redirects, and supplemented by timeouts, DNS issues, slow user connections, and so on.
Read: when users click your banner and are redirected to your tracking link, some users fail to ever make it to the tracking system.
The same rules apply for tracking system > affiliate link and this is often where you get the largest loss.
In some cases the networks/offers also use javascript straight away to redirect and do fancy shizzle, which will bleed anyone who uses NoScript (like myself) or who has javascript disabled.
Why is this important?
Because every click that doesn’t make it through to B is a potential loss of revenue to you.
If you are spending $10,000/day and you have 5% click loss from traffic source to offer, that’s $500/day you are effectively throwing away.
For tracking systems – if self-hosted, the general rule of thumb is to have the server as close as possible to users.
CDNs won’t help with reducing click loss to tracking systems (CloudFlare might have a small positive/negative effect due to the way it operates).
Other systems like
But affiliate networks suck at this!
Many use Cake – and from my testing it appears these networks operate with only a single server (or a set of servers) in the US, or in the UK/EU.
Well shit, what about the rest of the world?
Times are tight young Timmy.
Networks may be dealing with 6 to 7 figure daily revenues – and they have budgets to fill up, but likely don’t really care, since they aren’t directly affected by your loss of clicks.
But we could all make so much more money if less traffic was lost into the ether.
To this end, I am right here right now going to create “Project Mercury”, which is just a codename for my personal vendetta against this unintelligent tripe.
It will have to involve pitching to networks to see if I can convince them to make moves here. But that’s a topic for later discussion.
Affiliate Network Testing
For now – let’s have a looksee at global performance of affiliate networks.
I have done some rudimentary tests that look at global response times of affiliate network tracking domains.
These tests didn’t measure the time taken to get from A > B, only the time taken for a monitor to look up the tracking link domain and resolve to some address.
So, this data shows nominal redirect/load times at best. Real world values will be much higher.
I couldn’t easily test real timings due to that pain in the ass called geo-redirection.
Tests were conducted with CA APM Cloud Monitor - http://cloudmonitor.ca.com/en/ping.php
This provides measurements from about 100 locations and gives min, avg and max round trip times (RTT) as well as the IPs resolved.
5 consecutive tests were carried out and the average RTT data averaged to provide plots.
The plots show number of results that were within certain latency brackets (so the y-axis is number of results).
The full Excel sheet of data is available here: https://www.dropbox.com/s/7w4ocwhhkr...ison.xlsx?dl=0
Lets’ first have a look at the resolve times of some websites:
Google.com
Wow. Slick. As expected, Google.com responds very quickly from virtually anywhere on the planet. Not surprising given they have one of the most comprehensive infrastructures in existence and deal with a large % of the world’s internet traffic.
Facebook.com
Not as flash. Why? Probably because Facebook has a relatively limited number of data centres – they are very complex as well, and large, so it’s probably not cost effective to make a datacentre in Thailand or South Africa for example. I also suspect that these results are not indicative of Facebook's performance when supplying content - all the images and static files served to you on Facebook will be delivered through Akamai's network, so are very fast.
Let’s also look at some CDNs – note these values don’t take into account throughput of the CDN that will effect page load speeds.
Amazon CloudFront
Very nice.
Rackspace Cloud Files
Even nicer.
The results for Rackspace are similar to Google. Rackspace uses a small portion of Akamai’s network of PoPs and Google likely rivals Akamai (or has partnered with them) in terms of global latencies.
Amazon is newer and has fewer PoPs, which likely explains the slight shift to higher latency.
Conclusion? If you have to pick between Rackspace and Amazon, Rackspace has an edge but consult Cedexis first (http://stmforum.com/forum/showthread...ty-data-Part-2) for country-level advice!
Now, let’s get on to networks.
First up: CAKE-based networks – Adsimilis, GrabAds, Neverblue, F5 Media.
Adsimilis:
GrabAds:
Neverblue:
F5 Media:
For your enjoyment, here is a column graph showing the above response times from F5 by geographic region:
Now shit, this is a lot different to a CDN, right?
From a performance internet marketing standpoint, this is bad.
Remember – these values are nominal, so real world latencies will be worse due to the time taken for the network platform to decide where to send you next, as well as the multiple redirects that may happen after that.
From this data we can conclude several things:
1. Adsimilis shows a different latency profile to the other three. If we look at the actual country-level data in the Excel file, it is quite clear that Adsimilis’s tracking server is located in Europe. Much lower European latencies, 80+ ms to US locations. I think US – UK – Western Europe are so well connected that the latencies here won’t matter too much.
2. Neverblue, GrabAds and F5 Media use servers based in the US.
3. Because of the countries/servers involved in the tests, the EU-based Adsimilis showed a lower average latency to all servers. You should look at the Excel file if you want info on a specific country…
4. Three networks resolved to one IP only – so there was no geographic load balancing. Neverblue however resolved to 3 IPs but these varied randomly, suggesting it was simple load balancing between servers in the same location. This is not surprising since Neverblue undoubtedly does the highest volume of them all.
On closer inspection, we can see very low latency results which basically identify the server locations. One is in Dallas, one is somewhere around Ashburn, another somewhere near Philadeliphia.
Looking at AWS server locations (CAKE marketing uses AWS), we can conclude Neverblue load balances across servers in Dallas TX (US central/south), Ashburn VA and probably Newark NJ.
So load balancing is possible, but surely there’s little advantage of this vs. several servers in the same data center, considering the US latencies are mostly sub 50 ms. What about Europe and Asia??
It really surprises me that CAKE – which is marketed as being blazing fast, great for mobile etc., hasn’t pushed for delocalised hosting of tracking systems in more than one continent.
Let’s look at some others:
Matomy:
1 IP, very likely in Amsterdam
iQU:
1 IP, also in Amsterdam
Ad4Game:
2 IPs, randomised load balancing between two located in Ashburn VA (USA).
Clickbank:
4 IPs – two servers somewhere close to Philadelphia in the USA, one server close to Boulder, another which is also close to boulder (so likely load balancing between East and Central US).
Glispa:
Couldn’t test properly due to tracking domain bouncing pings, but other tests suggest one server location somewhere in Western Europe.
Now you might all glaze over a little at this data, but I think it is quite frightening.
Let’s look at data for F5 for example in Thailand:
That’s a minimum of 250 ms before the test server had even found the destination server.
For a user, that’s undoubtedly going to take longer, let’s say 300 ms on a good day with a great ISP with excellent routing.
Now it takes 50 ms for Cake to respond with the next destination.
Cake returns a header to the user and says go here > that takes another 150 ms.
We are now 500 ms in and best case have the destination of the advertiser lander, time for another lookup and probably another 200-500 ms before the user is even loading a page.
Nearly a full second has gone by and we haven’t even started loading a page!
This is not good considering mental context switching often occurs at about the 1000 ms mark.
Now imagine if you are on a mobile device.
They are slower and if you are connected via carrier then there is additional overhead – in the US, you can expect an extra 100-400 ms with AT&T or Sprint depending on if you’re on 3G or 4G.
There is something called the “time to glass” on mobile and Google/app developers aim to break the 1000 ms barrier.
Well we’re certainly screwed.
And in India, Indonesia, Egypt and South Africa, 40-70% of internet users are mobile only.
Want to run mobile in Thailand? Let me know how many clicks vanish into the ether from your traffic source.
What the hell is the point of this report?
Basically, I just wanted to let people know what click loss is, where it can come from, and why you should have very little faith in affiliate networks in this context.
Running 100,000 clicks a day in Vietnam? Have fun – many of those clickers are going to wait several seconds on a blank page being redirected.
How can we improve the situation?
Firstly, sort out performance on your end.
Use CDNs to deliver static content fast. If you have an HTML-only lander, for god sakes if you are working in a geo far away from your server – or using cloud-hosted tracking, put the entire lander on a CDN. You might even benefit from
doing this if your server is US-based and so is your traffic.
Tweak your server to be blazingly fast and get A > B as quickly as possible. If you are a server noob, running volume, and have a self-hosted tracker, then for god sakes hire a server admin.If using
Lastly, let me work on my pretentiously named Project Mercury and see what happens.
Who knows, in a year’s time we might find networks pushing the bleeding edge on click tracking speeds.
I know exactly how they could do this too – but it will cost them (will it be worth it?).
Love it, use cloudfront or rackspace boys!
Amazing case study Zeno!
All the data laid out nicely, it's obvious you've got a PHD 
As you said, while we can't improve redirect speeds on the network side we should max out the variables under our control ( our servers / CDNs ).
Zeno for president.
For certain geos I tend to check for click discrepancy. It's one of the first things I do. I have a setup which makes it all very easy. This also shows me the results by carrier.
This data (especially the carrier data) is highly interesting when compared to a campaign's results / data.
... and this is why zeno has a Ph.D.
Whoaaaa another cool guide again today, Great stuff Zeno
great stuff. so where's the noob guide to hosting your LP on Cloudfront/Rackspace? 
I would be really curious to hear what discrepancies you guys have when comparing the impressions served by your tracker and the impressions in the adnetworks. For Asian countries Im seeing it sometimes as high as 20%!
This is really interesting especially since I've recently be buying up a lot Asian traffic.
P.S. Zeno, I want to take you to a casino. We will start an STM Blackjack team. MIT will have nothing on us.
This is gold dust. Absolutely incredible work.
Networks please note - the first one of you guys to run a local server in East Asia is going to clean up bigtime, based on this data!
Honestly, it's almost enough to make me consider running my own offer in Thailand or similar - 800ms or more of load time is a hell of a competitive advantage to have and you could achieve it just by localising a server.
@Zeno - it would be interesting to see these tests for some of the Chinese networks! Fancy doing them too?
Also, how does Google's Play Store do on these tests?
Originally Posted by Mr Green
AND he actually entered the online marketing industry afterwards. He had a pretty nice success selling one of the largest online lead generation companies for a handsome amount several years back.
http://mashable.com/2007/10/11/washi...courseadvisor/
I am a newbie. Can you explain clearly what is known as 'Affiliate network links having 3+ redirection steps to get to the offer page' ?
It's when the initial affiliate link redirect users to URL A ---> which redirects users to URL B ---> which redirects users to URL C.
Redirects take time so having more, all else being equal, will increase how long it takes for users to get to the offer page.
what happen if the merchant change their affiliate network or affiliate link system when we setting up redirects for our affiliate links?
What is the difference between url cloaking and redirecting?
Could you please explain clearly what is known as round trip times (RTT) ? and what is the difference between url redirection and cloaking?
Hi iAmAttila,
I am following your blog for more than 6+ months and new to STM. Some of these thread so difficult to understand, As a newbie where do I need to start, I would like to see my first income $10 or $1000 I hope that will motivate me continue this business as full time, please advice me ASAP!
Great article.
I was wondering why I would send Neverblue (now globalwidemedia) 20,000 clicks, and yet only 14,000 would show up (for example).
Please edit your posts or ponder a bit longer... 4 posts in a row is righteous self-bumping lol.
Originally Posted by affiliatelk
Originally Posted by affiliatelk
Originally Posted by affiliatelk
RTT = how long it takes for you to contact something and have it contact you back.
Originally Posted by affiliatelk
Read your guides...You simply rock!
Don't know how I missed such a valuable post until now!
It's an old post, but most (if not all) of the information is still relevant. Now I know where to direct people that are asking about clickloss - what it is and how to minimize it. Thanks zeno!
Amy
Home > Tracking Campaigns > Tutorials, Tips and Guides