CDN Wars - Case Study with Results Inside (15)
In the past, when running big campaigns I've always been unsure what would be the best way to go to handle all the volume.
As we know every second counts and it could make or break your campaign.
After I tweaked my servers for optimal performance I thought about using a CDN to improve load speeds especially for non US campaigns.
For the non technical guys over here, a CDN ( content delivery network) according to Wikipedia is :
"A CDN is a large distributed system of servers deployed in multiple data centers across the Internet. The goal of a CDN is to serve content to end-users with high availability and high performance. CDNs serve a large fraction of the Internet content today, including web objects (text, graphics and scripts), downloadable objects (media files, software, documents), applications (e-commerce, portals), live streaming media, on-demand streaming media, and social networks.".
Basically it copies your files across many servers in different GEOs and when a user requests a file it serves it form the nearest server to the user.
So normally it improves the speed a bit ( ideally ) if your original server is too far from the user ( a different continent for example).
Everyone I talked to seemed to agree that it helps with load times so I signed up for one and ran with it for a month or so.
However being the curious person I am, I wanted to see REAL DATA about how much it improved ( if it did ) or lowered my ROI.
Also I wanted to split-test between the top 2 CDNs I heard were the most common to find out which was the best.
For this test I wanted to run 2 different campaigns, one for the US and one for Europe.
The US one because my server is in the US so wanted to see if a CDN made a change at all.
Europe because serving content from a European server ( the CDN in this case ), theoretically should be faster.
The 2 CDNs I used in this case ( feel free to suggest more , I plan testing more in the future ) were MaxCDN and CloudFlare.
After setting up the CDNs ( fairly easy ), I created the campaigns so it would rotate between three landers :
The first lander was hosted on my server.
The second using MaxCDN
The third using CloudFlare .
After couple thousand dollars spent here are the results.
For the US campaign :
For the French campaign :
So basically the results are pretty clear.
If you have a good server and tweaked it for max performance, a CDN in the US wont' be a huge improvement.
Between the 2 CDNs CloudFlare was the winner.
For campaigns outside the US ( always if your server is in the US) , the use of a CDN can help.
Also in this case my server is pretty good so even for the french campaign the difference wasn't huge.
However for most affiliates, I think using a CDN could help them squeeze and extra 15-20% in ROI.
As always , feel free to ask any questions you might have.
Thanks for the case study!
Do you think that the results should be the same both for mobile and web traffic campaigns ?
+1 for cloudflare!
What about rackspace?
Originally Posted by dario
The CDN will impact the campaign the most if it's mobile and non-US since for mobile speed is key.
Originally Posted by xpatwork
Might be a good option for round two if it's easy to setup.
I understood that Rackspace CDN (formerly Akumai) and Google are some of the best - may be worth checking. The LeadPages services uses google CDN and have been pushing that as one of their unique advantages (you have the hosted by them on google option).
Thanks for the data. I've tried CDNs a couple of times and never became convinced it was helping any, so any data helps in the decision making process.
Awesome case study! Your results match pretty closely with my experience.
I would be interested to see how Cloudflare does in other regions like LATAM and Asia. Harder to get a really good server there in my experience.
I saw speed/latency comparison CDN tests somewhere, and results vary per region depending on how particular CDN provider is set up. So Latam, APAC, Europe performance will be different per provider
I use Cloudflare... with custom settings.... a good solution when you're sending a lot of traffic .... not so good with less traffic
I am not sure 499 vs 439 clicks is significantly different,
1 important factor in split testing is split the traffic correctly, means 30% of the times send to landing page 1, 30% percent to other landing..
And not 100% to one landing for few hours, and after few hours sending 100% for other landing.. I am not sure how you test it, but this can influence the results more than what you actually testing..
(I used CDN before and but have zero experience compares to BBrock, that's just my two cents.. )
In total there were sent close to 1 million clicks. The test lasted about 10 days.
Hope this answers your questions.
I am not sure that even 499 vs 439 conversions are significantly different,
But for 1 million clicks over a week is probably enough,
Just to clarify my point was that split test not in parallel can add noise that we can't even notice its there, for example for certain hours we got more young population or more women and don't even think about it so when we see difference we say yea that option is better but the real lesson is missed.
Originally Posted by ropulos
Anyways nice test bbrock, sadly I've been noticing similar results with rackspace, I was of course hoping to see a bigger difference.
I think this test is great, definitely more of a practical approach.
Obviously speed plays a huge part, so perhaps split testing how fast everything loads under stress (say 1000 clicks in a minute) from different regions would make more sense..
I use MaxCDN and I (hope) it has been helpful, however I only use it to host CSS, Script and Image files.
Home > Hosting, Servers & Security >