PPV Hosting Split-Test: BeyondHosting vs. Storm on Demand (22)
In my on/off experiments with PPV, I've always had a sense that I was somehow losing a good bit of traffic due to the inadequacy of my server/hosting/tracking.
About a year ago I switched to Storm on Demand Dedicated cloud hosting to see if there would be a noticeable difference, based on the recommendation of a well-known PPVer in the aff space. I haven't been doing much with PPV lately, but after being on STM for a while, I knew that this time 'round, I needed to approach the issue more methodically. I've been doing some testing on Trafficvance lately, and was getting some promising metrics on a potentially large-volume campaign. Trouble was, my P202 was missing about 25-30% of the impressions that Trafficvance says it was serving me. So either these users are so adept at closing out pop windows before my page loads, or my server/tracking just isn't doing its job. Enter the split test.
StormonDemand Dedicated 2GB vs. BeyondHosting 202 Optimized VPS 1024 MB
Beyond Hosting
- 1024MB RAM VPS
- Litespeed
- Prosper 202 1.7.2
- php 5.3.9
- msql 5.1.56
- memcache yes
Storm on Demand
- 2GB Ram dedicated
- Apache
- Prosper202 1.7.2
- php 5.2.13
- msql 5.1.56
- memcache yes
The Test
I took my promising campaign (1 landing page) and duplicated it on both servers. Set up a new TV campaign with 2 creatives, one for each host, and the same 1 target across both creatives.
Note: Images and JS in both instances were hosted on MaxCDN. Same exact pull zones.
RESULTS: CLICKLOSS
Beyondhosting:
TV impressions: 1635
202 impressions: 1396
Discrepancy: -14.62%
Storm on Demand:
TV impressions: 1584
202 impressions: 1285
Discrepancy: -18.88%
Conclusion:
29.13% more click loss with StormonDemand
Additionally, I did some load testing on each version of the landing page using Blitz.IO. I have very little understanding of Blitz, but the results look BAD in both cases.
BLITZ.IO RUSH RESULTS
I ran the default recommended rush test on each LP:
-p 1-250:60 http://trackingurl.com/landingpage.php
BH Rush: 49.21% of the users during this rush experienced timeouts or errors!
The first timeout happened at 28.86 seconds into the test when the number of concurrent users was 120.
Storm Rush: 86.27% of the users during this rush experienced timeouts or errors! The first timeout happened at 33.15 seconds into the test when the number of concurrent users was 139.
Conclusions:
WTF?! While I'm no technician, it seems clear that something is seriously wrong here!
My rep at Trafficvance tells me that 10% click discrepancy is the average. So I'm about 50% above that on Beyond Hosting, and 90% above that on Storm.
Initially I thought that by moving to Storm and throwing some dedicated power at it, I'd be fine, despite my lack of server know-how. (Mind you, I'm doing very small volume here, maybe a few thousand impressions per hour). But it seems obvious that even at these low levels, I'm still doing something wrong.
Storm's Support has been very responsive, but they never seem to know much about what I'm trying accomplish. I don't think the needs of PPV traffic are well understood by their average support person. BeyondHosting, on the other hand, is clearly dedicated to optimizing PPV traffic. So at this point I'm inclined to order their dedicated service and just say "You guys figure it out". But the $300-$500 / month is a large increase over the $100 I'm currently paying at Storm, given my non-baller status at this point.
=========================
Throwing this out to STM for whatever purposes it may serve. If anyone has suggestions or feedback, or wants to point out obvious flaws in my tests/methodology, configurations, blah blah blah, let me know. I'm still convinced that a lot of my campaigns haven't been profitable because of these vague technical issues...
Yeah , def 50% click discrepancy is huge.
On my dedicated testing using blitz.io I sent about 1000 users per second and lost only less than 1%.
When it comes to huge traffic generally it's not about the hardware but about the software and how you configure it.
And with Beyond you don't need to spend $300 a month.
With $100-$150 and a good combo of nginx+memached+apc you can handle pretty easily 250 connections / second.
Just ask them to fine tune your nginx and php-fpm configuration.
Hope this helps.
apache suck big time, what bbrock said is true, use the correct softwares to gain speed, i use varnish cache...haha in front of nginx
See your ticket! :-)
Your litespeed has not yet been activated with us, so its funny to say your above test were simply us vs lw with nearly the same software, and yet we still dominated without final tuning.
Getting the litespeed activated for you and then you can bench it again, should be vastly improved.
From what I saw on LI, your page load speed theoretically shouldn't matter since they don't fade your pop in until it has fully loaded. No idea about TV and the different toolbars though.
Re-run your test!
LiteSpeed wasn't activated. Re-running the test and will report back asap. Thanks Tyler.
Re-tested.
Click loss results:
Beyond Hosting: -14.36% clicks compared to Trafficvance reporting
Storm: -18.19% clicks compared to Trafficvance reporting
Difference: 26.65% more clicks lost with Storm on Demand
Blitz.io Test results:
BeyondHosting: 6,829 hits, 62 timeouts
Storm: 564 hits, 3,476 timeouts
Clearly the BH setup is better equipped for handling higher traffic volumes, and the difference in click loss between the two providers was fairly similar across both tests. BeyondHosting for the win.
Hmmm although I'd think for this to be a truly fair test you'd need to be running the same software on both ends and have both hosting providers specifically tune your service for what you're doing. Given that Tyler and BH seem to be more knowledgeable with PPV traffic that's probably not easy to do.
IMO though, I think StormOnDemand is a great choice for a beginner (like myself) running low traffic and perhaps non-PPV, since they're cheaper and the service is excellent; and then BH takes the lead when it comes to high volume traffic and top-tier performance. Just my 2c.
Well, the point of my test wasn't to determine which provider would perform better with the same exact software under the same exact conditions. It was to compare my current setup, which has been tweaked several times by Storm, to Beyond Hosting's P202 optimized service. Despite my several attempts, Storm support never seems to really know what I'm trying to achieve, or how to achieve it. So I don't want to waste more time with them, I simply want to find a better solution to my dropped clicks.
Storm seems fine for organic or PPC traffic.
Originally Posted by brianb
Originally Posted by brianb
Just to be fair here, the first test you ran was apache on our side.
So out the gate our service outperformed storm, then after some further configuration on our side with litespeed we were able to really outperform storm.
@Zeno why would you start off on a low performance solution?
Money I suppose. Although now that I know a bit more about hosting, I look at your CPV/Prosper plans and realise that Storm isn't really any cheaper at all, and you cant compare specs to specs to decide which is better... especially when BH has three different hosting sections with different options tweaked for specific things.
Will probably move when I start running more volume, just cant be assed at the moment. On the bright side Storm charges by the hour so it makes it easy to pull out without losing any precious moneys.
this was an awesome case study....thanks to brianb for running this
Where are the data centers for each server?
On either test below, I saw 0 timeouts. Am I doing something wrong with the test?
I ran this test
This rush generated 6,784 successful hits in 1.0 min and we transferred 21.56 MB of data in and out of your app. The average hit rate of 108/second translates to about 9,396,617 hits/day.
-p 1-250:60 http://landingpage
Ran it a second time on another landing page, same server.
This rush generated 6,395 successful hits in 1.0 min and we transferred 30.43 MB of data in and out of your app. The average hit rate of 103/second translates to about 8,901,024 hits/day.
Another +1 for BeyondHosting. LiquidWeb is terrible.
Well in my opinion if you are serious in affiliate marketing, you just have no way but to "invest" in technology to ensure it doesn't contribute a factor to lead loss on the campaigns that you are running...
I started with Hostgator shared hosting (coz it is freaking cheap), and until some point, I realized that I need to move to a proper server, so I changed to BH Prosper202 Starter, and I have A LOT OF data in it. Recently, I'm moving to their Prosper 1024CLM solution (on a new tracking domain) as I am slowly getting more volume on my campaigns, and I wanna ensure technology part doesn't give me any problem...
The thing is, the shifting is a pain, maybe it's only because of me but to my experience, while I was shifting my Prosper202, I had to PAUSE all my profitable campaigns and resubmit them... and this process eventually took my time on profiting with my current campaigns...
I should have started with a decent server that can handle the traffic when I know myself is going to be one of the TOP affiliates out there... Furthermore, you can get 50% off with BH with the coupon shared here - STACKTHATMONEY ... So I really see no reason if anyone who isn't on a good hosting, not to shirt to beyond hosting...
Just my 2 cent.
*disclaimer - Tyler didn't pay me anything, but this is solely my personal opinion.
Originally Posted by kokofai
Originally Posted by getzlaf15
All of our gear is here in Cincinnati, Ohio.. were connected very well to Chicago, Dallas, Atlanta, and New York.
Originally Posted by hd2010
Home > POP / PPV / Redirect >