Check Whether Your Server Can Handle Your Traffic (33)
You're running a great campaign. ROI's through the roof. But for some reason, whenever you try to scale it up, all your visitors stop converting.
Is it because of your server?
Here's a way to tell for sure!
Stress-Testing Your Server With Blitz.io
Before You Begin: The entire point of stress-testing is to see whether your server will crash, freeze up, or otherwise misbehave. As Mr Baffoe points out below, that means that during testing, there's a good chance your server will crash, freeze or misbehave! Test when you have minimal traffic, or better yet, after you have temporarily turned off your campaigns. Otherwise it can get expensive fast!
Load up Blitz.io and hit "Free Sign Up". It will offer you a few ways to sign up, either with an email address, a Google or a Facebook account - use whichever's the most convenient.
Then, you'll end up on a page with this at the top:
Click in that text box, and type:
--pattern 1-250:60 URL_TO_YOUR_LANDING_PAGE
--pattern 1-250:60 http://www.foo.bar/LP.php
Give you an error.
Authenticating with Blitz
No problem. Blitz is just asking you to prove that the site you're testing is your own - as otherwise, you could use their service to launch Denial Of Service attacks!
Here's the page you're probably looking at:
It's written in Hardcore Techie, but it's actually pretty easy to get past even if you have no idea what a RESTful server is or why you'd want one. Just create a text file, type the number they ask for (usually 42) in it, and save it as the filename they ask for (the big string of letters and numbers.
Now upload it to the base (root) directory of your web server, so that you can browse to it at http://your.server.com/FILENAME.txt , where FILENAME is the name you just saved it as. If you do that, you should see "42" and nothing else on your screen.
Go back to Blitz.io and hit "Run" again, and the test should start up!
The Results And What They Mean
You'll now see a live graph of the response time of your server as the number of users hitting it goes up. If all's well, it should stay roughly the same. If it starts shooting up at some point, that shows that your server's becoming overloaded - it's time to optimise or get a more powerful server if you're going to need to serve that many users.
(Want to know how many users you NEED to serve? Read this article here.)
At the end of the test, you'll get a report on how it all went - which you can refer back to after optimising your setup. That report will also show if you had any timeouts - if you had more than .5% or so of the requests time out, that's also a problem that you'll need to fix!
Otherwise, you're all good to go!
Tips And Tricks
- If you want to test with more than 250 users, simply replace the "250" in "--pattern 1-250:60" with the number you want to test. This will cost you more credits. You can check if you need to run more than 250 users by following this article.
- You get 10 free credits to start with Blitz.io - that's enough for a fair bit of testing (particularly if you sign up with multiple Google accounts, *cough*). Additional credits are a very reasonable $1 per credit or so. A credit will let you run a single 250-user test.
- Remember, if you're testing a live landing page, you'll ALSO have real live users hitting the page at the same time. For a cleaner test (and to avoid wasting money) pause your campaigns, do the test, then restart them.
- Depending on what tracking software you use, Blitz's test may also show up in your stats -so remember to discount those visits.
Testing Prosper 202
Blitz doesn't quite simulate a real user - it only fetches the single page you give it, not the images, stylesheets and CSS on that page. Normally, that shouldn't be a problem - the PHP, database calls and so forth of the main landing page are what stress your server out. However, there are a couple of gotchas.
If your tracking software uses javascript to do its tracking - like Prosper - the Blitz test won't stress-test that element of your lander, which means you'll probably want to test that seperately.
Because of the way that Prosper works, that's a little bit tricky to do.
- Find the code that Prosper asks you to insert at the bottom of your landing page. From it, find the number right after "lpip="
- In Blitz.io, run a test as above on the following URL: Code:
http://YOUR.TRACKING.URL/tracking202/static/record.php?lpip=THE_NUMBER_YOU_JUST_FOUND
That will stress-test your Prosper installation's landing page script - which is the script that will get the most traffic.
To test your outbound redirects (which shouldn't be the bottleneck, but nonetheless are worth verifying) is somewhat simpler. Simply create a new campaign in Prosper, and use the "affiliate URL" of www.google.com. Use Prosper's Get Links option in Step 7 to get a direct link to that test campaign, and use that URL in your Blitz.io test.
In testing, I've found that Prosper seems to handle a slower ramp-up of users much better - and this also more accurately simulates real server load. To run a slower ramp-up test, replace the "60" in the query you're using with a larger number - this is the amount of time in seconds the test will run over. So, for a 180-second ( 3 minute ) test, run
--pattern 1-250:180 http://www.foo.bar/LP.php
ADVANCED - More Precise Testing
If you want to get really granular about your testing, you can actually test all the static content (images, stylesheets), your database, and your LP all at the same time. It's a bit more complicated, though.
To do that, use the following string in Blitz:
-v:url [LP.php, DATABASE.php, image1.jpg, stylesheet.css] http://your.site/#{url}
If you're doing this, you also need to adjust the number of visitors you test. There's a simple way to do this: multiply the number you already have from this article by the number of individual items you're testing, then divide by six.
So, for example, if Bob (from the example had a landing page with 10 items included, he'd take his 105 users, multiply by 10 for 1050, then divide by six, leaving him with a test needing 175 users.
Don't JUST do this test - but it's a good additional test to do to see if it uncovers any problems you didn't see before!
Confused? Have a suggestion or something I didn't cover? Let me know by commenting below!
(Big thanks to Mr Baffoe for his suggestions and clarifications below!)
Great content and info
Here are a few important things to watch out for:
It's VERY possible to overload and kill your server with Blitz.io when you run a test, do it when there are no campaigns live, or late at night when it won't matter as much. Because any down time could turn out to very expensive for you.
The above example tests your landing page. To test your redirects use a link from step 7. I'd recommend making a new campaign that sends traffic to a domain you own or google.com
Also there are a few other variable you can play around with.
-T will set how long in ms to wait before timing out the default is 1 second.
-r chooses the region to send traffic from. The current options are (california, oregon, virginia, singapore, japan, brazil or ireland).
Note: The region names seem to be case sensitive.
This can be used to stimulate the response times of your site based on where traffic is coming from. e.g if the majority of your traffic is coming from Asia you may want to use Singapore or Japan.
Here's an example with a 5 second time out running from Virginia.
-p 1-250:60 -T 5000 -r virginia http://www.foo.bar/LP.php
The other tips are also great! I only realised today how many countries Blitz.io actually have servers in - they cover Asia, Europe, Australia and South America, making them a decent first port of call for testing international response times too.
We're going to be getting into speed testing and optimisation next week - there's some fascinating research out there.
Blitz is a pretty poor test honestly. its all from a single source, you may as well just run apache ab.
Tyler - I'd argue it's better than ab, if only for the options. However, I agree, it's not ideal. Personally I feel LoadImpact is slightly better, but only on its paid tests - which aren't cheap at all.
What load testing software/site would you recommend? If there's a much better one out there, I might just rewrite this entire tutorial with it!
You will want to checkout software like Jmeter http://jmeter.apache.org/
Ah, yes - I know of JMeter, and like what I've seen of it, although I've not really dug into its full power.
( It's wicked user-unfriendly from what I've seen, though - might as well have a "if you don't speak at least 3 programming languages, don't even bother downloading this tool" button on the front page 
)
If there's a demand for a more advanced tutorial on similar lines, I'll give JMeter a thorough test for that.
Thanks for the tip! Do you use JMeter at Beyond? It'd be interesting to see what practical use-cases it has.
Yay I crashed my server!
What's that quote... "The breakdown comes before the breakthrough."
Heh, yeah - crashing your server is more than a bit likely when doing stress-tests Hope it comes up again OK!
Originally Posted by caurmen
Hope it comes up again OK!Anyway when I explained to BH that I was running tests and messing with server stuff without really knowing what I was doing and intended to keep doing so, they were happy to help with all the stuff I was asking about. Kekekeekke.
Hi caurmen is there anything we can do to prevent a Denial Of Service attack ? i think some one has done Denial Of Service attack on one of my sites
Yep, there are a few things you can do.
What web server are you running? If you're on Apache, check these two tutorials: http://stmforum.com/forum/showthread...Apache-Servers and http://stmforum.com/forum/showthread...-LiteSpeed-Ish). They'll help you secure the most obvious flaws in Apache that make DOSing it easy.
After that, you'd probably want to look at installing iptables on your server, and setting up a rule to limit DOS there. That's not a very simple thing to do - I'll confess that iptables confuses me from time to time - but there's an introductory article over here: http://blog.bodhizazen.net/linux/pre...with-iptables/ (tbh, this is probably something you want to hire someone to help with.)
thanks caurman im running on a linux vps with nginx do you still think the tuts will be good for me ?
Aha, right. In that case, the Apache tutorials won't be useful for you, but setting up iptables still definitely will be.
I'm going to be writing a tutorial about reducing Nginx's memory usage soon, but the short version is - check that your php-fpm setup isn't using up more memory than you have available. To do that, run a blitz.io test on your server, and use the "top" command at the command line of your server to watch the server's memory usage.
If you are running out of memory when you do that, change the pm.max_children in your php config file (usually /etc/php-fpm.conf) to a lower number. You'll also want to try reducing pm.max_requests .
That should get you off to a good start! For more suggestions, I'd recommend Googling "Tuning php-fpm", or alternatively wait a few weeks for my Nginx tuning tutorial!
I am pretty new here as you will see I have like 1 post . However isn't the easiest way if you do not know too much about servers to optimize nginx and php-fpm when you just check the CPUs you have available. Normally low-end VPS will have like 1 max 2 CPU cores and will also have like 256 MB RAM maximum. If you are on normal RAID anything with SATA drives which is pretty much everywhere just set:
-- nginx --
worker_processes # -- should be the number of CPUs cores you have
-- php-fpm --
pm = dynamic
pm.max_children = # -- this number depends on too many things but it can be double the nginx workers, dynamic means there will be one process running at least but these may go up to whatever you set here.
If you are on SSD or FlashCache enabled VPS this is a completely different thingy. I have one of those running a dozen of Wordpress websites, 256MB RAM 2 CPU Cores and SSD storage. This handles more concurrent request than a dedicated server with ATOM CPU and 1GB RAM.
Hope this helps. I noticed you are writing a tutorial about servers speed optimization I will be happy to help with that if you need any help . Considering your posts you are guru in that area.
Thanks for the post!
Yes, indeed, if you're running Nginx, that's a reasonably sensible guideline, not least because other things tend to bottleneck nginx servers A good tip!
As you say, the only real way to figure out your optimal php-fpm max_children is testing. Annoying but true!
(For anyone wanting to know more about tuning Nginx, there's a great tutorial over here.)
Hey,
Thanks man this tutorial is just great. 12k hits per second this will require some resources . Will test it and if I can get better results in low end VPS or at least on one of these I am creating will get you results.
However the creator seems to like BSD and that combined with nginx, php-fpm and memcache is some nice setup. The unfortunate thing is that you cannot run BSD on openVZ and it does not work fine on XEN so KVM or stand alone dedicated servers only. That increases the price a bit and if you have to pay for maintenance, managed server with that will be pretty expensive. Not too many hosting companies offering BSD as operating system out there. If you have used before any managed hosting with BSD on it please do share the info .
I must admit, I've not touched BSD in 20 years. I tend to use the rule that if I can't run it on a Debian / Ubuntu server, I don't want to know Too little brainspace for all the UNIX variants out there.
I'll be very interested to hear your results on a low-end VPS!
I have set a vps for the test. Let me know what stats do you need with which software. I am not really good with the AM software but I am eager to learn.
Thanks
Originally Posted by seowork12
Originally Posted by BeyondHosting-Tyler
CloudFlare is CDN. It is true they say it will protect against some malicious but me personally I do not believe that.
This depends on a lot of things and in any case bad coding cannot be protected with CDN it serves only the static content and if you have problem with a script coding security it will be there and exploitable even behind CDN. There is no ultimate protection but still mod_security is good for malicious attacks. The price you pay for the security check is a few milliseconds slower loading speed and this may cause real problem on server with high access rate and not too much resources.
As for the DDoS if it is well organized attack and the person knows what s/he is doing the only option is hardware firewall installed infront of your server. If you know how to check what is happening and the attacker is not experienced and uses method which creates identical entries in any log on the server ( different than a regular access entry for the service against which the attack is initiated ) CSF is pretty good. This also works for bursts of requests from small number of IPs it can be set to monitor for lots of patterns. It is not complex just monitors one log and blocks IPs which initiated a connection with certain pattern. As any system this requires some tweaking but the documentation is very good and simple. Note that CSF is not a firewall it is a system which functions with the iptables on the server.
I cannot find an article I wrote on how to setup CSF and mod_security but if you are interested I can write it again. CSF can be simple but that mod_security is kinda complex stuff.
If one knows a little for IDS setup it can be a pretty hard task to prevent DDoS and/or malicious attacks. However this is useful info here:
http://configserver.com/cp/csf.html
Again if you do not know how a firewall functions better hire someone who knows how to do that. Another thing for the DDoS is the web service configuration there are certain DDoS attacks which use web service bugs. There is a pretty good article here on how to protect against some of these here on STM.
@goshev - as far as stats go, it'd be very interesting to see how many redirects/sec you could get Prosper up to. That'll mostly be a database-limited challenge, mind, but it's one of the big hurdles in AM.
How is that result:
ANALYSIS
DATE : 6/13/2013
This rush generated 4,366 successful hits in 1.0 min and we transferred 8.65 MB of data in and out of your app. The average hit rate of 69/second translates to about 6,029,160 hits/day.
The average response time was 322 ms.
(Started at: Thu Jun 13 2013 19:15:40 GMT+0300 (FLE Daylight Time) | Finished at: Thu Jun 13 2013 19:16:42 GMT+0300 (FLE Daylight Time))
TEST
--pattern 4-200:60 -T 5000 http://domain/tracking202/redirect/lp.php?lpip=614
with 50 concurrent users I get 12 timeouts after 30 seconds which is also pretty good I think for that number of hits.
This is with Apache and suphp. Will test with nginx and php-fpm but that will run remote MySQL. Is 6mil hits a day a good result or someone has better setup?
@goshev - those are some pretty darn good numbers! What's the spec on the dedicated server?
It would be very interesting to see the results for record.php, too - that can often be the bottleneck on a Prosper server.
Hi Caurmen,
Can you take a look at my stats, I don't understand it completely:
http://imgur.com/7OnwKoP
The response times look good but the hit rate goes up.
What does hit-rate exactly mean?
This was a test with 20 simultaneous users
Originally Posted by BeyondHosting-Tyler
@dennis - that's just the number of simulated visitors hitting your site. If it goes up but response times stay the same, that's good.
The hit rate is as good as it can get, i.e 0 errors or timeouts.
The response time looks stable but if you are doing mobile lower would be better ( 100-200 ms ).
Originally Posted by dennis
@Caurmen: Thanks! I was a bit worried because it said: 12.5/s in the 2nd picture at the left side.
@bbrock32: Hmm, this ServInt VPS is in Amsterdam and the test is performed from the US.
So that might explain the higher response rate
Originally Posted by goshev
This happened the other day to one of our services hosted on Rackspace cloud:
08/07/14
We apologize for the inconvenience you may have been experiencing over the last couple days and would like to explain in further detail the issue and what we are doing to mitigate it.
Since August 5th, 2014 at approximately 4:29 PM CDT, a portion of our DFW Cloud Sites infrastructure has been experiencing a series of intermittent network interruptions as a result of a DDoS (Distributed Denial of Service) targeting a Cloud Sites customer with political content relating to the conflicts between Israel and Palestine.
This customer is no longer hosted with Rackspace. However, the DDoS initially targeted an IP and then grew to an IP block on Cloud Sites which has resulted in this ongoing malicious traffic.
Rackspace has a series of mitigation steps used to remediate these types of attacks however, customers behind the affected IP block may still continue to experience periods of network latency and packet loss. This network latency will result in customers experiencing intermittent site performance, site inaccessibility, timeouts, or slow load times.
Our engineers have the targeted IP addresses behind our mitigation systems and are customizing the rule sets to block malicious traffic while permitting legitimate traffic to pass. As mentioned above there are instances where some legitimate traffic may be inadvertently tagged as malicious and our engineers are actively working toward minimizing this impact.
Due to the nature of this issue, we are unable to provide a definitive time for complete resolution. We thank you for your continued patience and will continue to provide updates, as new information is available.
Hello cmdeal,
Pretty much any VPS/Cloud you get can be effected from that same thing. The cloud is basically a shared resources account. If it is very restricted and each container is well isolated they cannot oversell the resources and the price will be more than a good dedicated server. When they allow you to use shared resources you get better speed but if one user gets DDoSed all the users on the node will be effected. This depends on the DDoS of course as in certain cases it may effect the whole data-center if their infrastructure is not good enough.
The only way you can be sure you get what you pay for all the time is on a dedicated server. There are data-centers which will provide you with Cisco for up to 48 hours free of charge. If you are interested in some dedicated server prices I have a bunch of good offers for EU, US and India. Just PM me and I will be happy to help.
@caurmen
I'm wondering why you're using the LP as the target of the test and not the click tracker URL (so upstream), as it seem (from my basic understanding) more likely to put a high load on the server (LP= html / php files probably without mysql calls, versus the clicktracker which need to register the clicks) ?
Anyway very useful, it seems like my setup should be OK to start with.
Home > Hosting, Servers & Security >