Home >
Programming, Servers & Scripts >
Hosting, Servers & Security
traffic-losing problem with Amazon server (15)
10-22-2014 10:44 AM
#1
nicksolidclix (Member)
traffic-losing problem with Amazon server
I am using Amazon server all this time, I suppose Amazon is one of the best server providers in the world. but I has been frustrated by an awkward thing for some time.
I generated ten millions of clicks on FB and Google totally, but there were at least 30% to 40% clicks globally couldn’t be recorded by Amazon server, to some countries, the rate climbed to 80%!!
It’s so terrible that I dare not generate more traffic due to this tough issue. it really sucks!
As a matter of fact, this issue started from the second half of this year, it never happened in the previous years.
I am wondering whether this should attribute to Facebook and Google or something wrong with my current strategies?
Are there any buddies who generate large amount of clicks on FB or Google? what's your server? how did you solve the traffic-losing problem?
I am looking forward to your good ideas!
10-22-2014 11:01 AM
#2
zeno (Administrator)
Amazon servers are not generally considered 'amazing' unless you are dealing with higher tier (pricey!) instances and they are well optimised. Given their coverage I would rather get a dedicated server for what they charge or use cheaper, higher price to performance VPS providers.
In any case, where were your servers located?
If you were sending Malaysian traffic to an Amazon server in Dallas, TX then you were throwing money away.
Click loss is highly dependent on network latencies.
Geographical (assuming connectivity is directly related to distance) closeness of servers to your traffic source is a huge factor.
Try to never have connections going over oceans (stay within geographical regions!) or at least have your tracking servers within ~100 ms latency of your users. At least if you are running tens of millions of clicks on expensive traffic sources like FB/Adwords, this should definitely be the case.
10-22-2014 11:13 AM
#3
bbrock32 (Administrator)
Amazon is a cloud service and you don't know exactly what hardware you are running on. From what I know most instances don't even use SSDs.
Apart from that, the real change is how you configure and tune the server. Even a great dedicated running with vanilla apache and mysql will perform 3x-4x worse compared to the same hardware but with tuned services ( nginx instead of apache, mariadb instead of mysql and so on).
In your case I would suggest outsourcing the tech part and focusing on what you are good at. You could either hire someone to optimize your amazon server or get a dedicated with a company that offers good support.
Goshev from the forums or Tyler from BeyondHosting will be able to help you with a dedicated.
10-23-2014 04:08 AM
#4
nicksolidclix (Member)

Originally Posted by
zeno
Amazon servers are not generally considered 'amazing' unless you are dealing with higher tier (pricey!) instances and they are well optimised. Given their coverage I would rather get a dedicated server for what they charge or use cheaper, higher price to performance VPS providers.
In any case, where were your servers located?
If you were sending Malaysian traffic to an Amazon server in Dallas, TX then you were throwing money away.
Click loss is highly dependent on network latencies.
Geographical (assuming connectivity is directly related to distance) closeness of servers to your traffic source is a huge factor.
Try to never have connections going over oceans (stay within geographical regions!) or at least have your tracking servers within ~100 ms latency of your users. At least if you are running tens of millions of clicks on expensive traffic sources like FB/Adwords, this should definitely be the case.
my server is equipped with 8-core, 16G,CPU is large enough for collection.
it's located in US, SG and BR. click-loss occurs mostly in South-east Asia countries.
10-23-2014 04:16 AM
#5
hiro99 (Member)

Originally Posted by
nicksolidclix
my server is equipped with 8-core, 16G,CPU is large enough for collection.
it's located in US, SG and BR. click-loss occurs mostly in South-east Asia countries.
Amazon is great in SG as their POP is there but other Southeast Asia countries can/will perform much lower. Malaysia is generally okay but PH, ID, VN etc your talking "single" cables connecting to "interweb" and idiots cut them sometimes.. PIA but you really need local machines in those countries.
EDIT: Even with local machines in some of those countries you'll get click lose. ie. PH, machine in Manila will set get a higher than average traffic lose within PH as connectivity in generally is just not the same level as US/EU - although SG has better connectivity than US/EU
10-23-2014 04:21 AM
#6
nicksolidclix (Member)

Originally Posted by
bbrock32
Amazon is a cloud service and you don't know exactly what hardware you are running on. From what I know most instances don't even use SSDs.
Apart from that, the real change is how you configure and tune the server. Even a great dedicated running with vanilla apache and mysql will perform 3x-4x worse compared to the same hardware but with tuned services ( nginx instead of apache, mariadb instead of mysql and so on).
In your case I would suggest outsourcing the tech part and focusing on what you are good at. You could either hire someone to optimize your amazon server or get a dedicated with a company that offers good support.
Goshev from the forums or Tyler from BeyondHosting will be able to help you with a dedicated.
thanks for your great suggestion.
I have a tech team who is trying to figure out the problem. I hope I will not be plagued by this for too long.
10-23-2014 04:34 AM
#7
nicksolidclix (Member)

Originally Posted by
hiro99
Amazon is great in SG as their POP is there but other Southeast Asia countries can/will perform much lower. Malaysia is generally okay but PH, ID, VN etc your talking "single" cables connecting to "interweb" and idiots cut them sometimes.. PIA but you really need local machines in those countries.
EDIT: Even with local machines in some of those countries you'll get click lose. ie. PH, machine in Manila will set get a higher than average traffic lose within PH as connectivity in generally is just not the same level as US/EU - although SG has better connectivity than US/EU

click loss is terrible in those countries, I am trying to avoid sending large traffic to those countries of high click-loss
10-23-2014 06:00 AM
#8
hiro99 (Member)
Have a look at cedexis, there's some posts about it here.
The data is excellent for working out which cloud provider is best in a given country, and it does vary....
10-23-2014 06:43 AM
#9
zeno (Administrator)

Originally Posted by
hiro99
Have a look at cedexis, there's some posts about it here.
The data is excellent for working out which cloud provider is best in a given country, and it does vary....

Originally Posted by
hiro99
Have a look at cedexis, there's some posts about it here.
The data is excellent for working out which cloud provider is best in a given country, and it does vary....
http://stmforum.com/forum/showthread...ty-data-Part-2

Originally Posted by
nicksolidclix
my server is equipped with 8-core, 16G,CPU is large enough for collection.
it's located in US, SG and BR. click-loss occurs mostly in South-east Asia countries.
How much are you paying for each of those? And dear me I hope it's all SSD-based!
10-23-2014 09:53 AM
#10
nicksolidclix (Member)

Originally Posted by
zeno
thanks for your sharing of website.
it's all located with SSD,
approximate fees: US: 300$, SG:$400,BR:$400
10-23-2014 09:56 AM
#11
hiro99 (Member)

Originally Posted by
zeno
Thanks Zeno, I meant to add the link in before posting (forgot).
10-24-2014 03:47 AM
#12
zeno (Administrator)

Originally Posted by
nicksolidclix
thanks for your sharing of website.
it's all located with SSD,
approximate fees: US: 300$, SG:$400,BR:$400
Are you scaling up and down with volume?
If not, and you are constantly running high volume, a dedicated server would be a superior investment.
10-27-2014 11:08 AM
#13
nicksolidclix (Member)

Originally Posted by
nicksolidclix
I am using Amazon server all this time, I suppose Amazon is one of the best server providers in the world. but I has been frustrated by an awkward thing for some time.
I generated ten millions of clicks on FB and Google totally, but there were at least 30% to 40% clicks globally couldn’t be recorded by Amazon server, to some countries, the rate climbed to 80%!!
It’s so terrible that I dare not generate more traffic due to this tough issue. it really sucks!
As a matter of fact, this issue started from the second half of this year, it never happened in the previous years.
I am wondering whether this should attribute to Facebook and Google or something wrong with my current strategies?
Are there any buddies who generate large amount of clicks on FB or Google? what's your server? how did you solve the traffic-losing problem?
I am looking forward to your good ideas!
I got information from our tech team that site provides global access to users may result in website click- loss problems. We analyze that there may have following reasons causing the problems.
1. Georaphical problems;
2. Operator bandwidth exit problems in different country;
3. International link problems;
4. Network infrastracture problems in different country;
5. DNS delay problems;
6. Server bandwidth problems;
7. Server configuration problems;
8. Server optimization problems;
9. The bandwidth stability prblems of your server with your third party server provider if there is any communication with your third party server;etc.
We can come to a hypothesis as shown below: we lauch an advertising link address on facebook, the link address will allocate to the server which belong to American. Normally, the users can click the ads we put on the facebook. During the process, DNS will make analysis and direct the user to vist our American server and server will revert back to the users right page in return. Then the whole process could be completed. Then let us see the visiting process of different ares. 1. American users ----à server; 2.Indian users—à server; the visiting speed for two different users must be different. Then which country could suffer from the click- loss problems? Everyone must know the answers for this hypothesis. Then how should we solve this problems? Let us put CDA aside for some CDN are almost the dynamic website on our site.
As you can see above structure, according to the user requests from different country, DNS will automatically allocate to the server which is not far away from you. It’s very flexible. I know most of you are all proficient in this field. Is there anyone can contribute any ideas about this diagram as shown above? See what advantages and disadvantages does it have, regardless of its cost?
10-27-2014 11:11 AM
#14
nicksolidclix (Member)
Hi ZENO,
thanks for your ideas.
my tech team has tried their best to solve click-loss problem.
10-28-2014 04:15 AM
#15
zeno (Administrator)
DNS-level load balancing is definitely the way to go.
What you really should do is split-test with a different system (e.g. Voluum who also have DNS-level load balancing) and use something like NewRelic to collect data on your page load times, resource usage and database load, etc.
Home >
Programming, Servers & Scripts >
Hosting, Servers & Security