Is this a bot attack? (23)

Hey dudes,

After being a member here for more than a year, I finally found the
time away from my day job (copywriter) to create some campaigns
and make some shit happen.

As a result, I've got my first ever profitable campaigns this week,
which is EXCELLENT...

... but I woke up this morning to see my stats in CPVLab are all
totally fucked. Way more traffic showing than I'm actually getting,
and it's ruining the CTR/Conversion figures for my landers.

I contacted CPVLab support, and they replied within about 2
minutes, which is amazing, but they said it's probably just bots,
and that I should look through the logs and block their IPs to
stop them registering in my stats in future.

Great idea, trouble is these hits are showing from a wide range of
IP addresses. Going through them all to sort the real from the fake
and then add the IPs to the "blocked" list is a job that would take
days.

I feel like there's probably something I'm just not getting, being
relatively new to paid traffic, and I'm hoping one of my fine STM
brethren can point me in the right direction.

Thanks in advance for your time,

-David

I don't know what traffic source you are using but this is very common. There are a lot of spy tools out there or people using VPNs to spy on campaigns.
In my stats I get a TON of repeat hits from IP blocks owned by hosting companies. Some traffic sources are worse than others for some reason. I've got campaigns that have been dead for ages and they still get like 5-10 hits per day.

You could block them by IP because I know it sucks having those bots screw up your stats. I know there are also some services out there that will help you filter out incoming visitors if they come from known hosting IP blocks. I would block by IP , although your .htaccess file could become a huge mess that will slow down your server over time.

What traffic source is this, if you don't mind saying?

Caurmen, traffic source is Bing PPC.

Thanks for your help, Rafael. I'm not able to block the IPs because
it seems to rotate to a new one every few minutes, and doesn't
go back to the old ones. So essentially I'd only be blocking the IPs
that are already not going to be used any more.

-David

Oh if it's Bing PPC it might be the Bing Bot checking your LP. It's been quite a while since I did any Adwords, but back then I remember there were frequent visits from Google spiders.
You might want to check the User Agent for those IPs.

Also, unless you are planning to get some organic traffic for your campaigns you might want to block Search Engine spiders.

That's very interesting, thank you. What do you mean by the User Agent?
Sorry, I'm new to this.

I read in the CPVLab manual that disallowing a search robot from
your lander can get you in trouble with the traffic source, so I
hadn't wanted to do that. But no, no organic traffic for me.

http://en.wikipedia.org/wiki/User_ag...d_web_browsers

Most 'official' bots/crawlers have an easily identifiable User Agent.

You should be fine excluding search robots - not sure how this would get you in trouble with Bing/Google, surely they wouldn't check pages with bot-like user-agents...?

Thank you Zeno, that's very helpful.

How might I go about finding what the User Agent is for the bots I'm getting
through CPVLab?

Are you using the latest version of CPVLab? If so, check the detected browser of the suspicious visits. If they're listed as "Googlebot" or similar, then they're crawlers!

I thought I was running the latest version, but I just checked and I'm 2.16
and 2.17 was recently released. I'll see about getting upgraded today.

In the meantime, the only information I have about these visits is that a lot
of them originate from Ottawa in Canada and Illinois in the US, and a few
other states. The IPs are repeated for a little stretch, and then never
used again, so blocking by IP doesn't make much difference.

Attached an image so you guys can see what I mean:



As you can see this visits mostly have no referring domain. I know they're
not genuine because they're way more frequent than the traffic I'm paying
for.

I'm gonna pester the CPVLab support a little more today to see how I can
block whatever's happening. I was on the cusp of declaring a winning lander,
but now my stats are so fucked I'm going to need to start over.

-David

When you're getting lots of clicks excess to those you have paid for it is likely due to a) bots/crawlers, b) reviewers, c) spying tools.

Thanks for your help guys. With some diligent Googling and
the help of the support at Beyond Hosting, I fixed this problem
yesterday.

Turns out all I needed to do was upload a robots.txt file to my
server.

How I've been a successful marketer and copywriter for 8 years
and still not know to do that, I honestly don't know.

#neverstoplearning

-D

Okay false alarm.

The problem isn't fixed, it just happened to ease up as I uploaded the new
robots.txt, but now it's back with a vengeance.

Any ideas here? This has basically ruined what was a profitable campaign
for me with a lot of potential.

The only idea I'm left with is maybe hiring a web developer to have a looksee
and tell me what to do to fix it.

-David

Hmm - are the flood of clicks coming from any IP range in particular? You could get someone to set up a firewall rule to ban that IP range if so.

I really don't understand what you are concerned about.

If you are not paying for the traffic, you should not care less.

Originally Posted by caurmen

Hmm - are the flood of clicks coming from any IP range in particular? You could get someone to set up a firewall rule to ban that IP range if so.

Originally Posted by cmdeal

I really don't understand what you are concerned about.

If you are not paying for the traffic, you should not care less.

Originally Posted by the_writer

Impossible to tell which lander/creative/whatever is
making money when there's a thousand or so random impressions sprinkled into
every test.

I can see what you mean with it messing up the tracking, but cmdeal is right - if you aren't paying for the traffic it shouldn't matter in the end, just compare banner/placement/campaign spend to revenue and there's your profit. The extra clicks only influence intermediate metrics like CTR.

Thanks guys, really appreciate your input. You are absolutely
right.

As of today, this problem gets zero of my time.

I've probably spent three hours fucking around with things and
trying to figure it out over the last week or so. In that three hours
I could have set up a completely new campaign, or done any
number of other profitable things.

Thanks gentlemen, appreciated.

-David

P.S. If you decide to come back to this AND it turns out bing-bot was the culpurt, this will fix it:

<?php
if (strpos($_SERVER['HTTP_USER_AGENT'],'adidxbot') !== false) {
  // Direct link to offer page (bing-bot sees this)
   header("Location: http://www.example.com/"); 
}else{
  // Your tracking link (bing-bot won't see this)
   header("Location: http://www.example.com/"); 
}
?>

Originally Posted by snipe

P.S. If you decide to come back to this AND it turns out bing-bot was the culpurt, this will fix it:

Code:

<?php
if (strpos($_SERVER['HTTP_USER_AGENT'],'adidxbot') !== false) {
  // Direct link to offer page (bing-bot sees this)
   header("Location: http://www.example.com/"); 
}else{
  // Your tracking link (bing-bot won't see this)
   header("Location: http://www.example.com/"); 
}
?>

Just send traffic to that PHP script instead of your tracking link.
If that doesn't fix it, bing-bot is not your problem

Glad I could help!

I threw up a test an ad on Bing + a monitoring script when first saw your thread last week.

Here's a full breakdown of the traffic I got

April 1st - Ad setup
April 3rd - First bot visit
April 6th - 4 more bot visits
April 8th - 2 bot visits + 3 reviewer visits

On April 8th, 2 of the reviewers hit the link twice. (Running XP / Windows 7)
Unsure if they were all real reviewers or advance bots.

Either way, all those visits seems pretty pointless on Bing's part. Not sure what they are trying to accomplish!
I'm guessing each bot has a different function - i.e. checking for changed links, malware etc

They probably send bots out on a throttle based on account history / how much your ads etc are trusted.
This would explain why it's so random on your end.

Either way, this was fun to do! If it occurs again the code I posted will definitely help to some extent.

I got tens of thousands of hits from Bing bot when I ran on Bing. However these hits did not show up on CPV labs. They were blocked by my "redirection" software...