STEP BY STEP: Running A Bot Test On Any Campaign (32)
Step-By-Step: How To Bot Test Your Campaign
Bots. We hates them, my Precious, we hates them forever.
If you're not aware: almost all display traffic is plagued with bots, computer programs designed to click on your ads, pretend to be human, and get publishers more money by fraudulently inflating traffic figures.
For us, they're a nightmare: we end up spending cash on worthless traffic. And some placements are 100% bot traffic - meaning you can optimise your campaign all you like, it'll never convert.
But there are ways to spot them before you spend significant money. Here's a guide to a simple bot detection technique, which allows you to detect bots by their click-through patterns.
That means you can eliminate very high bot placements before they cost you money.
What You'll Need
- A campaign that you want to test for bot traffic.
You'll also ideally need: - A tracker that can run more than one lander in a sequence in a campaign. FunnelFlux and
Thrive can do this, and I think the latest version of Prosper may be able to as well.
If you're running a tracker that can't handle a 2-step lander sequence, you can get around it by creating two campaigns. See below.
Step By Step
Step 1: Create The Bot Trap
Create a new HTML page from the following code:
<!DOCTYPE HTML>
<script type='text/javascript'>
function redir(){
window.location.replace('YOURTRACKERURL');
}
</script>
</head>
<body onload="setTimeout(redir, 300);">
</body>
Upload this page to your server and set it up in your tracker as a landing page.
Step 2: Add It To Your Campaign
Add this lander as the first step in whatever campaign you want to test.
For FunnelFlux, for example, you'd add it as the first node, with Action 1 from this bot detection lander linking to the "real" lander you're using.
If you're using a tracker that can't run multiple lander steps in a campaign, create a new campaign with your bot detection lander as the landing page, and your "real" campaign as the offer page. Make sure to pass all tokens along appropriately so you don't lose data!
Test your campaign link. It should send you to your "real" lander as usual, with a very small pause before that happens.
Make sure that you're capturing placements in your tracker: you'll need those to tell what's bot traffic.
Step 3: Run Some Traffic
Set your campaign live on your chosen traffic source. Now watch your stats closely.
Unlike most AM techniques, this one should give you results very quickly if you're getting significant traffic.
Watch your stats on a per-placement basis. You can determine whether a placement has bot traffic by watching what percentage of the traffic makes it past your bot detection lander. The CTR of the bot detection lander is equal to the percentage of that traffic which is not a bot
If your bot detection lander has a CTR of 0% for a particular placement, after a statistically significant number of clicks onto the bot detection lander (30-50 should do it), then all of that traffic is bot traffic. Block it immediately.
If it has a CTR of 100%, then all that traffic is probably human. That's a good placement at least from a bot point of view.
Between those numbers, use a judgement call. If it's 50% bots, just assume that the true price of that placement is doubled, because half the traffic's fake. That doesn't mean you can't make it profitable, though!
I'd usually drop any placement with 80% or more bot traffic.
How This Works
Crude bots have two telltale flaws.
Firstly, they generally aren't running a Javascript engine, just a simple HTML engine. That means they won't get redirected by the Javascript code here, which otherwise redirects all visitors to your "real" landing page.
And secondly, they often close the page right after clicking through the ad. This page waits 300ms before redirecting the visitor. That's faster than the average human reaction speed, so anything closing the page on the redirect probably isn't human.
This detection method is crude, and far from foolproof. Some bots can interpret Javascript, and some will mimic human behaviour - this won't catch them. But a lot of clickbots are really simple, and this will definitely catch them out.
I've never seen this technique catch human traffic by accident as a false positive, but it's always a possibility. Let me know if you have that problem.
Ways To Extend
If you want to add in additional bot-capturing, here are a few ideas:
- Add an invisible, clickable link, either in your main lander or the bot-catching lander. I'd recommend a link that's almost but not quite the same color as the background - #fefefe if your background is white, for example. Anything that clicks through that is probably a bot. That'll catch Javascript-capable bots too.
- Have your lander redirect anyone who stays on it for 1 minute or more to a second URL. Very few humans will stay on your lander for a minute, so anything redirected is probably a bot. This is a somewhat more imprecise test.
- Check your carrier logs too. If you're getting traffic from Amazon, Digital Ocean or another hosting company, that's probably not human.
- More ideas for bot detection can be found in this StackExchange discussion. I particularly like, and must test, the one about tracking mouse X and Y coordinates.
- There are various bot-detection services out there if you want to get really serious about your bot prevention. I've heard good things about https://forensiq.com/ although I've not used them myself.
I hope that was helpful! If you're confused, have questions, or would like to suggest additional bot-detection techniques, let me know below!
Nice one Caurmen, Im using a similar auto-redirect script and it works well too. Its a simple method and doesnt work 100% but its the easiest one to use that Im aware of too - perfect for quick screening of a large amount of placements.
Thanks for this Caurmen. Is there a way that this can be used to test for Bots in PPV (Pops) Desktop or Mobile traffic as well?
@flashpacker - yep, it'll work fine for both of those, although the delay will probably impact profitability on PPV campaigns more than others.
This is awesome, thanks a lot Caurmen! I'll be implementing this once I go mobile.
Couple of questions:
- In your ways to extend you mention adding an "almost" invisible link. Why is that? If you make it completely invisible, would a bot still not click it? I have no idea how bots work, thought they would just parse the html and figure out from that what to click.
- When you mentioned: Is this
Originally Posted by caurmen
Voluum specific or can it be done in CPVLab? Haven't played around with Landing Page Sequences but I'm assuming the same can be achieved by having a single LP and the ad linking to the bot catcher, which then redirects to the campaign URL or am I then at the mercy of comparing traffic source clicks to my tracker visitors to see the difference (which can be quite inaccurate)?
Thanks again!
@francug - No worries!
The invisible link trick is quite well-known, so if I was designing a bot, I'd have it check if text is the same colour as the background. However, almost the same colour is much harder to detect. Hence going for a slightly different colour to dodge any slightly smarter bots.
As for CPVLab - sorry, I haven't used CPVLab for years, I'm afraid. I'd suggest setting up two campaigns if you can't do LP sequences, with the first sending visitors to the second. That should work in almost any tracker.
Awesome thanks a lot! And yeah the bot trick definitely makes sense. Guess it would be pretty easy for a bot to be designed with at least that much "intelligence" haha 
Awesome, simple and straight to the point tutorial @caurmen!
Just set it up and it seems that 36% of my traffic are bots, going to keep running it to get more significant data.
Hope to gain some ROI from those money sucker robots!
Originally Posted by francug
Hi again Caurmen,
So you say:
"Replace "YOURTRACKERURL" with the CTA URL from your tracker that you'd normally use on a landing page. "
So as I'm using Volume then that would be: http://xxxxx.voluumtrk.com/click
But I don't see how this will redirect to my PPV lander? Surely it would just redirect straight to the offer.
Because in
So if this Lander is the first step in the process
then it would be: BotLander-------->Offer
But what I think is required for this to work is: BotLander--------->ReallLander---------->Offer
and I cannot see how to make this work in Volume
Unless you mean I should replace "YOURTRACKERURL" with the Url of the PPV lander that I want to redirect to?
Hey @flashpacker I believe Caurmen means for you to create a LP sequence campaign. I'm guessing here, since I don't use
That was my confusion as well in the question I asked above actually
@francug ahh yes I see Caurmen's response to your question now:
"I'd suggest setting up two campaigns if you can't do LP sequences"
I don't see a way to setup landing page sequences in
this method
Awesome thx a lot ! Caurmen !
Hello Caurmen,
So you say:
"Replace "YOURTRACKERURL" with the CTA URL from your tracker that you'd normally use on a landing page. "
I'm using Imobitrax,,what the CTA URL would be like?
can you please tell me ?
thanks a lot
@phillip: it's called the "Offer Link" in iMobiTrax. Note that you'll need to do the two-campaigns trick to get this to work with iMobiTrax - one campaign for the bot tester which then has your main campaign as its offer page.
Working fine on Funnel Flux ;-) , traffic was ok Yesterday only 20% bots, but I implement the top link today to see the difference. Thx again Caurmen !
Originally Posted by caurmen
up two Voluum campaign Urls. I mean currently I have:
Traffic Source------>VolumecampaignUrl1---------->RealLander----------->Offer And so the offer can then postback data all the way back to Voluum and the traffic source.
With the suggested lander sequence we will have:
Traffic Source------->VolumecampaignUrl1------->BotTestLander-------VoluumcampaignUrl2------->RealLander----------->Offer
I really think that the Offer will not be able to postback the data to the traffic source or even the first campaign in Voluum (VolumecampaignUrl1).
I'm not really a technical guy so maybe I'm wrong with this.
I feel though that this can be fine to be run for a while at least to identify bot targets from the CTR at VolumecampaignUrl.
Then simply don't use those Targets and then use the original campaign sequence as before.
Anyway I will try this and update when I have some results.
@flashpacker I know in Cake at least you can add as many post back URL links as you'd like for an offer. When the advertiser fires their post back, your network fires all your postbacks you put in the tracking section of your offer. So you could add your 2 campaigns as well as your traffic source's if they have one.
Some one please correct me on the above if I'm wrong 
Edit: or is the issue that they get assigned a different sub id once they transition to the 2nd campaign?
Originally Posted by francug
Some one please correct me on the above if I'm wrong
Edit: or is the issue that they get assigned a different sub id once they transition to the 2nd campaign?
But guys : you shouldn't really be running your campaigns with this bot trap forever, consider it just a test when you need to screen the sources. The extra redirect is not something you want in your campaigns, especially not in mobile. When I run this test, I simply use the bot trap page as a regular LP, it will actually work as if you were directlinking. Keep it up for a brief period just to get the clicks and then get back to your regulars LPs. You dont need a lot of traffic to spot the obvious bot placements.
I'm in the process of talking with
do it and keep all the stats in one campaign. I will post here when I have more news.
@matuloo But ultimately yep, I totally agree. Just running this long enough to spot those bad placements should be enough.
Most bots today use javascript. Then what.
I want to make sure I got this right... Is this how you set it up in
Originally Posted by simon_89
If you want to run just a quick bot check, you can simply put the "global bot detection" as the only LP in any real campaign and let it run for a while. It will work as a direct linked campaign so you will probably loose on it, but as a quick test it works fine
Originally Posted by matuloo
If you want to run just a quick bot check, you can simply put the "global bot detection" as the only LP in any real campaign and let it run for a while. It will work as a direct linked campaign so you will probably loose on it, but as a quick test it works fine
You say put it the global bot detection Lander in as the only LP in any real campaign. But could you also run it
as a split test with another normal lander, just to see the difference in CR and so how effective or not the global
bot lander is when it comes to actual conversions. Plus you can still measure all the bot traffic targets just from that lander?
Originally Posted by flashpacker
The bot detection lander is nothing but a delayed redirect, there is nothing else on it, so it cant really improve your CVR. A directlink should give you pretty much the same results.You will of course see different CVR because BOTs that cant get through this bot trap will not get through, so the CVR could actually be better - its just a numbers game.
Keep pin mind this is just a simple way to detect SOME bots, nothing more, nothing less.
@iamatilla - Using this test you'll find a surprising amount of bot traffic doesn't pass this test.
It's still partially effective against JS-parsing bots: the 300ms delay is enough to catch anything that immediately closes the page after clickthrough.
If you want to build in more protections against JS-parsing bots, the suggestions I give at the bottom of the post will all do the job: I'd start with the invisible-link technique (which is a great one to run at all times - costs you no loading time at all) and go from there.
If you really want to be sure that you're catching all the bots, then I'd recommend going for a professional service - particularly once you're running at the sort of volume I know you do. I've not tested any of these services, but I know that there are some real experts out there at catching fake traffic, working on the problem 24/7 - find one of them and hire them or their service, and you'll get the state of the art.
Originally Posted by matuloo
If you want to run just a quick bot check, you can simply put the "global bot detection" as the only LP in any real campaign and let it run for a while. It will work as a direct linked campaign so you will probably loose on it, but as a quick test it works fine
I don't think I'm implementing this correctly. I tried three methods:
- Direct Linking the campaign with the bot lander page.
- Taking the bot lander page and the offer be the real campaign url. The method I Was talking above.
- See below example
The campaign I'm running, it's not generating any clicks so it can't measure the CTR.
For example here's a screenshot of what I did:
With the PopAds - Germany/Belgium/United States/Netherlands - iPhone6s campaigns, their path is just a regular lander(not the bot lander) w/ regular offer.
The PopAds - Global - Bot Detection campaign, the path is Flow campaign. I just created a flow that redirects the user to a specific bot lander within the country. So something like the below:
This is going to sound a bit confusing so let's put out some prefixes first:
Real campaign link(the one I actually use with real landers and offers for the TS): http://Voluum.tr.com/fcstmforum?websiteid=[websiteID]
Global - Bot - USA Lander: http://google.com/bot/botdetection.html <----this is my bot lander that redirects the user to http://voluum.tr.com/fcstmforum?websiteid=[websiteID](real campaign link).
Global - USA - PopAds Bot Test offer(Same as the REAL campaign link url): http://voluum.tr.com/fcstmforum?websiteid=[websiteID]
After all of this is completed, I take the PopAds - Global - Bot Detection URL link and go into PopAds and setup 4 different GEO campaigns, so from the stats screenshot from above you can see that the PopAds - Global - Bot Detection campaign is not registering any clicks, but the Popads - United States - iPhone6s is generating clicks. Even when the campaign is generating clicks the stats it gives me on which placement is a bot is just showing me this:
I think I have to reverse the flow. I'll see if that changes anything.
Originally Posted by whtang
Side Tangent: I think it's important to implement this process while running your real campaign, that way you can also track NEW placements that are bots. However, as stated earlier it depends whether you want to take that risk in terms of the speed of your page. If you're working with mobile pops and your page loads 100ms and adding an extra 300ms probably won't do much harm. But every ms or second counts for mobile. However, for desktop, I don't think it'll make that much of a difference. My page already loads under 1 sec.
Here's how I did it(may still be wrong but now it's actually showing the website ID and registering clicks):
Under your real campaign, setup your lander as the bot lander:
This lander in
<!DOCTYPE HTML>
<script type='text/javascript'>
function redir(){
window.location.replace('http://11111.voluumtrack.com/click');
}
</script>
</head>
<body onload="setTimeout(redir, 300);">
</body>
I understand that it's too early to tell, but it is possible that placement 763226 could be on it's way to be blacklisted given that it's 0 clicks.
That approach will work, but you're effectively direct-linking: so it might end up being a bit more expensive than just sticking the bot test in front of an already working campaign.
However, it WILL work, and that's the important thing.
And yes, that placement you highlight does look like it's on the way to ban-ville. Although it might pick up yet! The rest of the traffic you're seeing looks pretty decent quality - a couple of other slightly dodgy placements, but not the rain of crap I've seen on some sources in the past.
Originally Posted by koobs11
If you still have questions please post your stats and ask them and we'll take a look.
Amy
Sent from my SM-G930W8 using STM Forums mobile app
Hello, excuse me, I would like to ask a question, almost 2020, do you have a better way to filter the bot traffic? Are you still using cpvlab now? Thank you!
Originally Posted by guo19921230
Home > Tracking Campaigns > Tutorials, Tips and Guides