Home > Tracking Campaigns > Tutorials, Tips and Guides

How to Setup Your Own Bot Detector (11)


05-22-2017 09:17 AM #1 platinum (Veteran Member)
How to Setup Your Own Bot Detector

Bots bots bots… The always-relevant topic that has been haunting affiliates for ages.

Following the great information shared by Caurmen and other STM-ers in this guide, here is how I turned my campaigns into green faster than usual, by just putting the pieces together and cutting out money sucking publishers.


As you can see from the visual funnel builder there are two filters in my campaign that will make it possible to detect if the publisher I’m buying traffic from has a high percentage of bot traffic not worth spending money on.

Without further ado, let’s get into the details…

1. Javascript auto redirect delay node
Just create this simple HTML lander, place it on your Amazon S3 cloud storage and then create a new CloudFront distribution in order to have this redirect lander as close as possible to the geo you are running your camping on. I would highly recommend accessing this redirect lander through a CDN in order to avoid further delays.

Code:
<!DOCTYPE HTML>
<script type='text/javascript'>
function redir(){
window.location.replace('myfluxtracking.domain/?flux_action=1');
}
</script>
</head>
<body onload="setTimeout(redir, 300);">
</body>
2. The hidden link

Even though we’re in 2017, not all types of bots are smart enough to pass through an invisible detection system as Catherine Zeta Jones playing Entrapment.

Place the below code in a random place of your lander and make sure it renders as it should: (not visible for our eyes but present in the code – see screenshot below). The text color code in this case is almost equal to the background color but slightly different.
In order to pick the right color code I like using coolors.co as it makes it easier for me to compare two color codes close to each other.
Code:
<a href="http://myfluxtracking.domain/?flux_action=2" style="font-size: 2px; color: #E7E8E5; position: absolute; left: -900px;">Win Now</a>
As you may note from the code, the hidden link points to Flux Action 2 which goes to a dummy offer which no real user should ever click on.

This is how it should look when inspecting the html code using Chrome/Firefox Development Tools.


3. SmartLink Hidden Offer

In order to be sure that the hidden link is always going to be clicked by a bot and not a human, every click on Flux Action 2 goes to a SmartLink, while every Flux Action 1 goes to the Real Offer we are promoting.

I like doing this just to be sure, that if the hidden link gets clicked by a human that is about to convert, that conversion is not lost. This includes those very rare cases where the lander may not render correctly and the link is visible even to that real user.

By having our Bot Traps ready, all we need to do is to generate the campaign link and start sending traffic to our funnel by the traffic source of choice.

After sending a certain amount of traffic to our campaign, we will see that not all publishers will pass through the first javascript redirect lander. This means that, part of the traffic purchased from these publishers is bot traffic but we are not quite sure yet… Even though the biggest part of that traffic has gone through the javascript redirect lander we will notice that those same publishers that have gone through the first filter, now will randomly click on our call to action links, especially on Flux Action 2 link. This is the proof we need to classify them as bots.

So, to get to know which of the publishers are having that bad traffic we don’t want any more in our campaign by right clicking on our funnel we go to the Stats Drilldown and then setup our custom drilldown report.


When on the drilldown report page, we will need to generate the report with the following variables in this sequence: Funnel; Flux Lander; URL Tracking Field and Action Num as seen in the below secreenshot.
Note: For the URL Tracking Field, we need to select the publisher token name which in this example is publisher.


By expanding Lander -> Publisher -> Action Num on the generated report we will see that not all clicks recorded in our landing pages belong to the human visible link, lots of them are on the hidden one.
For each click on Flux Action 2 we have the same number of clicks for Flux Action 1.


Attention! As you may see, not all clicks are bot clicks and as such we will need to be sure that the placement will can still be profitable despite the fact it has even 50% of bot traffic.

Both explained methods are effective and can cover different types of traffic.

Personally I’ve found the first method is more useful when run on banner traffic, since when a user clicks intentionally on a banner, he will wait that extra 300-500ms of delay for the page to be displayed. While the second method is more suitable for POP/Redirect traffic where having an extra 300ms of delay continuously on a camp will decrease the ROI since the user will have more time to close the browser window/tab.

Another advantage of the second method is that we can keep that hidden link as long as we want in our lander since it will not cost any additional delay on our landers loading time while at the same time as we increase the bid in order to get more traffic, it will continue to detect new bad publishers as they are recorded in our tracker.

I hope this guide will be useful for those who are still struggling with their traffic quality and bot detection techniques.

And of course, if anyone has more advanced techniques they use for bot detection, feel free to share here. I would love to test and implement them


05-22-2017 12:54 PM #2 Tyoussef_PureLander (Member)

thanks for sharing the info .
i think you have covered every thing .
also if you saw some crazy ctr on some lander that means a bot .


05-22-2017 04:46 PM #3 vortex (Senior Moderator)

A beautiful way of catching bots! You have such a knack for laying things out step-by-step - would LOVE to see more wisdom from you in your future posts!

Thank you very much for taking the time to lay it all out!



Amy


05-23-2017 08:48 AM #4 bbrock32 (Administrator)

Great case study!

I think with the inclusion of a few extra conditions like super high ctr, datacenter ips etc it would become very accurate.


05-23-2017 07:54 PM #5 platinum (Veteran Member)

Thank you for guys!

Actually putting together the second technique and the power of the drill-down reports one can save some good money as well.
Even when running different GEOs for a given traffic source


05-24-2017 06:19 AM #6 Jitendra (Member)

Wow man you just have showed me how to stop bots


08-26-2018 12:06 AM #7 diamond (Member)

Why wouldn't you send 100% of the traffic through it?


08-26-2018 04:45 PM #8 platinum (Veteran Member)

Quote Originally Posted by diamond View Post
Why wouldn't you send 100% of the traffic through it?
Are you referring to the JS node? If so, yes you can send 100% of the traffic there but you’d need to check the capability of processing JavaScript on the lander (something like the one I’ve used in Voluum’s guide)


Sent from my iPhone using STM Forums mobile app


09-13-2018 04:00 AM #9 vintaych (Member)

Hi

I saw your blog post: http://blog.theoptimizer.io/610/how-...-using-voluum/

Have a few questions and hoping to get your reply:

1) I am not using Voluum (due to expensive pricing). If my tracker supports custom conversion tracking, it will work the same right?
2) From what I am understanding from your explanation and the code, this script basically does not add any additional ms due to redirection unlike one of the most common redirect bot detection script around right? Basically this script all exist in 1 LP.

Thank you!


09-13-2018 09:58 AM #10 platinum (Veteran Member)

Quote Originally Posted by vintaych View Post
Hi

I saw your blog post: http://blog.theoptimizer.io/610/how-...-using-voluum/

Have a few questions and hoping to get your reply:

1) I am not using Voluum (due to expensive pricing). If my tracker supports custom conversion tracking, it will work the same right?
2) From what I am understanding from your explanation and the code, this script basically does not add any additional ms due to redirection unlike one of the most common redirect bot detection script around right? Basically this script all exist in 1 LP.

Thank you!
Yeah, as long as your tracker allows you to post custom conversions that should work fine and it doesn't add any delay on loading time.


09-13-2018 01:32 PM #11 vintaych (Member)

Quote Originally Posted by platinum View Post
Yeah, as long as your tracker allows you to post custom conversions that should work fine and it doesn't add any delay on loading time.
Thank you!

I just checked and unfortunately it seems my tracker does not provide custom conversions.

In that case, these two codes should be alright to place in my LP for detecting bots as well right?

1) To be placed below body and change the color to match similar to my own LP background color. This will not be seen at all by humans and if clicked = bots
<a href="http://xxx.trackvoluum.com/click/1" style="font-size: 6px; color:#423c15; position: absolute; left: -900px;">Click Here</a>

2) Basically to have my CTA button to redirect using JS because most bots cannot read JS. So this helps in weeding majority of the bots.
<a href="javascript:void(0)" rel="noreferrer" onclick="exit_offer();PreventExitPop = false" class="claim">
<script type="text/javascript">
function exit_offer() {
window.onbeforeunload=null;
window.location = “http://xxx.trackvoluum.com/click/2”;
}
</script>


Home > Tracking Campaigns > Tutorials, Tips and Guides