Home > Technical & Creative Skills > Programming, Servers & Scripts

Bot detection implementation of a NewBie (3)

04-27-2018 06:02 AM #1 buiduckhanh1986 (Member)
Bot detection implementation of a NewBie

Bot is a big problem on Pop traffic. STMers discussed so much about it and there are quite much threads about bot detections. After reading and overloading with bot detection tip, I want to make a summary of some topics to make a step by step tutorial for beginner

I grab information from 3 topics which I think they resolve different kinds of Bot

  1. Bot traffic that opens our pop and do nothing. @caurmen gave us tip
    https://stmforum.com/forum/showthrea...n-any-campaign

    It bases on rules that bot will open POP and close it immediately ( less than 200 ms ) or Bot cannot parse javascript. I don’t really like this solution because It increases loading time, I use Voluum and this method requires I need create an additional campaign for bot test. @caurmen If you read this post, I really want to ask you about do you still using this solutions ?
  2. Machine bot without javascript parser opens our pop. @caurmen gave us tip
    https://stmforum.com/forum/showthrea...live-campaigns

    I love this, I also extend it to make it also have the same affect in method 1
  3. Sophisticated bot that analyze our landing page to get your offer link and click it @platinum give us tip
    https://stmforum.com/forum/showthrea...ffic-in-Voluum


So I try to wrap all of those 3 tips into one solution. I’m a newbie in AF but I’m a programmer and I understand this problem in view of a programmer. Hope that experience guys here can give me a comment If something is wrong and help me improve it

Firstly, We create custom conversions on volumn
On Voluum go Settings (gear icon on top right) -> Custom Conversion. Here we create 2 conversions

BotClickJS conversion


BotOpenJS conversion


Secondly, We create 2 offers for those 2 kind of bots
Botclick: create botclick.htm with this code
HTML Code:
<!DOCTYPE html>
<html>
<head>
<script type="text/javascript">
function getToken(variable)
{
   var query = window.location.search.substring(1);
   var vars = query.split("&");
   for (var i=0;i<vars.length;i++) {
           var pair = vars[i].split("=");
           if(pair[0] == variable){return pair[1];}
   }
   return(false);
};

var protocol = 'http';

var goal = "costantom-litylist.com/postback"; //your postback link

var pa = "?cid="+getToken("cid")+"&et=continue";

function redir(){
    window.location.replace(protocol + "://" + goal + pa);
};
</script>
<head>
<body onload="setTimeout(redir, 200);">
</body>
</html>
BotOpen: create botopen.htm with this code
HTML Code:
<!DOCTYPE html>
<html>
<head>
<script type="text/javascript">
function getToken(variable)
{
   var query = window.location.search.substring(1);
   var vars = query.split("&");
   for (var i=0;i<vars.length;i++) {
           var pair = vars[i].split("=");
           if(pair[0] == variable){return pair[1];}
   }
   return(false);
};

var protocol = 'http';

var goal = "costantom-litylist.com/postback"; //your postback link

var pa = "?cid="+getToken("cid")+"&et=open";

function redir(){
    window.location.replace(protocol + "://" + goal + pa);
};
</script>
<head>
<body onload="setTimeout(redir, 200);">
</body>
</html>
Upload those 2 htm to your host and add 2 offers into Voluum. Their offer links should look like this
http://yourdomain.com/botclick.htm?cid={clickid}
http://yourdomain.com/botopen.htm?cid={clickid}

Remember to include cid={clickid} on your offer link

Finally, update your landers
Note that we have click urls to our offers

http://xxx.trackvoluum.com/click/1 - Sends the bot click to the 1st offer in list (our botclick detection offer)
http://xxx.trackvoluum.com/click/2 - Sends the bot open to the 2nd offer in list (our bot open detection offer)
http://xxx.trackvoluum.com/click/3 - Sends the real visitor to the 3rd offer in list (our real offer)

For other Newbies who may not understand what are those link ( also mean that they have not added lander to Voluum ) like me 2 days ago. Read this first https://stmforum.com/forum/showthrea...pbystep-voluum

Now update lander with structure like this

HTML Code:
<!DOCTYPE html>
<html>
<head>
    <style>
        iframe {
            visibility: hidden;
            display: none;
            position: absolute;
            left: 0; top: 0;
            height:0; width:0;
            border: none;
        }
    </style>
    <script src="https://code.jquery.com/jquery-1.12.4.min.js" integrity="sha256-ZosEbRLbNQzLpnKIkEdrPv7lOy9C27hHQ+Xp8a4MxAQ=" crossorigin="anonymous"></script>
    
<head>
<body>
    <div id="btPanel">
        <a href="http://xxx.trackvoluum.com/click/1" style="font-size: 6px; color:#423c15; position: absolute; left: -900px;">Click Here</a>
    </div>
    
    <a href="javascript:void(0)" rel="noreferrer" onclick="exit_offer();PreventExitPop = false" class="claim">
        CTA link
    </a>

<script type="text/javascript">
     function exit_offer() {
            window.onbeforeunload=null;
            var protocol = "http";

            var goal = "xxx.trackvoluum.com";

            var pa = "/click/3";
            
            window.location = protocol + "://" + goal + pa;
    }
    
    function add_open_offer() {
        var protocol = "http";

        var goal = "xxx.trackvoluum.com";

        var pa = "/click/2";
            
        $( "#btPanel" ).append('<iframe src="' + protocol + '://' + goal + pa + '"></iframe>');
    }
    
    $( document ).ready(function() {
        setTimeout(add_open_offer, 2000);
    });
</script>
</body>
</html>
Explaination

Botclick: I use exactly method from @platinum https://stmforum.com/forum/showthrea...ffic-in-Voluum But I still explain it in my view
HTML Code:
<a href="http://xxx.trackvoluum.com/click/1" style="font-size: 6px; color:#423c15; position: absolute; left: -900px;">Click Here</a>
It creates hidden link, normal user cannot see it. If this offer get traffic, it can be only bot or someone who are hacking your lander

Our real offer

HTML Code:
<a href="javascript:void(0)" rel="noreferrer" onclick="exit_offer();PreventExitPop = false" class="claim">
        CTA link
    </a>
and

HTML Code:
function exit_offer() {
            window.onbeforeunload=null;
            var protocol = "http";

            var goal = "xxx.trackvoluum.com";

            var pa = "/click/3";
            
            window.location = protocol + "://" + goal + pa;
    }
It is your real offer, I hide CTA link in javascript variables to prevent html/javascript analyzer of bot

BotOpen

HTML Code:
<style>
        iframe {
            visibility: hidden;
            display: none;
            position: absolute;
            left: 0; top: 0;
            height:0; width:0;
            border: none;
        }
    </style>
HTML Code:
function add_open_offer() {
        var protocol = "http";

        var goal = "xxx.trackvoluum.com";

        var pa = "/click/2";
            
        $( "#btPanel" ).append('<iframe src="' + protocol + '://' + goal + pa + '"></iframe>');
    }
    
    $( document ).ready(function() {
        setTimeout(add_open_offer, 2000);
    });
Remember we need jquery for above code work
HTML Code:
<script src="https://code.jquery.com/jquery-1.12.4.min.js" integrity="sha256-ZosEbRLbNQzLpnKIkEdrPv7lOy9C27hHQ+Xp8a4MxAQ=" crossorigin="anonymous"></script>
This code means that after your landing page load for 1 second (1000 ms), your landers auto open BotOpen offers. If bot doesn’t have javascript engine, It cannot access BotOpen offer. If bot close your popup immediately in less than 1 second, It cannot access BotOpen offer. So if your visit to BotOpen offer/ visit of your lander less than 80% I think we may cut it


I have just implemented this solutions and I will run a campaign to get some stats. I'll be very happy If any guys find mistakes and give me a comment.

Thank you


04-27-2018 04:02 PM #2 noxnox (Member)



The great test, coincidentally, I tested two kinds of bot detections methods in Zero Park, the result is too big. Method 2 shows 61.99% of bots, and Method 3 shows only 1.24% of bots. Who to trust?



This is the result of my test on propellerads. The two methods show similar results.

04-27-2018 04:31 PM #3 buiduckhanh1986 (Member)

The great test, coincidentally, I tested two kinds of bot detections methods in Zero Park, the result is too big. Method 2 shows 61.99% of bots, and Method 3 shows only 1.24% of bots. Who to trust?
@noxnox I think both of those 2 methods are trust because each method detect different kind of bot
  1. Bot that open Pop only -> this kind of bot is very easy to make so It is quite popuplar
  2. Bot that download our lander, analyze html & js code and execute link -> this kind of bot is very sophisticated and hard to implement so It is rare


I have tested too, my result is similar to you

Home > Technical & Creative Skills > Programming, Servers & Scripts