Home > Technical & Creative Skills > Programming, Servers & Scripts

A Simple Way to Detect Bot Traffic in Voluum (33)


02-16-2018 04:09 PM #1 platinum (Veteran Member)
A Simple Way to Detect Bot Traffic in Voluum

Although Voluum is an amazing tracker, I've always had a hard time implementing bot detection techniques within my live campaigns.
In order to do that, I either needed to have some PHP server side scripts or a second campaign from where I could filter my traffic, ending up in a chaotic mess of stats.

Here's how I could get rid of all the headaches of additional scripts and additional redirects.

Requirements:
- Enable Custom Conversion tracking in Voluum.
- Use at least one landing page.
- Create your own offer for bot traffic.

Benefits:
- Detect both JS capable and incapable Bot clicks
- No additional delays in your landing page loading speed
- No need to create separate scripts to track your data (like passing parameters from one camp to another)
- Live Bot detection Always ON!

If you are using Voluum and want to have a bot detection setup in place, try implementing the following setup.

Step 1: Enable Custom Conversion tracking needed to detect JS capable Bots.
Login to your voluum account and create a custom conversion event


Give a name and a value to your custom conversion event then move to Step 2. (screenshot below)


Step 2: Add 2 Multi-offer links to one of your landing pages.

http://xxx.trackvoluum.com/click/1 - Sends the bot click to the 1st offer in list (our bot detection offer)
http://xxx.trackvoluum.com/click/2 - Sends the real visitor to the 2nd offer in list (our real offer)

Because the browser engine will read our lander’s HTML from top to bottom, in this example I’m placing the hidden link right after the opening body tag <body>

Code:
<a href="http://xxx.trackvoluum.com/click/1" style="font-size: 6px; color:#423c15; position: absolute; left: -900px;">Click Here</a>
This link is not visible to our visitors, but almost every type of Bot can click on it. So whenever we track clicks to our own bot offer, we know that the visitor is NOT real.

In addition to that, to minimize fake clicks towards our real offer, we will set our lander’s call to action link behind a java script function, so that bots that cannot process java will not be able to land to our real offer.
Here’s an example on how our cta code should look like and the java script function that contains our 2nd multi-offer link.
Code:
<a href="javascript:void(0)" rel="noreferrer" onclick="exit_offer();PreventExitPop = false" class="claim">
Add this to the bottom of your page before </body> tag for the cta button to work properly.
Code:
<script type="text/javascript">
     function exit_offer() {
            window.onbeforeunload=null;
            window.location = 'http://xxx.trackvoluum.com/click/2';
        }
  </script>
Step 3: Create your own JS Parse offer
This offer will fire only the custom conversion that we created on Step 1 without interfering to our campaign’s conversions. This means that each for each lp click that generates a custom conversion, it is coming from a bot that can parse java script, if no custom conversion is fired then this bot cannot parse java.

Code:
<!DOCTYPE html>

<script type="text/javascript">
function getToken(variable)
{
       var query = window.location.search.substring(1);
       var vars = query.split("&");
       for (var i=0;i<vars.length;i++) {
               var pair = vars[i].split("=");
               if(pair[0] == variable){return pair[1];}
       }
       return(false);
};
var goal = "http://xxx.trackvoluum.com/postback?cid="+getToken("cid")+"&et=continue";
function redir(){
window.location.replace(goal)
};
</script>

<body onload="setTimeout(redir, 200);">
</body>
When adding JS Parse offer to your voluum, make sure to append the Click ID at the end of its url by adding ‘?cid={clickid}’ after the .html extension

For a copy of the Landers and Bot offer used in this guide follow this link.

After sending some traffic to your campaign with a similar setup as described above your stats should look like this.


In case you might have any question or need assistance on the above setup, feel free to ask below.


02-16-2018 08:19 PM #2 rob_gryn (Member)

That's really clever, I'm impressed You'll be happy to know that we're working on a fully fledged solution for bot and fraud filtering/reporting in Voluum. We're utilizing our years of experience from Zeropark and Voluum DSP.


02-16-2018 08:19 PM #3 khzidan (AMC Alumnus)

I m gonna try this now!!!!


02-16-2018 08:57 PM #4 vortex (Senior Moderator)

That's a share worth its weight in platinum!

Thanks so much for taking the time to illustrate the entire process!



Amy


02-17-2018 12:18 AM #5 hyperlogic (Member)

Very cool thanks for sharing.

Excited to see the Voluum native implementation for this as well...


02-17-2018 03:36 AM #6 erikgyepes (Moderator)

Interesting approach with the custom conversion events.

Thanks for sharing this method ;-)


02-17-2018 05:34 PM #7 birdman (Member)

Hi Platinum,

Adding the code to the bottom (REAL CTA Button) doesn't seem to work. Here is a screenshot, where am I going wrong?

Name:  2018-02-17_1316.png
Views: 1905
Size:  109.3 KB



Step 2:

Name:  Screen-Shot-2018-02-17-at-1.18.50-PM.gif
Views: 1895
Size:  78.1 KB


Is this how you setup?

Thanks!


02-17-2018 06:18 PM #8 platinum (Veteran Member)

@birdman as a good practice, first you should move javascript function at the bottom of your html page just before the </body> closing tag.

I've included some notes in the screenshot below where you can see a real lander example on how the hidden link, the cta button and the script are setup
https://prnt.sc/ig4gfb

Also, when configuring landers in such setup, you need to set the number of offers to 2 when creating the lander (example below)
https://prnt.sc/ig4h9m

Please note that if you don't set your lander's number of offers to 2, both offers the JS Parse detection offer as well as the real one will receive an equal amount of traffic.

Let me know if you need further assistance


02-17-2018 10:25 PM #9 mrbraun (Moderator)

Thank you very much for this thread!

Looks really cool. Will try today.


02-18-2018 06:55 AM #10 birdman (Member)

Hey Platinum,

I changed up my lander so it has 2 offers now instead of 1. Major mistake on my part. However; I still cannot seem to get this to work.

I wrote this dummy lander for you to replace your own Voluum links and perhaps see where I am going wrong. I looked at your screenshots but I can't seem to see the difference of my page vs your example.

Name:  2018-02-18_0258.png
Views: 1857
Size:  583.0 KB


Download
: DummyLander HTML Code

Looking forward to your reply,
Aras

P.S. Is it normal to get a 404 Error for the BotJS Conversion page?


02-18-2018 09:18 AM #11 platinum (Veteran Member)

@birdman when testing the flow of the above setup, kindly make sure to visit your lander through your campaign link. If you’re visiting it from just your lander’s url, none of your tracker’s data is being utilized. Therefore you will receive an error 404 when clicking on click 1 or 2 as the tracker won’t know where this click os coming from.

Also, make sure to append the the click id at the end of your landing page URL. This is one is really important, as here it is where we are grabbing the clickid we need to fire the custom conversion back to our tracker.

PS: Will take a deeper look to the dummy lander once in front of my PC


02-19-2018 10:27 AM #12 chervenkov (Member)

this is very nice, but i have a question? Should i make this every time when start new campaign or once created i can use this bot for all my camps? Thanks


02-19-2018 10:51 AM #13 platinum (Veteran Member)

Quote Originally Posted by chervenkov View Post
this is very nice, but i have a question? Should i make this every time when start new campaign or once created i can use this bot for all my camps? Thanks
You can run this method on live campaigns and be able to identify placements bot % right from the bot offer clicks and custom conversions.
Also, about the JS Parser offer you need to create it only once and then use it on all campaigns you’d like to.


02-23-2018 02:20 PM #14 navidaffiliate (Member)

Quote Originally Posted by rob_gryn View Post
That's really clever, I'm impressed You'll be happy to know that we're working on a fully fledged solution for bot and fraud filtering/reporting in Voluum. We're utilizing our years of experience from Zeropark and Voluum DSP.
Hey Rob. That's great news. If Voluum as in music industry, it would be Rock 'n Roll

When can we see this bot filtering/reporting in voluum?


03-09-2018 04:49 AM #15 redirectad (Member)

Thanks for the tip!

for some reason I can't set it up, is someone willing to help me via skype?

Thanks


03-09-2018 06:51 AM #16 platinum (Veteran Member)

Quote Originally Posted by redirectad View Post
Thanks for the tip!

for some reason I can't set it up, is someone willing to help me via skype?

Thanks
Hey redirectad, in what part of the setup are you stuck?


03-09-2018 05:11 PM #17 redirectad (Member)

Hi platinum, I'm trying to create a LP that will only redirect users to offer and filter out the Bot traffic. this is what I did:


That's the LP (took it from the example):

<!DOCTYPE html>
<head>
<script type="text/javascript">
function getToken(variable)
{
var query = window.location.search.substring(1);
var vars = query.split("&");
for (var i=0;i<vars.length;i++) {
var pair = vars[i].split("=");
if(pair[0] == variable){return pair[1];}
}
return(false);
};
var goal = "http://track.voluumurl.com/postback?cid="+getToken("cid")+"&et=continue";
function redir(){
window.location.replace(goal)
};
</script>
</head>


<body onload="setTimeout(redir, 200);">


<a href="http://track.voluumurl.com/click/1" style="font-size: 6px; color:#423c15; position: absolute; left: -900px;">Click Here</a>
<a href="javascript:void(0)" rel="noreferrer" onclick="exit_offer();PreventExitPop = false" class="claim">


<script type="text/javascript">
function exit_offer() {
window.onbeforeunload=null;
window.location = “http://track..voluumurl.com/click/2”;
}
</script>


</body>



That's my Voluum configuration:

Click image for larger version. 

Name:	Screen Shot 2018-03-09 at 9.52.20.png 
Views:	65 
Size:	186.3 KB 
ID:	18337


Click image for larger version. 

Name:	Screen Shot 2018-03-09 at 9.59.44.png 
Views:	44 
Size:	269.6 KB 
ID:	18338


And this is the result:


Click image for larger version. 

Name:	Screen Shot 2018-03-09 at 9.54.12.png 
Views:	24 
Size:	19.0 KB 
ID:	18339


I'm sure I'm doing something wrong I just can't figure out what.


03-09-2018 06:38 PM #18 platinum (Veteran Member)

@redirectad it looks like you are somehow merging different landers & scripts and links in the same landing page.

The following code belongs to the Custom Bot Offer we are using to post custom conversions back to our tracker.

Code:
<!DOCTYPE html>

<script type="text/javascript">
function getToken(variable)
{
       var query = window.location.search.substring(1);
       var vars = query.split("&");
       for (var i=0;i<vars.length;i++) {
               var pair = vars[i].split("=");
               if(pair[0] == variable){return pair[1];}
       }
       return(false);
};
var goal = "http://xxx.trackvoluum.com/postback?cid="+getToken("cid")+"&et=continue";
function redir(){
window.location.replace(goal)
};
</script>

<body onload="setTimeout(redir, 200);">
</body>
Based on these custom conversions fired back to our tracker, we are able to determine if the bot click is coming from a bot placement that is able to parse javascript.

http://track.voluumurl.com/click/1 and http://track.voluumurl.com/click/2 are added to our landing page where in the guide example, click/1 is a hidden link to the Bot offer and click/2 is our real offer. One thing to note is that I have set the real offer click link behind a javascript function in order to avoid dumb bot clicks to the real offer.

To make things easier, I have included a copy of clean landers and custom bot offer in this link

Please take a look at the template materials in the above link and let me know if there's any thing you still need assistance with.


03-10-2018 09:54 PM #19 redirectad (Member)

I got it now!

Thank you very much for your help!!!


04-03-2018 07:25 PM #20 eurosen (AMC Alumnus)

Quote Originally Posted by rob_gryn View Post
That's really clever, I'm impressed You'll be happy to know that we're working on a fully fledged solution for bot and fraud filtering/reporting in Voluum. We're utilizing our years of experience from Zeropark and Voluum DSP.
Hi Rob,

Any update on when this will be available?


04-03-2018 07:50 PM #21 ThrvTrkr (Member)

As an aside to all of this extra work, Thrivetracker has been offering a built-in custom redirection and bot filter option for over two years. This provides you with the opportunity to customize your audience using 17 different data points, and it even allows you to push one campaign's wrong traffic into another campaign, for further capitalization. We include this functionality at no cost in all cloud-hosted plans, and we do not charge for any clicks filtered out of your campaigns.

For those who would like to put our solution to the test, we offer a 30-day free trial, allowing you the opportunity to see just how easy it is to have Thrive work for you.

For the STM audience, we are about to pull the curtain back on a VERY a special promotion. If you'd like to be inb4 the masses, let's chat!

Cheers all,


04-04-2018 08:22 AM #22 twinaxe (Senior Moderator)

In Binom you can use events, you can see it here https://docs.binom.org/events.php#p1
Just put that little script on your lander and Binom shows you a column with bot percentage directly in your stats.
Its really easy to implement and you have the stats directly in your tracker.


04-07-2018 10:55 AM #23 thien425 (Member)

I am not sure if I am doing this right. Can this set up be used for split testing offers and landing pages? Do I set this up on every landing page that I am split testing? Also I am testing 4 different offers, when I set the offer to 2 in voluumfor the landers, it will only allow me to add 2 offers when setting up the campaigns. I am unable to add the other offers that I am split testing.


04-07-2018 06:40 PM #24 platinum (Veteran Member)

Quote Originally Posted by thien425 View Post
I am not sure if I am doing this right. Can this set up be used for split testing offers and landing pages? Do I set this up on every landing page that I am split testing? Also I am testing 4 different offers, when I set the offer to 2 in voluumfor the landers, it will only allow me to add 2 offers when setting up the campaigns. I am unable to add the other offers that I am split testing.
Yes, using this method you can split-test offers and landers. Even if the lander setup will force the traffic to go only on one specific offer per click action, you can create a separate path using flows where you can set the specific rotation weight to each offer you are split-testing.

Let me know in case you might face any difficulties setting this up.


04-08-2018 03:47 PM #25 thien425 (Member)

I don't know how to use the flows feature in Voluum but this is what I came up with. Is this the correct way to set it up for split testing? I currently have 4 landers and 4 "real" offers in rotation.




04-09-2018 08:17 PM #26 platinum (Veteran Member)

Quote Originally Posted by thien425 View Post
I don't know how to use the flows feature in Voluum but this is what I came up with. Is this the correct way to set it up for split testing? I currently have 4 landers and 4 "real" offers in rotation.



Sorry for the late reply!

I'm afraid your setup is not correct, so let's fix it together.

First, you need to create a separate new lander where you configure click/1 and click/2 as instructed in the original post. When creating this new lander in your Voluum account make sure to set the number of offers to 2. This is really important as this will indicate on what offer the visitor will land when clicking one of our click links. The hidden link goes to the bot offer while the real link to the real offer.

Second, when whating to split test multiple landers and offers using this method, simply create a custom flow. You can create a separate path for each real offer then add landers you are split-testing on each path. The following screenshot should give you an idea how it should look.

As you may notice on the side of each offer the rotation weight percentage information is not visible, this because on our landers each offer has it's own click link.

Let me know if you'd need further assistance on this.


04-10-2018 10:19 AM #27 thien425 (Member)

So I only need to create 1 lander and configure click/1 and click/2 on only that lander? Or do I configure click/1 and click/2 for every lander I'm split testing. I am still not clear on that. But by your screenshot it looks like the same setup as what I did. It probably didn't look like it with the screenshot that I posted. Check the screenshot below, looks similar to yours when I use that one. I configured click/1 and click/2 on every lander. Offer 1 is the bot offer and offer 2 is my real offer. Then I created a separate path for every offer I am split testing and I added all of my landers in each path.


04-10-2018 10:57 AM #28 platinum (Veteran Member)

Quote Originally Posted by thien425 View Post
So I only need to create 1 lander and configure click/1 and click/2 on only that lander? Or do I configure click/1 and click/2 for every lander I'm split testing. I am still not clear on that. But by your screenshot it looks like the same setup as what I did. It probably didn't look like it with the screenshot that I posted. Check the screenshot below, looks similar to yours when I use that one. I configured click/1 and click/2 on every lander. Offer 1 is the bot offer and offer 2 is my real offer. Then I created a separate path for every offer I am split testing and I added all of my landers in each path.

Yes, this one looks correct! You need to add click/1 and click/2 on every lander you are split-testing. I might have miss-read the 25% rotation weight on the lander considering it as the offers rotation.

Drive some traffic to this campaign then read the results. You will be able to see that some placements are clicking on the hidden link as well and if a custom conversion fires, that will tell that the bot that clicked on the hidden link is able to process javascript.


04-11-2018 10:26 AM #29 chervenkov (Member)

How i can use it for direct-linking offers? Thanks


04-12-2018 03:53 PM #30 sunwukong (Member)

i have a question about if the bot only visit the page and didn't click anything . how can we tell those bots ? the pop traffic sell on cpm type. so even if the bot just view the page we still need pay the money. @platinum


07-29-2019 11:48 AM #31 alpha7776 (Member)

After running for a day, i realise that a placements have 1 visit and 13 clicks, but the BotJs did not indicate anything.
So i decide to conclude it as a 100% bot site.


07-30-2019 05:26 PM #32 vortex (Senior Moderator)

Quote Originally Posted by alpha7776 View Post
After running for a day, i realise that a placements have 1 visit and 13 clicks, but the BotJs did not indicate anything.
So i decide to conclude it as a 100% bot site.
When you say "botjs", do you mean the bot script you've set up as an offer in Voluum?

Can you show us a screenshot of Voluum stats?



Amy


08-02-2019 12:24 PM #33 platinum (Veteran Member)

Quote Originally Posted by alpha7776 View Post
Thank you for this infor
Set this up and started running the live campaign for 2 days.

However, there is nothing captured in the BotJs.
But when i manually went in to click, it does reflect on Voluum.

Does this means that there is no Bot in the traffic source?
Quote Originally Posted by alpha7776 View Post
After running for a day, i realise that a placements have 1 visit and 13 clicks, but the BotJs did not indicate anything.
So i decide to conclude it as a 100% bot site.
As you may notice from the screenshot, you have two metrics that show you if the clicks are valid or not.

1st metric
Are the clicks to the hidden link (to fake offer) where any type of bot can click on, but humans don't (obviously because it's a hidden link). So anytime you see a click happening in the hidden link, it means that it's not clicked by a real visitor.

2nd metric
The second metric (BotJS custom conversion) is used to test if clicks that happened in the hidden link are coming from basic dummy bots that can read and process basic HTML tags only or advanced ones, that are able to process javascript as well as fill in forms.

As Amy mentioned, please share a screenshot of your stats here so that we can help you to better read your stats.


Home > Technical & Creative Skills > Programming, Servers & Scripts