Home > Technical & Creative Skills > Programming, Servers & Scripts

---

About Bot Traffic Again... (21)

---

Hey guys!

Yes I'm back on ranting about bot traffic.

So I was reading @matuloo's "old" blog post about bot traffic, he mentioned there placing a pixel on the landing page as well as having a blank page javascript redirect before the landing page.

Is that still practical?

Now my question is whether most bots we get from Push/Pop traffic are able to click through the landing page? And are they usually doing so? Lets say its also a longer flow landing page with giftbox & questionnaire together.

---

Hey,


Can you post the URL of the post? I would love to have a look at it.

I won't say that most bots do it, but definitely some, and more than a few years ago, especially since they release Chrome Driver in 2017. I am finishing a custom JS library to track them client-side, so i will probably have more info in a few months once my system has been running for a while.
@Voluum published an article backed by a pretty technical research paper 2-3 months ago about that. Let me try to find the link.

---

Here:
https://voluumdsp.com/blog/demand-si...against-fraud/

And the paper:
https://link.springer.com/chapter/10...030-43229-4_23

Some of their work hypothesis are not clearly detailed, so it is not as useful as it could be, but still lots of ideas to test when tracking bot.

---

Hey man, here it is: http://www.matuloo.com/bot-traffic-h...o-do-about-it/

---

The methods I listed in the article should still work with the less advanced bots, but for the really good ones, it wont be that effective.

But anyways, your goal is not to catch all bots, that's not even possible. You want to identify the placements/spots that have too many bots, because those won't work no matter what you do. And for such screening, the simple methods are often sufficient.

These days, many trackers have bot detection built in, which helps a lot to identify this rally poor placements. What tracker are you using, if I might ask?

---

Hey man I use Bemob, they have bot rules which you can set up but I don't know too much which rules to setup

---

I'm not using bemob, so can't really help you with the setup

I did a quick search and found this article : https://blog.bemob.com/2017/12/25/be...h-bot-traffic/

Looks like they have several settings to use, for example IP based filtering, which can be of use in case you see unusual behavior for certain IPs... for example too many clicks from the same IP. But that's not that useful for initial traffic screening.

Try to talk to their support, they migh have better docs available. If you look at the stats, do you see some "bot %" breakdown or something like that? If there is something like that, you might use that to determine what placements are the most poluted by bots and block them at the source directly.

---

Just saw this And no, they do not have any bot % stats by default, I believe you need to set rules for it to show you some bot numbers.


Just ripped a lander and found this -

This is clearly a "bot trap" setup by the previous affiliate, now my question is how do I actually replace this link so that I could track it on the campaign level, or even placement level? Or just simply report the clicks in tracker?
(No I'm not going to be switching the link for every new campaign that I run )

---

You can use the multiple URL feature
https://help.bemob.com/docs/adding-bemob-click-urls

You assign one link to your normal offer/flow, and one to a fake offer that is the bot one. By doing so, you have the tracking at campaign level, and when analyzing the data in your tracker, you can find which placement it came from.

---

That seemed like a nice idea... However it comes out that I can't rotate more than one real offer for a landing page like this:


(in the example I set the offer value to "2", setting it to 1 will give a normal "/click" and will enable to rotate as much real offers as I want i.e. normal mode)

(("add new offer" just enables you to add a new URL to the existing 2 spots that you have there when setting the value to "2"))

---

I am not talking about rotating offers, but having multiple links which points to multiple offers. Please read the link i sent, section "Multi-Offer Click URL".

If you want to rotate, try to define a flow separately where you rotate your offers.

Point:
/click/1 to this flow
/click/2 to your fake offer/flow

I am not a Bemob user, so you may ask support for assistance on that.

---

I know you weren't talking about rotating offers But with the "Multi-Offer Click URL" method you can't really practically rotate real offers in your campaign.

Can't really do that In the click/N paths you can enter only offer URLs

---

Any other ideas @jeremie?

---

Note: I will assume you use the redirect method.

If you want the data in BeMob, you could define a dedicated bot tracking campaign, and use the campaign link as the bot link. By doing this, you will be able to see who visits this link.

You will have to add some javascript triggered when the page loads, to pass the query parameters of the current page, at least the placement ID and the network, to match the URL definition you define in the tracker. I would also add a parameter with the original campaign ID and/or landing page. So that, if you use this campaign for the tracking of all your other campaigns, you can identify which campaign the visit is related to, and have access to all stats.


The problem with this method is that it won't detect bots that do not support JS.

---

So I just went to my "IPs" tab on Bemob and I saw many IPs that have many "visits"

My IPs section divides into two groups:
Subnets (small)
Subnets (big)

On both I see many visitors, but only on Subnets (big) I see IPs with multiple legit offer conversions (CC submits), on Subnets (small) I see only 1 legit offer conversion per IP (not including small network redirect conversions)
My question is: are some of those bots? How do I optimize based on it?
I know some affiliates optimize based on IP ranges, databases or stuff, but I don't know how that works.




How can I use this to my advantage?

---

To simplify, a big subnet (/16) can contain 256 time more computers than a small subnet (/24). Which means you will naturally see more conversions in absolute. But you can not optimize just based on that fact. This is more useful when you want to analyze specific placements, to see if a bot operate from several IP from the same network, identify datacenters... I would not bother with that until you have a good optimization strategy at the placement level.

See more info on the CDR notation here:

---

@jeremie you just sent me 12 years back in time with this!


Quite often trackers will use the subnet address rather than showing the exact IP address of the visitor. Most probably it's a limitation of the database used to report stats. However, I wouldn't go into subnet level optimization, since that will involve taking in consideration lots of details of a specific IP range, and has no guarantee of helping you with the results.

Those who optimize by IPs quite often utilize data collected through time and mostly exclude known data center IP ranges and alike rather than go over specific IPs. On the other hand there are those who generate the fraud traffic and are pretty aware of these techniques, and keep on changing the IPs too, so that extra squeeze with such optimization can make sense at scale.

---

Now that @platinum mentions it, I remember that @voluum does IP Anonymization to comply with GDPR, by setting the last 2 bytes to 0.
See: https://doc.Voluum.com/en/ip_anonymization.html

They only have IP filtering using the full IP while redirecting the visit.
https://doc.voluum.com/en/ip_filtering.html

Therefore, I don't think you can do bot analysis afterwards by IP at least with your tracker. You could potentially do that using an analysis of your webserver logs. I would not bother with that while starting, unless you really know what you are doing.

---

This. Datacenters, IP ranges. How do affiliates optimize based on this? I believe this is the most common thing I heard when it comes to IP addresses.

---

It depends what you want to do.

1) If the goal is to absolutely avoid any visit coming from a data center and/or spy tool, then a good cloaker is what you need. For example MagicChecker
https://magicchecker.com/

Some trackers have built their own solution

Voluum has the anti-fraud kit
https://Voluum.com/anti-fraud-kit/

Thrive launches its module in 2020
https://thrivetracker.com/blog/the-c...ow-available/3

Binom has a MagicChecker integration, so I am not sure if they built there own
https://docs.binom.org/magicchecker.php

2) If the goal is to limit the impact of bots on your media buying cost, but you don't mind if they see your page from time to time, then you can use bot traps to detect the likeliness of a visit to be a bot. It is one of the feature of the Voluum anti-fraud kit. And you can also use tools to automate placement cutting like theoptimizer.io, Voluum automizer, brax.io. On this I let @platinum elaborate :-)

---

Like @jeremie mentioned, your best bet would be to use a cloaker to filter out junk bot traffic. However, if you're not marking the placements where this traffic is coming from, then you'll still have to go and check things manually on a 3rd platform to make sure you block these placements in order to prevent over-spending on them.

If you're looking to get rid of visits from spy tools or scrapers that inflate your campaign link visits count, or even visits from your competitors that navigate your campaign link from spy tools, then you can start with your tracker. Simply redirect all unmatched traffic to a smartlink like Monetizer or another one you can source from your preferred offer network. You can get this done quite easily by simply building a rule-based flow on a tracker like Voluum.
On top of that, instead of having your click URL end up on a Not found - Error 404 page, you can point to the smartlink again when landing page visitors aren't coming from your campaign link.

Furthermore, when it comes to automation tools like TheOptimizer, you have two options:
1- Block low engagement/performance publishers using your landing page clicks count or click-through rate. This is an early indicator based on which you can easily calculate the bare minimum clicks thresholds for getting the publisher profitable. However this won't give you any advanced insights on the quality of traffic related to bot or real visits/clicks.

2 - Combine automatic optimization capabilities with tracker custom conversion events. I've already tested this with TheOptimizer and Voluum and personally I think it works pretty well.
What you can do is to add a simple hidden link or something alike that a real visitor might not be able to click/fire an event for, then connect this event with a custom conversion. As this event is tracked within your tracker, then all you need to do is to specify the automatic rule conditions that will utilize this event to block fake visits/clicks.

Here's a simple example:



As you can see from the above example, we're checking the landing page clicks count vs. bot hits and if they are equal, then we're blocking campaign publishers.

However, it is important, to be a bit conservative when creating such rules, as it may happen that you get mixed traffic from some publishers, and as long as a placement is profitable, you have no reason to block it.

Also, something really neat that you can do with TheOptimizer Native (that other automation tools cannot do) is that by using Smart Lists you can automatically add the publishers blocked by such rules, directly on a block list. This way, if you're targeting the same geo/vertical on another campaign, all you'll need to do is to assign the list to the campaign and publishers will get blocked without having to spend much on them - a visit or two will be just enough to validate their presence on the campaign and get them blocked by TheOptimizer.

---

Home > Technical & Creative Skills > Programming, Servers & Scripts