(1) Weird Way FB Are Screwing Your Stats (34)

This is important so don't ignore it.

This facebook issue impacts your campaigns - whether you use Voluum, Thrive, 202, Noipfraud, JCI or any other tool.

If you are running Facebook Mobile ads you may have been wondering why your stats are all over the place!

Due to a massive change in Facebook's main Android app, volumes of tracked "clicks" exploded for Facebook advertisers recently.

Careful analysis by our team highlighted that these visits were “forced” by Facebook and as such can cause your stats to be a mess.

Its Facebook...as usual!



What a great idea FB Draining your users batteries & data plans even more....

In their wisdom to "speed up the web" they have started loading the ad URLs in the Android app when the ad is shown to the user.

Basically causing a hit to your ad URL on each impression!

This is madness and is making people's data usage and battery even worse than it was before.

​I am surprised by this action from Facebook as it's a very aggressive approach to "speeding up the web" and will no doubt get them in a bunch of trouble again.

Results from our analysis

Our team have investigated this and I want to share the conclusions with you here.

A few things on our initial investigation flagged up some interesting facts about this traffic:

• We saw an influx of traffic that did not send an Accept-Language header
• All of this traffic contained X-FB-* headers but did not necessarily send the usual FB User Agent identifier
• It only seems to affect Android App at the moment
• Connections are real mobile visits but behaviour wasz not
• A small percentage of this traffic does actually convert

Explains a lot, finally! Thanks.

Excellent info! Thanks NoIpFraud! You rock!

I wouldn't say this is specifically aimed at ads. More so all newfeed content with external links / links to other pages to make the overall experience faster. Messing with stats just a very unwelcome side effect.

I'd say all the trackers need to get on this ASAP and add an option to disable pre-fetch (done with some simple header checking code).

Originally Posted by bobliu

I'd say all the trackers need to get on this ASAP and add an option to disable pre-fetch (done with some simple header checking code).

Just for those that doubt the impact of this Facebook Android issue -

we process millions of facebook ad clicks a day - here is the % of traffic that was pre-fetch traffic for the first few days we understood the issue and were able to monitor it and roll out a fix:



And here is the last 8 days



As you can see it reduced from 35% of traffic down to around 2% as of today. This is as a result of our users implementing the fix we provided.

Is anyone seeing similar issues in your trackers? If so - what is the trackers support saying about this?

Just simply blocking these pre-fetch requests is not a solution. Because the blocked page will be cached and never get re-requested as a non-prefetch visit. Which means real clicks will not make it to your pages at that point.

Excellent info, please, keep continue

Thanks for the info, great help!

Thanks for the info. I hope Voluum fix this as well.

Originally Posted by hangman

Thanks for the info. I hope Voluum fix this as well.

Originally Posted by noipfraud

Has Voluum said anything about this ?

What about other trackers @shakedown - heard anything from them?

Hi guys,

We’d like to confirm that we are aware of this new Facebook update and the problems it may cause our customers. Our Dev. team is already working on solving this as quickly as possible, and we should have an update by the beginning of next week.

As "NoIpFraud" mentioned below, there is no easy fix, but we will do our very best to solve this issue as soon as possible; our users' satisfaction is the most important thing for us.

Kind regards,
Voluum Team

Thanks @Voluum for confirming this

Originally Posted by Voluum

We’d like to confirm that we are aware of this new Facebook update and the problems it may cause our customers. Our Dev. team is already working on solving this as quickly as possible, and we should have an update by the beginning of next week.

Originally Posted by Voluum

Hi guys,

We’d like to confirm that we are aware of this new Facebook update and the problems it may cause our customers. Our Dev. team is already working on solving this as quickly as possible, and we should have an update by the beginning of next week.

As "NoIpFraud" mentioned below, there is no easy fix, but we will do our very best to solve this issue as soon as possible; our users' satisfaction is the most important thing for us.

Kind regards,
Voluum Team

Here is the post explaining this fix

We found that with the android app if you scroll and an ad is in view, the FB app will make a request to your ad URL and follow any 301/302 redirects it sends.

These preview requests can be identified by the x-purpose: preview HTTP header that is sent.

We found this fix by sniffing traffic on a Facebook Android app, watching what adverts were triggering a reload of the destination URL on click (rather than using the cache), and then identifying and testing variations of the method they use.

We noticed that ads running on the Nanigans platform were not affected by this prefetch issue and caused FB to re-request the page, not using a cached copy.

They do this by sending a Content-Length greater than 0 along with their 302 redirect. This breaks the FB Android prefetch and forces the App to load your destination URL again when the Ad is clicked.

We found that this method is the most natural looking, avoiding any obvious footprint.

Other methods that work are letting the page fail with a non-200 OK HTTP codes (ex. 404, 500, ... ).

Therefore, for prefetch requests, we implement the Content-Length method in addition to a 302 to an unknown path.


Fix in PHP

Here is an example script of fixing this issue in PHP. Its got a ton of comments in there to explain what all the code is doing.

<?php
//only php on apache has getallheaders function
//so this defines the function for other webservers like nginx
if (!function_exists('getallheaders')) {
    function getallheaders() {
        $headers = '';
        foreach ($_SERVER as $name => $value) {
            if (substr($name, 0, 5) == 'HTTP_') {
                $headers[str_replace(' ', '-', ucwords(strtolower(str_replace('_', ' ', substr($name, 5)))))] = $value;
            }
        }
        return $headers;
    }
}

//make sure all header keys are lower case so its easier to test
$lc = array_change_key_case(getallheaders(), CASE_LOWER);

//set a fall back URL to redirect prefetch to in case fix stops working
//in our case were forcing prefetch to load non existing random page on your domain
//because a 404 response will also force the FB APP to reload
//the destination URL on ad click
$fallbackurl='/'.substr(md5(microtime()),0,rand(1,12));

//check if x-purpose header is sent with value "preview"
if (!empty($lc['x-purpose'])&&strtolower($lc['x-purpose'])=='preview') {
    //set a fallback redirect - if this fix fails
    //facebook android app will follow this redirect
    header("Location: $fallbackurl");

    //our testing showed that setting this header
    //results in the FB android app not following
    //the redirect and therefore not caching the result
    //when user clicks on ad - app will just load dest URL again
    header('Content-Length: '.rand(1,128));exit();
}
?>

AWESOME @noipfraud I've already passed the info to the dev team. I'll get back to you as soon as I have an update.

Thank you for helping not only the trackers but the whole STM community!

Can anybody advise a fix for prosper202? I guess I can use that php code but where to put it exactly?

Thanks a lot @noipfraud!

We have implemented this patch in FunnelFlux and it will be pushed to customers this week with our new upgrade.

We have taken a slighly different approach. Instead of using a fallback url, the whole funnel is being processed normally, even if it comes from FB's prefetcher. This allows to apply any rule / filter created by our users. The hits are just not saved in the DB and the content-length header is being appended to force reload the funnel at the time the user clicks.

Thanks again for sharing this.

Originally Posted by andyman

Can anybody advise a fix for prosper202? I guess I can use that php code but where to put it exactly?

Will be implementing. Thanks a lot guys

great stuff Tom!

One request though - make sure you do your own testing and verification - as this is a method not documented by facebook it is very possible the workaround may stop working in the future as and when they change things in their apps. We will of course keep an eye on it at our end but how it impacts your customers may be very different.

HI guys,

Thank you very much @noipfraud.

I implemented the patch on my tracker but it doesn't seem to work...




Does anybody has ever seen a loss of clicks that big on facebook ? Offer is targeted to Ecuador, 3 clicks from Facebook USA and then all the other ones come from me testing the click tracking...

Is this your own tracker zouyah (as in inhoude developed)?

If its not - dont try and implement this within a tracker that you have not developed yourself. Wait for the tracker support to provide a patch that has been properly tested.

If it is your own tracker i suggest using our code as guidelines but implement your own code that works with your inhouse tracker.

Thanks for your answer. Yes it's my own tracker. So this patch is still working for you... Was the loss of clicks that massive for you too before ?

This patch is not here to fix click loss, it's the other way round. Without this patch, we see MORE visits coming from facebook, not less, because the hits from their prefetching agent were processed as real visitors.

Hi Vitavee. My guess was that I was seeing a loss of clicks because facebook was prefetching the final destination URL... Thus bypassing my tracking by showing only the final destination (offer page) without going through my tracking. Any chances this is happening ?

Hi Zouyah. Difficult to say for sure. I would need to see your complete setup to give you an answer on this one. But ultimately, yes it's possible.

Switched to Voluum for the sake of testing, same issue... The campaigns are in tier tier 3 geos but I can't believe I loose 4/ of the traffic because of loading times. Any guess of what it could be ? Please help

hey zouyah - i think your issue is not related to the topic of this thread as vitavee suggested. If your loosing clicks because of load time best is to start a new thread and maybe post some example links and descriptions of setup. Certainly not use the script we posted as it is for a different issue.

It looks like this happened again? Just before the weekend clicks are like 10x more in my dashboard vs Facebook stats.
Also see it network wide, anybody knows why?

yes correct we have seen this with some users. Version 1.8 of noip has a filter build in which you can enable to remove these visits and ensure the prefetch result is not cached and force the visitor to reload your url when the visitor actually clicks the ad. no idea why its reared its head again. maybe facebook doing more testing.

Yes i have similar issues. i get less clicks through my url now on mobile. FB is reporting more click than my tracker.

@hmedia

That doesn't sound like a prefetch issue, and your loosing clicks somewhere. I suggest you get in touch with your tracker provider or review your traffic flow to figure out why there may be click loss like that. If you use noip then give us a shout on the chat in the app and we can help you diagnose it.