If you use social media at all then the chances are you’ve heard the “fake news” and “Russian bot clicks won the US Election for Donald Trump” clarion calls.
But how much truth is there in these allegations? Can bot clicks have that much influence on our daily digital lives? Well, if you work with digital advertising, then the answer is an unequivocal yes.
Half, or more, of paid online display adverts have never been seen by an actual person. – Google
This stark statistic from a landmark study conducted by the Association of National Advertisers (ANA) and digital security firm, White Ops, is backed up by Cnet and other studies and is something anyone with an ad spend budget should be aware of.
What Are Bot Clicks?
At its simplest, a bot click is when a script or software program using an auto bot commands information from a website link in the same way a human clicking on a link with their mouse would do.
Also known as click fraud in email campaigns or banner ads, it can frequently mean you – or your client – is paying for clicks to a website which aren’t from humans and who therefore won’t be real potential customers for your goods and services.
The scale of the click-bot problem
In 2016 Forbes exposed the Methbot scam which highlighted how security firm White Ops had discovered how a group of Russian criminals were making between $3 and 5 million every day through false bot clicks attacking the advertising market.
However, according to the company’s 2018-29 Fourth Bot Baseline: Fraud In The Advertising Industry Report, carried out in partnership with the Association of National Advertisers: “For the first time, the majority of fraud attempts are getting stymied before they are paid for, by DSPs and SSPs filtering fraudulent bid requests, by clawbacks, or by other preventative measures.”
Fraud attempts amount to 20-35% of all ad impressions throughout the year, but the fraud that gets through and gets paid for now is now much smaller.
Thanks to traffic sourcing transparency efforts led by the Trustworthy Accountability Group (TAG), traffic vendors have gone further underground, reducing “retail” bot buying on the open web.
ads.txt (Authorized Digital Sellers) has reduced domain spoofing. Additionally, TAG requires publishers to have completed ads.txt files if they want to be Certified Against Fraud.
This initiative from IAB Technology Laboratory specifies a text file that companies can host on their web servers, listing the other companies authorised to sell their products or services. It allows online buyers to check the validity of the sellers from whom they buy.
How effective is reCAPTCHA in preventing bots?
Google’s introduction of its simple captcha, which requires a click in a checkbox, could be considered as one reason why fraudulent sign-ups for services or filling in of forms are being abused less.
Instead of depending upon the traditional distorted word test, Google’s “reCaptcha” examines cues every user unwittingly provides: IP addresses and cookies provide evidence that the user is the same friendly human Google remembers from elsewhere on the Web.
And Vinay Shet, the product manager for Google’s Captcha team, says even the tiny movement a user’s mouse makes as it hovers and approaches a checkbox can help reveal an automated bot.
But it’s just one area of a very wide marketplace and not every website or service is one which would require the use of a captcha.
As White Ops’ co-founder and president Michael Tiffany says in his preface to the latest report: “Now is the time for advertisers to push for the ability to hold all ad impressions to the same high standard of validatability.
“Imagine if every CAPTCHA on the internet was the same. It wouldn’t work very well. Validation with only a pixel is like serving a CAPTCHA that never changes. Only a dynamic challenge can be used to catch a dynamic adversary.”
Not All Bots Clicks Are Bad
However, not every bot click is bad. There are a number of automated programmes which trawl the world wide web for good reasons, Google’s own spider bot Googlebot included. The most common of these include:
- Copyright Bots – does what it says it does, looks for articles or content which may have been plagiarized or illegally uploaded.
- Trade/Shopping Bots – these are the ones which search for the best available deals across multiple websites for whatever item you’re looking to buy.
- Data Bots – like the BBC Weather App, which will search for the most up to date information available on a particular subject.
- RSS Bots – these pull together news articles on a subject and present it in an easily digestible format.
These are just some of the examples of bot clicks being carried out for beneficial reasons. But how do you identify the good from the bad when it comes to website traffic or daily interactions on social media?
How do you know if a bot is clicking?
There are a number of methods to try and identify bot clicks, some are easier than others and most start with Google Analytics.
Checking your Google Analytics on a regular basis makes it easier to spot irregularities which may be down to bot clicks. Monitoring the average number of page views, average session time, and the source of referrer traffic makes it easier to spot bot visitors which don’t follow the same patterns.
Examples of bot click traffic
- Page duration;
- bounce rates and activity on-site;
- do the “visitors” go to more than one page;
- do they hit every single page on your website within seconds;
- are they hard bouncing the second they arrive;
Higher number of visits than usual, which results in big spikes in page views, can also be systematic of a bot directing traffic to your site. These can be doubly dangerous for a business as they can also slow down your site for genuine visitors.
All this requires someone to spend a lot of time on Google Analytics though. While it’s a fantastic free product and should be a staple in any business’ digital marketing toolset, it can be complex, and complicated, to get the best use of it.
How can Beacon help identify bot clicks?
Beacon’s all-in-one click fraud protection platform tracks every link click and website visit from the Beacon Links you create dynamically for your campaigns. Beacon uses Digital Journey Tracking to collate all visitor actions and uses this data for individual visitor behavioural and network analysis, as well as looking at dozens of parameters to perform advanced OS / device fingerprinting.
It also gives detailed analytics on your website traffic and page performances. And it provides valuable insights and intelligence into how your online marketing is going, making sure you’re targeting human visitors and blocking out bots.
The traffic from these links is analysed. Beacon then determines which are genuine human visits and which is likely to be a bot. Beacon displays the percentage of human/bot visits across the spectrum of digital marketing activity; from individual link, to channel, to campaign.
Get started with Beacon
Start your journey by having the best available data. At Beacon we’ve been champions of transparent marketing data since our foundation. With click fraud detection and mitigation across search and social, Beacon feeds other parts of the Martech stack with better quality data, to enable improved analytics, decision-making and results.
See what Beacon can do for you or the brands you represent by watching a 4 minute demo video. Or calculate how much you might be losing on your paid media campaigns to click fraud with our click fraud Calculator.
NOTE: Updated March 2022.