...with analysis & insight...
Online nobody knows you’re a bot...
Human beings are naturally good at deciding when somebody is trying to cheat them. Evolution
has created these skills over eons of time. Often we are so good at not being cheated that we don’t
even recognise how we successfully avoided a trap. Much of our skill lies in the interpretation of
minute visual facial cues and nuances that helped build the trust for early humans to live in
communities and work together. Yet today a growing number of our interactions take place online
and we have yet to evolve the skills we need to not only survive but prosper in this new
People worry about artificial intelligence (AI) as though this was a problem for the future.
Unfortunately the future is already with us. Surprisingly, it only takes an extremely tiny piece of
artificial intelligence to fool us, just a few relatively simple lines of scripting code. For example,
frequently those Facebook likes, or Twitter tweets, or trending stories on Reddit that you receive
haven’t originated from a human being but come from an automated “bot” and you probably didn’t
even notice. Today there is more traffic from automated bots on the Internet than there is human
traffic and as the chart above shows – unfortunately there are more bad bots than good bots. That
means a lot of people are being duped a lot of the time when they use the Internet.
Unless you are somebody who follows Internet trends closely you probably know very little bots
but its important that you start to learn about them - what they can do and how they fool people
into believing they are human. And their numbers are rising. Don’t get me wrong I’m not against
bots - some good bots can be very useful, its the bad bots you have to be concerned about. Bots
are a made up of a series of automated instructions (scripts) that can be run over the Internet.
They are becoming increasingly sophisticated and therefore very much harder to detect. In the
early days of the Internet a Web server could control the behaviour of a bot by having a “robot.txt”
file that instructed what a bot could or could not do with the information on that Web server. Web
servers identify humans apart from bots because humans use browsers to view Internet content
and bots don’t. Now many bots disguises themselves as humans by including a few extra lines of
code. Today Web servers are fooled by bots who appear to be using a Web browser and can also
appear to emulate human mouse click behaviour. Humans are just as easily fooled sometimes to
For example: a few rudimentary lines of code instructed a small army of bots masquerading as
humans to help Russia support Donald Trump’s election campaign. Maybe Trump would have won
in any event, we will never know the effect that thousands of non-human Twitter accounts firing off
their messages in the alphabetical order of their false identities, a few seconds apart, on the day of
the election had on the U.S. result. Each message was tailored to appeal different groups of people
based on their previous Twitter behaviour so the message would chime with their own personal
biases. Most people thought these messages were coming from real people, few people realised
these messages were from automated bots with fake human identities.
Gartner, the information technology consultancy, has amongst their 2018 predictions forecast that
“By 2022, the majority of individuals in mature economies will consume more false information
than true information.” To which I would add that the bulk of that false information will be coming
from bots disguised as humans. Of course some pundits think this prediction is absurd, to quote
Steve Andriole writing in Forbes magazine “While I agree that fake news will increase, it’s
impossible to predict what regulations might appear, what real-time fact-checking might occur, and
how the technology giants will address the fake news problem. Bad prediction.” I disagree and
think that this view greatly underestimates the rapid progress being made in bot technology.
Gartner being a technology consultancy has understood that we are about to experience an
invasion of bots.
Fake news is a new name for old style propaganda with a new social media twist. These days the
quantity of fake news is being greatly amplified by fake human messenger bots. Part of the reason
for this is that it is getting very, very easy to make a bot. Back in 2016 at Microsoft Build 2016
conference, CEO Satya Nadella said "Bots are the new apps," as he announced Microsoft’s Bot
Framework. This is a set of tools for software developers that does a lot of the lower level work in
building a sophisticated bot. This Bot Framework has now reached the marketplace as has
Facebook’s Workplace Bots. Facebook even has instructional videos like this one on how to build a
great bot. And you can even build a bot without coding using tool like Converse AI.
The chart above uses data from Incapsula’s 2016 Bot Traffic Report, so let’s look at some of the
good bots and the proportion of Web traffic they take up. Firstly, there are the “monitoring” bots
that check that websites and all their functions are up and running correctly. Monitoring bots make
up 1.2% of Web traffic and they help maintain the sites we visit. The second smallest proportion of
bot Web traffic is generated by “commercial” bots that crawl the Web automatically extracting data
from the authorised websites they visit. Think price comparison websites or advertising or
marketing trackers. To my mind some of these bots border onto the dark side of the Web although
compared to the bad bots these commercial bots behave in a much less malign way. These
represent 2.9% of Web traffic. Thirdly, there are search engine bots (sometimes called spiders and
crawlers) like the Googlebot or Bingbot that captures and indexes Web pages so they can be
retrieved by search engines. Most people want their website content to be discoverable by people
using search engines so these are considered good bots and they can be controlled using a
robots.txt file mentioned earlier. Search engine bots make-up 6.6% of Web traffic. The last and
most frequent good bots are the “feed fetchers” that capture content from websites and shuttle it
to mobile and Web applications e.g. news feeds. Feed fetchers make-up 12.2% of Web traffic.
Turning to the bad bots, these can range from being just a nuisance to the criminal or downright
diabolical. You will probably noticed the results of “spammer” bots, although they only make-up
0.3% of Web traffic. Using millions of captured email addresses spammer bots automatically
proliferate messages about Ukrainian ladies seeking love and sex or Nigerians seeking investment
partners. Spammer bots are the polluters of the Web although good spam filters can block the
majority of their output. 1.7% of Web traffic comes from “scraper” bots who lift unauthorised data
for websites, including content to repackage for free or the prices used by commercial websites.
2.6% of Web traffic originates from “hacker” bots that are permanently scouring the Web for
vulnerabilities to be exploited by data theft and malware injection. The creators of these bots can
be criminals after money or foreign states searching for secret information. A good example of the
latter is the constant attacks by both Western powers and the Russian FSB to use Kaspersky anti-
virus software as a search engine to detect top secret information.
By far the largest amount of Web traffic from any type of bot at the moment comes from
“impersonator” bots at 24.3% of Web traffic. Using false identities to bypass log-in details these
bots break most types of security. Identified as human these impersonator bots can be bought by
the thousand for Facebook Likes, Twitter or Instagram Followers. Hard to detect on social media
because their identities have usually been created manually, they are difficult to distinguish from
real people. They even come complete with matching mobile numbers for two step identification.
If you want to understand how this is done read this article, alternatively select a country and build
your own profile using an online Random Identity Generator. Some Facebook advertisers reckon
that 90% of their clicks can come from bots and not human beings. So far their only solution to the
problem is to try and claim a refund. When combined into automated armies, impersonator bots
can bring down government websites and freeze a postal tracking service, or extort money from
commercial websites. These attacks are called Distributed Denial of Service (DDoS) and are more
common than you think. Check out this website to view real-time attacks that are happening today,
or view previous attacks and the traffic they generated.
It is ironic, that Alan Turing, the mathematical genius who conceived the modern computer in the
1930s, created his famous “imitation test” to show how it was possible for a machine to appear to
convince a human that it was capable of thought. Now commonly called the Turing Test, examples
of this are played out by the millions everyday as humans interact with fake human bots on
Facebook, Instagram and Twitter in complete acceptance they are human. Without noticing it, we
are undoubtedly going into a future where an increasing amounts of our time will be spent
interacting with machine (artificial) intelligence at the expense of interaction with our fellow
humans. What the current data above indicates is that unfortunately more time is being spent
interacting with bad bots that good bots. You have been warned!