AI Can’t Measure What Isn’t Real: The Crisis of Fake Engagement

It seems like the term “Fake news” has been everywhere over the last decade. It’s evolved to the point now that it’s seen more as a flippant remark than an actual danger or threat. When everything is challenged, second-guessed or simply ignored, it’s lost a lot of its power. Fake news is no longer the significant threat that’s going to knock the news media’s house down.

What has grown up in the shadows and is emerging as an even bigger threat is fake speech. Free speech, supported and aligned with fake speech, has flipped the way we interpret public sentiment. Ideas that once seemed fringe are now echoed by thousands of accounts, gaining traction and complexity through widespread engagement.

Nefarious narratives are now amplified worldwide, blurring the line between public opinion and coordinated manipulation. What is behind this phenomenon? One of the answers is Bot farms.

Bot farms used to be regarded as sophisticated government actors working together on off-the-books operations, trying to do things like sway government elections, elicit riots and general nefarious behaviour. Today, they are far simpler from a setup perspective, but almost impossible to detect online.

Bot farms generally consist of thousands of smartphones that are controlled by one computer and can be programmed to use fake social media accounts and broadcast coordinated likes, comments, and shares. In days past, bots relied on software and API access. Today, forums have detailed explanations on how to use rows and rows of smartphones with real SIM cards connected to USB hubs. They’re almost impossible to detect as they also use mobile proxies, device fingerprinting, and geolocation spoofing. They are so advanced that, according to the New York Post, they can now pass the Turing test and escape detection.

The line between authentic and inauthentic activity has become incredibly blurred.

The improvements in the technology have also aligned with a decrease in cost. What was once state-aligned activity is now open to all. Platform marketplaces like Fiverr and Upwork have commercial bot farms openly advertising their trades under services such as “social media marketing panels” or “growth marketing services” according to Fast Company. Customers are charged around 1p per action, and this can include following accounts, liking posts, leaving comments, or visiting websites.

Industries that can be heavily influenced by public sentiment have been especially hard hit. Cryptocurrency has seen thousands of “Pump and Dump” schemes that have been facilitated through sophisticated bot farms targeting users on message boards like X (formerly known as Twitter), Discord, and Reddit. Websites like Stocktwits, which is often pushed as the premier place to discuss and learn stocks, utilise a sentiment indicator that offers a metered score of 0-100 to help market participants identify how bullish/bearish retail participants are around a stock. Great in principle, not so great in reality.

Market manipulators use bot farms to shape the online narrative they are looking for. They amplify fake posts about stocks that are supposedly in demand by using targeted language and messages on the aforementioned platforms. These messages are full of ticker symbols and positive financial slang like “BTD – Buy the dip”, “wen lambo”, and “diamond hands”. Hundreds of thousands of messages are coordinated by a handful of players who, in turn, make substantial profits based on the direction of the stock they have manipulated. Whilst the everyday investor has provided them with their hard-earned capital based on a speculative gamble that was never real to begin with.

With these smaller stocks, there was never a news story to begin with or a technical indicator. It’s bots who have been masquerading as real people.  

This is just one area of manipulation. It can be applied everywhere, and in today’s age, where everything is automated and analysed by AI, it’s a scary thought how easily systems can be manipulated at scale. The applications are endless. They can sway elections, tarnish reputations, incite riots, and amplify dangerous lies.

Hollywood has seen its fair share of this with the recent court cases of Johnny Depp and Amber Heard and more recently Justin Baldoni and Blake Lively. In both cases, each side accused the other of shifting the narrative through the use of smear campaigns and bots.

On the surface, it should be relatively easy to spot but these methods are generally used to get the ball rolling rather than paint the entire picture. Often, a real post will be identified as fitting into a specific narrative and, in turn, will be boosted by bot activity. This sparks further organic activity, and the ball is now rolling. If the narrative is a strong, salacious one, it will continue to amplify organic activity due to the algorithms of social activity. This gets put in front of more people and so on and so on.

Free speech is the pillar of modern democracy, but is it still free speech when it was never ‘real’ to begin with?

We are standing on the precipice. Free speech is the pillar of modern democracy but is it still free speech when it was never ‘real’ to begin with?

Every week, someone new goes viral and their time spent in the spotlight gets further and further reduced. Attention quickly shifts to something new, and the old is passed over. Just as quickly as it arrived, it’s forgotten. For a brand, this can be either a blessing or a curse.

You need to know where it came from, how it came about, and how it spread. Answering these questions is essential to determining your influence on a story, destroying negativity at the source, or simply working out what’s real and what’s not when it comes to public perception.

This is where a specialist in global media intelligence is imperative. Separate the fact from the fiction, measure what’s meaningful and demonstrate the value of your PR efforts.  

At CARMA, we combine cutting-edge AI tools with expert human analysis to ensure your reputation is grounded in truth, not manipulated perception. Talk to us to learn more.

تحدث مع أحد مستشارينا ذوي الخبرة حول الرصد الإعلامي والاتصالات الخاصة بك اليوم.