Pages

22 June 2019

Twitterbots: Anatomy of a Propaganda Campaign

Gillian Cleary

Internet Research Agency archive reveals a vast, coordinated campaign that was incredibly successful at pushing out and amplifying its messages.

Key Findings

The operation was carefully planned, with accounts often registered months before they were used – and well in advance of the 2016 U.S. presidential election. The average time between account creation and first tweet was 177 days.

A core group of main accounts was used to push out new content. These were often ”fake news” outlets masquerading as regional news outlets or pretended to be political organizations.

A much larger pool of auxiliary accounts was used to amplify messages pushed out by the main accounts. These usually pretended to be individuals.

The campaign directed propaganda at both sides of the liberal/conservative political divide in the U.S., in particular the more disaffected elements of both camps.

Most accounts were primarily automated, but they would frequently show signs of manual intervention, such as posting original content or slightly changing the wording of reposted contented, presumably in an attempt to make them appear more authentic and reduce the risk of their deletion. Fake news accounts were set up to monitor blog activity and automatically push new blog posts to Twitter. Auxiliary accounts were configured to retweet content pushed out by the main accounts.


The most retweeted account garnered over 6 million retweets. Only a small fraction (1,850) of those retweets came from other accounts within the dataset, meaning many of the retweets could have come from genuine Twitter users.

One of the main talking points of the 2016 U.S. presidential election campaign involved attempts to surreptitiously influence public opinion using social media campaigns. In the months after the election, it quickly became apparent that a sophisticated propaganda operation had been directed against American voters.

Not surprisingly, news of these campaigns caused widespread public concern, prompting social media firms to launch investigations into whether their services had been misused. In October 2018, Twitter released a massive dataset of content posted on its service by the Internet Research Agency (IRA), a Russian company responsible for the largest propaganda campaign directed against the U.S.

The dataset consisted of 3,836 Twitter accounts and nearly 10 million tweets. These accounts had amassed almost 6.4 million followers and were following 3.2 million accounts. The sheer volume of data was enormous, more than 275 GB.

The archive has proven to be a treasure trove of information on how the IRA’s propaganda campaign operated. For example, prior to the release, many people assumed that its posts were focused on just one side of the political spectrum. Once the data was made public, it quickly became obvious that in order to achieve its goal, the campaign directed propaganda at both sides of the liberal/conservative political divide in the U.S., in particular the more disaffected elements of both camps. The main objective of the campaign instead appeared to be sowing discord by attempting to inflame opinions on both sides. This was not just confined to the online sphere. Several of the accounts were used to organize political rallies in the U.S. and some of the most influential accounts in the dataset were used to promote these events to the largest possible audience.

However, believing that there is a lot to learn from this data beyond its messages and target audience, we decided to carry out some in-depth analysis of the archive to learn more about how this propaganda campaign worked. What we discovered was that this was not an ad-hoc response to political events in the U.S. Instead, the evidence points to a carefully planned and coordinated operation, with the groundwork often laid months in advance.

While the tactics employed changed somewhat over time, the basic template for this operation remained the same, utilizing a small core of accounts to push out new content and a wider pool of automated accounts to amplify those messages.

Along the way, we also came across some interesting bits of information, such as what appeared to be some rogue operators using monetized link-shortening services to make some money on the side.

Different account types

Once we started analyzing the data, it became apparent that the accounts could be divided into two main categories, which we called main accounts and auxiliary accounts. Each category had different characteristics and played a different role.

Main accounts had at least 10,000 followers but followed substantially fewer accounts. They were primarily used to publish new tweets.

Auxiliary accounts had less than 10,000 followers, but often followed more accounts than that. Their main purpose was to retweet messages from other accounts, although they were also used to publish original tweets. Not surprisingly, the majority of accounts were auxiliary accounts. We identified 123 main accounts and 3,713 auxiliary accounts within the dataset.

Main accounts generally were ”fake news” outlets masquerading as regional news outlets, or pretending to be political parties or hashtag games—the popular Twitter game that involves people sharing anecdotes or jokes based on a single theme, such as #5WordsToRuinADate. Based on their creation date they were usually created individually or in small batches. The default language selected for main accounts was always either English or Russian.

Auxiliary accounts usually pretended to be individuals, spreading the content created by the main accounts by retweeting it. These accounts were usually created in batches and sometimes hundreds of auxiliary accounts were created on the same day. For example, during May 2014, seven fake news accounts were set up by the agency, along with 514 auxiliary accounts.

Many of the accounts were created long before they were used. The average time between account creation and first tweet was 177 days. The average length of time an account remained active was 429 days.

Influential accounts

Some of the Twitter accounts created by the attackers managed to be extraordinarily influential. The most retweeted account within the dataset was TEN_GOP. Created in November 2015, the account masqueraded as a group of Republicans in Tennessee. It appears to have been manually operated.

In less than two years TEN_GOP managed to rack up nearly 150,000 followers. Despite only tweeting 10,794 times, the account garnered over 6 million retweets. Only a tiny fraction (1,850) of those retweets came from other accounts within the dataset. In other words, almost all of its retweets came from accounts outside the dataset, meaning many could have been real Twitter users.

Due to the popularity of this account, the IRA created two backup accounts in case TEN_GOP should be discovered and shutdown—ELEVEN_GOP and realTEN_GOP—both of which have the same profile description: “This is our backup account in case anything happens to @TEN_GOP”.

As has already been noted, the IRA’s campaign targeted both ends of the political spectrum in the U.S. and this is reflected in the breakdown of influential accounts. The top 20 most retweeted English-language accounts were split evenly between conservative and liberal messages.

Table 1. Twenty most retweeted English language accounts
User nameProfile descriptionRetweets
TEN_GOPUnofficial Twitter of Tennessee Republicans. Covering breaking news, national politics, foreign policy and more. #MAGA #2A6,000,296
Crystal1JohnsonIt is our responsibility to promote the positive things that happen in our communities.3,776,481
Pamela_Moore13Southern. Conservative. Pro God. Anti Racism2,223,854
wokeluisaAPSA. #Blackexcellence. Political science major1,217,081
JemiSHaaaZzzTeacher, Reader, Writer, Resister. #Resistance RESPECT LOVE LOYALTY1,171,652
KaniJJacksonFollow the example set by Mrs Obama; peace, love, acceptance & vigilance #Impeach45 #Resist #GunReformNow821,178
gloed_upNo black person is ugly #BRONZE #BlackLivesMatter #BlackToLive787,521
TrayneshaColeLove for all my people of Melanin. Your BLACK is BEAUTIFUL! #MyPussyMyChoice #BlackGirlsMagic #BlackLivesMatter585,768
Jenn_AbramsCalm down, I'm not pro-Trump. I am pro-common sense. Any offers/ideas/questions? DM or email me jennnabrams@gmail.com (Yes, there are 3 Ns)559,978
SouthLoneStarProud TEXAN and AMERICAN patriot #2a #prolife #Trump2016 #TrumpPence16 Fuck Islam and PC. Don't mess with Texas!557,781
BlackNewsOutletFreedom is never given; it is won. #BlackLivesMatter491,973
BlackToLiveWe want equality and justice! And we need you to help us. Join our team and write your own articles! DM us or send an email: BlackToLive@gmail.com478,238
TheFoundingSonBusiness Owner, Proud Father, Conservative, Christian, Patriot, Gun rights, Politically Incorrect. Love my country and my family #2A #GOP #tcot #WakeUpAmerica454,868
BleepThePoliceFor a second at least, I'm resurrecting the peace #Blacktivist #BlackLivesMatter306,157
USA_GunslingerTruth is strong, and sometime or other will prevail!267,192
tpartynews[NO DATA]144,508
PamelaKealer13[NO DATA]114,198
Jeblary20162016 US presidential candidate106,553
10_gopConservative, daily news that you won't find on mainstream media. #MAGA Retweeted by @realDonaldTrump 😇105,587
Blk_VoiceActivist. Feminist. Celebrating and highlighting Black excellence.103,967

There is a similar split in themes in the breakdown of the most followed English language accounts. Thirty-five percent pretended to support conservative causes, while 30 percent pretended to back liberal causes. The remainder masqueraded as general or political news outlets.

Table 2. Twenty most followed English language accounts
User nameProfile descriptionNumber of followers
TEN_GOPUnofficial Twitter of Tennessee Republicans. Covering breaking news, national politics, foreign policy and more. #MAGA #2A147,767
Jenn_AbramsCalm down, I'm not pro-Trump. I am pro-common sense. Any offers/ideas/questions? DM or email me jennnabrams@gmail.com (Yes, there are 3 Ns)79,152
Pamela_Moore13Southern. Conservative. Pro God. Anti Racism72,121
TodayNYCityNew York City's local news on Twitter. Breaking news, sports, events and international news. Tweet us or DM66,980
ELEVEN_GOPThis is our back-up account in case anything happens to @TEN_GOP59,279
wokeluisaAPSA. #Blackexcellence. Political science major57,295
Crystal1JohnsonIt is our responsibility to promote the positive things that happen in our communities.56,581
SouthLoneStarProud TEXAN and AMERICAN patriot #2a #prolife #Trump2016 #TrumpPence16 Fuck Islam and PC. Don't mess with Texas!53,999
TheFoundingSonBusiness Owner, Proud Father, Conservative, Christian, Patriot, Gun rights, Politically Incorrect. Love my country and my family #2A #GOP #tcot #WakeUpAmerica47,413
USA_GunslingerTruth is strong, and sometime or other will prevail!44,895
WashingtOnlineBreaking news, weather, traffic and more for Washington. DM us anytime. RTs not endorsements40,762
BlackNewsOutletFreedom is never given; it is won. #BlackLivesMatter40,458
PolitweecsTweets and politics36,094
NewOrleansONBreaking news, weather, traffic and more for New Orleans and Louisiana. DM us anytime. RTs not endorsements35,988
KaniJJacksonFollow the example set by Mrs Obama; peace, love, acceptance & vigilance #Impeach45 #Resist #GunReformNow34,276
WarfareWWPolitical & Military Analyst33,375
PigeonTodayAmerica’s Weakest Primetime Lineup Anywhere! Follow Merica's #1 cable news network, delivering you breaking news, insightful analysis, and must-see memes.32,347
BlackToLiveWe want equality and justice! And we need you to help us. Join our team and write your own articles! DM us or send an email: BlackToLive@gmail.com29,472
KansasDailyNewsLocal news, sports, business, politics, entertainment, travel and opinion for Kansas. DM us 24/729,357
gloed_upNo black person is ugly #BRONZE #BlackLivesMatter #BlackToLive28,943

Fake news

Most of the fake news accounts created by the agency pretended to be local news outlets, such as “New Orleans Online”, “El Paso Top News”, or “San Jose Daily”. The majority of these accounts were created between May and August of 2014, but lay dormant until January 2015, when most of them started tweeting. This suggests that the fake news element of the operation was planned well in advance.

The vast majority (96 percent) of these fake news accounts were fully automated, using services to monitor blog activity and automatically push new blog posts to Twitter. Another two percent of these accounts queued tweets for publication at scheduled times.

Most followed a broad pattern of activity, with activity trending upwards from the beginning of 2015 until the summer of 2016 when there was a sudden fall in activity. By August 2016, all but one fake news account had stopped tweeting.

This is likely because the fake news accounts had been using a service known as Twitterfeed to feed their blogs to Twitter. During 2016, Twitterfeed announced that it would close by October of that year and the fake news accounts began to transition to an alternative service called Twibble. The drop off in activity during August 2016 could have been caused by technical problems during the changeover. By December the changeover was complete and the fake news accounts had resumed business as usual.
Prolific accounts

Not surprisingly, a majority (55 percent) of the most prolific accounts were fake news accounts. However, a significant number of prolific accounts had their identities masked by Twitter because they had less than 5,000 followers. Most of these accounts acted as auxiliary accounts and were automated in the same manner as most of the fake news accounts. However, while fake news accounts were automated to publish original content, these other accounts were automated to retweet content.

Table 3. Top twenty most prolific English language accounts
User nameDisplay nameUser profile descriptionEnglish language tweet count
AnonymizedAnonymizedNo more #HappyHolidays shit!!! It's #MerryChristmas!!!!! #BuildTheWall #DrainTheSwamp #MAGAus¸ @realDonaldTrump133,315
TodayNYCityNew York City TodayNew York City's local news on Twitter. Breaking news, sports, events and international news. Tweet us or DM58,630
ChicagoDailyNewChicago Daily NewsLocal news, sports, business, politics, entertainment, travel and opinion for Chicago and Illinois. DM us 24/753,071
KansasDailyNewsKansasCityDailyNewsLocal news, sports, business, politics, entertainment, travel and opinion for Kansas. DM us 24/751,648
ScreamyMonkeyScreamy MonkeyFirst frontier.News aggreagtor.45,947
NewOrleansONNew Orleans OnlineBreaking news, weather, traffic and more for New Orleans and Louisiana. DM us anytime. RTs not endorsements45,894
DailySanFranSan Francisco DailyFollow for San Francisco's breaking news, special reports, entertainment and more from @DailySanFran44,711
AnonymizedAnonymizedNever Eat These Foods44,416
AnonymizedAnonymizedAll major information policy in the world, we are ready to give you the most accurate and timely information in the field of politics and economics35,016
AnonymizedAnonymizedWife, Mother, Patriot, Friend34,251
TodayPittsburghPittsburgh TodayPittsburgh's local news on Twitter. Breaking news, sports, events and international news. Tweet us or DM33,104
WashingtOnlineWashington OnlineBreaking news, weather, traffic and more for Washington. DM us anytime. RTs not endorsements32,880
RoomOfRumorRoom Of Rumor[NO DATA]31,126
Seattle_PostSeattle PostBe the first to know Seattle's local news. Politics, sport, entertainment, business, features and more that you need to know and share29,821
OnlineClevelandCleveland OnlineBreaking news, weather, traffic and more for Cleveland. DM us anytime. RTs not endorsements27,291
WorldOfHashtagsWorld Of HashtagsHosted by @GiselleEvns. Join #WorldOfHashtags game on Mondays at 9AM ET/ Wednesdays at 10AM ET26,417
PhoenixDailyNewPhoenix Daily NewsLocal news, sports, business, politics, entertainment, travel and opinion for Phoenix and Arizona. DM us 24/725,730
AnonymizedAnonymizedWorkout Advice for a better life.25,453
gloed_up1-800-WOKE-AFNo black person is ugly #BRONZE #BlackLivesMatter #BlackToLive25,366
DetroitDailyNewDetroit Daily NewsLocal news, sports, business, politics, entertainment, travel and opinion for Detroit. DM us 24/725,193

Following the links

Some tweets within the dataset contained links to other content. The majority of links were shortened URLs, masking the final destination. To get a picture of which sites these tweets were linking to, we followed each shortened URL tweeted during 2016 to find out the ultimate link destination. The largest number of links led to other Twitter posts, some of which were to other suspended accounts.



Aside from Twitter, other social media services such as YouTube, Instagram, and Facebook also figured highly in the list of websites being linked to. The rest of the list was mostly filled out by links to legitimate media outlets. This suggests that, along with delivering “fake news” the campaign also leveraged real news stories that supported messages it was pushing.
2016 U.S. presidential election

During the months prior to the 2016 U.S. presidential election, there was a marked increase in activity. Between January and November 2016, accounts within the dataset sent 771,954 English language tweets, with a marked uptick in activity as November approached.


Analysis of the topics tweeted by English language accounts during this time period reveals that content was heavily focused on the election and quite evenly split between topics relating to either side of the political spectrum in the U.S.

Political rallies

Perhaps the most overt aspect of the propaganda campaign was the IRA’s organization of a number of political rallies in the U.S. Despite the fact that the accounts comprised of fake personas and organizations, they nevertheless succeeded in mobilizing people to attend events. Besides its online activities, the campaign’s operators also organized rallies supporting positions on both sides of the political spectrum.


One account, @March_for_Trump, advertised more events than any other account in the dataset, promoting events 47 times. Most of the other accounts involved in promoting rallies are anonymized as they have less than 5,000 followers.

Only nine accounts in the dataset retweeted March_for_Trump’s tweets promoting rallies, but the majority of these retweets came from the most followed or retweeted accounts in the dataset, suggesting the campaign’s operators had prioritized promoting these events.
Professional campaign

While this propaganda campaign has often been referred to as the work of trolls, the release of the dataset makes it obvious that it was far more than that. It was planned months in advance and the operators had the resources to create and manage a vast disinformation network.

It was a highly professional campaign. Aside from the sheer volume of tweets generated over a period of years, its orchestrators developed a streamlined operation that automated the publication of new content and leveraged a network of auxiliary accounts to amplify its impact.

The sheer scale and impact of this propaganda campaign is obviously of deep concern to voters in all countries, who may fear a repeat of what happened in the lead-up to the U.S. presidential election in 2016.

A growing awareness of the disinformation campaigns may help blunt their impact in future. If you’re concerned about falling victim to similar campaigns, read our blog post How to Spot a Twitter Bot, which can be found on Symantec’s

No comments:

Post a Comment