Fighting the terrorist threat online: New research can identify extremists online, even before they post dangerous content

Key takeaway: By identifying and removing extremist users from social media platforms, such as Twitter, and preventing their return, we can significantly impact their ability to spread dangerous propaganda and incite violence.   

CATONSVILLE, MD, September 12, 2018 – New research has found a way to identify extremists, such as those associated with the terrorist group ISIS, by monitoring their social media accounts, and can identify them even before they post threatening content.

The research, “Finding Extremists in Online Social Networks,” which was recently published in the INFORMS journal Operations Research, was conducted by Tauhid Zaman of the Massachusetts Institute of Technology; Lieutenant Colonel Christopher E. Marks, U.S. Army; and Jytte Klausen of Brandeis University 

The number and size of online extremist groups using social networks to harass users, recruit new members, and incite violence is rapidly increasing. While social media platforms are working to combat this (in 2016, Twitter reported it had shut down 360,000 ISIS accounts) they traditionally rely heavily on users’ reports to identify these accounts. 

In addition, once an account has been suspended, there is little that can be done to prevent a user from opening up a new account, or multiple accounts. 

“Social media has become a powerful platform for extremist groups, ranging from ISIS to white nationalist “alt-right” groups,” said Zaman. “These groups use social networks to spread hateful propaganda and incite violence and terror attacks, making them a threat to the general public.” 

Identifying extremists before they pose a threat online 

The researchers collected Twitter data from approximately 5,000 “seed” users who were either known ISIS members or who were connected to many known ISIS members as friends or followers. They obtained their names through news stories, blogs, and reports released by law enforcement agencies and think tanks. 

In addition to reviewing the content of 4.8 million tweets from these users’ timelines (including text, links, hash tags, and mentions), they also tracked account suspensions, as well as any suspensions of their friends and followers accounts. 

For the purpose of this study, the researchers focused on the account networks forged by known ISIS and Al Qaeda sympathizers and known foreign fighters and content that had been flagged by Twitter as terrorist in nature. 

Using statistical modeling of extremist behavior with optimized search policies and actual ISIS user data, the researchers developed a method to predict new extremist users, identify if more than one account belongs to the same user, as well as predict network connections of suspended extremist users who start a new account.   

In addition, by tracking and comparing data on screen names, user name, profile images and banner images, the researchers were also able to identify 70 percent of additional Twitter profiles held by extremist users, with only a 2 percent incidence of misclassifying profiles. 

 “We created a new set of operational capabilities to deal with the threat posed by online extremists in social networks,” said Marks. “We are able to predict who is an extremist before they post any content, and then able to predict where they will re-enter the network after they are suspended. In short, we can automatically figure out who is an extremist and keep them of the social network.”

While the study was conducted using data from accounts belonging to ISIS extremists on Twitter, their methodology can be applied to any extremist group and any social network.

“Users that engage in some form of online extremism or harassment will have very similar behavioral characteristics in social networks,” said Klausen. “They will connect to a specific set of users which form their extremist group. They will create new accounts which will resemble their old accounts after being suspended, and when the return to the social network following a suspension, there is a high probability they will reconnect with certain former friends.” 

The full study is available at https://pubsonline.informs.org/stoken/default+domain/OPRE-09-2018/full/10.1287/opre.2018.1719.

About INFORMS and Operations Research

INFORMS is the leading international association for operations research and analytics professionals. Operations Research, one of 16 journals published by INFORMS, is a leading academic journal that is focused on research using quantitative approaches to study all aspects of operations research. More information is available at http://www.informs.org or @informs.

####