Youth and Online Polarization and Radicalization

As adolescent time on social media and video game platforms continues to rise in recent years, what can parents and caregivers do to protect them against radicalization in seemingly “safe” social spaces, and what are signs that your child may already be swayed to radical or extremist viewpoints or behaviors? What algorithmic and other features of these platforms are being used in order to create extremist content, and how might federal regulation help prevent this?

Children and Screens invited leading researchers, thought leaders, and other experts to provide tips and insights on youth and online polarization and radicalization.

How Are Youth Being Radicalized Online?

Increased Exposure

“In the days before computers, a young person who exhibited many of the vulnerabilities that we associate with radicalization would have to be very unlucky to meet the kind of recruiter or propagandist who could channel those vulnerabilities in the direction of political violence, extremism or hate,” says Brian Hughes, Associate Director at the Polarization & Extremism Research & Innovation Lab (PERIL). Yet nowadays, “it’s frankly impossible not to encounter that kind of propaganda. Any time we log on to our digital devices, we encounter extremist propaganda, hate and their various cognates,” he says. “And this is especially true of spaces and places online where youth are especially reachable and where vulnerable youth are more easily identified. [Extremists] consciously target those spaces and they consciously target those youth.”

Vulnerabilities that Increase Susceptibility to Radicalization:

• Trauma
• Disruption
• Loss
• Confusion
• Uncertainty
• Anger and Betrayal
• Rebellion and Status-Seeking Trolling
• Desire for connection with radicalized friend/someone they love
• Isolation and Lack of Belonging
• Boredom and Idleness

Source

Peril – Polarization & Extremism Research & Innovation Lab

Other Adolescents

It’s a common misperception that it’s only adults that are targeting and radicalizing youth online. Hughes notes that “it’s adolescents themselves who are out on these platforms acting as the propagators of extremist ideology and this sort of violent viewpoint.” Alex Newhouse, Deputy Director of the Center on Terrorism, Extremism, and Counterterrorism at the Middlebury Institute of International Studies at Monterey, agrees, citing an example of a terrorist organization tracked by his team that was founded by a 13 year-old. “Highly militant white supremacists often are adolescents themselves,” says Newhouse. “The people creating the propaganda are likely between the ages of 13 and 21, so the types of propaganda and the platforms that they choose are already geared towards adolescent audiences, which improves their efficacy in reaching other youth.”

Social Media Platform Design

Katie Paul, Director of the Tech Transparency Project, is a specialist in tracking online criminal behavior on major tech platforms. “Our research on extremism has shown that platforms are dangerous by design, targeting youth, amplifying extremism and profiting in the process,” says Paul. Paul notes that not only do social media platforms recommend harmful content using algorithms, but other algorithms on platforms such as Facebook actually auto-create pages and content for extremists.

“We’ve been following the auto generation issue for several years, and this is something that is unique to Facebook and not the other platforms,” says Paul. ”So one day, suddenly all of the things that you had put in your profile linked to pages that had been auto-generated by the platform. Facebook was doing this because it needed more digital real estate to offer for advertisements, and it was offering ads up on those pages.” While the auto generation process may have been created with innocuous intent, the end result is that if a Facebook user has an extremist interest on their page, Facebook can and will auto-generate a page for that interest, “which means Facebook is making it itself and amplifying these groups that thrive on propaganda,” says Paul, who noted examples of ISIS pages being created by the platform.

Online Video Games

Social Trust With Strangers

Video gaming has established itself as a large and sometimes even dominant presence in many contemporary adolescents’ lives. While there is a large body of research demonstrating many positive benefits of gaming, such as positive social interaction, “we have some initial psychological studies that suggest that video games might have a uniquely strong impact on certain psychological vulnerabilities to radicalization, “ says Newhouse. “The reason we think this is happening is because games can actually invert the ways that social relationships are created. In a multiplayer game, for instance, you’re put into a situation where you have to trust your teammate before you know them. So you end up having an implicit trust relationship created before you have any other types of relationship built on top of it. And that’s inverted from the normal of, you meet someone, you get to know them and then you trust them later.”

More Hours Gaming = Increased Risk

Researchers have found that additional hours spent gaming per week “can have an impact with a bunch of different sort of vulnerability indicators of radicalisation, including expressions of authoritarianism, white nationalism, Machiavellianism, and something called identity fusion, which is basically subsuming your own individual identity into the group,” says Newhouse. Extremists exploit this vulnerability, and evidence indicates that “there are increasing numbers of extremists actually creating social networks in games and on gaming platforms,” says Newhouse.

Gamification Of Violence

Recent acts of extremist violence have shown troubling signs of mimicking the experience and visuals of online shooter games. “Attackers and extremists who idolize those attackers have increasingly used the language, the aesthetics and the mechanics of video games in carrying out and celebrating mass casualty violence in pursuit of extremist ideologies,” says Newhouse. Live streams of recent shootings in the U.S.and internationally “have become increasingly illustrative of the mechanics of first person shooter games – this goes from everything from the way that the GoPro mounted on the shooter’s helmet distorted the image to the way that the shooter actually adorned his weapon with icons and logos. That’s something that comes from ‘Call of Duty.’”

Newhouse notes that this gamification of extremist violence contributes to a “dehumanizing” process of victims and a self-perpetuating impact on how this violence is processed in extremist communities, citing examples of “leaderboards” and publication of “kill/death/assist” ratios in reference to real-life violent events in extremist spaces online that mimic popular video games.

Person + Environment = Behavior

Understanding behavior to be shaped by a mixture of factors unique to the individual as well as the environment can help explain the complex impact that online spaces can have on radicalization. “Games can have a sort of pretty significant impact on this equation on impacting one’s way that they look at the world, one way that they process stimuli coming in and ultimately the way they interact both in game and out of game“, says Newhouse. “So you can think about video games as potentially sort of one one element in the full equation of a person’s behavior. It is not as simple as playing violent video games increases the potential of violence, but there is increasing indications that video games can sort of shift people towards certain directions that they might have already been predisposed to.”

Radicalization’s Real Impacts On Families And Individuals

“Radicalization, polarization, and extremism in youth can tear communities and families apart, sow mistrust and damage our democratic process,” says Hughes. For radicalized youth individually, “Radicalization is not a pathway with good outcomes. This is a process of development that leads to negative life outcomes. Organized extremist groups frequently are involved in criminal activity. Abuse and exploitation are rampant within extremist spaces. And the social costs of embracing hate drives people into further isolation from family and friends.”

Tips – What Can Parents Do to Identify and Protect Children From Radicalization?

Watch for Signs Of Possible Radicalization In Youth

Newhouse urges parents and caregivers to keep an eye open for these behavioral signs that their child has begun to be radicalized online:

• Interest in one single game or gaming community to the exclusion of any other interests
• Use of language or memes that appear to be particularly unusual, fringe, and edgy
• Detachment from normal relationships (family and friends, both online and offline)

Don’t Be Intimidated ByThe Pace Of Tech Change – Engage!

Many parents feel confused and alienated by the quick pace of technological change and feel unable to understand the nuances of the online spaces their children are occupying. As a result, “they feel that they can’t support safer, more inclusive experiences online for the young people that they most care about,” says Jennie King, Head of Civic Action and Education at the Institute for Strategic Dialogue.

King urges parents to look past feeling a need for technical expertise and to focus on being a constructive sounding board for young people trying to safely navigate online communities. “You’re never going to be able to keep up with the evolving landscape online. And so if you’re constantly holding yourself to that standard, you’re going to feel eternally on the back foot,” says King. “I would really encourage people to focus about on the core principles and competencies that help young people or any people to engage with information and to reflect on their own experience as a consumer of content and the way that they engage with that content in the safest and most constructive way, rather than the specifics of whichever trend is generating crisis or panic amongst the general public at a given time.”

Focus On Awareness And Critical Thinking

“I think it’s important to remember the necessity of technology in the current era and balance that with how we use that technology and what our knowledge about its function is,” says Paul.

“It’s important to remember how dominant platforms, especially Meta platform, are in the developing world, but by developing digital literacy skills, and an understanding of algorithms and how content is fed to you and pushed to you, you can do a lot to dissuade people from buying into something just because it comes across their screen.”

Teaching youth to start thinking critically about content they find with a sense of openness and curiosity is a strategy any parent can implement, says King. Encouraging self-investigation into their own online environment can encourage youth to take a leadership role in curating their own online experience. “One of the main things that we encourage both parents and educators and practitioners to do is to have that conversation. ‘What kind of web do you want to be a part of as a young person?’ ‘What is your manifesto for the Internet?’ and ‘How do you as an individual contribute to building that kind of environment, both in the way that you police your peers’ behavior and in the way that you police your own behavior?’” says King.

Youth can be encouraged to use the same open, question-based approach to evaluate the content they see online. King suggests asking questions like “‘What is the purpose of this information online? Is it there to inspire? Is it there to teach? Is it there to create belonging and identity? Is it there to polarize? Is it there to generate advertising revenue?” Answering these questions can inspire critical thinking about the relationship between the individual and what they’re consuming and expose how extremist ideologies are actually based in “fundamental misunderstandings, often compounded by algorithms and the online filter bubbles or echo chambers they may produce.”

Acknowledge Your Own Vulnerabilities And Hold The Judgment

The road to radicalization is not always marked with clear warning signs, and keeping a sense of empathy around the ease in which anyone can find themselves led into extremist spaces can help with productive intergenerational discussion. “It’s important to have that awareness that with things like conspiracy theories, which then lead into extremist paths, there are good reasons why average people are susceptible to those ideas,” says King, such as vulnerable life moments of social or economic crisis when people may be driven by fear or a need to make patterns out of chaos. “It’s not some shadowy group of ‘others’ who are going to fall prey to that. We all have the capacity to go down that path,” says King.

By acknowledging even their own vulnerability to being led into misinformation or echo chambers, parents can create a space where youth can see their own situations more clearly. “It’s important to get away from the concept that there are kind of neatly delineated heroes and villains, and the idea that there are ‘stupid’ people who are susceptible to that kind of content,” says King “As adults, I think coming to those conversations with humility and saying, ‘Oh, here’s a time that I fell prey to something that I actually thought was very legitimate, I didn’t check the sources on, and I accidentally shared it and provided oxygen to that idea. And I realize now that that was really unhelpful.’ Walk through those scenarios with them and be humble about how you might act in those particular encounters.”

Use Positive Role Models To Help Instill Values

Looking for positive role models or examples in broader culture that may make young people feel more incentive to proactively engage with issues like misinformation and polarizing online behavior, says King. “Understand who are the influences that your students or your children idolize online and really unpack with them. ‘Why do you like them?’ ‘How do they communicate?’ ‘How do they create spaces that are inclusive for people?’ ‘How are they challenging ideas that you find interesting?’” Identifying and understanding the people they see as trusted gatekeepers or intermediaries, and the behavior they would like to model themselves, “goes a long way to instilling them with values, which will then produce resilience against some of the biggest online harms,” says King.

Encourage Offline Activities

One way to decrease the risk of online radicalization in youth is to ensure more time spent offline in rewarding and social activities. “We know that COVID 19 was a mass radicalization event specifically because it isolated people, made them alienated from each other,” says Newhouse. “I think we often discount the shared social impacts of kids having things to do. Encourage your local community, vote for state policies that fund rec centers and after-school activities that just generally create those sort of ‘third spaces’ that give kids opportunities to be kids and to interact with one another and to get a bigger diversity of experience.”

Finding more opportunities for in-person community with people different from ourselves is key, says King. “We have to reengage with the people that are around us and find ways to increase basic contact theory, getting people from different profiles and walks of life to cohabit or even to be involved in programs together where diversity is kind of the byproduct, but it’s not the ultimate goal” will help thwart the pull of more polarizing online echo chambers, says King.

Advocate For Change In Regulation Of Social Media And Online Platforms

Society is in a “regulatory ‘Wild West’” with social platforms and online content, says Paul, despite the fact that “we’ve seen dozens of letters from members of Congress on both sides of the aisle regarding the need for platforms to stop targeting children and rein in tech.” Parents of all political persuasions can use this moment of bipartisan agreement to call or write elected officials like representatives, senators, and state attorney generals. “It’s a matter of the public needing to put pressure on them to make sure something actually happens,” says Paul.

A barrier to change in regulations for social media is that there isn’t a federal agency, at the national level, that oversees the social media industry, says Paul Barrett, Deputy Director and Senior Research Scholar at NYU Stern Center for Business and Human Rights. Even with federal oversight, regulating content on social media would be difficult, “because the business in question is one that trafficks in expression. And the First Amendment simply prevents Congress or regulatory agencies from getting involved in the substance of setting policies for these businesses, let alone in actually participating in or overruling decisions about how to enforce those policies,” says Barrett.

Requiring social media platforms to provide disclosure and transparency in understanding how and why algorithms prioritize certain content, and requiring some level of content moderation would be the best first steps in addressing social media’s influence in radicalization, says Barrett, who suggests enhancing the consumer protection authority the FTC has through its Bureau of Consumer Protection. ”The FTC in its already existing statutory authority to regulate unfair and deceptive commercial practice could impose certain transparency requirements and further impose requirements that the platforms demonstrate that they have in place procedurally adequate content moderation, which might address some of the concerns that we have about how platforms currently self-regulate,” says Barrett. In the meantime, he acknowledges that for now, “it is really these platforms that are in a position to limit the kind of damage that we’re talking about, both because of constitutional constraints, but also of technological constraints.”

Resources/Organizations Recommended By Experts Featured*:

• https://www.techtransparencyproject.org/
• https://www.middlebury.edu/institute/academics/centers-initiatives/ctec/ctec-publications 
• Building Resilience and Confronting Risk: A Parents and Caregivers Guide to Online Radicalization 
• Building Networks & Addressing Harm: A Community Guide to Online Youth Radicalization: Resources for Trusted Adults, Mentors & Community Leaders
• Be Internet Citizens curriculum
• Extreme Dialogue programme
• Young Cities: Practical Guide to Youth Capacity Building for Countering Hate and Extremism
• BBC Fact or Fake platform
• BBC Bitesize on Algorithms and Filter Bubbles
• SELMA: Hacking Hate