Much of today’s public conversation takes place online, where the volume of posts, comments and messages grows every year. For analysts, schools and public authorities, this creates a practical challenge: harmful language, personal attacks and signals of escalating behaviour appear at a pace that makes manual review difficult. 

At the Department of Computer and Systems Sciences (DSV) in Kista, researcher Lisa Kaati studies these environments. With a background in computer science and more than a decade at the Swedish Defence Research Agency (FOI), she develops methods that help analysts make sense of large text datasets and identify signals that are difficult to detect manually. 

 

The research: toxic communication and early warning signals

Kaati’s work at FOI helped shape her current research direction. There, she and her colleagues developed tools to support analysts working with large amounts of online text. Working close to public authorities made one thing clear: while research prototypes can test new ideas, operational users need systems that are reliable, robust and well-tested. That experience continues to influence how she approaches her methods today.  

At DSV, her research focuses on understanding online behaviour through data, divided into two main areas. One track focuses on toxic or harmful communication – language that targets a person rather than their arguments. In an interview with Universitetsläraren, Kaati explains: 

“In political discussions, for example, there can be quite a harsh climate, but what we analyse is when people make personal attacks or talk about irrelevant things, such as derogatory comments about people’s intellectual capacity or appearance, when discussions should really be about the person’s work.” 

The second track focuses on digital warning signs, signals that may indicate a heightened risk of violence. She continues: 

“There, we don’t analyse toxic language but try to identify actual threats or someone describing a planned violent act. We also analyse psychological factors in individuals. Is this person expressing a perceived injustice?” She adds that when several warning signals appear together, the risk level increases and the material requires closer review by human experts. 

This work is strongly interdisciplinary. Kaati collaborates with psychologists, criminologists and social scientists, while she develops the computational methods used to analyse large text collections. By building her own analytical models rather than relying solely on existing tools, she can tailor methods to the specific signals that analysts need to detect. 

Ms. Lisa Kaati

Practical applications and collaborations

Models based on Kaati’s research are now used primarily in the United States by schools and police departments. They help distinguish between harsh language and situations that may require follow-up — not by replacing human judgement, but by helping analysts prioritise which material needs closer review. 

“It is not possible to sit and read everything that is written on the web. You need to sort through it. Computers can never replace people and their analytical skills, but we can help filter out what people need to read and understand,” Kaati says to Universitetsläraren. 

Her methods also shape her teaching at DSV. The courses she has developed draw directly from her research and focus on online intelligence analysis and techniques for handling large volumes of digital text. As more work shifts into digital environments, these skills become increasingly relevant for future analysts.   

International collaboration is another part of the work. During an exchange at Boston University, Kaati began a partnership with an American researcher specialising in threat assessment. Together, they analysed threatening online texts and worked with U.S. analysts to understand how risk levels are evaluated in practice. The collaboration has resulted in joint publications and involvement from the FBI’s Behavioral Analysis Unit, whose analysts contribute expert assessments to the research.

 

Digital safety in a changing landscape

Digital environments evolve quickly, and the volume of online material continues to rise. This makes it harder for organisations to recognise when everyday conversations begin to shift toward more concerning behaviour, especially when relevant signals may be scattered across large datasets. 

In this context, analytical methods like those developed by Lisa Kaati give analysts a clearer starting point for their assessments. They highlight combinations of signals that might otherwise be overlooked and help ensure that potential risks are not lost in the speed and scale of online communication. 

Kaati’s work focuses on parts of the internet that many prefer not to look at, but which can shape real risks and real consequences. By refining analytical tools and deepening insight into harmful online behaviour, her research contributes to a broader question: how societies can manage the risks that come with a more digital public life. 

Further reading 

“Lisa Kaati studerar allt som är dåligt med internet” (Universitetsläraren): A profile article about Kaati’s background, research approach and personal experiences with online harassment.

“Hon vill bara att tiden ska gå” (the Swedish Gender Equality Agency): A report that maps digital profiles and websites that market sexual acts for payment, co-authored by Kaati.

P4 Extra interview: Kaati discusses online discourse, vulnerable groups and her collaboration with analysts connected to the FBI.

Mind Intelligence Lab: A company founded by Kaati and colleagues at Uppsala University that develops research-based tools for identifying and assessing digital threats.