Artificial intelligence (AI) is transforming how people seek advice online. From AI algorithms to generative AI chatbots, ensuring technology serves people ethically is a growing concern. The University of Minnesota is positioning itself as a leading public university for AI to serve all Minnesotans on issues like these, as outlined in the Elevate Extraordinary 2030 strategic roadmap.

Stevie Chancellor, an assistant professor in the College of Science and Engineering on the Twin Cities campus, is developing “human-centered AI” to safeguard the mental health of online users.

A troubling trend

As social networks (e.g., TikTok, Instagram) and the AI technology behind them have grown, people can find endless communities for their worries — especially when it comes to mental health and potentially stigmatized symptoms or conditions. The rise of generative AI now allows individuals to access that information even more quickly, with the ability to confide in and seek comfort from chatbots (e.g., ChatGPT, Gemini).

While there can be benefits, Chancellor has identified two potential harms of AI. The first is connecting people to accounts or programs that share dangerous advice. The second is the risk that users will no longer seek trained professionals for help, and instead trust incorrect and dangerous information.  

Recent research has shown that AI is not safe for providing therapeutic advice, as chatbots can lead to delusional spirals like encouraging self-harm, and algorithms like “explore” or “for you” pages promote “doomscroll” behaviors that can exacerbate psychological harms.

By designing systems that are sensitive to diverse backgrounds and perspectives, Chancellor aims to minimize those risks.

Working toward solutions

Chancellor and her colleagues audit the safety of AI chatbots and algorithms used as a support provider for mental health. 

“We work heavily with users who have mental health conditions. They’ve found themselves in unfortunate situations with social media or with chatbots, but we also hear about the benefits that social connection can bring when they are isolated,” says Chancellor. “We talk to real people about their problems, and that richness cannot be replicated in any simulations or experiments.”

So far, Chancellor's research has worked directly with more than 150 people. The process includes early interviews with volunteers and outreach to community nonprofits about content recommendation systems and social media to eliminate guesswork in their design. They use that input to build better AI systems that address the issues raised and bring the built technology back to participants for feedback and further testing.

For example, Chancellor had a study participant present an idea to improve content recommendation systems through a choose-your-own-adventure model. Instead of only getting what the algorithm suggested, users could make choices after viewing a set of content. Originally not something the team envisioned, this metaphor has become a central design plan going forward. 

“I am humbled and always grateful for learning about these experiences, not only because of the way that they shaped my research. Their perspectives fundamentally change the way that I see the world and the technical solutions we can invent,” says Chancellor. 

A safer space for all

Chancellor aims to encourage tech companies to adopt safer, sustainable practices when building AI systems. She is also interested in influencing legislation on digital safety and protecting people from harmful AI chatbot content, while supporting those doing ethical work to build AI tools that support people with mental health conditions.

“Better insights about the impacts of technology on well-being will help our state and nation develop more thoughtful policies on AI and social media. It will also improve nuanced and careful approaches to bringing technology into our lives,” says Chancellor.

To Chancellor, this work and others like it should always be grounded in people over progress. 

“I think the best work is interdisciplinary and makes the biggest impact when you can actually solve problems that many people care about. I'm really excited and eager to work with folks who are interested in health and well-being and AI and online spaces.”

Learn more about Chancellor’s work from her talk on TEDx.