How an AI Avatar Will Help Social Work Students Practice Their Counseling Skills

Meet Emma, a 20-year-old woman who is navigating the resurgence of depression symptoms following a breakup with her boyfriend. She answers questions, and her responses vary depending upon what is said to her. Sometimes she’s agreeable and sometimes she’s defensive—exactly as you’d expect from someone who is going through a hard time.  

But in this case, the mercurial behavior isn’t because of her pain—it’s just how she was coded. Emma is an AI program, designed to help social work students get comfortable with the kinds of sessions they’ll lead with future clients.

“We had to feed the AI all sorts of information and scholarship that were based on person-centered care, and then we started creating more of the scenario in terms of some of the back and forth that might happen during interactions between clinician and client,” says NYU Silver Clinical Professor Nicholas Lanzieri, who is the principal investigator for the project.

The program, which is nearing its pilot phase, requires users to put on a VR headset and hold a synchronous conversation with Emma, who is shown seated in a chair across from the user. Users can hold a clicker to select “Hold to Talk” or “End” as they speak or pause for Emma to respond.

Emma is the most recent effort by Lanzieri and Silver Clinical Associate Professor Anne Dempsey, the co-principal investigator, to create immersive and realistic scenarios for students to gain counseling experience. Previous projects have included role-playing with actors and computer simulations in which users can select pre-loaded responses to engage in client exchanges.

The use of VR was partially inspired by an earlier training program by Lanzieri that offered immersive, neighborhood-specific experiences for social work students preparing for practicum assignments across New York City. When it came to course work, VR was the natural next step to expand opportunities for students, particularly as community partners have limited resources to provide students with on-site experiences.

“They’re overloaded, and since the pandemic, there’s been a lot of turnover and difficulty in hiring mental health professionals,” says Dempsey of practitioners in the field. “So, we do have to think creatively to broaden the exposure that students have to different populations, different settings, and different approaches.”

In 2023, Lanzieri and Dempsey decided to take an interdisciplinary approach to develop a product that would incorporate their clinical perspective with Tandon faculty’s AI expertise and Steinhardt faculty’s experience with VR.

“The guidance for this development effort was to be as authentic as possible,” says Al Olsen, an assistant research scientist for the CREATE Lab at NYU Steinhardt. “Emma needed to enact the symptoms of this particular mental health condition.”

Olsen, along with Steinhardt Administration, Leadership, and Technology Professor and Paulette Goddard Chair in Digital Media and Learning Sciences Jan L. Plass and Tandon Professor Julian Togelius, began creating the Emma program, going through various iterations as they sought to make her more realistic in her expressions and movements. It has been challenging to develop a humanlike persona that users won’t find unsettling.

“Responses from people have varied,” Olsen says. “People who have experience with video games seem to be very familiar with this type of aesthetic, while people who’ve not been in VR before did report a little bit of an uncanny valley. So, we're still trying to figure out how to get across that gap to bridge fantasy with reality.”

Lanzieri and Dempsey said that they used their expertise to determine what optimal, neutral, and suboptimal responses from a social worker would sound like during these client interactions. Conversations with Emma are recorded, and users receive an AI-based evaluation on emotional attunement, therapeutic techniques, communication skills, and ethical decision-making. The feedback helps the social workers in training learn and reassess for future encounters.

“In terms of therapeutic techniques, the trainee asked a general why-question to explore the client’s feelings, which is a start, but more open-ended questions or reflective statements could have encouraged the client to elaborate at their own pace,” reads a passage from one evaluation.

While VR has lots of advantages, Lanzieri and Dempsey wanted to make sure this training could still be available to those who lack access to that kind of technology. So, they started work on a second, 2D version of Emma that’s available on a computer screen—without a VR headset. It’s a hyperrealistic figure on a white background, and it’s still in development.

“We were working on making the avatar look as realistic as possible. And there were some limitations for that in the VR space,” Dempsey explains. “In the meantime, AI had sort of just exploded, right? So, the ability to make an avatar in the 2D version look realistic had just increased in its capacity, and that's why we decided to do both.”

Now it’s time to begin testing these products. After that, Lanzieri and Dempsey are hoping to expand Emma’s capability, for example, by loading different scenarios into her programming so that users can practice addressing a range of clients’ problems.

“I think people are very excited about it,” Dempsey says. “When we test it, we’ll be looking at how realistic this experience feels to the user engaging with the avatar, and what makes it feel more or less realistic, and more or less useful, for student learning.”
 

Press Contact

Jade McClain
(646) 469-8496