NYU Steinhardt Professor and sound computing scientist Juan Pablo Bello uses AI to listen in on and track the migratory patterns of birds, the cacophony of New York City, and more.
Sjo/Getty Images
Every year, billions of birds migrate across the United States, diverse societies of long-distance commuters moving in vast waves high above our heads. They can log thousands of miles as they traverse oceans and continents—with no GPS or traffic lights to guide them, and often under cover of darkness—communicating with one another in short, subtle chirps whose patterns and purpose have long intrigued and eluded humans.
Though bird migrations have been observed since the time of the ancient Greeks, the methods for identifying species have been imperfect. Most commonly, birders have made their own observations—a technique limited by which species are actually visible in their vicinity and the fact that most migratory flights happen at night, when birds are hard or impossible to see. Or they have relied on Doppler radar (after filtering out weather), which can detect the large migrating masses overhead but cannot identify individual species, sort of like trying to pick out individual conversations happening in the top rows of a packed MetLife Stadium.
But a team of researchers from NYU, the Cornell Lab of Ornithology, and France’s Centrale Nantes has come up with a breakthrough solution: a neural network called BirdVoxDetect that is able to fill in many of the gaps, monitoring migratory flights by the tantalizingly brief nocturnal calls—sometimes on the edge of frequencies human ears can pick up—that birds make when they fly.
“When you put sensors in more remote places, you get a better sense of the species composition of flights,” says Juan Pablo Bello, an NYU Steinhardt professor of music technology (as well as computer science and engineering at NYU Tandon) who put together the team that began working on the BirdVoxDetect system in 2016. That’s important, he says, “because migratory flights are not all just one species—you have different compositions of species flying at different times of the year, with each species having different peaks of migratory movement.”
As part of the eight-year project, the BirdVox research took subtle information from around 15 sensors set up in a variety of rural areas, particularly around Ithaca, New York, home to Cornell. “The sensors are amazing because you get 24/7 monitoring, so continuous, in-depth observations of bird behavior that you can’t get from human observation because things change over the day and seasons, and, with people, you only get sparse observations,” Bello says. “When people think about bird sounds, they mostly think about their songs, which have specific biological purposes—for example, for mating or food. But those are very different from what we were looking at, which was fly calls, ridiculously short signals that birds send to manage their migratory flights.” Those calls vary depending on the bird; the familiar calls of large birds like geese are closer to the frequencies of the human voice and are easier for us to identify. But most migratory birds are smaller and make very fast, high-frequency chitters.
Working with the birding community and biologists, the researchers were able to put together massive datasets of real-world fly calls, which were then annotated by humans who could identify individual species. The BirdVox team used that information to train machine learning software how to ignore competing noises (like car alarms or, most vexingly, chirping spring peeper frogs) to pick out the flight calls and, ultimately, identify the birds by species.
Illustration by Brian Stauffer
“It’s important to understand the biology of the migratory patterns for different species to drive conservation activities, including how environmental conditions affect flights, and to monitor the health of bird populations and how they change,” says Bello. Specifically, this type of research aims at answering questions like what triggers migratory flights, how routes by the same species change year to year, why there is so much stability between generations of birds, and how the health of bird populations is affected by human activity. That information can affect decisions about sites like airports, wind farms, and even urban environments like New York City, where shutting off lighting in certain buildings can aid migratory flights.
Once the initial research was completed and the team published a paper on its findings, the BirdVoxDetect AI software was released for free so other researchers could use it to identify more birds and even other animals. “The data we generated is still being used by researchers all over the world to develop better systems or even adapt systems to their specific geographic locations and biodiversity needs,” says Bello. “It wouldn’t be a stretch to say that we were among the first teams to actually look at these approaches in machine learning for sound.”
Bello, who has been at NYU for two decades, has appointments in the Tandon School of Engineering and the Steinhardt School of Culture, Education, and Human Development, and he is director of the multidisciplinary Music and Audio Research Laboratory (MARL). A native of Venezuela, he has long been interested in sound: He began playing music as a child and joined bands as a teen and early adult. Bello still plays classical guitar, piano, and traditional Venezuelan instruments, and he says music is a passion he shares with his two daughters. “We have the same tastes—we’ll see for how long,” he says, chuckling. “But I’m having a blast. It’s a great way to reconnect with what I found exciting about music in the first place.”
Early on, Bello was also interested in music production and engineering, so as an undergraduate, he studied electronic engineering, going on to earn his doctorate at the Queen Mary University of London in digital signal processing, as applied to everyday sounds and music. In all of his work, he says, “I try to get machines to understand the world through sound—to give computers the capability to understand their environment through sound.”
That goal was part of another NYU project Bello was deeply involved in: SONYC, which stands for Sounds of New York City. Unlike BirdVox, SONYC focused on a man-made soundtrack: the cacophonous urban environment of the City That Never Sleeps. Starting in 2012, SONYC placed about 100 microphones on second stories of buildings throughout Manhattan, Brooklyn, and Queens to randomly record 10-second clips of everyday street noise, from blaring police sirens, barking dogs, and jackhammers to air conditioners, car horns, and snowplows.
“Noise is such a central part of everyone’s life in cities, and it’s something we all have a kind of ‘learned despair’ about,” says Bello. “We think there’s not much we can do about it,” compared with, say, water pollution or garbage in the street. But by recording and analyzing the sounds of the city, SONYC showed that noise can actually be quantified so that affected communities can use the data to push for changes that benefit them. For example, in Red Hook, Brooklyn, the project helped the local community gather evidence to reroute noisy delivery trucks so they affected fewer people. And in Manhattan’s Chinatown, the researchers worked with eldercare groups who lived near a massive jail construction project to help them advocate for better regulation to mitigate the noise.
Bello’s work with sound continues through his ongoing role as the director of MARL, an interdisciplinary community at Steinhardt of about 30 faculty members, PhD students, and postdocs that started in 2010. “The three big specialist areas in MARL are computing for sound and music, cognition and neuroscience for music, and immersive sound,” says Bello.
One of his current projects is a collaboration between Steinhardt and Sony that focuses on developing machine learning tools for spatial audio analysis and generation. “Imagine that you have a video or part of a movie and you need to create sound effects that are well synchronized with the images,” he says. “We are creating artificial intelligence systems that can generate sounds that match,” whether for movies, immersive video games, or virtual reality systems. Another collaborative MARL project, this one with Bosch, focuses on sound detection technology, using audio signals to monitor machine failures in manufacturing environments and also explore industrial and medical applications.
The goal of a proposed third group project at MARL is to improve AI music generation by helping machines understand aesthetic quality—specifically, why some people perceive certain pieces of music, from Mozart to Monk, to be beautiful (or not) and how those judgments differ from person to person. The ultimate goal would be to build AI systems that generate music tuned to individual human preferences by linking cognition, music theory, and computation. MARL “is really a wonderful community,” Bello says. “It is my happy place, my lifelong project.”
—Alison Gwinn, originally published under the title “Bioacoustics: Super Sonics” in Scope, NYU's Research Magazine, Issue 7, 2026
Related Articles
Music Tech’s Moon Ha Organizes Mise-En Festival of Contemporary Music
The festival features six concerts of new music as well as panel discussions and workshops at Mise-En’s new Harlem venue—and at NYU and Brooklyn College.
Music Technology's Magdalena Fuentes Receives $800K Award from National Science Foundation
Along with NJIT researchers, Fuentes is developing AI audio captioning technology for deaf and hard-of-hearing viewers in a three-year project.
NYU Steinhardt’s Nordoff-Robbins Center and MARL Receive Renée Fleming Neuroarts Investigator Award
The team’s research examines how music therapy may be able to improve social communication skills in autistic children.
Related Programs
Related Department
Music and Performing Arts Professions
35 W. 4th Street, 2nd Floor
New York, NY 10012
212-998-5424
mpap@nyu.edu
