Skip to main content

Search NYU Steinhardt

Accessible Music and Creative Code


While pursuing his PhD in Music Technology at NYU Steinhardt, Willie Payne (PhD ’23) partnered with The Filomen M. D’Agostino Greenberg Music School (FMDG), which helps people pursue their study of music while addressing the challenges posed by vision loss. 

There, Payne served as the Accessible Music Technology Fellow, where he worked to research and implement accessible musical instruments and creative coding environments. 

As an undergraduate, Payne double-majored in music composition and computer science at CU Boulder, where he was named the Outstanding Graduate of Engineering. While in school, Payne was pulled onto a project with computer scientist Clayton Lewis, composer Michael Theodore, and physics educator Noah Finkelstein to make physics simulations more accessible to the visually impaired using sound.

“I had written code to produce music and animation in live performance, but I hadn't considered that those skills could be applied elsewhere,” says Payne. “I became interested in how we can leverage different modalities to make learning more accessible for diverse students.”

Willie Payne, in a dark blue shirt and glasses.

The music technology program at NYU Steinhardt represented an ideal interconnection between Payne’s ranging interests in music composition, computer science, and accessibility.

“FMDG emerged as an amazing partner as I was working on my dissertation about the nexus of creativity, accessibility, and learning,” says Payne, who is now an assistant professor of user experience and design at the University of North Carolina Chapel Hill’s School of Information and Library Science.

Payne’s first projects included transcribing printed scores to help create the braille music read by FMDG students and interviewing blind and low-vision musicians across the country. He and his advisor, Amy Hurst, associate professor of occupational therapy and director of The Ability Project, co-authored a paper on the challenges and design opportunities for providing access to music notation in braille and large print formats.

“We discovered a bunch of innovations that vastly simplify the process of creating braille music compared to even 20 years ago,” says Payne. “However, music notation is still thought to be a visual medium, and significant labor is required to take a single print score and accurately convert it to braille – not to mention a library of print scores.”

This scholarly endeavor led to Payne developing a tool called SoundCells with a team of FMDG technologists and NYU undergraduate research assistants. SoundCells allows people to write music that is automatically created accessible, with both braille and print notation generated simultaneously. The work culminated in an online six-week class at FMDG for blind and low-vision musicians who learned how to compose short pieces using the software. The class then performed their works at a Zoom concertPayne’s first article on SoundCells won the Best Paper Award at the Web4All Conference.

The current iteration of Payne’s collaboration with the school is FMDG’s FiLOrk (Fil’ Laptop Orchestra), an electronic music ensemble led by high school students who identify as blind or low vision writing code to joint-compose and perform original music. FiLOrk had its first concert in December of 2022 and several more throughout 2023

“Laptop orchestras have been around for a while, but mainly at universities,” says Payne. “Our challenge here is to make the curriculum friendlier to younger learners and the classroom more inclusive. Since they are responsible for writing the score and building the instruments with code, everyone wears a lot of hats. Each decides what role they want to take in a performance in pursuit of a collective musical vision. It’s a lot of fun and at times a bit chaotic!”

Payne still collaborates with FMDG as a vital research partner; FiLOrk is currently supported by a Project Grant from the Consumer Technology Association (CTA) Foundation. In addition, the NYU undergraduates who worked with him and FMDG during his fellowship continue their research from New York while Payne contributes remotely from North Carolina.

Related Articles

Nancy Rico-Mineros Takes Music Technology by Storm

Nancy Rico-Mineros, third year Music Technology student, discusses her role with NYU’s radio station, serving on the panel of a non-profit founded by Alicia Keys, and more.

Graduate Student Peter Traver Shares What Led Him to Music Technology

After years of studying composition in the hopes of becoming an opera singer, Peter Traver changed course and found the right fit in Music Technology.

Embodying Healing through Art Therapy

NYU Steinhardt’s Art Therapy program blends clinical efficacy with holistic wellness for communities across New York City — and the world.

Related Programs

Music Technology

Develop your expertise in an academic program that emphasizes creative experimentation and the integration of musical and technical skills.

Read More

Related Departments

Music and Performing Arts Professions

Discover programs in education, composition, technology, business, theatre, administration, therapy, performance, and dance.

Read More