A team of researchers from the University of Rochester’s Eastman School of Music, Warner School of Education & Human Development, Hajim School of Engineering & Applied Sciences, and School of Arts & Sciences, along with Northwestern University’s McCormick School of Engineering, are collaborating to leverage and better utilize Artificial Intelligence (AI) to empower musicians to produce and disseminate their art more effectively and independently.
This project, which received a $1.8M grant award from the National Science Foundation Future of Work at the Human-Technology Frontier program, builds on the preliminary work of an interdisciplinary team of sixteen faculty, staff, and students across the fields of music, audio engineering, ethics, and education. Five principal investigators lead the team – the University of Rochester’s Raffaella Borasi (Warner School of Education), Rachel Roberts (Eastman School of Music, Institute for Music Leadership), Zhiyao Duan (Hajim School of Engineering & Applied Sciences), and Jonathan Herington (School of Arts & Sciences), and Northwestern University’s Bryan Pardo (McCormick School of Engineering). This grant marks the first sizable NSF-funded research project with deep collaboration between Eastman, Hajim, ASE, and Warner.
Raffaella Borasi, the lead principal investigator, says of the collaboration, “The most exciting and rewarding aspect of this project is the opportunity to work together with people contributing such different expertise to address the many complementary components of a complex problem.”
There is a growing demand for digital music content, including music for film, television, games, and advertising, as well as the traditional recorded music market. AI-powered tools have the potential to enable musicians to create digital music products on their own and at a very low cost in ways that are not yet possible. By eliminating their dependence on centralized music production, musicians will achieve greater creative freedom and reach larger audiences while increasing their chances of making a living with their art.
However, musicians must overcome many obstacles before realizing this potential. Despite significant advances in audio AI research, today, only a few musicians (even among electronica and hip-hop artists who have made the most use of technology in their work) are using AI-powered tools. Obstacles that need to be addressed are cost, limitations of currently available tools, specialized technical skills needed to use specific tools, as well as musician concerns about how use of AI may impact their creativity, intellectual property, and professional identity.
Two essential things need to happen to unleash AI’s full potential in music production. First, more robust and user-friendly AI-powered tools to support various aspects of music production need to be created and disseminated. At the same time, researchers continue to create novel, deep-learning AI models, so this is only a step towards building user-friendly tools that most musicians can use. Second, we need to empower more musicians to use these new tools in creative and transformative ways in their everyday endeavors.
The project, titled “Toward an Ecosystem of Artificial-intelligence-powered Music Production” (TEAMuP), proposes to achieve these goals by working in the following complementary areas:
- Creating an open-access framework enabling musicians and AI researchers to collaborate in developing new music production solutions, which will run on Audacity (an open-source and free DAW that more than two million people have used to date).
- Understanding both advances and challenges that may arise from using AI for music production.
- Providing new learning opportunities to empower current and future musicians to utilize technology better.
Through this project, the research team aims to increase the democratization of music production by helping musicians become self-sufficient in their music creation. The open-access framework they will develop will also accelerate research in audio engineering by providing a vehicle to effectively deploy and refine new AI models for music production. Insights gained from this project will be relevant for many other occupations at the human-technology frontier besides musicians. Those insights include how to create more ethical and user-friendly AI-powered products, develop the mindsets and skills needed by domain specialists to leverage technology, and understand the implications of the pandemic.
Lauren Sageer, Assistant Director of PR and Digital Content, Eastman School of Music at University of Rochester, (585) 451-8492, firstname.lastname@example.org
Amanda Morris, Senior Science and Engineering Editor, Northwestern University, (847) 467-6790, email@example.com
About the University of Rochester:
The University of Rochester is one of the nation’s leading private research universities, one of only 62-member institutions in the Association of American Universities. Located in Rochester, NY, the University gives undergraduates exceptional opportunities for interdisciplinary study and close collaboration with faculty through its unique cluster-based curriculum. Its College, School of Arts and Sciences, and Hajim School of Engineering and Applied Sciences are complemented by the Eastman School of Music, Simon School of Business, Warner School of Education and Human Development, Laboratory for Laser Energetics, School of Medicine and Dentistry, School of Nursing, Eastman Institute for Oral Health, and the Memorial Art Gallery.
About the Eastman Institute for Music Leadership:
The Eastman Institute for Music Leadership (IML) was the first center of its kind in the country, created in 2001, with a particular focus on the changing state of classical music, with generous support from the Andrew W. Mellon Foundation. Always at the forefront of educating musicians of the future, the Eastman School of Music is dedicated to the comprehensive education of world-class musicians, scholars, and leaders. The IML serves as a center – and benchmark – for music leadership programs and activities and respond to and help shape America’s changing musical and cultural environment.
The establishment of the IML, along with the generous support of foundations and philanthropists, has allowed the Eastman School to create an atmosphere where ideas can flourish, and students can develop the skills and networks they need to adapt to the changing arts world. The IML leads, reacts, adapts, and morphs its curriculum to meet the needs of the challenging musical world that our students will inhabit as professionals.
About Northwestern University:
Founded in 1851, Northwestern University is one of the country’s leading private research and teaching universities with an enrollment of approximately 8,000 full-time undergraduate students, approximately 8,000 full-time graduate and professional students, and approximately 2,000 part-time students on campuses in Evanston and Chicago, Illinois, and Doha, Qatar. Northwestern combines innovative teaching and pioneering research in a highly collaborative environment that transcends traditional academic boundaries. Northwestern provides students and faculty exceptional opportunities for intellectual, personal, and professional growth in a setting enhanced by the richness of Chicago.