Dancers make beautiful music through movement in this integrative approach to performance.
When you’re already making music together, dance is a natural next step. At least that’s what two Adelphi faculty members decided when they created an interdisciplinary collaboration to produce the dance performance “It Happens This Way Once” in November 2024 for Fall Dance Adelphi. Orion Duckstein, associate professor and chair of the Department of Dance, and Lee Stemkoski, professor and associate chair of the Department of Mathematics and Computer Science, created a way for dancers to produce music through their movements.
The professors collaborated on the design of a software program that analyzes live video of dancers’ movements and generates music. Based on their discussions, Stemkoski created a series of prototypes that Duckstein tested in the studio. Using an iterative design process, Stemkoski incorporated his colleague’s feedback and suggestions to continuously update and improve the software.
“It was really fun watching the dancers interact with material in a new way,” Duckstein said. “The dance is called, ‘It Happens This Way Once,’ and it’s really just like life. It’s the same and different every day, and the choices we make affect things in small and beautiful ways.”
Generative Music
The collaboration took off during a rehearsal for the Mad Professors Band, which Duckstein and Stemkoski are in with two other Adelphi faculty members.
“I had this idea kicking around for a long time and mentioned it to Lee after a band rehearsal,” Duckstein said. “I told him that I wasn’t sure how to set something up that would react the way I had in mind, and two days later Lee had worked something up. I was stunned at how fast.”
Stemkoski explained that their inspiration was a mutual interest in Brian Eno, an experimental musician and visual artist who coined the term “generative music.”
“Eno created the iOS smartphone apps Bloom and Trope, where the user taps and swipes the screen to create a randomly generated musical composition, which we were both familiar with,” he said. “We tossed around some ideas about, instead, using the movement of dancers to generate music, and thus the concept was born.”
Stemkoski said he and Duckstein implemented the music-generating program as a web app for ease of use. Anyone can open the app’s webpage in their browser.
“It uses the video feed from your camera and scans for clusters of pixels within a specified color range—shades of reds, blues, etcetera. As these clusters enter or exit different regions across the screen, different preselected tones are played,” he said.
Duckstein said the tones were recorded and provided by composer Carson Moody, who also had great ideas for tones and how to adjust the musical dynamics for future versions of the software.
Choreographing in Real Time
While the students were excited to be part of the piece, knowing that their movements would impact the music, Duckstein said that starting the choreography without music was challenging.
“It was definitely work getting 12 individual students to think as a group and develop a dance in real time together,” he said. “But in both instances, it was rewarding to come out on top. Every dancer in the piece was seen, every dancer contributed and came to the stage alive and in the moment.”
Duckstein added that it was also a challenge—yet exhilarating—to let go, trusting the dancers to produce something interesting in the moment. “The cast and I were really excited to see what was going to happen every night. In fact, that’s how we came up with the title of the piece,” he said.
Potential for More Interaction
For Dr. Stemkoski, who is also the director of the Adelphi University Innovation Center, this interdisciplinary collaboration is just one example of how to use a collaborative and innovative mindset to support the University at large.
“The mission of the Innovation Center is to inspire, support and showcase innovative and interdisciplinary activities across the University,” he said. “This collaboration between dance and computer science is a perfect example of what we’re trying to accomplish. In fact, early versions of our music generation app were featured as the weekly tech exhibit in the Innovation Center. Adelphi students were able to try out the app and provided valuable feedback that was used to improve the app.”
Duckstein and Stemkoski are already thinking of their next dance collaboration. “While the November dance performance was my first time collaborating with Lee, it’s sparking a whole bunch of other ideas,” Duckstein said. “I think we’re going to come up with some bigger ideas for next time.”
Stemkoski added, “It would be really cool to also tie this in to dynamically adjusting the stage lighting or to project a dynamic art piece in the background, all driven by and synchronized with the movements of the dancers.”