Synesthetica: 2024-2025
musical instrument, artistic canvas, physics sandbox
We've conceptualized an interactive touchscreen installation for the CURRENTS 2025 new media festival.
The audiovisual musical instrument Synesthetica (a portmanteau of synesthesia and melodica) uses a playful perspective on simulated particle systems to create a musical experience inspired by molecular dynamics.
The above video shows a screen recording of the early demo version of the instrunent's software running live.
Play the live web demo at this link: il41.github.io/synesthetica
Please note that the web demo does not currently support reverb or audio effects, so it will sound less immersive than the native software version heard in the above video.
Users interact with a 21.5 inch touch screen to create simulated molecules that play musical notes.
The instrument is controlled with a variety of dials that change physical, musical, and visual parameters simultaneously:
Color changes the colors and visual effects of the particles while changing the musical key in which they play notes.
Type controls the instrument that will be played by newly created molecules, along with their physical shape and texture.
Space simultaneously changes the audio reverb and echo effects while modulating the visual effects and colorful feedback trails.
Other dials and buttons may be added to increase the interactivity of the instrument. Please note that the functions, behavior, look, and sound of Synesthetica will mosty likely change as we continue to develop the installation and the instrument's software.
We would like to continue experimenting with the simulated behavior of the molecules to find the best balance between striking visuals, fun interactions, scientific engagement, and musicality.
scientific background:
Our key goal is to create a musical instrument and new media installation inspired by the physics of density and temperature in molecular particle systems to generate stochastic musical and visual experiences.
Inspired by animations and simulations of molecular dynamics, we conceptualized a simplified real-time musical system of particles to create music based on abstracted scientific concepts.
With playful symbolism, the ‘temperature’ knob gives players control over the tempo and speed of their instrument while engaging conversations about scientific topics such as the behavior of particles in a gas.
The generative animations of particles bouncing around evoke scientific animations of molecular-level processes, like water molecules bouncing around quickly when hot in a vapor cloud or moving slowly when cold in solid ice.
New Mexico’s relationship with molecular physics also provided inspiration for Synesthetica. We added a fission functionality that enables chain reactions of particles releasing energy and increasing velocity until a feedback loop state is reached. This make particles continuously stay at a high speeds, resulting in sustained dense generative musical sound and explosive visuals, experienced when creating many particles all at once.
Thank you for reading!
Please refrence the submitted materials for the full proposal document and details. As mentioned above, we are still developing the exact functionality of the software, however we are very excited for the installation potentials and look forward to building and sharing the physical musical instrument!
- Artific3