A smartphone app developed at Drexel University deconstructs music into discernible elements like tone, intensity, and rhythm, and facilitates a fuller way of experiencing a live performance. Drexel University’s Expressive & Creative Technologies Center (ExCITe) uses the “Science of Jazz” app to translate some of what’s measurable about music into visual form.
Microphones capture sound and the app transforms it to images in real time: one for how sound waves reach different parts of a concert hall, another to approximate which notes musicians are playing on their instruments, and another to depict the pitch and intensity of each instrument.
Dr. Youngmoo Kim, ExCITe’s director, is behind the app, which he says makes the live concert more educational and meaningful. Limited to the iPhone - and used only for jazz performances so far - see how it works:
http://youtu.be/zJcUX3wmElk
