Artist, composer, researcher working with voice, attention, and machine listening
I build musical systems from human attention, language, and machine perception. My work explores what happens when imperfect sensing, memory, and human behavior become compositional material rather than errors to be corrected.
Beyond Babel selected for Listening Place, Seoul International Computer Music Festival 2026.
Watch the fixed-media version
Effect-Time Cue Scheduling for Distributed Gaze-Based Musical Performance — accepted, Sound and Music Computing Conference 2026.
Paper
Peak-Residual Modal Estimation with Learned Calibration and High-Band Density Correction — DAFx26 Parameter Estimation Challenge, Task B.
Proceedings
Beyond Babel
2026 — Live audiovisual systemVoice, machine listening, custom software, sound
Add the demo video URL to SITE.videos.beyondBabel
Beyond Babel is a live audiovisual composition system that treats machine listening not as a transparent transcription tool, but as a material process.
Spoken language enters the system through separate microphones and is transformed into linguistic and acoustic traces. Each trace retains its source while becoming material that can be positioned, combined, and woven with traces produced by other speakers.
The work asks what survives when language is heard by a machine, what becomes altered or lost, and what can be composed from those unstable residues. Rather than correcting imperfect listening, Beyond Babel uses it as the beginning of a new musical process.
images/Beyond_babel_artwork_1.jpg
Traces from separate speakers, woven into a single passage.
A fixed-media version of Beyond Babel was selected for Listening Place at the Seoul International Computer Music Festival 2026. Watch it here.
In development
A public, multi-participant version
Beyond Babel currently runs with two microphones. The next stage is to open it to many speakers in a public space, where the composition is assembled by whoever happens to be present.
Visitors contribute short spoken fragments in the languages they use or think in. Speech is analyzed rather than recorded: what enters the piece is a trace made of the recognized words, the timing and contour of the delivery, and sound mapped from that prosody. The voice itself is not kept. Each trace stays attributed to its speaker while becoming material that can be placed beside material left by others in a shared weaving field.
At the end of each public working day, I preserve the state of the piece as a Daily Weave Record: a visual capture of the field paired with a short audio render. Because no voices are stored, these records hold the relationships between contributions rather than recordings of the people who made them.
images/beyond_babel_storefront_mockup.jpg
Conceptual mockup of a street-facing public iteration. Not documentation of a realized installation.
How a day is made
Visitor A speaksMicrophone 1 — Trace A
Visitor B speaksMicrophone 2 — Trace B
↓
Shared weaving fieldVisitors take turns placing and weaving traces
↓
Sonic resultThe day's composition, heard in the room
↓
Daily Weave RecordOne image and one short audio render, kept
A record that accumulates
New people bring new traces, so each day produces a different arrangement and a different sound. Set side by side, the daily records become a record of encounters between speakers over a working period, and can be shown as a wall of images and renders.
Each cell: one visual capture of the field, one short audio render.
Add the performance video URL to SITE.videos.gazeComposer
Gaze Composer is a live audiovisual performance system in which attention becomes a compositional action.
The performer's gaze does not simply move a cursor across a screen. Looking creates visual fragments, accumulates a history of attention, and changes the musical material available in the performance. Earlier fragments remain in the visual field as traces of previous decisions, and returning to them can bring degraded memories of earlier sound back into the present.
The work therefore treats looking as both an immediate control gesture and a form of memory. A performance gradually constructs its own visual and sonic history, which is transformed, recalled, and eventually allowed to disappear.
images/gazecomposer_ss2.jpg
The field fills with earlier decisions.
images/gazecomposer_ss3.jpg
A fragment releases its stored sound.
images/gazecomposer_ss4.jpg
The full history, immediately before dissolution.
Four movements
I
Inscription
The first movement establishes the act of looking as inscription. Each sustained moment of attention produces a visual fragment and associates it with the sonic state of that moment.
II
Accumulation
As the performance continues, fragments remain rather than disappearing immediately. The screen becomes a growing history of attention, and new musical decisions take place inside a field already shaped by previous ones.
III
Recall
When attention returns to an earlier fragment, the past can re-enter the performance. Stored sound is not reproduced perfectly; it returns in altered or degraded form, allowing the present to be composed with its own memory.
IV
Dissolution
In the final movement, the accumulated field is revealed as a whole before it disappears. The disappearance is not a reset of the work, but the closing gesture of the performance: the live visual memory is removed from the screen while its history remains elsewhere as documentation.
What the disappearance leaves
At the end of the performance, accumulated traces briefly appear together before disappearing from the live field. This final gesture separates temporary memory from recorded memory: what existed during the performance vanishes from the screen, but the performance does not vanish completely. Its residue remains in two forms: the audio recording and the graphic score generated from the history of gaze.
One performanceLive visual memory, removed from the screen at the end
↓
Audio recordingSonic memory
Graphic scoreVisual memory
Audio recording
Full performance, single take.
Graphic score: residue of attention
Each performance generates a graphic score from its accumulated gaze history. The score is not conventional notation for reproducing the piece exactly; it is a visual record of where attention accumulated, returned, and disappeared during that particular performance.
Archived alongside the audio recording, the score forms a second record of the same event: one sonic, one visual.
images/gc-score.jpg
Graphic score generated from a single performance.
Why noise
The visual traces are built from TV-static-like noise rather than representational images.
I use noise because gaze in this work is not meant to become a literal diagram of where the performer looked. Noise allows attention to appear instead as unstable material: something that can accumulate, overlap, degrade, be recalled, and disappear.
Its visual instability also parallels the way memory functions in the sound. Earlier sonic material does not return as an exact copy; it returns altered. The noise fragments therefore function simultaneously as traces of attention and containers of imperfect musical memory.
Proposed, not yet realized
Conceptual installation view
images/GazeComposer_artwork_mockup.jpg
Conceptual museum-scale performance visualization using the existing Gaze Composer visual system.
About
Doohyun Jung is an artist, composer, and researcher who develops musical systems from language, attention, bodily behavior, and machine perception. Trained initially as a classical vocalist, she works across interactive sound, audiovisual performance, machine listening, and custom software she writes herself.
Her recent work treats imperfect sensing as compositional material. Beyond Babel turns speech heard by a machine into linguistic and acoustic traces that can be positioned and woven together; Gaze Composer makes the performer's looking into an act of inscription, accumulation, and recall. Alongside her artistic practice she publishes technical research on gaze-based and distributed musical performance systems.
Sound and Music Computing Conference (SMC 2026), Zagreb, CroatiaPaper presentation and poster
Publications
2026
Effect-Time Cue Scheduling for Distributed Gaze-Based Musical Performance.Proceedings of the Sound and Music Computing Conference (SMC 2026), Zagreb. Accepted. PDF
2026
Peak-Residual Modal Estimation with Learned Calibration and High-Band Density Correction.Proceedings of the 29th International Conference on Digital Audio Effects (DAFx26), pp. 591–594. 1st Parameter Estimation Challenge, Task B. PDF
Research and artistic practice
2025 – present
Independent artist-researcher and developerInteractive musical systems combining machine listening, eye tracking, computer vision, embodied interaction, and audiovisual composition.
Education
2022 – 2024
Master of Music in Voice PerformanceBoston University, Boston, MA
2015 – 2019
Bachelor of Music in Vocal MusicSeoul National University, College of Music, Seoul
Areas of practice
Interactive and audiovisual composition; machine listening; voice and language
Eye tracking and embodied musical interaction; generative sound
Creative coding and performance systems — Python, JavaScript, Web Audio, MIDI and OSC
Additional professional practice
Classically trained vocalist and voice educator, with professional teaching and performance experience in South Korea and the United States.
