Expressive Acoustic Guitar Sound Synthesis

In modern days, music production extensively employs Digital Audio Workstations, where MIDI (Musical Instrument Digital Interface) serves as the primary format for music manipulation. In these cases, imbuing expressiveness into MIDI to achieve realistic audio rendering is a vital yet challenging task, commonly referred to as MIDI-to-audio synthesis.

Current off-the-shelf methods heavily rely on sample-based synthesis using prerecorded, high-quality audio samples. However, sample-based synthesis typically lacks the necessary flexibility to faithfully capture the extensive range of acoustic nuances inherent in expressive musical instruments. This work focuses on the case of acoustic guitar instrument, which is known for exhibiting subtle yet clear differences between audio recordings of realistic performances and those generated solely through MIDI.

Musical Instrument Timbre Transfer

Musical instrument timbre transfer aims to transform the instrumental identity of audio tracks while preserving core musical content such as melody and rhythm. This task becomes especially challenging when the fine-grained structural conditions derived from the source audio contain instrument-specific playing characteristics (e.g., attack transients and vibrato), since these details can lead to timbral ambiguity, shifting the transferred output away from the target instrument's natural timbral distribution.

Motivated by this observation, our work introduces target-adaptive control mechanism to address timbral ambiguity caused by source-derived structural conditions whose fine-grained details may conflict with the target instrument's acoustic properties. Rather than copying all source expressive details uniformly, the method uses text prompts within a ControlNet framework to adaptively scale the frame-wise influences of pitch and loudness controls, allowing the model to attenuate or emphasize those details in accordance with the target instrument's identity.