AudioTurbo: Fast Text-to-Audio Generation with Rectified Diffusion
Anonymous author
AbstractDiffusion models have significantly improved the quality and diversity of audio generation but are hindered by slow inference speed. Rectified flow enhances inference speed by learning straight-line ordinary differential equation (ODE) paths. However, this approach requires training a flow-matching model from scratch and tends to perform suboptimally, or even poorly, at low step counts. To address the limitations of rectified flow while leveraging the advantages of advanced pre-trained diffusion models, this study integrates pre-trained models with the rectified diffusion method to improve the efficiency of text-to-audio (TTA) generation. Specifically, we propose AudioTurbo, which learns first-order ODE paths from deterministic noise-sample pairs generated by a pre-trained TTA model. Experiments on the AudioCaps dataset demonstrate that our model, with only 10 sampling steps, outperforms prior models and reduces inference to 3 steps compared to a flow-matching-based acceleration model. |
|---|
Model ArchitectureArchitecture of the proposed model AudioTurbo. AudioTurbo consists of four key components: 1) a text encoder for generating text embeddings; 2) a LDM based on rectified diffusion for predicting audio latent representations; 3) a VAE decoder that reconstructs the mel-spectrograms; and 4) a vocoder for generating audio samples.
|
|---|
AudioTurbo (10 steps) VS baseline models (200 steps) on AudioCaps
|
|
|
|
|
|
|
|---|---|---|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|---|---|---|---|---|---|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
5-Step Generated Results on AudioCaps
|
|
|
|
|
|---|---|---|---|
| "Birds are squawking, and ducks are quacking" |
|
|
|
| "Water running with a man is speaking" |
|
|
|
|
|
|
|
|
|---|---|---|---|
| "A woman speaking followed by a porcelain plate clanking as food and oil sizzles" |
|
|
|
| "Blowing of a horn as a train passes" |
|
|
|
| "A crowd is clapping at an animal of some kind" |
|
|
|
Acknowledgement
TODO.
Page updated on 10 May 2026
