AudioTurbo: Fast Text-to-Audio Generation with Rectified Diffusion



Anonymous author



Abstract

Diffusion models have significantly improved the quality and diversity of audio generation but are hindered by slow inference speed. Rectified flow enhances inference speed by learning straight-line ordinary differential equation (ODE) paths. However, this approach requires training a flow-matching model from scratch and tends to perform suboptimally, or even poorly, at low step counts. To address the limitations of rectified flow while leveraging the advantages of advanced pre-trained diffusion models, this study integrates pre-trained models with the rectified diffusion method to improve the efficiency of text-to-audio (TTA) generation. Specifically, we propose AudioTurbo, which learns first-order ODE paths from deterministic noise-sample pairs generated by a pre-trained TTA model. Experiments on the AudioCaps dataset demonstrate that our model, with only 10 sampling steps, outperforms prior models and reduces inference to 3 steps compared to a flow-matching-based acceleration model.

Model Architecture

Architecture of the proposed model AudioTurbo. AudioTurbo consists of four key components: 1) a text encoder for generating text embeddings; 2) a LDM based on rectified diffusion for predicting audio latent representations; 3) a VAE decoder that reconstructs the mel-spectrograms; and 4) a vocoder for generating audio samples.

Figure 1: Framework of AudioTurbo.



AudioTurbo (10 steps) VS baseline models (200 steps) on AudioCaps

Text Query
Ground Truth
AudioTurbo
LAFMA
AudioLDM2
Tango
"A musical horn"
fname
fname
fname
fname
fname
"A man is speaking followed by a child speaking and then laughter"
fname
fname
fname
fname
fname
Text Query
Ground Truth
AudioTurbo
LAFMA
AudioLDM2
Tango
"Two men speaking followed by plastic clacking then a power tool drilling"
fname
fname
fname
fname
fname
"Brief speech followed by whistling a tune"
fname
fname
fname
fname
fname
"A train running on a railroad track followed by a vehicle door closing and a man talking in the distance while a train horn honks and railroad crossing warning signals ring"
fname
fname
fname
fname
fname


5-Step Generated Results on AudioCaps

Text Query
AudioTurbo
LAFMA
Auffusion
"Birds are squawking, and ducks are quacking"
fname
fname
fname
"Water running with a man is speaking"
fname
fname
fname
Text Query
AudioTurbo
LAFMA
Auffusion
"A woman speaking followed by a porcelain plate clanking as food and oil sizzles"
fname
fname
fname
"Blowing of a horn as a train passes"
fname
fname
fname
"A crowd is clapping at an animal of some kind"
fname
fname
fname

Acknowledgement

TODO.




Page updated on 10 May 2026