Most of a recording is silence
The challenge
Feeding raw audio to the model spends a large share of the compute transcribing pauses, which is the main reason long-file throughput is bad.
The approach
Voice activity detection segments speech first, so only speech reaches the model. The batching is then built around those segments rather than around file boundaries.