Whisper GitHub

Whisper is a general-purpose speech recognition model developed by OpenAI. It was trained on a large and diverse dataset of audio data and is designed as a multi-task model capable of performing tasks such as multilingual speech recognition, speech translation, and language identification. Whisper utilizes the Transformer sequence-to-sequence architecture and has been trained for various speech processing tasks, including multilingual speech recognition, speech translation, spoken language recognition, and speech activity detection. These tasks are represented collectively through a series of labels that are predicted by the decoder, enabling a single model to replace multiple stages in traditional speech processing pipelines. The multi-task training framework makes use of specialized labels that serve as task identifiers or classification targets.

Whisper can be used via the command line or in Python. When using it through the command line, you can specify the audio file and the model size to transcribe the audio content. When using it in Python, you can load the model and employ the `transcribe()` method to process the audio files.

Multilingual speech recognition
voice translation
Language recognition
Voice Activity Detection