Tag

Provides research-grade implementations and pretrained models for sequence tasks (translation, LM, speech). Offers multi-GPU training, fast generation (beam/sampling/lexical constraints), mixed-precision, and state sharding — aimed at researchers reproducing or extending papers.

Provides a PyTorch-based framework for building, training, and deploying speech and multimodal models (ASR, TTS, speech LLMs). Includes NVIDIA-optimized components, pre-trained checkpoints, streaming/low-latency features, and Hugging Face integration to accelerate prototyping to production.

Multilingual sequence-to-sequence speech model and toolkit for speech recognition, speech-to-text translation, and language identification. Offers several model sizes (tiny → large/turbo) for different speed/accuracy trade-offs and ships with a CLI and Python API for offline transcription workflows.

FunASR is an open-source end-to-end speech recognition toolkit (ASR) led by Alibaba DAMO Academy. It supports ASR, voice activity detection (VAD), punctuation restoration, speaker verification/diarization, multi-talker ASR, emotion recognition and more. FunASR provides many industrial-grade pretrained models, inference scripts, and deployment runtimes for research and production use.

Reimplements OpenAI's Whisper with CTranslate2 to speed up transcription (up to ~4×) and reduce memory use. Supports CPU/GPU, INT8 quantization, batched inference, word timestamps and VAD—aimed at fast, low-memory Whisper inference rather than training.

Listens to the microphone and transcribes speech to text in real time with voice-activity detection, wake-word support, and Faster-Whisper GPU transcription. Provides callback hooks and server/client modes for low-latency voice assistants and real-time apps; now community-driven.

Converts videos between languages by transcribing audio, translating subtitles, and producing AI dubbing—supports local and online ASR/LLM/TTS providers, speaker diarization, voice cloning, and GUI/CLI workflows for batch or headless use.

Terminal CLI for on-device Whisper ASR using Hugging Face Transformers + Optimum, with optional Flash Attention 2, batching, and diarization support — focused on high-throughput transcription on NVIDIA GPUs and Apple Silicon (mps).

Performs speaker diarization (who spoke when) with pyannote-audio: combines voice-activity detection, speaker-change and overlapped-speech detection to produce time-stamped speaker segments; compatible with Hugging Face Endpoints and ASR pipelines.

Hands-free voice-first companion with a Live2D avatar for real-time conversations with LLMs. Cross-platform web and desktop clients, runs locally or via cloud APIs, supports local ASR/TTS and modular customization for personas and models.

Framework for building realtime, programmable voice and multimodal agents that combine STT, LLM, TTS, WebRTC and telephony. Includes job scheduling, semantic turn detection, plugin ecosystem and built-in test/judges for server-side conversational apps.

Pipecat is an open-source Python framework for building real-time voice and multimodal conversational AI agents. It provides pluggable integrations for STT, TTS, LLMs, transports (WebSocket/WebRTC), and composable pipelines to enable low-latency voice-first assistants, multimodal interfaces, and conversational agents.