Tag

NVIDIA TensorRT is an SDK and tool-suite that compiles and optimizes trained neural-network models for ultra-fast, low-latency inference on NVIDIA GPUs.

Provides NumPy-compatible array operations with composable program transformations — automatic differentiation, JIT compilation to XLA, and vmap/pmap for vectorization and parallel execution. Optimized for GPUs/TPUs and widely used for research and large-scale model training.

Provides a PyTorch-based framework for building, training, and deploying speech and multimodal models (ASR, TTS, speech LLMs). Includes NVIDIA-optimized components, pre-trained checkpoints, streaming/low-latency features, and Hugging Face integration to accelerate prototyping to production.

JIT-compiles Python kernels to efficient CPU/GPU-executable, differentiable primitives for simulation and spatial computing — integrates with PyTorch/JAX/Paddle for end-to-end optimization; best suited for physics, geometry, and robotics workloads (GPU acceleration requires NVIDIA CUDA).

Fast, memory-efficient exact attention kernels that cut attention memory from quadratic to near-linear and speed up training/inference on GPUs (A100/H100). Offers multi-backend support (CUDA/Triton/ROCm), KV-cache for decoding, and successive FlashAttention versions for different hardware.

Provides GPU-accelerated simulation and ready-to-train robot-learning environments (RL, imitation, motion planning) built on NVIDIA Isaac Sim. Includes RTX-based sensors, LIDAR, 30+ environments and 16+ robot models, and supports local or distributed runs for sim-to-real workflows.

A local, offline web UI and API for running and interacting with LLMs (text + vision). Supports multiple backends, tool-calling, fine-tuning and portable builds—designed for private, offline use and flexible model/backend switching.

High-throughput inference and serving engine for LLMs that reduces KV-cache memory use with PagedAttention. Provides continuous batching, CUDA/HIP acceleration, Hugging Face integration, quantization, and an OpenAI-compatible API for production LLM serving.

Scans LLMs for security and safety failures — probing for hallucination, data leakage, prompt injection, jailbreaks, toxicity, and misinformation. A CLI red‑teaming kit that runs static, dynamic and adaptive probes across many providers and outputs structured JSONL reports.

Terminal CLI for on-device Whisper ASR using Hugging Face Transformers + Optimum, with optional Flash Attention 2, batching, and diarization support — focused on high-throughput transcription on NVIDIA GPUs and Apple Silicon (mps).

Provides a Python inference pipeline and unified API for running AI weather and climate models — load model wrappers (FourCastNet3, GraphCast, AIFS), swap data sources (GFS/IFS/ERA5), and write forecasts to Zarr or serve them via inference utilities.

A research codebase and model family for vision–language models that experiments with data‑centric post‑training strategies and long‑context multimodal reasoning. Includes model reports, released research weights (non‑commercial), grounding tools (LocateAnything) and integrations for inference/optimization.