AI Model2026
Performs one-shot, long-horizon OCR parsing of multi-page documents by using Reference Sliding Window Attention (R‑SWA) to keep the KV cache constant, enabling single-pass transcription of dozens of pages. Key features: constant KV cache, DeepEncoder visual compression, MoE-backed 3B model—suited for large-scale PDF/image parsing on GPU.
AI Audio2026
Generates directed, multi-character, long-form speech from text with natural-language directions and voice cloning. Built on a 35B MoE backbone (3B active per token) with a codebook multi-token-prediction decoder to enable multi-turn revision, infilling edits, streaming, and expressive control.
AI Model2026
Provides a locally deployable, FP4-quantized variant of Qwen3.8-Flash-Next for multimodal (text, image, video) text generation. Uses NVIDIA Model Optimizer with quantization-aware distillation (QAD), a ~96 GiB checkpoint, and is validated on vLLM for Blackwell and DGX Spark.
AI Model2026
Multimodal image/video-to-text model optimized for on-device inference: the 1B variant uses early visual-token compression and multiple quantization formats to improve capability-per-compute, and supports mobile platforms and common inference frameworks.
AI Model2026
Text-only sparse Mixture-of-Experts language model for long-context reasoning, coding, and agentic tool use; 501B total / ~23B active parameters, 1M-token context, and BF16/FP8/NVFP4 checkpoints released under Apache 2.0.
Xiaohan Qin, Xiangdong Zhang +8School of AI, Shanghai Jiao Tong University, Shanghai Innovation Institute +3
Enables multi-head, depth-wise residual retrieval in Transformers by expanding a single residual stream into multiple context-aware streams using causal depthwise convolutions. Improves pretraining convergence and downstream benchmark scores with minimal overhead, yielding notable iso-loss compute speedups in large MoE models.
AI Model2026
Generates calibrated decisions (choice/score/noul) from a prompt and optional images, up to 256 options. Routed MoE experts are quantized to NVIDIA NVFP4 to reduce weights to ~25.6 GB and GPU memory to ~23.3 GiB; provided as a vLLM-compatible System 1 decision model with a /v1/decide endpoint.
AI Model2026
A System 1 multimodal decision model that picks among up to 256 options and returns calibrated probabilities for choice, yes/no and 0–5 score decisions from text and images. Fine-tuned on Qwen3.5-35B-A3B, available in bf16 and NVFP4 builds (NVFP4 fits ~23.3 GiB GPU), and served via vLLM's POST /v1/decide API.
AI Model2026
Quantized multimodal vision-language model for reasoning, coding, tool use, document and video understanding; 120B total (12B active) with NVFP4 quantization, native Multi-Token Prediction for speculative decoding, and up to 1M-token context — designed for deployment on NVIDIA GPU stacks.
AI Model2026
Runs a 24B-parameter mixture-of-experts reasoning model with only ~2B active parameters per token for fast on-device agentic workflows; provides a 131K-token context and 128K vocabulary, post-trained for tool calling and agentic RL—pair with retrieval for knowledge-heavy tasks.
Explores asymmetric compute allocation in looped Mixture-of-Experts models by compressing the first loop visit's sequence and reallocating saved compute to enlarge hidden size, yielding consistent NLL reductions and predicted compute savings across 1.3B–7B configurations.
Internalizes context-induced performance gains into model parameters by generating prospective queries and distilling context-conditioned outputs into mergeable low-rank or structural feed-forward updates, balancing consolidation and retention across 4B–120B dense and MoE models.