Tag

Unified framework for few-shot evaluation of generative language models across 60+ academic benchmarks. Supports multiple model backends (Hugging Face, vLLM, APIs, local servers), configurable prompts/YAML configs, and reproducible exports for leaderboards and research comparisons.

Provides Kubernetes-native CRDs and runtimes to deploy and serve predictive models and LLMs at scale — supports scale-to-zero autoscaling, GPU acceleration, multi-framework runtimes, and ModelMesh for high-density model loading.

High-throughput inference and serving engine for LLMs that reduces KV-cache memory use with PagedAttention. Provides continuous batching, CUDA/HIP acceleration, Hugging Face integration, quantization, and an OpenAI-compatible API for production LLM serving.

Runs LLMs, vision, audio and multimodal models locally with an OpenAI-compatible API, supporting CPU-only and GPU acceleration across 35+ backends. Includes built-in agents, multi-user access controls, a model gallery, and privacy-first local inference.

Unifies efficient fine-tuning and deployment workflows for 100+ LLMs and VLMs — via CLI, web UI, and reproducible Docker images. Integrates many training methods (LoRA/QLoRA/OFT/PPO), optimizers, and inference backends (vLLM), making cross-model experiments and production deployment easier.

Serves language, speech, and multimodal models through a single OpenAI‑compatible inference API that supports heterogeneous hardware, automatic batching, multiple backends (ggml, vLLM, TensorRT) and distributed deployment.

Provides a scalable RLHF framework combining Ray, vLLM and DeepSpeed for distributed training and high-throughput generation. Uses an agent-based token-in-token-out execution, ships PPO/REINFORCE++ variants, and supports VLMs—suited for large-scale RLHF experiments and production pipelines.

Deploy arbitrary AI models and pipelines as custom inference servers in Python — write the inference logic, control batching/routing/streaming, and run self-hosted or one-click to Lightning Cloud with autoscaling and GPU support.

Performs document OCR, layout analysis, reading-order detection and table recognition across 90+ languages using a ~650M-parameter vision–language model; offers per-page and per-block modes and supports GPU (vllm) and CPU/Apple Silicon backends.

Pocket-sized multimodal LLM for efficient image- and video-understanding on mobile and edge devices, featuring mixed 4x/16x visual-token compression (MiniCPM‑V 4.6), compact 1.3B variants, and ready guides for iOS/Android/HarmonyOS deployment.

High-performance GPU kernel library and JIT kernel generator that accelerates LLM inference serving by optimizing attention (block-sparse KV-cache, customizable attention templates), dynamic scheduling, and multiple backends (FlashAttention/CUTLASS/cuDNN/TensorRT).

Streamlines the full lifecycle of foundation models — data prep, fine-tuning (SFT/LoRA/QLoRA/GRPO), evaluation, and deployment — with ready-to-run recipes, multi-engine inference support, and cloud/CLI workflows for both laptop experiments and large-scale runs.