Tag

Prefect

Orchestrates and schedules Python data pipelines and workflows with primitives for retries, caching, parameters, and deployments. Provides either a self-hosted server or managed Prefect Cloud for monitoring, observability, and integrations across common data tools.

Triton Inference Server

Serves machine learning and deep learning models for cloud, data center, edge and embedded environments. Supports multiple frameworks and backends, dynamic and sequence batching, HTTP/gRPC APIs, Docker deployment and NVIDIA-optimized runtimes.

ONNX Runtime | Home

Runs ONNX models faster on CPU, GPU, and NPU by routing graph subgraphs to backend execution providers (CUDA, TensorRT, OpenVINO, DirectML, CoreML). One engine serves the same model across cloud, browser, mobile, and edge, for both inference and training.

Bento: Run Inference at Scale

Turns Python ML code into production inference APIs that scale on Kubernetes or any cloud. Bundles models, dependencies, and serving logic into versioned "Bentos" with autoscaling, scale-to-zero, and multi-GPU serving for LLMs and custom models.

NVIDIA NeMo Speech

Provides a toolkit and codebase for building, training, and deploying speech and multimodal models — Automatic Speech Recognition, Text-to-Speech, and speech-aware LLMs — with modular neural components and pre-trained checkpoints for PyTorch. Supports streaming/low-latency inference, multi-language models, and optional compiled kernels for acceleration.

Nango

Manages OAuth, credential storage, API proxying, and deployable TypeScript integration functions so products and AI agents can access 800+ external APIs. Includes AI-assisted function generation, a production runtime with scaling and observability, and cloud or self-hosted deployment options.

FinRL-X

An AI-native, weight-centric infrastructure for quantitative trading that produces target portfolio weight vectors to unify data ingestion, strategy composition, backtesting, and live/broker execution. Modular pipeline supports ML/DRL allocators, LLM-ready preprocessing, multi-source data, and Alpaca integration for paper/live trading.

KServe

Serves predictive and generative ML models on Kubernetes via a single InferenceService CRD, with scale-to-zero, canary rollouts, and an OpenAI-compatible LLM path on vLLM. One autoscaling abstraction over PyTorch, XGBoost, ONNX, and HuggingFace.

ZenML — The AI Control Plane

Orchestrates ML training pipelines and production agent workflows from one Python codebase that runs unchanged from a laptop to Kubernetes or any cloud. Auto-versions artifacts, models, and agent checkpoints, with no orchestrator or framework lock-in.

CowAgent

Framework for building multi-channel AI assistants that autonomously plan tasks, invoke tools/skills, and keep long-term memory; supports many LLM providers and channels (WeChat, Feishu, QQ, web) for local or server 24/7 deployment.

Text Generation Inference

Rust-and-Python toolkit that serves open-source LLMs (Llama, Falcon, Mixtral, StarCoder) over HTTP/gRPC with tensor parallelism, continuous batching, Flash/Paged Attention and quantization. Now in maintenance mode, pointing users toward vLLM and SGLang.

Reflex

Build full‑stack web apps entirely in Python — write frontend components and backend state as Python classes with a reactive model. Provides fast refresh, deployment tooling, and AI-focused integrations such as an AI Builder and an Agent Toolkit for connecting LLMs and image models.