Category

A lightweight open-source platform for running, managing, and integrating large language models locally via a simple CLI and REST API.

Provides an open-source platform for building, training, and deploying machine learning models across CPU, GPU, and TPU. Includes high-level APIs, model hubs, and tools for production deployment and optimization.

NVIDIA TensorRT is an SDK and tool-suite that compiles and optimizes trained neural-network models for ultra-fast, low-latency inference on NVIDIA GPUs.

ONNX (Open Neural Network Exchange) is an open ecosystem that provides an open source format for AI models, including deep learning and traditional ML. It defines an extensible computation graph model, built-in operators, and standard data types, focusing on inferencing capabilities. Widely supported across frameworks and hardware, it enables interoperability and accelerates AI innovation.

Orchestrates and scales Python-based AI/ML workloads from laptop to thousands of GPUs — exposing task and actor primitives plus high-level libraries for training, hyperparameter tuning, serving, RL, and data processing. Designed for heterogeneous accelerators and production ML pipelines.

Streamlit is an open-source app framework that turns Python scripts into shareable web apps in minutes. It enables data scientists and AI/ML engineers to build interactive data apps like dashboards, reports, or chat apps using pure Python, without front-end experience.

Optimizes and deploys deep learning models for inference across CPU, GPU and Intel NPUs. Converts models from PyTorch/TensorFlow/ONNX, exposes C++/Python/NodeJS APIs, and integrates with Hugging Face and Optimum for LLM/GenAI workloads.

Provides an inference platform to package, serve, and scale ML/LLM models across cloud and on-premises. Uses a Python-first SDK, model store, containerized builds, and features like adaptive batching, multi-model pipelines, and BentoCloud for managed deployments.

An AI-native, weight-centric infrastructure for quantitative trading that produces target portfolio weight vectors to unify data ingestion, strategy composition, backtesting, and live/broker execution. Modular pipeline supports ML/DRL allocators, LLM-ready preprocessing, multi-source data, and Alpaca integration for paper/live trading.

Serves machine learning models in production with TensorFlow Serving — an extensible model server that supports TensorFlow SavedModel, REST/gRPC APIs, model versioning, and deployment via Docker or Kubernetes.

SkyPilot is an open-source MLOps / AI infrastructure project that provides a unified control plane and CLI to run, manage, and scale AI workloads on any compute — Kubernetes, Slurm, 20+ clouds, or on-prem clusters. It supports job-as-code (YAML/Python), intelligent scheduling and cost optimization (spot instances, autostop), automatic setup/sync of environments, auto-recovery, and integrations for training, serving and inference workflows.

Provides Kubernetes-native CRDs and runtimes to deploy and serve predictive models and LLMs at scale — supports scale-to-zero autoscaling, GPU acceleration, multi-framework runtimes, and ModelMesh for high-density model loading.