Metaflow

Netflix’s human-centric framework for building and operating real-life data-science and ML workflows with idiomatic Python and production-grade scaling.

Visit Website

Introduction

Overview

Metaflow abstracts infrastructure so data scientists can prototype locally and scale to the cloud with a single decorator, while maintaining versioned data and code snapshots.

Key Capabilities

Pythonic DAG definition with automatic resume/replay
Built-in artifact store & data versioning
Seamless scaling to AWS Batch / Kubernetes / Argo
Integrated notebook, CLI and UI for monitoring
Supports R, Conda, and integrations with SageMaker & Ray

Back

Information

Websitemetaflow.org
AuthorsNetflix
Published date2019/12/03

More Items

ZenML

2021

ZenML

An extensible open-source MLOps framework that lets teams design portable, reproducible pipelines decoupled from infra stacks.

ai-development mlops

Flyte

2020

Lyft, Flyte Community

A Kubernetes-native workflow engine (originally at Lyft, now LF AI & Data) that provides strongly-typed, versioned data/ML pipelines at scale.

ai-development mlops

DeepFlow

2022

Yunshan Networks

A Yunshan Networks open-source observability stack that delivers zero-code eBPF-based tracing, metrics and continuous profiling for cloud-native & AI workloads.

ai-development mlops

Metaflow

Introduction

Overview

Key Capabilities

Information

Categories

Tags

More Items

ZenML

Flyte

DeepFlow