AIAny
AI Infra2023
Icon for item

The Data Engineering Handbook

Curated learning hub that aggregates roadmaps, tutorials, bootcamps, books, projects, and tool recommendations for learning data engineering and production data infrastructure. Focuses on practical applied learning (projects, interview prep, community links) rather than code libraries.

Introduction

Most people learning data engineering struggle to find a single, up-to-date place that connects fundamentals (ETL, storage, warehouses), modern lakehouse patterns, and the operational practices that production ML/AI teams need. This handbook consolidates those threads into a navigable learning path and resource index prioritized for applied skills and job readiness.

What Sets It Apart
  • Comprehensive curated index: organizes roadmaps, free bootcamps, project ideas, interview prep, books, newsletters, and community links so learners can move from fundamentals to production topics without hunting across dozens of disparate blogs and repos.
  • Practical focus on infrastructure that matters to ML/AI: includes recommended tools and vendors across orchestration, data lakes/lakehouses, warehouses, data quality, real-time systems, and LLM app libraries — useful when you need to design pipelines that feed ML models or productionize inference.
  • Lightweight, link-first format: instead of reinventing tutorials, it points to canonical guides, whitepapers, and projects, lowering the time-to-resource for concrete hands-on practice and cohort bootcamps.
Who it's for and tradeoffs

Great fit if you are building a study plan to transition into data engineering or MLOps, preparing for interviews, or assembling a list of practical tools and projects to learn by doing. Look elsewhere if you need an opinionated, end-to-end code library or turnkey software package — the handbook is a curated index and learning guide, not a maintained SDK. Content quality depends on linked upstream sources and contributor maintenance, so verify versions for production decisions.

Information

  • Websitegithub.com
  • OrganizationsDataExpert-io, DataExpert.io
  • Published date2023/11/19

Categories

More Items

GitHub
AI Infra2025

Measures generative AI inference performance with token-level metrics (TTFT, inter-token latency), latency, and throughput under realistic traffic patterns. Provides a multiprocess engine, real-time TUI dashboard, extensible plugins, and integrations for telemetry and result uploads, aimed at inference benchmarking and capacity planning.

GitHub
AI Train2019

Train and experiment with multi-billion to trillion-parameter transformer models on large GPU clusters using GPU-optimized building blocks and reference training scripts; offers advanced parallelism and mixed-precision support for research teams and ML engineers.

GitHub

Indexes full text of visited web pages and local files on a self‑hosted server so you can search your personal knowledge from a web UI, terminal, CLI, or an AI assistant. Runs without mandatory telemetry, offers a browser extension for automatic capture, and supports optional semantic search via a configurable embeddings endpoint.