AIAny
AI Infra2023
Icon for item

The Data Engineering Handbook

Curated learning hub that aggregates roadmaps, tutorials, bootcamps, books, projects, and tool recommendations for learning data engineering and production data infrastructure. Focuses on practical applied learning (projects, interview prep, community links) rather than code libraries.

Introduction

Most people learning data engineering struggle to find a single, up-to-date place that connects fundamentals (ETL, storage, warehouses), modern lakehouse patterns, and the operational practices that production ML/AI teams need. This handbook consolidates those threads into a navigable learning path and resource index prioritized for applied skills and job readiness.

What Sets It Apart
  • Comprehensive curated index: organizes roadmaps, free bootcamps, project ideas, interview prep, books, newsletters, and community links so learners can move from fundamentals to production topics without hunting across dozens of disparate blogs and repos.
  • Practical focus on infrastructure that matters to ML/AI: includes recommended tools and vendors across orchestration, data lakes/lakehouses, warehouses, data quality, real-time systems, and LLM app libraries — useful when you need to design pipelines that feed ML models or productionize inference.
  • Lightweight, link-first format: instead of reinventing tutorials, it points to canonical guides, whitepapers, and projects, lowering the time-to-resource for concrete hands-on practice and cohort bootcamps.
Who it's for and tradeoffs

Great fit if you are building a study plan to transition into data engineering or MLOps, preparing for interviews, or assembling a list of practical tools and projects to learn by doing. Look elsewhere if you need an opinionated, end-to-end code library or turnkey software package — the handbook is a curated index and learning guide, not a maintained SDK. Content quality depends on linked upstream sources and contributor maintenance, so verify versions for production decisions.

Information

  • Websitegithub.com
  • OrganizationsDataExpert-io, DataExpert.io
  • Published date2023/11/19

Categories

More Items

GitHub
AI Infra2026

Compresses tool outputs, logs, files and RAG chunks locally before they reach the LLM to reduce token usage while caching originals for on-demand retrieval. Provides proxy/library/wrap modes, JSON- and code-aware compressors, and reversible compression.

GitHub
AI Infra2026

Provides a self‑hosted OpenAI‑compatible gateway that exposes one API endpoint to route requests to 40+ model providers, issue revocable virtual keys, enforce budgets before spend, and record usage.

GitHub
AI Coding2023

A statically typed, parallel programming language that prevents AI-introduced bugs by making correctness claims (laws) require machine-checked proofs. Compiles to native code and targets CPU/GPU parallelism with a fast proof checker.