nanochat is a full-stack, minimal codebase for training, fine-tuning, evaluating, and deploying a ChatGPT-like large language model (LLM) from scratch on a single 8xH100 GPU node for under $100.
MiniMind is an open-source GitHub project that enables users to train a 26M-parameter tiny LLM from scratch in just 2 hours with a cost of 3 RMB. It provides native PyTorch implementations for Tokenizer training, pretraining, supervised fine-tuning (SFT), LoRA, DPO, PPO/GRPO reinforcement learning, and MoE architecture with vision multimodal extensions. It includes high-quality open datasets, supports single-GPU training, and is compatible with Transformers, llama.cpp, and other frameworks, ideal for LLM beginners.
The Claude Cookbooks is a GitHub repository by Anthropic featuring a collection of notebooks and recipes that demonstrate fun and effective ways to use the Claude AI model. It offers copy-paste Python code snippets for developers to integrate into projects, covering topics like classification, retrieval-augmented generation, tool use, third-party integrations, multimodal capabilities, and advanced techniques.
Awesome LLM Apps is a curated open-source repository collecting awesome LLM applications built with RAG, AI Agents, Multi-agent Teams, MCP, Voice Agents, and more, using models from OpenAI, Anthropic, Gemini, xAI, and open-source alternatives like Qwen or Llama that can run locally.
AI Engineering Hub is a comprehensive GitHub repository offering in-depth tutorials and 93+ production-ready projects on LLMs, RAGs, AI agents, and real-world AI applications for all skill levels.
This tutorial offers a detailed, line-by-line PyTorch implementation of the Transformer model introduced in "Attention Is All You Need." It elucidates the model's architecture—comprising encoder-decoder structures with multi-head self-attention and feed-forward layers—enhancing understanding through annotated code and explanations. This resource serves as both an educational tool and a practical guide for implementing and comprehending Transformer-based models.
The best introduction to how large language models (LLMs) like ChatGPT works in the world. It covers the three main stages of their training: pre-training on vast amounts of internet text, supervised fine-tuning to become helpful assistants, and reinforcement learning to improve problem-solving skills. The video also discusses LLM psychology, including why they hallucinate, how they use tools, and their limitations. Finally, it looks at future capabilities like multimodality and agent-like behavior.
The best introduction on how to use LLMs like ChatGPT. It covers the basics of how LLMs work, including concepts like "tokens" and "context windows". The video then demonstrates practical applications, such as using LLMs for knowledge-based queries, and more advanced features like "thinking models" for complex reasoning. It also explores how LLMs can use external tools for internet searches and deep research. Finally, the video delves into the multimodal capabilities of LLMs, including their use of voice, images, and video.
