Tag

DeepSeek, founded in 2023, is dedicated to developing world-class foundational models and technologies for general artificial intelligence, tackling cutting-edge research challenges in AI. Leveraging its self-developed training framework, an in-house intelligent computing cluster, and tens of thousands of GPUs, the DeepSeek team released and open-sourced several large-scale models—each with tens of billions of parameters—within just six months. These include the general-purpose large language model DeepSeek-LLM, the code-specialized DeepSeek-Coder, and, in January 2024, China’s first open-source MoE (Mixture-of-Experts) model, DeepSeek-MoE. Across public benchmarks and real-world generalization tests, these models consistently outperform peers in their class. You can converse with DeepSeek AI or access its capabilities easily via API.

This paper presents DeepSeek-V2, a 236B-parameter open-source Mixture-of-Experts (MoE) language model that activates only 21B parameters per token, achieving top-tier bilingual (English and Chinese) performance with remarkable training cost savings (42.5%) and inference efficiency (5.76× throughput) compared to previous models. Its innovations—Multi-head Latent Attention (MLA) and DeepSeekMoE—reduce memory bottlenecks and boost specialization. The paper’s impact lies in advancing economical, efficient large-scale language modeling, pushing open-source models closer to closed-source leaders, and paving the way for future multimodal and AGI-aligned systems.

This paper introduces DeepSeek-V3, a 671B-parameter Mixture-of-Experts (MoE) language model that activates only 37B parameters per token for efficient training and inference. By leveraging innovations like Multi-head Latent Attention, auxiliary-loss-free load balancing, and multi-token prediction, it achieves top-tier performance across math, code, multilingual, and reasoning tasks. Despite its massive scale, DeepSeek-V3 maintains economical training costs and outperforms all other open-source models, achieving results comparable to leading closed-source models like GPT-4o and Claude-3.5, thereby significantly narrowing the open-source vs. closed-source performance gap.

This paper introduces DeepSeek-R1, a large language model that improves reasoning purely through reinforcement learning (RL), even without supervised fine-tuning. It shows that reasoning skills like chain-of-thought, self-reflection, and verification can naturally emerge from RL, achieving performance comparable to OpenAI’s top models. Its distilled smaller models outperform many open-source alternatives, democratizing advanced reasoning for smaller systems. The work impacts the field by proving RL-alone reasoning is viable and by open-sourcing both large and distilled models, opening new directions for scalable, cost-effective LLM training and future development in reasoning-focused AI systems.