Daily AI

DistilBERT (distilbert-base-uncased)

A compact pretrained bidirectional Transformer distilled from BERT base for masked language modeling and downstream NLP fine-tuning. Trained via knowledge distillation on BookCorpus and English Wikipedia, it reduces size/latency (≈67M params) while preserving BERT-like representations.

BERT base model (uncased)

Pretrained uncased English BERT base model for masked language modeling and next-sentence prediction. ~110M parameters, pretrained on BookCorpus and English Wikipedia; commonly fine-tuned for classification, token labeling, and question answering.

GPT-2

Autoregressive transformer that generates English text from a prompt and can be fine-tuned for downstream NLP tasks. This is the 124M-parameter GPT-2 checkpoint (causal LM, byte-level BPE, 1024-token context) hosted on Hugging Face.

Spark-X2.5-4B

A 4B-parameter on-device general-purpose LLM for chat, writing, translation, coding and agentic workflows with native 1,000,000-token context. Uses a hybrid attention design to enable long-context efficiency, pretrained on ~20T tokens, and compatible with vLLM, llama.cpp, Ollama and LM Studio.

all-MiniLM-L6-v2

Encodes sentences and short paragraphs into 384-dimensional semantic vectors for retrieval, clustering, and similarity tasks. Fine-tuned from MiniLM-L6-H384 on ~1.17B sentence pairs with a contrastive objective; defaults to 256 word-piece truncation.

Text-to-Speech Human Preferences (315K)

Provides 315,000 pairwise human-preference votes comparing 15 English TTS models over 300 operational prompts, with 4,500 high‑quality audio renders and structured vote/pair/prompt records for training or evaluating preference/reward models. Metadata under CC-BY-4.0; audio use governed by model providers' terms.

Qdrant-FineWeb-10B

A 10‑billion‑document retrieval benchmark with per‑document 768‑dim unit‑norm dense embeddings and mGTE sparse embeddings, FineWeb text/metadata, and exact top‑1000 MS MARCO ground truth for ~120k queries. Built for large‑scale evaluation of dense/sparse/hybrid retrieval, filtered search, indexing, ANNS algorithms, and embedding compression.

TikTok Videos, 4.5 Billion

Provides 4.5 billion TikTok video records with captions, timestamps, music IDs and engagement counts for research; split across 27 zstd-compressed Parquet files (~289 GB) and sampled via TikTok's mobile API; released for research-use only with privacy and ToS caveats.

Language Models Can Control Their Own Attention

Introduces Declarative Attention (DA), a zero-shot protocol that has LMs declare which parts of long context to attend to during chain-of-thought, letting the runtime build dynamic attention masks and skip most KV-cache reads. Produces large token savings (up to ~52% on Gemma-4-31B) with modest accuracy loss.

It Takes Two to Match: Co-Evolving Generative Retriever with Reinforcement Learning

Generates compact keyword sets for both queries and items with LLMs and matches them directly via an inverted index. Uses supervised fine-tuning to align keyword spaces, then alternates GRPO-based reinforcement learning on query- and item-side generators to co-evolve representations and maximize retrieval F1 while staying compatible with keyword-based infrastructure.

EarlyEval: Cheaper Agent Evaluation via Early Outcome Prediction

Predicts an LLM agent's final success or failure from partial execution traces and halts runs when outcomes are confident to save per-task compute. Uses LightGBM success/failure classifiers on behavioral, textual, and reference features; cuts 13–26% steps and up to 44% input tokens on benchmarks.

SolarWM: Open Data and Scalable Training for Long-Horizon Video World Models

Provides an end-to-end, reproducible foundation for camera‑controllable, long‑horizon video world models — converting 1.43M clips from 10 datasets into a unified canonical corpus and releasing data, pipelines, recipes, and weights. Introduces backbone‑native adaptation and a three‑stage training recipe to produce 5B–33B models that enable minute‑to‑hour rollouts after training on 5s sequences.

Repo-To-Skill: Distilling GitHub Repositories Into AI4AI Skills

Distills operational know‑how from ML GitHub repositories into compact, verified 'skills' that research agents can load and reuse. Produces a skill format (SKILL.md, references, scripts), the AREX‑Skill Library (5,000+ skills from 1,000 repos), and demonstrates sizable benchmark gains when agents use skills.