publications

2025

1. Blog

   Layer Sharding for Large-Scale Training with Muon

   2025

   Abs Bib Blog

   Innovated layer sharding to improve the performance of the state-of-the-art Muon optimizer by 30x, making it viable for large-scale LLM pre-training.

   @article{vanjani2025layersharding,
     title = {Layer Sharding for Large-Scale Training with Muon},
     author = {},
     year = {2025},
   }

2. arXiv

   Rethinking Reflection in Pre-Training

   2025

   Abs arXiv Bib Blog

   Investigates the role of reflection mechanisms during large language model pre-training and proposes improvements to training methodology.

   @article{vanjani2025reflection,
     title = {Rethinking Reflection in Pre-Training},
     author = {},
     year = {2025},
   }

3. arXiv

   Essential-Web v1.0: 24T Tokens of Organized Web Data

   2025

   Abs arXiv Bib HTML

   Presents Essential-Web, a large-scale curated web dataset comprising 24 trillion tokens organized for efficient LLM pre-training.

   @article{vanjani2025essentialweb,
     title = {Essential-Web v1.0: 24T Tokens of Organized Web Data},
     author = {},
     year = {2025},
   }

4. arXiv

   Practical Efficiency of Muon for Pretraining

   2025

   Abs arXiv Bib Blog

   Evaluates the practical efficiency of the Muon optimizer for large-scale language model pre-training, demonstrating significant improvements in training throughput and convergence.

   @article{vanjani2025muon,
     title = {Practical Efficiency of Muon for Pretraining},
     author = {},
     year = {2025},
   }

2024

1. Blog

   Data Curation Just Got Smarter: How Essential AI Used AMD GPUs to Label the Web for Smarter AI

   Yash Vanjani and others

   2024

   Abs Bib Blog

   Describes the large-scale data curation pipeline built at Essential AI using AMD GPUs for efficient web data labeling and annotation.

   @article{vanjani2024datacuration,
     title = {Data Curation Just Got Smarter: How Essential AI Used AMD GPUs to Label the Web for Smarter AI},
     author = {Vanjani, Yash and others},
     year = {2024},
   }