Tianhua Tao led the project and carried out its core research and development: conceptualization, system design and implementation, experimental evaluation, profiling, analysis, visualization, and manuscript preparation.
Akshita Bhagia contributed broadly to the Olmo-core software, including but not limited to MoE code refactoring and integration, software maintenance, and testing.
Dirk Groeneveld, Pete Walsh, and Tyler Romero made foundational contributions to Olmo-core and early MoE prototyping, establishing the software foundation on which the Olmo-core DDP system described in this report builds.
Yashas Samaga and Jacob Morrison participated in technical discussions on MoE integration and downstream use. Their experience with the software informed feedback on design, training workflows, and usability.
Iz Beltagy provided team leadership, project direction, and strategic coordination.
Taira Anderson provided technical program management, including compute-resource allocation, scheduling, project administration, and coordination.
Noah A. Smith and Hannaneh Hajishirzi provided strategic guidance and research advising. Hannaneh Hajishirzi also served as the project’s primary academic advisor.
G.1 Use of AI Tools
We used AI tools to assist with developing the codebase and GPU kernels, drafting and revising this report, managing experiments, and creating visualizations. Building and documenting the training stack required repeated iteration across distributed systems code, kernel behavior, experiment orchestration, and technical explanation. We used AI assistance to support that iteration, explore implementation alternatives, and reduce repetitive coding and editorial work. This was a practical part of a human-led research process: the research direction, design decisions, and interpretation of results remained the responsibility of the authors.
AI-generated suggestions were assessed through code inspection, testing, profiling, experimental comparisons, and source checks, as appropriate to the task. We also kept a citation audit that records source evidence and review notes for individual cited claims. These checks can themselves miss errors; AI assistance does not transfer responsibility for the work. The authors remain responsible for the methods, results, and statements in this report.