Extracted Insight:
- The School of AI, Bangalore, built and pre‑trained LightningLM, a 120‑billion‑parameter large language model, proving frontier‑scale AI can be designed, trained, and scaled within India.
- Training employed a progressive growth strategy on a single 8‑GPU node, taking 40‑50 days and approximately $15,000 in compute; the full training run is projected to cost about $100,000, far below the $1‑2 million typical cost for comparable models.
- The model was trained on roughly 100 billion tokens selected from a curated 1‑trillion‑token corpus, with at least 25 % Indic content guaranteed in each training batch, and utilizes a Mixture‑of‑Experts architecture routing computation across 460 expert networks.
- Three accompanying open‑source research papers were released:
1. BrahmicTokenizer‑131K reduces Indic token count by 26.7 % overall (over four‑fold reduction on Odia) while matching or beating leading tokenizers on English, code, and math benchmarks.
2. Kronecker embeddings replace large embedding tables, removing 91‑94 % of input‑side parameters and shrinking storage from gigabytes to megabytes.
3. Reversible Foundations documents a state‑preserving scaling method that grows a 120 B sparse MoE model stably on constrained hardware, a contribution typically seen only from the largest global labs.
- All models, code, and papers are publicly available at www.lightninglm.theschoolofai.in; the work was conducted as part of ERA V4, a cohort of more than 300 students.
- Contact details: Rohan Shravan, Founder, The School of AI, Inkers.ai (rshravan@theschoolofai.in); Shenaz Bapooji, Skyful Marketing Advisory Pvt Ltd (shenaz.bapooji@sky-ful.com).