BLOOM 176B

Pioneering open-access multilingual model with 176B parameters designed and built through collaboration of 1,000+ researchers. Released July 2022 as first multilingual LLM trained in complete transparency. Decoder-only Transformer trained on ROOTS corpus covering 46 natural languages and 13 programming languages (59 total). Trained for 117 days (nearly 4 months) on 416 A100 80GB GPUs via Jean Zay supercomputer (French government). Training used 1.6TB multilingual dataset (350B tokens) achieving 150 TFLOPs throughput. Largest publicly available open multilingual model. Released under Responsible AI License. Represents unprecedented international research collaboration in AI.