Llama 2 70B

Meta's largest open-weights Llama 2 model with 70B parameters. Released July 2023, trained on 2 trillion tokens with 40% more data than Llama 1. Features 4K context length (expanded from 2K in Llama 1) and Grouped-Query Attention for improved inference scalability. Supports both research and commercial use with permissive license. Pretrained on data through September 2022 with tuning data through July 2023. Global batch size of 4M tokens during training. Fine-tuned chat variant optimized for dialogue using supervised fine-tuning and RLHF. Widely adopted as foundation for custom models and applications. Superseded by Llama 3.1 70B (July 2024).