Gemma 2 27B

Google's largest open-weights model in Gemma 2 family with 27B parameters. Released June-July 2024, trained on 13 trillion tokens of web documents, code, and mathematics. Features interleaved local/global attention, logit soft-capping for stability, Grouped Query Attention for efficiency. Outperforms Qwen1.5 32B baseline and performs "only a few percent below" Llama 3 70B despite being significantly smaller. Optimized to run full precision on single Google Cloud TPU host, NVIDIA A100 80GB, or H100. 256K vocabulary with 4K context using RoPE. Commercially-friendly license. Available via Google AI Studio, Kaggle, Hugging Face.