CodeLlama 34B

Meta's specialized code generation model with 34B parameters built on Llama 2 architecture. Released August 2023, trained on 500 billion tokens of code and code-related data from January-July 2023. Features extended 100,000 token context window for handling large codebases. Supports Python, C++, Java, PHP, TypeScript/JavaScript, C#, and Bash. Three variants: Base (code completion), Python (Python-specialized), Instruct (instruction-following). Achieves 53.7% HumanEval and 56.2% MBPP. Trained on Meta's Research Super Cluster requiring 400K GPU hours on A100-80GB hardware across all 9 CodeLlama models. Open weights enable customization and self-hosting.