Efficient Training of Large Language Models: A Survey

This comprehensive survey examines various approaches to make the training of large language models more efficient and environmentally sustainable.

The research analyzes different techniques including model compression, efficient attention mechanisms, and hardware-aware training strategies that can significantly reduce the computational and energy costs.

The authors provide a systematic comparison of different efficiency methods and their impact on model performance, training time, and energy consumption.