Efficient Transformers: A Survey of Modeling and Training Approaches

This comprehensive survey examines various approaches to making transformer models more computationally efficient and environmentally sustainable.

The research analyzes different architectural innovations and training strategies that reduce the computational and energy requirements of transformer models while maintaining their effectiveness.

The authors provide a systematic comparison of different efficiency techniques and their impact on model performance, training costs, and environmental footprint.