NLP

This pioneering study examines the carbon footprint of training natural language processing models. The authors quantify the financial and environmental costs of training various NLP models. The study reveals that training a single BERT model can emit as much CO2 as a trans-Atlantic flight, and that the computational costs of NLP models double every 3-4 months. The authors provide concrete recommendations to reduce environmental impact, particularly by prioritizing energy efficiency in model design and using renewable energy sources for training.