Key Takeaways
- ML and AI have a role to play in the climate problem - what role is up to us
- Sustainable, or green, software means minimising the negative impact of our software on the climate
- You can use the Green Software Foundation’s Software Carbon Intensity Specification to assess the carbon score of your product
- Measuring the carbon emissions of your software as a rate gives you more actionability than only measuring a total
- Through the ML life cycle there is research and techniques available for you to make your software more energy efficient, hardware efficient and carbon aware. These include things like time and region shifting, shrinking the size of your ML models, selecting the right hardware, and improving your server density
Climate action and green software engineering have risen to the top of many technology companies' agenda, with accuracy hungry algorithms ML software consuming more and more computational resources, largely benefiting from the increasingly better hardware. Are the results worth the environmental cost?
This article introduces the field of green software engineering, showing the Green Software Foundation’s Software Carbon Intensity Specification, which is used to estimate the carbon footprint of software, and discuss ideas on how to make machine learning greener. It aims to give you the tools to take an active part in the climate solution.
Environmental impact of machine learning systems
When it comes to environmental impact, machine learning systems behave just like all software systems in many senses. They use energy and hardware, which in turn generates carbon emissions. What is particularly interesting with ML is the scale and the rapid increase of ML that we are currently experiencing. There is no denying that we live in the age of AI and machine learning. Never before has it had as profound an impact on our lives as it has right now. AI makes splashy headlines in all areas of life, from art, to medicine, warfare, and school papers, to climate.
The use of ML is rapidly increasing, as are the size of ML models, and the amount of problem areas to which we apply ML. The state of AI in 2022 report by McKinsey shows that AI adoption has more than doubled between 2017 and 2022, and the number of AI capabilities used by organisations has also doubled between 2018 and 2022, with NLP taking the lead. The State of AI in the Enterprise report from Deloitte in 2022 found that 94% of their respondents say AI is critical to success and 79% of respondents say they've fully deployed three or more types of AI, compared to just 62% in 2021. This growth alone makes ML and sustainability interesting to talk about.
Green software is often very context specific; there is no magic potion or silver bullet that will work for all software systems, so some tailor-made conversations and patterns are needed. In the end, we want software to be a part of the climate solution, not the climate problem. While the software industry does not contribute to the most greenhouse gases out of all the industries, our impact is still significant and we are a very small number of people who make up the builders of our carbon footprint, which is good in some sense, because that means we don’t have to change the minds of so many people to change the trajectory.
Measuring the environmental impact of a software system
There are many aspects of environmental impact and they can be measured in different ways. But if we focus on carbon emissions, which I will use as the cover-all term for all greenhouse gas emissions, there are a few ways. Perhaps we are the most used to seeing a total, often for our entire organisation. This is one way and it is great at informing one set of decisions. On the other hand, as a software practitioner, it does not give you the granularity, or actionability, to let you know what things you can do to reduce carbon emissions. Knowing my organisation’s carbon emission total is simply too high a level for that conversation. We might need a different metric.
Let’s back up: what makes software green in the first place? When we say green software, we mean software that limits the amount of carbon emissions that are released into the atmosphere. To achieve this, we want our software to treat carbon emissions as a valuable resource; we want our software to be carbon efficient. Carbon efficiency, using the least amount of carbon possible, is determined by three things:
- Energy efficiency: use the least amount of energy
- Hardware efficiency: use the least amount of embodied carbon possible
- Carbon aware: do more when the electricity is clean and less when it is dirty
The Green Software Foundation’s (GSF) Software Carbon Intensity (SCI) specification is sensitive to these three qualities. The SCI is a rate of emissions, not a total, and the goal is to give software practitioners a tool to take action. We want you to measure it, change some things, and then measure it again.
[Click on the image to view full-size]
Green Software Foundation. This image is licenced under Creative Commons (CC BY 4.0)
The SCI is pretty straightforward and GSF wants it to be easy to implement. It consists of the energy usage (E) multiplied with the location-based carbon intensity of the grid (I), to this the embodied carbon (M) is added and this all gets divided by (R), a functional unit of your choice which describes how your software scales.
Resulting in the formula SCI = ((E*I)+M)/R
Let’s take a look at a made up example to get our feet wet. Let’s assume we have a server exclusive for our use to run our machine learning workload. This server is serving API requests to provide personal shopping recommendations. We want to find out the SCI where our functional unit (R) is "per recommendation". We decide that our software boundary is this server only and do not include any networking equipment or client-side emissions.
We take some steps to measure the energy use (E) for our server and see that it is 0.0656 kWh for a 1-hour period. Next we consult an open data source; you can find examples of such data sources on the GSF guidance page, and given that this imaginary server is located in Ireland, we see that the carbon intensity (I) is 0.38 kg per kWh. Next up, we contact our server provider and learn that the total embodied emissions for this server is 1205.52 kg CO2eq. We will assume a 4-year, or 35,040 hours, lifespan for this server and that we have exclusive use. This gives us M = 1205.52 * (1/35040) CO2eq per hour. During this hour we observed 500 recommendations, R = 500.
Putting all these together we get:
SCI =
((E*I)+M)/R =
((0.0656*0.38)+(1205.52 * (1/35040)) / 500 kg CO2eq per recommendation =
(0.025 + 0,0344) / 500 kg CO2eq per recommendation =
0,12 gram CO2eq per recommendation
You can find a quick start guide and more in-depth guidance, as well as case study examples in the quick guide on the SCI guidance.
Making machine learning greener
There are several things we can do to mitigate the negative impact of software on our climate. They will be different depending on your specific scenario. But what they all have in common is that they should strive to be energy-efficient, hardware-efficient and carbon-aware. GSF is gathering patterns for different types of software systems; these have all been reviewed by experts and agreed on by all member organisations before being published. In this section we will cover some of the patterns for machine learning as well as some good practices which are not (yet?) patterns.
If we divide the actions after the ML life cycle, or at least a simplified version of it, we get four categories: Project Planning, Data Collection, Design and Training of ML model and finally, Deployment and Maintenance.
The project planning phase is the time to start asking the difficult questions, think about what the carbon impact of your project will be and how you plan to measure it. This is also the time to think about your SLA; overcommitting to strict latency or performance metrics that you actually don’t need can quickly become a source of emission you can avoid. Consider matching your SLA and SLO to your actual needs. If you want to include some carbon-aware features, like demand-shaping, this is a great time to start these conversations. Demand-shaping means changing the behaviour of your product depending on the carbon intensity of your grid. Think of how the video quality of a video conferencing call changes depending on your internet bandwidth. The same thinking can be applied to software. Maybe you serve lower accuracy, and less computationally intense, recommendations when the user’s grid is dirty?
Data collection is, for some reason, often seen as a less glamorous part of the ML lifecycle. However, we do know that data quality issues have a cascading effect such as reducing accuracy. As we build larger AI models, they need larger datasets to prevent overfitting and to be representative of the full data. Large datasets can also potentially be reused for other projects later, hence, they become even more attractive. Since datasets are growing, it means that green data collection is becoming increasingly important. Even so, there is surprisingly little research on how much of the carbon footprint the data collection stands for. Something that works for all workloads which are not on-demand, is time shifting and region shifting, e.g. making this part carbon aware by using sustainable regions and times. Also critically think about how much data you actually need and if there are already open source datasets which could suit your scenario. Using already gathered data means that you do not have to spend additional carbon emissions building your own data pipeline.
The next phase is the phase that most people might consider the "actual ML part", the design and training of the ML model. Here we have significantly more data on the environmental impact and mitigation approaches! Possibly because people think this is more fun and glamorous? Or possibly because results and experiments are more easily reproducible? Or perhaps because it is fun to compete in accuracy metrics? Whatever the reason, we know that training of ML models is growing in size and usage and they are growing rapidly. One aspect of this was investigated in this blog post by Open AI about AI and compute. The authors found that starting in 1959, they saw a ~2-year doubling time for the compute used in the largest AI training runs for their time. This 2-year doubling time can be compared with a 3.4-month doubling time starting in ~2012, continuing to 2020 when the blog post was written.
ML training has the great benefit of very rarely being urgent. It is important, yes, but rarely urgent. This means it is a great candidate for carbon aware actions like time or region shifting. There are several research initiatives that are exploring how to train models faster and more efficiently that rely on pruning, compression, distillation, and quantization among other techniques with the goal of shrinking down the size of the models and utilising fewer compute cycles and using less storage space. One example is from the paper Can Federated Learning save the planet?, where the authors’ findings show that FL, despite being slower to converge, can be a greener technology than data centre GPUs. Edge computing and IoT means we are seeing more and more devices with limited capabilities; smaller models will be the way to go here. Another perk of Edge computing is reducing the energy consumption by providing local computing and storage for data.
The last phase is deployment and maintenance, and here ML behaves very similarly to other software systems, so we can use tools that might already be familiar to us. This includes things like thinking about server density, turning off zombie workloads, leveraging Edge computing when possible, and using your spending power to host your workloads on a platform which is committed to sustainability.
Another thing to consider is if you can use specialised hardware like ASIC or TPU (or to some extent FPGAs) to speed up run time? Custom application-specific integrated circuits (ASICs) and field-programmable gate arrays (FPGAs) are provided or supported by cloud vendors which provide better energy efficiency and inference for AI models than conventional chips. However, if you are self-hosted and you already have general purpose hardware, like CPUs, it is likely better to optimise the use of this hardware over buying new specialised hardware, to avoid spending more embodied carbon.
What software developers can do to reduce climate impact
Hopefully this article has given you some ideas for hands-on actions that you can take to reduce your climate impact. However, you might be left with the feeling of where do I start? A good question to ask oneself then is "Where in the life cycle does my software spend the most carbon?" I’m afraid the answer to this question is, "It depends on your project". If you are working with very large models, it might very well be data collection and that is where you should focus your first efforts. Or perhaps you are trying to achieve super high accuracy, like a cutting-edge research scenario? Then very likely the training of the ML models will be your biggest culprit. If you are running ML operations in your production workloads at a large scale, most likely your deployment and maintenance carbon cost is the highest.
Another good thing to do is to learn more about these topics, or maybe inspire someone else to learn more. The Linux Foundation has a free course, which GSF has produced, where you can learn the basic concepts a software practitioner needs to know to build, maintain and run greener applications.
I also think it is good to start causing some good trouble; dare to ask the slightly uncomfortable questions and advocate for things you can to reduce your impact. Also know that there are also great resources out there, for example you can use the Green Software Foundation’s Software Carbon Intensity Specification to assess the carbon score of your product. As we have seen in this article, there is already research and techniques available to you to reduce your impact as well.
Sustainable artificial intelligence
In conclusion, I hope this article has inspired you to approach your own software emissions with curiosity. With sustainable AI we want to create change in the entire lifecycle of AI products, from idea generation, training, re-tuning, implementation, deployment and maintenance toward a smaller environmental impact. In this article, we have seen that there are tools available for measuring and reducing your impact, making use of them and paying your learnings forward to the community.