E11: Data centers and the race to power AI (part 2)

Madison Goldberg: Welcome to Ask MIT Climate. I’m Madison Goldberg, and this is the second half of our deep dive into data centers and the energy system. If you haven’t listened to part one, by the way, we’d suggest catching up on that first; it’ll help this episode make more sense.

But as a quick recap: last time, we talked about the rush to build data centers amid the AI boom. And more specifically: how the rapid buildout of these huge, concentrated electricity users can create challenges for people, and for a U.S. power grid that…might not be prepared for this. 

Costa Samaras: The power sector is under threat from extreme impacts from climate change. It's under threat from new demands that are appearing faster than the grid is used to. And it's under threat from its own age.

MG: Costa Samaras is a professor at Carnegie Mellon University, where he directs the Scott Institute for Energy Innovation. In our last episode, he asked this question.

CS: How do we add lots of new electricity demand in ways that are good for communities and good for the country and good for the climate?

MG: Because, ultimately, he says, this task—growing the grid while also making the whole system cleaner, cheaper, and more reliable—matters regardless of what happens with data centers. It’s also what we’re going to need to do to tackle climate change—as we switch from gasoline cars to electric vehicles, and from natural gas furnaces to heat pumps.

CS: Data centers are only part of the new electricity demand that we're going to see.

MG: Of course, electricity use and climate change are not the only concerns people have when it comes to data centers. And before we dig in, we wanted to acknowledge that communities all over the country are grappling with lots of decisions in this moment, including whether to allow data centers at all.

For this episode, we spoke with our guests about managing data centers’ electricity demand with the climate and our energy system in mind. So when it comes to data centers that do get built: We hope this episode helps you think about the policies you’d want to see in place for their energy use—to give ourselves a shot at emerging from this moment more prepared for the cleaner cars and buildings we’ll need, and not less.

Okay, let’s get into it. There are way more ideas on the table than we have time for, but we’ll go over some of the main ones our guests are thinking about. Let’s start by sorting them into a couple of buckets. Here’s the first:

CS: Get more out of the existing grid immediately.

MG: These are ways to bring more demand onto the grid—whether it’s data centers or electric vehicles or whatever—without stressing the system or triggering costly upgrades.

One option is good old energy efficiency. If I switch out incandescent bulbs for LEDs, I can make my room just as bright with much less electricity. Sure, we need to think way bigger when it comes to data centers, but we can push for efficiency from them, too.

Vijay Gadepally: And so, whether it's better cooling technology for the data center, more efficient AI algorithms, more efficient computing technologies. I think those are all coming.

MG: That’s Vijay Gadepally. He’s a senior scientist at MIT Lincoln Laboratory and the co-founder of two AI infrastructure companies.

Now, the industry has already seen big efficiency improvements over time. But Dr. Gadepally points out a caveat here: As it gets less energy-intensive—and cheaper—to use AI models, that could have its own knock-on effects.

VG: When something becomes cheaper, there is often a rebound effect where the net usage actually goes up. For example, what if we come up with a brand-new model that's inherently far more efficient, that requires a lot less compute? I still think that even if that happens, our demands are insatiable, and there's a fairly good chance that we'll just use it more, and we'll start to use it in things that we weren't necessarily using it in before.

MG: There’s another set of ideas under this “get-more-from-the-grid” umbrella. One of Dr. Gadepally's companies aims to make data centers’ power use more responsive to real-world conditions—offering what's called “load flexibility.” For example, consider what happens when electricity demand is at its highest—like when lots of people are cranking air conditioners on a summer afternoon. If a data center adds to those peaks, that could be a recipe for grid stress and extra costs.

But what if you could shape a data center’s electricity demand so it doesn’t pile on when the grid is already stressed?

VG: So, flexibility can mean a lot of different things depending on who you're asking. Flexibility could mean, hey, we might turn off a few servers for some time. It could be order of hours. It could be that, hey, we're going to, you know, reduce the amount of voltage that's going to the processors, which means that it consumes less power for a small period of time. It could mean that I'm just going to delay some results for people, right? If you told me that you're submitting a ChatGPT query, and if you tell me, ‘I'm not in a rush to get the answer, I'm heading home for the evening, as long as I get it by morning, I'm fine’—that could be flexibility.

MG: The idea here is that data centers may be able to shift some of their work around to avoid periods of grid stress. Workloads that aren’t super time-sensitive, like model training, could be scheduled for when there’s plenty of spare capacity.

Or, instead of shifting in time, you could shift in space, passing off some work to another data center somewhere else in the country, where the grid isn’t seeing peak demand.

VG: And it could mean also, like, hey, I'm going to run my processors a couple of degrees warmer during this time. I'm not going to set my air conditioning to 21 degrees centigrade, but I'm going to go to 25. Flexibility could also mean that I'm going to have batteries in place so that there is supplemental power during certain periods of the day that I utilize.

MG: So far we’ve been talking about avoiding grid stress, but you could also think about other puzzles—like tailoring workloads to cut down on climate pollution.

For example, you could have flexibility in which AI model you use when: like, opting for a less accurate but more energy-efficient model when the grid is running mostly on fossil fuels, and vice-versa when there’s lots of solar and wind. Dr. Gadepally and his colleagues built a system that looks at the grid and adapts—including by shifting which models it relies on—to minimize pollution without sacrificing too much accuracy or speed.

VG: We were able to reduce carbon emissions by 80 percent with just a reduction in accuracy of about three percent. So that is just because we timed it very well.

MG: Flexibility has also been proposed as a way of supporting other uses of electricity: for instance, by shifting around the timing of when EVs charge. And maybe, Professor Samaras says, we can get all these energy consumers to work together.

CS: We have an opportunity for broader flexibility in the grid outside the data center. How do we leverage flexibility in when people are charging their vehicles, or when different electrification loads in houses, like hot water heaters, turn on and off. And utilities could, with data center companies, pay households to have flexibility built into their systems, so that the grid can operate more reliably and that customers can benefit from a lower rate, and that we don't add more polluting power plants when grids are stressed, and we avoid blackouts.

MG: So we’ve gone over a couple of ideas for how to squeeze more out of the grid we have. But there’s another bucket of approaches that, for Professor Samaras, are really front of mind.

CS: At the same time—this is not to be put off until later, this is start tomorrow—is building more clean power plants, adding lots of new solar and lots of new wind, lots of new energy storage, but also deploying advanced geothermal, deploying advanced nuclear plants, and reinvesting in the clean power sources that can provide electricity to communities without carbon emissions and without air pollutant emissions.

MG: Data centers haven’t been the only cause of the country’s rising electricity demand over the last few years…and they definitely won’t be in the future, either. Professor Samaras argues that if we don’t think carefully about how to power this data center boom, it could be a big setback to building the bigger, cleaner grid we’ll need to address climate change.

CS: Infrastructure lasts a long time. So if we go through a building spree right now of lots of new, inefficient fossil power plants, maybe spread out at data center campuses, without a lot of new clean energy, without a lot of new energy storage, without a lot of new transmission lines—we are at risk of having basically a two-tiered electricity system going forward, where the data center companies are running their own systems to power their own operations, and everybody else is relying on an aging commercial grid.

And so that's why, in this moment, it's especially important that we're not just building for next year, we're building for the next several decades. Companies have large financial incentives to build out power systems and build them out rapidly. We as a society have the opportunity to help guide what that looks like in ways that are good in the near term and also good in the long term.

MG: Lately, Professor Samaras has been thinking a lot about how to do that.

CS: There's a lot of talk about bringing your own clean power for data centers. The way I think about it is more like a potluck picnic. It's “bring your own clean power and enough to share with the community,” because we're not only building out the grid for data centers, we're building out the grid for electrification.

Then we have to think about the boring part of the power system. Power lines. Transformers. Substations. That equipment is old and that equipment is at risk of extreme weather amplified by climate change. And those costs roll on to households and electric bills. So there's an opportunity now that states and firms could say, we're going to reinvest in this grid infrastructure in ways that doesn't make the household customer pay for all of it.

MG: On that front, lawmakers and regulators are thinking about how to strengthen protections so that other customers are shielded from the costs of bringing data centers onto the grid. For instance: requiring data centers to pay for big power lines and other grid infrastructure that’s built to accommodate them.

And an important question in all this is: What if a bunch of the projected demand from data centers… just doesn’t pan out? If a utility invests in expensive infrastructure to connect a data center that never gets built, or that shuts down, households could be left footing the bill. Financial measures, like early-exit fees and collateral, could help address that.

So communities are setting ground rules to say: If a data center company wants to build here, these things are non-negotiable.

CS: The public needs more assurance and information that their representatives and their regulators have got this under control. And I think more and more states are trending towards making this development conditional on being good neighbors.

MG: Professor Samaras has also thought about policies that incorporate incentives. After all, these developers have shown they’re willing to pay up to get online fast.

CS: And so that can be done with fast-track permitting for best practices in bringing their own clean power with enough to share, rebuilding the transmission and distribution systems so that those costs don't fall on the ratepayers, adding lots of new flexibility, building out additional efficiency for communities.

MG: Making all these choices will also be easier if we have good information about how AI companies use energy. And Dr. Gadepally says there are still big gaps there.

VG: I mean, when you buy a car, they give you so much information about the emissions, you know, where it came from, what's the mileage, all of that. Most of these big commercial models that are out there, I do not know what power they're using, what the average carbon emissions are. And so we just try to push very hard on the community to say, we need to have better visibility, better transparency. This has to be something we demand.

MG: Even after all that, there are whole categories of options being explored that we haven’t touched on: A bunch of states are rethinking their tax breaks for data centers. New York’s governor announced a statewide moratorium on large data centers, to give regulators time to create energy and environmental standards. There are efforts focused on transparency, and on getting data centers to bring meaningful benefits—like jobs and investment—to the communities where they operate.

But hopefully a key takeaway from today’s show is that, in this moment, we can shape the kind of future we want to see. And as we evaluate our options, we should consider the big picture: what our choices mean for people; for an energy system that faces challenges related and unrelated to data centers; and for the climate.

CS: A clean power grid that is affordable and reliable is table stakes for dealing with climate change. There's a bunch of lights blinking red right now. Residential electricity prices are rising faster than inflation in many parts of the country. Power sector emissions are not dropping as they were previously. And impacts from climate change are affecting the infrastructure of the grid in ways that are costing households money.

Now is an opportunity to have shared investment between companies and states and the federal government and even climate philanthropies. There are risks that things go wrong. There's risk that the benefits don't materialize. Absolutely. And that's why we need strong governance and we need capable public utility commissions. And we need an engaged public to make sure that projects are maximizing the benefits and minimizing the risks. We, society, should not be spectators in this moment as the grid is being built around us.

MG: Ask MIT Climate is the climate change podcast of the Massachusetts Institute of Technology. Aaron Krol is our executive producer. David Lishansky is our sound editor and producer. Michelle Harris fact-checks our episodes, and the music is by Blue Dot Sessions. And I’m your host and associate producer, Madison Goldberg. I also wrote this episode.

A huge thank you to Dr. Vijay Gadepally and Professor Costa Samaras for digging into this topic with us. You can find more episodes of the show at climate.mit.edu, and we’re on TikTok, Instagram, and Youtube @askmitclimate. And last but not least: If there’s something about climate change that’s just not computing, send us your questions at askmitclimate@mit.edu.