“A lot of people want local inference. And in some ways, they assume the problem is solved now because we’re going to have open source models, we’re going to be able to run them on our computers, so we’re not going to need all the stuff in the data center. But from a global allocation perspective, actually, it’s pretty terrible that everyone has a bunch of the world’s limited DRAM supply on their own computer and using it five percent of the time. What you want is maximum utilization in the cloud if you’re trying to deal with “This environment of very limited supply and in some ways the intuition of everyone, of many people, is the opposite of that.”
