John Giannandrea, Apple’s AI chief, speaking at John Gruber’s The Talk Show Live from WWDC 2024:

The inference of large language models is incredibly computationally expensive. So it’s a combination of bandwidth in the device, it’s the size of the ANE [Apple Neural Engine], it’s the oomph in the device to actually do these models fast enough to be useful.

You could in theory run these models on a very old device. but it would be so slow it would not be useful.

Greg Joswiak, Apple’s marketing boss, also pushed back on the idea that Apple Intelligence is a scheme to sell new iPhones. “Otherwise, we would have been smart enough just to do recent iPads and Macs, too.”