Deep dive into Apple Intelligence models, optional ChatGPT and privacy approach

Cupertino, California: Summarise a lengthy document or long emails in seconds, priority notifications and reduced interruptions to prevent notification clutter, smart replies for emails that come in the form of choice based questions and AI writes the replies for you, or even rewrite complete emails, improve handwriting in notes or detailed searches within Photos – just some of the new functionality that relies on artificial intelligence, which are making their way to the Apple iPhone, iPad and Mac, later this year.

Critics, over the past year in particular, hardly ever missed a chance to opine that Apple had fallen behind in the artificial intelligence (AI) race. On the face of it, they had a point, since Google and OpenAI had, built AI tools such as chatbots and circle to search, which benefited Android phones as well as Microsoft’s Windows platform. The annual Apple Worldwide Developer Conference (WWDC) gives us a clear illustration that the opposite is true. Apple hadn’t fallen behind in the AI race. In fact, what it was building for iOS, iPadOS and macOS, now gives it the sort of advantage that Android phones, Windows PCs and standalone generative AI tools, don’t have.

It is, as Apple CEO Tim Cook describes, a “personal intelligence system”. He hopes Apple Intelligence and the company’s approach to AI, will “set a new standard for privacy and AI”. Core to Apple Intelligence are multiple models that they’ve built in-house, on-device processing, a smarter Siri, the privacy focused Private Cloud Compute for server-side processing, updates for App Intents framework that’ll allow third-party apps to plug into the intelligence features and an optional OpenAI ChatGPT-4o integration.

This extent of detailing to the proposition for integrating AI features, is a conversation no AI company, phone maker or software maker have done in such detail. One element of this is Private Cloud Compute, which is to be used by an iPhone, iPad or Mac for tasks that cannot be done on-device. These servers run the same kind of Apple Silicon hardware as the devices that users have to verify data privacy.

“What’s happening there is your request to our servers is actually first of all anonymized, so your IP address is masked. Then it talks to a server that has no permanent storage, cannot log, but most importantly is running software where the image is publicized for security researchers to audit or your iPhone won’t even talk to that server is a clever kind of blockchain issue,” details Craig Federighi, Senior Vice President for Software Engineering at Apple, in a briefing session of which HT was a part.

The security audit for the Private Cloud Compute servers means that an iPhone, for example, will only trust the software that’s been publicly put out there. If it hasn’t been audited and verified, the connection will not complete. Even the smallest of updates to the server software or configurations, must be audited.

The clarity of thought with how Apple Intelligence has been structured, is clear. “We think that the right approach to this is to have a series of different models and different sizes, for different use cases. And of course, we put a lot of time and effort on the 3 billion parameter model that will run on your iPhone, and it is one of the most capable models today,” explains John Giannandrea, who is Senior Vice President for Machine Learning and AI strategy at Apple.

This isn’t just one. Giannandrea details that there are a number of models that have been developed and are at work, including the ones on Private Cloud Compute. “We built a series of models, and we don’t think users care about model size. We think they care about the functionality, having enough performance and enough quality that it’s useful every day,” he says. For simplification, Apple simply refers to the models presently as ‘Apple on-device’ and ‘Apple server’.

Where ChatGPT fits in Apple’s AI approach

Apple is integrating OpenAI’s ChatGPT-4o into iOS 18, iPadOS 18, and macOS Sequoia, but this isn’t the default AI model at work. In fact, reaching ChatGPT will be optional for any queries or tasks a user may have, and this will be invoked if Apple’s own models feel they do not have the necessary expertise or response to offer. “We’re excited to partner with Apple to bring ChatGPT to their users in a new way. Apple shares our commitment to safety and innovation, and this partnership aligns with OpenAI’s mission to make advanced AI accessible to everyone,” says Sam Altman, CEO of OpenAI.

The integration however is seamless in Siri and within the operating systems, including variable functionality that OpenAI may allow between free and paid subscription tiers. Users, along with any query being sent to ChatGPT, will also be able to share any additional information such as an image or a document which would help with a detailed and relevant response.

“We wanted to start with the best and the new GPT-4o model represents the best choice for our users today. We wanted to make sure that users can tap into that model without having to leave where they were to go to a different tool and break the flow, but instead take advantage of the kind of integration that we built for Apple Intelligence,” says Federighi.

Apple also confirmed this is simply the beginning of the journey, and users can expect more AI models to find integration and optional invoke capabilities within iOS, iPadOS and macOS in the future. Federighi specifically mentioned Google Gemini but didn’t elaborate any specific plans or timelines. By all estimates, that specific integration, is unlikely to happen till iOS 18, iPadOS 18 and macOS Sequoia roll out for consumers later this year.

But what is the need for more models to plug into Apple Intelligence, when there are multiple models at work already? The reason, Federighi and Giannandrea explain, is the concern about expertise.

“There are lots of times when Siri has definitive knowledge on a particular question you may ask, and it’s great because that’s from a database that has references. We know it’s accurate information with which Siri will answer, but the times you might end up taking a picture of a dragon fruit, asking what to I make out of this, and you’ll end up with a bizarre meal. In that case, drawing on another model can be really exciting,” they say.

OpenAI has also detailed the privacy measures put in place for queries generated from Apple devices. They say privacy protections are built in when accessing ChatGPT within Siri and Writing Tools, to the extent that the user requests are not stored by OpenAI, and users’ IP addresses will be obscured. Additionally, users with a ChatGPT account can sign in to have their selected data preferences apply under ChatGPT’s policies.

Apple Intelligence (AI), their take on the broader AI conversation, rolls out later this year for iPhone, iPad and Mac users with iOS 18, iPadOS 18 and macOS Sequoia. That is, unless the public beta versions are something you have an appetite for – those release next month.