Apple Intelligence

What is Apple Intelligence?

Apple Intelligence is the platform name for a suite of generative AI capabilities that Apple is integrating across its products, including iPhone, Mac and iPad devices.

Apple announced Apple Intelligence during the opening keynote at its Worldwide Developers Conference 2024 event.

Apple has had AI technologies in its platform for many years, including natural language processing capabilities, most notably in its Siri voice assistant. However, Apple did not have the generative AI technology of tools such as OpenAI's ChatGPT. Generative AI enables users to easily summarize content and generate new content, including text and images.

As part of the Apple platform, Apple Intelligence isn't just a single app, rather it is a foundational set of generative AI capabilities that are deeply integrated across multiple applications and operating system utilities.

How Apple Intelligence provides more personalization

Apple Intelligence provides personalized assistance by drawing on the user's context across their apps and devices. The technology enables supported Apple devices to understand and generate language, create images and take actions to simplify interactions across apps.

The personalized AI powers enhanced capabilities across Apple's core apps and services. With Apple Intelligence, Siri gains more natural conversation abilities, orchestration of multiapp workflows and awareness of personal context from calendars and messages.

What are the features of Apple Intelligence?

Apple Intelligence provides a broad array of features to Apple's users across iPhone, Mac and iPad devices.

Among the core features and capabilities are the following:

• Language functionality. Apple Intelligence integrates deep natural language understanding (NLU) and generation, which enables different capabilities, such as automated text summarization across email threads and notifications. It also offers writing assistance across apps, such as providing smart reply suggestions in apps such as Mail.
• Personalized image generation. Users can generate custom images for conversations in iMessage or Mail. The Genmoji feature generates personalized emojis based on text descriptions or photos of real people. There is also a dedicated Image Playground app for image creation.
• Enhanced Siri. Apple's original AI, Siri, benefits from improved NLU and context awareness by maintaining conversations for follow-up questions and responses. Siri is able to complete in-app actions by request, such as updating or finding photos.
• Automatic audio transcription. Apple Intelligence can help transcribe and summarize audio recordings from voice notes and phone calls.
• Prioritized notifications. The context awareness that Apple Intelligence brings enables Apple's platforms to organize notifications by importance and provide summaries of stacked notifications.
• Memory movies. Apple users have been able to create custom movies, but with Apple Intelligence, the process is almost entirely automated. Users can create custom memory movies based on a prompt and then select the best photos and videos to craft a storyline.

How does Apple Intelligence work?

There are multiple technologies under the Apple Intelligence banner. Apple has developed its foundation models, developed and trained using the Apple AXLearn framework. Among the models is a 3 billion-parameter large language model that runs on Apple devices to power NLU, text generation and summarization capabilities. There are also larger models that Apple is running in the cloud to support Apple Intelligence.

Additionally, Apple is partnering with OpenAI to bring ChatGPT to Apple devices as an additional option for generative AI. Apple Intelligence has generative AI image diffusion models for image creation capabilities.

The full Apple Intelligence experience requires the A17 Pro chip -- iPhone 15 Pro and newer models -- or M-series chips in Apple Mac and iPad devices with the advanced Neural Engine.

A fundamental attribute of how it all works is the understanding of context. At the system level, Apple Intelligence has a semantic index that organizes and surfaces personal context from the user's apps and data. When a request comes in, it identifies the relevant personal context and feeds it into the right generative model to ground the AI's response in that personalized information.

A key aspect is the on-device processing architecture. Over the last several years, Apple has invested heavily in powerful silicon with its M-series chips, which are critical to providing on-device inference. This enables Apple Intelligence to tap into personal data, such as photos, messages and notes, without that private information ever leaving the device.

What is Private Cloud Compute for Apple Intelligence?

Apple has designed an architecture that processes AI operations on device as much as possible, avoiding the need to send personal data to the cloud. For cases requiring more computing power, Apple has its new Private Cloud Compute (PCC).

PCC provides specialized Apple silicon servers that process only the minimum data needed for a given request and cryptographically ensure no data can be stored or accessed improperly for user privacy and protection.

What is the security of Apple Intelligence?

Apple has pledged to provide responsible AI that is secure and private for its users.

Several elements are part of the Apple Intelligence security model, including the following:

• On-device processing. The Apple Intelligence architecture processes AI operations on device as much as possible, enabled by Apple's advanced silicon chips. By running the generative AI models locally on iPhones, iPads and Macs, personal data can be used to provide intelligent and contextually relevant experiences without ever leaving the device.
• PCC. With PCC, only the minimum data relevant to specific tasks is sent to the cloud. Data is used solely to fulfill requests, it is never stored or accessible to Apple, and cryptographic measures ensure devices only communicate with verified servers.
• Data minimization. The system collects only the minimum data necessary for each specific request whether it's for PCC or on-device processing.
• Transparency and verifiability. The Apple Intelligence software stack is designed for transparency, from on-device components to cloud systems. The stack also benefits from the use of the Swift programming language, which provides strong security properties.

How Apple Intelligence integrates ChatGPT

Apple Intelligence also has integrations with third-party generative AI tools. The initial publicly disclosed integration is with OpenAI's ChatGPT.

With the integration, ChatGPT's capabilities are accessible to users directly within Apple's existing experiences and platforms rather than using an external application.

One significant integration of ChatGPT is through enhancements to the Siri personal assistant. Siri is able to determine when a user's question may benefit from ChatGPT's expertise, such as for generating menu ideas or decorating suggestions. Siri then asks the user's permission to share their question with ChatGPT. Users can also include photos or documents with their questions.

Apple is also building ChatGPT directly into its new systemwide writing tool called Compose. When using Compose, users have the option to use ChatGPT's abilities to assist with content generation for various styles of writing, such as custom stories.

ChatGPT integration does not require account creation or data logging, and Apple has pledged that users remain in control over when and how ChatGPT is utilized.

When is Apple Intelligence available?