Finally, The Promised New Siri
Far and away, the biggest headlining feature of the new iOS/iPadOS/macOS 27 announcement was the new Siri AI powered by Apple Intelligence. While I knew deep down to not expect this new version of Siri to come anywhere close to the capabilities of AI tools offered by Anthropic, OpenAI, and Google, I was still excited about this announcement. I use Siri all the time to do simple things on my phone, from setting timers, making calendar events, adding tasks to my to-do list, and sending the occasional text. Siri could handle these tasks fine in a pinch, but it was far from perfect. A Siri that was more capable of taking actions on my behalf in more apps with cross-app awareness sounded amazing. More than anything else, the idea of doing the tasks I just listed with better dictation abilities was enough to sign me up day 1.
As excited as I was for a better Siri, an extremely large asterisk loomed in my head as I watched the keynote for the new iOS/macOS/iPadOS versions being announced. These new flashy features are powered by AI, and AI equals significant privacy implications. The thought of having an AI on my phone harvesting every byte of my personal data never sat right with me, regardless of the company making and hosting the AI. I know this is necessary for making an AI assistant “contextually aware”, but control over my data is important to me. For me to be completely on board with using an AI with full access to my device, I would need to have full, fine-grained control of what that AI can see and do with the data on that device.
What quelled my tinfoil hat fears of Apple taking all of my data and training an AI on it was their commitment to Private Cloud Compute announced in 2024 and the use of an on-device LLM.
Private Cloud Compute (PCC), from how I understand the concept, basically work as specialized, locked-down servers dedicated to running Apple’s AI. When a user makes makes a request to Siri–asking a question or performing some action–Siri will take the request, encrypt it on the device, and send that information over to the PCC server. When the PCC server gets the request from Siri, it will decrypt the message, run the LLM, generate the necessary text or actions, re-encrypt the response and send it back to Siri on the user’s device. Apple then deletes the data PCC used to process the users request the moment it is no longer needed. For the sake of user privacy, Apple is committing that the only time user data is available to be read on Apple’s end is by the PCC node at runtime. Apple says they have but sufficient safeguards in place so that Apple and hackers would find it almost impossible to get at this data even if they wanted to. While this blog post is just a promise and not personally verifiable schematics of the system, Apple has an excellent track record on respecting its user’s data privacy.
The more interesting part of Apple’s privacy commitment was Siri AI’s use of an on-device LLM. While PCC’s privacy capabilities only go so far as Apple’s commitment to uphold its guarantees, a local LLM is truly privacy respecting as no data needs to leave the device in order to be processed. Macs in particular have exploded in popularity over the past few years for their local AI capabilities because of the power of Apple Silicon. If Apple can replicate the capabilities of the popular open source models on a device that fits in your pocket, that has the potential to be a gamechanger for security and privacy conscious people like myself.
Both PCC and an on-device LLM was enough for me to be a believer in the new Siri AI, and I excitedly waited for the various OS betas to wrap up for me to try it out.
Testing Siri AI
After a few days of iPadOS being released (I waited in case any major bugs or security issues were caught in the first days of its release), I downloaded the update and immediately started testing Siri AI out. When you first update to iPadOS 27, you don’t immediately get the new Siri right away. You start off with the old Siri, and you have to opt into the Siri Beta and join a waitlist. This waitlist last 10 minutes for me, but that time could take more or less time depending on when in the beta you opt in. After being approved, you get the new Siri app, and you are all set to start using Siri AI.
I started out my testing of the new Siri by asking it some basic questions about itself, like “Hello, how are you?” and “Give me a list of your new capabilities”. I was pretty impressed off the bat - the responses came back to me quick, were short and to the point, no annoying follow-up questions, and links to the information sources were provided below each response section. That last point pretty quickly piqued my interest; if Siri is able to provide links to web resources, then the request I was making was going to Apple’s PCC servers. While this made sense for some of my requests that required fact checking information from the web, some of my simpler requests appeared to also be computed over the web.
“That’s odd, I thought Apple promised an on device LLM to handle simple tasks. Surely, a top of the line iPad Pro with the M1 chip in it could handle that.”
If you go to the Apple Intelligence Page on the Apple’s website and scroll to the bottom you see that an iPad Pro M1 and later is compatible with Apple Intelligence.
“Then surely, some of my requests should be using the on-device LLM and not Private Cloud Compute. What do I need to do to use it?”
I tested Siri some more, asking as simple of questions as I could. From what I could tell based on personal experience running local LLMs on a Mac, the responses I got from Siri on my iPad came to me too fast. No matter what I asked, I got PCC responses. Frustrated, I turned to Google searches to get a better understanding of how Apple Intelligence works. In the fine print at the very bottom of this Siri AI press material, you can find the disappointing information about “Apple’s most powerful on-device model”:
Apple’s most powerful on-device model and the features it enables, like expressive voices and more advanced dictation, are available on iPhone Air, iPhone 17 Pro, iPhone 17 Pro Max, iPad (M4) or later with at least 12GB of unified memory, and Mac (M3) or later with at least 12GB of unified memory. The model and expressive voices are also available on Apple Vision Pro (M5).
That blows. My excitement a local AI on my Apple devices dashed away. With the price increases of all of Apple’s products, I don’t see myself upgrading any time soon.
What is particularly frustrating is Apple’s lack of transparency with this feature. Apple advertises Siri AI the same for all of their compatible products without clearly specifying how the technology works from product to product. I get that there is most likely a technology limitation in the older hardware that makes a locally hosted LLM not work. But I wish this information was not relegated to a gray text asterisk at the bottom of a very long press release. And come on, if my M1 MacBook can run a tiny LLM, I feel my iPad with the same processor can do it.
Testing Siri AI Offline
With hopes and dreams of an on-device LLM out of the picture, I turned back to understanding Private Cloud Compute in greater detail. Siri AI may still be in beta, but all up-to-date devices that are compatible with the iOS/macOS/iPadOS 27 will eventually get the new Siri. Specifically, I was interested in the information that Siri is aware of, and the information that leaves the device to be acted on by PCC. I wanted to know about its features and limitations, what capabilities stay the same, what improve, and what is new. In just 20 minutes of testing I found myself going down a concerning rabbit hole.
In a previous attempt to get Siri to use its on device model, I instinctually disabled the Wi-Fi on the device to prevent Siri from calling out to PCC. Asking Siri to generate me a short story offline would return me an error stating, “I’m having trouble with the connection. Please try again later”. This makes sense, with no on-device model and no ability to connect to the internet, there isn’t anything for Siri to do. Thinking I would give Siri an easy task to try next, I asked it to add a chore to my to-do list. Again, “I’m having trouble with the connection. Please try again later”. That’s not right, Siri doesn’t need the internet to add an item to the Reminders app. I tried asking Siri to add an event to my calendar. Same response, “I’m having trouble with the connection. Please try again later”. To my shock, I had discovered that Siri AI no longer works without an internet connection.
(Except for starting a timer and responding to the words “Hello”. From my brief testing, these two commands were the only ones that would not return a connection error message. If I had to guess, this is because there is no iCloud integration with timer, and responses to “Hello” are hard coded into Siri.)
I don’t own modern enough Apple devices to test if the on-device model allows Siri to work offline, but I would assume this is the case. I couldn’t find any posts at the time of writing this blog from someone who has tested this. That being said, it doesn’t change the reality for people like me with older Apple products: if you use Siri in your day-to-day life, you will soon be required to send your Siri messages directly to Apple’s Private Cloud Compute servers. This is not good as it effectively disables all offline Siri capabilities. If you use Siri to write personal notes, send text messages, set reminders, or make calendar events, you are—or will soon be—having Apple’s AI servers parse all of Siri requests, regardless of whether it needs to or not. If you happen to be in an area where you cannot connect to the internet, tough luck: no Siri for you.
An Always Online Siri
Now I can hear the devil’s advocate voice in the back of my head shouting “But wait! Why does any of this matter if Apple keeps all of my notes, calendar events, reminders, etc. in iCloud? Couldn’t Apple just view the responses Siri generates because everything is stored in the cloud?”. I have no clue if you, the reader, were thinking this or not, but I do feel it’s a good point to bring up. Apple has a very interesting read on their support page regarding their ability to access your data in iCloud. Apple offers two security and data encryption tiers: Standard data protection and Advanced Data Protection for iCloud (Apple’s naming scheme for their products and services drive me insane sometimes). Using the chart near the bottom of the page, we can see that many of Apple’s apps and services are not end-to-end encrypted unless you have the Advanced Data Protection turned on. This extra security feature is turned off by default for data recovery reasons, but it may be a good idea to turn this on if you feel confident you won’t lose the recovery keys. Without this feature enabled, Apple does in fact have the keys to your kingdom and could view all of your iCloud data if they wanted to. Again, Apple has no reason to view an individual’s data, but the mechanism to do so is readily available; all you can really do is trust that they don’t ever go back on their word.
Let’s say that you are okay with your Siri conversations going to PCC and that you never take your device off the internet. “I do not ask Siri anything personal, private, or interesting to even warrant Apple cracking into my chats and viewing my messages”, you may be thinking to yourself. I would strongly reject this way of thinking. Even the mundane things you do in your day-to-day life real a shocking amount of details about who you are and what you are doing. While I could get into the weeds about OpSec and why it is important to everyone regardless of status, what I discovered next in my investigation sealed the fate of this Siri AI completely.
Apple Intelligence Report
While I was performing my initial testing with the new Siri AI, I asked it a few questions about Private Cloud Compute: how it determines when to compute a message on device and when to send the chat to the cloud. I typically do this with all AI I am testing just to see how much self awareness the AI has about itself. While most of the conversation was very boilerplate and generic information I can find on Apple’s website and in their press materials, Siri AI mentioned a single fact that piqued my interest. It mentioned that deep in the Settings app, there is a feature to export your devices Apple Intelligence activity. If you navigate Privacy & Security tab in the Settings app, you will find a section at the bottom of the list of options call Transparency Logs. This holds a setting called Apple Intelligence Report. The Apple Intelligence Report description reads:
This report is stored on-device and may include personal data such as messages and text you enter in intelligent writing tools.
Besides the button to Export Activity, there is one selector field, allowing you to select the duration in which you want to the report to encompass. The report is set to Off be default, but this can be changed to 15 minutes or 7 days. Pressing the Export activity button allows you to save a large text file to your Files app.
The contents of this text file were shocking. This export was hundreds of thousands of words long, and it contained everything from the system prompt for Siri AI, the tool calls it is capable of, the personality it should have, and what looks to be almost every text, email, phone call, calendar event, reminder, note I had ever written.
This is crazy - asking a couple of prompts about “What is Private Cloud Compute” and “Set a timer for 1 minute” results in a file containing everything I have personally sent, received, and done on my device is compiled into a text file and sent to Apple’s PCC servers without me knowing or asking permission. The security and privacy implications of this feature is significant. Traveling over the wire every time Siri is called is a transcript of everything that you have used your phone to do over the past however many years you have been an Apple customer. And this information is neatly stored in a plaintext readable text file which can be called by anyone with user access to the device. While I need to give Apple credit for making this information so transparent and easy to access, it is wild that the private text messages I send to my girlfriend are needed by Siri to now “add eggs to my shopping list”.
(I will be making a follow-up blog post deep diving into the contents of this export file, including what information was included and missing, any patterns I discovered in the data, Siri AI’s system prompt, and what appears to be a large amount of encoded data at the bottom. Check back later for that post and more.)
The data being used by Siri and sent to PCC, while shocking, is what Apple advertised in their keynote announcement and press materials.
[…] Siri AI can help users find what they need in the moment, from answering questions from the web on virtually any topic, to surfacing relevant information from a user’s personal messages, emails, photos, and more.
Apple advertises Siri as being context aware of your personal messages, emails, photos, and more, so it’s expected that the prompt to the PCC servers would have all of this information bundled up in one place. In a way, it is a good thing that Apple built the system this way, as this information affirms Apple’s claims that PCC does not store user information and conversations are deleted immediately after processing. The only way Apple’s LLMs could have awareness of your messages, emails, photos, and more would either be PCC has built a profile for you based off of your data in the cloud, or it is delivered this information by your device during off-device compute. With that being said, I still dislike this system and what this means in the industry going forward. Siri, without an on-device LLM, is essentially useless without sending all of your device data to the cloud. All of my data is directly coupled with my ability to verbally ask my phone to take actions on my behalf. I can’t ask Siri to tell me the weather without also seeing my sensitive emails. I can’t make a calendar event without sending my private text messages. I can’t remind myself to do a chore without sending my family photos. This will become the new normal for all Apple device users over the coming months as Siri AI goes from beta to full release and I hate it.
Conclusion
I believe that one’s opinion on this issue will ultimately boil down to one’s trust in Apple. While Apple’s track record with data privacy has a good track record, recent product announcements and rumors suggest Apple is walking on a tightrope. The new AirPods Pro’s are rumored to have cameras in them which record what you look at, similar to Meta’s AI Glasses. The new Apple Watch Series 12 and Ultra 4 ship with a microphone and CPU designed to be always listening and transcribing what it hears. I feel Apple is walking on a very, very fine privacy line here, and entrusting this much data to a company based on goodwill may be playing with fire.