When you send a message to ChatGPT, that conversation travels across the internet to OpenAI’s servers, gets processed, and is stored — potentially used to train the next version of the model. Most people assume their AI conversations are private. They are not. Understanding what actually happens to that data is the first step toward making an informed choice.
What AI Companies Actually Collect
Every mainstream AI assistant — ChatGPT, Gemini, Claude, Copilot — operates on a cloud infrastructure model. That means every message you type is transmitted over the internet, received by a corporate server, and processed there. The response travels back to your device. Your message never stayed local.
What gets collected in that process varies by provider, but the baseline is consistent: your input, the AI’s output, timestamps, your account details (where an account exists), device information, and IP address. According to OpenAI’s privacy policy, this includes “content you provide us” — which is, plainly, your conversations.
OpenAI retains conversation data for 30 days by default even for users who opt out of training, and longer for logged-in users with chat history enabled. That’s a 30-day window in which your messages exist on a server you don’t control.
Google’s approach with Gemini is similar. Google states that Gemini Apps conversations are reviewed by human reviewers and may be stored for up to three years. Three years of conversations, including anything sensitive, personal, or professionally confidential you’ve ever asked about.
The Training Data Problem
Cloud AI companies improve their models by training on user interactions. This is how large language models get better — they learn from the vast corpus of human conversation. The problem is that your conversations are part of that corpus, whether you consciously consented to it or not.
Most providers offer an opt-out. OpenAI lets you disable “Improve the model for everyone” in settings. Google has a similar toggle for Gemini. But opt-out is not opt-in. The default state — the one most users never change — is data collection for training purposes.
Research from the Electronic Frontier Foundation and other privacy organizations has documented the gap between what users expect and what terms of service actually permit. A 2023 study on AI privacy practices found that fewer than 5% of users read AI provider privacy policies before use. The policies themselves average over 5,000 words.
Even when users do opt out of training, the underlying data transmission problem remains. Your message still travels to a server. It is still processed. It is still retained for some period. The opt-out removes one use case for that data — it does not remove the data.
What “Anonymous” Really Means
AI companies frequently describe their data practices as involving “anonymized” or “de-identified” data. This language sounds reassuring. The technical reality is more complicated.
Anonymization typically means stripping obvious identifiers: your name, email, account number. What it does not remove is the content of your conversation — the specific phrasing you use, the topics you ask about, the context you provide about your life, work, or health.
Re-identification research has repeatedly shown that sufficiently detailed text is nearly impossible to truly anonymize. A landmark study published in Nature demonstrated that 87% of Americans can be uniquely identified using only three data points: zip code, birth date, and gender. Conversations with AI assistants typically contain far more than three data points worth of identifying information.
If you tell ChatGPT you’re a nurse in a mid-sized city dealing with a specific professional dilemma, you have provided enough context to narrow identification significantly — even with your name removed. When you add your conversational style, the specific questions you ask, and the details you share across multiple sessions, “anonymous” becomes a much weaker guarantee.
Policy Promises vs. Architectural Guarantees
There is a fundamental difference between a policy promise and an architectural guarantee. This distinction matters more than almost anything else when evaluating AI privacy.
A policy promise is a company’s commitment to behave in a certain way. “We will not sell your data.” “We use industry-standard encryption.” “We will delete your data within 30 days of a deletion request.” These statements may be entirely sincere. They may also change. Companies get acquired. Terms of service get updated. Governments issue legal demands. Employees make mistakes. Policies are enforced by people, and people are fallible.
An architectural guarantee is different. It means the data collection cannot happen because the architecture makes it impossible. If your AI model runs entirely on your device and makes no network requests, your conversations physically cannot reach a server. There is no policy to trust, no promise to evaluate, no terms of service to read. The data does not exist outside your device.
According to a Pew Research survey, 79% of Americans are concerned or very concerned about how companies use the data they collect. The problem is that concern does not translate into protection unless the architecture supports it. No matter how careful a user is, a cloud-based AI assistant is structurally incapable of providing privacy at the architectural level.
Who Can Access Your AI Conversations
Even if you trust OpenAI, Google, or Anthropic to honor their privacy policies, trust is not the only variable. Your data on their servers is subject to legal demands from governments and courts.
All three major AI providers — OpenAI, Google, and Anthropic — are US-based companies. They are subject to US law, including subpoenas, court orders, and the broad surveillance authorities established under the Foreign Intelligence Surveillance Act and related statutes. When a government entity with proper legal authority requests data, companies comply. They are legally required to.
OpenAI’s transparency report, published annually, documents the number of government requests it receives. The numbers are not zero. In a six-month period, AI companies collectively receive thousands of legal requests for user data. Under most of these frameworks, companies are prohibited from notifying users that their data has been disclosed.
Beyond government access, there’s the question of third-party vendors. Large AI companies use external services for infrastructure, logging, customer support, and security monitoring. Each of these relationships extends the chain of entities that may have access to data fragments. A privacy policy covering the primary provider does not necessarily cover every subprocessor.
The data you put into a cloud AI system does not stay contained within a single trusted relationship. It distributes outward — legally, operationally, and technically.
How On-Device AI Changes the Equation
On-device AI inference means the model itself runs on your hardware. Your message is processed by a neural network stored on your phone — not transmitted to a server, not processed in a data center, not stored on infrastructure owned by someone else.
This is not a new privacy policy. It is a different architecture entirely.
Modern smartphones have the computational hardware to run capable language models. Apple’s A-series and M-series chips include a Neural Processing Unit — dedicated silicon for the matrix operations that power AI inference. The MLX framework from Apple enables quantized models to run at practical speeds on this hardware. A 3 billion parameter model can run locally on an iPhone 15 Pro with response times under a second per token.
The models used in on-device apps are open-source — Llama from Meta, Gemma from Google, Phi from Microsoft, Mistral from Mistral AI. These are the same underlying model families that power many cloud AI services, run locally instead of in a data center. The output quality has converged significantly with cloud models for most everyday use cases.
When inference runs on-device, the privacy guarantees are not policy-dependent. There is no server. There is no transmission. There is no company that could hand over your conversations because no company possesses your conversations. This is “we can’t, not we won’t” — a technical constraint rather than a corporate promise.
What This Means in Practice
The conversation topics most likely to be sensitive are exactly the ones people most commonly bring to AI assistants. Health questions. Relationship struggles. Financial worries. Professional challenges. Legal situations. Career decisions. These are the conversations that benefit most from an intelligent, non-judgmental interlocutor — and they are the conversations where data exposure carries the highest real-world risk.
A medical professional asking an AI for help drafting a patient communication has a professional obligation to patient privacy. A lawyer researching case precedents has attorney-client privilege obligations. An executive discussing a business strategy has fiduciary responsibilities. These users may not realize that their AI assistant is transmitting that conversation to a corporate server.
For everyday users, the risks are less acute but still real. Conversation history that reveals mental health struggles, relationship problems, or financial difficulties represents a data profile that could be exploited — by data brokers who aggregate profiles, by future employers who request account data, or by adversaries in legal proceedings.
The calculus is simple: if the data never leaves your device, none of these risks apply. The only question is whether the trade-off in model capability is acceptable for your use case. For most conversations, it is.
FAQ
Does ChatGPT store my conversations?
Yes. By default, OpenAI stores your conversations and uses them to improve its models. You can opt out of training data use in settings, but your data is still transmitted to OpenAI’s servers and retained for a period of time. Opting out of training does not mean opting out of storage or transmission.
Is there a truly private AI chat app?
Yes — apps that run the AI model entirely on your device, like Cloaked. Because inference happens locally using your phone’s chip, your messages never leave your device. There are no servers to store data, no accounts to link it to, and no company that could hand it over even under a subpoena.
What does “anonymous” data mean for AI companies?
When AI companies say your data is anonymized, they mean identifying information has been removed. But research consistently shows that conversation content — especially detailed or sensitive topics — can be re-identified even after anonymization. The only meaningful privacy guarantee is data that never leaves your device in the first place.
Can AI companies share my conversations with governments or third parties?
Yes. Under most AI providers’ terms of service, they may disclose user data in response to legal requests, court orders, or government demands. OpenAI, Google, and Anthropic are all US-based companies subject to US law, including broad surveillance authorities. If your data is stored on their servers, it is accessible.
Does using a VPN make AI conversations private?
No. A VPN encrypts the connection between your device and the AI provider’s servers, but the provider still receives your messages in plaintext. The data is still stored, still processed, and still subject to the provider’s data policies. A VPN addresses network-level surveillance, not provider-level data collection.
Further Reading
- What Actually Happens to Your ChatGPT Conversations — A technical breakdown of the data lifecycle from input to storage
- Your AI Conversations Are More Sensitive Than You Realize — The categories of information users unknowingly disclose
- The AI Privacy Checklist: What to Verify Before You Share — A practical framework for evaluating any AI tool’s privacy posture
The Only Guarantee Is Architecture
Privacy policies change. Companies get acquired. Legal interpretations shift. The only guarantee that holds across all of these variables is one grounded in architecture: a system that physically cannot transmit your data because it never needs to.
Cloaked runs 15+ open-source models — including Llama 3.2, Gemma 3, Phi-4 Mini, Mistral 7B, and DeepSeek R1 — entirely on your iPhone using Apple’s MLX framework. No accounts. No cloud. No analytics. No network requests during inference. Your conversations exist only on your device, encrypted at rest by iOS, and are never accessible to anyone but you.
This is not a policy position. It is a structural fact.
If your conversations matter to you, they deserve an AI that is built around protecting them.
Download Cloaked on the App Store — free, no account required.
Frequently Asked Questions
Does ChatGPT store my conversations?
Yes. By default, OpenAI stores your conversations and uses them to improve its models. You can opt out of training data use in settings, but your data is still transmitted to OpenAI's servers and retained for a period of time. Opting out of training does not mean opting out of storage or transmission.
Is there a truly private AI chat app?
Yes — apps that run the AI model entirely on your device, like Cloaked. Because inference happens locally using your phone's chip, your messages never leave your device. There are no servers to store data, no accounts to link it to, and no company that could hand it over even under a subpoena.
What does 'anonymous' data mean for AI companies?
When AI companies say your data is anonymized, they mean identifying information has been removed. But research consistently shows that conversation content — especially detailed or sensitive topics — can be re-identified even after anonymization. The only meaningful privacy guarantee is data that never leaves your device in the first place.
Can AI companies share my conversations with governments or third parties?
Yes. Under most AI providers' terms of service, they may disclose user data in response to legal requests, court orders, or government demands. OpenAI, Google, and Anthropic are all US-based companies subject to US law, including broad surveillance authorities. If your data is stored on their servers, it is accessible.
Does using a VPN make AI conversations private?
No. A VPN encrypts the connection between your device and the AI provider's servers, but the provider still receives your messages in plaintext. The data is still stored, still processed, and still subject to the provider's data policies. A VPN addresses network-level surveillance, not provider-level data collection.