Skip to main content
Back to newsletter

NEWSLETTER

What You Told the AI at Work May Be Exposed at a Company That Promised Not to Keep It

November 17, 2026

Illustration of a payroll clerk with glasses typing at her office desk late in the evening. Her monitor shows an “Ask AI” box, and a tag hanging from it reads “She thinks it stays here.” A framed photo of her husband sits on the desk.
Denise thinks what she types stays in the app.

Notion reported on November 12, 2026 that intruders who accessed Fireworks AI reached Notion AI chat transcripts used for its Insights feature, and Notion says it learned data was retained from the Fireworks endpoints it used despite a contractual commitment to zero data retention. A written retention promise and a security questionnaire describe a supplier's controls; they do not show the buyer what the supplier actually keeps. The edition asks every company with an AI feature in a work app which companies receive what employees type, which can keep it and for how long, and who is responsible for finding out when the vendor adds another supplier.

Anything you type in an AI box in a work app could be sent to another firm and kept there. This month, Notion said saved AI conversations had been stolen from a company that, according to Notion, had promised not to save them.

You open an application that your company uses, see the box “Ask AI,” type your question, and after a few seconds the AI gives an answer.

Seemingly the exchange is happening within that app. You ask, it replies, simple. But some of the work may be carried out by another company, and a copy of your exchange may stay there.

This story is fictional. The employees and their employer are invented. It draws on events that two real software companies reported this month. Their reports, with dates, follow the story.

Denise’s story

Denise has worked in payroll at a dental supply company in Ohio for nine years. There are roughly three hundred employees, and she needs to make sure they receive the correct amount on the correct day.

She uses a workspace app, a website where employees write and store notes, plans and meeting records together. Her company did not build it. It pays a monthly fee to the company that did.

There is also an AI assistant in the app, introduced last year, which Denise uses almost every day to convert her rough notes into emails.

Some months ago, when she had a problem with a payroll file not importing, her screen was covered with an error message. She pasted it all into the AI box to find out why the file would not load. In the middle of that text were the username and password for the payroll system. She had not noticed.

In September, her husband’s doctor discovers he has a tumor. He will begin chemotherapy, a treatment for cancer, on October 9 and will need her at home for three weeks. Denise has not told anyone at the office and cannot find the words to ask her boss for the time off.

One night, when everyone else has gone, she asks the AI assistant:

“Help me write to my manager. My husband starts chemotherapy on October 9 and I need three weeks off. I have not told anyone here and I want to keep it private.”

The assistant’s response is brief and kind. She copies it into an email, changes a few words and sends it to her manager. She deletes the conversation where she drafted it. She thinks it is too private to leave in the app.

Her manager does not make a fuss about the leave. Denise takes her three weeks off, comes back and forgets about the exchange.

The notice

Tom maintains the computers and software for the company. In November, he sends an email to everyone concerning a notice he has received from the app maker.

He explains that the app maker does not run the whole AI assistant itself. Another company does part of the work. Criminals got into that company’s computers over the internet and copied some of the AI conversations it had saved. Some came from their company.

Anyone who has typed a password into the assistant should change it today.

Denise recalls the payroll error message and changes the password before heading to Tom’s desk.

Which conversations were copied?

The app maker has sent him a file listing what was taken from their company. He turns his screen towards her. There are requests from her coworkers asking for help with a sales forecast or drafting a warning letter to an employee.

Halfway down, she reads her own words.

“My husband starts chemotherapy on October 9.”

Illustration of the clerk standing, hand to her mouth, beside the IT manager at his desk. His monitor shows a list titled “What was taken” with one row highlighted. A tag reads “She had deleted the page.”
Halfway down the list, she reads her own sentence.

Her name is not attached to the sentence, but she does not need it to recognize her words. Someone else at work might remember that she was off for three weeks.

“That cannot be right,” she says. “I deleted the page the night I wrote it.”

Tom has spent the morning finding out why the words were still there. The app had a feature that showed managers how staff used the assistant, but the reports were not supposed to identify individual employees or show what they had written.

To produce those reports, the app saved what people typed and what the AI answered. The conversations needed to be sorted into categories, so the app maker sent them to another company to do that work. Denise’s conversation was among them.

She had deleted the conversation from the app, but the copy sent to the other company was still there. That was one of the conversations the app maker had listed in the file telling Tom what had been stolen.

Denise asks which company. Tom reads out the name on the notice, but neither of them recognizes it.

Who in her company had agreed that her message could go there?

Tom describes how the app was bought. Two years ago, before the purchase, he sent the app maker a security questionnaire. That is a list of questions a buyer asks a supplier about how the buyer’s information will be protected. The answers were satisfactory, and the purchase went ahead.

At that time there was no AI assistant. It arrived in a later update, along with the second company’s involvement. Tom was not asked to review the app again.

Denise has one last question: was that other company allowed to keep what she wrote?

Tom asked the app maker the same question earlier that day. The answer was no. Under its contract, the supplier was supposed to sort each record, erase it and keep nothing. It kept the records anyway. The criminals copied what it had kept.

Denise wrote a message in a work app. The app recorded it and passed the record to another company. That company had promised in writing not to keep it, yet it did. Criminals then took the copy.

Her employer had chosen and checked just one company in that chain: the app maker. It did so before the app had an AI assistant.

Illustration of four stops joined by arrows: Denise at her desk, a laptop labeled “Work app” with a tag reading “Checked,” an office building labeled “Second company” with a file drawer open, and a man carrying papers labeled “Criminals.”
Her employer checked one stop on this path.

What the companies reported

Let's get back to the real world. One of the companies that reported an incident this month was Notion. Companies use it's workspace app to write and store notes and documents. It includes and AI assistant is called Notion AI.

The second company is Fireworks AI. It runs AI systems for other software businesses. An app maker does not have to run all the AI on its own computers; it can pay a company like Fireworks to do part of the work.

On November 12, 2026, Notion published a security advisory, a public notice about a security problem. It said intruders had accessed Fireworks and reached data that included transcripts of Notion AI chats. A transcript is a saved written conversation containing what the person typed and what the AI answered.

The transcripts were at Fireworks because of Insights, a Notion feature that reports to customers on how people in their workspace use Notion AI. Notion said it used Fireworks to sort chats for that report.

According to Notion:

On November 2, Fireworks told Notion that intruders had accessed a small amount of its data. At first, it appeared to be test data containing no customer information.

Notion disconnected from Fireworks on November 3 and deleted its data there on November 4.

On November 6, Fireworks said customer information might also be involved.

On November 8, Fireworks confirmed which data had been accessed.

On November 9, Notion informed affected customers.

Illustration of six cards pegged to a string: Nov 2 First notice, Nov 3 Disconnected, Nov 4 Data deleted, Nov 6 Second notice, Nov 8 Data confirmed, Nov 9 Customers told.
The dates Notion gave in its advisory.

Notion says fewer than 1% of its workspaces were affected. A workspace is a customer’s area within Notion, usually belonging to a company or team.

For the vast majority of those workspaces, Notion says, the transcripts had no identifying labels showing the author or company. What people wrote could still give them away.

The contents of the conversations were still in the transcripts, and Notion found software keys in some of them. These are credentials that allow one program to access another, much like passwords.

Notion advised affected customers to treat the data as exposed and replace any passwords or keys they had pasted into Notion AI.

Changing their password would not be sufficient, because their messages may also contain information about their organisation or private information that a new password cannot protect.

For now, customers Notion has not contacted do not need to take action, according to the advisory.

The promise not to keep it

According to Notion, features such as chat history and Insights require it to save conversations and are not covered by its own zero-retention promise. Fireworks, however, had separately agreed not to retain the data. On a page answering questions about the incident, Notion said Fireworks had agreed to “zero data retention.” In ordinary terms, it could handle the data to do the work but was not supposed to retain a copy afterwards.

Notion said its investigation found that

“data was retained from the Fireworks endpoints that we used.”

The endpoints were the connections through which Notion sent the data. Notion’s account is that Fireworks retained data despite a contract requiring it to keep none.

Fireworks published its own security bulletin, last updated on November 9. It does not name customers or address Notion’s account. It says three Fireworks products retain no data by default and that it found no evidence those three products were affected. It does not explain where the Notion conversations were retained or for how long.

It does explain how the attackers accessed its systems. Fireworks rented some of its computers from a cloud provider, a company that rents out computers over the internet. That provider had a security incident on October 23. The next day, attackers used that incident to reach Fireworks’ machines.

There are four companies in this chain: the employee’s employer, Notion, Fireworks and the cloud provider. The employer bought the app from Notion. Fireworks and the cloud provider were companies behind that service.

Illustration of four buildings on a street joined by arrows: Employer, Notion, Fireworks and Cloud provider. The first arrow is labeled “Direct supplier” and the next two “Further suppliers.”
Four companies in the chain. The employer chose one.

Cursor reported an exposure too

Cursor is an AI coding tool. Programmers type instructions, and it helps write or repair software. Cursor saved records of some of those work sessions.

On November 13, Cursor reported that an attacker had copied some of the stored sessions. It had given Fireworks an access key. According to Cursor, Fireworks exposed the key, and an attacker used it to access a storage area Cursor used to share data with Fireworks.

Cursor said it had permission to use the sessions to train AI models. Training means feeding examples to an AI system to improve how it performs. Most of the copied sessions were from 2025.

Credentials had not been removed from a small number of them. The attacker used those credentials to access some of the outside software tools Cursor uses.

Cursor says the copied sessions concerned fewer than 0.05% of users who were not using Privacy Mode. For users with Privacy Mode enabled, it says prompts and code were not affected.

Neither Notion’s disclosure nor Cursor’s says that a private chat has been published.

What could the buyer check?

A company selling software to another company is called a vendor. Buyers often expect the vendor to take responsibility for what happens inside its product. Choose a reputable supplier, ask the security questions and get the commitments in writing, and the information should be protected.

This is the belief I call The Accountability Assumption on sougataroy.com: once you buy the tool, the company selling it owns the answer for what happens within it.

Notion says it had Fireworks’ commitment in writing. That commitment did not establish that the information was actually being deleted. Whether it was kept depended on what happened inside Fireworks.

This is also the limit of a security questionnaire. The supplier describes its controls. The buyer reads and stores the answers. The buyer does not thereby gain a view into the supplier’s computers. “We keep nothing” remains the supplier’s claim.

Tom’s review had another limitation in the fictional story. He checked the app as it existed when his company bought it. His questionnaire was already on file when an update added the AI assistant and brought another company into the handling of employee conversations.

For a company seeking to establish where its information goes, three questions apply to every app with an AI box.

Which companies receive what employees type? Name every company involved, including those working for the app maker, not just the company on the invoice.

Which companies can keep it, and for how long? What supports that answer beyond the wording of a contract?

Who at our company is responsible for knowing this, and how will that person find out when the vendor adds another supplier?

Illustration of the IT manager scratching his head over a security questionnaire stamped “We keep nothing.” A note on the corkboard asks “Who receives it? Who can keep it? Who is responsible?”
“We keep nothing” remains the supplier’s claim.

An employee can ask the people responsible for the company’s software where AI conversations go and who keeps a copy. Ask what information the company allows you to use with each AI tool. Ask about AI features inside spreadsheets and documents too, not just the boxes where you type a question.

Do not put passwords or software keys into an AI tool. Until you know what is allowed, do not use it with customer data, confidential company information or personal details. Something you would not say out loud in the office does not become safe to share because you are typing it into a work app.

Denise deleted her conversation the night she wrote it. A copy of what she wrote in the chat stayed behind at a company whose name she first saw on a breach notice.

Denise, Tom and their employer are invented. Their story shows how a worker’s words can travel past the vendor the employer chose. It does not describe any Notion or Cursor customer. This article is not legal advice.

Sources: Notion, “Fireworks AI incident,” security advisory, November 12, 2026. Notion Help Center, “Notice about the Fireworks AI incident that may impact some Notion workspaces.” Fireworks AI, “Update on October 2026 security incident,” last updated November 9, 2026. Cursor, “An update on our recent security incident,” November 13, 2026.