CYB3R53C ICS / OT Security

AI Supply-Chain Risk Made Concrete: Notion AI Chats Exposed Through a Breach at Fireworks AI

Cybersecurity Trends and Emerging Threats Jairo Rodriguez 2 min read
Illustration: linked nodes with one link broken

Notion has told customers that some AI chat sessions may have been exposed after a breach at Fireworks AI, a company it used to run AI features. Notion puts the possible impact at under 1% of workspaces.

What happened

  • 2 November: Fireworks informs Notion that a small dataset was accessed. Notion’s first assessment is that it contained only synthetic test data.
  • 3 November: Notion disconnects from Fireworks.
  • 6 November: Fireworks says customer data may have been accessed after all.
  • 8 November: Fireworks confirms the final dataset.

The data consists of Notion AI chat sessions, not tied to named users, that had been used for Notion’s Insights feature. Notion has disabled Insights. Reports say Fireworks had disclosed an earlier intrusion in October, and that this second incident came through a third-party cloud provider.

What Notion advises

Treat the affected chat content as exposed, and rotate any passwords, API keys or tokens that were pasted into Notion AI.

Why this matters beyond Notion

People paste things into AI assistants that they would never put in an email: configuration files, error logs with credentials, network details, customer data. That text then travels to vendors the organisation has never assessed, and may be stored for purposes such as analytics.

The first statement in this case said no customer data was involved. It was revised four days later. That is normal during an investigation, and a good reason to act on the first notice instead of waiting for the final one.

What to do

  • List the AI features switched on in your tools and find out which third parties process the data.
  • Set a clear rule: no credentials, keys or network diagrams in AI chats.
  • Ask vendors how long prompts are kept and whether they are used for secondary features.
  • Scan for secrets in the tools that allow it, and rotate what you find.

For OT teams

Engineers increasingly use AI assistants to debug PLC logic, parse device logs or draft procedures. Those inputs describe how your plant works. Decide what may leave the site before someone decides for you.

Sources