Samsung engineers leak confidential code via ChatGPT

Background

A chatbot built on a large language model answers a question by generating text, and to be useful at work it has to be given something to work on: a block of code, a draft document, a set of meeting notes. That is also where the trouble starts. Whatever is pasted into the input box leaves the machine it was typed on and goes to a service run by another company, outside the employer's own systems and outside its control.

Samsung's semiconductor business is run by its Device Solution (DS) division. The company had let DS staff begin using ChatGPT for work on March 11, 2023 ^[2]. Fewer than 20 days later, the leaks surfaced ^[2].

What happened

In April 2023 the DS division found three cases in which engineers had uploaded sensitive company information to ChatGPT in the course of their work ^[1]. The material was not idle chat. It included source code and meeting minutes: the code for a semiconductor equipment measurement database, software related to production and to defective equipment, and a summary of a meeting transcript ^[2]. Material of this kind describes how internal systems are built and how problems on them are handled — information a company normally keeps inside its own walls rather than publishing.

The three cases were found inside a single division, within weeks of that division being given access to the tool.

The response

Samsung issued a memo banning the use of generative AI tools on company-owned computers, tablets and phones. The ban took effect on May 1, 2023 and started with the Device eXperience (DX) division ^[1]. Employees were also asked to refrain from uploading company-related information to AI chatbots when using personal devices, and were warned that violations could lead to disciplinary action up to and including termination ^[1].

The company did not rely on rules alone. It limited the amount of information that could be entered into ChatGPT to 1,024 bytes per question — about a kilobyte of text, roughly a short paragraph — and launched an internal investigation ^[2]. A cap that size sharply limits what the tool can be asked to do with source code, which usually runs longer than a paragraph, while still leaving room for short questions.

Samsung also said it was developing its own internal AI tools for translation, document summarization and software development ^[1]. Tools of that kind would keep the work inside systems the company controls, instead of sending it to an outside service.

The concern was not confined to management. A Samsung survey of DX division staff found that 65 percent of respondents believed security risks could arise from using ChatGPT for work ^[1].

Why it matters

Samsung was not alone. Other companies, including SK hynix, Posco, Amazon and several major US banks, introduced similar bans or restrictions on generative AI tools ^[1]. The episode became a reference case for a practical problem that arrives with these tools: they are useful precisely because employees can hand them real work, and real work at a technology company is often confidential.

The sequence Samsung followed — access granted, leak found, a ban on company devices, a size limit on what can be typed in, an internal investigation, and plans for in-house alternatives — was repeated at other firms in the same period. It set out the two directions companies could take: restrict what goes into outside chatbots, or build something on the inside that employees are allowed to use.