Your Team’s AI Use Is a Data Risk: Here’s How to Protect Your Business

Artificial intelligence (AI) offers powerful tools for businesses, from automating complex processes to improving decision-making and enhancing customer experiences. However, using AI, especially with sensitive company data, introduces significant security and privacy risks. Data leaks and privacy breaches can lead to financial losses, reputational damage, and legal liabilities. This guide outlines the dos and don'ts for using AI with company data, helping you to minimize those risks.  

In 2023, several Samsung employees allegedly leaked confidential company information to the AI-powered chatbot, ChatGPT. One engineer entered Samsung’s source code into ChatGPT while looking for a solution to a bug. Another recorded a company meeting, transcribed it, and then inputted the transcription into ChatGPT to create meeting notes. A third employee used ChatGPT to optimize a test sequence for identifying yield and defective chips. As ChatGPT is a machine learning platform, all input data is used to train its algorithm, so this proprietary information is now available to all those using the platform. It is for this reason that ChatGPT itself warns users not to enter sensitive information. This incident highlights the risk associated with employees inputting sensitive data into AI tools, especially when those tools’ data handling practices are not fully understood or controlled.  

Understanding the Risks 

It’s critical to understand how data flows in AI systems. "Training data" teaches the AI, "input data" is what you provide to the tool, and "output data" is the AI's response.

Risks include:    

• Sensitive Data Collection: Gathering data without proper safeguards, like personally identifiable information (PII) or confidential business information, is a key concern.    

• Lack of Consent: Collecting data without explicit consent or a legal basis can have severe consequences.    

• Data Misuse: Using data for purposes beyond its original intent is a risk.    

• Unchecked Surveillance: AI can be used in ways that violate individual rights.    

• Bias in AI Output: Flawed data or algorithms can lead to biased results.    

• Data Exfiltration: Unauthorized theft of data from AI systems is a serious threat.    

• Data Leakage: Inadvertent exposure of private information can cause privacy breaches.    

• AI-Powered Cyberattacks: Attackers can use knowledge of AI systems to launch sophisticated attacks.    

• Adversarial Attacks: Manipulating input data to deceive AI models.    

• Data Manipulation: Injecting false information to skew AI's learning process.    

• Theft of AI Models: Attackers may steal proprietary AI models and the sensitive data used to train them.    

• Model Supply Chain Attacks: Targeting components used to develop AI models.    

• Algorithmic Bias: AI can make unfair or discriminatory decisions.    

• Extended Data Storage: Keeping data for long periods increases the risk of unauthorized access.    

Dos and Don'ts for Using Company Data in AI – What You Need to Tell Your Employees  

Dos 

• Strict Data Governance: Categorize data by sensitivity, grant access on a need-to-know basis, and define data lifecycle policies.    

• Prioritize Data Minimization: Only input essential data, anonymize data when possible, and use aggregated data.    

• Utilize Secure APIs: Prefer API integrations for better control and use strong authentication.    

• Establish AI Usage Policies: Create guidelines for AI use and provide employee training.    

• Conduct Security Audits: Perform vulnerability assessments, monitor data flow, and stay updated on security threats.    

• Emphasize Prompt Engineering Best Practices: Train your employees to craft secure prompts, use generic data in prompts, and use code to handle data before it is input into AI.    

Don'ts 

• Don't Input Sensitive Data Directly: Avoid pasting confidential information into AI prompts.  

• Don't Rely Solely on AI Output: Always verify AI-generated content.    

• Don't Neglect Employee Training: Educate employees on AI risks.    

• Don't Assume AI Providers Are Always Secure: Vet providers and understand their data policies.    

• Don't Ignore Legal Requirements: Comply with data privacy regulations.    

• Don't Use Public AI Tools for Sensitive Data: Only use vetted AI tools.  

Provider	Opt-Out Method	Notes
OpenAI (ChatGPT)	Data controls within account settings or API usage with specific parameters.	API usage offers more granular control. OpenAI has updated their policy to not train on business API data.
Google (Bard/Gemini)	Activity controls in Google account; opt out of ‘Bard activity’ or ‘Gemini activity’. For workspace, Data is not used for training.	Google Workspace has stronger data privacy but review specific product settings.
Microsoft (Azure/OpenAI/Copilot)	Azure OpenAI Service: Data is not used for training. Microsoft 365 Copilot: Data remains within tenant boundary.	Azure OpenAI Service is designed for enterprise with robust security. Microsoft 365 copilot is within the boundaries of the tenant's data.
Anthropic (Claude)	API use with specific parameters. Review their privacy policy.	Similar to OpenAI, API use offers more control.

 

So, what can we learn from the Samsung ChatGPT data leak? 

Educate your employees. Teach them about the risks of sharing data with AI, there are plenty of examples where it has gone wrong. Create clear rules. Set policies on how AI tools can and should be used. Minimize data sharing. Only share the necessary data with AI. Use security measures. Implement online tools to prevent data leaks. Assess AI Risks. Always make sure to check the security of AI tools before your staff use them. Control API access. If using API’s, ensure they have strict data access controls. Review the situation often. AI technology is rapidly evolving, and so are security measures, so make sure to keep informed and updated on the latest changes.