Don't Forget About Data Protection Compliance When Using Artificial Intelligence
The growing popularity of artificial intelligence (AI), particularly generative AI like Chat GPT, presents opportunities and challenges for the higher education sector.
We anticipate that higher education institutions (HEIs) will want to explore the benefits that AI has to offer. Data protection law does not stand in the way of this innovation but it does require HEIs to carefully consider how to be compliant before processing personal data using AI.
Carry Out a Data Protection Impact Assessment
HEIs must carry out a Data Protection Impact Assessment (DPIA) before processing personal data in a way that is likely to result in a high risk to individuals. In many cases the use of generative AI will meet this threshold. Even if it is not a strict legal requirement, a DPIA is a useful exercise to methodically consider and document data protection compliance.
Security
You must consider how to avoid a personal data breach when using AI. It is an obligation under the UK GDPR to implement appropriate technical and organisational measures keep personal data secure. What measures are appropriate will depend on the specific AI system, and what you're using it for, but they are likely to include controls around what personal data you permit staff to input into the AI system and considering how the AI integrates into your existing IT system.
In addition, the National Cyber Security Centre (NCSC) has highlighted the cyber risks created by generative AI. One of the risks is cyber criminals using AI to write convincing phishing emails - spam emails that try to trick people into providing information, click on bad links or open harmful attachments. Your HEI's information security training for staff must include how to spot phishing emails. This training must be supplemented by policies, procedures and technical measures to keep personal data secure.
Controller or Processor?
Under the UK GDPR those processing personal data are either a controller, a joint controller or a processor. If your HEI uses an external AI provider, your data protection obligations will be partly dependent on the AI provider's role. For example, if they are your processor, you must have a contract with them that contains the mandatory data protection provisions and only use processors that provide sufficient guarantees of UK GDPR compliance. It's possible for an AI provider to be a controller or joint controller for some phases or purposes of processing, and a processor for others.
Fairness and Transparency
There are risks around the use AI leading to discriminatory outcomes, for example, when the AI is trained on information that is unbalanced or biased. This has implications for data protection compliance because the Information Commissioner's Office (the data protection regulator) is clear that any processing of personal data using AI that leads to unjust discrimination will violate the fairness principle of the UK GDPR. There are other laws, notably the Equality Act 2010, that you will also need to consider.
Transparency sits alongside fairness in the first principle of the UK GDPR. Be clear with individuals about how their personal data will be used. This can be challenging with AI because of its complexity. It is essential that the personal data is not used in a way that is unexpected to the individuals concerned.
What Do Your Staff Need to Know?
Consider what training, policies and procedures are needed to ensure that those who are processing personal data using AI do so in line with the legal and regulatory requirements. Any training should remind staff what counts as personal data to guard against staff assuming that removing names is sufficient to anonymise information.