Informal applications of artificial intelligence, such as the use of tools by university faculty to grade student papers, could breach existing legislation, warns a policy expert, according to The Times Higher Education.
Many European universities may need to „change everything” about how they use artificial intelligence once historic European Union regulations take effect, an expert has warned.
Thomas Jørgensen, director of policy coordination and foresight at the European University Association, highlighted the growing practice of university staff using tools like ChatGPT to grade student work as a particular concern. Such informal uses of artificial intelligence could violate the EU's AI Act, he said.
„”If you use artificial intelligence for assessment, according to the AI Act, there's a wide range of requirements you need to meet, both as a user and as a provider,„ Jørgensen told Times Higher Education. "And I think it's fair to say that large language models [LLMs] don't meet the criteria, as it relates to transparency and the data they are trained on.".”
Universities sometimes use artificial intelligence tools to grade students without any institutional oversight, but compliance challenges would also affect universities that have created their own formal AI tools, he added.
„For teachers using ChatGPT for assessment, there is a risk that it may be illegal; but we don't know yet, as we don't have the guidelines from the European Commission's Office for Artificial Intelligence.” , he said, referring to the body overseeing law enforcement.
The EU's AI Act, a landmark piece of legislation covering a wide range of sectors, went into effect in August 2024. Its remaining provisions are set to come into force over the next six to 36 months.
In March, the European Parliament voted to postpone key provisions of the legislation to give companies and regulators more time to prepare for the changes. These provisions relate to high-risk artificial intelligence systems and were initially set to come into effect in August of this year. According to the law, AI used to evaluate students is classified as high-risk – falling into the same category as AI used for hiring decisions and credit scoring.
Jørgensen emphasized that universities responded to the emergence of generative AI tools in 2022 and 2023 with working groups and guidelines, and have developed quite clear strategies for responsible use in learning and teaching.
„The next step will be when the AI law truly comes into effect and the AI Office guidelines arrive. This will be a big challenge.”
Once the provisions are fully in effect, he said, the biggest risk will be that universities will have to „change everything” and revise a lot of practices they have implemented regarding AI in order to comply with the law.
Jørgensen said that the use of artificial intelligence in research was already substantial, especially in STEM fields where researchers often worked with large and complex datasets. He was cautiously optimistic about what this could mean for the speed of scientific progress, which he said had been slow in recent years, but there were questions about data privacy.
„Usually, in healthcare, we need to have policies in this regard, because otherwise there is a risk that we won't follow them. But this is a conversation that is just beginning,” he said.
In January, the EU published a report, led by Jørgensen, that examined the use of artificial intelligence in European universities. Emphasizing that the primary source of data for training many models comes from the English-speaking world, and overwhelmingly from the US, it highlighted the need for European AI models.
Jørgensen warned that European universities using the same commercial models practiced in the US risk eroding the intellectual diversity that made collaboration worthwhile. „Things become bland and uniform,” he said. „The idea of cooperation, because you bring something different, disappears, because you're testing the same model, trained on American data.”
He pointed to the emergence of large European language models, including EuroLLM, as a potentially more suitable alternative for universities. „The big American models are not really suitable for us. They are very US-centric,” he said. „If you speak a [different] language, they might not be as good for you.”