On average, across more than 28,000 assessment questions, human students generally outperformed the chatbot.

Across all assessments, students scored an average of 76.7%, while ChatGPT scored 47.4% based on fully correct answers.

ChatGPT performed better when there were multiple choice questions but also often simply made up answers which had no relevance to the exam questions and made basic errors in calculations. It also struggled to handle long, written questions with multiple parts, showed the global research led by Brigham Young University.