How to tweak assessments to limit the use of generative AI

BACK
^
Listen to this article

If you are looking to tweak an assessment to make it harder for students to use generative AI, there are several approaches you can consider. While it may not be possible to completely eliminate the use of AI, the suggestions below can help mitigate its impact and ensure a fair evaluation.

Adjust task weightings

Making deliberate adjustments to your assessment criteria by assigning varying levels of importance to different tasks is an easy to implement, fast way to assess the use of generative AI tool in students assessments. For instance, you can prioritize oral presentations over written reports, thereby discouraging or reducing the tendency for students to heavily rely on generative AI tools. By placing a higher weight on tasks that require critical thinking, creativity, and verbal communication skills, you can foster a learning environment that promotes genuine student engagement and discourages overreliance on automated content generation.

Focus on critical thinking and analysis

By requiring students to demonstrate critical thinking and analysis in their assessments, you make it much more difficult for the students to use generative AI to carry  out the task.  Developing students’ higher-order thinking skills (HOTS), such as critical analysis, problem-solving, synthesis of information, interpretation, human judgment, and evaluation, is a central mission of Higher Education, and these are precisely the skills that cannot be accurately replicated by AI models.

Contextualize questions

Craft questions that require students to apply knowledge to specific real-world scenarios or case studies. This requires students to have a deep understanding of the subject matter and demonstrate their ability to apply the knowledge they have learned.

This contextualization makes it harder for AI models to generate appropriate and plausible responses without a clear understanding of the subject matter.

Incorporate open-ended questions

Questions that demand complex, nuanced answers cannot be easily generated by AI. Instead of multiple choice or simple factual questions, require critical thinking, analysis, and creative problem-solving in your assessments. Get students to analyse, evaluate, or synthesise information.

These types of questions are more challenging for AI models to generate accurate responses for.

Require personal experiences or examples

To provide appropriate personal experiences, anecdotes related to a topic, or relevant and plausible case examples requires students unique insights and individual perspectives. AI models have not (yet) been trained with the necessary skills to generate authentic and contextually relevant content.

Make it specific and localized

The more the assessment is highly specific and localized to a particular place, time, or organization, the narrower the scope of available information for the AI model as it often involves unique details and nuances that may not be present in the AI training data. Consequently, generating coherent and informed responses in such specific contexts becomes more difficult for the AI model.

Create dynamic or evolving assessments

Create assessments that incorporate real-time data, unpredictable scenarios, or dynamic variables, which can present greater difficulties for AI models to produce precise and prompt responses.

Utilize in-class discussions or presentations

Incorporate elements of oral communication or in-class presentations, which are harder for AI models to replicate. These activities allow students to demonstrate their understanding, engage in critical dialogue, and respond to spontaneous questions.

Set reasonable time constraints

Setting reasonable time constraints and deadlines for assessments poses challenges for students using generative AI as it limits the model’s ability to generate accurate and well-developed responses within a given timeframe. Moreover, time constraints may induce stress on students, leading to compromised decision-making and potentially lower quality output.

Combine multiple assessment formats and comparison with previous work

Incorporate a variety of assessment formats, such as written essays, presentations, oral exams. By diversifying the assessment methods, it becomes harder for AI models to generate appropriate responses across different formats. 

When ensuring the authenticity of a piece of work, comparing it with other student-created works helps identify characteristics such as spelling, grammar, writing style, vocabulary, complexity, comprehension, and mode of production.

Peer-feedback for assessing group individual

Peer-feedback focuses on the specific strengths and weaknesses of individual group members, which generative AI cannot accurately capture or address. This skill requires critical thinking and analysis that cannot be easily replicated by AI models.

Initiate a brief discussion with students to discuss their work

Engaging students in a short dialogue about their work requires active participation, critical thinking, and personalized responses that cannot be effectively generated by AI models alone.

Monitor student performance and behaviour

Be vigilant of suspicious patterns or anomalies in students’ performance, writing style, or language proficiency, as sudden improvements or inconsistencies could signal the use of AI-generated content.

To discourage cheating, GIHE has implemented academic integrity measures like Turnitin Similarity reports now identifying AI-generated text, as well as the LockDown Browser proctoring system.

Engage in ongoing assessment development

Continuously update and refine assessment questions, formats, and evaluation criteria to stay ahead of AI advancements. Monitor the latest developments in AI and adapt your assessments accordingly to maintain their difficulty level.

_____________________________

Reflective questions for assessment design

Have a look at our “Thinking about assessment” article on assessment design. We have created a set of reflective questions to guide you in critically examining your assessment design process and fostering continuous improvement.

_____________________________

JISC – Assessment ideas for an AI enabled world

JISC have recently released assessment cards, inspired by Lydia Arnold’s Top Trumps, led by Dr. Isobel Bowditch from UCL and supported by their assessment working group, designed to foster students’ foundational knowledge, skills, and attributes for success in an AI-enabled world, featuring 6 categories and over 40 modes:

Access it here.

_____________________________

Useful resource for considering and explaining the purpose of assessments to students

In their article, Damon Thomas collaborated with 6 academics from Australian universities to present an analytical framework designed to facilitate the understanding and development of assessments within various disciplines in higher education: The Assessment in Higher Education Framework (Thomas et al., 2018, p. 4.). 

Descriptive task

Analytical task

Reflective task

Persuasive task

Creative task

Responsive task

Design-based task

Engagement-based task

Definition

Students are prompted to provide information or factual details related to the learning content.

Students are prompted to unpack and organise information from various sources, often into categories, types, relationships.

Students are prompted to introspect on personal experiences, opinions, events, and learning.

Students are prompted to present one or more perspectives on a topic with the aim of persuading others.

Students are prompted to generate original and imaginative responses and narratives.

Students are prompted to provide responses to hypothetical situations or scenarios, often assuming vocational roles.

Students are prompted to design or plan novel texts, resources, or solutions.

Students are prompted to actively participate in learning opportunities, such as attending tutorials or contributing to group discussions.

Typical instructions

Use instructions like describe, record, summarise, or define.

Use instructions like analyse, report, relate, compare, synthesise.

Use instructions like reflect, respond, or react.

Use instructions like argue, persuade, defend, or discuss.

Use instructions like narrate, recount, or imagine.

Use instructions like respond, apply, or review.

Use instructions like design, plan, create, or build.

Use instructions like prepare, share, practice, or learn.

Assessment types

This can be applied in descriptive essays, articles, or abstracts.

This can be applied in critical reviews, reports, viva voce, and case studies.

This can be applied in interviews, journals, or presentations.

This can be applied in persuasive essays, presentations, articles, or reports.

This can be applied in performances, portfolios, artifacts, or projects.

This can be applied in reports, journals, or case studies.

This can be applied in portfolios, performances, or demonstrations. 

This can be applied in debates, role-plays, placements, or teamwork.

_____________________________

Useful glossary of assessment types

Assessment types - Glossary

Abstract

A concise description of an extensive research paper. This type of assessment evaluates the students’ ability to concisely summarise and articulate the key points and main ideas of a given topic or research project within a limited word count or time frame.

Annotated bibliography

A comprehensive list of sources accompanied by brief summaries and evaluations that provides a critical analysis of each source’s relevance, reliability, and applicability to a specific research topic.

Artefact

A tangible or digital piece of work such as a project or a portfolio, created by a student to demonstrate their skills, knowledge, or understanding in a particular subject or discipline.

Article

A written piece that students produce, typically following academic conventions, to present a specific topic, and communicate their research findings, or analysis, often demonstrating their understanding, critical thinking, and ability to communicate effectively.

Case study

A detailed analysis and examination of a specific individual, group, or situation, requiring students to identify problems and offer solutions by applying theoretical knowledge, critical thinking, and problem-solving skills to understand complex real-life scenarios.

Class tests

An assessment conducted within a classroom setting, invigilated, and typically consisting of short-duration tests or quizzes that assess students’ understanding and retention of recently covered course material.

Critical review

A comprehensive analysis and evaluation of a piece of work, such as an academic article or book, examining its strengths, weaknesses, and implications, while providing a well-supported and objective assessment of its context.

Demonstration

A practical showcase or performance by students to illustrate their skills, knowledge, or competence in a specific task or subject area.

Debate

A structured discussion or argument between individuals or teams presenting opposing viewpoints on a specific topic, engaging in reasoned discourse and persuasive communication to support their respective positions.

Discussion

An interactive exchange of ideas and opinions among students, guided by specific prompts or questions, to assess their ability to analyse, critically reflect, and engage in constructive dialogue on a particular topic or subject matter. This type of assessment can be useful for group oral assessments.

Dissertation

A substantial and in-depth research project or written work undertaken by students under the guidance of a tutor as a requirement for an academic degree, involving original research, analysis, and academic writing on a specific topic withing their field of study.

Essay

A written academic composition that requires students to present a structured and coherent argument, analysis, or interpretation of a topic or subject, typically involving critical thinking and the effective use of evidence and reasoning.

Exams

Formal and invigilated assessments that test students’ knowledge, understanding, and application of course material within a specific timeframe, usually through a series of questions or tasks.

Forum posts

A written short contribution on an online platform posted by students to demonstrate their understanding, engagement, and ability to communicate ideas within a specific topic of subject area.

Interview

A formal conversation where an interviewer assesses an individual or group by asking questions to gather information and evaluate their qualifications, skills, or opinions.

Literature review

A comprehensive examination and analysis of existing academic sources and research on a particular topic, highlighting key findings, gaps, and the overall state of knowledge in the field. This type of assessment requires students to demonstrate their ability to critically evaluate relevant literature.

Journal

A written record where students reflect, document, and analyse their thoughts, experiences, and learning throughout a course or specific educational activity.

Observation

A systematic and purposeful monitoring of and gathering of information about a student’s behaviour, performance, or skills in a specific context or task to assess their abilities, progress, or competencies.

Online quizzes

A brief and typically timed evaluation consisting of questions or tasks designed to test knowledge, understanding, or recall of specific content or concepts. This can typically be completed using online platforms such as Moodle.

Performance/recital

A public presentation or demonstration of skills, talent, or artistic expression, such as music, dance, or theatre, to evaluate a student’s proficiency and mastery in a live performance setting.

Placement

An evaluation of students’ abilities, knowledge, or skills to determine their appropriate level or position within an educational program, course, or job.

Portfolio

A collection of work, often accompanied by summaries, justifications for their inclusion, and reflections on their significance or impact.

Poster

A visual representation or display that combines research or learning outcomes into a self-explanatory display by communicating information, research findings, or creative work in a concise and visually engaging manner, allowing students to showcase their knowledge and presentation skills.

Practicals

An oral or visual communication of information, ideas, or findings, typically in front of an audience, to demonstrate knowledge, convey arguments, or showcase skills in a structured and engaging manner. This mode of assessment can be used to assess students’ practical skills and knowledge.

Presentation

An oral or visual communication of information, ideas, or findings, typically in front of an audience, to demonstrate knowledge, convey arguments, or showcase skills in a structured and engaging manner.

Project

A task or assignment that requires students apply their knowledge, skills, and creativity to plan, execute, and present a tangible outcome or deliverable, showcasing their application of knowledge, problem-solving abilities, and project management skills.

Reflective writing

A thoughtful and introspective exploration of personal experiences, observations, or learning, providing insights and critical analysis that demonstrate self-awareness and growth.

Report

A structured and formal document presenting detailed information, findings, analysis, and recommendations on a specific topic or research inquiry. Reports frequently rely on original research conducted by the students.

Role-play

Students adopt assigned roles or characters to simulate real-life scenarios, demonstrating their understanding, application of skills, and ability to interact and problem-solve within a given context.

Self-evaluation

The process in which students critically assess their own performance, progress, or skills, often through self-reflection and self-assessment tools, to gauge their strengths, weaknesses, and areas for improvement.

Take-home examinations

Students are given an examination outside of the classroom within a specific timeframe, typically allowing them to research, analyse, and thoughtfully respond to questions or prompts in a more extended format.

Teamwork

An evaluation of the collaborative efforts and abilities of students working together towards a common goal, assessing their communication, coordination, cooperation, and problem-solving skills within a group setting.

Viva voce

An oral examination or interview conducted to assess students’ knowledge, understanding, and ability to discuss and to articulate ideas on a particular subject, as well as to defend their academic work or research.

_____________________________

Further resources

Department for Education. (2023, March). Generative artificial intelligence in education: Departmental statement. Gov.uk. https://assets.publishing.service.gov.uk/government/uploads/system/uploads/attachment_data/file/1146540/Generative_artificial_intelligence_in_education_.pdf

Hillier, M. (2023, March 3). Assessment tweaks in response to generative artificial intelligence. TECHE. https://teche.mq.edu.au/2023/03/assessment-tweaks-in-response-to-generative-artificial-intelligence/

Joint Council for Qualifications. (2023). AI use in assessment: Protecting the integrity of qualifications. JCQ. https://www.jcq.org.uk/exams-office/malpractice/artificial-intelligence/

Lee, J. (2023, May 8). Effective assessment practices for a ChatGPT-enabled world. Times Higher Education. https://www.timeshighereducation.com/campus/effective-assessment-practices-chatgptenabled-world

Morrison McGill, R. (2023, April 21). Generative AI: Using ChatGPT in classrooms. @TeacherToolkit. https://www.teachertoolkit.co.uk/2023/04/21/chatgpt-in-classrooms/

Thomas, D., Rundle, O., & Emery, S. (2018). Elaborating a framework for communicating assessment aims in higher education. Assessment & Evaluation in Higher Education, 44(4), 546-564. https://doi.org/10.1080/02602938.2018.1522615