Writing learning outcomes for GIHE courses

The background

In Europe, the Bologna Declaration of 1999, led to the Bologna process, which was instrumental in creating the European Higher Education Area (EHEA) (https://ehea.info/ )

To make the EHEA a reality, the Bologna process had several goals including:

• The adoption of a system of easily readable and comparable degrees.
• The promotion of mobility by overcoming obstacles to the exercise of free movement by students, teachers, researchers, and administrative staff.
• The establishment of a system of academic credits – the European Credit Transfer System (ECTS).

(EHEA, 1999).

In order to make degrees comparable across the EHEA and to enable mobility between different countries and different educational systems, the Bologna process created the European Credit Transfer System (ECTS). According to the European Union (EU, 2015, p. 10), the “ECTS is a learner-centred system for credit accumulation and transfer, based on the principle of transparency of the learning, teaching and assessment processes”.  The ECTS is a way of defining programs of learning in terms of Learning Outcomes and Student Workload. Traditionally, academic programs, syllabi, curricula, had been defined in terms of content: what the teacher was going to teach. In contrast, the outcomes-based approach of the ECTS allowed programs to be compared, and students’ achievement in one program to be recognized by another program, in another educational system. This transferability facilitated student and teacher mobility and exchanges (see Erasmus Student Network).

Outcomes based programs and Student-Centered Learning

 The shift in the focus of program specification from content (what the teacher teaches) to outcome (what the learner knows or is able to do as a result of successfully completing the program) was also part of a wider change in educational approach towards learner or Student-Centered Learning (SCL) which became more and more influential during the final decades of the 20^th century (for more information about SCL, see Weimar, 2013; Blumberg, 2008; Cullen et al., 2012).

The EU defined SCL as:

“a process of qualitative transformation for students and other learners in a learning environment, aimed at enhancing their autonomy and critical ability through an outcome-based approach. The SCL concept can be summarized into the following elements:

• Reliance on active rather than passive learning;
• Emphasis on critical and analytical learning and understanding;
• Increased responsibility and accountability on the part of the student;
• Increased autonomy of the student;
• A reflective approach to the learning and teaching process on the part of both the student and the teacher.

(EU, 2015, p.15)

In 2004, Adam predicted that

“Learning outcomes and ‘outcomes-based approaches’ have implications for curriculum design, teaching, learning and assessment, as well as quality assurance. They are likely to form an important part of twenty-first century approaches to higher education and the reconsideration of such vital questions as to what, how, where and when we teach and assess” (p.3).

The intervening years have proved the accuracy of Adam’s prediction.

As a result of this convergence of European Union strategy and a student-centered, outcomes-based approach to learning and teaching, Learning Outcomes have become the principal way of defining programs of study, courses, and learning activities.

Aligning learning outcomes to learning and teaching

We have briefly considered how the move to student-centered learning and the need in Europe for credit-based programs led to the predominance of an outcomes based approach to defining programs, courses, learning activities. Before we look at learning outcomes in detail, it may be helpful to bear in mind this advice about how abstract learning outcomes relate to concrete learning and teaching activities and the fundamental questions we should consider as we design GIHE courses:

Aligning learning outcomes to teaching and learning is about connecting the abstract idea of a learning outcome to what teachers actually do to help students learn, and the things that students do to learn.

The outcomes approach requires teachers to pose and answer the questions:

(a) what do I intend students to learn (what learning outcomes do I want them to achieve)?

(b) what teaching methods and curriculum design can be used to encourage students to behave in ways that are likely to achieve these outcomes?

(c) what assessment tasks and criteria will tell me that students have achieved the outcomes I intend?

(d) how can formative and summative assessment be combined to support the learning process and to clarify whether outcomes have been achieved?

(Cedefop, 2017, pp.54-55)

With these questions in mind, it is now time to find out exactly what Learning Outcomes are, how to define them and how to write them. This will be explored in detail in the following sections.

What are learning outcomes?

Learning outcomes (LOs) are short statements that express the level of competence attained by the student and verified by assessment after a period of learning. There are many definitions of the term learning outcomes:

Learning outcomes are statements of what the individual knows, understands and is able to do on completion of a learning process. The achievement of learning outcomes has to be assessed through procedures based on clear and transparent criteria (ECTS, p.10).

Learning outcomes: Short, focused statements that indicate what you will learn or be able to do as a result of studying a particular course or module (The Quality Assurance Agency [QAA], 2022 p. 1).

[Learning outcomes]: A statement of what a learner is expected to know, understand and/or be able to demonstrate after completion of a period of learning (SEEC et al. 2001, p.8).

A learning outcome is a statement of what a learner is expected to know, understand and be able to do at the end of a period of learning. Learning outcomes are linked to the relevant level and since they should generally be assessable, they should be written in terms of how the learning is represented (Moon, 2007, p.12).

Learning outcomes (sometimes known as objectives) define the knowledge and abilities that you hope learners will learn by completing your course (or part of it). They’re the essential building blocks of course design. They shape learning activities and assessment tasks (University College of London [UCL], 2023).

Learning Outcomes These identify what a student will know, be able to do and be able to demonstrate by the end of a module or programme. (King’s College London, [KCL], 2023).

While there are many slightly differing definitions of the term learning outcomes, all the definitions identify three essential elements:

1. The learning outcome specifies what the student knows or can do on successful completion of the learning experience (degree, course, lesson, activity), not what the teacher does. The LO is student centred: it is expressed from the student’s perspective, and it describes the transformation in the student’s behaviour or knowledge caused by the learning experience.
2. The learning outcome specifies the level of learning that the student is expected to achieve. The Europe Qualifications Framework (EQF) and the Dublin Descriptors provide overall guidance about what a student is expected to have achieved in five areas: knowledge and understanding, applying knowledge and insight, making judgements, communication, and learning skills. The descriptors identify the differences between Associate Degree level, Bachelor level and Master level. GIHE uses a more detailed framework – the QAA’s The Frameworks for Higher Education Qualifications (FHEQ) of UK Degree-Awarding Bodies to anchor its courses at a consistent level which is comparable with other institutions using the same framework.
   

   BBA1, BBA2 – FHEQ level 4

   BBA3, BBA4 – FHEQ level 5

   BBA5, BBA6, BBA7 – FHEQ level 6

   MSc1-3 – FHEQ level 7

3. The learning outcome specifies what the student is expected to do to demonstrate achievement of the outcome. This means that the LO has to be measurable, that there has to be some external proof of the cognitive transformation in the students’ knowledge or skills. This external proof is provided by the assessment of the student’s attainment. The assessment may take many forms depending on what is most appropriate: practical work, exams, project work, case studies, reflections, tests, quizzes and so on. In this way LOs and assessment are very closely linked.

A good working definition from the ECTS users’ guide (2005, p. 10) might be:

Learning outcomes are statements of what a student is expected to know, understand and/or be able to demonstrate after completion of a process of learning.

Learning outcomes and Bloom’s Taxonomy

 To assess achievement of a learning outcome, we need to measure a student’s performance or behaviour – an external demonstration as evidence of internal cognitive transformation. To define the demonstration (what does the student have to do and at what level to show they attained the learning outcome), educators often make use of Bloom’s Taxonomy. Benjamin Bloom categorized thinking behaviours into a hierarchy (revised in 2001) moving from the least sophisticated cognitive process – Remembering up to the most complex Creating. This hierarchy is often represented as a pyramid or as columns:

Source: Global digital citizen foundation based on Churches, 2008

Bloom’s taxonomy is useful for defining learning outcomes in two ways:

Firstly, the hierarchical nature allows us to specify the level of learning that is expected of the student. We can imagine that GIHE BBA1 courses at FHEQ level 4 have more learning outcomes which are pitched at the remembering or understanding levels and that BBA 5-7 and MSc courses at FHEQ levels 6-7 have more learning outcomes pitched at the evaluating or creating levels. This is a general principle, not a hard and fast rule: BBA1 students may be required to create, MSc2 students may be required to apply, but the general idea of increasing cognitive complexity and sophistication can be used to pitch your learning outcomes at the appropriate level.

Secondly, Blooms taxonomy is a way of categorizing many specific cognitive action verbs which are used to define the cognitive behaviour that the student is required to demonstrate. In order to create your course learning outcomes, you can select the most appropriate verb at the appropriate level.

There are many lists of “Bloom’s verbs”; here is a concise list:

Bloom’s taxonomy is currently the most influential and widely used framework for defining levels of understanding, but it is not the only one. Biggs et al. (2022, pp. 83-91) propose the Structure of Observed Learning Outcomes (SOLO) taxonomy which “provides a systematic way of describing how a learner’s performance grows in complexity when mastering many academic tasks” (p. 86). They claim that SOLO is an improvement on Bloom’s taxonomy because “while the Bloom’s taxonomy is a useful adjunct for suggesting a wider list of verbs for designing LOs, it does not specify the criteria for each of its six levels, whereas SOLO does”(p. 88). Reflecting on the SOLO taxonomy may help you reach a deeper understanding of how cognitive levels can be defined and then expressed in cognitive action verbs in LOs.

What does a learning outcome look like?

 A learning outcome usually consists of four parts:

Condition : “On successful completion of this course, you should be able to…” This is how the condition is specified in GIHE LOs.

Actor: “You” or “the student”. This is the standard definition of the actor at GIHE.

Behaviour : what you want your students to be able to do on completing the condition (the GIHE course). This part should consist of a cognitive action verb from Bloom’s taxonomy at the appropriate learning level and the content or subject matter which the student has to learn.

Degree : how well we want the behaviour to be performed, the standard against which will measure the performance. In GIHE LOs, the degree may be expressed by an adverb “effectively”, “accurately” or by a phrase “to a professional standard”, “respecting industry conventions” or the degree may be implicit, for example a 50% pass grade for a test or exam.

Since the Condition and the Actor are defined in the same way for all GIHE courses, you only have to define the behaviour and the degree.

Here are some examples of GIHE course LOs:

On successful completion of this course (condition), you (actor) should be able to:

Learning level 4 – BBA1

Handle challenging situations (behaviour) and provide solutions (degree) for the guests

Describe the importance of Sustainability in Hospitality (behaviour)

Learning level 5 BBA3-4

Interpret (behaviour) the elements of the balance sheet and income statement (content, subject matter).

Carry out (behaviour) effective (degree) research by selecting, evaluating, and integrating reliable data and sources (content, subject matter).

Learning level 6 BBA6-7

Use (behaviour) quantitative and qualitative data (content, subject matter) to develop strategic business recommendations (degree).

Defend decisions (behaviour) that solve (degree) ethical challenges faced by the business professional today using normative ethical theory (content, subject matter).

Learning level 7 MSc1-3

Critically analyse (behaviour) a project’s profitability (subject matter, content) to make informed investment recommendations (degree)

Critically appraise (behaviour) the digital performance of companies, employing evaluative frameworks and metrics (subject matter, content) to drive continuous improvement and innovation (degree).

How to write your course learning outcomes

1. Consider what your students should know, be able to do after your course. Discuss your ideas with the program manager, program director or academic dean. Look at the GIHE BBA or MSc graduate profile / program outcomes and think about how your course outcomes will contribute to the program outcomes. Look at the courses that your students have already taken and the courses they will take in the future to ensure that your course is a good fit, covering gaps, avoiding repetition and redundancy. Discuss your ideas with relevant faculty colleagues. Consider the learning level (FHEQ 4-7) of the semester that your course is in and look at the Dublin Descriptors to help you pitch your course at the right level.
2. Use a cognitive action verb from Bloom’s taxonomy to define behaviour – what the student will have to do to demonstrate that they have achieved the outcome. This verb is the key word in your learning outcome because:

• It defines the level of cognitive activity that is required to ensure that your LO is pitched at the correct learning level / Dublin descriptor level for the semester. Use a verb from the highest appropriate cognitive level since the higher levels include the lower levels. For example, it is not necessary to write “Remember, understand and apply” – the word “apply” covers all three levels.

• It also enables you to identify the relevant assessment task that you will require students to do in your assessments. If you choose the verb “explain” for the learning outcome, then as part of your course assessments you will probably get students to explain something. If you choose the verb “evaluate”, then you should have students appraise, judge, critique something. This direct link between the cognitive activity verb in the learning outcome and the assessment activity is part of the meaning of “alignment” in the ”Constructive Alignment” approach to course design.

3. Identify the content or subject matter that the cognitive activity verb applies to. Keep the phrase brief, you do not need to go into detail in the LO.
4. Include a phrase indicating the degree of performance that constitutes success or achievement of the outcome.

Some points to consider

1. Use only one cognitive action verb per LO. If you have two verbs, then separate them out into two LOs.

2. Make sure your LOs are specific; avoid long complex sentences.

3. Make sure your LOs are realistic in terms of the course length, student workload, corresponding assessment tasks and so on.

4. Make sure your LOs are observable and measurable. The cognitive action verb should indicate the assessment task. If your LO begins with the verb “Examine…” then the corresponding assessment task should require the students to analyze something. If your LO begins with “Appraise” then the assessment task should require the student to evaluate something.

5. Avoid verbs such as “understand” “Be aware of” “learn about” because you cannot measure them. As a last resort you may decide to use the verb phrase “demonstrate an understanding of … but this begs the question of what that demonstration might look like? If you imagine what the demonstration would be then it is usually better to directly use the cognitive action verb: “explain” “apply” “analyze” “evaluate” and so on.

6. There is no hard and fast rule about how many LOs you should define for a course. A rule of thumb which works for our context at GIHE is

one course LO for 15 hours of class contact time

meaning in practice one or two LOs for a 15-hour course, two or three LOs for a 30-hour course, three to five LOs for a 45-hour course.

7. Use assessment tasks to get the student to carry out or demonstrate the cognitive action defined in your learning outcome. Design all learning activities to help students perform well on the course assessments. This direct relationship between learning outcome, assessment task, and learning activities / course materials is the meaning of alignment in constructive alignment.

8. Make sure that all of your LOs are assessed in some way and that all of your assessment tasks get students to demonstrate a LO. If you do not assess a LO, then you have no way of knowing if the student has achieved it or not. If you are assessing something which was not expressed as a learning outcome, then this is outside the scope of the course and unfair to your students. If your course LOs, assessments and learning activities / course materials work together and reinforce each other in this way then your course will be aligned (constructive alignment) and your students engaged.

Formulating learning outcomes (ECTS guidelines)

The following guidelines are published in the ECTS user’s guide (2015, p. 23):

Considerable care needs to be taken in formulating learning outcomes.

The learning outcomes should adequately reflect the context, level, scope, and content of the programme.

The statements of learning outcomes have to be succinct and not too detailed.

The learning outcomes have to be mutually consistent.

The learning outcomes should be easily understandable and verifiable in terms of what the student has actually achieved at the end of the programme.

The learning outcomes have to be achievable within the specified workload.

The learning outcomes have to be linked with appropriate learning activities, assessment methods and assessment criteria.

There are no rules on the ideal number of learning outcomes at programme level. GIHE experience suggests that between 3 and 5 is appropriate for a 45hour conta.

A widely accepted way of formulating learning outcomes is based on three essential elements.

1. Use an active verb to express what students are expected to know and be able to do (e.g., graduates can ‘describe’, ‘implement’, ‘draw conclusions’, ‘assess’, ‘plan’).
2. Specify what this outcome refers to (object or skill e.g., can explain the ‘function of hardware-components’, or can present the ‘design of a living-room by hand’).
3. Specify the way of demonstrating the achievement of learning outcomes (e.g., ‘to give an overview of the materials most often used in electro-engineering’; ‘to develop a research design by applying up-to-date scientific methods’, etc.).

Resources

Cedefop (2017). Defining, writing, and applying learning outcomes: A European handbook. Publications Office. http://dx.doi.org/10.2801/566770 – A comprehensive and clear guide to writing learning outcomes. An EU produced, definitive guide.

Kennedy, D. (2006). Writing and using learning outcomes: a practical guide. University College Cork. https://cora.ucc.ie/server/api/core/bitstreams/88bdd1f3-4e1c-4cf8-baf4-df28d4f094c5/content – As the title indicates, this is a very practical, teacher-friendly guide. Although the original Bloom’s taxonomy is used rather than the revised taxonomy of 2001, the principles and how to apply them remain valid.

VideoScribe – Writing Learning Objectives – a short clear explanation of the purpose of learning objectives and how to write them.

How to Canvas – How to write measurable learning outcomes – an explicit demonstration of how to write effective learning outcomes: the narrator models the process of designing LOs by ‘thinking out loud’.

References

 Adam, S. (2004). Using learning outcomes: A consideration of the nature, role, application, and implications for European education of employing ‘learning outcomes’ at the local, national and international levels. United Kingdom Bologna Seminar, 1-2 July 2004, Heriot-Watt University. https://www.ehea.info/media.ehea.info/file/Learning_Outcomes_Edinburgh_2004/76/8/040620LEARNING_OUTCOMES-Adams_577768.pdf

Blumberg, P. (2008). Developing learner-centered teaching: A practical guide for faculty. Jossey Bass. Council of Europe. (n.d.). European higher education area. https://www.coe.int/en/web/higher-education-and-research/european-higher-education-area

Cedefop (2017). Defining, writing, and applying learning outcomes: A European handbook. Publications Office. http://dx.doi.org/10.2801/566770

Churches, A. (2008). Bloom’s digital taxonomy. https://www.pdst.ie/sites/default/files/BloomDigitalTaxonomy-AndrewChurches.pdf

Cullen, R., Harris, M., & Hill, R. (2012). The learner-centered curriculum: Design and implementation. Jossey Bass.

European Higher Education Area. (1999). The Bologna declaration. https://www.ehea.info/page-ministerial-conference-bologna-1999

European Union. (2015). ECTS users’ guide. https://education.ec.europa.eu/sites/default/files/document-library-docs/ects-users-guide_en.pdf

King’s College London (KCL). (n.d.). Glossary: Learning outcomes. https://www.kcl.ac.uk/campuslife/acservices/academic-regulations/assets-20-21/Glossary.pdf

Moon, J. (2007). Linking levels, learning outcomes and assessment criteria. https://www.ehea.info/media.ehea.info/file/Learning_Outcomes_Edinburgh_2004/77/4/040701-02Linking_Levels_plus_ass_crit-Moon_577774.pdf

NICATS NUCCAT SEEC. (2001, November). Credit and HE Qualifications:  Credit guidelines for HE qualifications in England, Wales, and Northern Ireland. https://www.ehea.info/media.ehea.info/file/Qualification_structures_Copenhagen_2003/10/9/BritQF_576109.pdf

The Quality Assurance Agency (QAA). (2022). Understanding learning outcomes and assessment. https://www.qaa.ac.uk/docs/qaa/members/understanding-learning-outcomes-and-assessment-infographic-text-only.pdf

University College London (UCL). (2023). Set measurable learning outcomes for your course. . https://www.ucl.ac.uk/short-courses/staff-resources/course-design/set-measurable-learning-outcomes-your-course

Weimar, M. (2013). Learner-centered teaching: Five key changes to practice. Jossey Bass.