Guide for Busy Academics
Using Learning Outcomes to Design 
a Course and Assess Learning

 

Introduction

HE learning comprises complex mixtures of knowledge, understanding, skills and broader capabilities that can be more or less demonstrated and assessed. It will also embody values, attitudes and behaviours that are difficult to assess directly but which are an important part of a student’s academic performance.

The higher education system is being encouraged to be more explicit about the nature of the learning that programmes and modules are intending to promote. For example, through QAA policies like programme specifications and subject benchmark statements (http://www.qaa.ac.uk). In the last decade most HE institutions have adopted a learning outcomes approach in order to explain more precisely the learning that teachers are seeking to promote. 

HE teachers are expected to be able
 to show how:

•	the educational outcomes for a programme and learning outcomes for a module are being achieved;
•	that the assessment methods used are appropriate to test the achievement of  the intended outcomes;
•	and that the criteria used to judge achievement are aligned to the intended learning outcomes.

This guide is designed to provide a basic introduction to these things.

What are learning outcomes?

•	An outcome is simply a result or consequence of an action or process. 
•	A learning outcome is what results from a learning process. 
•	Intended learning outcomes are statements that predict what learners will have gained as a result of learning.

From the students’ perspective, the outcomes approach communicates what they are expected to be able to do and the criteria that will be used to assess them.

Vocabulary of course design

Learning outcomes are the actual results of learning (or rather the aspects of a student’s learning that tutors choose to assess and reward). Course designs set out the tutor’s intentions for learning (intended learning outcomes). These goals are part of an aspirational / expectational framework that includes:

Aims – broad purposes or goals e.g. this course aims to…They are generally aspirational at programme level but are more specific and achievable at the level of modules/courses.

Objectives – the specific steps that take us from where we are now towards our goals. They can be formulated as teaching objectives (what the teacher does to promote students’ learning), as curriculum objectives (how the curriculum supports the achievement of the intended learning) and as objectives for students’ learning (what the students do to learn). 

Intended learning outcomes – what students will know and be able to do as a result of engaging in the learning process. They represent statements of achievement expressed from the learners’ perspective.. at the end of the course learners will know … and be able to do….Course/module learning outcomes must be achievable and measurable. They should connect directly to the assessment criteria that are used to judge achievement.

Course documents, programme and module specifications may or may not list all three elements of the expectational framework. Sometimes the term objectives is used in the same sense as an intended learning outcome. Sometimes there is confusion because the intended learning outcomes are poorly articulated.

The outcomes approach to learning

The outcomes model is predicated on a
teaching and learning system that is aligned. Reduced to its simplest form an outcomes approach to learning has three interconnected components.

•	an explicit statement of learning intent  (intended learning outcome) expressed in a form that permits their achievement to be demonstrated and measured. 

•	the process and resources to enable the outcomes to be achieved and demonstrated (curriculum, teaching, learning methods and materials, assessment and support and guidance methods); 

•	the criteria for assessing whether the intended outcomes have been achieved and for differentiating the performance of students.

Learning outcomes and the theory of constructive alignment  

Underlying the outcomes approach to defining, designing, promoting and assessing students’ learning is a useful theory of learning known as constructive alignment (Biggs 1999). The theory connects the abstract idea of a learning outcome to the things teachers actually do to help students learn, and the things that students do to actually learn.  

The outcomes approach requires teachers to pose and answer the questions:

•	what do I intend students to learn (what learning outcomes do I want them to achieve)?
•	what teaching methods and curriculum design will I use to encourage students to behave in ways that are likely to achieve these outcomes?
•	what assessment tasks and criteria will tell me that students have achieved the outcomes I intend?

How does it work?

Constructive alignment starts with the notion that the learner constructs his or her own learning through relevant learning activities (where students’ learning is concerned – what the student does is more important than what the teacher does). The teacher’s job is to create a learning environment that supports the learning activities appropriate to achieving the desired learning outcomes. The key is that all components in the teaching system –      the curriculum and its intended learning outcomes, the teaching methods used, the resources to support learning, and the assessment tasks and criteria for evaluating learning – are aligned to each other and facilitate the achievement of the intended learning outcomes.

The main steps in the alignment process are:
1	Defining the intended learning outcomes (which determine the teaching and curriculum objectives – the steps we take to achieve the learning outcomes.)
2	Choosing teaching/learning activities likely to lead to, help and encourage students to attain these intended learning outcomes.
3	Engaging students in these learning activities through the teaching process.
4	Assessing what students have learnt using methods that enable students to demonstrate the intended learning and, in the case of formative assessment, giving feedback to help students improve their learning.
5	Evaluating/judging how well students match learning intentions: a process that is guided through explicit and manageable criteria.
6	Awarding marks/grades in line with these judgements.

Not all students will meet the intended learning outcomes) as perfectly as we would want (to those that do we award an ‘A’), a few will only meet minimally acceptable standards (call that ‘D’), others will fall in between at ‘B’ and ‘C’. It helps to define those levels of performance by verbs. These verbs are associated with objects e.g. the subjects being taught and the contexts for learning. The assessment question is how well the learning outcomes match the desired outcomes. 

Types of learning

Learning outcomes are often presented in terms of different types or categories of learning. Institutions and subjects (in their benchmarking statements)  are developing their own frameworks for describing learning. Typically they include the following categories:

•	Knowledge
•	Understanding 
•	Skills: variously described as:
•	Cognitive skills 
•	Subject specific skills (including practical/professional skills) 
•	Transferable skills
•	Employability skills
•	Capabilities 
•	Values (others often link values with attitudes)
•	Personal development 
•	Progression to employment and/or further study

Writing learning outcomes

The best learning outcomes are descriptions of what the student will be able to do as a result of studying the course or module.  They can be tested for effectiveness with the question: “and how would this be assessed?” If a clear assessment emerges, with straightforward differences between poor and excellent standards, then you have probably got a useable learning outcome.

Intended learning outcomes:
•	normally written in the future tense;
•	identify the most important learning requirements (don’t try to do too much - keep the number of learning outcomes to a manageable number typically 4 to 6 for a module);
•	be achievable and assessable; 
•	use language which students can understand;
•	relate to explicit criteria for assessing levels of achievement.

Learning outcomes are at their most useful when they focus on describing what students can do – they might have developed understanding, but how does this manifest itself? What do they do differently as a result of their enhanced understanding or their greater knowledge?

This outcomes approach is useful because it makes explicit the design and assessment of units and of whole programmes – so students, examiners and quality assurance people know what is going on. It also moves students into the centre of the picture - What do they need to learn? And how do we (and they) know they have learnt it? – so it is one element in the shift of focus from teaching to learning which has been so productive in recent years. 

For this approach to be valuable (in planning teaching activities and methods of assessment, and in linking with standards and levels) the best guidance is to move the statements about increased knowledge and deeper understanding to the “Aims” of the course, and to concentrate on preparing outcomes that are direct and assessable.

Verbs that define understanding

From the website of the American
Association of Law Libraries

‘Since the learner's performance should be observable and measurable, the verb chosen for each outcome statement should be an action verb which results in overt behaviour that can be observed and measured. Sample action verbs are: compile, create, plan, revise, analyse, design, select, utilize, apply, demonstrate, prepare, use, compute, discuss, explain, predict, assess, compare, rate, critique.

Certain verbs are unclear and subject to different interpretations in terms of what action they are specifying. Such verbs call for covert behaviour that cannot be observed or measured. These types of verbs should be avoided: e.g. know, become aware of, appreciate, learn, understand, become familiar with.’

Examples of verbs that you might think of using to specify different sorts of learning outcome:
The following lists are derived from 1) Alan Jenkins (Oxford Brookes) and Dave Unwin (Birkbeck) for the National Centre for Geographic Information and Analysis (www.ncgia.ucsb.edu): 2) Mike Laycock (UEL).

For Knowledge: Arrange, order, define, recognise, duplicate, label, identify, recognise, recall, list, repeat, memorise, name, state, relate, reproduce, record. 

For Comprehension/understanding: Classify, locate, describe, observe, recognise, discuss, report, explain, restate, express, review, select, indicate, translate.

For Application: Apply, operate, choose, practice, demonstrate, schedule, dramatise, sketch, employ, solve, illustrate, use, interpret, write.

For Analysis: Analyse, differentiate, plan, appraise, discriminate, calculate, distinguish, categorise, examine, compare, experiment, contrast, question, criticise, test.
For Synthesis: Arrange, organise, initiate, formulate, assemble, manage, collect, compose, plan, construct, prepare, create, propose, write, conceptualise, elaborate, distil, synthesise, associate, connect, develop, produce.

For creativity: Create, imagine, visualise, hypothesise, generate ideas, associate, connect, design, consider possibilities, adapt.

For evaluation Evaluate, estimate, measure, assess, judge, criticise, compare, appraise, discriminate.

For problem working – Solve, resolve, identify, recognise, apply, propose, chose, implement, assess, formulate, select, define, provide options, plan, describe a course of action.

For communication – communicate, examine, debate, respond, defend, demonstrate, express, question, explain, advocate, articulate, formulate, justify, illustrate, summarise, present.

Verbs that reflect different levels of understanding 

Biggs (1999, 2003) suggests that certain verbs reflect different levels of understanding. Some illustrative verbs for each level of Biggs’ SOLO Taxonomy are given below. Each discipline will have its own verbs as well and each verb has a topic object or context. 
Minimal understanding, sufficient to deal with terminology, basic facts:  Memorize, identify, recognize

Descriptive understanding, knowing about several topics: Classify, describe, list.

Integrative understanding, relating facts together and understanding basic theory: apply to known contexts, integrate, analyse, explain the aetiology.

Extended understanding, being able to go beyond what has been taught, deal creatively with new situations: Apply to novel contexts, hypothesize, reflect, generate. 

Outcomes in the wider context

Anybody writing learning outcomes for a module should be aware of how the module fits into the overall programme. External Examiners may also be asked to comment on the extent to which students are demonstrating the achievement of the educational outcomes for the whole course. Given the complexity of many courses External Examiners might expect to be given the maps that the course team have constructed to show how the programme outcomes are developed and assessed through the course. The key document is likely to be the programme specification but course teams may also prepare a curriculum map which shows which outcomes are developed and assessed in each module.

External Examiners may also be expected to comment on the extent to which the educational outcomes listed in the relevant subject benchmarking statement are reflected in the programme specification. The External Examiner can ask the course team to explain this relationship if it is not clear in the programme specification.

Designing a course 
is a messy business

It is one thing to describe in an abstract way how to do it. It is another thing to actually do it. Designing a course using learning outcomes is a messy iterative process. It is likely that you will start off with some provisional learning outcomes that change as you think through what it is you want to do and want students to learn. This is particularly the case when you come to think about assessment. The criteria you use to generate the evidence of learning will have an important influence on the way you express the intended learning outcomes.

An example of the process of designing a course using learning outcomes is given in an appendix.

Useful sources of information

Website of the American Association of
Law Libraries http//:www.allnet.org

Biggs, J.B. (1999) Teaching for Quality Learning at University. Buckingham: Society for Research in Higher Education  & Open University Press. (new edition in 2003).
Biggs J B (2002) aligning teaching and assessment to curriculum objectives. LTSN imaginative curriculum web site www.ltsn.ac.uk/genericcentre.

University of Hertfordshire Learning and Teaching Development Centre. http://www.herts.ac.uk/tli

LTSN Generic Centre Imaginative Curriculum web site http//:www.ltsn.ac.uk/genericcentre/curriculum

QAA web site – information on programme specifications and subject benchmark statements http://www.qaa.ac.uk.

Authors Norman Jackson, James Wisdom and Malcolm Shaw, Imaginative Curriculum project team.

Version 1 March 2003

Appendix 

Example of designing a course using learning outcomes
Norman Jackson University of Surrey

These materials are intended to provide insights into the processes of designing an outcomes-based curriculum. It is based on two teaching sessions in the University of Surrey Teaching and Learning in HE programme 1) An introduction to curriculum/ course design  2) Assessment, marking and course evaluation. For the purpose of the exercise I am imagining that this is a module in a programme.

1 Course Aims – inherited from course document
•	Introduce and review practice of course module design
•	Help participants identify and write appropriate learning outcomes and consider planning strategies, teaching, learning and assessment.
•	Help participants develop strategies to promote quality learning, practice marking and consider methods of course and self-evaluation.

2  What I want course participants to learn - content
•	What learning outcomes are and how they can be used to design modules, plan teaching and students’ approaches to learning and assess student learning.
•	The theory of constructive alignment that underlies the outcomes model of learning
•	How to apply this knowledge and learning through doing.
•	How to construct an assessment strategy, a range of methods, how to construct criteria, grading models, marking cultures, ways in which feedback might be given and assessment issues.
•	Methods for evaluating curriculum designs and the impact of teaching on students’ learning (including student feedback mechanisms).

3 This translates into a set of provisional Intended Learning Outcomes
For participants
1 To develop their knowledge about the subjects of curriculum design and assessment in the contexts of an outcomes approach to learning
2 To be able to apply this knowledge to the evaluation of course and assessment designs
3 To construct new understandings about the curriculum and assessment that will improve their ability to design  courses and modules from basic principles
4 To recognise their own learning outcomes and develop their conceptions of teaching and learning

I also have learning outcomes - the development of the knowledge, understanding and resources to teaching these subjects in these contexts.

4 My teaching and learning system

In order to enable students to achieve these outcomes I design a teaching and learning system – these are my objectives for teaching and the way students will learn. It shows that I am basing my approach on the theory of Constructive Alignment. My model of teaching embraces didactic (teacher directed) and collaborative (peer learning) and I explain this at the start of each session.

Figure 1 My teaching and learning system
What I do as a teacher to promote students’ learning

•	Create design for teaching session
•	Select appropriate teaching model
•	Identify intended learning outcomes
•	Content-learning theories/approach to modelling session
•	Research topics
•	Objectives for teaching
•	Invite participants to think about key concepts in advance
•	Request resources to support approach.
•	Prepare teaching notes and power point slides – (learning resources) 
•	Prepare a concise guide to main topic of learning outcomes and their use in course design
•	Create a series of analytical tools for the evaluation of course designs (learning resources)
•	Teach session – part transmission – part facilitation
•	Capture knowledge of participants
•	Identify key knowledge sources for learners
•	Gain feedback from students to a) inform strategy during session b) gain new knowledge for teaching in future.	What students do to learn 

In the classroom
•	Preparation – think about key concepts before session and bring own resources (module specs)
•	Actively participate in session by sharing knowledge
•	Read and learn from the learning materials provided
•	Take notes
•	Participate in small group discussions
•	Evaluate, process/analyse course designs and assessments
•	Engage in in-class assessment exercise

Outside the classroom
•	Reflect on what was learnt and construct meaning for their contexts
•	Read Guide for Busy Academics on Using Learning Outcomes to Design Courses and other recommended sources of information
•	Build portfolio of evidence of critical appraisal of own course designs.
•	Visit LTSN Subject and Generic Centre web site to see what help/materials are available.

5 Designing assessment

In order to demonstrate how I would assess students learning I have to develop an assessment process that will test the learning I am intending. I think again about my provisional list of learning outcomes and design a new set that are more useful for evaluating learning ie my learning intentions are influenced by what I finally decide to assess.

I create an assessment task using my own design as the basis for the problem. I will it as an in class learning activity and involve students in a) designing a range of assessment strategies and b) generating criteria to evaluate these strategies c) assessing the results of the task and d) drawing out the principles they used to assess learning. It serves to provide them with experiential learning geared to assessing learning.

6 Simultaneously I create a framework for assessing students’ learning for the hypothetical course

My design solution (Figure 2) to enable the ILOs to be demonstrated and assessed includes ensuring that the method will generate the type of evidence that will enable me and students to judge whether learning has taken place.

Figure 2 My assessment design

Revised Intended Learning Outcome	Assessment method/process	Criteria to guide evaluation of whether intended learning outcomes have been achieved	
1 developed their knowledge about the outcomes approach to designing modules and courses and assessing learning and demonstrate understanding by applying this knowledge to the critical evaluation of course and assessment designs, identifying ways in which they might be improved;

2 developed knowledge of the theory of constructive alignment and how it underlies the learning outcomes approach;

3 developed knowledge of programme specifications and curriculum maps and understand how they can be used to make course designs more explicit;
 
4 developed knowledge of the ways in which the curriculum and the processes it supports can be evaluated. 

5 reflect on, evidence and evaluate their own teaching and learning in respect of 
the assessment task and feedback they obtain from students and identify how they can improve their course from the self-evaluation.
	Creation of a Reflective 
Teaching File 

After 4 weeks –Progress review based on 1000 word distillation of learning against criteria 
10% self-assessment
10% tutor assessment.

End course assessment
of teaching file
30% self-assessment
50% tutor assessment

Self-and peer-assessed learning activity that simulates an assessment process.
Not assessed,	
Students will need to
1 Demonstrate that they have developed an analytical tool to evaluate course designs and assessment processes based on the twin concepts of constructive alignment and learning outcomes.

2 Demonstrate the use of the tool to undertake a critical evaluation of the curriculum design, teaching and learning activities and assessment processes and criteria on a module that they are teaching.

3 Consider the ways in which feedback is gained on the achievement of intended learning outcomes and identify ways through which feedback might be improved.

4 Show how their module connects to the educational outcomes for the whole programme by locating it within the programme specification.

5 Identify directions for change with reasoned arguments for change and show how they will be implemented and evaluate these changes.	

Standards reference points
1 Examples of completed portfolios
2 Products of simulated assessment exercise