Designing Assessment for Creativity: an imaginative curriculum guide (June 2005) Lewis Elton Jackson (2003) in his introduction to the Imaginative Curriculum Network, ‘Nurturing creativity through an imaginative curriculum’ has this to say on assessment: The current assessment model with its atomised approach to assessing learning at module/curriculum unit level is a major inhibitor of designs for creative learning which may need to foster development over a longer period of time and a range of contexts before assessing capability The idea of synoptic assessments that enable students to draw together and apply their learning throughout a course (such as final level projects and dissertations) provides important opportunities for students to demonstrate their creativity Strategies that require students to reveal their understanding of how they have acquired core learning outcomes from a course (e.g through reflective report or portfolio) offers students another way of demonstrating their unique creativity Student instrumentalism driven by the teachers’ belief that students only learn when they are assessed inhibits creativity Narrow, summatively-driven assessment practices and criteria that focus on what is known, which not recognise the process of learning and how people come to know, or recognise emergent unanticipated learning outcomes, will smother creativity This Guide builds on these ideas and in particular attempts to make them accessible to practising teachers in higher education In the process it may be accused of oversimplifying, but it is at least arguable that references to Shakespeare, Newton and Mozart are not helpful when discussing the creative work of most undergraduate students How can one recognise and assess creative work? It is necessary to operationalise the abstract word ‘creativity’ if it is to be linked to assessment Hence the use of the phrase ‘creative work’ For work to be considered creative, it has to be – within its context and, in the case of students, at a level appropriate for them – both new and significant An important aspect of that context is the creator While the word ‘new’, even when modified by ‘within its context’ is comparatively simple to interpret, this is not so for ‘significant’, which introduces elements of critical judgment This links the concept of creativity to that of criticality and, while not all criticality is allied to creativity, all creativity must be allied to criticality [The definition of creativity as ‘shared imagination’ (DeWulf and Baillie , 1999) has similar implications – sharing introduces critique.] Assessing creativity – guide 1 Hence, in order to recognise and assess creative work, it is necessary to assess both the creativity and the criticality involved, within the appropriate context This approach should provide a frame of reference, without implying an inappropriate reductionism At the same time, it does not imply that there may not be other criteria to be considered in assessing a creative piece of work, such as relevance, difficulty, cost etc Clearly, it is in general neither possible nor desirable, to assess a piece of creative work against predetermined criteria – the criteria have to be interpreted in the light of the work In general, this places the assessment of creative work in the realm of interpretivist assessment, as opposed to positivist assessment, in which the criteria are determined in advance of the work [For a discussion of these terms, see section 7.] Illustrations from Fine Art and Architecture courses It is of course essential to convey all these matters to students, teaching staff and examiners It is interesting to see how this is done, eg in the University College London Slade School BA in Fine Art, Student Handbook 2003/2004, where the cruteria are stated as follows: BA in Fine Art: Criteria for Assessment You will be assessed on the evidence of ambition, experimentation, innovation and understanding of the subject and its contexts, as developed in the work Your progress in and development of the following will be taken into account: critical awareness; relevant use of processes and materials; the depth and scope of investigation; the ability to realise ideas; contribution to and participation in the course Similar criteria are laid down e.g in the Fine Art Studio Practice Module at Manchester Metropolitan University (MMU) where, on completing Level Two, students will be able to: explore and identify appropriate uses of visual language in processes of research, analysis, ideas generation and development and expression of outcomes; interpret and evaluate and synthesise theoretical, professional and contextual issues relating to an emergent personal practice; appreciate the relationship between the artist, products of practice and audience, maker and processes of consumption; and in general terms: use critical reflection as a means of identifying progress, personal strengths and overall development needs; understand and apply a range of research strategies for the planning and information retrieval procedures; select, apply and evaluate personal strategies for the planning and organisation of work Assessing creativity – guide Although the MMU criteria are more comprehensive, in fact both sets can only be understood and applied in terms of a tacit understanding of what they mean This is appropriate so as not to constrain the two most important aspects – creativity and criticality Similar approaches are used in Schools of Architecture [see Doidge et al (2000) for the famous ‘crit’ and thanks to Rosie Parnell for her excellent description of it, see Appendix]; the approach in general has been described rather well by Parnell (2004) who found that her ‘worst teaching moment’ was in the lecture theatre, because it was difficult to engage in dialogue! That all may not be well in Art assessment under the pressures of the bean counters is argued by Canatella (2001) Marking schemes In the Slade School statement, there follows a statement of an overall marking scheme This is not however broken down according to the above areas which are to be taken into account and thus suffers from the flaw in all such schemes, namely that the same grade can be obtained on the basis of quite different strengths and weaknesses and that it therefore hides details of a student’s performance This fundamental flaw in classification schemes has led to doubts concerning the value and desirability of degree classification schemes as such, and this in turn has led to the idea of a profile in which each part is separately classified if appropriate, and not if not, the final outcome being then reported in the form of a profile [It may be noted that the phrase ‘and not if not’ radically differentiates such a scheme from the one used in the United States, where all parts are separately classified, with the implied restriction that only parts which can be so classified are included.] Resulting schemes of this kind, which in general include assessment on the basis of a portfolio, have been advocated by Knight and Yorke (2004, pp.103 - 105 ), Johnston and Elton (2002) and Johnston (2004) Hidden curriculum and constructive alignment There needs to be some alignment between the programme specifications in different subject areas: there needs to be some core account of the nature of ‘critical thinking’, ‘creativity’ and other core elements of graduateness Knight and Yorke (2004), pp 47 – 48 A problem, first explored by Snyder (1973), is that what is being assessed in a course is not in agreement with proclaimed learning outcomes, in general because assessment of the proclaimed learning outcomes cannot be achieved with the reliability desired In such circumstances, the curriculum addressed by the majority of students is that on which they are assessed – the ‘hidden’ curriculum [At this point I wish to distance myself somewhat from Jackson (2003) above regarding ‘the teachers’ belief that students only learn when they are assessed’; to the extent that once they are satisfied that their assessment needs are met, many students transcend them (Elton 1988).] This phenomenon has been generalised by Biggs (1999, pp 25 - 29) through the concept of ‘constructive alignment’, ie that teaching, learning and assessing must be aligned to each other How this can be done differs for different disciplines, but it is often particularly absent in the sciences, where students are expected to learn problem solving, but problems in examination tend to be stated in a stepped fashion, so that each step follows from the previous one with the usually most important aspect of problem solving, ie the open-endedness of the process, deliberately left out in the interest of marking reliability Assessing creativity – guide Disciplinary differences and Departmental cultures While the use of portfolio assessment of creative work has been traditional in Fine Art and Architecture for a very long time, the opposite is the case in other disciplines, in most of which it may indeed be argued that creative work is only appropriate for a comparatively small part of the curriculum Such work is then usually confined to Project work in the final term of a course (an innovation adapted from Fine Art departments in the 1960s and now almost universal in Britain), but even there its creative purpose may be subverted through inappropriate assessment, such as assessment based on the written report of the project, rather than the work itself which is harder to document, or the allocation of only a small proportion of the total degree mark on the grounds that project assessment is less reliable than the assessment of written papers Such forms of subversion arise usually in teachercentred traditional curricula, where students are allowed little freedom This leads to negative attitudes towards a change to a creative curriculum – far more than the perceived inappropriateness of a creative approach to the study of a particular discipline Evidence for this has come in the past decade from the development of curricula, which use problem based learning (PBL) (Savin-Baden 2000), but while PBL certainly makes it easier to introduce creativity into a curriculum, it is by no means essential However, it is worth identifying the features of PBL which encourage creative work and which individually are present in other types of curricula than PBL Problem based curricula PBL was originally introduced into medical curricula in order to improve the diagnostic skills of students However, in the process it led to curricular changes which had nothing to with the development of diagnostic skills but which were valid across the disciplines The most important of these are: learning is student centred and student initiated (instead of teacher driven); the teacher becomes a facilitators of learning (instead of dispensers of knowledge), remaining an authority, but not in authority; knowledge is acquired through solving problems (instead of problems being solved through the application of knowledge); learning is co-operative in groups (instead of individual), with the consequence that what might traditionally be called plagiarism becomes collaboration All four of these features encourage student creativity, with the additional benefit that teachers, in retaining their role as authorities, encourage the kind of criticality that is such an essential feature of meaningful creativity Another important feature in encouraging creativity is that of group learning – e.g students bounce their ideas off critical fellow students or through brain storming develop creative ideas At the same time, allowance has to be made for the creative loner, even if he is probably not a budding Shakespeare, Newton, Mozart The assessment process A summative approach to assessment may be fit for the purpose of establishing how much determinate information a student remembers, but not for making judgments related to a fuzzy social construct such as practical intelligence Assessing creativity – guide Knight and Yorke (2004), p 52 It is apparent that the traditional principle of fair assessment, ie that all students are assessed in the same way and compared with each other through normative assessment on the same materials and individually, is inappropriate for the assessment of creative work What has to be understood is that to treat everyone the same when people are so obviously different from each other is the very opposite of fairness Instead, for authentic assessment, students should be assessed in their performance against their potential and as a rule on the basis of negotiated learning agreements, largely on work presented by them in portfolios based on their own selfevaluation of their work No two portfolios are the same and the assessment has to be criterion and not norm referenced Such criterion referencing on the part of examiners implies what Eisner (1985) has called connoisseurship: the educated ability of experts in a particular field to assess work in it Such connoisseurs not necessarily agree in their judgment, but they can defend it to other connoisseurs, so that an agreed judgment becomes possible Also, they are usually aided by the need to relate their overall judgments to a series of criteria, like the ones listed for the Slade School in section However, to form a judgment of a particular piece of work on the basis of such a list of criteria can normally be done only through discussion between assessors after students have presented their work; it is rarely possible to lay down assessment criteria in advance, as it is exactly this feature of traditional assessment which takes away from the originality, criticality and creativity of the work For that reason such assessment has been called interpretivist, as opposed to the traditional positivist approach, in which performance is matched against previously determined criteria Another aspect of a ‘fair’ assessment that becomes inappropriate is that such a ‘fair’ assessment must be carried out under the same controlled conditions for all, generally through a written examination in an examination hall As has recently been confirmed (Wagner 2004), an essential aspect of creative work is being able to sleep on it: “Our data support the concept that sleep, by hippocampal – neocortical replay, not only strengthens memory traces quantitatively, but can also ‘catalyse’ mental restructuring, thereby setting the stage for the emergence of insight.” Portfolios Portfolios are another idea that has made the transition from Fine Art and Architecture to other disciplines In principle, a student’s portfolio contains all that the student wishes to be assessed on and it thus starts with the idea of assessing what a student claims to be good at, in contrast to most examinations which all too often aim to find out the gaps and deficiencies in a student’s achievements As a rule, portfolios are built up during students’ university careers and thus document their development as well as their final achievement This makes it possible to assess processes of learning through intermediate products and to use the outcomes of these intermediate products in a formative manner This use of interim summative assessments in a formative manner is a great strength of portfolio assessment (see eg Knight and Yorke, 2004, p 106) and in striking contrast to the traditional approach which keeps formative and summative assessment strictly separate More generally, Knight (2002, p 145) divides assessment into ‘high stake’ which covers most traditional forms of assessment (see eg Johnston and Elton 2002) and is essentially Assessing creativity – guide summative, and ‘low stake’, which aims at authenticity rather than reliability, tends to be fuzzy and for that reason is used mainly formatively It is one of Knight’s major contribution to the debate between formative and summative assessment that he argues that such fuzzy assessment must play an essential part also in summative assessment and that one way to achieve this is through the use of portfolios The two main approaches to portfolio assessment – positivist and interpretivist – have been discussed by Johnston (2004) and, from what has been said earlier, the latter approach is particularly appropriate for the assessment of such matters as creativity and criticality Degree classification A serious obstacle can arise in attempting to fit unorthodox forms of assessment into regulations not designed for them, although it is often possible to bend regulations and stay within the rules as laid down by the management of a university However, this may not always be the case, particularly if the proposed form of assessment is interpretivist At that point the innovator may well be asked to produce sets of marks which can be fitted into a traditional scheme, in order to produce final degree grades The choice may then lie between shoehorning the innovation into the ill fitting constraints of traditional assessment practice and abandoning the innovation If this happens, it is necessary to give in at least temporarily, but at the same time to join the increasing band of iconoclasts who consider the award of degree classes an outdated and largely meaningless practice (Winter 1993, Elton 2004) While it is not impossible to devise assessment schemes leading to degree classifications for programmes which are either totally positivist or totally interpretivist, it is quite meaningless to that for programmes in which parts are legitimately assessed on the basis of positivist and other parts which are equally legitimately assessed on the basis of interpretivist principles As such programmes ought to be the rule rather than the exception, something has to give and it ought to be the classified honours degree 10 Practical experiences with assessment Practical experiences with creativity assessment does not seem extensive A recent workshop on ‘How can Creativity be Taught’ reported on a substantial number of instances where creativity was ‘taught’ (or rather where its learning was facilitated), but there were only two mentions of its assessment, both very negative [Jackson (ed) 2004]: “We have not in this case found a solution to explicitly assessing creativity” (L Pybus) “However, the difficulty arises in the assessment which tends to be output based.” (K Slater) This guide is clearly much needed 11 Conclusion This brief account has not been able to more than touch on the problems that arise when assessing mental capacities like creativity and criticality and in particular it has not looked at the problem of persuading traditional teachers to change their practices, evidenced for instance in the following (Oliver and Plewes 2003): Assessing creativity – guide The study has revealed a complex picture in which revisions are composites of countless small adjustments, generic information is viewed as a poor substitute for disciplinary knowledge, and enculturation plays a far stronger role than training or set texts Other difficulties are indicated in the title of Mitchell, Smith and Kenyon (2004), “It’s not for lazy students like me ” There is however reassurance in that capacities like creativity and criticality have been assessed in a small number of disciplines for many years and that they are important in all S/he who dares, wins Appendix by R Parnell The traditional ‘crit’ …if it is running well Students present their work using visual aids, providing a verbal explanation to an audience comprising staff and peers (and potentially ‘visitors’) Both staff and students offer feedback and engage in dialogue with the presenter Students are thus given an opportunity to learn from each other and build their critical skills Some limitations: The review is primarily a means of formative assessment, but some institutions use it as a way for staff to decide on the summative assessment The argument for separating summative assessment from this process is to permit the students to be less defensive and to perceive the review as a vehicle for learning rather than an opportunity to be judged Unless the format of the review is carefully structured the staff tend to dominate the discussion and marginalise student participation/peer learning This can happen simply because the staff have all taken a seat at the front Students tend to be concerned that any negative criticism of their peers will influence their marks (if tied to assessment) Students ideally need to have prior input on how to give constructive criticism as this is not a skill that necessarily comes naturally Large groups obviously introduce problems with being able to see the work being presented and being able to participate in discussion etc (the alternative - reviewing lots of smaller groups - can be staff or time intensive) References J Biggs (1999), ‘Teaching for Quality Learning at University’, SRHE & Open University Press H Canatella (2001), ‘Art Assessment’, Assessment and Evaluation in Higher Education 26, pp 319 – 326 S DeWulf and C Baillie (1999), ‘Creativity in Art, Science and Engineering: How to Foster Creativity’, Department for Education and Employment C Doidge, R Parnell, and R Sara (2000), ‘The Crit’, Oxford:Architectural Press E W Eisner (1985), ‘The Art of Educational Evaluation’, Falmer Press L Elton (1988), ‘Student Motivation and Achievement’, Studies in Higher Education 13, pp 215 – 221 L Elton (2003), ‘Seven Pillars of Assessment Wisdom’, LTSN Generic L Elton (2004), ‘Should classification of the UK honours degree have a future?’, Assessment and Evaluation in Higher Education, to be published Assessing creativity – guide N Jackson (2003), ‘Nurturing Creativity through an Imaginative Curriculum’, Imaginative Curriculum Project, LTSN (Generic) N Jackson (ed.) (2004), ‘How can creativity be taught? Personal Accounts of Teaching to Promote Students’ Creativity’, LTSN Generic Centre B Johnston (2004), ‘Summative Assessment of Portfolios: An Examination of Positivist and Interpretivist Approaches to Agreement over Outcomes’, Studies in Higher Education, in the press B Johnston and L Elton (2002), ‘Assessment in Universities: a critical review of research’, LTSN (Generic), ASS 013 P Knight (2002), ‘Being a Teacher in Higher Education’, SRHE & Open University Press P Knight and M Yorke (2004), ‘Assessment, Learning and Employability’, SRHE & Open University Press J Mitchell, J Smith and A.J Kenyon (2004), ‘’It’s not for lazy students like me ’, Engineering Education Conference , Wolverhampton, June M Oliver and L Plewes (2003), ‘Academics’ curriculum practices at UCL: a preliminary study for the Imaginative Curriculum project’, LTSN Generic R Parnell (2004) in P Leon, ‘What is your experience of teaching?’, Times Higher January, p 25 Savin-Baden, M (2000), ‘Problem-based Learning in Higher Education: Untold Stories’, SRHE and Open University Press B R Snyder (1973), ‘The Hidden Curriculum’, Boston: MIT Press U Wagner et al (2004), ‘Sleep inspires insight’, Nature 427, pp 352 – 355 Acknowledgments Thanks are due to Sharon Morris (University College London), Tim Dunbar (Manchester Metropolitan University) and Rosie Parnell (University of Sheffield) for their help; all responsibility is mine 3733 words Assessing creativity – guide ... criteria to be considered in assessing a creative piece of work, such as relevance, difficulty, cost etc Clearly, it is in general neither possible nor desirable, to assess a piece of creative work. .. allowed little freedom This leads to negative attitudes towards a change to a creative curriculum – far more than the perceived inappropriateness of a creative approach to the study of a particular... assessment (see eg Johnston and Elton 2002) and is essentially Assessing creativity – guide summative, and ‘low stake’, which aims at authenticity rather than reliability, tends to be fuzzy and for