Teaching Today A Practical Guide Fourth Edition - part 9 potx

496 Summative, or terminal, assessment takes place at the end of a module, course or academic year. As mentioned at the beginning of the previous chapter, the aim may be to sum up what the candidate can do (criterion referencing). This might be done with the aid of a checklist of skills or competences, and/or by reports or profi les. Alternatively, the aim may be to grade candidates, or place them in a rank order (norm referencing). This is usually done by means of an examination, designed to differentiate between candidates on the basis of the breadth and depth of their learning. Summative assessment in school and post-school education is in the middle of a turbulent revolution at present. Practice varies from subject to subject, but it also varies from year to year for any given subject. I will give only a brief outline of the basic methods here. It is vital that you discover the detailed requirements of assessment for the courses on which you teach, and any personal responsibility you may have in this respect. Do this as soon as you know what you are teaching, and ask experienced teachers for advice. Methods used for summative assessment: a brief outline Profi les Everyone is familiar with the school or college report. Profi les and records of achievement extend reporting to include a systematic coverage of the learner’s achievements, abilities, skills, experiences and qualities. As mentioned in the section on self-appraisal in the previous chapter, they can be used formatively, as well as summatively, and are commonly used for both. Like any report or refer- ence they are subjective, but they give information which cannot be measured objectively. They typically report only positively, and are written by the learner, but drafts are agreed by the teacher. They give information on the learner’s: personal and social development, self-awareness and social skills• attainment progress and motivation• career aspirations• interests and hobbies, both in and out of school or college• achievement in key skills such as problem-solving, communication, informa-• tion technology, numeracy or other skills, such as manual dexterity, etc. Ideally, the students should be self-assessing and setting themselves targets for improvement as described in Chapter 34, with the profi le acting as the outcome of this process. Profi les are the property of the learners, and can be used, if they wish, when seeking employment or educational progression. 44 Summative assessment P04.indd 496 2/3/09 16:45:47 Summative assessment 497 Some profi les are in a grid format, listing core skills and achievement in those skills in terms of hierarchically ordered descriptors. This may mislead readers of the profi le into believing the assessment was more objective than it really was, and the format is too restrictive to allow adequate description of more subjective criteria. Ticking a box labelled ‘Can present a logical argument’ is meaningless unless the context is clear, as everyone can present a logical argument at some level. Other profi les are in an open-response format: effectively, they are a series of headings under which the student records their accomplishments. Combinations of the grid and open formats are common, and profi ling design and practice vary markedly from institution to institution. Profi les have been criticised by teachers for the work they generate; for lacking validity and being unreliable; and for giving unrealistic impressions, in that they report only positively. Some commentators doubt whether employers read the longer grid-style profi les. However, since the learner’s academic achievements often make it very clear what the learner cannot do, it seems fair to redress the balance with a profi le, especially as the self-assessment involved is so valuable. If you use profi ling, make sure the learners do as much of the work as possible! Competences Assessment can also be carried out on the basis of checklists or a set of compe- tences; this is a widely used method where a criterion-referenced assessment is required. The achievement of these competences is usually on a ‘passed’ or ‘not passed’ basis. Re-attempts are encouraged when a pass is not attained. Competences are the method used to defi ne the content and organise the assess- ment of National Vocational Qualifi cations (NVQs). Let’s take as an example an NVQ in horticulture, which addresses the ‘key purpose’ of ‘providing ornamental beds and borders’. The units of competence in this NVQ might be: Produce plants from seed.• Establish ornamental beds and borders.• Maintain ornamental beds and borders.• Each of these ‘units of competence’ has a number of ‘elements of competence’. For example, the unit ‘Maintain ornamental beds and borders’ might have the following elements: Weed a bed and/or a border by hand, with or without the aid of tools.• Maintain the appearance and health of plants.• Maintain the soil condition and physical appearance of the bed or border.• Each of these competences can be assessed separately, or in any combination, at any convenient time by an accredited assessor. The assessment of each compe- tence is then checked by an external verifi er from the awarding body offering the qualifi cation, such as City & Guilds. The ‘scope’ of a competence is usually given. For example, a competence such as ‘Assist with planting ornamental plants’ might be given the following ‘scope’: ‘Container-grown shrubs, herbaceous plants, bedding plants and bulbs’. P04.indd 497 3/3/09 15:04:56 498 The learner, or ‘candidate’, submits evidence to an assessor in an attempt to demonstrate the attainment of a particular competence or competences. This might involve the candidate being observed. If the assessor agrees that the compe- tence has been achieved, it is ‘signed off ’ by the assessor; if not, any further work required to demonstrate the competence adequately is usually made clear. Such competence-based schemes have the advantage that they set realistic work- based standards, agreed by experts in the vocational fi eld (the industry’s ‘lead bodies’). Hence they ought to have the support of the relevant industry. They are accessible to learners in work, in that they encourage (indeed, may even require) work-based evidence and do not require the candidate to attend a course. Past evidence of skills can be used to meet competences by a process called ‘Accredi- tation of Prior Learning and Experience’ (APL/E) – though this can be a time- consuming and costly process. Criticisms of NVQs include the suggestion that the lack of grading means both that the able are not stretched, and that potential employers have no means of using NVQs to differentiate between candidates with the same qualifi cation. Some say they lower standards, and put too little emphasis on the candidate’s understanding of the skills and techniques assessed, but good teaching can overcome this diffi culty. NVQs are overseen and ‘kite-marked’ by the QCA (Qualifi cations and Curriculum Authority). There are fi ve levels: Level 5: professional Level 4: for people in a supervisory role Level 3: roughly A-level standard Level 2: roughly GCSE standard Level l: basic level – introductory. Do not let yourself be tyrannised by competences. It is almost never a good idea to teach a course competence by competence, or even unit by unit. Teach fi rst, get your candidates to do real work in a real context, then look for evidence for assessment. If your tasks and course are well designed, the assessment will fall into place quite easily. For example, some horticultural students could be given the task to design, stock and maintain a fl ower bed. Photographic records could be taken and they might write about their experience, putting references in the margin as to the competences they are claiming for each part of the job. This holistic experi- ence is much more natural and meaningful than picking off competences one by one in the order given in documentation. Continuous assessment or coursework This is the process by which work done during a course is assessed as part of the learner’s summative assessment. Most full-time vocational courses, and part of some GCSEs and A-levels are assessed in this way. Like most developments in education, this ‘internal assessment’ is more work for the teacher, though it does have the advantage of increasing student motivation considerably. In many instances, the assessment conditions, and therefore its fi ndings, are much more realistic – who, for example, would want to write a poem or complete an engineer- ing design in one time-constrained sitting? Putting it all together P04.indd 498 2/3/09 16:45:47 Summative assessment 499 Don’t assess key or ‘common’ skills such as ‘problem-solving’ or ‘working with others’ without teaching these skills! In order to ensure that internal assessment has been carried out in the prescribed manner – and to the same standard – in different schools and colleges, an external verifi er or moderator will usually view all or part of the marked coursework (see Chapter 49). Again, procedures vary greatly, and it is imperative to ascertain quickly exactly what is expected of you. For example, when must coursework be submitted to the moderator or verifi er? The examination board or validating body will provide written guidance on such matters, but some of this material is famously voluminous and opaque. Seek guidance from an experienced teacher in the fi rst instance. Examinations Make sure you are aware of the form of the fi nal examination, and ensure students have had some months of practice in answering papers of the appropriate type. Past papers are available from the appropriate board and are an excellent homework source. Examiners’ reports on past papers are published by some examining boards, and these give valuable information on common mistakes and omissions made by candidates. If students fi nd past-paper questions daunting, it is often because they fi nd the language used to frame the questions diffi cult. Give them a glossary, and play ‘deci- sions, decisions’ games to develop their ability to tell an evaluation from a descrip- tion. Work through past-paper questions yourself on the board; then do some as worked examples with the class volunteering the answers; then let them loose on a few questions in pairs. Make use of spoof and peer assessment. Even so, they may take some months to gain confi dence. If the fi rst time they see a past paper is in their mock or practice examination, their marks will be a big disappointment. Graded tests Graded tests use the mastery-learning philosophy for summative assessment. A pioneer in this fi eld has been the Graded Objectives in Modern Languages (GOML) movement, with tests similar in principle to the music examinations of the Associ- ated Board of the Royal Schools of Music. The GOML tests are criterion referenced, taken when the learner is ready, and can be retaken. They have been popular with students, aiming to provide the frequent positive reinforcement of certifi cated success through the setting of attainable short-term goals. Psychometric tests Psychologists have devised special tests to measure intelligence quotients, verbal reasoning, non-verbal reasoning, manual skills, basic skills in reading P04.indd 499 2/3/09 16:45:47 Putting it all together 500 and arithmetic, etc. Other tests measure aptitude – for example, the candidate’s aptitude for learning how to use a computer, or their mechanical aptitude. Yet more tests claim to measure personality, and to indicate whether a candidate would be suitable for management training. Such tests can be expensive, and most require special training in their use. It is generally recognised that there is a danger in relying too heavily on the results of such tests since, for example, they do not take motivation into account. Moreover, the results, are not as stable as is sometimes claimed; education can raise IQ scores by as much as 30%. Research reviews, like that of Ericsson et al. (1993), have shown that potential or aptitude is very hard to measure and that ability is more learned than innate (Chapter 45). Question styles Here is some advice if you are about to write examination questions. Don’t, if you can possibly help it! Writing examination questions, especially objective test items (multiple-choice questions), is very time-consuming. Why reinvent the wheel? Try to obtain a store of past papers, and also to fi nd internal papers used in your school or college in previous years; rifl e textbooks or books of questions. Adapting these saves time. Be clear on the purpose of your examination. Is it to grade and differentiate, or to diagnose learning problems? Are your questions fi t for your purpose? All questions should be clear, concise and unambiguous, and written in everyday language. This is harder than it sounds, and it’s easy to make a slip, so if you adapt or write questions, it is worth getting them checked by another teacher. The diagram below shows issues related to the type of questions you use. Problems with assessment Validity The validity of an assessment depends on whether it actually measures the knowl- edge or skills it is designed to assess. For example, an objective test cannot measure a candidate’s practical skill, or his or her ability to develop a coherent argument. Characteristics of question types P04.indd 500 2/3/09 16:45:47 Summative assessment 501 To be valid, an assessment must also sample across a large proportion of the topics on the syllabus, and sample all the appropriate levels in Bloom’s taxonomy. The breadth and depth of learning sampled by the assessment must be correctly weighted in the marking. Validity is also compromised if questions are diffi cult for the candidates to under- stand, or are culturally biased. It is common for teachers to confuse poor learning with a student’s diffi culty in understanding examination questions. Reliability In public examinations, different examiners should award the same mark to the same script, and each year’s paper should award the same grade to a student of a given standard. In addition, the same examiner should give the same mark if they unknowingly mark a script twice on different days. In practice, perfect reliability is impossible to achieve, and, in particular, essay questions are less reliable than objective test questions. ‘During the 1960s schools were regularly putting pupils in for the same subject with different boards and getting totally different results.’ Peter Newsom, Times Educational Supplement, 16 October 1992 The 11-plus was (is, in a few areas) a much more reliable examination than most. Yet the defi nitive research published by the National Foundation for Educational Research in England and Wales in 1957 showed that, with the same pupils taking the same examination after a period of a few days, something like 10% ‘passed’ on one occasion and ‘failed’ on the other, and vice versa. The reliability of examinations is considerably increased by the use of carefully designed marking schemes which allot marks on objective criteria, rather than leaving the mark to the general impression of the examiner. ‘You will never amount to very much.’ Comment made by a Munich schoolmaster to a 10-year-old pupil called Albert Einstein P04.indd 501 2/3/09 16:45:47 Putting it all together 502 VACSAR ‘VACSAR’ stands for valid, authentic, current, suffi cient and reliable. As well as valid and reliable, as discussed above, assessments need to be: authentic• – if you want to measure a student’s ability to design, it would not be realistic to give them a design problem to solve in half an hour. That’s not how designers work. current• – an electrician may be used to out-of-date regulations, but can he or she work to the new ones? I could give mouth-to-mouth resuscitation last year – can I do it now? suffi cient• – how much writing must a student do before we can make a judgement on their ability to spell? Developing an assessment strategy Every course needs an assessment strategy. This should be related to the aims and objectives for the course, and should respond to considerations such as: What are the purposes of your assessment: to grade or to diagnose? (You may • also assess to motivate, get feedback, acknowledge progress, certifi cate, select learners, evaluate courses or some combination of these.) What is to be assessed? How is it to be done?• Who will assess whom, and when? Are marks required for reports or modera-• tors? What will happen as a result of the assessment, particularly to those who have • done badly or very well? Once the strategy has been decided, methods appropriate to this strategy, and to the aims and objectives of the course, need to be devised. For example, a computer training course for adults of varied experience may choose to use a checklist of competences that learners tick off for themselves. A school mathematics teacher may decide on a series of mastery tests and a grading examination at the end of the year. A course to develop counselling skills may use learning journals, and have periodic one-to-one tutorial sessions where issues in the journal are discussed. As always in education, the choice is made on the basis of fi tness for purpose and value for effort. Devise mark schemes along with the tests, and keep them safe for use next year, along with a monitoring copy of the paper, on which you can write suggested amendments. This enables you to improve the assessment process, and allows comparison of students from year to year. Saving and amending tests and mark schemes takes organisation, but it saves many precious hours of work. MARK SCHEMES • Contrary to fable, it is unusual to give most marks for the more diffi cult parts of a question or paper, as this strongly biases the test in favour of the most able. It is usual to apportion marks on the basis of the likely time taken by the candidate to complete the answer. P04.indd 502 2/3/09 16:45:48 Summative assessment 503 You will, of course, need to keep records of your assessments; fi nd out what has been done before. Don’t keep more records than you need. Chapter 41 shows different approaches. Coda Because only the measurable can be reliably assessed, much of importance is usually ignored by the assessment process – and therefore, all too often, by the teaching process. Both teachers and students tend to the pragmatic view: ‘If it’s not assessed – ignore it’. And so the assessment tail is rightfully accused of wagging the dog. At least a third of young people emerge from school branded as failures. The emotional damage infl icted on our children and young people by this process can only be guessed at by people like you and me, who for the most part have succeeded in our learning. Some of these ‘failures’ go on to reject the norms of the society which has rejected them, and pass into a twilight world of Giros, drugs, petty crime and imprisonment. It is no accident that over 50% of those in prison are functionally illiterate, in many cases as a result of dyslexia that was not adequately diagnosed or attended to. Failure also has its economic consequences. Advanced economies like ours cannot compete on the world market with cheap labour, but only with the skills passed on by education and training. In 1993, the Audit Commission reported that less than 50% of 17-year-olds were in full-time education. They found that one-third of those in education either dropped out of their courses or failed them. The situation has improved slightly since, but the introduction of new vocational qualifi cations has not been as helpful as you might think. These qualifi cations are vocational by name, but academic by nature, and so offer little to students in search of an alternative to ‘book and biro’-based education. This social, psychological and economic damage is due in large part to a curricu- lum which is heavily academic (see pages 125–6); to norm- rather than criterion- referenced assessment, with a consequent bias towards the achievements of the able; and to a tendency not to recognise and reward qualities which are diffi cult to measure. Try not to mirror these mistakes in your own assessment. Whatever • Candidates should be aware of how marks are allocated. • Work out solutions to numerical problems in advance, to ensure the ques- tions are possible and valid. • Mark a very good script fi rst to check the mark scheme, and your answers! • If you wish to grade or discriminate, set a large number of moderately diffi cult questions, rather than a small number of very hard ones. P04.indd 503 2/3/09 16:45:48 Putting it all together 504 the summative assessment of a course, for formative assessment consider using competence-based systems, profi les, graded tests and other mastery methods. These reward the effort and successes of every learner, and encourage the self- belief on which future learning relies. Remember that formative assessment has much more impact than summative on learning. ‘Not everything that counts is countable, and not everything that is countable counts.’ Albert Einstein Checklist for your assessment system Is your assessment system related directly to the aims and objectives of the ❏ course? Do you use frequent diagnostic tests to discover weaknesses in learning? ❏ Do your students make efforts to overcome the above weaknesses? ❏ Are students allowed to improve and resubmit inadequate work? ❏ Do your methods recognise the efforts to learn which the less able are ❏ making? Do your methods stretch the able and recognise their achievements? ❏ Do your methods recognise and reward important qualities which are ❏ diffi cult to measure objectively? If you use mastery methods, are your tests easy enough? ❏ Further reading Bloom, B. S., Madaus, G. F. and Hastings, J. T. (1981) Evaluation to Improve Learning, New York and London: McGraw-Hill. Ericsson, K. et al. (1993) ‘The role of deliberate practice in the acquisition of expert performance’, Psychological Review, 100, 3: 363–406. Gipp, C. and Stobart, G. (1993) Assessment: A Teacher’s Guide to Issues (2nd edition), London: Hodder and Stoughton. Rowntree, D. (1987) Assessing Students: How Shall We Know Them? (2nd edition), London: Kogan Page. EXERCISE Compile a summary of the advantages and disadvantages of the main methods of assessment outlined in this and the previous chapter. P04.indd 504 2/3/09 16:45:49 P04.indd 505 2/3/09 16:45:49 [...]... Peace Laureate 514 Values and what teachers can achieve adapt to the diverse nature of our students Further reading Adey, P and Shayer, M ( 199 4) Really Raising Standards: Cognitive Intervention and Academic Achievement, London: Routledge An overview of cognitivist approaches to teaching intelligence’, including a useful section on Feuerstein Coles, A (ed.) (2004) Teaching in Post-Compulsory Education:... learned I have a summarising paper on this: do email from my website for a copy Gray, J ( 199 9) Improving Schools: Performance and Potential, Buckingham: Open University Press Hattie, J A ( 199 9) Influences on student learning This can be downloaded from Professor John Hattie’s staff home page: www.uoa.auckland.ac.nz/education/staff/ cfmj.hattie/j.hattie-homeeducation See also www.geoffpetty.com Martinez, P... effective, but also more adaptable and better able to solve problems Effective teachers are always changing what they do; this is because they are continually learning how to teach better Learning and teaching are not simple, and if your experience is anything like mine, you will never arrive at a final decision about the nature of learning and teaching, but will continue to develop your understanding all through... materials and methods, for which they were specially trained Two years after the programme had ended, the students were found to have average IQs (around 100) and to be quite independent They had started Feuerstein’s programme with a mental age three years behind the average Values and what teachers can achieve and had now caught up! Some of his ex-students have since become university lecturers and... Those teaching adults basic skills such as numeracy and literacy will go about it in a radically different way from those teaching English in schools, or A- level business studies in a college For example, the basic skills teacher will have control over what they teach, and can mould it to each individual learner’s goals, as described in Chapter 34 on selfdirected learning If, for example, the learner... support is adequate, and to increase it until it is sufficient This flexibility explains in part why some colleges, even though they serve the most socially and economically deprived areas, have an audited average achievement rate (pass rate) of over 90 per cent (Martinez 2000) 5 09 The professional in practice Many teachers believe that whether students pass or fail depends on their innate characteristics,... the same lesson plan again next year Note that in this case there is no attempt to learn the general principles of effective teaching from the lesson Example 2 Learn 2 ‘I suppose the main reason the lesson was successful was the classification game They enjoyed it, but it also enabled me to check their learning And it made them think It also made them apply their learning, but in a playful way And they... career achievements, even that of academics, and Dweck cites a Nobel Prize winner in science with a very unremarkable IQ I am not arguing that there is no such thing as ability or talent, but that these attributes are largely learned This is a difficult message to accept in the West, where our culture assumes that exceptional ability is a gift However, the evidence is emphatic that ability can be learned... today is that Feuerstein, Ericsson and others have shown us that huge improvement is possible You will play a part indeed – as a teacher, the most important part of all – in realising this potential What values should guide, inform and inspire you as you square up to this challenge? EXERCISE 1 Work alone for a few minutes and write down the main values you bring to teaching The following questions may... career You will never ‘arrive’, but that is not a problem, as the journey is so fascinating Pablo Casals, one of the world’s greatest cellists, was asked why he still practised at the age of over 80 He answered, ‘Because I think I am beginning to make progress’ Learning never stops, especially for the most able, and it was this inclination to learn that made them so able in the first place 520 Evaluating . examiners should award the same mark to the same script, and each year’s paper should award the same grade to a student of a given standard. In addition, the same examiner should give the same. VACSAR ‘VACSAR’ stands for valid, authentic, current, suffi cient and reliable. As well as valid and reliable, as discussed above, assessments need to be: authentic• – if you want to measure a. alternative to ‘book and biro’-based education. This social, psychological and economic damage is due in large part to a curricu- lum which is heavily academic (see pages 125–6); to norm- rather