School - University Partnerships for Science and Mathematics Reform,

Final Report (submitted to Arizona Board of Regents in August 2004, updated slightly in May 2012): School - University Partnerships for Science and Mathematics Reform, Year A grant for $48,900 funded in February 2003 by the Arizona Board of Regents: Improving Teacher Quality Program Written by P.I Jane Jackson, Faculty Associate, Dept of Physics & Astronomy, ASU 480-965-8438, Jane.Jackson@asu.edu The Co-P.I was David Hestenes, Research Professor of Physics & Astronomy, ASU Summary: School - University Partnerships for Science and Mathematics Reform was the second year of a pilot project to establish the Arizona Science and Technology Education Partnership (AzSTEP) as a statewide program to provide K-12 schools with the resources they need for highquality, systemic reform in science, mathematics and technology education Horizontal and vertical integration of science and mathematics was enhanced in the second year of establishing a district-wide learning community of teachers of junior high science and mathematics in Mesa Public Schools (MPS) Twenty-three teacher participants, apprentice leaders, and mentors from the previous year's workshop participated in a 3-week summer workshop in physical science with mathematical modeling The workshop included thematic strands in scientific modeling, structure of matter, energy, and use of calculators and computers as scientific tools Mathematics instruction was coupled to this thread through an emphasis on mathematical modeling MPS science and mathematics specialists coordinated recruitment Workshop leaders led follow-up meetings Increased content knowledge and better instructional strategies of teachers correlated with measured improved learning of students I Overview of the Modeling Workshop: project design and procedures Physical science and mathematics teachers in 11 of Mesa's 13 junior high schools were selected by Mesa's science and mathematics specialists, Rick Vanosdall and Sandra Nagy, for their interest, ability and experience Most selected teachers were of grades and 9, because the workshop curriculum is considered to be too hard for th graders, and the 7th grade curriculum is mostly life science About two-thirds of the 23 participants teach science, and one-third teach math Two participants teach industrial technology, which use math and science for technical applications Three of the 23 participants repeated the workshop, having previously attended in summer 2002 School-level teams of a math and a science teacher were sought, because research shows that the best follow-up is daily interaction of teachers about the reform Few teams could be found However, some interaction was possible, because 23 teachers had participated in the previous summer from most of the same schools Support of principals was enlisted by Rick Vanosdall and Sandra Nagy Some principals were already aware and supportive The specialists organized a morning workshop in March for principals, led by workshop leaders Larry Dukerich and Stella Ollarsaba, to acquaint them with the project design, involve them in a modeling activity, and solicit their support They were reminded of the value to their school because improved coordination between science and math teachers will lead to better performance on the AIMS test, since students will have opportunity to learn math concepts in a physical context Workshop leaders Larry Dukerich and Stella Ollarsaba invited workshop participants from last year to serve as apprentice leaders or mentors, to establish continuity and enhance the reform Summer activities started the week of June with re-design of the previous summer's workshop materials and evaluation instruments (PSCI and MCI) by the two apprentice leaders and Larry and Stella to align with Mesa's standards and state standards in mathematics The workshop itself was held from June – 27, 2003 at Dobson High School in Mesa Apprentice leaders conducted parts of some sessions under guidance of Larry and Stella Teachers met daily for hours; included were mentors on a few days Most participants, including a few teachers in the previous year's workshop, gave the revised Physical Science Concepts Inventory (PSCI) or Math Concepts Inventory (MCI) to a class this year We wanted to learn if their implementation improved during their second year, because we find that, in physics, implementation usually improves for or years Unfortunately for the project, Rick Vanosdall left the district in the summer for another job He had been enthusiastic about this project, and he had planned to assume leadership in the fall His guidance was sorely missed because he and Sandy Nagy were best positioned (1) to coax teachers in each school to collaborate, and (2) to work with principals to implement short inservices During the academic year, teachers met informally in small groups in their schools and in a large group on two Saturdays with Larry and Stella to deepen the learning Teachers questioned and debated, and shared materials, methods, and reflections on progress They evaluated effectiveness of instructional materials, and modified and re-designed curriculum materials for future use The 23 participants from the summer 2002 Modeling Workshop were invited, and most attended Teachers were encouraged to request classroom visitations and informal advice from apprentice leaders and mentors Larry and Stella reported on the commitment of participants in implementing their workshop learning, and they provided informed feedback and support to participants For long-term professional development, teachers were subscribed to a local junior high school modeling listserve managed by Modeling Instruction staff Contact time was 72 hours, plus individual work (readings, written reflections, learning technology, adapting instructional materials for their courses), totaling about 135 hours Teachers earned semester hours of graduate work in physics or in math education - their choice II Recommendations for Improved Modeling Workshop implementation We recommend that the 40 Mesa Public Schools teachers who have learned Modeling Instruction have additional follow-up and support, to retain and deepen their learning Most of the following actions can be spearheaded by district math and science specialists • Mentors should be given specific training Larry Dukerich gave a vision in his e-mail of June 4, 2003 He wrote me, "We met with three of the mentors (Carl Harms, Jacob Davis and Tony Garibay) yesterday and hope to see the others today They seem very intrigued by the expanded role they will play You see, for the longest time in Mesa, you either put on a workshop or attended one There was no one encouraging you to actively work with your colleagues to continue the work started in the workshop It was like • • • • • • • • planting a seed and walking away Here we are watering it and nurturing it This is very different and will require encouragement on the part of you and Stella and me to remind these mentors (and leaders) that they can and should take an active role in bringing about reform Speaking of that, I would like to give each of the participants a copy of "Before It's Too Late", the final report of the Glenn Commission I found that it clearly articulated most of the arguments that teachers would like to make (but find themselves unable to) when talking with administration about what needs to change You should have heard Jacob - that boy is a ball of fire and really wants to see change take place He suggested it would be a good idea to let the math and science teachers break away during a late-start day to review what was learned in the workshop - pretty neat that he came up with the same idea we proposed, no? Well, the steps outlined in the Glenn Commission report are the kind of ammunition teachers need when trying to convince administration to change." Teachers should re-design curriculum together, with guidance from Modeling Instruction Project faculty at Arizona State University Teacher networks are the most effective means for consolidating and sustaining reform (Adams, 2000) Thus teams of math and science teachers in each school should meet to plan some lessons together, discuss how they can use modeling strategies, and observe each other's teaching; they could report to science and math coordinators in writing on this Also, they should study samples of student work MPS science and math specialists can coax teachers to collaborate Workshop leaders (Larry and Stella) should be given release time and substitutes to visit selected classes of workshop participants, or vice-versa, to help improve implementation Admittedly, this is difficult, given the present organization of a teacher’s day; but it is important 'Best practice' videos of modelers should be viewed by teachers Two such 20-minute videos of Larry Dukerich's teaching were made in 2003 at ASU in Ann Igoe's and James Middleton's NSF Preparing Tomorrow's Teachers in Technology-PT3 grant James.Middleton@asu.edu Streaming format: http://vimeo.com/channels/modelingphysics Explicit instruction should be given to under-prepared teachers on conservation of weight (or mass), conservation of volume, proportional reasoning and other rational number concepts (see section below on measurement of teacher learning) All teachers should be given instruction on how to teach thinking skills such as conservation of weight and volume, proportional reasoning Some resources for this are by "Thinking Science" developer Adey (1999) and Adey (2003) We suggest that you buy each science teacher a copy of Anton "Tony" Lawson's book, Science Teaching and the Development of Thinking (Wadsworth, 1995 & 2002; paperback & hardback) The revised edition is expected to appear in 2005 The book is the text for BIO480, "Methods of Teaching Biology", which is a senior level course but could perhaps be cross-listed as a graduate course and a special section be organized for inservice teachers: You could form study groups with that book as focus Principals should be asked to try to provide common planning time for teams Principals should be asked to target some monthly school late-start days for an inservice for all math/science teachers, led by workshop participants Additionally, workshop participants could lead school-wide inservices on whiteboarding and discourse • Principals should be made aware that implementation of modeling instructional materials in the 8th grade science course will provide students with skills necessary to succeed in physics if the district eventually moves to a physics-first sequence (i.e., mostly physics in grade 9, mostly chemistry in grade 10, biology and advanced courses in sciences in grades 11 and 12) III Measurement of teacher learning A Physical Science Concepts Inventory All 23 teachers took the Physical Science Concepts Inventory (PSCI) pretest (version 7) on the first workshop day and the posttest on the last workshop day Teachers' pretest mean was 76%, posttest mean was 83% (pretest median was 80%, posttest median was 88%) These scores are higher than the previous year's group, when the Mesa teachers’ pretest mean was 69% and posttest mean was 80% An earlier version of the PSCI had been used, but most questions were the same Appendix A is a graph of each teacher’s scores Seven teachers were initially low-performing Two of these teachers improved tremendously; however, the other did not improve (their scores remained at 50% to 65%) This is troubling, for they lack basic reasoning skills that are crucial for math and science, as we see in the following analysis The content subscales of PSCI on which teachers (specifically, this group of lowestperforming teachers) did badly were these: • Conservation of weight or mass (questions & 2): teachers got these wrong on the posttest This is shocking, for most 10-year old children have mastered this thinking skill, according to Arizona State University Professor Anton "Tony" Lawson, whose research field is scientific thinking skills (private communication; also, Lawson and Bealer, 1984) (Tony Lawson developed and teaches the Learning Cycle Modeling Instruction is a refinement of the Learning Cycle; it is the Learning Cycle structured around scientific and mathematical models.) • Conservation of volume (questions & 4): teachers got #3 and #4 wrong on the posttest This is 1/4 of the class; a very disturbing situation, because most 11 to 12-year old children have mastered this thinking skill! (Anton Lawson, private communication, June 2003.) • Proportional reasoning (questions 5-8): teachers got these wrong on both the pretest and posttest This is one-quarter of the class; a disturbing situation because proportional reasoning is crucial in math and in physical science! • Energy & states of matter (4 questions): 14 teachers (60% of the class) still had misconceptions at the end of the workshop This content was the last focus of the workshop, so the poor results aren't surprising because teachers hadn't had time to reflect on their new learning, which was novel Larry wrote me, "Stella told me that I was a slave driver, as I pushed the teachers to complete the materials, knowing that the energy treatment would be novel for most of them." Regarding the 25% of the class who failed to proportional reasoning: in December 2002, after I related similar results from the previous workshop to Larry Dukerich, he wrote to me, "The questions borrowed from Lawson's Classroom Test of Formal Reasoning (items 1-8 on test v5) have been banging around for quite some time The majority of the teachers already knew how to items 5-8 Those who did not, showed only slight gains primarily because there was little instruction on proportional reasoning in the workshop." Other researchers have found similar gaps in middle school teachers' understanding For example, referring to teachers' understanding of rational number concepts, Behr, Harel, Post and Lesh (1992) reported, "In our recent survey of over 200 intermediate-level teachers in Minnesota and Illinois, one-quarter to one-third did not appear to understand the mathematics they were teaching (Post, Harel, Behr, & Lesh, 1988)." Clearly, explicit instruction on conservation reasoning, proportional reasoning and other rational number concepts must be included in future workshops As Behr et al say, "Although there is no indication that teachers cannot learn these concepts, large-scale in-service in these areas becomes a logical necessity." (ibid) We offer a companion course, "Integrated math and science for middle school", that focuses on these thinking skills Most teachers achieved well on subscales of graphing skills/motion, geometrical & physical properties of matter, and atomic structure of matter In fact, their achievement on atomic model of matter was MUCH better than the previous group's achievement Larry Dukerich was aware that he needed to improve this component Apparently he did; congratulations to Larry! B Math Concepts Inventory All 23 teachers took the MCI pretest (version 4) on the first workshop day and the posttest on the last workshop day Teachers' pretest mean was 79%, posttest mean was 84% (pretest median was 76%, posttest median was 84%) Appendix A is a graph of each teacher’s scores The first 10 questions on both instruments (PSCI and MCI) were the same (they are questions #1 to 10 of Anton Lawson's Classroom Test of Formal Reasoning) Teachers answered these 10 questions the same way on each test, a sign that they took the tests seriously We find it alarming that five teachers scored low on both pretests and both posttests (their scores remained at 50% to 65%) Two of these teach science, two teach math, and one teaches both subjects From the test scores, it appears that these teachers don't understand the math and the scientific reasoning skills that they are teaching The problem is apparently not in the workshop design, for other teachers with low pretest scores improved dramatically on the posttests Current versions of the PSCI (vs 8) and MCI (vs 7) (which don't include Lawson's questions #7 and on proportional reasoning, for they are deemed too hard for junior high students) are at http://modeling.asu.edu/MNS/MNS.html The password to open them can be obtained from Jane.Jackson@asu.edu IV Follow-up meetings and implementation On Nov 15, the first of two Saturday morning meetings was led by workshop leaders Larry Dukerich and Stella Ollarsaba Amazingly, 29 teachers participated! Among them were several of the 2002 Modeling Workshop participants The agenda was: * break into subject groups (math & science), discuss progress, fill out whiteboards to report to main body *reconvene as large group – show and tell lessons we’ve tried or plan to try * discuss how to effectively work together in the building – how to increase communication – how to modify the workshop content to make it more meaningful to junior high teachers Let’s be positive! * divide into groups to review use of Graphical Analysis software or graphing calculator Teachers were positive and enthusiastic, reported Larry Dukerich The second Saturday meeting was held on Feb 21 The group discussed implications of the new draft science standard Teachers and workshop leaders thought that it would be useful to prepare a document representing the views of a significant fraction of Mesa junior high teachers Prior to the November 15 meeting, 21 teachers e-mailed to me a report on their implementation A summary of their responses follows Keep in mind these facts: 1) 7th and 8th graders have only semester of science 2) 7th grade science was mostly life science, with a couple of weeks on phases of matter 3) Motion was taught only in 8th grade, and it was taught after mid-November, in only weeks, including instruction on Newton's laws Energy and atomic nature of matter weren’t in the th grade curriculum 4) In 9th grade science, about 3/4 of the year focused on chemistry, including atomic nature of matter 5) Energy wasn’t taught in-depth in grades 7, 8, or A serious omission! 6) District math and science curricula were jam-packed full of required objectives 7) 9th grade alternative algebra is double period, for low achievers in math Thus implementation of teaching methods is expected to be more evident than that of content SUMMARY OF TEACHERS' REPORTS ON IMPLEMENTATION • Median # years experience teaching junior high was Range was from to 31 years • 13 teachers gave the PSCI to students, most to grade Three teachers who gave it to grade students were in their 2nd year of implementation; one teacher of grade was in nd year of implementation • teachers gave the MCI, most in grade Two gave it in grade alternative algebra (double period) Answer on scale of to (1= not at all or insignificant, = somewhat, = fully or substantial) CONTENT: To what extent have you already implemented the units this semester? • Unit 1: models of measurement Half of the teachers wrote (somewhat) No difference between math and science teachers Range to • Unit 2: motion Most teachers wrote (not at all) math teachers & science teacher wrote Range - • Unit 3: structure of matter/energy 9th grade science teachers wrote or Most others wrote METHODS:To what extent have you used this semester: • whiteboarding? 50% of the teachers wrote or 40% wrote • Socratic questioning? 45% of the teachers wrote or 45% wrote • circle whiteboarding? 20% of the teachers wrote or Half wrote • cooperative groups? 60% of the teachers wrote or 25% wrote COORDINATION OF MATH AND SCIENCE: To what extent are you coordinating your math or science course with your colleague(s)? (so that the courses enhance each other, and thus students learn more) 75% wrote (not at all) Range - SUMMARY: Overall, to what extent has the Modeling Workshop enhanced your teaching? 25% of the teachers wrote or 50% wrote Teachers voluntarily made extensive comments about their implementation Most comments were similar to those expressed by the previous year's workshop group (see that ABOR report) We repeat them here because they are informative and typical * "The workshop brought about a change in my teaching style This is shown in the increase in my students’ enthusiasm for science They hardly ever work alone now They are much more engaged My students’ test scores also show improvement." * "Big whiteboards really promote interaction between the students While one is writing, the others are helping and checking." "While working in groups, the students are debating and critiquing each other Oral and written communication skills are being improved." * A math teacher was so enthused that she and her principal made sure that every one of the math teachers in her school had a classroom set of whiteboards * They all felt that they knew enough about whiteboarding that they could implement this aspect of modeling in their classrooms * "The workshop experience changed the way I look at science in general I find myself looking for the models in the rest of the th grade science curriculum It helps me realize the way students learn so that I am a more effective teacher." * Several teachers reported that they are “converting” colleagues who didn't participate * The level of the activities was too high for 7th grade students * Everyone expressed the concern that what was expected of them (either preparing for district tests or conforming to the existing curriculum) hindered their ability to implement as much of the materials as they would have liked This was the greatest roadblock to implementation! * Several teachers commented that it would have been much better if a team of math and science teachers from their school had participated in the workshop The Nov report included several more questions Teachers responded as follows (in both years): • groups of students especially well served by Modeling Instruction are lower ability kids concrete learners; shy, bright kids, ELL (visual representations; input from other kids; whiteboarding helps a lot with vocabulary!); gifted students (they can use software without supervision); special needs students special ed (graphing skills increase their self-esteem), and girls (they enjoy science more) • Obstacles to implementation are: modeling takes time; too much hassle to go to the computer lab to use MathWorlds or Graphical Analysis software • One teacher used SimCalc MathWorlds software for multiple periods Most didn't use it • Desires for future professional development are: review strategies in modeling, questioning, and cooperative groups; review use of TI-83+ graphing calculator; review SimCalc MathWorlds software; how to use modeling instruction in reading, social studies, and civics; • • take a few days in summer to develop lessons for one's classes and share them; specific modeling strategies in mathematics; strategies for low-level math students All said that it would be very valuable if they had an hour scheduled regularly (every other week? once a month?) to coordinate their math and science courses with colleagues in their school Technology desires: several have small classrooms (too small for desktop computers) and would love to have a classroom set of TI-83+SE graphing calculators (one said that calculators are preferable to computers because graphing calculators force students to work, whereas students can be lazy if they're in a group that's working on a computer) Two teachers said that they wanted 10 computers in their classroom, and I was able to provide StRUT-donated computers for them Larry Dukerich gave his classroom set of older computers to a teacher Summary of teachers’ implementation: • Large increase in use of whiteboards • Some increase in student discourse (Socratic questioning) • Less lecturing • Less use of a standard textbook • The workshop moderately enhanced their teaching (3 on a scale of to 5) • Little coordination of courses with colleagues • No improvement in classroom technology Typically teachers have to computers in their classroom Some have access to a computer lab (often with difficulty in scheduling) Most teachers don't have TI-83+, but rather older models for which SimCalc can't be used • The degree of encouragement from administration to implement the workshop learning was vastly different in different schools It tended to be either very low or very high Listserv for participants: I subscribed all teachers to a listserv to which 150 Phoenix area junior high Modeling Workshop participants, workshop leaders, and district coordinators subscribe Hardly anyone ever posts, but this is typical for a teacher listserv of this size, in my experience I posted occasionally on resources: how to borrow classroom sets of TI-83s from Texas Instruments, how to write a $500 grant to the Wells Fargo Teacher Partner program, recent research on 'not giving the answer' to middle school students, exemplary middle school science programs, and the like V Measurement of student learning I preface this section by quoting Larry Dukerich (e-mail to me, June 2003): "I am hesitant to draw far-reaching conclusions about the merits of individual questions (or categories of questions) based on the scores of some teachers in Mesa who frankly admitted that they were unable to implement the curriculum in any meaningful way I cannot tell you how dismayed I was to learn (in our follow-up sessions) that the teachers could only this or that activity here or there because they felt compelled to cover all the topics required of them in order to prepare their students for the end-of-year tests Having spent a significant chunk of a year devising a set of activities designed to carefully build skills and a coherent view of the atomic model of matter and energy, I felt as if it had been a real waste of time Some of these teachers may as well be a control group! So, I would my best to compare gains to the teachers' self-rating of the degree to which they were able to implement the curriculum they learned - just as we did with the Modeling Instruction in High School Physics workshops It seems reasonable to me that one would only find gains for the students in classes in which the teachers were able to implement the curriculum in a coherent way." Although 21 teachers administered the PSCI or MCI (see above), the usual difficulties in matching students' pretest and posttest scores occurred, with the result that our staff could analyze significant numbers of matched pre- and posttest student data only for science teachers and math teachers Results follow A Results from the previous year (2002 – 2003): As background and for comparison: in classes of teachers in the previous modeling workshop (in summer 2002), the highest student gain in Mesa was th grade math (7% gain; i.e., percentage points gained); most students tested were in alternative algebra, a 2-period course for low achievers (representative of the group most in need of improved instruction!) Specific gains in scores for the 135 9th grade students who took BOTH the MCI pretest and the posttest were: 7% overall gain 8% conservation of weight & volume 7% proportional reasoning 9% graphing skills and motion 6% geometrical and physical properties of matter Actual MCI scores for that Mesa group are recorded in last year's ABOR final report Appendix B is a tabulation of last year's MCI and PSCI scores for grades 7, 8, and in Mesa, and for grades 7, 8, and in Phoenix-area districts: Glendale Elementary USD, Washington Elementary USD, Glendale Union High School District, Peoria Unified, and Mesa All data are matched pre- and posttest Numbers in parentheses are the number of students in the sample B This year's results (2003 – 2004): Appendix C is a spreadsheet of (mostly) grade and student pretest and posttest mean scores in the MCI and the PSCI in 2003-04 All data are matched, pre- and posttest Numbers in parentheses are the number of students in the sample, i.e., the number who took both the pretest and posttest We don't have time nor staff to analyze data by gender, and the numbers of disadvantaged minority students in the samples are too small to yield good statistics on race/ethnicity Math Concepts Inventory (vs 7) This year (2003 – 2004), we don't have enough matched MCI data points, i.e., not enough students took both the pretest and posttest, to split it into grade levels The 68 students for whom we have matched pre- and posttest MCI scores were in grades and (of teachers: th grade alternative algebra, 8th grade pre-algebra, and industrial technology) Their overall gain was percentage points (posttest average of 43%) Gains in subcategories were: 9% conservation of volume 5% graphing skills 11% measurement and geometry 5% statistics and data analysis Students had no gain in the new category of equation skills (average of only 25%) I wonder why, for the questions are conceptual and basic - designed by Mesa teachers to be similar to the 8th grade AIMS test The new version of the MCI is markedly different from previous versions; hence the new subcategories, which better reflect state standards in math Physical Science Concepts Inventory (vs 8) In the PSCI, the 58 9th graders (general science: teachers, all of whom are in their nd year of implementation) gained percentage points (posttest average of 43%) Gains in subcategories were: 13% conservation of volume 10% control of variables 6% geometric and physical properties of matter 10 17% atomic nature of matter Students had no gain in graphing skills & equations (average of 30%) and in energy & states of matter (average of only 8%) The low energy & states of matter result is not surprising, for in-depth instruction on energy is not in the curriculum - in any grade, it appears That omission is alarming! Energy is more fundamental to all sciences than forces Furthermore, because energy is a scalar quantity, it is easier for students of that age to deal with than vectors States of matter are in the grade curriculum and are reviewed only cursorily (2 to days) in 9th grade Clearly, students don't understand states of matter, even after the review Science teachers lament the poor graphing skills of students; graphing is supposed to be taught in math before grade 9, they tell me, yet their incoming students haven't a clue about slope In the PSCI, the 101 8th graders (6 teachers of science) are analyzed in categories, for one of the teachers piloted the modeling curriculum, and the other used Mesa’s 8th grade curriculum Category 1: The 84 students of the teachers who used Mesa's regular science curriculum gained only percentage points, overall (posttest average of 35%) Gains in subcategories were: 12% conservation of volume 13% control of variables 9% atomic nature of matter Students had no gain in graphing skills & equations, and in geometric & physical properties of matter, and in energy & states of matter Category 2: One of the teachers of 8th grade science piloted the modeling curriculum during the spring semester PSCI posttest scores and gains for that class of 27 students were much higher than any other class: students gained 13 percentage points overall (54% posttest average)! The teacher has many years of junior high science teaching experience Gains in subcategories were much better than any other class: 21% control of variables 15% geometric and physical properties of matter 11% atomic nature of matter 36% energy & states of matter! Unfortunately, this class had a LOSS of 17 percentage points (17%) on conservation of volume This correlates with the fact that the teacher got wrong answers on the PSCI posttest on the questions about conservation of volume, as well as on two other questions involving volume If "Do no harm" is a principle for teaching, then ways to prevent such harm as this must be found Since 1/4 of the teachers missed the volume conservation questions on the posttests, the Modeling Workshop materials must be improved in the "models of volume" section Students had no gain in graphing skills & equations, although the teacher answered this category of questions correctly The teacher is weak in math, as indicated in this self-report: "My limited math training sometimes gets in the way, but my advanced math students usually bail me out I've also started talking to some of our math teachers and they are more than willing to get me caught up." This teacher scored in the lowest quarter of the Modeling Workshop teachers on the PSCI posttest, with a score of 70% 11 Student achievement is surprisingly good overall, considering the teacher's weaknesses The results fit a pattern; we have previously found in high school physics that even teachers with poor understanding can have strong student achievement when Modeling Instruction is used fully This teacher taught the entire Modeling Workshop curriculum and consistently used all components of Modeling Instruction, including most activities and worksheets; Socratic questioning strategies, whiteboards, and in-depth group work with cooperative groups of three or four The teacher is in a school with mostly middle class students A caution: Larry Dukerich wrote to me, "I would like to caution you against reading too much into the test results from the Physical Science Concepts Inventory Unlike the FCI [Force Concept Inventory], which was developed over years based on interviews with students to come up with plausible distractors, this test is an amalgam of items selected ad hoc from a variety of sources with different goals in mind This is not to say that the individual questions are not good, but that this collection of loosely related items does not constitute a coherent test design." Unlike the previous year's group, teachers' PSCI scores in the atomic nature of matter subcategory showed improvement, so we analyzed them for students They improved, too! When looking at the data in Appendix C, it's important to realize: 1) Science teachers were frustrated at not being given commitment for time to science activities because of focus on the math AIMS test On the other hand, math teachers found creative ways to use Modeling Workshop activities to teach Arizona math standards, not by rote, but by science applications 2) Most teachers could not implement some content The physical science content is specified to be other than what was taught in the workshop (ex motion and Newton's laws in th grade - a bad idea, because energy is more fundamental to all sciences) 3) ALL teachers expressed the concern that what was expected of them hindered their ability to implement as much of the materials as they would have liked (Note that the existing science curriculum was put in place to be compliant with the 1997 Arizona standards; also, that th and 8th grades have only semester of science.) C Weaknesses in student learning: The weakest subcategories on the PSCI and MCI that were analyzed in the previous year's group of Mesa students were: • Proportional reasoning (questions #5 and were included; success rates, i.e., scores, were typically under 10%) These scores were the same as those for 104 th graders in suburban east Mesa in January1982, as reported by Prof Anton Lawson of Arizona State University (Lawson and Bealer,1984-1) In both studies, students were told that a given quantity of water occupies units in a wide cylindrical container and units when poured into a narrow one They are asked to predict how high a given quantity of water that occupies units in the wide container would rise if poured into the narrow container, and state their reason Eighth graders in Berkeley, California scored twice as high as east Mesa students, in spring 1978: 104 students averaged about 20% (ibid) • Energy and states of matter (actual scores were typically under 10%) Four research-based conceptual questions about energy were included, one of which is the following: As water in an ice cube tray freezes, a it absorbs energy from its surroundings b its surroundings absorb energy from it 12 c it absorbs coldness from and releases energy to its surroundings d it only absorbs the coldness from its surroundings e it neither absorbs nor releases energy, because its temperature stays constant Students don't BEGIN to understand energy Scores were BELOW random, indicating the powerful alternative conceptions Incidentally, the choices are expressed in the language of children who were interviewed Middle school textbooks a poor job of treating energy The Modeling Instruction Program is attempting to improve instruction on energy, the most fundamental concept in the physical sciences The modeling website has relevant articles by Gregg Swackhamer and a Basic Energy Concept Inventory for 8th and 9th grade students In this year's group (2003 – 2004), we couldn't analyze proportional reasoning, for a 'glitch' in the pdf format prevented teachers who downloaded the MCI or PSCI on Windows machines from getting the diagram that accompanies those questions (# and 6) Weakest subcategories on the PSCI that were analyzed in this year's group of students were energy & states of matter (scores of about 10%; no gains), and graphing skills & equations (scores of about 30%; no gains) Weakest on the MCI were equation skills (scores of about 25%; no gains) Conservation of weight (or mass) and volume are fundamental scientific/mathematical reasoning abilities; and junior high students should well on them However, this was not the case Conservation of weight (or mass) was tested on the first questions of the MCI and the PSCI About 3/4 of 8th and 9th graders answered these questions correctly; few significant gains occurred Only 3/4 of Mesa's 8th & 9th graders understand that when you flatten a piece of clay, its weight doesn't change Science students did better than math students (9 th grade science students scored highest; success rate in the 80%'s) Anton Lawson has found that 10 year old children typically understand this concept (private communication, and Lawson and Bealer, 1984-2), so these Mesa results of 13 and 14 year olds are weak The previous year's scores in the school districts were similarly low Conservation of volume was tested in questions #3 and on each test Although students made gains during the year (see above), their average score was low th graders started out at only 40% success rate; and 9th graders started at 47% This concept is typically mastered by 11 or 12 year olds, according to Anton Lawson (private communication, and Lawson, 1990), so these Mesa results are poor The previous year's scores in the school districts were similarly poor Quantitative evidence of this poor understanding of volume conservation is shown in Appendix D, which is a graph from a published paper (Lawson, 1990) More than 3000 th, 8th, and 9th graders in North Carolina were given the same two questions on conservation of volume; their mean score (success rate) was considerably higher, at about 60% Japanese students of the same age group did even better, scoring about 70% (The category of reasoning skill is called DV=displaced volume, on the graph.) (The test in the1990 paper was not in multiple choice format, yet scores are consistent with the multiple choice format, Anton Lawson assured me.) The question is this: given two identical cylinders filled to the same height with water If you drop marbles, one of glass and the other of steel, both the same size, into the water, compare the new heights of the water Of course, since the marbles have the same volume, the water rises by the same height in each cylinder The same graph shows that Mesa students did well in the reasoning skill called "control of variables" The two questions concern three strings hanging from a bar, with weights attached to their ends; decide which strings should be used to find out if length or weight affects the time it 13 takes to swing back and forth Mesa science students started higher than North Carolina students, and they made good gains during the year, ending up a bit higher than the Japanese students I wonder if students of teachers who hadn't had the Modeling Workshop achieved as high gains It would be illuminating if these first questions on Tony Lawson's test were given each year VI Teachers' stories about their learning Comments by teachers give insight into the human side of a project Here are excerpts from emails to me " and I are the only two teachers at Jr High School who have participated in the Modeling Workshop, and we also both teach th grade science As a result, we talk with some regularity about ways to incorporate modeling into our course More recently, after spending time individually reviewing the new science standard, and I spent several hours over the past week discussing changes that will forthcoming We went through the new th grade standard line-by-line, identifying performance objectives that will lend themselves readily to modeling techniques, and others that will not ." " I have found that using the TI-83+ has been very beneficial in teaching seventh grade science I somewhat modified what we learned this summer as an experiment to see if the seventh graders were capable of grasping the graph-matching motion program I took a day after a test and created a lesson to see what would happen I set up the CBR on the side of the class and had the calculator graphing me real time as it showed up on the overhead for the whole class to see I started the graph and walked and ran back and forth, while letting them watch it to see what was going on I was hoping that the students would be able to figure out what was being presented on the graph, but I had major doubts that they would not understand this and it would require an incredible amount of instruction I was truly shocked at how quick the seventh graders were able to label, understand, and even explain what they were seeing on a graph about it graphing motion vs time By the second time I did the demonstration, it felt like everyone in the class got it Not only in one class, but all six of my classes that day, which was a Friday and is sometimes hard to get them to focus ." " _ (a mentor) came to my room to observe me teach using the modeling method I taught five of the lessons straight from the curriculum, as it is in the book ) on Jan 21 and 22) I taught as best I could using the modeling approach in the same way, as it was presented to us in the workshop I did not modify the curriculum or lessons in any way Then we met on Friday, Jan 23rd after school to discuss what happened, what went well, what didn't, and how to improve Our reflections were that students in seventh grade are a bit too immature still, for this curriculum the way it is written We both think that it is way better suited for the ninth graders ." " and I have modified the way we teach Introduction to Technology Electronics unit We set up the labs for discovery of Ohm's Law instead of teaching it first and doing the labs to verify it We intend on using whiteboarding to report the findings, and through the class discussion of the results of changing resistance in electronic circuits to change the current, we hope to guide the class to Ohm's Law We prepared the questions for the whiteboards In addition, it is our intent to present this to all the junior high Industrial Technology teachers at an in-service and the beginning of school next year." 14 "We [three 8th grade science teachers] modified a F=ma lab to use the Buggies, we also made changes to a lab we with pulleys to use the whiteboarding We have found the whiteboarding to be very effective We will start our physics unit in the 4th quarter." VII Long-term outlook Larry Dukerich wrote the following to Rick Vanosdall, former science specialist in Mesa Public Schools, during his workshop in June 2002: "On the larger issue of meaningful change, I find it hard to be optimistic in the face of the obstacles we face The history of the fragmented approach we have been using in the Mesa junior high program has shaped the way teachers view how instruction should take place Most see this workshop as a source of good activities that they could plug in here or there to supplement or bolster what they already I had this vivid image of garage sale junkies picking through the pile of dishware for a piece here and there, not recognizing that a whole set was in front of them It was with considerable difficulty that I managed to keep the look of incredulity off my face as I listened to (who has been a real pillar in this workshop) tell me just how much she felt she was expected to in 8th grade science Force and motion, energy, circulatory system, astronomy and ecology in 18 weeks???? In her defense, she is gamely trying to "cover" the standards set forth for grades - It would require a major paradigm shift for teachers to consider the possibility of using these materials as a complete course They universally express the view that they could be far more effective if they could focus on just a few topics and them well, but they just don't see that in the cards However, expectations of performance on the current math test and future science test hangs like a sword of Damocles over their heads I expressed the view that perhaps we don't need to worry so much about the tests; just focus on developing the skills, and the kids will well enough ." 15 VIII The bottom line: how can jr high students in Mesa learn math & science better? Larry Dukerich said it (above): • focus on just a few topics and them well, • replace the fragmented approach with a coherent curriculum/pedagogy: Modeling Instruction, • don't worry so much about the tests; just focus on developing the skills To implement Larry’s ideas, Mesa's science and math directors and specialists are urged to: • coordinate math and science instruction in each building and at each grade, as discussed in the first section of this report This involves regular meetings of teachers to work together, with peer leaders giving instruction Nearly 40 peer leaders in 12 junior high schools are available from the Modeling Workshops; use them! Develop them! Begin with the apprentice leaders and a few of the mentors that Larry and Stella chose • pay Larry Dukerich and Stella Ollarsaba to lead workshops in their respective areas of strength, including the following technology inservices: Larry: Graphical Analysis software; Stella: SimCalc MathWorlds software used with TI-83+SE handhelds • persuade teachers who didn't take our Modeling Workshop to enroll in our ASU summer course, PHS534/MTE598: Methods of Teaching Physical Science/Physical Science with Math Modeling Workshop Give them financial help in paying tuition See http://modeling.asu.edu/MNS/MNS.htmlfor course information ASU can help Here are ideas for how Mesa's science and math directors and specialists can work with ASU to improve your teachers' content knowledge in science and math • Encourage teachers who have already taken PHS534/MTE598 to enroll in courses for science or math majors at a community college (e.g., general physics, general chemistry) If they choose wisely, they can get quality instruction from instructors who use the Learning Cycle or the Modeling Cycle Give teachers financial help to pay tuition • Encourage 9th grade teachers who have taken general physics to enroll in PHS530: Methods of Teaching Physics (Modeling Workshop in Mechanics) • Encourage teachers without a Master's degree to consider ASU's MNS degree programs for high school teachers (some include middle school teachers) Faculty are Steve Semken (geology), Jim Middleton (middle school), and Marilyn Carlson (math) • Encourage teachers who have a Master's degree to take MNS courses • Encourage 7th and 8th grade science and math teachers to enroll in our companion course, PHS 594: Physical Science with Math Modeling Workshop II (force, motion, intro chem.) • We are glad to help school districts such as Mesa to implement these ideas Call on us! References: Adams, Jacob E (2000) Taking Charge of Curriculum, Teachers College Press Adey, Philip (1999) "The Science of Thinking, and Science for Thinking: A Description of Cognitive Acceleration through Science Education (CASE)", Innodata Monographs - 2, UNESCO International Bureau of Education On the world wide web, downloadable in pdf format at http://www.ibe.unesco.org/International/Databanks/Innodata/inograph.htm 16 Adey, Philip (2003) Scientific Reasoning Skills (a PowerPoint presentation) Downloadable at http://modeling.asu.edu/Projects-Resources.html Behr, Harel, Post and Lesh, R (1992) The Handbook of Research on Mathematics Teaching and Learning, 326 Hestenes, D (1987) Toward a Modeling Theory of Physics Instruction, Am J Phys 55: 440454 At http://modeling.asu.edu/R&E/Research.html Lawson , A E (1990) Science Education in Japan and the United States: Are the Japanese beating us at our own game? Science Education 74(4): 495-501 Lawson, A E and Bealer, J M (1984-1) Cultural Diversity and Differences in Formal Reasoning Ability, Journal of Research in Science Teaching 21, 735-743 Lawson, A E and Bealer, J M (1984-2) The acquisition of basic quantitative reasoning skills during adolescence: Learning or development? Journal of Research in Science Teaching 21(4): 417-423 National Council of Teachers of Mathematics (2000) Principles and Standards for School Mathematics, Reston, VA National Research Council (1996) National Science Education Standards, National Academy Press, Washington DC National Research Council (1999) Designing Mathematics or Science Curriculum Programs, a Guide for Using Mathematics and Science Education Standards, Natl Academy Press, Wash.ington DC Sparks, Dennis (2000) Designing Powerful Professional Development for Teachers and Principals, National Staff Development Council At http://www.nsdc.org/sparksbook.html Wells, M, Hestenes, D., and Swackhamer, G (1995) A Modeling Method for High School Physics Instruction, Am J Phys 63: 606-619 At http://modeling.asu.edu/R&E/Research.html 17 ... in Science Teaching 21(4): 41 7-4 23 National Council of Teachers of Mathematics (2000) Principles and Standards for School Mathematics, Reston, VA National Research Council (1996) National Science. .. the new science standard, and I spent several hours over the past week discussing changes that will forthcoming We went through the new th grade standard line-by-line, identifying performance... 8th grade science Force and motion, energy, circulatory system, astronomy and ecology in 18 weeks???? In her defense, she is gamely trying to "cover" the standards set forth for grades - It would