Assurance-of-Learning-Report-Dec21-2008

Assurance of Learning Report Prepared for: AACSB International Date: December 15, 2008 School of Business | Camden Rutgers University Address questions and inquiries to: Mitchell P Koza, PhD Dean and Distinguished Professor of Management Eugene Pilotte, PhD, FRM Professor of Finance Senior Associate Dean for Academic Affairs Chair, Learning Assessment Committee School of Business - Camden Rutgers, The State University of New Jersey 227 Penn Street Camden, NJ 08102-1656 Table of Contents I Executive Summary II Assessment Tools and Procedures A Step 1: Establishing Learning Goals B Step 2: Aligning Curricula with Adopted Goals C Step 3: Identifying Assessment Instruments and Measures D Step 4: Collecting, Analyzing and Disseminating Assessment Information E Step 5: Planning for Continuous Improvement III Appendices A Detailed Results of Assessments of Learning Goals B Rubrics Used to Assess Learning Goals C Indirect Assessments (Exit Surveys) I Executive Summary The Rutgers School of Business - Camden (SBC) efforts for Assurance of Learning have been facultydriven and have adhered to the steps outlined by the AACSB in the November 2007 White Paper titled “AACSB Assurance of Learning Standards: An Interpretation.” Learning goals and objectives were defined by a process that included committee input, external benchmarking, and faculty workshops facilitated by assessment experts Curricula were mapped to learning goals to ensure that the curriculum sufficiently addresses each of the learning goals adopted by the faculty Learning was assessed through course-embedded measures and processes were put in place to collect, analyze, distribute, and use the assessment results for continuous improvement The initial rounds of assessment activity occurred during the Spring 2008 semester and 2008 summer sessions, when data were collected and assessed for four of six MBA learning goals and three of the six undergraduate learning goals Collection and assessment of data for all remaining learning occurred during the Fall 2008 semester For most learning objectives, 80 percent or more of students met or exceeded expectations and no immediate action is required For learning objectives where 10 to 20 percent of students failed to meet expectations, the Learning Assessment Committee has charged the undergraduate and MBA task forces with alerting relevant faculty, encouraging individual course-level initiatives, and monitoring for improvement For learning objectives for which less than 80 percent of students met or exceeded expectations, the task forces are charged with designing targeted remediation strategies Two undergraduate goals fall in this category, the general management knowledge goal and the communications impact and effectiveness goal In response to the shortfall in performance for the communications impact and effectiveness goal, two sections of a new undergraduate Business Communications course were offered during the Fall 2008 semester and three sections are planned for Spring 2009 The course is designed to address the shortcomings identified through learning assessment The success of the course in addressing the identified shortcomings will be monitored and the course will be considered for addition to the required curriculum Also, the rubrics that were used to evaluate written and oral communications skills were circulated among the faculty Several have used it in their courses during the Fall 2008 semester in giving feedback to their students The results of the assessment of general management (functional area) knowledge indicate that students retain less knowledge from courses outside their majors and courses taken earlier in the curriculum Remedies in the early stages of implementation include developing systems for early signaling and systems for review of key topics for retention, creation of an undergraduate continuous improvement website to facilitate student review of key concepts in preparation for the capstone business policy course, and encouraging students to enroll in specially designated courses that integrate knowledge from the functional areas Also under consideration is the addition to the curriculum of additional integrative courses that draw on all core business topics II Assessment Tools and Procedures A Step 1: Establishing Learning Goals Multiple sources of input contributed to the current set of SBC learning goals that was accepted by the faculty in January 2008 The Undergraduate (UG) and MBA task force each provided initial guidance on primary objectives for student learning within each program External benchmarking of 34 peer and aspirant schools was conducted to identify themes in learning assessment within our comparative schools The faculty then assembled in a series of workshops to finalize the learning goals and translate them into specific objectives which each focused on a measurable attribute or skill AACSB Assessment Expert, Kathryn Martell, played a vital role in facilitating the final faculty workshop and in syndicating our ultimate end-product The SBC Learning Assessment Committee (LAC) was then responsible for finalizing the language used to express the overall set of goals and objectives The LAC is chaired by the Senior Associate Dean for Academic Affairs (Eugene Pilotte) and includes the Assistant Dean for Special Projects (Sam Rabinowitz), the MBA Program Director (Rakesh Sambharya) and the UG Program Director Carol Kaufman-Scarborough A total of learning goals have been defined, which are largely consistent across two programs Each of the learning goals, where applicable, has one or more specific learning objectives associated While the number of learning objectives is nearly the same for the UG and MBA programs, the focus and wording differ, as shown below in Table Table - MBA and UG Learning Goals and Objectives Learning Goal General Management Knowledge MBA Objectives • Not assessed in MBA program Critical Thinking and Analytical Decision Making • Students will interpret and synthesize analysis to draw meaningful conclusions Teamwork, Interpersonal Relations • and Leadership • Communications Impact and Effectiveness • Global Mindset • Ethical Reasoning • Technological Fluency • Students will be able to work well in teams and relate interpersonally with others Students will demonstrate the ability to evaluate their leadership capabilities, identify areas of weakness, and develop a leadership blueprint to maximize their leadership potential Students will demonstrate effective oral presentation skills Students will possess strong written language skills Students will understand and evaluate business environments outside of the United States Students will demonstrate awareness of various ethical frameworks, how they relate to their own decisionmaking, and the impacts on various stakeholders Students will use software/technology to enhance research, analysis and presentation • Undergraduate Objectives • Students will understand basic functions of business and functional area concepts • Students will analyze, interpret and solve business problems using quantitative approaches where relevant • Students will work well in teams and relate interpersonally with others • Not assessed in undergraduate program • Students will demonstrate effective oral presentation skills Students will possess strong written language skills Not assessed in undergraduate program • • • Students will identify ethical issues related to the management of people, and/or the marketing of products and services • Students will demonstrate Information Technology Literacy B Step 2: Aligning Curricula with Adopted Goals In order to ensure that the curriculum sufficiently addresses each of the learning goals, relevant courses at both the UG and MBA level were reviewed The mapping in Table demonstrates clear evidence that students complete work in one or more classes that directly supports achievement of the learning goals Coursework within scope for Assurance of Learning efforts is restricted to courses that all students, either UG or MBA must take Courses that are required, but not taught in the school of business (e.g., Undergraduate Pre-Business Requirements), are not in scope For learning goals where coverage is limited to less than courses with individual work products, a specific learning objective will not be assessed (i.e., General Management Knowledge at the MBA level, and Leadership and Global Mindset at the UG level) The following key should be used to interpret the table below: E = Exam, O = Oral Presentation, S = Simulation or Hands-on exercise, W = Written Assignment; (t / i) = team / individual Table – Mapping of Curricula to Learning Goals Learning Goals Courses MBA Managerial Economics Statistical Methods for Managerial Decision Making Personal Leadership and Communications Skills Accounting for Financial Reporting Accounting for Managerial Decisions Financial Management Managing People and Organizational Processes International Business Environment Operations Management Managing Information Technology and Resources Marketing Management Strategic Management: Integrating the Enterprise Undergraduate Introduction to Financial Accounting Management Accounting Introduction to Business Computing Business Law I Principles of Finance Organizational Behavior Operations Management Management Science Management Information Systems Principles of Marketing Business Policy and Strategy General Management Knowledge Critical Thinking and Decision Making Teamwork, Interpersonal Relations and Leadership S(t) W(i), S(t) E E S(t) W(i), O(t) E E W(i), E W, O(t) W(i), E W(i), E W(t) W(i + t) W(i), O(t) W(i), E W(i), E W(i), E W(i), E W(i), E W(t+i), E W(t+i) W(i), E W(i), E S(t) S(t) W(i), E W(i), E W(i), E W(i), E W(i), E W(t+i), E W(t+i) W(i), E W(i), E S(t) S(t) Communications Impact and Effectiveness Global Mindset O, W(i) O(i), W(i) Ethical Reasoning Technological Fluency O(i), W(i) W(i) E W(i) W, O(t) W(t), O(t) S(t) W(t) W(t) O(t) O(t), W(t) W(i) W(i) W(i) W(i + t) O(t), W(i) W(i) W(i) W(i) E, W(t) S(t) O(t), W(i) W(i), E W(i), E O(t), W(t) W(t) S(t) W(t+i), O(t) W(i) O(i), W(t) W(t) E E W(t) W(i) W(i) W(i) S(t) S(t) C Step 3: Identifying Assessment Instruments and Measures The SBC LAC considered the range of acceptable approaches and developed the following plan for direct assessment of student learning The courses selected were those where the scope of instruction is most closely aligned with the specific learning objectives and goals In all cases, a course-embedded instrument will be used The plan is to assess each goal either during the Spring, Summer, or Fall of 2008 Table – Individual Student Work Products for Assessment of Learning Learning Goal General Management Knowledge Critical Thinking and Analytical Decision Making Teamwork, Interpersonal Relations • • • MBA Student Work Product Not assessed in MBA • program Written case assignment • Peer evaluations from computer simulation project Personal Leadership Action Plan • Relevant Course Not assessed in MBA program Operations Management (Spring 2008) Marketing Management (Spring 2008) Undergraduate Student Work Product Relevant Course • Exam • Business Policy (Fall 2008) • Written case • Business Policy assignment (Fall 2008) • Peer evaluations from case study team project • Organizational Behavior (Spring 2008) Personal Leadership and Communications (PL&C) (Spring 2008) PL&C (Spring 2008) • Not assessed in undergraduate program • Not assessed in undergraduate program • Individual oral presentation • Organizational Behavior (Spring 2008) and Leadership • Communications Impact and Effectiveness • 6- minute oral presentation • • Written case assignment • Operations Management (Spring 2008) • Individual written research paper • Organizational Behavior (Spring 2008) International Business Environment (Spring 2008) PL&C (Fall 2008) Management of Information Technology (Summer 2008) • Not assessed in undergraduate program • Not assessed in undergraduate program • Exam questions • Individual assignment s • • • Organizational Behavior (Fall 2008) MIS (Summer, Fall 2008) • Global Mindset • International Business 15page “Country Report” • Ethical Reasoning • • Technological Fluency • Ethics assignment or PL&C ethics case Individual assignments • The direct assessment data from the instruments above will be complemented by student input obtained through the SBC Exit Assessments For both the UG and MBA programs, an internally-developed survey is administered to each graduating class Through this instrument, students will self-assess how well their coursework enhanced their skills and knowledge across the seven goal areas D Step 4: Collecting, Analyzing and Disseminating Assessment Information Data collection process According to plan, data were collected during the Spring 2008 semester for four of the six MBA learning goals and data were collected for three of the six undergraduate learning goals during Spring and Summer of 2008 Collection of assessment data for all remaining learning goals occurred during the Fall of 2008 In all cases, an assessment rubric is applied to a course-embedded work product that is collected by the SBC Learning Assessment Committee (LAC) from the instructor of a targeted class Each work product is assigned to an assessor who reports results to the Senior Associate Dean of Academic Affairs for tabulation, storage, and provision to the LAC for review Outcomes data for MBA courses Table summarizes the overall results of assessment activity for the MBA program Six goals (eight objectives) were assessed The detailed results of the assessments of individual traits under each goal/objective combination appear in Appendix A of this document The rubrics applied appear in appendix B Table - Summary of MBA Assessment Activity (Spring, Summer & Fall 2008) Goal (objective) Critical Thinking and Analytical Decision Making (Students will interpret and synthesize analysis to draw meaningful conclusions) Teamwork, Interpersonal Relations, and Leadership (Students will be able to work well in teams and relate interpersonally with others) Teamwork, Interpersonal Relations, and Leadership (Students will demonstrate the ability to evaluate their leadership capabilities, identify areas of weakness, and develop a leadership blueprint to maximize their leadership potential) Communications Impact and Effectiveness (Students will demonstrate effective oral presentation skills) Communications Impact and Effectiveness (Students will possess strong written language skills) Global Mindset (Students will understand and evaluate business environments outside of the United States) Ethical Reasoning (Students will demonstrate awareness of various ethical frameworks, how they relate to their own decision-making, and the impacts on various stakeholders) Technological Fluency (Students will use software/technology to enhance research, analysis and presentation) Does not meet expectations (6%) Meets expectations 14 (39%) Exceeds expectations 20 (56%) Sample Size 36 (19%) 11 (42%) 10 (38%) 26 (13%) 15 (48%) 12 (39%) 31 (17%) 14 (47%) 11 (37%) 30 (14%) 0% 13 (36%) 43% 18 (50%) 12 57% 36 (5%) 25 (33%) 47 (62%) 76 (17%) (42%) (42%) 12 21 The results of MBA assessment activity indicate that at least 81 percent of assessed students met or exceeded expectations for each learning objective The objectives for which 13 to 19 percent of students failed to meet expectations were associated either with the teamwork, interpersonal relations, and leadership goal, the communications impact and effectiveness goal, or the technological fluency goal Outcomes data for undergraduate courses Table summarizes the overall results of assessment activity for the undergraduate program Six goals (seven objectives) were assessed The detailed results of the assessments of individual traits under each goal/objective combination appear in Appendix A of this document The rubrics applied appear in appendix B Table - Summary of Undergraduate Assessment Activity (Spring, Summer, and Fall 2008) Goal/Objective General Management Knowledge (Students will understand basic functions of business and functional area concepts) Critical Thinking and Analytical Decision Making (Students will analyze, interpret and solve business problems using quantitative approaches where relevant) Teamwork, Interpersonal Relations (Students will work well in teams and relate interpersonally with others) Communications Impact and Effectiveness (Students will demonstrate effective oral presentation skills) - first presentation Communications Impact and Effectiveness (Students will demonstrate effective oral presentation skills) - second presentation Communications Impact and Effectiveness (Students will possess strong written language skills) Ethical Reasoning (Students will identify ethical issues related to the management of people, and/or the marketing of products and services) Technological Fluency (Students will demonstrate Information Technology Literacy) – Summer 2008 Technological Fluency (Students will demonstrate information technology literacy) – Fall 2008 Does not meet expectation 19 (40%) Meets expectations 27 (56%) Exceeds expectations (4%) Sample Size 48 (12%) 37 (76%) (12%) 49 (11%) (6%) 30 (83%) 36 (18%) 23 (61%) (21%) 38 13 (42%) 13 (42%) (16%) 31 (27%) (5%) 15 (45%) 15 (68%) (27%) (27%) 33 (17%) 16 (67%) (17%) 24 (10%) 12 (24%) 32 (65%) 49 22 The results of undergraduate program assessment activity identified two areas of immediate concern, retention of general management knowledge and communications impact and effectiveness E Step 5: Planning for Continuous Improvement Continuous improvement is ensured by providing feedback to the faculty on the results of learning assessment and by making specific curricular changes or programmatic enhancements The LAC facilitates this process by categorizing goals according to the percentage of student outcomes that not meet expectations and providing the results to the undergraduate and MBA task forces which are charged with implementing actions for improvement The categories and relevant actions follow Less than 10% not meet expectations In this case no immediate action is taken In the current round of assessment activity the MBA critical thinking and analytical decision-making goal, global mindset goal, and ethical reasoning goal fall in this category At the undergraduate level the ethical reasoning goal falls in this category Between 10% and 20% not meet expectations In this case task forces are charged with alerting relevant faculty, encouraging individual course-level initiatives, and monitoring for improvement In the current round of assessment activity the MBA teamwork, interpersonal relations, and leadership goal and the communications impact and effectiveness goal fall in this category The undergraduate goals that fall in this category are critical thinking and analytical decision making, teamwork and interpersonal relations, and technological fluency More than 20% not meet expectations In this case a targeted remediation strategy is designed Two undergraduate goals fall in this category, the general management knowledge goal and the communications impact and effectiveness goal The assessment of general management knowledge was an exam administered to students early in the Fall Business Policy course This exam was designed to test retention of basic business concepts covered in core courses The results indicate that students retain less knowledge from courses outside their majors and courses taken earlier in the curriculum Remedies proposed by the Undergraduate Task Force include: 1) developing a system for early signaling to students in required courses as to key topics for retention, 2) developing a system for student review of key topics and skills, and 3) consideration of additional integrative courses that draw on all core business topics Two meetings attended by faculty who teach required courses produced additional recommendations Based on these recommendations an Undergraduate Continuous Improvement website is being created to facilitate student review of key concepts in preparation for the capstone business policy course The faculty also recommended creating a designation for certain business courses that integrate information from all the majors, rather than focus on one major versus another The designation would signal students that a major goal of such courses is the integration of foundational business knowledge Because of the shortfall in performance for the communications impact and effectiveness goal, two sections of a new undergraduate Business Communications course were offered during the Fall 2008 semester and three sections are planned for Spring 2009 The course is designed to address shortcomings in oral and written communications skills that were identified through learning assessment The success of the course in addressing the identified shortcoming will be monitored and the course will be considered for addition to the required curriculum In addition, the rubrics that were used to evaluate written and oral skills were circulated among the faculty Several have used it in their courses during the Fall 2008 semester in giving feedback to their students APPENDIX A Detailed Results of Assessments of Learning Goals 10 ORAL PRESENTATION RUBRIC (undergraduate and MBA) TRAIT Preparedness Confidence BELOW EXPECTATIONS Trouble with responding to most questions even with notes at hand MEETS EXPECTATIONS Can address most questions with correct information and some use of notes Lack of confidence as evidenced by excessive nervousness and/or distracting mannerisms such as shifting feet and/or bad posture Makes minor errors but recovers quickly from them with little or no tension displayed EXCEEDS EXPECTATIONS Fully prepared presentations with complete knowledge of material, minimal use of notes and ability to answer impromptu questions Exudes confidence when presenting Shows command of topic and comfort in presentation POINTS Appearance Eye contact Does not meet minimum standards for business attire Makes little or no eye contact Tone, Pace, and Enunciation Noticeable amount of the following: hard to understand, monotone, erratic pace, little excitement or enthusiasm projected, mumbling, too fast, too slow, too loud, too soft Completeness relative to supporting materials Sloppy and/or unprofessional visual aids May be difficult to read or interpret Noticeable lapses in presentation content and focus on audience Meets minimally acceptable dress standards Maintains eye contact most of the time with part of the audience Maintains an understandable and appropriate pace and Maintains an understandable and appropriate pace and volume Mostly clear and natural in presentation style Appropriate dress relative to quality, style, and color choice for a working professional Maintains eye contact with everyone in the audience Excellent delivery Conversational where appropriate, modulates voice and projects enthusiasm and interest Adequate use of supporting materials with minimal inconsistencies or relevance in their use and application Uses resources and visual aids effectively while focusing on audience and not the visual aids themselves Transition and Organization Presence in front of an audience No opening statement or one that is irrelevant Choppy and uncertain switching from point to point Questionable organization Offers opening statement with adequate outline of intentions Mostly logical sequencing of material A few minor points may be confusing Clear opening statement with overview and total focus Smooth, logical and interesting transitions from point to point (and to the next speaker- if applicable) Unprofessional in word choice and interactive style with audience Content and/or style are occasionally inappropriate Maintains a professional relationship with the audience regarding style and content of presentation TOTAL: Comments 19 WRITTEN COMMUNICATION RUBRIC (undergraduate) TRAIT Organization BELOW EXPECTATIONS Unclear, uneven, or missing Introduction/overview of paper; Unclear or missing objectives and/or Conclusions Uneven and ineffective overall organization Development Of Ideas Most ideas are presented without the application of theoretical concepts or other forms of documented evidence Spelling & Grammar Numerous errors in spelling and grammar which interfere with comprehension Lack of editing throughout paper Fewer sources than required are utilized Variety of sources is lacking Bibliographic references are missing or not completed in acceptable style Writing lacks necessary documentation Formatting is consistent or absent Documentation & Formatting MEETS EXPECTATIONS Adequate introduction/overview and statement of objectives with minor weaknesses Mostly organized and coherent ideas, but with possible illogical and/or unrelated aspects of the paper presentation Adequate conclusions and summary with minor shortcomings Support of ideas with some use of theoretical concepts, as well as mostly effective use of examples and details May have some inconsistencies in presenting support for ideas or reasoning Some errors, mostly minor in nature may present There is evidence of attempts at proofreading EXCEEDS EXPECTATIONS Clear, effective introduction/overview of paper, clearly stated objectives; consistent and thoughtful conclusions and summary Includes and integrates the number and type of sources required for the assignment Bibliographic references are mostly correctly presented Content within the paper is mostly cited correctly and adequately Formatting is logical and consistent in most places in the paper Includes and integrates more sources and a greater variety of sources than is required Bibliographic references are correctly presented Content within the paper is cited correctly and adequately Formatting is consistent and logical throughout the paper TOTAL: COMMENTS 20 POINTS -Explores ideas thoroughly; supports points with accurate application and balance of theoretical concepts and subjective and objective evidence -The writing is essentially error free in terms of spelling and grammar Undergraduate Ethical Considerations Rubric Date: _ Rater: Course: Student: Trait Does not meet expectation Meets Expectations Exceeds Expectation (1 point each) (2 points each) (3 points each) Identifies the ethical dimensions of the issue Does not identify the ethical dimensions of the issue Analyzes and reflects on the ethical dimensions of the issue Makes recommendation based on ethical considerations Is unable to analyze and reflect on the ethical implications of the issue Has difficulty identifying a course of action that is tied to analysis of ethical dimensions of the issue Identifies the ethical dimensions of the issue in a general way Analyzes and reflects in a general way on the ethical implications of the issue Identifies a course of action generally supported by analysis of ethical implications of the issue Identifies the ethical dimensions of the decision clearly and precisely Accurately analyzes and reflects on the ethical implications of the issue Recommended course of action is tied closely to analysis of the ethical implications of the issue Total Points 21 Points Undergraduate Technological Fluency / IT Literacy Evaluation Matrix Assesssment Beginning User Intermediate User Expert User Beginning User _ IT Skills Intermediate User _ Expert User typing skills, _ can use Reasonable basic word processing and power point skills, create simple spread sheets, can use email with attachments and simple web access Can use commonly used features of application software (e.g databases, spreadsheets) to create simple business applications and customized graphs and presentation by integrating data from more than one application Can develop very simple web pages Can learn to use a new piece of software without help; mastered advanced skills in using common application software such as use of Scenario Manager and Solver in Excel, exchanging data between Office software, inserting java applets in web pages etc Beginning User _ IT Concepts Intermediate User _ Expert GeneralUser awareness of _ IT issues, BeginningIT User _ Research / Intermediate User _ Information Literacy Expert User basic information _ Demonstrates but little specific knowledge of retrieval skills using popular IT hardware and software usage portals such as Google Has basic knowledge of how computers and internet work, and how information is stored and retrieved Can use technology in the learning process (e.g., on-line tutorials, web resources, discussion boards Thinks about technology on an abstract level and understands the role of IT as a strategic tool; understands the underpinnings of algorithmic thinking Can visualize use of IT across disciplines 22 Uses the College’s library catalog and periodical databases, but not aware of the need to evaluate sources Has rudimentary awareness of copyright and plagiarism Navigates and evaluates information using a variety of academic and non-academic sources, formats, and media in research and problem solving Understands the basics of copyright and plagiarism MBA CRITICAL THINKING AND ANALYTICAL DECISION-MAKING RUBRIC (adapted from California State University – Chico Critical Thinking/Problem-Solving Abilities Rubric) Course: Semester: Student Identifier: _ Please rate the paper on each of the following traits Try to stay close to the descriptors used in the rubric Granted, not every situation will fit perfectly within a particular rating category and some degree of personal interpretation may be necessary For each trait please give a rating from 1-3 using whole numbers only (No half-point scores are allowed.) DOES NOT MEET EXPECTATIONS TRAIT Problem Identification: Identifies and summarizes the problem, question, or issue This dimension focuses on task or issue identification, including secondary or implicit aspects of an issue and the relationships between factors that may be integral to effective analysis POINT EACH Does not attempt to or fails to identify and summarize the problem accurately Evaluates Quality of Evidence: Identifies, assesses, and analyzes the quality of supporting data/evidence This dimension focuses on evidence of search, selection, and source evaluation skills—including accuracy, relevance and completeness Repeats information provided without question or dismisses evidence without adequate justification MEETS EXPECTATIONS POINTS EACH Summary of issue is mostly accurate but some aspects are incorrect or confused; nuances and critical details are absent or glossed over EXCEEDS EXPECTATIONS POINTS EACH Clearly identifies and summarizes main problem, question or issue Identifies secondary or implicit issues If applicable, notes relationships between factors in the situation and how they relate to each other - Does not distinguish among fact, opinion, and value judgments Sources are not on topic or are inappropriate No evidence of search, selection or source evaluation skills Use of evidence is qualified and selective, though perhaps unintentional Discerns fact from opinion and may recognize bias in evidence though attribution is spotty, inappropriate, or exaggerated Examines the evidence and source of evidence; questions its accuracy, precision, relevance, completeness Information need is clearly defined and is related to assignment, course or personal interests Sources selected adequately meet the information need, though little evidence of more than routine exploration Evidence of search, selection, and source evaluation skills; notable identification of uniquely salient resources Demonstrates adequate skill in searching, selecting, and evaluating sources to meet the information need Business Analysis: Undertakes appropriate quantitative or qualitative analysis This dimension focuses on the appropriate use of quantitative or qualitative analysis of information to clarify issues and facilitate decision-making POINTS Quantitative or qualitative analysis conducted is inappropriate, inaccurate, and superficial (or nonexistent) Quantitative or qualitative analysis is appropriate and accurate, but rather superficial Analysis doesn’t help clarify the issues or facilitate decision-making Analysis has limited ability to help clarify the issues and facilitate decision-making Quantitative or qualitative analysis is appropriate, accurate, and thorough Analysis is used to clarify the issues and facilitate decision-making - 23 24 Considers Alternative Perspectives: Integrates issue using other perspectives and positions This dimension focuses on the treatment of diverse perspectives, effective interpretation and integration of contrary views and evidence through reflective and nuanced judgment and justification Deals only with a single perspective and fails to discuss other possible perspectives, especially those held by others If more than one idea is advanced, alternatives are disjointed or bolted together Begins to relate alternative views to qualify analysis Rough integration of multiple viewpoints and comparison of ideas or perspectives Ideas are investigated, if in a limited way, and integrated, if unevenly May dismiss alternative views too hastily Adopts a single idea or limited ideas with little question Reasoned and Logical Conclusion: Identifies and assesses conclusions, implications, and consequences This dimension focuses on integrating previous dimensions and extending them to draw conclusions and solve a problem(s) Well-developed conclusions more than summarize They establish new directions for consideration in light of context and the breadth and depth of the evidence Fails to identify conclusions, implications, and consequences of the issue or the key relationships between the other elements of the problem, such as other perspectives, assumptions, or data and evidence Doesn’t propose solution to problem Conclusions begin to reflect influence of other perspectives, assumptions, and evidence that leads to consequences that extend beyond the borders of a discipline or single issue Fully integrated ideas and perspectives from variety of sources Analogies may be used effectively Integrates own and others’ ideas through a complex process of judgment and justification Can clearly present and justify own view or hypothesis while respecting other views Identifies and discusses conclusions, implications, and consequences considering assumptions, data, and evidence Objectively qualifies own assertions Confuses correlations with cause Recognizes limitations of correlations or association and qualifies implications of assertions accordingly Considers knowledge as relative collection of opinions and perspectives, and makes little attempt to compare - Proposes solution to problem(s) based on previous dimensions noted in rubric Proposes solution to problem(s) that is somewhat related to previous dimensions noted in rubric Mistakes correlations with cause Considers knowledge as absolute when confirmed by one or another authority Addresses additional diverse perspectives drawn from outside information to qualify analysis Views knowledge as the best available evidence within the given context, even in the face of uncertainty and ambiguity - TOTAL POINTS - Overall Assessment Score: expectations 5-7 points = Does not meet expectations; 8-12: Meets expectations; 13-15: Exceeds Please make comments related to the quality of the student’s critical thinking and analytical decision-making abilities 25 MBA Team Performance Evaluation Marketing Management Management, Spring 2008 Prof Julie Ruth Regarding the team project, you are asked to rate your performance as well as the performance of group members These ratings will remain confidential and will be reviewed only by the instructor This information will be used in conjunction with the instructor’s observations regarding team process, progress, and outcomes Please return this evaluation no later than the day that your final exam is due to Professor Julie Ruth The evaluation can be returned via email attachment (jaruth@camden.rutgrs,edu) or postal mail (pre-addressed, postage-paid (already), envelope provided) Name (indicate names of team members) Quantity of Quality of Contributions contributions Cooperation Dependability Initiative or Leadership (10=Excellent, 1=Very Poor) (10=Excellent, 1=Very Poor) (10=Excellent, 1=Very Poor) (10=Excellent, 1=Very Poor) (10=Excellent, 1=Very Poor) Overall Evaluation (rate each team member from 0% to 100%; 100% represents fully meeting expectations and responsibilities Any additional comments: After careful consideration, the above ratings represent my genuine assessment of the performance of each team member Signature _ Print Name 26 PERSONAL LEADERSHIP RUBRIC TRAIT Demonstrated ability to evaluate leadership capabilities BELOW EXPECTATIONS Not sure of what he/she wants to with life and/or unsure of course values and/or unclear of what kind of leader to be or what to create as a leader (1 point each) MEETS EXPECTATIONS Adequate identification of personal purpose, core values, and leadership vision (2 points each) Identification of areas of strength and weaknesses Student minimally identified or discussed strengths and weaknesses Inconsistent or lacking in integrated information from various sources Did not have an understanding of implications related to leading others (1 point each) Student reported on strengths and weaknesses from various sources with adequate understanding of themes Acceptable thought given to implications (2 points each) Development of a leadership blueprint to maximize their leadership potential Student’s action plan is too general to carry out with any consistency (2 points each) Student developed a generally compelling and specific action plan tied to key development aspects but may be lacking in consistent or sufficient detail on following up on some of these goals (4 points each) EXCEEDS EXPECTATIONS Clear understanding of personal purpose with a thorough discussion of core values and a clear and specific identification and reflection on a leadership vision (3 points each) Student fully utilized and integrated feedback from significant people in his/her life along with his/her own selfassessment in discussing strengths and weaknesses Student clearly understood common themes and implications related to the identification of strengths and weaknesses (3 points each) Student developed a compelling and specific and doable action plan tied to each of to key development outcomes (6 points each) 4-6 points = Below expectations; 7-9 points = Meets expectations; 27 10-12 points = Exceeds expectations -TOTAL: Overall Assessment Score: POINTS Course _ Semester _ Student identifier _ MBA WRITTEN COMMUNICATION RUBRIC FOR CASE WRITE-UP TRAIT Problem Statement Alternatives and Recommendations Logical Connection of Ideas Organization Spelling & Grammar Appearance, Documentation & Formatting DOES NOT MEET EXPECTATIONS (1 POINT EACH) Either unable to clearly communicate a central problem or to link analysis and recommendations to the problem statement Unable to clearly communicate perceived pros and cons of alternatives and recommendations No or limited logical connection of ideas Presentation of arguments, evidence, and other information are disjointed MEETS EXPECTATIONS (2 POINTS EACH) Some ambiguity in stating the central problem Adequate linkage of subsequent analysis and recommendations to the problem statement EXCEEDS EXPECTATIONS (3 POINTS EACH) Clear statement of central problem with analysis and recommendations clearly linked to problem statement Clearly communicates perceived pros and cons of most alternatives and recommendations Organization of information interferes with a reader’s ability to clearly understand the problem and outcomes of the analysis Numerous errors in spelling and grammar which detract from effective communication Lack of editing throughout paper Lack of clear and consistent labeling of headings and exhibits Writing lacks necessary documentation Formatting is inconsistent or absent Page numbers are missing Some minor problems with organization of information which slightly interferes with a reader’s ability to clearly understand the problem and outcomes of the analysis Some errors, mostly minor in nature may present There is evidence of attempts at proofreading Clearly communicates all perceived pros and cons of alternatives and recommendations All of the alternatives and recommendations are clearly connected to the central problem in a logical and effective manner Organization of information enhances a reader’s ability to clearly understand the problem and outcomes of the analysis Most of the alternatives and recommendations are connected to the central problem in a logical and effective manner -The writing is essentially error free in terms of spelling and grammar Most headings and exhibits are clearly and consistently labeled Content within the paper is mostly cited correctly and adequately Formatting is logical and consistent in most places in the paper Polished and professional appearance with headings and exhibits clearly and consistently labeled, pages numbered consecutively Content within the paper is cited correctly and adequately Formatting is consistent and logical throughout the paper TOTAL POINTS: Overall Assessment Score: POINTS 6-9 points = Does not meet expectations; 10-14: Meets expectations; 28 15-18: Exceeds expectations - Course: AACSB Assessment - Global Perspective Rubric - MBA Student: Rater: _ Trait Identifies and locates elements of Economic and Financial Environment of a country Does not meet expectation Cannot locate and identify basic facts about country such as exports, imports, foreign direct investment, tariffs, balance of payments, and non-tariff barriers point Analyses the competitiveness of a Country’s major industry using the Porter’s International Competitiveness Diamond Is not able to discuss the Porter International competitiveness Diamond in terms of factor conditions, demand conditions, related and supporting industries, and firm strategy, structure and rivalry points Is not able to clearly provide and discuss the basic elements of a country’s culture point Analyzes the cultural and social environment of a country Meets expectations Is able to locate and identify basic facts about country such as exports, imports, foreign direct investment, tariffs, balance of payments, and nontariff barriers points Is able to analyze and discuss the Porter International competitiveness Diamond in term s of factor conditions, demand conditions, related and supporting industries, and firm strategy, structure and rivalry points Is able to discuss in detail the different aspects of a country’s culture such as do’s and don’ts, negotiation strategies, business specific cultural practices, and can compare American culture using Hofstede’ scales points Exceeds expectations Is able to not only locate and identify basic facts about country such as exports, imports, foreign direct investment, tariffs, balance of payments, non-tariff barriers but also discuss in detail points Does an exceptional job of analyzing and discussing the Porter International competitiveness Diamond in terms of factor conditions, demand conditions, related and supporting industries, and firm strategy, structure and rivalry points Does an exceptional job of discussing in detail the different aspects of a country’s culture such as do’s and don’ts, negotiation strategies, business specific cultural practices, and can compare American culture using Hofstede’ scales points Total Overall Assessment Score: 4-5 points = Did not meet expectations; 7-9: Meets expectations; Greater than = Exceeds expectations 29 Total MBA IT Fluency Evaluation Matrix Assessment Beginning User Intermediate User Expert User Beginning User _ IT Skills & Concepts Intermediate User _ Expert _ Can useUser commonly used features of application software, has word processing and power point skills, and basic knowledge of how computers and internet work Can develop simple business applications using commonly used application software, has understanding of developments in computer hardware and software and their impact business environment Understands information processing and security issues Can learn to use a new piece of software without help; has advanced skills in using common application software, capable of doing critical analysis of trends in hardware/software for business environment and their impact on information security Beginning User _ IT Social & Ethical Intermediate User _Issues Expert User the societal _impact Understands and limitations of technology Aware of issues such as ID theft, spam, their impact on business & society, and can take simple steps to prevent these problems Understands challenges and impact of integration of technology in business and social environment Also aware of political challenges and business concerns in addressing these issues globally A critical thinker with an understanding of the economic, legal, and social issues surrounding the use of information and its impact on global workforce, environment and social structure 30 BeginningIT User _ Research / IntermediateStrategy User _ Expert User _catalog Uses the College’s library and periodical databases, but not fully aware of the process to evaluate sources Has basic awareness of copyright and plagiarism issues Navigates and evaluates information using a variety of academic and non-academic sources, formats, and media in research and problem solving Understands copyright and plagiarism issues Critically evaluates online sources including ebooks, journal, newspaper, and magazine articles and websites for issues of currency, authorship, authority, and design Uses a wide range of source of information to formulate business solutions APPENDIX C Indirect Assessments 31 Results of 2007 - 2008 Graduating Senior Student Survey Perceptions of Effectiveness (n=37) Please rate how effectively your business school courses contributed to your: Dimension Understanding of the functional areas of business Critical thinking and analytical decisionmaking skills Ability to work well in teams and relate interpersonally with others Oral presentation skills Written communication skills Awareness of ethical issues in business settings Proficiency in using software to analyze business problems Overall preparation for a career in business Extremely Effective Highly Effective Somewhat Effective Not Very Effective Not at All Effective (11%) Very Highly Effective 13 (36%) 14 (39%) (11%) (0%) (3%) (11%) 14 (38%) 12 (32%) (16%) (3%) (0%) (22%) 10 (27%) 14 (38%) (11%) (0%) (3%) (14%) (11%) 11 (30%) 11 (30%) 14 (38%) 13 (35%) (16%) (16%) (3%) (5%) (0%) (3%) (16%) (19%) 15 (41%) (19%) (3%) (3%) (16%) 10 (27%) 13 (35%) (16%) (3%) (3%) (11%) (24%) 15 (41%) (19%) (5%) (0%) 32 Results of 2007 - 2008 MBA Survey Perceptions of Effectiveness (n=43) “How effectively did the MBA program contribute to your development on the following dimensions?” Dimension Critical thinking & analytic decisionmaking skills Ability to maximize leadership skills Teambuilding skills Oral presentation skills Written communication skills Understanding of and ability to evaluate global business environment Ability to make ethical decisions Ability to use software/technology to enhance research, analysis and presentations Overall preparation for a career in business Very Highly Effective (5%) (5%) (2%) (5%) (5%) (2%) (2%) (2%) Highly Effective Somewhat Effective Not Very Effective Not at All Effective 19 (45%) 22 (51%) 24 (59%) 13 (30% 22 (51%) 17 (40%) 17 (40%) 20 (47%) 20 (48%) 17 (40%) 15 (37%) 25 (58%) 19 (44%) 22 (51%) 23 (53%) 20 (47%) (2%) (5%) (2%) (7%) (0%) (7%) (5%) (5%) (0%) (0%) (0%) (0%) (0%) (0%) (0%) (0%) (5%) 17 (40%) 22 (51%) (5%) (0%) 33