RISE Evaluation of Seminars Revised

UNIVERSITY OF THE VIRGIN ISLANDS RESEARCH INITIATIVE FOR SCIENCE ENHANCEMENT (RISE) PROGRAM: MID YEAR EVALUATION REPORT AUGUST 2001-JANUARY 2002 Background The RISE program is one of several collaborative training/research programs in the Sciences Division of the University of the Virgin Islands These programs include: (1) The Minority Access to Research Careers ( MARC) program funded by NIH, a faculty/student research program conducted, in part, in mainland university labs, (2) a NASA scholarship program, and a National Science Foundation (NSF) grant designed to promote excellence in science, mathematics , engineering and technology These programs are under the umbrella of Emerging Caribbean Scientists program and commonly share the goal of increasing the number of UVI students who are skilled, motivated and confident about their preparation for biomedical and other scientific careers Overview of the RISE Program The goal of the RISE Program is to overcome the geographic and intellectual isolation of the University of the Virgin Islands and increase the number of graduates that are skilled, motivated, and confident about their preparation to enter graduate studies in the sciences This is accomplished through a three-step process involving students, faculty and The University of the Virgin Islands Collaboration is structured between UVI faculty in different biomedical disciplines, students, the university, and mainland scientists in approved laboratories The student development portion is supported by exposure to a inquiry-based curriculum, research experiences, presentations of research results, participation in seminar series and special workshops, clubs, study groups and travel to mainland universities Participation in research programs on the mainland provides opportunities to interact with role models and broaden experiences in a student’s chosen field The faculty development portion provides faculty the opportunity to initiate collaborations with outstanding research scientists and travel to mainland laboratories with the goal of enhancing their professional careers and productivity The institutional development portion provides renovated space, capital equipment, materials, staffing, upper level course redesign to modernize molecular biology training, and the revamping of general chemistry Description of RISE Participants and Mentors RISE Participants 2001-2002 There are ten students supported by the RISE program in the 2001-2002 academic year Eight are female and two are male Four are sophomores, four are juniors and two are seniors All participants have a GPA of 2.5 or better, are US citizens and are classified as a minority student All students selected for the RISE program have demonstrated, by participation in other research based activities and written documentation, an interest in a career in biomedical research and motivation to obtain a Ph.D In addition to the ten students who are the cohort for the 2001-2002 school year, three additional students were designated RISE students during the summer term, and received support for mainland research projects Mentors Mentors for the RISE Fellows consist of two groups : There are five UVI based mentors at the University of the Virgin Islands and three at mainland universities throughout the United States The criteria for being a mentor are being an experienced researcher associated with a funded research laboratory.( NSF, NIH, etc.) and a willingness to mentor undergraduate students Mentor/Collaborators University researchers located at mainland universities also serve as collaborators with UVI faculty in the science and math division RISE Goal and Specific objectives Goal The specific goal of the RISE program is to provide sufficient opportunities, training, mentoring and advising to students coupled with experiences to strengthen faculty productivity that result in 50 percent or more of RISE students enrolling in graduate programs leading to a Ph.D degree Objectives The objectives listed are designed to be implemented during the projected four years of UVI RISE project Therefore, and not all objectives are addressed during the first year ( see appendix 1) Students will be provided the support to excel in gatekeeper courses (general chemistry, general biology, calculus, and physics) Retention of RISE freshmen students in biology, marine biology and chemistry programs will exceed 60% in school year 2002-2003 Student will have greater interest in and awareness of diversity of biomedical research after attendance at seminars and scientific meetings as measured by survey responses Students will become active scientific researchers as demonstrated by presentations at science seminars, scientific meetings, and reports from mentors At least RISE students will co-author publications in refereed journals Students will be knowledgeable in their field as demonstrated by course grades, mentor and faculty evaluations Student confidence will increase as demonstrated by performance in course work, application to graduate programs, scientific presentation, and co-authorship of articles Faculty in the Division of Science and Math will develop significant collaborative relationships which will lead to productive research careers as measured by progress reports from summer research experience, submitted manuscripts and grant proposals 8a.Modernized molecular biology instruction as demonstrated by refurbished lab, modular course development and implementation 8b.Restructuring of the General Chemistry curriculum Evaluation Plan of the RISE program Introduction The goal of the RISE Evaluation Plan is to measure the progress in achieving the stated goals and objectives of UVI RISE by assessing whether stated objectives are being met, activities implemented, and the efficacy of methods, materials, curriculum, and design being utilized to achieve them The evaluation is a collaborative effort among MBRSRISE staff, UVI office of Institutional Research, Program Directors of other MBRS projects , and the outside evaluators, Lce Consulting Evaluation is formative as well and summative in order to provide timely feedback to stakeholders which provides the opportunity to improve the project as it develops The ongoing evaluation process gathers data during the course of the project The MBRS-RISE staff collects, tabulates, and forwards to the external evaluators a variety of data gathered on RISE students within the University’s program The External evaluators analysis and interpret this data in addition to designing instruments to gather data, conducting observations, interviews, focus groups, record reviews, research and providing a written report Evaluation Plan Table ( appendix 1) presents the evaluation questions that framed the evaluation design, methods of data collection, and responsibility allocations Figure A summarizes the research methods and data collection used to complete the present evaluation Overview of Evaluation Activity August 2001- February 2002 RISE evaluation activities consisted of observations of Poster Presentations ( July 30 and September 22), design of student and mentor survey forms (see appendix 2), administration of student and mentor survey forms, faculty/mentor interviews, FIGURE A LCE EDUCATIONAL CONSULTING EVALUATION ACTIVITIES AUGUST 2001 -JANUARY 2002 OBSERVATION AND EVALATION: uU POSTER PRESENTATIONS July 30 September 22 OBSERVATION: MOLECULAR BIO LAB RENOVATION STUDENT: RISE PROGRAM SURVEY FORMS -Summer Research -Presentations RISE: EVALUATION METHODS AND DATA COLLECTION DOCUMENT REVIEW: -GRADES – GATEKEEPING COURSES -FACULTY EVALU ATIONS -STUDENT EVALUATIONS -CURR DOCUMENTS -FACULTY PAPERS -RESPONSES: ABSCMS EVALUATION FORMS MENTORS: RISE PROGRAM SURVEY FORM INTERVIEWS: RISE FELLOW FACULTY/ S ADMINSTARTORS - FOCUS GROUPS: RISE FELLOW S participation/ presentations at the joint MBRS Marc and RISE advisory council meeting, meetings with RISE project Director, conducting focus groups, document reviews, data collection and analysis, and reporting Poster Presentation: The observation of Poster presentation (July 30) was to gather baseline data relative to objectives and and to establish rapport with participant stakeholders Only three of the six RISE students who had a summer research experience presented on July 30 due to weather condition that prohibited St Croix participants from attending The participants were in the areas of Math, Computer Science/Engineering, and Psychology The three poster presentations were observed, photographed and RISE presenter interviewed The evaluator, Dr Creque, gathered data on confidence level, opinions on summer mentoring experience and plans for graduate school Each participant was observed for approximately 25 minutes Confidence level was defined as the ability to:  -answer all questions posed by visitors and evaluator  -explain poster exhibit in detail  -speak knowledgeably about content  -make eye contact with visitors and observer  -remain relaxed (smile, speak slowly and distinctly, etc) Each criterion had a value of 20 Using these criteria , during the period of observation all three students met the five criteria of exhibiting confidence (minimum scored: 80) Summer Mentoring Experience: RISE participants were asked to describe their summer mentoring experience All responses were positive Comments were as follows: Got to see what I was doing Wonderful to have one-on-one I had an opportunity to participate in almost everything The experience more intense than two weeks in a regular class The research encouraged me to go on to graduate school (I see) what I am moving toward, I can only achieve at the Masters level Helped me to focus on what to study in grad school Plans for Graduate school All three students interviewed plan to go to graduate school, however, their discussion and comments center on the Master’s level Only one student, when asked, responded the continuing on to a Ph.D was “possible.” Development and Administration of RISE Student and Mentor Evaluation Forms RISE Students Survey forms were designed to collect baseline data on RISE students assessment of their likelihood of entering a Ph.D program, the quality of counseling and support , level of proficiency in their field and awareness of the biomedical or other research areas after Graph 1: Student/Mentor Comparason: Proficiencies (Means) = Very Proficient 5 1= Not Proficient at All 4.9 4.7 4.4 4 4 4 3.7 Overall Grasp of Material Appying to Graduate Program Preparation For GRE Students Passing Oral Exam Mentors Making paper presentation at conference CO-authoring articles their summer RISE experience These questions relate to the overall goal of the RISE program and specific objectives, 2,3, 4, and RISE Mentors Survey forms for RISE mentors paralleled question in RISE Student Self-report survey instrument Mentors were asked to assess how likely it was for their mentee to enter Ph.D program, proficiency in content area and medical research, and their level of awareness of biomedical and other research fields Mentors were also asked to assess their ability as a mentor Direct contact was made with RISE Students and Mentors to explain the survey and respond to concerns Mentors located on the mainland were telephoned Only one mentor was not spoken to directly and contact was made through Email Results: RISE Program for 2001 There were seven students and five mentors that responded to the program evaluations Six out of seven of the students said that they intended to pursue a Ph.D program Of those expressing a desire to continue, students said that they were planning to wait at least a year before going on, and one student said that they were planning to go in the next 10-12 months The person who said that they didn't want to go on said that they wanted to go into medicine rather than get a Ph.D Some of the universities students are considering are Virginia Commonwealth, Howard, Georgia State, Florida State, Georgia Institute of Technology, Stanford, Boston and Ross Both students and advisors tended to rate nearly all areas of the quality of academic advising high Graph compares student and advisor responses There were only two areas where there appears to be a significant difference between the means of the student and mentor responses (p