This lecture includes these contents: How to do meta analysis, the popularity of meta analyses, four steps of meta analysis, identify your studies, how to search for literature,... Invite you to consult this lecture.
How to do Meta Analysis Arindam Basu Associate Director, Fogarty International Training Program Kolkata, India February, 2005 arin.basu@gmail.com phone: 919830153666 What is meta analysis? Quantitative approach for systematically combining results of previous research to arrive at conclusions about the body of research What does it mean? ● Quantitative : numbers ● Systematic : methodical ● combining: putting together ● previous research: what's already done ● conclusions: new knowledge The popularity of meta analyses publications 3000 Number of Publications 2500 2000 1500 1000 500 93-94 94-95 95-96 96-97 97-98 98-99 99-00 2000-1 2001-2 2002-3 2003-4 Year of Publications Number of Meta Analysis publications are steadily increasing since 1993. We graphed the counts of journal articles included “meta analysis” as “publication type” from Pubmed, from years 1993 through 2004 Four Steps of Meta Analysis ● Identify your studies ● Determine eligibility of studies Inclusion: which ones to keep – Exclusion: which ones to throw out – ● Abstract Data from the studies ● Analyze data in the studies statistically Identify your studies ● Be methodical: plan first ● List of popular databases to search ● – Pubmed/Medline – Embase – Cochrane Review/Trials Register Other strategies you may adopt – Hand search (go to the library ) – Personal references, and emails – web, eg. Google (http://scholar.google.com) How to Search for literature ● Formulate your question appropriately ● If you are searching pubmed Use Medical Subject Headings (MeSH) [1] – Lookup word in text word, abstract, title [2] – Combine [1] with [2] using boolean logic – Set up proper filters – ● For Others, use text word, abstract Huh?!, what is boolean logic?? Boolean Logic: AND this is AND zone, covering common area between two ellipses Boolean Logic OR this is boolean OR, covering the two ellipses Example: Research Issue Let's say we want to know whether streptokinase is protective for death from acute myocardial infarction. How should we set up a search strategy? We will search pubmed only Choose your DATABASE here Remember to choose both PUBMED, and MESH for formulating search. Choose PUBMED CENTRAL for free articles! Keep some, throw out others ● ● Cannot include all studies Keep the ones with – – – ● ● ● high levels of evidence good quality check with QUOROM guidelines Usually, MA done with RCTs Case series, and case reports definitely out Selection problems are major problems – read the article I sent MA = Meta Analysis; RCT = Randomized Controlled Trial Plan of Action ARE THE STUDIES ELIGIBLE FOR MA (STEP I)? NO DISCARD YES ABSTRACT THE DATA ENTER INTO A SPECIFIED FORMAT How to Abstract Data: Guidelines ● Create a spreadsheet (Excel, or OpenOffice Calc) ● For each study, create the following columns: – name of the study – name of the author, year published – number of participants who received intervention – number of participants who were in control arm – number who developed outcomes in intervention – number who developed outcomes in control arm Let’s do that to our streptokinase myocardial infarction study, next … Spreadsheet Data for Strepto Study We got like 22 studies to do our meta analysis, after all We created seven columns trial: trial identity code trialname: name of trial year: year of the study pop1: study population deaths1: deaths in study pop0: control population deaths0: deaths in control Analyze Data Statistically ● ● Combine data to arrive at a summary, 3 measures – Effect Size (Odds Ratio) – Variance with 95% Confidence Interval – Test of heterogeneity Two Graphs – Forest Plot – Funnel Plot ● Examine why the studies are heterogeneous, if they are ● Use Statistical Packages, several choices Let’s see what we got for streptokinase versus deaths from AMI Summary Estimates for Strepto Study Mantel Haenszel OR=0.77 95% Confidence Interval [0.72, 0.83] Test of Heterogeneity: Chisquare (df=21) = 31.5 PValue = 0.07 The pooled Odds Ratio shows that those receiving streptokinase at AMI are about 77% at risk of death (23% less likely to die) That in 95 out of 100 such meta analyses, the pooled Odds Ratio would lie between 0.72 and 0.83, indicating a statistically significant protective effect That these studies were not significantly heterogeneous Forest Plot The dotted line passes across null, or 1.0 The Risk Estimate of each study is lined up on each side of the dotted line, with 95% CI spread as the line The diamond in the below is the summary estimate The two ends of the diamond indicate 95% CI The size of the black square box indicates weight of the study They call it a forest plot so that you don’t miss the wood for the trees! Funnel Plot: what and how to read Plots the effect size against the sample size of the study To study a funnel plot, look at its LOWER LEFT corner, that’s where negative or null studies are located If EMPTY, this indicates “PUBLICATION BIAS” Note that here, the plot fits in a funnel, and that the left corner is not all that empty, but we cannot rule out publication bias Issues in meta analysis ● Choosing a model – ● ● Bias in meta analysis – poor quality of trials – publication bias Quality control in meta analysis – ● Fixed effects model or random effects? QUOROM guidelines Statistical Software for meta analysis Fixed Effects or Random Effects Model? Fixed Effects Model Random Effects Model • conduct if it is reasonable • Conduct if test of to assume underlying Rx heterogeneity is effect is SAME for all significant (shows studies heterogeneity) • Pooling: Mantel Haenszel • Assume that TRUE log OR odds ratio comes from a • Test: test of heterogeneity normal distribution • If significant, go for • Method: DerSimonian random effects model • short 95% CI for summary Lair’s method (DSL) of • smaller summary estimate calculating Odds’ Ratio • OR=0.78 [0.69,0.88] • OR=0.77 [0.72,0.83] ● ● Bias in Metaanalysis Poor Quality of Trials – To avoid them, learn more at CONSORT statement [http://www.consortstatement.org] Publication Bias – study showing beneficial effects of new treatment more likely to be published than one showing no effect – negative trials assumed to contribute less; never show up in the literature base – use several approaches to avoid this – Use Funnel Plots to examine the influence of publication bias Quality Control in MA:QUOROM Table • Detailed Guidelines • A Good Checklist • Use it for reporting • Meta Analysis • Systematic reviews Statistical Software for Meta Analysis ● Huge Checklist [http://faculty.ucmerced.edu/wshadish/] ● ● Free Software: – EpiMeta: from Epi Info – Revman: from Cochrane Collaboration – “meta” package in R for statistical computing Nonfree – meta module in STATA Summarizing ● Defined meta analysis – ● Outlined basic steps – – – – ● quantitative research synthesis Information retrieval Data Abstraction Data Analysis Model Selection: Fixed Effects or Random effects Outlined some issues and listed software Feel free to shoot questions at arin.basu@gmail.com ... number who developed outcomes in control arm Let’s do that to our streptokinase myocardial infarction study, next … Spreadsheet Data for Strepto Study We got like 22 studies to do our meta analysis, after all We created seven columns... publication bias Issues in meta analysis ● Choosing a model – ● ● Bias in meta analysis – poor quality of trials – publication bias Quality control in meta analysis – ● Fixed effects model or random effects?... Statistical Software for meta analysis Fixed Effects or Random Effects Model? Fixed Effects Model Random Effects Model • conduct if it is reasonable • Conduct if test of to assume underlying Rx