Lecture Meta-analysis

This lecture includes these contents: Meta analysis, a review of literature, function of meta analysis, the historical background, fixed effect model,... Invite you to consult this lecture.

Meta-analysis The EBM workshop A.A.Haghdoost, MD; PhD of Epidemiology Ahaghdoost@kmu.ac.ir Definition Meta-analysis: a type of systemic review that uses statistical techniques to quantitatively combine and summarize results of previous research A review of literature is a meta-analytic review only if it includes quantitative estimation of the magnitude of the effect and its uncertainty (confidence limits) EBM workshop Meta analysis A.A.Haghdoost Function of Meta-Analysis(1) 1-Identify heterogeneity in effects among multiple studies and, where appropriate, provide summary measure 2-Increase statistical power and precision to detect an effect 3-Develop ,refine, and test hypothesis continued EBM workshop Meta analysis A.A.Haghdoost Function of Meta-Analysis(2) continuation 4-Reduce the subjectivity of study comparisons by using systematic and explicit comparison procedure 5-Identify data gap in the knowledge base and suggest direction for future research 6-Calculate sample size for future studies EBM workshop Meta analysis A.A.Haghdoost Historical background Ideas behind meta-analysis predate Glass’ work by several decades – R A Fisher (1944) • “When a number of quite independent tests of significance have been made, it sometimes happens that although few or none can be claimed individually as significant, yet the aggregate gives an impression that the probabilities are on the whole lower than would often have been obtained by chance” (p 99) • Source of the idea of cumulating probability values – W G Cochran (1953) • Discusses a method of averaging means across independent studies • Laid-out much of the statistical foundation that modern metaanalysis is built upon (e.g., inverse variance weighting and homogeneity testing) EBM workshop Meta analysis A.A.Haghdoost Basic concepts The main outcome is the overall magnitude of the effect It's not a simple average of the magnitude in all the studies Meta-analysis gives more weight to studies with more precise estimates – The weighting factor is 1/(standard error)2 EBM workshop Meta analysis A.A.Haghdoost Main magnitude of effects Descriptive Mean Prevalence Analytical Additive Mean difference Standardized mean difference Risk, rate or hazard difference Correlation coefficient Multiplicative Odds ratio, Risk, Rate or Hazard Ratio EBM workshop Meta analysis A.A.Haghdoost Statistical concepts(1) You can combine effects from different studies only when they are expressed in the same units Meta-analysis uses the magnitude of the effect and its precision from each study to produce a weighted mean EBM workshop Meta analysis A.A.Haghdoost Statistical concepts(2) The impact of fish oil consumption on Cardio-vascular diseases EBM workshop Meta analysis A.A.Haghdoost Forest plot the graphical display of results from individual studies on a common scale is a “Forest plot” In the forest plot each study is represented by a black square and a horizontal line (CI:95%).The area of the black square reflects the weight of the study in the meta-analysis A logarithmic scale should be used for plotting the Relative Risk EBM workshop Meta analysis A.A.Haghdoost Quality score Some meta-analysts score the quality of a study – Examples (scored yes=1, no=0): • Published in a peer-reviewed journal? • Experienced researchers? • Research funded by impartial agency? • Study performed by impartial researchers? • Subjects selected randomly from a population? • Subjects assigned randomly to treatments? • High proportion of subjects entered and/or finished the study? • Subjects blind to treatment? • Data gatherers blind to treatment? • Analysis performed blind? EBM workshop Meta analysis A.A.Haghdoost Quality score Use the score to exclude some studies, and/or… Include as a covariate in the metaanalysis, but… Some statisticians advise caution when using quality EBM workshop Meta analysis A.A.Haghdoost Quality scoring A very common practice is to weight studies on a quality score usually based on some subjective assignment For example, 10 quality points for a cohort design, points for a nested case control design, and points for a population based case control design EBM workshop Meta analysis A.A.Haghdoost Quality scoring Quality scoring submerges important information by combining disparate study features into a single score It also introduces an unnecessary and somewhat arbitrary subjective element in to the analysis EBM workshop Meta analysis A.A.Haghdoost Quality scores as weighing factors study weight=1/var Quality adjusted weight= quality score /var EBM workshop Meta analysis A.A.Haghdoost Quality scores The judgment that the studies should or should not be combined should be stated and justified explicitly There is some of a tendency to make this judgment on the basis of the quantitative results, but it’s critical to make a qualitative judgment EBM workshop Meta analysis A.A.Haghdoost What is an IPD Meta-analysis? Involves the central collection, checking and analysis of updated individual patient data Include all properly randomised trials, published and unpublished Include all patients in an intention-to-treat analysis EBM workshop Meta analysis A.A.Haghdoost IPD Meta-analysis Individual patient data used Analysis stratified by trial IPD does not mean that all patients are combined into a single mega trial EBM workshop Meta analysis A.A.Haghdoost IPD Analyses Collect raw data from related studies, whether or not the studies collaborated at the design stage, exposures measures and other covariates that can be applied uniformly across the studies combined The major advantage of a IPD over an MA is the use of individual-based rather than group-based data EBM workshop Meta analysis A.A.Haghdoost sensitivity analysis In sensitivity analysis, the sensitivity of inference to variations in or violations of certain assumptions is investigated For example, the sensitivity of inference to the assumption about the bias produced by failure to control for smoking can be checked by repeating the meta-analysis using other plausible values of the bias EBM workshop Meta analysis A.A.Haghdoost sensitivity analysis If such reanalysis produces little change in an inference, one can be more confident that the inference is insensitive to assumptions about confounding by smoking In influence analysis, the extent to which inferences depend on a particular study or group of studies is examined; this can be accomplished by varying the weight of that study or group EBM workshop Meta analysis A.A.Haghdoost sensitivity analysis Thus , in looking at the influence of a study, one could repeat the meta-analysis without the study, or perhaps with half its usual weight If change in weight of a study produces little change in an inference, inclusion of the study can not produce a serious problem, even if unquantified biases exist in the study EBM workshop Meta analysis A.A.Haghdoost Sensitivity and influence analysis On the other hand, if an inference hinges on a single study or group of studies, one should refrain from making that inference EBM workshop Meta analysis A.A.Haghdoost conclusion Most meta-analysis will require from each study both a point estimate of effect and an estimate of its standard error A point estimate accompanied only by a P value will generally not provide for accurate computation of a standard error estimate, and should not be considered sufficient for reporting purposes EBM workshop Meta analysis A.A.Haghdoost Over conclusion Like large epidemiologic studies, metaanalysis run the risk of appearing to give results more precise and conclusive that warranted The lager number of subjects contributing to a meta-analysis will often lead to very narrow confidence intervals for the effect estimate EBM workshop Meta analysis A.A.Haghdoost ...Definition Meta-analysis: a type of systemic review that uses statistical techniques to quantitatively combine... effect and its uncertainty (confidence limits) EBM workshop Meta analysis A.A.Haghdoost Function of Meta-Analysis( 1) 1-Identify heterogeneity in effects among multiple studies and, where appropriate,... 3-Develop ,refine, and test hypothesis continued EBM workshop Meta analysis A.A.Haghdoost Function of Meta-Analysis( 2) continuation 4-Reduce the subjectivity of study comparisons by using systematic