Evidence-Based Dentistry: An Introduction pptx

EVIDENCE BASED DENTISTRY 0011–8532/02 $15.00 ‫00 ם‬ WHAT IS EVIDENCE BASED DENTISTRY? Gary R Goldstein, DDS The volume of literature and lectures directed at the modern dental practitioner has created some problems How does one resolve the often contradictory information? How does one determine what is a cuttingedge technique and what is useless? In resolving a clinical decision, evidence rather than empiricism should dictate treatment Evidence based dentistry (EBD), based on the concepts developed at MacMaster University,13, 14, 17–22 presents guidelines to determine the validity of study results and whether they can be applied to clinical practice The foundation for evidence based practice was laid by David Sackett who has defined it as ‘‘integrating individual clinical expertise with the best available external clinical evidence from systematic research.’’23 Evidence based dentistry supplies guidelines to help the clinician make an intelligent decision In and of itself, EBD does not give definitive answers It does not exchange the tyranny of the expert for the tyranny of the literature As Sackett’s definition states, EBD relies first on clinical expertise This expertise is especially critical in dentistry, where the number of randomized, controlled clinical trials and prospective cohort studies is limited In a perfect world, full of quality prospective studies, one would only have to pull up a well-performed metaanalysis or systematic review of the evidence on the clinical question to solve the problem at hand Unfortunately, these studies are too few, and clinicians must apply the best available evidence to make a decision The Cochrane Collaboration, an international nonprofit organization whose goal is to make up-to-date, accurate information on the effects of health care available worldwide, has an Oral Health Group that has produced some systematic reviews Their web site (http://hiru.mcmaster ca/cochrane/default/htm) is an excellent place to see what the evidence based dental practice in the future will be like From the Department of Prosthodontics, New York University College of Dentistry; and Department of Dental Material Science, New York University Graduate School of Arts and Sciences, New York, New York DENTAL CLINICS OF NORTH AMERICA VOLUME 46 • NUMBER • JANUARY 2002 GOLDSTEIN The internet has made it easy to initiate an evidence based practice (see article by Felton on page 45 of this issue) Guidelines for EBD are applicable to peer-reviewed literature and also to publications and lectures that provide a case report or, at best, a case series done under conditions that may not be similar to those seen in the average dental office Armed with the tools of EBD, the clinician can readily evaluate the mass of data and choose, in an educated manner, what to use and what to discard Unfortunately, most of what is seen in dentistry is product testing done in laboratories, not operatories The studies are usually univariate analyses, because the researcher has been trained to homogenize the study so that only one variable is tested Clinicians, however, live in a multivariate environment For example, an in vitro study on a dental cement might deal with retention of castings on extracted teeth Retention, however, is not the only variable that a clinician evaluates in choosing a cement A clinician must also be concerned with postoperative sensitivity, film thickness, setting time, working time, longevity, ability to clean up, setting expansion, and so forth One might also wonder how good the retention would be in a clinical milieu where isolation, crevicular fluid, saliva, and intraoral humidity become confounding variables Clinicians, seeing only one variable tested, should be reluctant to change their cement based on the limited laboratory study Needed instead are controlled, long-term clinical trials to help clinicians make decisions, but such studies are expensive and require a long time to supply the information Chambers questioned whether ‘‘there is clinical evidence showing that this restorative material will last longer in patient’s mouths then it will be on the market’’ (see article by Chambers on page 29 of this issue) Using EBD is quite simple:3 Create an answerable question Track down the best evidence to answer the question Critically appraise the information Apply the results to one’s patients Evaluate one’s performance The Journal of Prosthetic Dentistry has published a series similar to the User’s Guide to the Medical Literature,13, 14, 17–22 specific to dentistry, to help appraise the information.1, 2, 5, 6, 8, 11, 12, 15, 16 Although the guidelines differ for the different clinical question being asked, certain characteristics pertain to all studies THE USE OF EVIDENCE BASED DENTISTRY IN DETERMINING THERAPY Was the Assignment of Patients to Treatment Randomized? Randomization eliminates allocation bias In theory, randomization ensures that variables, over which the study has control and the un- WHAT IS EVIDENCE BASED DENTISTRY? known variables that come in to play in all studies, are equally distributed among the test groups To ensure equal distribution, the study population (N) must be sufficiently large A randomized controlled trial (RCTs) is considered the optimal research design and is the reference standard for most clinical questions Not all RCTs, however, are properly planned and carried out The reader must still examine the methodology Also, as Sackett concluded, ‘‘some questions about therapy not require randomized trials (successful interventions for otherwise fatal interventions) or cannot wait for the trials to be conducted And if no randomized trial has been carried out for our patient’s predicament, we follow the trail to the next best external evidence and work from there.’’23 Feinstein9 has questioned the blind faith often put in randomized trials and has suggested that prognostic stratification is critical to the utilization of the data He maintains that if data are to be evaluated in prognostic subgroups, those subgroups should be identified, where possible, before the study starts, and that subjects should be allocated to those subgroups before they are randomly allocated to treatment.10 For example, in a study on implants in which the site (anterior mandible versus posterior maxilla) is a major variable, it would be sensible to identify the site before randomizing to ensure that chance alone does not place most of the anterior mandibles in one group and most of the posterior maxillae in the other Another potential confounder would be smoking Although it would be unwieldy, if not impossible, to identify every possible variable, certain dominant ones known to affect the outcome of the therapy should be identified at the start of the project Were All Patients Who Entered the Trial Properly Accounted For and Attributed For at its Conclusion? It is critical that all patients who enter a trial are properly accounted for at its conclusion It is not enough to say that a certain number of patients dropped out One must include the dropouts in the statistical analysis (see article by Clive on page 137 of this issue) The most common reason patients drop out of a therapy trial is because they are unhappy with the therapy Some subjects die, and some move out of the area, but the number in these categories should be relatively equal in the control and test groups If the drop-out rate exceeds 20%, the clinician should be concerned about the external validity or generalizability of the project Were Patients, Their Clinicians, and Study Personnel Blinded to Treatment? Blinding means that someone was not aware of the treatment being rendered Double-blinded means that both the evaluators and the patients GOLDSTEIN were unaware of the therapy being rendered Blinding is easily done in a drug trial in which the pills look and taste the same and the patient is identified only by a code number unknown to the evaluator looking at the outcome Blinding can also be easily done in a study of toothpastes or mouthwashes It is not always possible to blind a clinical trial For example, in a study comparing implant-retained overdentures with either two or four fixtures in place, it would be impossible to blind the patient or the researcher if intraoral examinations were necessary Although a nonblinded trial is not ideal, it can still be an excellent experiment that can generate usable, reliable data Were the Groups Similar at the Start of the Trial? To ensure validity, it is critical that the cohorts (groups) be similar in all pertinent demographic, medical, and dental factors Although in a large study randomization should ensure equivalence, it is the investigators responsibility to assess equivalence among cohorts in detail Aside From the Experimental Intervention, Were the Groups Treated Equally? Anything one studies, one alters Patients who agree to participate in a study tend to be more compliant than the average Knowing they are to be examined may cause them to exercise better home care before presenting in an effort to please the investigator It is tempting for investigators to recall a test group more often when the outcome is uncertain or side effects are suspected Co-interventions, such as an extra prophylaxis, can affect the primary outcome being examined and the validity of the study All groups need be treated equally Were All Clinically Important Outcomes Considered? The reader must decide whether all clinically important outcomes have been considered If, for example, in evaluating a new cement for ceramic restorations, the investigator reports only that the restoration was in place after the time of the study, it is obvious that other important considerations have been ignored If the investigator also evaluates postoperative sensitivity, film thickness, setting time, working time, longevity, ability to clean up, setting expansion, and so forth, the important clinical factors have been evaluated More commonly, the investigation might evaluate only two of the factors Some clinicians would find the study adequate; other readers might not An implant study, for example, might speak of prosthesis stability and neglect the number of implants WHAT IS EVIDENCE BASED DENTISTRY? remaining If six implants were placed and three were lost, the prosthesis might be stable, but the clinician has cause to question the data Was Follow-up Sufficiently Long and Complete? Too often a study is not long enough to be valid to the clinician (chronology bias) Although a 1-year follow-up may be sufficient in a study of the efficacy of tetracycline-impregnated cord, the same followup time is not adequate in a study on a new composite resin restoration For restorative procedures, a minimum of to years may be necessary to convince a dentist to change therapy Were Objective and Unbiased Outcome Criteria Used? Outcome criteria are chosen by the investigator, and it is easy to err by choosing an assessment that best serves the theory of the investigator The adage, ‘‘I would not have seen it if I didn’t believe it,’’ readily comes into play Picture a study that compares a Lexus with a Yugo and chooses the following criteria for the study: Does Does Does Does Does it it it it it have have have have have an engine? a radio? four wheels? windshield? seat belts? Using these criteria, one concludes that the Lexus and Yugo are similar Any rational person, however, clearly sees that results based on questionable outcome assessments are useless In more sophisticated studies, such a flaw may not be so obvious Will the Results help Clinicians in Caring for Their Patients? The critical question for clinicians is whether the results will help them provide better care for their patients, because that question involves all the others If the methodology is good, if the statistically significant results have clinical relevance, and if the data interpretation is rational, one would lean towards accepting the study If, however, the population is not representative of a clinician’s practice or if the inclusion and exclusion criteria not match the practice population, clinicians should be hesitant about applying the results to the population they are treating.3 GOLDSTEIN USING EVIDENCE BASED DENTISTRY TO EVALUATE THE NEED FOR A DIAGNOSTIC TEST Was There an Independent, Blind Comparison with a Reference Standard? A gold (reference) standard is important In histopathology, the biopsy is considered the gold standard, but even the biopsy does not result in 100% agreement among pathologists The disagreement is magnified when the pathologists are deprived of the clinical findings supplied by the surgeon If a reference standard exists, one might question the need for the new test If the test cannot offer the advantages of being less expensive, or less invasive, or easier to perform, one should question its use Unfortunately, often there is no reference standard, or the reference may be controversial Lack of a suitable reference standard does not mean that the new test is not useful, but a heavier burden of proof is demanded from the investigator, and the clinician must exercise more caution Were the Methods for Performing the Test Described in Sufficient Detail to Permit Replication? If the reader cannot perform the test, it is of no use Were Sensitivity, Specificity, Positive Predictive Value, Negative Predictive Value, and Likelihood Ratios Presented? It is not the reader’s responsibility to undertake statistical analysis when reading an article Rather, it is the researchers’ obligation to supply the appropriate data (see article by Brunette on page 87 of this issue) Because EBD puts the onus of decision making on the clinician, readers must be familiar with the terms so they can determine if the new test would have merit in their practices Will the Patient be Better Off as a Result of the Test? Routine testing, if it does not affect the diagnosis, prognosis, or treatment, has questionable value If the results not potentially change the course of treatment, the test is unnecessary A patient who fell and knocked out the coronal portion of a tooth would benefit from a radiograph to determine the extent of the fracture but not from a pulp test to determine vitality An adolescent with an ulceration from biting the cheek would be better served by a reexamination in a week rather than by a biopsy Evidence based dentistry will surely be abused.4 Insurance compa- WHAT IS EVIDENCE BASED DENTISTRY? nies have already developed evidence based care policies that require dentists to prove that patients need the services.3 The possibility of abuse does not mean that dentistry should reject EBD Indeed, dentists have been practicing EBD, in part, for many years When clinicians tell patients to brush and floss, they so because the evidence supports the efficacy of these interventions When dentists advocate fluoride, they so because the evidence supports its efficacy Although many areas of dental practice are supported by numerous high-quality research projects, many more areas are supported only by anecdotal data Hence, the validity of the data and who evaluates it become critical Aurbach4 has questioned: ‘‘Who will be the anointed one or group that determines which evidence is valid? Who will set the research agenda and determine where the results will be maintained? Who will validate the research? Who will maintain the data base to make sure that it is up to date? How will the results be used?’’3 It is obvious that to control the data, clinicians need to own it If clinicians are not sophisticated enough to force good research practices by their ability to evaluate and reject poor science, they will be at the hands of third parties who can use dubious research as justification to control clinicians’ practices The sooner dentistry as a profession universally embraces EBD, the sooner the profession will command the use of research and prevent its misuse WHAT EVIDENCE BASED DENTISTRY IS NOT Evidence based dentistry is not a veil to mask the same old, inadequate research It is disturbing to see lecturers invoke EBD and present the same anecdotal lectures they gave before, with different slide titles As the profession of dentistry becomes more sophisticated, researchers and lecturers will be forced to grow also Evidence based dentistry does not take the clinical decisions out of clinicians hands and put them into the hands of the literature In fact, the opposite is true Evidence based dentistry gives guidelines for the clinician and relies first on clinical expertise Evidence based dentistry does not mean that third parties will control dental practices In fact, educated dentists, understanding the literature, will be able to prevent the misrepresention of data by commercial interests Evidence based dentistry does not mean the clinician need not study basic and dental material sciences In fact, the opposite is true To evaluate the research presented, clinicians need a solid background on which to base their evaluations and decisions Evidence based dentistry does not mean clinicians abandon everything they learned in dental school It does not force clinicians to go backwards to justify things the profession universally accepts GOLDSTEIN WHO BENEFITS FROM EVIDENCE BASED DENTISTRY • The ultimate beneficiaries of EBD are members of the public, who will reap the rewards of better care The internet allows patients, as well as professionals, access to health care information The public, however, does not have the tools to evaluate the data adequately and must rely on their educated dentists to help sort fact from fiction Patients will be more educated, more involved in their treatment decisions, and more appreciative of quality care • Dentists, who will also benefit from EBD Instead of conducting free product testing for dental product manufacturers, practitioners will have at their disposal more valid research on which to predicate their clinical decisions • Researchers, who will benefit by being called upon to the clinical testing necessary before new products are placed on the market References Anderson JD: Need for evidence based practice in prosthodontics J Prosthet Dent 83:58–65, 2000 Anderson JD, Zarb GA: Evidence based dentistry: Prognosis J Prosthet Dent 83:495– 500, 2000 Anderson V: Evidence based care, is the defense ready? Dental Economics 28–32, 2000 Aurbach FE: Evidence based dentistry: A practitioner’s perspective J Am Col Dent 66:17–20, 1999 Carr AB, McGivney GP: Measurement in dentistry J Prosthet Dent 83:266–271, 2000 Carr AB, McGivney GP: Users’ guides to the dental literature: How to get started J Prosthet Dent 83:13–20, 2000 Chambers D: Research for practitioners or research for researchers? J Am Coll Dent 65:2–4, 1998 Eckert SE, Goldstein GR, Koka S: How to evaluate a diagnostic test J Prosthet Dent 83:386–391, 2000 Feinstein AR: An additional basic science for clinical medicine: II The limitations of randomized trials Ann Intern Med 99:544–550, 1983 10 Feinstein AR: An additional basic science for clinical medicine: III The challenges of comparison and measurement Ann Intern Med 99:705–712, 1983 11 Felton DA, Lang BR: The overview: An article that interrogates the literature J Prosthet Dent 84:17–21, 2000 12 Goldstein GR, Preston JD: How to evaluate an article about therapy J Prosthet Dent 83:599–603, 2000 13 Guyatt GH, Sackett DL, Cook DJ: Users’ guides to the medical literature II How to use an article about therapy or prevention A Are the results of the study valid? Evidence-Based Medicine Working Group JAMA 270:2598–2601, 1993 14 Guyatt GH, Sackett DL, Cook DJ: Users’ guides to the medical literature II How to use an article about therapy or prevention B What were the results and will they help me in caring for my patients? Evidence-Based Medicine Working Group JAMA 271:59–63, 1994 15 Jacob RF, Carr AB: Hierarchy of research design used to categorize the ‘‘strength of evidence’’ in answering clinical dental questions J Prosthet Dent 83:137–152, 2000 16 Jacob RF, Lloyd PM: How to evaluate a dental article about harm J Prosthet Dent 84:8–16, 2000 17 Jaeschke R, Guyatt G, Sackett DL: Users’ guides to the medical literature III How to WHAT IS EVIDENCE BASED DENTISTRY? 18 19 20 21 22 23 use an article about a diagnostic test A Are the results of the study valid? EvidenceBased Medicine Working Group JAMA 271:389–391, 1994 Jaeschke R, Guyatt GH, Sackett DL: Users’ guides to the medical literature III How to use an article about a diagnostic test B What are the results and will they help me in caring for my patients? The Evidence-Based Medicine Working Group JAMA 271:703–707, 1994 Laupacis A, Wells GA, Richardson S, et al: Users’ guides to the medical literature V How to use an article about prognosis JAMA 272:234–237, 1994 Levine M, Walter S, Lee H, et al: Users’ guides to the medical literature IV How to use an article about harm JAMA 271:1615–1619, 1994 Oxman AD, Cook DJ, Guyatt G: Users’ guides to the medical literature VI How to use an overview JAMA 272:1367–1371, 1994 Oxman AD, Sackett DL, Guyatt GH: Users’ guides to the medical literature I How to get started The Evidence-Based Medicine Working Group JAMA 270:2093–2095, 1993 Sackett D, Richardson WS, Rosenberg W, et al: Evidence based Medicine: How to Practice and Teach EBM New York, Churchill Livingstone, 1997 Address reprint requests to Gary R Goldstein, DDS NYU College of Dentistry Division of Restorative and Prosthodontic Sciences 345 East 24th Street Clinic 5W New York, NY 10010–4086 e-mail: gary.goldstein@nyu.edu EVIDENCE BASED DENTISTRY 0011–8532/02 $15.00 ‫00 ם‬ THE QUESTION James D Anderson, BSc, DDS, MScD HOW QUESTIONS ARISE There are two aspects to the clinical practice of dentistry The surgical component includes all the manipulation of hard and soft tissue that is performed every day in dental practice Examples are tooth preparation and restoration, scaling, orthodontics, and prosthesis fabrication The other element involves decision making The diagnosis of unlocalized dental pain, the prognosis for a periodontally compromised tooth, the choice of posterior restorative materials, and the risks/benefits assessment of third molar extractions are examples Early in the career, decision making may be the most difficult aspect of clinical practice There is an overwhelming array of choices with little or no structure on which to build an approach to solving the problems As a practitioner gains experience, he or she acquires the advantage having seen the results of previous decisions, good and bad, and can recall how a problem was dealt with previously The practitioner also develop habits that make each task easier Habits, too, are the result of decisions made but not re-examined As a start, the thoughtful practitioner will ask first if there is a compelling reason to intervene for a patient, and second if there is a compelling reason to intervene at this time The answers to these questions can be obvious or elusive The patient who has severe, throbbing pain and tender swelling over the apex of a heavily carious lateral incisor with a large periapical radiolucency clearly needs treatment and needs it promptly On the other hand, whether or when to treat the young patient with impacted but asymptomatic third molars is less From the Faculty of Dentistry, University of Toronto; and the Craniofacial Prosthetic Unit, Toronto-Sunnybrook Regional Cancer Centre, Toronto, Ontario, Canada DENTAL CLINICS OF NORTH AMERICA VOLUME 46 • NUMBER • JANUARY 2002 11 BIOSTATISTICAL CONSULTATION FOR DENTAL RESEARCH 149 lead to loss of interest or motivation in the particular type of research being performed On the other hand, it is likely that a false effect will be exposed sooner or later in the course of further research The researcher also needs to provide estimates of the magnitude of the effect of the intervention on each of the outcome variables in one of the research groups, together with an estimate of the variability; these estimates are called pilot data Making these estimates may seem counterintuitive, because it may reasonably be asked what purpose the research serves if some concept of the size and variance of the intervention is available a priori In fact, researchers are not presuming to estimate the effect of the experimental intervention but rather to make a reasonable speculation on the response that could be expected in the control or nonintervention group The pilot data form the basis for estimating the sample size required to observe a given difference Assume, for example, that an investigator is planning to test the effect of an intervention hypothesized to reduce the rate of loss of attachment Suppose further that it is known that over some time period untreated individuals will lose an average of mm attachment, with a standard deviation of 3.5, and that these estimates apply to the type of patient population being studied The biostatistical consultant can use this information to estimate the number of patients needed to detect a specified difference based on given values of ␣ and ␤ and the type of analysis to be used SOURCES OF PILOT DATA The division of labor is straightforward in sample size estimation The researcher needs to supply estimates of the appropriate summary measures for the important outcome variables for at least one of the groups in the study (probably the control or non-intervention group) The researcher also needs to provide an idea of the magnitude of a clinically significant effect The statistician uses these data, together with specifications of ␣ and ␤ and the statistical method to be used, to estimate a sample size Table outlines the main steps in performing a power analysis It is permissible to estimate sample sizes for a range of values of ␣ and ␤, as illustrated in Table 3, which shows a sample size table for the hypothetical experiment concerning loss of attachment discussed previously There are several sources of assistance for the researcher in determining what constitutes a clinically significant difference, as well as providing pilot data for use by the statistical consultant in estimating sample size or power The scientific literature is a valuable source of background data for this planning It is quite likely that the researcher has exhaustively reviewed the literature in the course of formally developing the research plan The literature review may provide multiple sources of pilot data as well as indications of the variation in response across different classes of patients or potential research subjects 150 CLIVE Table PRIMARY STEPS IN CARRYING OUT A POWER ANALYSIS Step Activity Responsibility Specify clinical hypotheses Determine primary outcome measures Transcribe clinical hypotheses to statistical hypotheses Specify range of clinically meaningful differences for outcome measures Specify ␣, ␤ Obtain pilot data for calculations Obtain power/sample size estimates Evaluate results for final sample size/power specification DR DR, B DR, B DR DR DR B DR, B DR ‫ ס‬dental researcher; B ‫ ס‬biostatistician; ␣ ‫ ס‬probability of a type I error; ␤ ‫ ס‬probability of a type II error Input from colleagues is also a useful source of data for research planning, especially when inquiries can be focused, so that matters of pilot data and significant effect can be addressed directly A researcher who is also a practicing dentist may have a set of patient records worth examining Many providers use the readily available software (such as spreadsheet packages) to construct their own databases, which may be useful for planning; however, it is important to verify the standards under which the data were assessed Some commercially available software packages for power analysis provide interactive prompts for assisting users through the steps of a sample size determination.6 These steps include a variety of techniques for generating pilot data summary measures from limited input (e.g., estimating the standard deviation for an outcome variable based on estimation of the range or percentile values) These techniques are useful when prior knowledge or data are limited In some cases, suitable pilot data are lacking altogether Such a situation may arise, for example, early in the history of a line of scientific inquiry or when first testing a new drug or intervention Here, the researcher may consider implementing a pilot study A pilot study is a Table SAMPLE SIZE ESTIMATES FOR HYPOTHETICAL LOSS OF ATTACHMENT STUDY* ␤ Value Percent Mean Difference 0.10 0.20 10 15 20 25 40 33 29 25 35 28 24 19 *The table is designed to show the number of subjects needed in each of two groups, assuming a repeated measures design with hypothesis testing carried out at ␣ ‫ 50.0 ס‬Sample size estimates are shown for each of two values of ␤, and each of several effect sizes ␣ ‫ ס‬probability of a type I error; ␤ ‫ ס‬probability of a type II error BIOSTATISTICAL CONSULTATION FOR DENTAL RESEARCH 151 small-scale, preliminary study designed to assess the feasibility of the proposed research and entails evaluating all aspects of the research (including administrative matters if a clinical trial is being contemplated) One of the primary objectives is to acquire a database for use in formal calculations of sample size needed for a larger experiment or clinical trial to be carried out at a later date Although relatively small in scale, pilot studies often require as much interaction between researcher and statistician as more formal research, particularly in determining which variables will be assessed Stopping rules need to be established because, by definition, detailed power analyses are not possible in a pilot study DATA MANAGEMENT Data management is a deliberately broad term, incorporating a variety of tasks concerned with data acquisition, storage, confidentiality, editing, and retrieval These aspects of research execution are of vital importance in assuring the quality of the research Data management is especially vital in large-scale projects involving the determination of many variables from many research subjects and possibly at multiple time points Data acquisition begins with the design and planning of datagathering forms Completed forms need to be machine entered, although machine-readable forms can facilitate this activity Data should be checked thoroughly One procedure for checking is dual entry, in which data forms are entered twice, by independent data technicians The final data files for the two operators are compared data point by data point With dual entry, the only way an incorrect value can enter the final file is for each operator to make the identical mistake in the identical location Although dual entry is an effective mechanism for data-entry quality control, it is not always feasible In large studies or clinical trials, the researcher should plan on printing a randomly selected subset of the entire data file for verification against the data-gathering forms It may also be prudent to check all values of any variables that are particularly significant This checking will provide an estimate of the overall dataentry error rate and may suggest variables or areas in the data file that need further attention in the editing process Once a data file has been checked, edited, and found satisfactory, further examination should involve the search for potential outlying values This search can be done for all variables in the file and is a basic procedure in the exploratory data analysis phase of the research All values above the 95th percentile and below the 5th percentile (or some other specified cutoff points) are listed, together with identifying information indicating which record in the data file contains the value The researcher can then consider this output and flag blatantly out-of-range 152 CLIVE values for further checking This procedure should be followed for all major study variables Data confidentiality is important in clinical trials involving human subjects Assuring confidentiality usually entails an intermediate step in the data-entry process in which identifying information is replaced with some numeric patient identifier This is the responsibility of the researcher, who establishes and maintains the key relating the two data fields Access to the key is limited by and is under the direction of the researcher The file supplied to the statistical consultant should contain no unique patient data that could be used for identification or to breach subject anonymity RESEARCH ADMINISTRATION The biostatistical consultant can help the researcher with other aspects of the actual administration and execution of the research project Several such topics noted here arise in clinical trials and include the randomization of patients, protocol deviations, and the analysis of dropouts and missing data The randomization of patients refers to the assignment of patients to study groups, usually by some random mechanism Randomization is generally a straightforward task, and the method of randomization depends largely on the study design Both the researcher and the statistical consultant need to keep careful track of any protocol deviations Protocol deviations involve changes in the study design or plan once subject intake has begun These planning changes are sometimes unavoidable Subject dropout can be a major problem in longitudinal clinical trials Patients can leave for a variety of reasons Some may decide not to continue participating, especially if the experiment involves some unpleasant or invasive procedures Others may leave the area Some may become injured or ill The researcher and the biostatistician will hope fervently that such dropout is random; that is, one is not primarily losing the treatment responders or non-responders or only the most compliant or non-compliant participants Random dropout implies that subjects who leave a clinical trial before completion of the protocol so at random, and that the remaining subject groups are still homogeneous with respect to potentially confounding factors The term intent to treat refers to the analysis of data from all subjects, including those who drop out The rationale for this method of analysis is discussed in most clinical trial guides; see, for example, Spilker’s Guide to Clinical Trials19 and Piantadosi’s Clinical Trials: A Methodological Perspective.15 Most researchers will also analyze those subgroups of participants completing the protocol A crucial phase of the analysis of data from a clinical trial is the analysis of dropouts and their comparison with subjects who remained BIOSTATISTICAL CONSULTATION FOR DENTAL RESEARCH 153 Missing data can be a problem in surveys or retrospective studies The difficulty, as with patient dropouts, arises when missing data occur in a nonrandom fashion Although the treatment of missing data (involving, for example, multiple imputation procedures) is the responsibility of the biostatistician, the researcher needs to participate in planning the procedures for carrying out the study so that that the occurrence of missing data is minimized or the variables prone to absence are of relatively minor importance in the study DISCUSSION The primary theme of this article is that it is not essential that the researcher have a strong working knowledge of elementary (or higher) statistics to perform valid scientific research Rather, the researcher should be prepared to work with a biostatistical consultant on an extensive and ongoing basis to plan and execute a research project carefully The need for this emphasis was recognized by Moses and Louis, who suggested that effective collaboration between clinician and statistician can help identify tractable scientific and statistical problems that need attention and can help avoid undertaking intractable ones.13 Furthermore, they assert that the ‘‘central requirement for successful collaboration is clear, broad, specific, two-way communication on both scientific issues and research roles.’’13 The researcher will need to assist the biostatistician in estimating sample size, in understanding the basics of the science involved, and in relating scientific and statistical hypotheses The biostatistician should come to appreciate the scientific and clinical issues and underlying principles; likewise, the researcher will come to appreciate how appropriately executed data analysis can extract valuable scientific knowledge from experimental data Over time, the researcher will acquire the statistical knowledge needed to interpret and present study results The statistical understanding may be focused and restricted to the methods relevant to the particular study, but it will constitute a useful body of knowledge, appropriate for future studies or as a basis for using other statistical methods in different studies Most, if not all, scientists are convinced of the utility of mathematical models in representing and studying natural phenomena Statistical models are mathematical models that incorporate probabilistic measures of uncertainty In the study of oral health, two primary sources of variation impart this uncertainty The first is the natural variation among patients in measures of oral health; the second is the variation resulting from sampling, or selecting a subgroup of patients for study, because the entire population of such patients is impossible to access The progression of data analyses intended to account for this variation, from simple independent group t-tests through complex multivari- 154 CLIVE ate methods, is one of increasing technologic, mathematical, and statistical sophistication and advancement It is also a progression that describes considerable theoretic and applied advances by researchers attempting to understand dental disease and how to deal with it Often, theoretic clarification and understanding derive from the application of more detailed models, as new information about the processes being modeled derive from formalization and logical representation The influence of this trend of increasing detail and complexity in data analysis for dental research will become more profound in the immediate future, as new developments in dental science occur simultaneously with new advances in statistical theory and computer science This progression will only increase the need for dental researchers to establish and develop lines of communication with data analysts References Armitage GC, Jeffcoat MK, Chadwick DE, et al: Longitudinal evaluation of elastase as a marker for the progression of periodontitis J Peridontol 65:120–128, 1994 Chugal N, Clive J, Spangberg L: A prognostic model for assessment of the outcome of endodontic treatment: Effect of biologic and diagnostic variables Oral Surg Oral Med Oral Pathol Oral Radiol Endod, in press Clive J, Woodbury MA: Continuous and discrete global models of disease Mathematical Modeling 7:1137–1154, 1986 Diggle PJ, Liang K-Y, Zeger SL: The Analysis of Longitudinal Data New York, Oxford University Press, 1994 Elashoff J: nQuery Advisor௡ Version 4.0 User’s Guide Los Angeles, CA, 2000 Feinstein A: Clinical Biostatistics Boca Raton, FL, CRC Press, 2002 Friedman G: Primer of Epidemiology, ed New York, McGraw Hill, 1994 Glantz A: Primer of Biostatistics, ed New York, McGraw Hill, 1997 Imrey PB, Chilton NW: Design and analytic concepts for periodontal clinical trials J Periodontol (suppl) 63:1124–1140, 1992 10 Kilburn KH, Alsmundsson T: Anteroposterior chest diameter in emphysema Arch Intern Med 123:379–382, 1969 11 Littell RC, Milliken GA, Stroup WW, et al: SAS௡ System for Mixed Models Cary, NC, SAS Institute, 1996 12 Loe H, Anerud A, Boysen H, et al: Natural history of periodontal disease in man ă Rapid, moderate and no loss of attachment in Sri Lankan laborers 14 to 46 years of age J Clin Periodontol 13:431–440, 1986 13 Moses LE, Louis TA: Statistical consulting in clinical research: The two-way street Stat Med 3:1–5, 1984 14 Neely A: The natural history of periodontal disease in man Risk factors for progression of attachment loss in subjects receiving no oral health care J Periodontol 72(8):1006–1015, 2001 15 Piantadosi S: Clinical Trails: A Methodological Perspective New York; John Wiley & Sons, 1997 16 Riegelman RK: Studying a Study and Testing a Test, ed Philadelphia, Lippincott Williams & Wilkins, 2000 17 Sackett DL, Haynes RB, Tugwell P, et al: Clinical Epidemiology ed Philadelphia, Lippincott Williams & Wilkins, 1991 18 Sheehan TJ: The medical literature: Let the reader beware Arch Intern Med 140:472– 474, 1980 19 Spilker B: Guide to Clinical Trials New York, Raven Press, 1991 BIOSTATISTICAL CONSULTATION FOR DENTAL RESEARCH 155 20 Ten Have TR, Landis JR, Weaver SL: Association models for periodontal disease progression: A comparison of methods for clustered binary data Stat Med 14:413– 429, 1995 21 Voelkl KE, Gerber S: Using SPSS for Windows New York, Springer-Verlag New York, 1999 Address reprint requests to Jonathan Clive, PhD Department of Biostatistical Consultation University of Connecticut Health Center MC3805 Farmington, CT 06030 e-mail: clive@nso1.uch.edu EVIDENCE BASED DENTISTRY 0011–8532/02 $15.00 ‫00 ם‬ APPLYING EVIDENCE BASED DENTISTRY TO YOUR PATIENTS James D Anderson, BSc, DDS, MScD GENERAL ISSUES A common criticism of evidence based practice is that it seeks to usurp the individual clinician’s judgment, imposing instead an external authority found in the literature that may or may not be appropriate This criticism is not valid Indeed, the fourth step of the Evidence based Practice Model (Fig 1) reserves a place for the individual practitioner’s judgment in the application of the literature to the clinical problem Evidence based practice therefore seeks to inform clinical decisions, not to impose them After converting the patient’s problem into an answerable question, searching the literature, and critically appraising the found articles, the clinician must to decide if the valid information that has been revealed can be applied to the patient whose problem triggered the process To so, the clinician must consider certain specific factors First, clinicians cannot allow themselves to be dazzled by elaborate statistics showing ˚ extreme measures of statistical significance In a trial comparing Branemark and IMZ implants under mandibular overdentures, Boerrigter et al1 found a statistically significant difference in bone level changes between the implant types year after implant placement The mean ˚ scores were 0.5 mm for the IMZ implants and 1.0 mm for the Branemark implants This difference was found to be statistically significant (P Ͻ From the Faculty of Dentistry, University of Toronto ; and the Craniofacial Prosthetic Unit, Toronto-Sunnybrook Regional Cancer Centre, Toronto, Ontario, Canada DENTAL CLINICS OF NORTH AMERICA VOLUME 46 • NUMBER • JANUARY 2002 157 158 ANDERSON Figure The steps in the model of evidence-based practice (From Anderson JD: Need for evidence-based practice in prosthodontics J Prosthet Dent 83:58–65, 2000; with permission.) 0.003), meaning there was only a 0.3% chance that such a difference could have occurred by chance This difference seems major until one realizes that it is only a 0.5-mm difference and therefore is unlikely to be clinically significant A highly significant statistical difference is therefore no indicator of a clinically significant difference Most articles that describe clinical research report their findings on a sample of patients Often, the sample of patients is intended to represent the whole population The selected patients therefore should have demographic and disease characteristics similar to those of the population at large The distribution of age, sex, socioeconomic status, education, nutritional status, and occupational range all should reflect society in general Similarly, the prevalence, severity, and duration of disease status should also mirror the general population Clearly, a sample of patients in any given study is unlikely to fulfill all these criteria Often, the authors not want to reflect the whole population and limit their sample to persons of a certain age group, or with a history of exposure to an agent such as smoking, or with a clinical condition such as edentulousness In applying the findings from such studies to the individual patient, a clinician must decide if the patient is similar enough to the study patients for the results to be applicable One way to so is to see if the clinician’s patient would have met the inclusion and exclusion criteria to be included the study Often, some differences are found between the study sample and the present patient These differences may not make the article useless A more useful approach may be to reverse the question and ask whether the study population is so different APPLYING EVIDENCE BASED DENTISTRY TO YOUR PATIENTS 159 from the patient that the results cannot possibly be applied This approach makes it possible to apply some information from the article If the study population is divided into subgroups, it may be possible to match the reader’s patient to one of the groups for more focused information The setting in which the study was gathered can have a major impact on the findings The results of a new, experimental periodontal treatment tested in a major teaching institution may not be applicable to the patients of a general practitioner because of an effect called referral filter bias The major teaching institution is likely to attract patients who have more severe periodontal problems than those seen in general practice Similarly, the treatment they receive at a major center may not be feasible in general practice The patients’ response to the new treatment, therefore, may not be applicable to the patients of a general practice An example is the series of patients treated with severe (apical third) periodontal bone loss who were rehabilitated with extensive fixed bridges and aggressive oral hygiene maintenance.6 Such a report offers little help to the general practitioner who sees less severe periodontal destruction, is less likely to undertake such extensive reconstructions, and may not be able to expect such a high degree of patient compliance in oral hygiene Therefore, if readers are seeking information to apply to their general practice, it will be necessary to pay special attention to how the patients were selected with respect to the severity of their disease and the feasibility of the treatment approach The important question for the practitioner to ask is, ‘‘Could such circumstances be duplicated in my office?’’ No clinical decisions are made without some element of patient input The patient’s preferences, priorities, and resources will therefore affect clinical decisions Stated another way, the social and cultural issues that are important to the patient must be considered when deciding how to apply the information found in a literature search related to the patient’s problem A new, highly effective treatment approach that takes too long, is likely to be painful, or is too expensive is not appropriate if it is not consistent with the patient’s wishes Similarly, treatment solutions exist for problems that are not important to some patients The use of effective veneering techniques makes sense only if the social and cultural pressures on a patient exceed the risks inherent in the technique For many people, a less than perfect smile is simply not important To suggest a solution where there is no problem invites disaster Marketing techniques aimed at creating demand are a concern here SPECIFIC APPLICATIONS The general issues discussed previously apply to any situation in which one is contemplating the application of valid information found in the literature to a specific patient situation There are, however, other 160 ANDERSON points to be considered when applying certain types of information to the patient’s situation These are considered in turn Diagnostic Tests Once valid information about a diagnostic test has been recovered from the literature, the practitioner must decide whether the test will be useful for a given patient To make this decision, the answers to a few questions will provide guidance7 Is the diagnostic test available, affordable, accurate, and precise in this setting? Can a clinically sensible estimate of the patient’s pretest probability of disease be generated? • Can personal experience or prevalence statistics be drawn on? • Are the study patients similar to this patient? Would the results of the test affect the management and help the patient? • Could the results influence the decision to treat the condition? • Would the patient be a willing partner in the treatment? First and most sensibly, practitioners must be assured that the test is available, affordable, accurate, and precise in their setting The answer to the first two parts of this question are probably obvious An electric pulp tester is easily available and usable at reasonable cost in most dental offices On the other hand, computed tomography and the associated software are less available, and their use is certainly more expensive The answer to the latter two parts of the question may be less apparent A diagnostic test that has performed well in the office of a general practitioner may not perform as well in a specialist’s office or university clinic The reason is that the prevalence of the condition being tested will probably be different in the two settings Therefore, the difference in the rate of false-positive (or false-negative) findings will change the likelihood ratios of the test Because the prevalence of the disease may be a major component of the pre-test probability of disease, the test may behave differently in different settings Thus, the interpretation of the electric pulp test may be different in a general practitioner’s office and in a university teaching endodontic clinic Similarly, a report of a new test that was validated in a tertiary care center must be applied with caution in a general practice setting The appropriate use of a diagnostic test begins with a pretest estimate of the likelihood of disease This estimate may be no more precise than the prevalence of the condition in the population A patient who presents with throbbing pain and facial swelling, however, raises the pretest estimate of the likelihood of apical periodontitis beyond the general prevalence in the population Even if the diagnosis is a guess, under these conditions it is a better estimate than simple prevalence On the other hand, it is more difficult to generate an estimate of the APPLYING EVIDENCE BASED DENTISTRY TO YOUR PATIENTS 161 pretest likelihood of a malignancy in a young patient with an unexplained ulcer in the palate Where a practitioner estimates pretest likelihood, the most recent or most dramatic previous events encountered influence his or her judgment.5 The third question Sackett and colleagues pose is whether the results of the test will change the practitioner’s treatment behavior.7 For example, a patient presents with a maxillary lateral incisor consumed by decay that is sensitive to heat and percussion, with throbbing pain and swelling above the apex It is highly unlikely that an electric pulp test of this tooth will change the treatment behavior of the dentist In this situation, the treating dentist has already crossed a decision threshold to treat the tooth based on other clinical findings, and the electric test will add no new information On the other hand, the dentist would not a biopsy of a lesion in the palate of a teenager who reported burning his mouth on a hot pizza the night before Similarly, the dentist has crossed a decision threshold in the other direction, deciding not to test the lesion because of the invasiveness of the test and the low probability of a finding that warrants treatment It is in the area between these extremes, where the results of a test will influence the treatment behavior, that the time, cost, and discomfort of a test are warranted Finally, if the test is painful or costly, the patient may choose not to know the results of the test rather than submit to the test For example, a patient may be reluctant to submit to a CT scan with three-dimensional reconstruction to measure bone volume before the placement of two implants in his edentulous mandible when a conventional panoramic film and clinical examination confirm more than enough bone thickness and height Clearly, the patient must be a willing participant in the diagnostic procedure with an expectation of obtaining valuable new information that will influence the outcome to justify the additional cost or discomfort Prognosis Whether the information in an article about the prognosis of a condition should be applied to a specific patient can be decided by answering questions specific to this type of article: Will the results lead directly to selecting or avoiding treatment for an individual patient? Are the results useful for reassuring or counseling patients? Knowledge of the natural history of a condition clearly will influence the decision to select or avoid treatment For example, with the clarification of the prognosis of juvenile periodontitis,8 treatment can be more focused and aggressive On the other hand, an article by de Leeuw et al3 suggests that in patients with osteoarthrosis and reducing temporomandibular joint disk displacement, the prevalence of pain dropped from 43% to only 17% in to years and dropped further to 162 ANDERSON only 2.4% after as much as 30 years It would be difficult to suggest invasive surgical treatment in the face of this information So, in addition to the general issues noted at the start of this article, readers of an article that describes a prognosis must ask whether the results will lead directly to selecting or avoiding treatment for an individual patient Unless the results of an article about prognosis can be used in this way, it is unlikely that the results will have any application to the individual patient In situations such as temporomandibular disorders, providing information for the patient may be enough treatment Simply giving the patient some understanding of the natural history of the condition can much to relieve anxieties by providing realistic expectations A second question for the reader of articles about prognosis, then, is, ‘‘Are the results useful for reassuring or counseling patients?’’ Therapy In dentistry, numerous articles advocate improved techniques or materials over existing therapies It is not always appropriate to apply the results of every therapeutic improvement to every patient, even if the evidence was found to be compelling when critically appraised Certain questions specific to articles about therapy will help determine when to apply improvements to patients and when not to: Are the results reported as outcomes that are important to patients? Were all clinically important outcomes reported? Are the likely treatment benefits worth the potential harms and costs? Evidence for the improvement usually takes the form of increased longevity (such as implants or fixed partial dentures), reduced numbers of failures (such as tooth loss), or improvement in subjective parameters (such as comfort or chewing ability) All of these outcomes are important to patients Sometimes, when these outcomes are rare or take a long time to realize, substitute outcomes, such as attachment loss, bleeding on probing, and mobility, are used to predict those events that are important to patients The use of these surrogates is reasonable and expedient only to the extent that they do, in fact, predict the events that are important to patients A meta-analysis presented recently2 suggested that guided tissue-regeneration procedures would result in a mean increase in attachment level of 4.0 mm This result is impressive, but the application of this information to an individual patient requires that an increase in attachment level predicts greater tooth longevity—an outcome more likely to be of interest to the patient than the level of attachment If this link has been previously established, this information is meaningful; if this link has not been established, the usefulness of this information is limited, even though it is based on a meta-analysis (a strong design) A further problem was that the underlying studies used APPLYING EVIDENCE BASED DENTISTRY TO YOUR PATIENTS 163 in the meta-analysis were limited to year of follow-up The reader of articles that report the results of trials of therapy must therefore be sure that the outcomes reported are important to patients and not merely surrogates lacking in predictive value To ensure predictive value, it is important that all clinically important outcomes have been reported in the article In a randomized trial of the efficacy of flurbiprofen taken for months after implant surgery in reducing alveolar bone loss around implants, Jeffcoat and others4 noted that two patients had to be withdrawn from the study, one because of stomach upset and another because of a decrease in red blood cell counts thought to be related to the medication The trial found a statistically significant reduction in bone loss between the third and sixth months after surgery, but at no other time up to year The difference in bone mass lost was between 8.6 and 12 mg The reader therefore must consider whether the additional risks involved in that dosage of the drug are worth the benefit of saving those few milligrams of bone The significance of saving those few milligrams of bone must also be considered The clinician must balance the potential benefits of the treatment against the potential harms or costs of the treatment The information presented in the article informs but does not dictate the clinician’s decision to apply the findings to the particular patient SUMMARY It should be evident by now that evidence based dentistry leaves much room for the application of clinical judgment to the literature This article points out that judgment in evaluating certain factors is essential and that the practice of evidence based dentistry is not a process of blindly following the conclusions found in the literature Clinicians can safeguard the patient and themselves against the inappropriate use of weak or irrelevant evidence in the conduct of daily practice This skill adds confidence to decision making in clinical practice and prevents the decline in skills throughout a career References Boerrigter EM, van Oort RP, Raghoebar GM, et al: A controlled clinical trial of implantretained mandibular overdentures: Clinical aspects J Oral Rehabil 24:182–190, 1997 Bragger U: Evidence Based Outcomes of Periodontal Therapy: Clinical Decision Making in Prosthodontics and the Impact of Implants Bern, Switzerland, 2001 de Leeuw R, Boering G, Stegenga B, et al: Symptoms of temporomandibular joint osteoarthrosis and internal derangement 30 years after non-surgical treatment Cranio 13:81–88, 1995 Jeffcoat MK, Reddy MS, Wang IC, et al: The effect of systemic flurbiprofen on bone supporting dental implants J Am Dent Assoc 126:305–311, 1995 Kassirer JP, Kopelman RI: Cognitive errors in diagnosis: Instantiation, classification, and consequences Am J Med 86:433–441, 1989 164 ANDERSON Nyman S, Lindhe J: A longitudinal study of combined periodontal and prosthetic treatment of patients with advanced periodontal disease J Periodontol 50:163–169, 1979 Sackett DL, Straus SE, Richardson WS, et al: Evidence-Based Medicine How to Practice and Teach EBM, ed Edinburgh, Churchill Livingstone, 2000 Zambon JJ, Christersson LA, Genco RJ: Diagnosis and treatment of localized juvenile periodontitis J Am Dent Assoc 113:295–299, 1986 Address reprint requests to James D Anderson, BSc, DDS, MScD Department of Dentistry University of Toronto 124 Edward Street Toronto, Ontario M5G 1G6 Canada e-mail: jim.anderson@utoronto.ca ... about costs, and cultural issues An implant-supported fixed reconstruction cannot be done for an edentulous patient without significant surgical procedures and considerable discomfort and cost over... Understanding the terminology is more important than simply recognizing the terminology Randomization, however, cannot make up for a poorly planned and implemented study Just because the design is an. .. of root canal therapy Outcome of root canal therapy and RCT Outcome of root canal therapy and prospective trial Outcome of root canal therapy and retrospective trial Outcomes of root canal therapy