Ebook Applied epidemiologic principles and concepts - Clinicians’ guide to study design and conduct : Part 2

184 145 0
Ebook Applied epidemiologic principles and concepts - Clinicians’ guide to study design and conduct : Part 2

Đang tải... (xem toàn văn)

Tài liệu hạn chế xem trước, để xem đầy đủ mời bạn chọn Tải xuống

Thông tin tài liệu

Part 2 book “Applied epidemiologic principles and concepts - Clinicians’ guide to study design and conduct” has contents: Ecologic studies - Design, conduct, and interpretation, case-control studies, cross-sectional studies, cohort studies, causal inference in clinical research and quantitative evidence synthesis,… and other contents.

7 Ecologic studies Design, conduct, and interpretation 7.1 Introduction Ecologic studies are sometimes not understood in epidemiologic or medical settings due to the limited or low frequency of its utilization in evidence discovery Ecologic designs reflect investigations in which the unit of observation is the group and the analysis is performed on the group and not individual level This design is feasible in assessing an association when the exposure and outcomes are available on the group level In this context, ecologic studies may serve the purpose for generating hypothesis for individual-level studies Often, data are available on aggregate measure of the exposure and outcome of interest Additionally, causal association with ecologic design is difficult to establish Assuming that prostate cancer (CaP) mortality is higher in zip code 19810 with lower level of pesticide exposure relative to 19960 with higher pesticide exposure, does this design imply the protective effect of CaP mortality by pesticide exposure? Such an inference cannot be drawn given the reality of population dynamics in terms of migration since CaP mortality in 19810 may be due to CaP patients or individuals moving to 19810 from 19960 and other zip codes with higher levels of pesticides Also, such a design restricts the individual-level data on confounding in addressing the confounding effect of comorbidity, income, education, etc., factors observed to influence CaP mortality This chapter describes ecologic design as group level or aggregate data study design, presenting hypothetical and real-world examples (Figure 7.1) The different types of ecologic designs are presented as well as ecologic fallacy The advantages and limitations of ecologic design are addressed, notably confounding as a mixing effect of the third variable in the association between the exposure and the outcome of interest The advantages include public access data, low-cost data acquisition, and the feasibility of evaluating community-level intervention 7.2 Ecologic studies: Description Ecologic designs, also called group-level ecologic studies, correlational studies, or aggregate studies, obtain data at the level of a group or community often by making use of routinely collected data.1,2 Second, if the population, 134  Applied epidemiologic principles and concepts Exposure Population A with aggregate level Outcome Population B with aggregate level Exposure Outcome Figure 7.1  Ecologic study design illustrating aggregate level exposure by two groups or populations and the assessment of outcome rather than the individual, is the unit of study and its analysis, such a study is correctly characterized as ecologic This design involves the comparison of aggregate data on risk factors and disease prevalence from different population groups in order to identify associations Because all data are aggregate at the group level, relationships at the individual level cannot be empirically determined but are rather inferred Thus, because of the likelihood of an ecologic fallacy, this type of study provides weak empirical evidence.1 Basically, the focus of ecologic studies is the comparison of groups, and not individuals, implying the missing of individual data on the joint distribution of variables within the group This focus places the need for ecologic inference about effects on group levels.1 The questions is: Why conduct an ecologic study given critique of this method among some epidemiologists? The rationale involves feasibility and data reliability and accuracy in generating testable hypothesis Ecologic studies are conducted given no initial data on a health problem, no individual-level data, data are available at grouplevel, and limited research resources and time (rapid conduct) The advantages of ecologic design include its ability for hypothesis generation for individual-level data design and analysis and it is inexpensive and requires a short period for its conduct The required data for ecologic studies may be obtained from published literature or public access database, rendering the conduct less time-consuming and inexpensive While ecologic fallacies had been observed to be the main disadvantage of ecologic design, individual-level studies are not completely immune from ecologic fallacies, implying careful ascertainment of the exposure and disease variables in the conduct of ecologic studies Ecologic studies 135 Ecologic designs are used to for geographical comparisons of diseases such as the correlation between childhood brain cancer in certain regions in the United States, implicating clusters in some regions relative to the state If in the same region, there is a low consumption of extra virgin olive oil, a hypothesis can be generated on the association between brain cancer and extra virgin olive oil However, care must be exercised in the interpretation of such ecologic studies given the potentials for confounding such as age, sex, socioeconomic status, education, access to primary care, etc Ecologic studies are also useful in assessing time and secular trends in disease While rates of acute conditions such as bronchiolitis fluctuates over time, chronic disease rates tend to remain stable over time Ecologic studies may be used to generate hypothesis if disease rates illustrate a correlation with environmental changes For example, increasing rates of upper respiratory disorders in children during summer may be due to carbon monoxide car emission during the summer months Migrant studies remain a typical example of ecologic designs for hypothesis generation These studies provide the opportunity to examine genetic, environmental, and gene–environment association or determinant of disease For example if a migrant population in the United States (i.e., Asians) are observed to have a higher incidence of CaP in the United States compared to Asians in Asia, the higher rate of CaP among Asians in the United States may be due to environmental condition in the United States Also, the observed higher incidence of CaP among Asians in the United States may be due to selective emigration from Asia, implying those who were more susceptible to prostatic adenocarcinoma or hormonally related malignancies in general Race/ethnicity, mortality, chronic obstructive pulmonary disease, and occupation illustrate correlation African Americans have higher age-adjusted mortality compared to whites and are more likely to be employed in a lowpaying job which correlates with higher mortality On the basis of these correlations, hypothesis on the association between COPD incidence and low paying jobs could be generated BOX 7.1  ECOLOGIC DESIGN Ecologic design or aggregate study refers to an observational study in which all variables are group measures, implying the group as the unit of analysis, in contrast with either a case-control or prospective cohort design, where the unit of analysis is the individual level measures May be classified on two dimensions, namely, exploratory versus analytic, and whether subjects are grouped by place (multiple-group model), time (time-trend model) or place and time (mixed model) H Morgenstern, “Ecologic Studies in Epidemiology: Concepts, Principles, and Methods,” Annu Rev Public Health 16 (1995): 61–81 136  Applied epidemiologic principles and concepts 7.2.1 Conducting an ecologic study To conduct an ecologic study, we need to have aggregate information on groups or subpopulations on the dependent or outcome variable of interest and the independent, explanatory, or predictor variable For example, to determine the effect of alcohol consumption on stroke, we can obtain information on the prevalence of stroke in populations A, B, and C and then determine alcohol consumption in these three populations Finally, we correlate alcohol consumption and stroke prevalence in these three populations If alcohol is a risk for the development of stroke, the population with the highest alcohol consumption will be associated with the highest prevalence of stroke Alternately, if alcohol consumption per capita is protective, then the population with the highest alcohol consumption will be associated with the lowest prevalence of stroke BOX 7.2  TYPES OF ECOLOGIC DESIGNS • Exploratory—refers to the design where there is no specific exposure of interest or the exposure of potential interest is not measured during the investigation phase of the study • Etiologic—refers to the design where there is a measurement of the primary variable or exposure of interest, which becomes part of the analysis • Multiple-group design—involves the comparison of rate of disease among regions during the same period, with the purpose of identifying spatial patterns that may be suggestive of environmental etiology • This may be etiologic or exploratory in nature • Time trends—also called time series, refers to the ecologic design that compares disease or specific events occurrence over time in a geographically defined population • Like multiple-group design, this may be etiologic or exploratory in nature • Mixed designs—refers to the combination of the basic features of multiple-group designs and time series This design could be exploratory or etiologic K J Rothman, Modern Epidemiology, 3rd ed (Philadelphia: Lippincott, Williams & Wilkins, 2008) 7.2.2 Importance of ecologic data One situation where ecologic data are particularly useful is where a powerful relationship that has been established at the individual level is assessed at the ecologic level in order to confirm its public health impact If a risk factor is a Ecologic studies 137 major cause of a condition (in terms of population attributable fraction as well as the strength of association), then a lower presence of that factor in a population should presumably be linked to a lower rate of the associated outcome 7.2.3 Examples of ecologic studies in epidemiologic evidence discovery A study on CaP mortality and dietary practices and sunlight levels as environmental risk factors for geographic variability in CaP rate was conducted using ecologic design CaP mortality rate was compared in 71 countries The per capita food consumption rate and sunlight levels were correlated with age-adjusted mortality rate in CaP The study indicated an increase CaP mortality rate with total animal fat calories, meat, animal fat, milk, sugar, alcoholic beverages, and stimulants consumption, but inverse correlation with increased sunlight level, soybeans, oilseeds, onions, cereal grains, and rice.2 Caution must be exercised in the application of these aggregate data in clinical decision making regarding the recommendation of these food products and sunlight exposure Like all aggregate or group-level data, the geographic variation in lifestyle and potential CaP confounders remain to be assessed Additionally, countries may differ in the presence of effect measure modifiers, which may explain part of the observed correlation in these data 7.3 Statistical analysis in ecologic design The estimate of effect of exposure/s on disease or health-related events involves not just a correlation coefficient but a predictive parameter.3–5 To estimate the effect or association, we have to regress the group-specific disease or outcome rates (Y) on the group-specific exposure prevalence (X) Therefore, the linear regression model remains useful in estimating the effect or association in ecologic studies The prediction equation in this context is represented by Y = β0 + β1X1, where β0 is the intercept on the Y axis (coefficient without the effect of the exposure) and β1 is the slope The predicted outcome rate (Yx  = 1) in a group with the exposure or entirely or mostly exposed is β0 + β1(1) = β0 + β1, and for the group that is less exposed or unexposed, the predicted rate for (Yx = 0) is β0 + β1(0) = β0 Consequently, the estimated rate difference is β0 + β1 − β0 = β1, while the rate ratio is (β0 + β1)/β0 = + β1/β0 BOX 7.3  ECOLOGIC FALLACY AND INFERENCE • Ecologic studies are based on the average or aggregate effect of exposure on a group, and not individual risk • The inability to determine if the individuals who are part of the group are at risk due to the group variable exposure of interest renders such designs noncausal or rather very limited in causality 138  Applied epidemiologic principles and concepts • The average effect of fat consumption in geographic locale, while it may show an association with average or cumulative incidence of breast cancer in regions with high fat consumption, may not necessarily link individuals with high fat with breast cancer • Because of the possibility of other risk factors at individual level influencing the outcome, ecologic studies are limited in causal inference as well as in temporal sequence • Ecologic fallacy implies invalid, unreliable, and unreasonable inference on individual risk factor in a disease causation given the group level assessment of the outcome as a function of exposure • Despite this fallacy, this design provides the initial step in examining the relationship between the exposure of interest at group level and the prevalence of the outcome, disease of interest, or health-related event of interest • Hypothesis generating on the basis of ecologic design requires appropriate interpretation and inference 7.4 Ecologic evidence: Association or causation? Data from ecologic designs indicate association and not causation However, etiologic studies are often based on initial ecologic investigations Biological pathway: The association involves a possible biological mechanism For example, extra virgin olive oil consumption (group-level) and female breast cancer mortality rate among women in the five selected countries (hypothetical) The biologic pathway reflects the inhibition of abnormal cellular proliferation given extra virgin olive oil consumption These data showed an inverse correlation between extra virgin olive oil consumption and breast cancer mortality, implying that countries with high extra virgin olive oil consumption have lower breast cancer mortality rates (Figure 7.2) Group level: The association is based on group-level exposure in the countries examined For example, women who consume extra virgin olive oil may be exposed to vegetable and fruit consumption, as well as a physically active lifestyle, than women who not; and breast cancer had been associated with vegetable and fruit consumption as well as physical activities Contextual: The association involves biological mechanism as well as ecologic or group level exposures (Figure 7.3) 7.5 Limitations of ecologic study design Despite several practical advantages of ecologic studies, there are methodological issues that limit causal inference, including ecologic fallacies and crosslevel bias, unmeasured confounding, within-group misclassification, lack of 20 30 40 50 60 Ecologic studies 139 40 60 Extra virgin olive oil consumption Fitted values 80 Breast cancer mortality (age-adjusted) rate Figure 7.2  Hypothetical ecologic study of the association between extra virgin olive oil and breast cancer mortality (age-adjusted) rate in five countries BOX 7.4  ADVANTAGES AND DISADVANTAGES OF ECOLOGIC STUDY DESIGN • • • • • Low cost and convenience Measurement limitations of individual-level studies Design limitations of individual-level studies Interest in ecologic or aggregate effect Simplicity of analysis and presentation of results K J Rothman, Modern Epidemiology, 3rd ed (Philadelphia: Lippincott, Williams & Wilkins, 2008) adequate data, temporal ambiguity, colinearity, and migration across groups.6 Ecologic fallacies typically represent the absence of association observed at one level of grouping to correlate to effect measure at the group level of interest Specifically, ecologic bias or fallacies refer to the absence of an association at the individual-level data despite the observed association at the group level Additionally, study limitation using this design reflects geographic variability in the ascertainment of exposure and disease, implying the need for the uniformity of the exposure and disease exposure across geography 140  Applied epidemiologic principles and concepts Ecologic study design Study populations (A, B, C) Groups or subpopulations? Outcome (high stroke prevalence) Outcome (moderate stroke prevalence) Population A YES Exposure (level 1) high alcohol use Population B Exposure (level 2) moderate alcohol use Outcome (low stroke prevalence) NO Exposure levels describe the per capita alcohol consumption in these three subpopulations The outcome describes cerebrovascular accident (stroke) in these subpopulations The ecologic design indicates a direct correlation between subpopulations alcohol consumptions and stroke prevalence Population C Exposure (level 3) low alcohol use Figure 7.3  Design of ecologic study 7.6 Summary Ecologic design assesses the relationship between the exposure and outcome of interest at a group level and does not involve individual-level analysis as applicable to cohort, case-control, and cross-sectional classic designs These designs are feasible when group-level data are available and inference is required on such a level and not at individual-level risk It is analytic since the measure of effect, being the correlation coefficient, is an essential inference when provided with the coefficient of determination Since individual data Ecologic studies 141 are not assessed in ecologic design, an inference about individual risk from such a design remains invalid, leading to an ecologic fallacy The analysis of ecologic design involves different measures of effect and association and includes the correlation coefficient but mainly predictive parameters, namely, the linear regression model While the linear regression model is appropriate in the analysis of ecologic research data, temporality (cause-and-effect association) and confounding remain major issues in the interpretation and application of ecologic findings in public health policy formulation as well as in intervention mapping Unlike other nonexperimental designs, it is extremely difficult, if not impossible, to assess and control for confounding Additionally, unmeasured confounding and the inability to assess effect measure modifier or biologic interaction may render ecologic data misleading to the scientific audience Therefore, while there is a need to conduct ecologic studies, caution is required in the interpretation and application of such findings in clinical and public health decision-making Questions for discussion A study is planned to investigate the benefits of agent X in drinking water (agent X is measured at group level) and the risk of developing dental caries in children a Which design should be used if individual-level data are not available? b What are the advantages and disadvantages of ecologic design? Comment on ecologic fallacy c What is the measure of effect or association in ecologic design, and how is it interpreted? Suppose you are required to examine the effect of maternal education on learning abilities in Sweden, Norway, Finland, Austria, Australia, the United Kingdom, and the United States, and there are no data on individual cases How will you begin to conceptualize the study? What design will be most feasible to draw an inference on the association between maternal education and learning disabilities in children? Consider a study to determine whether or not there is an association between extra virgin olive oil consumption and breast cancer If data are obtained from several countries on extra virgin olive oil consumption and the incidence of breast cancer, what design could be feasible is assessing this relationship? Second, on the basis of these aggregate data, what sort of causal inference could be drawn if any? Comment on the distinction between biologic and ecologic causality Suppose you are required to assess poverty among children and education attainment in adulthood and data are only available in different countries on poverty level and mathematical skills in children What design should be feasible in this case? What will be the measure of effect? Comment on the association between education and poverty, and discuss the implication of this with subsequent health status of children 142  Applied epidemiologic principles and concepts References K J Rothman, Modern Epidemiology, 3rd ed (Philadelphia: Lippincott, Williams & Wilkins, 2008) C H Hennekens and J E Buring, Epidemiology in Medicine (Boston: Little Brown & Company, 1987) D A Savitz, Interpreting Epidemiologic Evidence (New York: Oxford University Press, 2003) S Greenland, “Epidemiologic Measures and Policy Formulation: Lessons from Potential Outcomes,” Emerg Themes Epidemiol (2005):1–4 J L Colli, A Colli, “International Comparison of Prostate Cancer Mortality Rates with Dietary Practices And Sunlight Levels,” Urol Oncol Semin Orig Investig 24 (2006): 184–194 H Morgenstern, “Ecologic Studies in Epidemiology: Concepts, Principles, and Methods,” Annu Rev Public Health 16 (1995): 61–81 302  Applied epidemiologic principles and concepts Effect measure modification as previously described reflects a situation in which the result of a study, for example, odds ratio (OR), is different across stratum Consider a study conducted to assess the effect of radiation of the development of second primary thyroid cancer (SPTC) in children If the OR on the association between radiation therapy received and SPTC is 2.46, 95% CI, 2.00–4.35, and the stratified analysis by race/ethnicity shows, Hispanic, OR = 3.95, non-Hispanic white, OR = 1.85, and non-Hispanic Asians, OR = 2.87, then race/ethnicity remains an OR effect modifier The accountable and responsible approach to nonmisleading results is to present these findings with race-specific stratum, adjusting for potential confounding as applicable Whereas the effect size assessment is fundamental to study appraisal in medicine, clinical research, and population-based research, the generalization of findings beyond the study population or the sample used to generate the data requires the quantification of random error with a probability value termed p value The alternate to the p value is the precision as measured by the CI, which is a more informative measure of precision As previously sustained, CI provides information on the margins of error as well as the location of the point estimate within these limits However, p value merely reflects the arbitrary cut off point of 5% type I error in most medical and clinical research ambience It is worthy to note that most studies in clinical medicine not apply probability sampling technique in the recruitment of study subjects In such situation, it is difficult to apply a probability value of variables that were not subjected to random selection, implying equal and known probability of the subjects that generate the variables to be included in the study The exception in the application of p value to a nonprobability sample involves the assessment of consecutive sample and disease registry These situations reflect representative sample, hence the application of p value 15.6 Translational epidemiology (TransEpi): Consequential or traditional Epidemiologic designs and methods had been used to assess laboratory data (T0/ T1) for evidence transfer to phases I and II clinical trials and likewise the application of such methods to phases III and IV clinical trials (T2/T4) (Figure 15.1) Clinical research largely applies epidemiologic designs to evidence discovery in enhancing patient care and improving clinical practice guidelines Epidemiology is challenged by public health and clinical medicine to provide assessment that is adequate and clearly transferable to intervention mapping and implementation TransEpi is the attempt to apply epidemiologic methods and principles in enhancing studies conduct from T0 to T5, and the steps in transferring the knowledge gained in a systematic and synthesized manner in improving patient and public health TransEpi by this characterization is the intersection of all epidemiologic initiatives in improving human health including, though not limited to, applied epidemiology, field epidemiology, occupational/environmental, nutritional, chronic disease, infectious disease, aging, neurology, social/behavioral Consequentialist epidemiology and translational research implication 303 Human population research only Application of advanced epidemiologic knowledge and methods to biomedical, clinical, and populationbased research, and the transfer of such evidence in improving animal and human health through team science Genome study Animal and human No Yes Translational epidemiology T0–T5 conduct and knowledge transfer Consequential epidemiology Biomedical research Yes Clinical research Research involving discovery and therapeutics Yes Genomic and epigenomic epidemiology Epigenetic alteration that results in gene expression— regulatory protein modification and disease manifestation Populationbased research Traditional epidemiology Figure 15.1  Translational epidemiology—intersection of epidemiologic methods Epidemiologic investigation is complex given environmental and genetic factors in disease causation As environmental influences introduce epigenetic (epigenotype) and genetic (genotype) changes in the causal pathway of diseases, translational epidemiology remains a relevant strategy in the assessment of the role of epidemiology in genomics and epigenetic research in disease etio-pathogenesis epidemiology, clinical epidemiology field epidemiology, and the recent field, molecular and genetic epidemiology, as well as health outcomes and health disparities epidemiology By characterization, TransEpi is the application of advanced and reliable epidemiologic methods and principles to advance translational research (T0–T5) and the transfer of this knowledge in the improvement 304  Applied epidemiologic principles and concepts of animal and human health through collective biomedical sciences, clinical medicine, and public health effort (team science) The importance of TransEpi cannot be overemphasized given the complexities in the assessment of risk and protective factors in the pathway of therapeutics Specifically, in the current era of gene–gene interaction, gene–environment interaction and sociogenomics in disease causations, translational epidemiology is required in assessing these risk factors with respect to incidence rates disease variation, temporal trends, risk ratios, prevalence of genetic risk factors, risk–risk interaction, sensitivity, specificity, and genetic variants’ predictive values TransEpi has a significant contribution to make in phase III randomized controlled trials where knowledge from such fields are transferred for evidence-based recommendation for professional guidelines development For example, the recommendation of women whose family history is associated with an increased risk for deleterious mutations in BRCA1 or BRCA1 genes for genetic counseling and BRCA testing was based on the breast cancer susceptibility gene mutation testing for and breast cancer testing.6 TransEpi signals a departure from the traditional notion of epidemiology that restricts its application to human animals at a population level by facilitating the conduct of studies across all spectra of human health, implying bench to bedside and the populations The growth of this field is needed more now than ever before in training of basic scientists, clinicians, and public health researchers With such training and application, clinical medicine and public health are better strategized in transferring reliable and accurate data for advancement of knowledge in discovery and therapeutics, as well as enhancing further research in human health, changing clinical practice guidelines, and rational health and healthcare policy development (science-based policies) With enhanced responsibility of TransEpi, there remains a shared focus and pathways with consequentialist epidemiology 15.7 Summary Epidemiology of consequence primarily encourages epidemiologic studies to be conducted not for studies’ sake but to improve population health This approach recommends causal inference and global epidemiologic investigation in addressing health problems in the populations at greatest risk Philosophically, two research traditions have been applied in epidemiologic investigation, namely, deontologic and consequentialist While the later applied the concept of the end justifying the means, deontologic tradition stresses rigorous methods implying appropriate means in arriving at the end The goal of epidemiology is to assess disease distribution and determinants at the population level and to apply this knowledge in intervention mapping to control disease and promote health Epidemiology of consequence thus reaffirms the overall goal of epidemiology with more emphasis on intervention research and population health improvement While we support consequentialist epidemiologic approach, we caution the superficial use of this pathway in deemphasizing the need for rigorous Consequentialist epidemiology and translational research implication 305 statistical modeling in the conduct of epidemiologic studies With this balance, epidemiology could exert its responsibility as a profession in public health indicative to improve population health Questions for discussion Compare and contrast between deontological and consequentialist approach to evidence discovery in clinical research Suppose you are required reduce the incidence of sudden infant death syndrome in the populations at risk, outline the steps necessary to accomplish this task using epidemiology of consequence approach The director of population health at a pediatric health system requests your assistance in improving asthma readmission rate Using the deontological approach, suggest a feasible approach in achieving this Two drugs are developed to control BP Drug A was administered to 100 participants and illustrated a mean systolic BP reduction of units ( p < 0.0001), while drug B was administered to 25 patients and showed a mean systolic BP reduction of 10 units ( p = 0.25) Which of these two drugs would you recommend for BP control? Should you prefer drug A or B, please provide the rationale Additionally, if you prefer none of these agents, provide the rationale If you are required to design a course in epidemiology for medical students, please provide an outline of such course to reflect the goal of epidemiologic of consequence References Epidemiologic Society of London, Transactions of the Epidemiological Society of London: Sessions 1866 to 1876: Objects of the Epidemiological Society (London, UK, 1876) S Scheffler, Consequentialsm and Its Critiques (New York: Oxford University Press, 1988) A F Karatas, L Miller, L Holmes Jr et al., “Cerebral Palsy Patients Discovered Dead During Sleep: Experience from a Comprehensive Tertiary Pediatric Center,” J Pediatr Rehabil Med (4) (2013): 225–231 B Borkhuu, D Nagaraju, F F Miller, M H Moamed Ali, D Pressel, J AdelizziDelany, M Miccolis, K Dabney, and L Holmes Jr., “Prevalence and Risk Factors in Postoperative Pancreatitis after Spine Fusion in Patients with Cerebral Palsy,” J Pediatr Orthop 29 (3) (2009): 256–262 doi: 10.1097/BPO.0b013e31819bcf0a K M Keyes and S Galea, Epidemiology Matters: A New Introduction to Methodological Foundations (New York: Oxford University Press, 2014) K Keyes and S Galea, “What Matters Most: Quantifying an Epidemiology of Consequence,” Ann Epidemiol 25 (2015): 305–311 doi: 10.1016/j.­annepidem​ 2015.01.016 Epub 2015 Feb Review PMID: 25749559 US Preventive Services Task Genetic risk assessment and BRCA mutation testing for breast and ovarian cancer susceptibility Washington, DC: US Preven­ tive Service Task Force, 2005 (http://www.ahrq.gov/clinic/upstf/uspsbrgen.htm) (Accessed 6/29/2017) (pubmed—www.ncbi.nlm.nih.gov/pubmed/16144894) http://taylorandfrancis.com Index A Absolute risk (AR), 84 ACE inhibitor, see Angiotensin converting enzyme (ACE) inhibitor ACL injury, see Anterior cruciate ligament (ACL) injury Acute lymphocytic leukemia (ALL), 45 Acute myeloid leukemia (AML), 61 Adjusted rate, 112 ADT, see Androgen-deprivation therapy (ADT) Affordable Care Act 2012, 288 Aging epidemiology, 273 AIDS malignancy (AM), 103 ALL, see Acute lymphocytic leukemia (ALL) Alpha error rate, 78 Ambidirectional cohort design, 198 AML, see Acute myeloid leukemia (AML) Analysis of covariance (ANCOVA), 22 Analysis of variance (ANOVA), 7, 213 Analytic epidemiology, 96 Androgen-deprivation therapy (ADT), 44, 190, 272 Angina pectoris (AP), 215 Angiotensin converting enzyme (ACE) inhibitor, Anterior cruciate ligament (ACL) injury, 175 Attributable risk (AR), 179 Atypical case-noncase design, 290, 291 B Bayesian theorem, 76 BC, see Bronchial carcinoma (BC) Beta error rate, 74 Bias, 242, 248 lead-time, 82 length, 82–83 observation, 230 recall, 152, 163, 249 referral, 82 surveillance, 249 types of, 258–259 volunteer, 82 Biased estimate, confounding and, 58–60 Big data, 279, 281, 288 Biologic interaction, statistical versus, 63–67 Body mass index (BMI), 54, 57, 146 Bronchial carcinoma (BC), 111 C Cancer epidemiology, 271–273 CaP, see Prostate cancer (CaP) Cardiovascular epidemiology, 273–274 Case ascertainment (disease screening and diagnosis), 71–89 alpha error rate, 78 balancing benefits and harmful effects in medicine, 84–86 beta error rate, 74 diagnostic or screening test accuracy/ validity, 80–81 diagnostic tests and clinical reasoning, 83–84 diagnostic test validity, 72 disease screening, 78–84 early disease detection, issues in, 81–83 false-positive error rate, 78 lead-time bias, 82 length bias, 82–83 measures of diagnostic value, 74 307 308  Index negative predictive value, 76 odds ratio, 77 positive predictive value, 76 predictive values, 75 questions for discussion, 87–88 receiver operating characteristic curve, 81 referral bias, 82 risk difference, 85 screening (detection) and diagnostic (confirmation) tests, 71–78 screening test, 73, 74 sensitivity, 74 specificity, 75 type I error rate, 78 type II error, 74 Case-control studies, 143–162 aggregate data, analysis using, 159 “ambidirectional case-control” design, 145 ascertainment of exposure, 151 basis of case-control design, 146–152 case-cohort design, 152 cases ascertained, 147–152 definition, 143 design analysis, 158 effect measure modifier, 159 hybrids, 149–150, 160 hypothetical case-control, 160 incidence rate ratio, 153 matched case-control, measure of disease effect obtained in, 155 measure of effect of association, 151–152 questions for discussion, 161 randomized placebo-controlled clinical trials, 143 scientific reporting, 156–160 selection of cases, 147–148 selection of controls, 148–149 sources of controls, 150–151 sources of data, 148 variance of case-control design, 152–156 Causal inference in clinical research and quantitative evidence synthesis, 227–262 additive model, 253 bias, 242, 248, 258–259 causal inference, 228–229 chi square test of homogeneity, 254 clinical trial assessment, 233 confidence interval, 241–242 confounding, 251–252 DerSimonian–Laird method (random effects models), 247–252 effect measure modifier, 254 epidemiologic causal inference, 255 errors in clinical research, effects of, 230 errors of measurements, minimizing, 228 errors in observations, types of, 229 fixed effects model, 246–247, 260 heterogeneity test, 246 hypothesis testing, errors in, 241 interaction, 253 intermediate variate, 252 log rank test, 232 Mantel–Haenszel method (fixed effects model), 246–247 meta-analysis, 227 multiplicative model, 253 nonstatistically significant result interpretation, 238 null hypothesis, 240 observation bias, 230 Peto method (fixed effects model), 247 power and sample size, 243 precision, 252–253 public health/epidemiologic research, critical appraisal of, 239–243 quantitative evidence synthesis applied meta-analysis, 244–246 questions for discussion, 261–262 random effects models, 247–252 random error, 252 random error, role of, 239–240 randomization and blinding, 242 randomized clinical trials, critique of, 229–238 Rothman’s component cause model (causal pies), 256–257 scale of measurement and distribution, 239 scientific literature, critical appraisal of, 260 significance level, 240–241 STATA syntax, 231 statistical/analytic methods, 246 study accuracy, 235 study validity, 248 univariable Cox regression model, 233 Causal pies model, 256–257 CDs, see Chronic diseases (CDs) Centers for Disease Control and Prevention (CDC), 286 Index 309 Cerebral palsy (CP), 29 Challenges of epidemiology, see Perspectives, challenges, and future of epidemiology CHD, see Coronary heart disease (CHD) CHF, see Congestive heart failure (CHF) Chi-square test, 22, 41, 254 Chronic diseases (CDs), 265 Chronic kidney disease (CKD), 166 Chronic obstructive pulmonary disease (COPD), 109, 135 CI, see Confidence interval (CI) Clinical epidemiology, 267–268 Clinical reasoning, 83–84 Clinical research, purpose of, 17–18, 27 Clinical research proposal development and protocol, 27–48 clinical trial, 27 confounding, 29 data collection, management, and analysis, 37–46 dependent variable, 32 elements of study proposal, 37 hypothesis-specific or main analysis, 44–46 hypothetical introduction, 35 intended purpose to the design, 44 intermediate variate, 30 meta-analysis, 35 multivariable Cox regression model, 44 protocol implementation, 36 purpose of clinical research, 17–18, 27 qualitative data collection, 42 quantitative data collection, 41–42 questions for discussion, 47–48 reliability, 31 repeatability, 40 reproducibility, 40 research question, 32–33 study accuracy, 31 study background, 34–36 study conceptualization, 28–32 validity and reliability, 30–32 variable ascertainment, 28–29 Clinical trials (CTs), 205–226 assessment, 233 basic design, 211 blinding, 211–212 community trials, 209 conceptualization and conduct of, 214–217 cross-over design, 213 definition, 205 design, 207 double-blinded trials, 221 elements of, 214 ethical considerations, 217 example, 217–221 experimental design, CT as, 206–208 factorial design, 213 feasible design, 211 goals of, 210 human experimental design, basics of, 210 intervention trial, analysis strategies in, 221 Kaplan–Meier survival estimates, 224 mortality rates, 221, 222, 223 notion of, 206 open-label trial, 212 parallel design, 212–213 phases of, 208–212 placebo control, 221 postmarketing, 224 purpose, 205–206 questions for discussion, 224–225 randomized clinical trials 205 randomized cross-over CT, 208 reliable clinical trial, 208 sample size, 216–217 series design, 213 single-blinded trials, 221 study population, 216 triple-blinded trials, 221 types of CT designs and statistical inference, 212–213 Clinical versus population-based research, 13–14 Cochran–Mantel–Haensel Stratification Analysis, 195 Cognitive-related symptoms (CRSs), 56 Cohort studies, 125, 175–203 advantages and disadvantages, 201 ambidirectional cohort design, 198 attributable risk, 179 cohort designs, 178–198 confounding, 194 crude odds ratio, 196 distinctions of study designs, 189 historical cohort study, 190 hybrids, 198 longitudinal (cohort) studies, 182 measure of disease frequency and association/effect, 179–180 310  Index population attributable fraction, 187 prospective cohort design, 201 prospective cohort studies, 176, 180–189 questions for discussion, 202 rate ratio estimation, 198–199 retrospective cohort study, 176, 190–198 retrospective design, 182 risk ratio estimation, 184 STATA syntax, 188, 189 Component cause model (Rothman), 256–257 Confidence interval (CI), 47, 241–242 Confounding biased estimate and, 58–60 in cohort design, 194 consequentialist epidemiology and, 301–302 elements and characteristics, 29–30 magnitude of, 55 qualifications for, 68 random effects models, 251–252 residual, 56 study conceptualization and, 29 types of, 55–58 Congestive heart failure (CHF), 109 Consequentialist epidemiology and translational research implication, 297–305 accountability (sampling and confounding, adequate modeling), 301–302 consequentialist epidemiology (methods), 300–301 consequentialist pathway, 299 consequentialist science, 298–300 incomplete and inconsistent clinical findings, 300 questions for discussion, 305 translational epidemiology, 302–304 COPD, see Chronic obstructive pulmonary disease (COPD) Coronary heart disease (CHD), 9, 217 Correlation coefficient, 22 Cox Proportional Hazard Model, 22, 233 Cox regression model, 44 CP, see Cerebral palsy (CP) Cronbach’s alpha, 44 Cross-sectional design (CSD), 163 Cross-sectional studies (CSSs), 128, 163–173 basis of, 164 case-control versus, 166 definition, 163 feasibility of, 164–169 purpose, 170 questions for discussion, 172–173 recall bias, 163 CRSs, see Cognitive-related symptoms (CRSs) Crude rates, 110–111 CSD, see Cross-sectional design (CSD) CSSs, see Cross-sectional studies (CSSs) CTs, see Clinical trials (CTs) D Data collection, management, and analysis, 37–46 data analysis, 43–44 evidence discovery process, 38 hypothesis-specific or main analysis, 44–46 modern data collection techniques, 42 qualitative data collection, 42 quality of measurement instrument, 44 quantitative data collection, 41–42 repeatability, 40 reproducibility, 40 research data management, 43 Dental disorders (DD), 290 Dependent variable, 32 DerSimonian–Laird method (random effects models), 247–252 Descriptive epidemiology, 96 Design challenges (confounding and effect measure modifier), 49–70 assessment for confounding, 51–53 confounding and biased estimate, 58–60 confounding, covariates, and mediation, 50, 54 consistency, 54 effect measure modifier, 49, 60–63 mediation, 50 questions for discussion, 69 residual confounding, 56 standardization, 54 statistical versus biologic interaction, 63–67 stratified analysis, 52 types of confounding, 55–58 Index 311 Diabetes mellitus (DM), 12, 56, 109 Diagnostic (confirmation) tests, 71–78 Discovered dead during sleep (DDDS), 109, 298 Disease causation, models of, 99–101 Disease occurrence and association, measures of, see Epidemiology, historical context, and measures of disease occurrence and association Disease screening, 78–84 advantages and disadvantages of screening, 81 alpha error rate, 78 clinical phase, 82 diagnostic or screening test accuracy/ validity, 80–81 diagnostic tests and clinical reasoning, 83–84 early disease detection, issues in, 81–83 false-positive error rate, 78 lead-time bias, 82 preclinical phase, 81 receiver operating characteristic curve, 81 type I error rate, 78 DM, see Diabetes mellitus (DM) Double-blinded trials, 221 E EBE, see Evidence-based epidemiology (EBE) Ecologic studies, 125, 133–142 advantages and disadvantages of ecologic study design, 139 conducting of, 136 description, 133–137 ecologic design, 135 ecologic evidence (association versus causation), 137–138 ecologic fallacy, 141 examples, 137 feasibility of designs, 140 focus of, 134 group-level ecologic studies, 133 importance of ecologic data, 136–137 limitations of ecologic study design, 138–139 linear regression model, 137 questions for discussion, 141 statistical analysis, 137 time trends, 138 types of ecologic designs, 138 Effect measure modifier, 159, 254; see also Design challenges (con­ founding and effect measure modifier) Environmental epidemiology, 266 Environmental and occupational epidemiology (EOE), 274 Epidemiologic Triad, 119 Epidemiology, historical context, and measures of disease occurrence and association, 93–120 adjusted rate, 112 basic notion of epidemiology, 94–95 causal inference, 97 crude rates, 110–111 cumulative incidence, 104 determinants of health-related events, 95–96 disease effects, common measures of, 93 distribution, 95 epidemiologic classification (descriptive versus analytic), 96 Epidemiologic Triad, 119 epidemiology, clinical medicine, and public health research, 97–98 history and modern concept of epidemiology, 99 incidence density, 105 measures of disease association or effect, 110–114 measures of disease comparison, 114–116 measures of disease frequency, occurrence, and association, 101–110 models of disease causation, 99–101 objectives of epidemiology, 98–99 observational designs, 93 point prevalence, 106 prevalence proportion, 106 proportion, 102 proportionate mortality, 108–110 questions for discussion, 119–120 rate, 103 ratio, 102 sources of epidemiologic data, 117, 118 specific rates, 111 standardized rate, 94, 119 Error alpha, 78 beta, 74 in clinical research, effects of, 230 312  Index false-positive, 78 of measurements, minimizing, 228 in observations, types of, 229 random, 239–240, 252 type I, 78 type II, 74 Evidence-based epidemiology (EBE), 288 Evidence discovery process, 38 Experimental design, CT as, 206–208 Extra virgin olive oil (EVOO), 51, 138 F False-positive error rate, 78 Family history of hypertension (FMH), 168, 187 Fasting blood level (FBL), Fixed effects model, 246–247, 260 Friedman’s test, 213 Future of epidemiology, see Perspectives, challenges, and future of epidemiology G Generalization (research), 21 Genetic epidemiology, 269–271 Geographic information system (GIS), 290 Gleason Score, 72 Gold standard, 122, 143, 208, 223 Gram-negative bacterial pathogen, 73 Group-level ecologic studies, 133; see also Ecologic studies public health policy, 286 questions for discussion, 296 resource allocation, 294 “Healthography,” Health policy formulation, epidemiology and, 274–280 High-density lipoprotein (HDL), 64 Historical cohort study, 190 Historical context, see Epidemiology, historical context, and measures of disease occurrence and association Historical design, 127, 182 H1N1 pandemic, 124 Hormone replacement therapy (HRT) trial, 217 HR, see Hazard ratio (HR) Human experimental designs, see Clinical trials (CTs) Hyperdense data, 279 Hypertension (HTN), 64, 102 Hypothesis definition of, 23 -specific analysis, 44–46 testing, 11, 241 Hypothetical case-control, 160 I Immunoglobulin G (IgG), 49 Incidence density (ID), 105 Incidence density ratio (IRD), 170 Incidence rate (IR), 93 Incidence rate ratio (IRR), 153 Infectious disease epidemiology, 268–269 H K Hazard ratio (HR), 56, 233 HDL, see High-density lipoprotein (HDL) Health disparities epidemiology, 275–277 Health and healthcare policies, 285–296 atypical case-noncase design, 290, 291 decision-making (management), 294–295 decision-making (policy), 293–294 evidence-based epidemiology and “big data” practice, 288 health policy, 287–288 health policy formulation, 289–293 policy cycle, 289 Kaplan–Meier survival estimates, 224 Kruskal–Wallis test, 213 L LBW, see Low birth weight (LBW) LDL, see Low-density lipoprotein (LDL) Lead-time bias, 82 Legacy data, 279 Length bias, 82–83 Likelihood ratios (LRs), 76 Linear regression model, 137 Logistic regression, 22, 41, 45 Log rank test, 232 Log relative hazard (LRH), 233 Longitudinal (cohort) studies, 182 Index 313 Low birth weight (LBW), 45 Low-density lipoprotein (LDL), 64 LRH, see Log relative hazard (LRH) LRs, see Likelihood ratios (LRs) M Magnitude of confounding (MC), 55 Mann–Whitney test, 213 Mantel–Haenszel method (fixed effects model), 246–247 Mantel–Haenzel (M-H) stratified analysis, 52, 128 Measles–mumps–rubella (MMR) vaccination, 155 Measurement instrument, quality of, 44 Medical care/healthcare outcomes epidemiology, 271 mHealth data source, 279 MI, see Myocardial infarction (MI) MMR vaccination, see Measles– mumps–rubella (MMR) vaccination Molecular epidemiology, 269–271 Multiple sclerosis (MS), 130 Myocardial infarction (MI), 215 N National Cancer Institute (NCI), 190 National Institutes of Health (NIH), 19, 287 National Survey of Children’s Health (NSCH), 290 Negative predictive value (NPV), 76, 86 Nonconcurrent design, 127, 182 Null hypothesis, 240 Number needed to harm (NNH), 71, 97, 237 Number needed to treat (NNT), 71, 87, 237 Nutritional epidemiology (NE), 268 O Observation bias, 230 designs, 131 types of errors in, 229 Obstructive sleep apnea syndrome (OSAS), 16 Odds ratio (OR), 77, 146, 302 One-way analysis of variance, 213 Open-label trial, 212 P Paired t-test, 213 Pancreatic neoplasm (PN), 111 Parametric test statistic, 13 Partial intracapsular tonsillectomy (PITA), 16 Perspectives, challenges, and future of epidemiology, 265–283 aging epidemiology, 273 cancer epidemiology, 271–273 CD and cardiovascular epidemiology, 273–274 clinical epidemiology, 267–268 collaboration, 266 environmental epidemiology, 266 environmental and occupational epidemiology, 274 future challenges and opportunities, 278–279 health disparities epidemiology, 275–277 infectious disease epidemiology, 268–269 medical care/healthcare outcomes epidemiology, 271 molecular and genetic epidemiology, 269–271 nutritional epidemiology, 268 process and outcomes measures (health and healthcare), 277 questions for discussion, 281 research design innovation, 279–280 social epidemiology, 266 Peto method (fixed effects model), 247 PITA, see Partial intracapsular tonsillectomy (PITA) Placebo control, 221 PN, see Pancreatic neoplasm (PN) Pool estimate, 245 Population-based research, clinical versus, 13–14 Population medicine, 13 POR, see Prevalence odds ratio (POR) Positive predictive value (PPV), 76, 86 Postmarketing, 224 Predictive values, 75 Prevalence odds ratio (POR), 65, 168 proportion, 106 ratio (PR), 170 314  Index risk ratio (PRR), 292 studies, 128 Primary versus secondary outcomes (research), 12–18 clinical versus population-based research, 13–14 epidemiologic/population-based research, 14 purpose of clinical research, 17–18 research rationale, 15–17 scales of measurement, 13 Proportionate mortality (PM), 108–110 Prospective cohort studies, 176, 180–189; see also Cohort studies Prostate cancer (CaP), 10 case-control study of, 150 diabetes mellitus and, 56 Gleason Score used in, 72 mortality, 17, 133 Prostate-specific antigen (PSA), 12, 72 Protocol implementation (clinical research), 36 Purpose of clinical research, 17–18, 27 Q Qualitative data collection, 42 Quality of measurement instrument, 44 Quantitative data collection, 41–42 Quantitative evidence synthesis (QES), 57, 227, 288; see also Causal inference in clinical research and quantitative evidence synthesis Quantitative evidence synthesis (QES) applied meta-analysis, 244–246 meta-analysis and pool analysis, 245–246 methodology, 246 quantitative systematic review, 244–245 Questionnaire types, 42 Questions for discussion case ascertainment (disease screening and diagnosis), 87–88 case-control studies, 161 causal inference in clinical research and quantitative evidence synthesis 261–262 clinical research proposal development and protocol, 47–48 clinical trials, 224–225 cohort studies, 202 consequentialist epidemiology and translational research implication, 305 cross-sectional studies, 172–173 design challenges, 69 ecologic studies, 141 epidemiology, historical context, and measures of disease occurrence and association, 119 health and healthcare policies, 296 perspectives, challenges, and future of epidemiology, 281 research conceptualization and rationale, 24 study designs, 130–131 R Random effects models, 247–252 Randomized clinical trials (RCTs), 205 causal relationship, 237–238 critique of, 229–238 elements of, 212 external validity of, 236 internal validity of, 230–235 placebo-controlled, 143 Random variables, 11 RD, see Risk difference (RD) Recall bias, 152, 163, 249 Receiver operating characteristic (ROC) curve, 81 Referral bias, 82 Relative risk (RR), 84, 175 Repeated measures ANOVA (RANOVA), 7, 16 Research conceptualization and rationale, 3–25 cases, 17 clinical versus population-based research, 13–14 control, 17 epidemiologic/population-based research, 14 generalization, 21 human experiment, 10 hypothesis, definition of, 23 hypothesis testing, 11 objective of study/research purpose, 7–8, 23 parametric test statistic, 13 primary versus secondary outcomes, 12–18 Index 315 purpose of clinical research, 17–18 purpose of study, 23 questions for discussion, 24 random variables, 11 rationale for research, 18 repeated-measure analysis of variance, research questions, 9–11 research rationale, 15–17 sample size and power estimations, 22 sampling, 19–20 scales of measurement, 13 structure and function of research, 4–7 study hypothesis, 11–12 study subjects, 19 Residual confounding, 56 Retrospective cohort study, 127, 176, 190–198 Risk difference (RD), 63, 85 ROC curve, see Receiver operating characteristic (ROC) curve RR, see Relative risk (RR) S Screening (detection) tests, 71–78 SE, see Social epidemiology (SE) Secondary outcomes, primary versus (research), 12–18 Second primary thyroid cancer (SPTC), 302 (SPTC) SEER data, see Surveillance, Epidemiology, and End Results (SEER) data Single-blinded trials, 221 Skeletal dysplasia (SKD), Social epidemiology (SE), 266 Social media data source, 279 Specific rates, 111 SPTC, see Second primary thyroid cancer (SPTC) SSI, see Surgical site infection (SSI) Standardized rate, 94, 119 STATA syntax, 188, 189, 231 Statistical interaction, biologic versus, 63–67 Statistical power (SP), 215 Statistical tests (for selected data types), 243 Strengthening the Reporting of Observational Studies in Epidemiology (STROBE), 156 Study designs, 121–131 case-control design, 124, 128 classifications of study designs, 121 cohort studies, 125 cross-sectional design, 125 cross-sectional studies, 128 descriptive and analytic epidemiology, 129–130 disease distribution, 124–125 disease outbreak determinants, 128 ecologic studies, 125 factors, 122–123 gold standard, 122 H1N1 pandemic, 124 Mantel–Haenszel method, 128 nonexperimental versus experimental design, 125–128 observational designs, 131 prevalence studies, 128 questions for discussion, 130–131 retrospective cohort, 127 traditional epidemiologic design, 127–128 Study proposal, elements of, 37 Surgical site infection (SSI), 294 Surveillance bias, 249 Surveillance, Epidemiology, and End Results (SEER) data, 20, 56, 190 Survival analysis, 22 Survival time, 116 Systems science, 280 T Thoraco-lumbar kyphosis (TLK) progression, 192 Time trends, 138 Transdisciplinary research, 279 Translational epidemiology (TransEpi), 302–304; see also Consequentialist epidemiology and translational research implication Triple-blinded trials, 221 Type diabetes (T2DM), 56 Type I error, 22, 78 Type II error, 74 U Ultraviolet (UV) radiation, 56 Unicameral bone cysts (UBCs), 268 Univariable Cox regression model, 233 316  Index V X Variable ascertainment, study conceptualization, 28–29 Volunteer bias, 82 Xeroderma pigmentosum (XP), 63 W Wilcoxon signed-rank test, 213 Women’s Health Initiative (WHI), 217 Z z statistics, 12, 45 ... 018 019 020 021 022 023 024 025 026 027 028 029 030 Age 50 45 60 70 66 65 48 50 55 63 43 67 75 64 65 48 50 55 63 43 67 65 48 50 55 63 43 67 59 61 Race/ethnicity 2 2 3 1 1 2 2 2 Exposure (selenium... disease-truenegative) Nonexposure prevalence (n, %) Figure 8 .2 Case-control and cross-sectional designs Exposure prevalence (n, %) 146  Applied epidemiologic principles and concepts 8 .2 Basis... Public Health 16 (1995 ): 61–81 136  Applied epidemiologic principles and concepts 7 .2. 1 Conducting an ecologic study To conduct an ecologic study, we need to have aggregate information on groups

Ngày đăng: 21/01/2020, 08:44

Từ khóa liên quan

Mục lục

  • Cover

  • Title Page

  • Copyright Page

  • Dedication

  • Table of Contents

  • Foreword

  • Preface

    • Epidemiology Conceptualized

    • Epidemiology Today

    • Acknowledgments

    • Author

    • Section Section I : Basic research design principles and study inference

    • Chapter 1: Epidemiologic research conceptualization and rationale

      • 1.1 Introduction

      • 1.2 Structure and function of research

      • 1.3 Objective of study/research purpose

      • 1.4 Research questions and study hypotheses

        • 1.4.1 Research questions

        • 1.4.2 Study hypothesis

        • 1.5 Primary versus secondary outcomes

          • 1.5.1 Scales of measurement

          • 1.5.2 Clinical versus population-based research

          • 1.5.3 Epidemiologic / population-based research

          • 1.5.4 Research rationale

Tài liệu cùng người dùng

Tài liệu liên quan