Statistical Methods in Medical Research - part 7 ppsx

calculate p H  1=2n when r  0 and p H 2n À1=2n when r  n, and obtain y from p H . A further point is that the probit transformation does not stabilize variances, even for observations with constant n. Some form of weighting is therefore desirable in any analysis. A rigorous approach is provided by the method called probit analysis (Finney, 1971; see also §20.4). The effect of the probit transformation in linearizing a relationship is shown in Fig. 14.1. In Figure 14.1(b) the vertical axis on the left is the NED of p, and the scale on the right is the probability scale, in which the distances between points on the vertical scale are proportional to the corresponding distances on the probit or NED scale. Logit transformation The logit of p is defined as y  ln p 1 Àp : 14:5 Occasionally (Fisher & Yates, 1963; Finney, 1978) the definition incorporates a factor 1 2 so that y  1 2 lnp=1 Àp; this has the effect of making the values rather similar to those of the NED (i.e. probit À 5). The effect of the logit (or logistic) transformation is very similar indeed to that of the probit transformation. The probit transformation is reasonable on biological grounds in some circumstances; for example, in a quantal assay of insecticides applied under different controlled conditions, a known number of flies might be exposed at a number of different doses and a count made of the number killed. In this type of study, individual tolerances or their logs may be assumed to have a normal distribution, and this leads directly to the probit model (§20.4). The logit transformation is more arbitrary, but has important advantages. First, it is easier to calculate, since it requires only the log function rather than the inverse normal distribution function. Secondly, and more importantly, the logit is the logarithm of the odds, and logit differences are logarithms of odds ratios (see (4.22)). The odds ratio is important in the analysis of epidemiological studies, and logistic regression can be used for a variety of epidemiological study designs (§19.4) to provide estimates of relative risk (§19.5). 14.2 Logistic regression The logit transformation gives the method of logistic regression: ln m 1 Àm   b 0  b 1 x 1  b 2 x 2  b p x p : 14:6 488 Modelling categorical data Fitting a model Two approaches are possible: first, an approximate method using empirical weights and, secondly, the theoretically more satisfactory maximum likelihood solution. The former method is a weighted regression analysis (p. 344), where each value of the logit is weighted by the reciprocal of its approximate variance. This method is not exactÐfirst, because the ys are not normally distributed about their population values and, secondly, because the weights are not exactly in inverse proportion to the variances, being expressed in terms of the estimated proportion p. For this reason the weights are often called empirical. Although this method is adequate if most of the sample sizes are reasonably large and few of the ps are close to 0 or 1 (Example 14.1 was analysed using this method in earlier editions of this book), the ease of using the more satisfactory maximum likelihood method with statistical software means it is no longer recom- mended. If the observed proportions p are based on n  1 observation only, their values will be either 0 or 1, and the empirical method cannot be used. This situation occurs in the analysis of prognostic data, where an individual patient is classified as `success' or `failure', several explanatory variables x j are observed, and the object is to predict the probability of success in terms of the xs. Maximum likelihood The method of estimation by maximum likelihood, introduced in §4.1, has certain desirable theoretical properties and can be applied to fit logistic regression and other generalized linear models. The likelihood of the data is proportional to the probability of obtaining the data (§3.3). For data of known distributional form, and where the mean value is given in terms of a generalized linear model, the probability of the observed data can be written down using the appropriate probability distributions. For example, with logistic regression the probability for each group or individual can be calculated using the binomial probability from (14.6) in (3.12) and the likelihood of the whole data is the product of these probabilities over all groups or individuals. This likelihood depends on the values of the regression coefficients, and the maximum likelihood estimates of these regression coefficients are those values that maximize the likelihoodÐthat is, the values for which the data are most likely to occur. For theoretical reasons, and also for practical convenience, it is preferable to work in terms of the logarithm of the likelihood. Thus it is the log-likelihood, L, that is maximized. The method also gives standard errors of the estimated regression coefficients and significance tests of specific hypotheses. By analogy with the analysis of variance for a continuous variable, the analysis of deviance is used in generalized linear models. The deviance is defined 14.2 Logistic regression 489 as twice the difference between the log-likelihood of a perfectly fitting model and that of the current model, and has associated degrees of freedom (DF) equal to the difference in the number of parameters between these two models. Where the error distribution is completely defined by the link between the random and linear parts of the modelÐand this will be the case for binomial and Poisson variables but not for a normal variable, for which the size of the variance is also requiredÐthen deviances follow approximately the x 2 distribution and can be used for the testing of significance. In particular, reductions in deviance due to adding extra terms into the model can be used to assess whether the inclusion of the extra terms had resulted in a significant improvement to the model. This is analogous to the analysis of variance test for deletion of variables described in §11.6 for a continuous variable. The significance of an effect on a single degree of freedom may be tested by the ratio of its estimate to its standard error (SE), assessed as a standardized normal deviate. This is known as the Wald test, and its square as the Wald x 2 . Another test is the score test which is based on the first derivative of the log- likelihood with respect to a parameter and its variance (see Agresti, 1996, §4.5.2). Both are evaluated at the null value of the parameter and conditionally on the other terms in the model. This statistic is less readily available from statistical software except in simple situations. The procedure for fitting a model using the maximum likelihood method usually involves iterationÐthat is, repeating a sequence of calculations until a stable solution is reached. Fitted weights are used and, since these depend on the parameter estimates, they change from cycle to cycle of the iteration. The approximate solution using empirical weights could be the first cycle in this iterative procedure, and the whole procedure is sometimes called iterative weighted least squares. The technical details of the procedure will not be given since the process is obviously rather tedious, and the computations require appropriate statistical software (for example, PROC LOGISTIC in SAS (2000), LOGISTIC REGRESSION in SPSS (1999), or GLIM (Healy, 1988). For further details of the maximum likelihood method see, for example, Wetherill (1981). Example 14.1 Table 14.1 shows some data reported by Lombard and Doering (1947) from a survey of knowledge about cancer. These data have been used by several other authors (Dyke & Patterson, 1952; Naylor, 1964). Each line of the table corresponds to a particular combination of factors in a 2 4 factorial arrangement, n being the number of individuals in this category and r the number who gave a good score in response to questions about cancer knowledge. The four factors are: A, newspaper reading; B, listening to radio; C, solid reading; D, attendance at lectures. 490 Modelling categorical data Table 14.1 A2 4 factorial set of proportions (Lombard & Doering, 1947). The fitted proportions from a logistic regression analysis are shown in column (4). (1) (2) (3) (4) Factor combination Number of individuals n Number with good score r Observed proportion (2)=(1) p Fitted proportion (1) 477 84 0Á176 0Á188 (a) 231 75 0Á325 0Á308 (b) 63 13 0Á206 0Á240 (ab) 94 35 0Á372 0Á377 (c) 150 67 0Á447 0Á382 (ac) 378 201 0Á532 0Á542 (bc) 32 16 0Á500 0Á458 (abc) 169 102 0Á604 0Á618 (d) 12 2 0Á167 0Á261 (ad) 13 7 0Á538 0Á404 (bd) 7 4 0Á571 0Á325 (abd) 12 8 0Á667 0Á480 (cd) 11 3 0Á273 0Á485 (acd) 45 27 0Á600 0Á643 (bcd) 4 1 0Á250 0Á562 (abcd) 31 23 0Á742 0Á711 Although the data were obtained from a survey rather than from a randomized experiment, we can usefully study the effect on cancer knowledge of the four main effects and their interactions. The main effects and interactions will not be orthogonal but can be estimated. There are 16 groups of individuals and a model containing all main effects and all interactions would fit the data perfectly. Thus by definition it would have a deviance of zero and serves as the reference point in assessing the fit of simpler models. The first logistic regression model fitted was that containing only the main effects. This gave a model in which the logit of the probability of a good score was estimated as À1Á4604  0Á6498A  0Á3101B 0Á9806C 0Á4204D SE: 0Á1154 0Á1222 0Á1107 0Á1910 z:5Á63 2Á54 8Á862Á20 P: <0Á001 0Á011 <0Á001 0Á028 Here the terms involving A, B, C and D are included when these factors are present and omitted otherwise. The significance of the main effects have been tested by Wald's testÐthat is, the ratio of an estimate to its standard error assessed as a standardized normal deviate. Altern- atively, the significance may be established by analysis of deviance. For example, fitting the model containing only the main effects of B, C and D, gives a deviance of 45Á47 with 12 14.2 Logistic regression 491 DF. Adding the main effect of A to the model reduces the deviance to 13Á59 with 11 DF, so that the deviance test for the effect of A, after allowing for B, C and D, is 45Á47 À 13Á59  31Á88 as an approximate x 2 1 . This test is numerically similar to Wald's test, since 31Á88  5Á65 p , but in general such close agreement would not be expected. Although the deviance tests of main effects are not necessary here, in general they are needed. For example, if a factor with more than two levels were fitted, using dummy variables (§11.7), a deviance test with the appropriate degrees of freedom would be required. The deviance associated with the model including all the main effects is 13Á59 with 11 DF, and this represents the 11 interactions not included in the model. Taking the deviance as a x 2 11 , there is no evidence that the interactions are significant and the model with just main effects is a good fit. However, there is still scope for one of the two-factor interactions to be significant and it is prudent to try including each of the six two-factor interactions in turn to the model. As an example, when the interaction of the two kinds of reading, AC, is included, the deviance reduces to 10Á72 with 10 DF. Thus, this interaction has an approximate x 2 1 of 2Á87, which is not significant (P  0Á091). Similarly, none of the other interactions is significant. The adequacy of the fit can be visualized by comparing the observed and fitted proportions over the 16 cells. The fitted proportions are shown in column (4) of Table 14.1 and seem in reasonable agreement with the observed values in column (3). A formal test may be constructed by calculating the expected frequencies, E(r) and En Àr, for each factor combination and calculating the Pearson's x 2 statistic (8.28). This has the value 13Á61 with 11 DF (16 À5, since five parameters have been fitted). This test statistic is very similar to the deviance in this example, and the model with just the main effects is evidently a good fit. The data of Example 13.2 could be analysed using logistic regression. In this case the observed proportions are each based on one observation only. As a model we could suppose that the logit of the population probability of survival, Y, was related to haemoglobin, x 1 , and bilirubin, x 2 by the linear logistic regression formula (14.6) Y  b 0  b 1 x 1  b 2 x 2 : Application of the maximum likelihood method gave the following estimates of b 0 , b 1 , and b 2 with their standard errors: ^ b 0 À2Á354 Æ 2Á416 ^ b 1  0Á5324 Æ 0Á1487 ^ b 2 À0Á4892 Æ 0Á3448: 14:7 The picture is similar to that presented by the discriminant analysis of Example 13.2. Haemoglobin is an important predictor; bilirubin is not. An interesting point is that, if the distributions of the xs are multivariate normal, with the same variances and covariances for both successes and failures (the basic model for discriminant analysis), the discriminant function (13.4) can also be used to predict Y. The formula is: 492 Modelling categorical data Y  b H 0  b H 1 x 1  b H 2 x 2   b H p x p , where b H j  b j and b H 0 À 1 2 b H 1   x A1   x B1  b H p   x Ap   x Bp  lnn A =n B : 14:8 In Example 13.2, using the discriminant function coefficients b 1 and b 2 given there, we find b H 0 À4Á135, b H 1  0Á6541, b H 2 À0Á3978, which lead to values of Y not differing greatly from those obtained from (14.7), except for extreme values of x 1 and x 2 . An example of the use of the linear discriminant function to predict the probability of coronary heart disease is given by Truett et al. (1967). The point should be emphasized that, in situations in which the distributions of xs are far from multivariate normal, this method may be unreliable, and the maximum likelihood solution will be preferable. To test the adequacy of the logistic regression model (14.6), after fitting by maximum likelihood, an approximate x 2 test statistic is given by the deviance. This was the approach in Example 14.1, where the deviance after fitting the four main effects was 13Á59 with 11 DF (since four main effects and a constant term had been estimated from 16 groups). The fit is clearly adequate, suggesting that there is no need to postulate interactions, although, as was done in the example, a further refinement to testing the goodness of fit is to try interactions, since a single effect with 1 DF could be undetected when tested with other effects contributing 10 DF. In general terms, the adequacy of the model can be assessed by including terms such as x 2 i , to test for linearity in x i , and x i x j , to test for an interaction between x i and x j . The approximation to the distribution of the deviance by x 2 is unreliable for sparse dataÐthat is, if a high proportion of the observed counts are small. The extreme case of sparse data is where all values of n are 1. Differences between deviances can still be used to test for the inclusion of extra terms in the model. For sparse data, tests based on the differences in deviances are superior to the corresponding Wald test (Hauck & Donner, 1977). Goodness-of-fit tests should be carried out after forming groups of individuals with the same covariate patterns. Even for a case of individual data, it may be that the final model results 14.2 Logistic regression 493 in a smaller number of distinct covariate patterns; this is particularly likely to be the case if the covariates are categorical variables with just a few levels. The value of the deviance is unaltered by grouping into covariate patterns, but the degrees of freedom are equal to the number of covariate patterns less the number of parameters fitted. For individual data that do not reduce to a smaller number of covariate patterns, tests based on grouping the data may be constructed. For a logistic regression, grouping could be by the estimated probabilities and a x 2 test produced by comparing observed and expected frequencies (Lemeshow & Hos- mer, 1982; Hosmer & Lemeshow, 1989, §5.2.2). In this test the individuals are ranked in terms of the size of the estimated probability, P, obtained from the fitted logistic regression model. The individuals are then divided into g groups; often g  10. One way of doing this is to have the groups of equal sizeÐthat is, the first 10% of subjects are in the first group, etc. Another way is to define the groups in terms of the estimated probabilities so that the first group contains those with estimated probabilities less than 0Á1, the second 0Á1to0Á2, etc. A g Â2 table is then formed, in which the columns represent the two categories of the dichotomous outcome variable, containing the observed and expected numbers in each cell. The expected numbers for each group are the sum of the estimated probabilities, P, and the sum of 1 À P, for all the individuals in that group. A x 2 goodness-of-fit statistic is then calculated (11.73). Based on simulations, Hosmer and Lemeshow (1980) showed that this test statistic is distributed approximately as a x 2 with g À 2 degrees of freedom. This test can be modified when some individuals have the same covariate pattern (Hosmer & Lemeshow, 1989, §5.2.2), provided that the total number of covariate patterns is not too different from the total number of individuals. Diagnostic methods based on residuals similar to those used in classical regression (§11.9) can be applied. If the data are already grouped, as in Example 14.1, then standardized residuals can be produced and assessed, where each residual is standardized by its estimated standard error. In logistic regression the standardized residual is r Àn^m n^m1 À ^m p  , where there are r events out of n. For individual data the residual may be defined using the above expression, with r either 0 or 1, but the individual residuals are of little use since they are not distributed normally and cannot be assessed individu- ally. For example, if ^m  0Á01, the only possible values of the standardized residual are 9Á9 and À0Á1; the occurrence of the larger residual does not necessarily indicate an outlying point, and if accompanied by 99 of the smaller residuals the fit would be perfect. It is, therefore, necessary to group the residuals, defining groups as individuals with similar values of the x i . 494 Modelling categorical data Alternative definitions of the residual include correcting for the leverage of the point in the space of the explanatory variables to produce a residual equiva- lent to the Studentized residual (11.67). Another definition is the deviance residual, defined as the square root of the contribution of the point to the deviance. Cox and Snell (1989; §2.7) give a good description of the use of residuals in logistic regression. The use of influence diagnostics is discussed in Cox and Snell (1989) and by Hosmer and Lemeshow (1989, §5.3). There are some differences in leverage between logistic regression and classical multiple regression. In the latter (see p. 366) the points furthest from the mean of the x variables have the highest leverages. In logistic regression the leverage is modified by the weight of each observation and points with low or high expected probabilities have small weight. As such probabilities are usually associated with distant points, this reduces the leverage of these points. The balance between position of an observation in the x variable space and weight suggests that the points with highest leverage are those with fitted probabilities of about 0Á2or0Á8 (Hosmer & Lemeshow, 1989, §5.3). The concept of Cook's distance can be used in logistic regression and (11.72) applies, using the modified leverage as just discussed, although in this case only approximately (Pregibon, 1981). In some cases the best-fitting model may not be a good fit, but all attempts to improve it through adding in other or transformed x variables fail to give any worthwhile improvement. This may be because of overdispersion due to some extra source of variability. Unless this variability can be explained by some extension to the model, the overdispersion can be taken into account in tests of significance and the construction of confidence intervals by the use of a scaling factor. Denoting this factor by f, any x 2 statistics are divided by f and standard errors are multiplied by f p . f may be estimated from a goodness-of-fit test. For non-sparse data this could be the residual deviance divided by its degrees of freedom. For sparse data it is difficult to identify and estimate overdispersion. For a more detailed discussion, see McCullagh and Nelder (1989, §4.5). The model might be inadequate because of an inappropriate choice of the link function. An approach to this problem is to extend the link function into a family indexed by one or more parameters. Tests can then be derived to deter- mine if there is evidence against the particular member of the family originally used (Pregibon, 1980; Brown, 1982; McCullagh & Nelder, 1989). The strength of fit, or the extent to which the fitted regression discriminates between observed and predicted, is provided by the concordance/discordance of pairs of responses. These measures are constructed as follows: 1 Define all pairs of observations in which one member of the pair has the characteristic under analysis and the other does not. 2 Find the fitted probabilities of each member of the pair, p  and p À . 3 Then, 14.2 Logistic regression 495 if p  > p À the pair is concordant; if p  < p À the pair is discordant; if p   p À the pair is tied. 4 Over all pairs find the percentages in the three classes, concordant, discordant and tied. These three percentages may be combined into a single summary measure in various ways. A particularly useful summary measure is c % concordant  0Á5% tied=100: A value of c of 0Á5 indicates no discrimination and 1Á0 perfect discrimination. (c is also the area under the receiver operating characteristic (ROC) curve (see §19.9).) For small data sets the methods discussed above will be inadequate, because the approximations of the test statistics to the x 2 distribution will be unsatisfac- tory, or a convergent maximum likelihood solution may not be obtained with standard statistical software. Exact methods for logistic regression may be applied (Mehta & Patel, 1995) using the LogXact software. It was mentioned earlier that an important advantage of logistic regression is that it can be applied to data from a variety of epidemiological designs, including cohort studies and case±control studies (§19.4). In a matched case± control study, controls are chosen to match their corresponding case for some variables. Logistic regression can be applied to estimate the effects of variables not included in the matching, but the analysis is conditional within the case± control sets; the method is then referred to as conditional logistic regression (§19.5). 14.3 Polytomous regression Some procedures for the analysis of ordered categorical data are described in of Chapter 15. These procedures are limited in two respects: they are appropriate for relatively simple data structures, where the factors to be studied are few in number; and the emphasis is mainly on significance tests, with little discussion of the need to describe the nature of any associations revealed by the tests. Both of these limitations are overcome by generalized linear models, which relate the distribution of the ordered categorical response to a number of explanatory variables. Because response variables of this type have more than two categories, they are often referred to as polytomous responses and the corresponding procedures as polytomous regression. Three approaches are described very briefly here. The first two are general- izations of logistic regression, and the third is related to comparisons of mean scores (see (15.8)). 496 Modelling categorical data The cumulative logits model Denote the polytomous response variable by Y, and a particular category of Y by j. The set of explanatory variables, x 1 , x 2 , , x p , will be denoted by the vector x. Let F j xPY j, given x and L j  logit F j x  ln F j x 1 ÀF j x ! : The model is described by the equation L j xa j À b H x, 14:9 where b H x represents the usual linear function of the explanatory variables, b 1 x 1  b 2 x 2   b p x p . This model effectively gives a logistic regression, as in (14.6), for each of the binary variables produced by drawing boundaries between two adjacent categories. For instance, if there are four categories numbered 1 to 4, there are three binary variables representing the splits between 1 and 2±4, 1±2 and 3±4, and 1±3 and 4. Moreover, the regression coefficients b for the explanatory variables are the same for all the splits, although the intercept term a j varies with the split. Although a standard logistic regression could be carried out for any one of the splits, a rather more complex analysis is needed to take account of the interrelations between the data for different splits. Computer programs are available (for instance, in SAS) to estimate the coefficients in the model and their precision, either by maximum likelihood (as in SAS LOGIST) or weighted least squares (as in SAS CATMOD), the latter being less reliable when many of the frequencies in the original data are low. The adjacent categories model Here we define logits in terms of the probabilities for adjacent categories. Define L j  ln p j p j1  , where p j is the probability of falling into the jth response category. The model is described by the equation L j  a j À b H x: 14:10 14.3 Polytomous regression 497 [...]... Institute of Health and Welfare) Veterans Age Non-veterans Number of cancers Number of cancers Subject-years Subject-years ±24 25±29 30±34 35±39 40±44 45±49 50±54 55±59 60±64 65±69 70 ± 6 21 54 118 97 58 56 54 34 9 2 60 840 1 57 175 176 134 186 514 135 475 42 620 25 001 13 71 0 6 163 1 575 273 18 60 122 191 108 74 88 120 141 108 99 208 4 87 303 832 325 421 312 242 165 5 97 54 396 40 71 6 33 801 26 618 17. .. effect after allowing for age is obtained from the deviance difference after adding in a treatment effect to a model already containing age; that is, 37 63 À 28Á84 8 79 (1 DF) Similarly, the effect of age adjusted for treatment has a test statistic of 2 70 (2 DF) There is no evidence of an interaction between treatment and age, or of a main effect of age The log-linear model fitting just treatment... of subdividing a contingency table In Example 15.5, the elementary table (a) could have been chosen as any one of the 2 Â 2 tables forming part of the whole table The choice, as in that example, will often be data-dependentÐthat is, made after an initial inspection of the data There is, therefore, the risk of data-dredging, and this should be recognized in any interpretation of the analysis In Example... described, in which the x2 statistic calculated for a contingency table is subdivided to shed light on specific ways in which categorical variables may be associated In §§15.6 and 15 .7 some of the methods described earlier are generalized for situations in which the data are stratified (i.e divided into subgroups), so that trends can be examined within strata and finally pooled Finally, in §15.8 we... wi di = wi p0i q0i : Example 15 .7 Table 15 .7 shows some results taken from a trial to compare the mortality from tetanus in patients receiving antitoxin and in those not receiving antitoxin The treatments were allocated at random, but by chance a higher proportion of patients in the `No antitoxin' group had a more severe category of disease (as defined in terms of incubation period and period of onset... well be included in the calculations The overall results favour the use of antitoxin, but this may be due partly to the more favourable distribution of cases Cochran's test proceeds as follows: Group I II III p0i 37= 47 0 78 72 10=29 0Á3448 2=3 0Á66 67 di 0Á1319 0Á3233 0Á5000 p0i q0i 0Á1 675 0Á2259 0Á2222 wi 11Á62 6Á83 0Á 67 19Á12 d 4Á 075 8=19Á12 0Á2132, p 3Á6381=19Á12 0Á09 97, SEd... variables relating to the patient and the injury was analysed using the cumulative logits model (14.9) of polytomous regression The final model included seven bs indicating the relationship between outcome and seven variables, which included age, whether the patient was transferred from a peripheral hospital, three variables representing injury severity, two interaction terms, and four as representing the... Â 17 1822 = 77 Á455 Â 43 77 3 15:12 5Á66: This is approximately a x2 statistic and is significant (P 0Á0 17) In this example most of 1 the difference between column means lies in the trend This is clear from examination of the column means, and the corresponding result for the test statistics is that the overall test statistic of 5 70 (2 DF) for equality of column means may be subdivided into... would be logical, in 2 2 calculating X2 and X4 , to use separate denominators for the contributions from 2 the two insecticides Thus, in calculating X2 the first term in the numerator would have a denominator 0Á93850Á0615 0Á0 577 , and the second term would have a denominator 0Á83160Á1684 0Á1400 The effect of this correction is usually small If it is made, the various x2 indices no longer... however, we were particularly interested in a certain form of departure from the null hypothesis, it might be possible to formulate a test which was particularly sensitive to this situation, although perhaps less effective than the portmanteau x2 test in detecting other forms of departure Sometimes these specially directed tests can be achieved by subdividing the total x2 statistic into portions which . 0Á 372 0Á 377 (c) 150 67 0Á4 47 0Á382 (ac) 378 201 0Á532 0Á542 (bc) 32 16 0Á500 0Á458 (abc) 169 102 0Á604 0Á618 (d) 12 2 0Á1 67 0Á261 (ad) 13 7 0Á538 0Á404 (bd) 7 4 0Á 571 0Á325 (abd) 12 8 0Á6 67 0Á480 (cd). two-factor interactions to be significant and it is prudent to try including each of the six two-factor interactions in turn to the model. As an example, when the interaction of the two kinds of. 191 312 242 40±44 97 135 475 108 165 5 97 45±49 58 42 620 74 54 396 50±54 56 25 001 88 40 71 6 55±59 54 13 71 0 120 33 801 60±64 34 6 163 141 26 618 65±69 9 1 575 108 17 404 70 ± 2 273 99 14 146 Total

Định dạng
Số trang	83
Dung lượng	521,26 KB