A-Novel-Estimation-Method-Based-on-Maximum-Likelihood

See discussions, stats, and author profiles for this publication at: https://www.researchgate.net/publication/330204895 A Novel Estimation Method Based on Maximum Likelihood Thesis · January 2014 CITATIONS READS 287 author: Md Mobarak Hossain University of Nevada, Reno PUBLICATIONS   12 CITATIONS    SEE PROFILE Some of the authors of this publication are also working on these related projects: A novel weighted likelihood estimation with empirical Bayes flavor View project All content following this page was uploaded by Md Mobarak Hossain on 08 January 2019 The user has requested enhancement of the downloaded file University of Nevada, Reno A NOVEL ESTIMATION METHOD BASED ON MAXIMUM LIKELIHOOD A thesis submitted in partial fulfillment of the requirements for the degree of Master of Science in Mathematics By Md Mobarak Hossain Dr Tomasz Kozubowski/Thesis Advisor December 2014 c 2014 Md Mobarak Hossain ALL RIGHTS RESERVED THE GRADUATE SCHOOL We recommend that the thesis prepared under our supervision by Md Mobarak Hossain entitled A NOVEL ESTIMATION METHOD BASED ON MAXIMUM LIKELIHOOD be accepted in partial fulfillment of the requirements for the degree of MASTER OF SCIENCE Tomasz Kozubowski, Ph.D., Advisor Anna Panorska, Ph.D., Committee Member Krzysztof Podg´orski, Ph.D., Committee Member Minggen Lu, Ph.D., Graduate School Representative David Zeh, Ph.D., Dean, Graduate School December, 2014 i ABSTRACT A Novel Estimation Method Based on Maximum Likelihood By Md Mobarak Hossain The method of maximum likelihood (ML) is perhaps the most widely used statistical approach to estimate unknown parameters in a parametric setting However, the required optimization of the likelihood function is rarely possible explicitly, and finding the estimators may be computationally challenging On the other hand, maximum likelihood estimators are often simple to compute when the sample size is equal to one Based on this observation, we propose a novel approach to estimation, where each individual observation in a random sample is used to derive an estimator of an unknown parameter using the ML principle These individual estimators are then put together as a weighted average to produce the final estimator The weights are chosen to be proportional to the likelihood function evaluated at the estimators based on each observation It turns out that this method can be related to a Bayesian approach, where the prior distribution is data driven In case of estimating a location parameter of a unimodal density, the prior distribution is the empirical distribution of the sample, and converges to the true distribution that generated the data as the sample size increases ii We provide several examples illustrating the new method, and conduct simulation studies to assess the performance of the estimators It turns out that this straightforward methodology produces consistent estimators, which seem to be comparable with those obtained by the ML method in large sample setting, and may actually outperform the latter when the sample size is small iii ACKNOWLEDGMENTS At first I would like to appreciate my honorable thesis advisor, Professor Tomasz J Kozubowski, who showed me a great interest in the field of statistics Professor Kozubowski not only helped me to complete the thesis but also encouraged me, supported me, guided me with great patience which accelerated me to complete my graduate study in the department of Mathematics and Statistics at the University of Nevada, Reno (UNR) I am really grateful to him I am also very grateful to the graduate school representative, Dr Minggen Lu, for his great support, and to Krzysztof Podg´orski from Lund University, Sweden, for his suggestions A special thanks also goes to the former Graduate Director, Professor Anna K Panorska, who not only gave me the best suggestion but also carefully helped me academically and spiritually to complete my graduate study in UNR I am really grateful for her excellent support for the graduate students I also would like to acknowledge my family’s support and encouragement Moreover, I am very much thankful to all of the faculty, graduate students, and staffs of the department of Mathematics and Statistics at UNR iv TABLE OF CONTENTS Abstract i Acknowledgments iii List of Tables vii List of Figures viii Chapter Introduction Point Estimation 2.1 Maximum Likelihood Estimation 2.2 The Method of Moments 2.3 Bayesian Approach 11 Maximum Likelihood Estimation for Cauchy Distribution 15 A New Approach to Point Estimation Based on Maximum Likelihood 20 4.1 Description of the new method 20 4.2 Bayesian interpretation 22 4.3 The case of uniform distribution 24 Examples and Simulation Studies 27 5.1 Exponential distribution 27 5.2 Cauchy distribution 33 5.3 Continuous Pareto distribution 43 Appendices v A R coding which we used in this thesis A.1 58 Finding the estimated values and MSEs of the estimated values for the exponential distribution (Table 5.1 was developed using the following R coding) A.2 59 Drawing the Q-Q plot for estimated values using MLE for exponential distribution (One part of the Figure 5.1 was developed using the following R coding) A.3 Drawing the Q-Q plot using new method (One part of the Figure 5.1 was developed using the following R coding) A.4 60 60 Drawing boxplot of the estimated values using MLE for exponential distribution (One part of the Figure 5.2 was developed using the following R coding) A.5 61 Drawing boxplot for the estimated values using new method for exponential distribution (One part of the Figure 5.2 was developed using the following R coding) A.6 62 Drawing boxplot of the estimated sigma values using new method for Cauchy distribution (One part of the Figure 5.2 was developed using the following R coding) A.7 63 Finding the estimated values of theta and sigma for the Cauchy distribution (Table 5.2 was partially developed using the following R coding) 64 vi A.8 Finding the estimated values of sigma, MSE and develop a boxplot for the Cauchy distribution (Table 5.2 and Figure 5.5 were partially developed using the following R coding) A.9 65 Drawing a histogram for the estimated values of sigma for the Cauchy distribution (Figure 5.4 was partially developed using the following R coding) 66 A.10 Finding the estimated values of beta, MSE and Drawing a boxplot of estimated values of beta for the continuous Pareto distribution (Table 5.4 and Figure 5.7 were partially developed using the following R coding) 67 A.11 Drawing a the histogram for the estimated values of beta for the continuous Pareto distribution (Figure 5.6 were partially developed using the following R coding) 68 A.12 Drawing the Q-Q plot for the estimated values of beta for the continuous Pareto distribution (Figure 5.8 were partially developed using the following R coding) 69 55 normally distributed for the new method Case : Both parameters α and σ are unknown Table 5.5: Estimated parameters for continuous Pareto distribution when both parameters are unknown Sample size 50 50 50 50 500 500 500 1000 1000 1000 1000 σ(True) 2 2 2 2 2 α(True) 3 3 3 3 3 Input α 10 10 10 10 10 10 10 10 10 10 10 # iteration 100 500 10 500 10 100 500 σ ˆ 0.9546541 2.035922 2.036456 2.036456 0.9551209 1.847418 1.847582 0.9518466 2.090154 2.092593 2.092593 α ˆ 4.481546 2.62648 2.625832 2.625832 4.555328 2.978288 2.978113 4.693975 3.015114 3.013269 3.013269 Table 5.5 was developed as follows: We generate a sample of size n with true parameters, α = and σ = Then we develop a coding where we use the above sample to estimate the parameters, α and σ using k iterations To obtain the estimates of both parameters, we first set an initial value of α, and then to estimate the value of σ, and then we use that estimated value of σ to estimate the value of α And again, we use the estimated value of α to estimate the value of σ, and then we continue this process k times From Table 5.5 we see that when both parameters, α and σ, are unknown, then the simulation study shows that estimated values of the parameters converge to the true values of the parameters regardless how we choose the initial value of α 56 REFERENCES [1] G Haas, L Bain, and C Antle Inferences for the Cauchy distribution based on maximum likelihood estimators, Biometrika, 57, 403-408 (1970) [2] J A Reeds Asymptotic number of roots of Cauchy location likelihood equations, The Annals of Statistics, Vol 13, No 2, 775-784 (1985) [3] P McCullagh Mobius transformation and Cauchy parmeter estimation, Annals of Statistics, Vol 24, No 2, 787-808 (1996) [4] M H DeGroot Probability and Statistics, Second Edition, Addison-Wesley (1986) [5] N L Johnson, S Kotz and N Balakrishnan Continuous Univariate Distributions, Second Edition, Wiley and Sons (1994) [6] T S Ferguson Maximum likelihood estimates of the parameters of the Cauchy distribution for sample of size and 4, Journal of American Statistical Association, 73, 211-213 (1978) [7] T J Kozubowski, Lecture Notes for Stat 754, Mathematical Statistics, University of Nevada, Reno (2014) [8] V D Barnett Evaluation of the maximum likelihood estimation when the like- 57 lihood equation has multiple roots, Biometrika, 53, 151-165 (1966) [9] X L Meng and D B Rubin Maximum likelihood estimation via the ECM algorithm: A general framework, Biometrika, 80, 2, 267-78 (1993) 58 59 APPENDIX A R coding which we used in this thesis A.1 Finding the estimated values and MSEs of the estimated values for the exponential distribution (Table 5.1 was developed using the following R coding) 60 A.2 Drawing the Q-Q plot for estimated values using MLE for exponential distribution (One part of the Figure 5.1 was developed using the following R coding) A.3 Drawing the Q-Q plot using new method (One part of the Figure 5.1 was developed using the following R coding) 61 A.4 Drawing boxplot of the estimated values using MLE for exponential distribution (One part of the Figure 5.2 was developed using the following R coding) 62 A.5 Drawing boxplot for the estimated values using new method for exponential distribution (One part of the Figure 5.2 was developed using the following R coding) 63 A.6 Drawing boxplot of the estimated sigma values using new method for Cauchy distribution (One part of the Figure 5.2 was developed using the following R coding) 64 A.7 Finding the estimated values of theta and sigma for the Cauchy distribution (Table 5.2 was partially developed using the following R coding) 65 A.8 Finding the estimated values of sigma, MSE and develop a boxplot for the Cauchy distribution (Table 5.2 and Figure 5.5 were partially developed using the following R coding) 66 A.9 Drawing a histogram for the estimated values of sigma for the Cauchy distribution (Figure 5.4 was partially developed using the following R coding) 67 A.10 Finding the estimated values of beta, MSE and Drawing a boxplot of estimated values of beta for the continuous Pareto distribution (Table 5.4 and Figure 5.7 were partially developed using the following R coding) 68 A.11 Drawing a the histogram for the estimated values of beta for the continuous Pareto distribution (Figure 5.6 were partially developed using the following R coding) 69 A.12 Drawing the Q-Q plot for the estimated values of beta for the continuous Pareto distribution (Figure 5.8 were partially developed using the following R coding) View publication stats