Enrico Fermi (1901–1954) took great delight in astonishing his.colleagues with his remarkably accurate predictions of experimental results. . . his “guesses” were really derived from the statistical sampling techniques that he used to calculate with whenever insomnia struck
Monte Carlo Inference Methods Iain Murray University of Edinburgh http://iainmurray.net Monte Carlo and Insomnia Enrico Fermi (1901–1954) took great delight in astonishing his colleagues with his remarkably accurate predictions of experimental results his “guesses” were really derived from the statistical sampling techniques that he used to calculate with whenever insomnia struck! —The beginning of the Monte Carlo method, N Metropolis Overview Gaining insight from random samples Inference / Computation What does my data imply? What is still uncertain? Sampling methods: Importance, Rejection, Metropolis–Hastings, Gibbs, Slice Practical issues / Debugging Linear regression y = θ1 x + θ2 , p(θ) = N (θ; 0, 0.42I) y -2 -4 Prior p(θ) -6 -2 x Linear regression y (n) = θ1x(n) + θ2 + (n) (n) , ∼ N (0, 0.12) y -2 -4 -6 p(θ | Data) ∝ p(Data | θ) p(θ) -2 x Linear regression (zoomed in) y -0.5 -1 -1.5 p(θ | Data) ∝ p(Data | θ) p(θ) -2 x Model mismatch −2 −4 −6 −2 What will Bayesian linear regression do? Quiz Given a (wrong) linear assumption, which explanations are typical of the posterior distribution? 2 0 −2 −2 −2 −4 −3 A −2 −1 −4 −3 B −2 D All of the above E None of the above Z Not sure −4 −1 −3 C −2 −1 ‘Underfitting’ −2 −4 −6 −4 −2 Posterior very certain despite blatant misfit Peaked around least bad option Roadmap — Looking at samples — Monte Carlo computations — How to actually get the samples ... beginning of the Monte Carlo method, N Metropolis Overview Gaining insight from random samples Inference / Computation What does my data imply? What is still uncertain? Sampling methods: Importance,... Peaked around least bad option Roadmap — Looking at samples — Monte Carlo computations — How to actually get the samples Simple Monte Carlo Integration f (θ) π(θ) dθ = “average over π of f ” ≈ S.. .Monte Carlo and Insomnia Enrico Fermi (1901–1954) took great delight in astonishing his colleagues