2.4 Combinations of SMC and MCMC
As the theoretical structures for the main Monte Carlo methods (e.g.SMC, MCMC) are implemented gradually by researchers, we learn that every method has its short- comings and the common way to overcome these shortcomings is adjusting the method case by case. With little effort, one can follow the foundation of the original method to get similar theoretical conclusions for the customized Monte Carlo method. In this thesis, we only introduce two types of extensions, the sequential Monte Carlo sam- plers (SMC samplers) method and the particle Markov chain Monte Carlo (PMCMC) method, which are both, in a different manner, the combination of SMC and MCMC.
2.4.1 SMC Samplers
SMC samplers is a Monte Carlo method which involves Markov kernels within an SMC algorithm, it can potentially benefit from both the resampling procedure of SMC and the exploration spirit of Markov moves. The study of SMC samplers arises in [22], the researchers then use SMC samplers to draw from a fixed target density, where a tempered and related sequence of probabilities are defined as a bridge and MCMC kernels are considered. Later, [23] introduces an adaptive version of SMC samplers method in the field of approximate Bayesian computation, to automatically choose parameters and control the computational complexity. [45] then applies an adaptive SMC to do inference for Levy-Driven stochastic volatility models; and [34] provides
many empirical results to show the great performance of adaptive SMC algorithm in sampling from posterior distributions of parameters. Moreover, the convergence of adaptive SMC samplers are discussed in [5]. In Section 4.2 of this thesis, the framework of adaptive SMC samplers is used and its superiority is shown there in approximating permanents.
Compared with general SMC algorithms, MCMC kernels are often involved in the SMC samplers algorithms to efficiently move particles within the state space. At time n(≥ 2), SMC samplers algorithm proposes to obtain X(i)n using a special kind of importance density, a Markov kernel Kn,ξn of invariant distribution πn, where the parameters{ξn}are user-specified. Depending on the basis of general SMC Algorithm 2.2, the SMC samplers algorithm is described in Algorithm 2.7.
Algorithm 2.7 SMC Sampler For i∈ {1,2, . . . , N},
(1) At time 1,
(a) Sample X(i)1 from η1(x1) and compute the weights w1(X(i)1 ) = γ1(X
(i) 1 ) η1(X(i)1 ), standardized weights W1(i) and ESS.
(b) If ESS < NT, resample N weighted samples {X(i)1 }1≤i≤N to obtain N perfect samples {X(i)1 }1≤i≤N of π1(x1), then set X(i)1 =X(i)1 , w1(X(i)1 ) = 1 and W1i = N1.
(2) At timen ≥2,
(a) Sample X(i)n from the Markov kernel Kn,ξn(X(i)n−1,ã) and compute the weightswn(X(i)n ) =wn−1(X(i)n−1) γn(xn|X
(i) n−1)
Kn,ξn(X(i)n−1,X(i)n ), standardized weightsWn(i)
and ESS.
(b) If ESS < NT, resample N weighted samples {X(i)n }1≤i≤N to obtain N perfect samples{X(i)n }1≤i≤N of πn(xn), then setX(i)n =X(i)n , wn(X(i)n ) = 1 and Wni = N1.
2.4 Combinations of SMC and MCMC
One of the drawbacks of the above SMC samplers method is the fact that the pa- rameters{ξn}are critical but can not be set straightforwardly. A possible improvement is using the adaptive MCMC techniques given in [3] to update the parameters, which brings the adaptive version of SMC samplers in [45, 23, 34, 5]. The idea is one can adaptively determine the parameter ξn of the Markov kernel Kn,ξn by exploiting the information contained in the simulated samples {X(1)n−1, . . . ,X(N)n−1}. Namely, at time n, we adaptively sample X(i)n from Markov kernel Kn,ξ
n(X(1)n−1,...,X(N)n−1). For example, in the most general proposal, the normal random walk Metropolis moves, for parameters ξn, we approximate the mean and the variance of the marginal at time n−1, and use this variance as parameters ξn for timen. This is reasonable since that if πn is similar to πn−1, the variance at time n−1 can provide sensible scaling for time n. We now describe the adaptive SMC samplers algorithm in Algorithm 2.8.
Algorithm 2.8 Adaptive SMC Samplers For i∈ {1,2, . . . , N},
(1) At time 1,
(a) Sample X(i)1 from η1(x1) and compute the weights w1(X(i)1 ) = γ1(X
(i) 1 ) η1(X(i)1 ), standardized weights W1(i) and ESS.
(b) If ESS < NT, resample N weighted samples {X(i)1 }1≤i≤N to obtain N perfect samples {X(i)1 }1≤i≤N of π1(x1), then set X(i)1 =X(i)1 , w1(X(i)1 ) = 1 and W1i = N1.
(2) At timen ≥2,
(a) SampleX(i)n from the Markov kernelKn,ξ
n(X(1)n−1,...,X(N)n−1)(X(i)n−1,ã) and com- pute the weights wn(X(i)n ) =wn−1(X(i)n−1) γn(xn|X
(i) n−1) Kn,ξn(X(1)
n−1,...,X(N)
n−1)(X(i)n−1,X(i)n ), stan- dardized weights Wn(i) and ESS.
(b) If ESS < NT, resample N weighted samples {X(i)n }1≤i≤N to obtain N perfect samples{X(i)n }1≤i≤N of πn(xn), then setX(i)n =X(i)n , wn(X(i)n ) = 1 and Wni = N1.
Note that for both the SMC samplers Algorithm 2.7 and the adaptive SMC sam- plers Algorithm 2.8, they have the same expressions for approximating πn and Zn as the general SMC Algorithm 2.2, which are given in (2.1.19) and (2.1.21) (or (2.1.22)).
In [22], an extension of the SMC samplers algorithm is considered to draw from a fixed target density. By using a similar tempering procedure, the adaptive SMC samplers method described above can also be further extended to simulate from a fixed target density, but in a more effective way. A series of densities, which are related to the target density in the manner of tempered parameters, are considered as a bridge to the target density. The extension of the above adaptive SMC samplers is that one considers an additional layer of adaptivity for the tempered parameters, then follow the procedures of adaptive SMC samplers algorithm. More details are discussed in [45, 23, 5]
2.4.2 Particle MCMC
Particle Markov chain Monte Carlo (PMCMC) methods are a class of MCMC algorithms, which involve SMC approximations in the proposals of traditional M-H updates. PMCMC is usually used in Bayesian inference for sampling from posterior densities of parameters ([1, 85]). Suppose we are interested in sampling from a posterior density of parameter θ given thatyn is the observed data: π(θ|yn). Given the current position θ0 and a proposed position θ?, the traditional MCMC move will accept the
2.4 Combinations of SMC and MCMC
proposed position θ? with probability
1∧R(θ0, θ?)
where
R(θ0, θ?) = π(θ?|yn)ãq(θ0|θ?) π(θ0|yn)ãq(θ?|θ0)
is known as the acceptance ratio and q(ã|ã) is the transition density. Whilst it is often impossible to compute the posterior density and according to Bayes formula, π(θ|yn) is proportional to the product of the marginal density of the data and the prior density of the parameter: p(yn|θ)×f(θ), hence the acceptance ratio is usually expressed as
R(θ0, θ?) = p(yn|θ?)ãf(θ?)ãq(θ0|θ?) p(yn|θ0)ãf(θ0)ãq(θ?|θ0)
However, there is also high chance that the marginal densityp(yn|θ) cannot be exactly calculated. In such cases, an SMC approximation to the marginal density is naturally considered in the above M-H update, denoted by ˆp(yn|θ), the acceptance ratio becomes
R(θˆ 0, θ?) = p(yˆ n|θ?)ãf(θ?)ãq(θ0|θ?) ˆ
p(yn|θ0)ãf(θ0)ãq(θ?|θ0) (2.4.1)
the above combined technique is known as the particle marginal Metropolis-Hastings method (PMMH). Note that PMMH is the most popular class of PMCMC algorithms and it is displayed in details in Algorithm 2.9 as a representative.
The ergodicity of the above PMCMC samplers is proved in [1, 85].
Algorithm 2.9 PMCMC algorithm (1) initialization,i= 0,
(a) set θ0 arbitrarily.
(b) run the SMC algorithm in Section 2.1 to obtain the estimate of the marginal density p(yn|θ0), denoted by ˆp(yn|θ0).
(2) for iteration i≥1,
(a) sample θ? fromq(ã|θ0).
(b) run the SMC algorithm in Section 2.1 to obtain the estimate of the marginal density p(yn|θ?), denoted by ˆp(yn|θ?).
(c) with probability
1∧R(θˆ 0, θ?) (listed in (2.4.1)) set θi =θ?; otherwise set θi =θi−1.