Ensemble Kalman Filters Part 1: The basics Peter Jan van Leeuwen Data Assimilation Research Centre DARC University of Reading p.j.vanleeuwen@reading.ac.uk
Model: 10 9 unknowns P[u(x1),u(x2),T(x3),.. Observations: about 10 8 Satellites: only sea surface u(x1) In situ: irregular in space and time u(x2) T(x3)
The basics: probability density func;ons P(u) 0.5 u (m/s) 1.0
Data assimila;on: general formula;on Bayes theorem: Solu;on is pdf! NO INVERSION!!!
Propaga;on of pdf in ;me: Kolmogorov s equa;on Model equation: Pdf evolution: Kolmogorov s equation (Fokker-Planck equation) advection diffusion
Making the Gaussian approxima;on If we make the Gaussian approximation we only have to propagate the mean and the covariance. If the state dimension is 10 9, the covariance has of order 10 18 elements. Too large to store And we need to propagate the mean, noting that
Mo;va;on ensemble methods: EfDicient propagation of pdf in time (for nonlinear models)
Ensemble methods Filters: - Ensemble Kalman Dilter (Evensen, 1994; Burgers et al., 1998) (Assumption of Gaussianity/linearity) - Particle Dilters (Gordon et al., 1993) (Not so) Smoothers - Ensemble Kalman smoother (Evensen and Van Leeuwen, 2000) - Particle smoothers (Gordon et al., 1993)
Ensemble Kalman Filter - Propagation of pdf in time via ensemble - At observation times assume that pdf s are Gaussian Use Gausianity in Bayes Hence
(Ensemble) Kalman Filter Complete the squares to Dind (only for linear H!!!): with influence region K the Kalman weighting Gain innovation These are the standard Kalman Dilter equations, the ensemble comes in via estimating the mean and the covariance from the ensemble.
A naive Ensemble Kalman filter 1 Assume at analysis time: Which can be rewritten as: The mean of this gives the correct Kalman- Dilter update. However, the posterior error covariance is too small:
A naïve Ensemble Kalman filter 2 The Kalman Dilter update for P reads: The naïve method misses the KRK T term because each ensemble member uses the same vector Ky..
Possible solu;ons: EnKF Use perturbed observation for each member: With drawn randomly from N(0,R). Disadvantages: - Introduction of extra noise into the method - Error covariance update correct in statistical sense only Advantage: - Extra noise useful for nonlinear systems
Ensemble Square- root filters (EnSRF) Define prior perturbation matrix: such that: Note that this is not unique, the prior covariance can also be : with:
The transform in which and or
Ensemble Transform Kalman Filter Use Shermann-Morrison-Woodbury to find Perform an SVD on the second part Finally, write and hence
The full ETKF
Ensemble Kalman Smoother (EnKS) The smoother solution is given by: Using the prior ensemble runs we simply have Adjoint equations and A very cheap smoother!
Conclusions Ensemble Kalman Filters well established Square- root Dilters with mean- preserving random rotation of transform matrix seems most efdicient for small ensemble sizes But ad- hoc tricks are necessary to make them work for large- dimensional systems Ensemble Kalman smoothers can be obtained extremely efdicient Nonlinear observation operators can be handled by outer loops
However, model pdfs are non- Gaussian Probability density function of layer thickness of first layer at day 41 during data-assimilation 10 8 6 4 Kalman filter? Variational methods? 2 0 408 412 416 420 424 428 432 436 440 444 448 452 456
Sigma- point or Unscented Kalman filter I Julier and Uhlman, 1996 The idea is to choose the ensemble members symmetrically to optimally capture the covariances. Scaling factors are used to enhance the spread, but by taking these into account in the analysis no ad hoc Dixes are introduced. Generate N ensemble members (sigma points) with Alpha is the scaling factor that determines the spread.
Sigma- point or Unscented Kalman Filter II Integrate each sigma point forward with the nonlinear model equations At analysis time generate the mean and covariance as: Use the Kalman filter equations to generate the posterior, calculate new sigma points etc.
Alpha determines spread in the sigma points