Difference between revisions of "Nonlinear Filtering for Data Assimilation of SemiLagrangian Ocean Glider Data"
(→Outline/Timeline of Research: NOT DONE YET) 

Line 4:  Line 4:  
= Outline/Timeline of Research =  = Outline/Timeline of Research =  
+  
+  == 2/16/11 ==  
+  
+  === Info ===  
+  This version contains the following changes:  
+  * The parameter file and particle filter function now contain a field <code> strengths_known </code> to determine if the particle cloud knows the strengths of the vortices.  
+  * The parameter file and particle filter function now contains fields <code> alpha </code> and <code> beta </code> which determine the st. dev. of the initial distribution of vortex strengths in the particle cloud and the st. dev. of the perturbations on the vortex strengths during resampling.  
+  * The parameter file contains a field for the number of simulations to run with the given parameters.  
+  * Wrote a condor submission file to run multiple experiments on ''pere'' using condor.  
+  * Wrote a bash script wrapper/driver for the condor submission file to call. This script sets up a directory for each condor job to save its results.  
+  
+  === Notes ===  
+  * The file/function <code> g_strengths_known.m </code> should be renamed to <code> g_strengths_constant </code> since the function is used whenever the vortex strengths are assumed to be constant, but also used in both the case where the vortex strengths are known and unknown by the particle filter.  
+  
+  
+  === Experiments ===  
+  
+  { border = "1"  
+  !Job/Experiment Number  
+  !Description  
+    
+  0  
+  Strengths known, system parameters from the paper, tracer initial condition: 0.3  0.6i  
+    
+  1  
+  Strengths known, system parameters from the paper, tracer initial condition: 1  0.6i  
+    
+  2  
+  Strengths known, system parameters from the paper, tracer initial condition: 1  1i  
+    
+  3 and 30  
+  Strengths known, system parameters from the paper, tracer initial condition: 2.4  2.4i  
+    
+  4 and 40  
+  Strengths unknown, alpha = 0.1, beta = 0.02, system parameters from the paper, tracer initial condition: 0.3  0.6i  
+    
+  5 and 50  
+  Strengths unknown, alpha = 0.1, beta = 0.02, system parameters from the paper, tracer initial condition: 1  0.6i  
+    
+  6 and 60  
+  Strengths unknown, alpha = 0.1, beta = 0.02, system parameters from the paper, tracer initial condition: 1  1i  
+    
+  7 and 70  
+  Strengths unknown, alpha = 0.1, beta = 0.02, system parameters from the paper, tracer initial condition: 2.4  2.4i  
+    
+  8  
+  Strengths unknown, alpha = 0.01, beta = 0.02, system parameters from the paper, tracer initial condition: 0.3  0.6i  
+    
+  9  
+  Strengths unknown, alpha = 0.05, beta = 0.02, system parameters from the paper, tracer initial condition: 0.3  0.6i  
+    
+  10  
+  Strengths unknown, alpha = 0.1, beta = 0.02, system parameters from the paper, tracer initial condition: 0.3  0.6i  
+    
+  11  
+  Strengths unknown, alpha = 1, beta = 0.02, system parameters from the paper, tracer initial condition: 0.3  0.6i  
+    
+  }  
+  
+  
+  === Files ===  
+  
+  Directory <code> (amallen)~/research/2_16_11/ </code> and <code> 5007mallena@pere.marquette.edu:~/2_16_11 </code>  
+  
+  
+  { border = "1"  
+  !Filename  
+  !Description  
+    
+  <code> bash_experiment_script.m </code>  
+  Driver script for reading system/experiment parameters from the file <code>parameter_*.txt</code> and running a collection of simulations of the particle filter using the function <code> point_vortex_particle_filter.m</code>. Saves all the errors, the average error over each simulation, the average error across all simulations, and the number of failures (error in vortex position of greater than 1).  
+    
+  <code> point_vortex_particle_filter.m </code>  
+  Generic function for performing a single simulation using the particle filter. Vortex strengths can be known or unknown but are assumed to be constant in time (not time varying). Allows arbitrary number of observations, arbitrary time steps, arbitrary number of particles, arbitrary number of tracers, arbitrary number of vortices, and arbitrary strengths for vortices. Returns a time series of the errors between the weighted mean of the cloud and the true state of the system. Also, returns a time series of the variances of the weighted distributions of particles for each small time step. Also, returns the analysis (after observation updates) errors at the observation times.  
+    
+  <code> SRK4.m </code>  
+  Numerical RungeKutta integration for stochastic differential equations. Function <code> f.m </code> is the deterministic part. Function <code> g_strengths_known.m </code> is the stochastic part.  
+    
+  <code> f.m </code>  
+  Deterministic part of the stochastic differential equation describing the system used in <code> two_point_vortex_particle_filter.m </code>.  
+    
+  <code> g_strengths_known.m </code>  
+  Stochastic part of the stochastic differential equation describing the system used in <code> point_vortex_particle_filter.m </code>. The stochastic part of the change in vortex strengths is 0 because they are assumed to stay constant (though sometimes unknown).  
+    
+  <code> H.m </code>  
+  Observation operator. Here, it maps the entire state vector to a vector containing just the the tracer locations.  
+    
+  <code> R.m </code>  
+  Fitness function for determining weights of particles after comparison with an observation.  
+    
+  }  
+  
== 2/4/11 ==  == 2/4/11 ==  
Line 13:  Line 105:  
* The wrapper/driver scripts read parameters in from the files <code>parameter_*.txt</code>  * The wrapper/driver scripts read parameters in from the files <code>parameter_*.txt</code>  
−  Directory <code> ~/research/  +  Directory <code> ~/research/2_4_11/ </code> 
{ border = "1"  { border = "1"  
Line 62:  Line 154:  
=== Files ===  === Files ===  
−  Directory <code> ~/research/  +  Directory <code> ~/research/1_24_11/ </code> 
{ border = "1"  { border = "1"  
Line 103:  Line 195:  
=== Files ===  === Files ===  
−  Directory <code> ~/research/  +  Directory <code> ~/research/1_20_11/ </code> 
{ border = "1"  { border = "1" 
Latest revision as of 22:15, 18 February 2011
Contents
Nonlinear Filtering for Data Assimilation of SemiLagrangian Ocean Glider Data
The goal of this project is to develop efficient techniques for assimilating glider data into nonlinear ocean models. First, it will involve taking current particle filtering data assimilation techniques for 2D ocean models and scaling them to work on higher dimensional, more realistic, and more complex models—such as models with unknown time varying strengths of vortices—in a computationally feasible way. To start, these techniques will focus on assimilating data from tracers which only move with the flow, unlike gliders which can maneuver themselves. Then, the next task is to develop techniques for assimilating gliderlike data (instead of tracer data) into these models and utilizing modelbased prediction to help inform future flight paths of the glider. The final task will be to scale the ocean model to include a third spatial dimension and adapting the previous method to inform the glider's future flight path in three dimensions.
Outline/Timeline of Research
2/16/11
Info
This version contains the following changes:
 The parameter file and particle filter function now contain a field
strengths_known
to determine if the particle cloud knows the strengths of the vortices.  The parameter file and particle filter function now contains fields
alpha
andbeta
which determine the st. dev. of the initial distribution of vortex strengths in the particle cloud and the st. dev. of the perturbations on the vortex strengths during resampling.  The parameter file contains a field for the number of simulations to run with the given parameters.
 Wrote a condor submission file to run multiple experiments on pere using condor.
 Wrote a bash script wrapper/driver for the condor submission file to call. This script sets up a directory for each condor job to save its results.
Notes
 The file/function
g_strengths_known.m
should be renamed tog_strengths_constant
since the function is used whenever the vortex strengths are assumed to be constant, but also used in both the case where the vortex strengths are known and unknown by the particle filter.
Experiments
Job/Experiment Number  Description 

0  Strengths known, system parameters from the paper, tracer initial condition: 0.3  0.6i 
1  Strengths known, system parameters from the paper, tracer initial condition: 1  0.6i 
2  Strengths known, system parameters from the paper, tracer initial condition: 1  1i 
3 and 30  Strengths known, system parameters from the paper, tracer initial condition: 2.4  2.4i 
4 and 40  Strengths unknown, alpha = 0.1, beta = 0.02, system parameters from the paper, tracer initial condition: 0.3  0.6i 
5 and 50  Strengths unknown, alpha = 0.1, beta = 0.02, system parameters from the paper, tracer initial condition: 1  0.6i 
6 and 60  Strengths unknown, alpha = 0.1, beta = 0.02, system parameters from the paper, tracer initial condition: 1  1i 
7 and 70  Strengths unknown, alpha = 0.1, beta = 0.02, system parameters from the paper, tracer initial condition: 2.4  2.4i 
8  Strengths unknown, alpha = 0.01, beta = 0.02, system parameters from the paper, tracer initial condition: 0.3  0.6i 
9  Strengths unknown, alpha = 0.05, beta = 0.02, system parameters from the paper, tracer initial condition: 0.3  0.6i 
10  Strengths unknown, alpha = 0.1, beta = 0.02, system parameters from the paper, tracer initial condition: 0.3  0.6i 
11  Strengths unknown, alpha = 1, beta = 0.02, system parameters from the paper, tracer initial condition: 0.3  0.6i 
Files
Directory (amallen)~/research/2_16_11/
and 5007mallena@pere.marquette.edu:~/2_16_11
Filename  Description 

bash_experiment_script.m

Driver script for reading system/experiment parameters from the file parameter_*.txt and running a collection of simulations of the particle filter using the function point_vortex_particle_filter.m . Saves all the errors, the average error over each simulation, the average error across all simulations, and the number of failures (error in vortex position of greater than 1).

point_vortex_particle_filter.m

Generic function for performing a single simulation using the particle filter. Vortex strengths can be known or unknown but are assumed to be constant in time (not time varying). Allows arbitrary number of observations, arbitrary time steps, arbitrary number of particles, arbitrary number of tracers, arbitrary number of vortices, and arbitrary strengths for vortices. Returns a time series of the errors between the weighted mean of the cloud and the true state of the system. Also, returns a time series of the variances of the weighted distributions of particles for each small time step. Also, returns the analysis (after observation updates) errors at the observation times. 
SRK4.m

Numerical RungeKutta integration for stochastic differential equations. Function f.m is the deterministic part. Function g_strengths_known.m is the stochastic part.

f.m

Deterministic part of the stochastic differential equation describing the system used in two_point_vortex_particle_filter.m .

g_strengths_known.m

Stochastic part of the stochastic differential equation describing the system used in point_vortex_particle_filter.m . The stochastic part of the change in vortex strengths is 0 because they are assumed to stay constant (though sometimes unknown).

H.m

Observation operator. Here, it maps the entire state vector to a vector containing just the the tracer locations. 
R.m

Fitness function for determining weights of particles after comparison with an observation. 
2/4/11
Info
This version contains many changes:
 The particle filter accepts an arbitrary number of vortices and tracers.
 Rotated frame plots are only printed if there are exactly two vortices.
 The wrapper/driver scripts read parameters in from the files
parameter_*.txt
Directory ~/research/2_4_11/
Filename  Description 

experiment_script_three_vortices.m

Driver script for running a single test simulation of the particle filter with one tracer and three vortices. This script uses system parameters found in parameter_1.txt . The tracer starts at location 0.3 − 0.6i. The vortices all have a strength of 2π. The vortices start at locations 1, 1, and 0 (all along the real axis). All variables saved in workspace.mat . Number of failures (error in vortex position of greater than 1) is saved in N_f_1.mat , average errors for each simulation is saved in d_1.mat , and the overall average error for all simulations is saved in d_avg_1.mat .

experiment_script_two_tracers.m

Driver script for running a single test simulation of the particle filter with two tracers and two vortices. This script uses system parameters found in parameter_2.txt . The tracers start at locations 0.3 − 0.6i and 1  0.6i. The vortices both have a strength of 2π. All variables saved in workspace.mat . Number of failures (error in vortex position of greater than 1) is saved in N_f_2.mat , average errors for each simulation is saved in d_2.mat , and the overall average error for all simulations is saved in d_avg_2.mat .

point_vortex_particle_filter.m

Generic function for performing a single simulation using the particle filter. Vortex strengths are known. Allows arbitrary number of observations, arbitrary time steps, arbitrary number of particles, arbitrary number of tracers, arbitrary number of vortices, and arbitrary strengths for vortices. Returns a time series of the errors between the weighted mean of the cloud and the true state of the system. Also, returns a time series of the variances of the weighted distributions of particles for each small time step. 
SRK4.m

Numerical RungeKutta integration for stochastic differential equations. Function f.m is the deterministic part. Function g.m is the stochastic part.

f.m

Deterministic part of the stochastic differential equation describing the system used in two_point_vortex_particle_filter.m .

g.m

Stochastic part of the stochastic differential equation describing the system used in two_point_vortex_particle_filter.m .

H.m

Observation operator. Here, it maps the entire state vector to a two vector containing just the first two entries (the tracer location) of the state vector. 
R.m

Fitness function for determining weights of particles after comparison with an observation. 
1/25/11
Incomplete version. I am saving it though because I may want to come back and edit this code. The next version will be significantly different, and I want to be able to come back to this working version if I need to.
1/24/11
Info
The experiment_script.m
from this version is, again, a script that performs an experiment to look at how the particle filter performs with different noise parameters and time step size. I used time steps of Δt = 1.0, 1.5, 2.0, 3.0 and noise standard deviation σ = 0.02, 0.03, 0.05. The tracer started at the location 0.3 − 0.6i. The vortices both have a strength of 2π. The strengths of the vortices are known for this experiment.
Notes
 Strengths are known.
 Small step size of 1/200 is used for integration (instead of 1/100 which was used previously).
Files
Directory ~/research/1_24_11/
Filename  Description 

experiment_script.m

Driver script for running numerous simulations of the particle filter. This version runs the particle filter with time steps of Δt = 1.0, 1.5, 2.0, 3.0 and noise standard deviation of σ = 0.02, 0.03, 0.05. The tracer started at the location 0.3 − 0.6i. The vortices both have a strength of 2π. All variables saved in workspace.mat . Number of failures (error in vortex position of greater than 1) is saved in N_f.mat and average error is saved in d.mat .

two_point_vortex_particle_filter.m

Generic function for performing a single simulation using the particle filter. Vortex strengths are known. Allows arbitrary number of observations, arbitrary time steps, arbitrary number of particles, arbitrary strengths for vortices. Can only handle two vortices. Returns a time series of the errors between the weighted mean of the cloud and the true state of the system. Also, returns a time series of the variances of the weighted distributions of particles for each small time step. 
SRK4.m

Numerical RungeKutta integration for stochastic differential equations. Function f.m is the deterministic part. Function g.m is the stochastic part.

f.m

Deterministic part of the stochastic differential equation describing the system used in two_point_vortex_particle_filter.m .

g.m

Stochastic part of the stochastic differential equation describing the system used in two_point_vortex_particle_filter.m .

H.m

Observation operator. Here, it maps the entire state vector to a two vector containing just the first two entries (the tracer location) of the state vector. 
R.m

Fitness function for determining weights of particles after comparison with an observation. 
1/20/11
Info
The experiment_script.m
from this version is a script that performs an experiment to look at how the particle filter performs with different noise parameters and time step size. I used time steps of Δt = 1.0, 1.5, 2.0 and noise standard deviation σ = 0.02, 0.03, 0.05. The tracer started at the location 0.3 − 0.6i. The vortices both have a strength of 2π. Results are poor; they are much worse than those from Chris's paper using the same parameters. In my experiment the strengths of the vortices were unknown, but in Chris's they were known. Next, I will run the same experiment with the strengths known.
Notes
 Strengths were unknown, but in Chris's paper they were known.
 May need more particles. I used 400 (the same number used in Dr. Spiller's paper).
Files
Directory ~/research/1_20_11/
Filename  Description 

experiment_script.m

Driver script for running numerous simulations of the particle filter. This version runs the particle filter with time steps of Δt = 1.0, 1.5, 2.0 and noise standard deviation of σ = 0.02, 0.03, 0.05. The tracer started at the location 0.3 − 0.6i. The vortices both have a strength of 2π. All variables saved in workspace.mat . Number of failures (error in vortex position of greater than 1) is saved in N_f.mat and average error is saved in d.mat .

two_point_vortex_particle_filter.m

Generic function for performing a single simulation using the particle filter. Vortex strengths are unknown. Allows arbitrary number of observations, arbitrary time steps, arbitrary number of particles, arbitrary strengths for vortices. Can only handle two vortices. Returns a time series of the errors between the weighted mean of the cloud and the true state of the system. Also, returns a time series of the variances of the weighted distributions of particles for each small time step. 
SRK4.m

Numerical RungeKutta integration for stochastic differential equations. Function f.m is the deterministic part. Function g.m is the stochastic part.

f.m

Deterministic part of the stochastic differential equation describing the system used in two_point_vortex_particle_filter.m .

g.m

Stochastic part of the stochastic differential equation describing the system used in two_point_vortex_particle_filter.m .

H.m

Observation operator. Here, it maps the entire state vector to a two vector containing just the first two entries (the tracer location) of the state vector. 
R.m

Fitness function for determining weights of particles after comparison with an observation. 
1/19/11
Info
The main changes in this version are mostly just code clean ups. Previously, an entirely different cloud of particles (with different realizations of the random noise process used during integration) was used for the Lagrangian frame, instead of just appropriately rotating the cloud to get the Lagrangian frame. The particle filter now uses the same cloud for plotting the movement of the cloud in both frames, but simply rotates the axes through time according to the constant angular velocity of the rotation of the vortices. Also, now the particle filter now computes and records a time series of the variances of the weighted distributions of particles (for each state variable).
Notes
 The particle filter still only handles 2 vortices.
 I need to make a script to extract all the given/assumed system parameters used in the experiment and output them into a nice, neat, easily readable file.
Files
Directory ~/research/1_13_11/
Filename  Description 

experiment_script.m

Driver script for running numerous simulations of the particle filter. All variables saved in workspace.mat . Error time series for the weighted mean of the particle cloud are saved in errors.mat . Time series of the variances of the distributions of particles are saved in variances.mat .

two_point_vortex_particle_filter.m

Generic function for performing a single simulation using the particle filter. Vortex strengths are unknown. Allows arbitrary number of observations, arbitrary time steps, arbitrary number of particles, arbitrary strengths for vortices. Can only handle two vortices. Returns a time series of the errors between the weighted mean of the cloud and the true state of the system. Also, returns a time series of the variances of the weighted distributions of particles for each small time step. 
SRK4.m

Numerical RungeKutta integration for stochastic differential equations. Function f.m is the deterministic part. Function g.m is the stochastic part.

f.m

Deterministic part of the stochastic differential equation describing the system used in two_point_vortex_particle_filter.m .

g.m

Stochastic part of the stochastic differential equation describing the system used in two_point_vortex_particle_filter.m .

H.m

Observation operator. Here, it maps the entire state vector to a two vector containing just the first two entries (the tracer location) of the state vector. 
R.m

Fitness function for determining weights of particles after comparison with an observation. 
1/13/11
Info
This version marks the start of Spring 2011 research. I have turned the particle filter script into a generic function that takes the system and filter parameters as arguments. It is restricted to just two vortices, but allows for different strengths. Also, the vortex strengths are unknown to the particle filter. I also have included the experiment_script.m
script which runs numerous simulations of the particle filter. The resulting error time series are saved in errors.mat
and all the variables are saved in workspace.mat
.
Notes
 The variances for the initial distribution of particles is still hardcoded into
two_point_vortex_particle_filter.m
instead of being arguments.  The variance (actually st. dev) of the stochastic part of the differential equation is hardcoded into
g.m
instead of being an argument to the particle filter. I'm not sure which is more appropriate, so I'm keeping it ing.m
for now.
Files
Directory ~/research/1_13_11/
Filename  Description 

experiment_script.m

Driver script for running numerous simulations of the particle filter. All variables saved in workspace.mat . Error timeseries for the weighted mean of the particle cloud are saved in errors.mat .

two_point_vortex_particle_filter.m

Generic function for performing a single simulation using the particle filter. Vortex strengths are unknown. Allows arbitrary number of observations, arbitrary time steps, arbitrary number of particles, arbitrary strengths for vortices. Can only handle two vortices. Returns a time series of the errors between the weighted mean of the cloud and the true state of the system. 
SRK4.m

Numerical RungeKutta integration for stochastic differential equations. Function f.m is the deterministic part. Function g.m is the stochastic part.

f.m

Deterministic part of the stochastic differential equation describing the system used in two_point_vortex_particle_filter.m .

g.m

Stochastic part of the stochastic differential equation describing the system used in two_point_vortex_particle_filter.m .

H.m

Observation operator. Here, it maps the entire state vector to a two vector containing just the first two entries (the tracer location) of the state vector. 
R.m

Fitness function for determining weights of particles after comparison with an observation. 
Fall 2010
Info
This directory contains all files used in Fall 2010.
 It contains a script for a particle filter with resampling applied to the two point vortex system with equal and known vortex strengths.
 It contains a script for a particle filter with resampling applied to the two point vortex system with different, but known, vortex strengths.
 It contains a script for a particle filter with resampling applied to the two point vortex system with different and unknown vortex strengths.
 It contains a numerical RungeKutta integration routine for stochastic differential equations.
Files
Directory ~/research/Fall_2010