D-Flow FM related information

Writing Kalman gain

Since OpenDA 3.1, the Kalman gain can be written fully according to the NetCDF CF format.

An example can be found in model_dflowfm_blackbox/tests/dcsmv5_kalman_rst/algorithm/Enkf.xml.

The Kalman gain is saved by configuring:

<saveGain>
    <times type="fixed" timeFormat="dateTimeString" >202201010100,202201010110,...,202401010400</times>
    <file dirPrefix="kgStorage_" fileName="KalmanGainStorage.nc" fileType="netcdf_cf"/>
</saveGain>

OpenDA will use the first two explicit times to determine the time step. It then fills all subsequent times according to this step until the end time, indicated after the dots. For each configured time, a directory is created with the prefix kgStorage_ and the time stamp according to the date format yyyyMMDDHHmm. In these directories, the file with the configured name KalmanGainStorage.nc is written.

For each noise term, one variable will be created in the KalmanGainStorage.nc. A two-dimensional (gridded) noise variable contains three dimensions: the first equals the number of observation stations of the state (station_dimension) and the other arguments are related to the noise dimensions. If we use the following configuration in model_dflowfm_blackbox/tests/dcsmv5_kalman_rst/stochModel/2DPressureNoise.xml:

<grid type="cartesian" coordinates="wgs84">
    <x>-12,-11,...,14</x>
    <y>48,49,...,63</y>
</grid>

then the following output is generated:

double 2DNoise(station_dimension=4, 2DNoise_dimension_0=16, 2DNoise_dimension_1=27)

where 2DNoise_dimension_0 is related to the \(y\)-coordinate, and 2DNoise_dimension_1 is related to the \(x\)-coordinate.

For the state variables s1 and unorm (coming from the *_rst.nc restart file, related to water levels and currents), the first dimension is again the station dimension of the state and the second dimension is related to the number of points where model results are compared to observations (number of predictors):

double s1(station_dimension=4, s1_dimension_0=17843);
double unorm(station_dimension=4, s1_dimension_0=17843);

These lines are included in the output when writing the Kalman gain.

Reading Kalman gain

When a Kalman gain is available, like from a previous EnKF run, it can be used in a steady-state Kalman filter. An example can be found in model_dflowfm_blackbox/tests/dcsmv5_kalman_rst/SSKF.oda. In model_dflowfm_blackbox/tests/dcsmv5_kalman_rst/algorithm/SSKF.xml, a reference to a previously written Kalman gain is configured as follows:

<readGain>
    <dirPrefix>kgStorage_L100km_</dirPrefix>
    <time timeFormat="dateTimeString">202209011200</time>
    <file>KalmanGainStorage.nc</file>
</readGain>

With the above elements the reference to kgStorage_L100km_202209011200/KalmanGainStorage.nc is constructed and the Kalman gain read. This way, there is no need to run multiple ensembles in order to determine the Kalman gain and the model can just be run once.

Maps Noise Model

When Kalman Filtering is applied, we introduce a noise term. This two-dimensional noise is correlated in time and space (an auto-regressive order 1 (AR1) model). This noise term is added as an extra state variable (state augmentation) and is therefore updated in the analysis step.

In model_dflowfm_blackbox/tests/dcsmv5_kalman_rst/stochModel/dflowfmStochModel.xml, an example of applying a two-dimensional/grid-formatted noise model is given (in OpenDA, this type of noise model is called a mapsNoiseModel). It refers to model_dflowfm_blackbox/tests/dcsmv5_kalman_rst/stochModel/2DPressureNoise.xml for the definition of the noise model. The noise is applied to the model exchange item p which is based on the air-pressure variable p in the %meteofile_p% NetCDF file that is read and edited via the data object <dataObject className="org.openda.exchange.dataobjects.NetcdfDataObject"> in dflowfmWrapper.xml. This way, OpenDA manipulates the air-pressure field in order to correct the water levels in the model. In order to overwrite the forecast noise (initial noise) with the analysis noise (after the state update), the following options should be configured in StochModelConfig.xml (at the definition of the exchange item <noiseModel>): allowOverwriteForecastWithAnalysisNoise="true" and transformation="set". For the defintion of this transformation variable, we refer to the section on exchange item configurations.

This way, OpenDA will set the value of the calculated noise directly in the exchange item (and thus file) instead of adding it to or multiplying it with existing values. Because OpenDA stores the gridded-noise values as a plain one-dimensional array of values, some care must be taken with applying it to two-dimensional data structures (as there is no check on matching dimensions or matching coordinates). For this reason, the airpressure_noise.nc variable p has the following order of dimensions: double p(time=49, Y=16, X=27) and the \(y\)-coordinates are descending: 63.0, 62.0, …, 49.0, 48.0. See again model_dflowfm_blackbox/tests/dcsmv5_kalman_rst/stochModel/2DPressureNoise.xml.