BRISEIDE Project, Geoserver, GeoNetwork and GeoBatch for the management of dynamic MetOc data

Dear All,
In this post I wanted to talk about the work we are doing for the BRISEIDE european project.

The ambitious aim of the project, as stated on its website, is the delivery of:

• time-aware extension of MetOc data models developed in the context of previous/ongoing EU INSPIRE related projects (e.g. in the context of GMES, eContentPlus)
• application (e.g. Civil Protection) based on the integration of existing, user operational information
• value added services for spatio-temporal data management, authoring, processing, analysis and interactive visualisation

Within the context of the project, GeoSolutions will work under the leadership of SinerGIS in order to provide near-real time ingestion, cataloging and publishing of meteorological data provided by the stakeholders to be used as inputs for running processes to perform, as an instance, fire propagation models in emergency situations.

The infrastructure we are setting up is depicted in the deployment diagram here below. Basic building blocks are as follow:

• GeoServer for providing WMS, WCS and WFS services with support for the TIME dimension. It is worth to point out that it will also provide WPS capabilities.
• GeoNetwork, for publishing metadata for all data with specific customizations for managing the TIME dimensions in the dataset (we are going to biriefly describe them later on)
• GeoBatch, for performing preprocessing and ingestion in near real time of data and related metadata with minimal human intervention.

For this project we have customized the metadata indexing (thanks Lucene!) in GeoNetwork in order to be able to index meteorological model runs in terms of their run time as well as in term of their forecast times. Generally speaking the data we are dealing with is driven by a meterological model which produces daily a certain number of geophysical parameters with forecast valid for around 5 days, moreover some additional fire risk indexes are produced by processing these results in near real time. As days go by forecasts from different runs of the model are available as indicated in the picture below.

To achieve our gaols we have slightly customized the indexing configuration as well as the user interface in order to be able to make searches on run times and forecast times fast. If you are interested in having a look at one of the ISO metadata XML documents that we are publishing, here you can find an example here.

Here below you can find a diagram depicting the automatic ingestion flow we have created for the BRISEIDE project (actually, one of the few we have created) using the GeoBatch framework (we will soon release version 1.0 on which we are working in our own repository).

This flow makes extensive use of an orchestration Groovy script that implements specific business logic for the use case. Internally it also make use of various other atomic action for performing tasks like, publishing an ImageMosaic in GeoServer or its metadata in GeoNetwork.

Incoming files are composed by a compressed set of .asc (ascii images) files which are:

1. Converted into re-tiled GeoTiff images
2. Embedded overviews are added to each image

The groovy script produces an ImageMosaicCommand which is essentially an xml command which is sent to the ImageMosaicAction which:

1. Check for the layer existence on the target GeoServer
2. If success copy all the files to the target directory
3. Create the Store and the layer to contain the ImageMosaic
4. Configure the layer on the GeoServer using desired parameters
5. Produce an XML file with the ImageMosaic properties

The groovy script read the produced ImageMosaic output, enrich it with some other useful information then pass that object to the FreeMarkerAction which using a template and the passed data model will produce the xml metadata file (as described above).

The GroovyScriptAction ends sending this file to the next action which is the GeoNetworkAction which will send the metadata to the target GeoNetwork server using the desired (specified by configuration) options.

If you have questions about the work described in this post or if you want to know more about our services could help your organization to reach its goals, do not hesitate to contact us.

Regards,
the GeoSolutions Team.