Oceanographic Database, managing oceanographic data with GeoServer and GeoBatch

Dear All,
in this post we'd like to introduce some work we have done recently at GeoSolutions towards an applications that is able to ingest in real-time, manage, visualize, retrieve and chart data acquired by oceanographic and meteorological sensors (like sensors mounted onboard underwater gliders as an instance) over time. We called this application, Oceanographic Database.

The infrastructure we have put together is shown in the picture below.

Summarising, we can identify the various components and their roles as follows:

• Data is ingested via GeoBatch in near real-time with proper prepocessing
• GeoServer is used in conjunction with some custom REST extensions for performinng visualization, dissemination and export to Google Earth
• A REST framework wrapping on an open source charting library is used to generate charts on the fly from the ingested data.

In the following videos we are going to describe the main functionalities of the infrastructure.

The first video below shows the basics of the front-ed as well as the following functionalities:

• basic and advanced filtering for visualization
• creation of charts from ingested data
• search and download of data
• export in Google Earth with support for temporal animation and underwater visualization

The second video below covers the extensive charting functionalities.The O&M Database is capable of plotting 2D charts  of the acquired geophysical parameters like:

• Temporal evolutions of a certain parameter
• Evolution of of a certain parameter over a depth range
• Correlation of two parameters in a certain spatiotemporal range
• Scatter diagram of a parameter with JET colormap

Going into details, the O&M Database allows users to filter data and then move onto the Diagrams Panel for selecting the parameters to plot as well as the type of diagram, Notice that charts are plotted in real time using the live data as the are stored in the sytem. This makes the tool a perfect mean to monitor the acquisition of data since very refined query can be used to plot chats of specific subsets of the data.

The third video below shows the O&M Database Data Download capability, which allows users to search for  the original data and then to download them in the original format. As an instance we can search for Glider's data acquisitions and download them as netCDF files.




The forth video below shows the experimental Temporal Animation Capability of the O&M Database that allows users to perform simple temporal animations to visualize the temporal evolution of the acquisitions on the map.

The application is entirely built exploiting Open Source frameworks and libraries. Our intent is to make it availabe to our customers as part of our Enterprise Support Plans as Open Source under the GPL license (the proper license is still under evaluation).

Are you curious to know more about the features of the Oceanographic Database? Interested in knowing how we can help your organization reach your goal? Contact us!

The GeoSolutions team,

Preview: GeoRepository, advanced authorization manager for GeoServer

Dear All,
in this post we would like to introduce an application which we have been working lately here at GeoSolutions in order to cope with the requirements coming from enterprise deployments in terms of Authentication and Authorization for GeoServer, which we have called GeoRepository since we want to make it grow even further in the near future.

Georepository short introduction

View more presentations from GeoSolutions

Whoever has been facing the technical problems involved with deploying GeoServer inside an enterprise infrastructure following the SOA knows in such context Authorization and Authentication capabilities are usually provided by external services in order to centralize A&A services, fostering reuse of existing capabilities, minimize risk of duplication of functionalities and as well as to avoid duplication of users' credentials. Moreover in such context Single Sign On (SSO) services are usually a must-have.

GeoServer comes with an integrated security subsystem that is able to handle both Authentication and Authorization which is quite powerful and customizable, however we needed even to extend it further in order to be able to provide fine grained control over users' authorizations as well as in order to better integrate with existing A&A services.

A typical and yet simple usage scenario for GeoRepository is depicted here below.

In these setups GeoServer is normally not exposed directly, there is normally a load balancer as well as an Apache configured to perform duties such as single sign on authentication, SSL encryption and connection with external monitoring tools such as Nagios or Hyperic.
As suggested the authentication is performed by Apache HTTPD via LDAP; GeoServer will therefore be responsible only for the authorization based on the current user. In order to do this, GeoServer communicates with GeoRepository in order to retrieve users' authorizations query limits (we will get to this later) and then applies them accordingly to the incoming requests (thus avoiding post-encoding filtering, see also this white paper comparing internal vs external security systems).

In order to make this approach possible GeoServer was modified by Andrea Aime, the author of the existing Security Subsytem as described in the picture below. GeoRepository obtains users' credentials from LDAP and allows administrator to decorate them with additional parameters which are specific for restricting GeoServer access to services and data (we will describe them shortly).

Now that we have seen the 10000 feet view let's delve into details of how us know provide additional details on how GeoRepository seamlessly applies specific geospatial access policies to one or more GeoServer instances

The possible types of authorizations that can be applied using GeoRepository are controlled via generic rules with priorities , are shown below.

Each rule comprises of:

• User
• Profile
• Instance (since GeoRepository can control more than one instance of GeoServer, the typical enterprise scenario having a cluster of machines serving geospatial requests)
• Service and Request which can be used to control grants on OGC services
• Workspace which can be used to control grants on GeoServer workspaces
• Layer which can be used to control grants on GeoServer layers
• Grant
• • ALLOW grants access if the rule is satisfied, DENY denies access if the rule is satisfied
  • LIMIT applies only when we have spatial restrictions available, in this case if the ruse is satisfied we basically perform an allow with spatial restrictions.

For ALLOW and LIMIT grants we can, under some conditions, set also additional limits which are called details.

Available detail levels are:

• Layer Details allows us to specify refined rules for a specific GeoServer layer. As an instance it is possible to override the default style or to filter the available styles' list. It is also possible to specify CQL filters for reading and/or writing (when possible) GeoServer layers' attributes.
• Layer Attributes Details allows us to limit access to vector layer attributes. By default all attributes are readable/writable, but with this capability we can hide some of them or make them read only.

It is worth to point out that GeoRepository interprets rule by priority: higher priority rules (represented by a smaller integer number) are applied before rules at lower priority. Like in IPTABLES, the first match is considered valid and then executed.

The possibilities introduced by having GeoRepository work right next to GeoServer are quite a lot, let us list some of them:

• Fine grain control over which services, layers, workspaces, operations (of a service) a user can access
• Fine grain control over which geospatial area for a layer (being it raster or vector) a user can access
• Fine grain control over the WMS style that a use can apply for a layer with the possibility to perform style substitution on a per request basis depending on the credentials received
• Fine grain control over which attribute (for vector layers) or bands (for raster layers) a user can access (this way it is possible to hide/show sensible attributes depending on credentials)

As an example of full versus limited access to resources, check the difference in the following two layers previews. First one show administrator's preview, second one shows unprivileged user's preview, where some the attributes are hidden, the default style is forced by the access control and the geometries outside the restriction area have been .

We should set up an inline demo soon, meanwhile, if you want to know more about the features we currently support or about our roadmap, please, do not hesitate to contact us.

Regards,
the GeoSolutions Team

ImageI/O-Ext 1.1.1 Released

Dear all,
GeoSolutions is pleased to announce the ImageI/O-Ext 1.1.1 release. With respect to 1.1.0, it adds support for the IDRISI Raster Format. This format should be soon be available in GeoTools and GeoServer trunk.
Release artifacts have been deployed on the GeoSolutions maven repository, as well as on the OSGEO one.

Regards,
the GeoSolutions Team

Developer corner: taming a massive style and get a performance speedup in the process

Hi all,
recently a customer contacted us with an interesting problem: how do I display a large dataset fast, with the requirement that we want every feature with a different color (unique color approach)?

I cannot show you the original dataset, but let's consider Natural Earth level 1 administrative subdivision: all provinces in the world. And let's say we want to give each of them a unique color, considering the are 3791 of them.

The common SLD approach would make for a Rule with filter, and symbolizer, for every single one of them. Assuming 15 lines of SLD per rule, that would make for a SLD with roughly 57000 lines of XML. Ugh! So large that nothing will allow the upload of it... double ugh!
As icing on the cake, for each feature on average 1900 rules would have to be evaluated in order to find the right color. Are you ready to way minutes for you map to be drawn?

Enter SE 1.1 and the recode function. Recode allows a compact representation of a style in which one attribute value is to be associated to a particular color, or width, in a very compact form.
Here is a portion of the SLD for the example at hand:


The full SLD can be downloaded here. It's around 7600 lines, still quite big, but remember it condenses in a compact form over 3700 rules.

Now, the function surely makes the SLD smaller, but was still quite slow. So we spent some time optimizing it for the common case where all the values and colors are static values, getting a tremendous speedup.

How fast is it now? Brace yourself: by the naked eye, it's as fast as the plain style that assigns the same color to all the polygons, the style that has just a single rule.


A larger version of the same map is available here. This large version took half a second to generate.

We hope that similar optimizations can be applied, in the future, to other SE useful functions, such as Categorize and Interpolate. Interested in the topic? Let us know!

Regards,
the GeoSolutions Team.

Developer's Corner: GeoNetwork-Manager, super-simple Java client library for GeoNetwork

Dear All,
we just wanted to spend a few words about a Java library we have just released under the MIT license for talking to the GeoNetwork Open Source Project. We have named this library GeoNetwork-Manager (once again, a lot of brainstorming behind that name, really...).

Basically the goal of this library is to provide a simple yet usable way to interface a Java application with the GeoNetwork that would actually ease the work of the developers hiding away from them the relative complexity of theCSW protocol without introducing a complex communication layer.

Here you have some basic facts about the library:

• minimum set of external dependencies, namely apache-http-common and jdom
• dead-easy API, see examples here
• works with GeoNetwork 2.1 and 2.0.X
• allows to performa basic operations:
• Insert
• Reset Privileges
• Delete

GeoNetwork-Manager is open source and licenced under the MIT License. This library draws its essence from the efforts endured inside GeoSolutions in various projects. Partial funding has been provided by the BRISEIDE project..

Our intent is to keep this library lean and mean, therefore we do not intend to add complex stuff like a de/deserializable entity model into it, since we want to use it as a simple yet powerful communication layer with the GeoNetwork. 

If you have questions or if you just want to talk to us about the using the library in your project, please, subscribe to the mailing list here. In any case, do not hesitate to contact us.

Regards,
the GeoSolutions Team.

GeoSolutions @ FOSS4G 2011

Dear All,
it looks like we will have some busy times at FOSS4G this year.
If you check the schedule here you can see a few interesting talks that we will give, let's group them by topic:

GeoServer

• The State of GeoServer
• GeoServer on steroids
• Advanced cartographic map rendering in GeoServer
• Raster data in GeoServer and GeoTools: Achievements, issues and future developments
• The status of GeoServer WPS

GeoTools

• Raster data in GeoServer and GeoTools: Achievements, issues and future developments
• Status of the GeoTools project

JAITools

• jai-tools: advances in Java image processing

Well, it is encouraging to see that people are interested in what we are doing, all the effort we put into doing Open Source software is producing value for the larger community.

Are you curious to know more about the latest developments we have done? Interested in knowing how we can help your organization reach your goals? Contact us!

The GeoSolutions team,