GeoSolutions presentation @ OSGIS 2010

GeoSolutions will be present at OSGIS 2010, the Open Source GIS UK (OSGIS) Conference, which will be held on 21-22 June at the Centre for Geospatial Science University of Nottingham.

Ing. Simone Giannecchini, GeoSolutions' founder as well as GeoServer PSC and GeoTools PMC member, will give a presentation titled "Using GeoServer for interoperable management and dissemination of geospatial data", which will provide a quick introduction on the GeoServer geospatial framework.

If you are attending the conference and are interested in knowing more about GeoServer or GeoSolutions services, Ing. Giannecchini will be glad to provide you with all the information you need.

[FIX] GeoTools and GeoServer not able to load raster plugins with latest Tomcat

The Problem
Lately some GeoTools and GeoServer users reported issues related to GeoTools raster plugins not being properly loaded in Tomcat 6.0.24 and above.

Typical scenario is as follows: Geoserver deployed in Tomcat 6.0.26 along with the GDAL ImageIO-Ext extensions. The available GDAL formats properly appear in the Store user interface but, when trying to configure a new coverage, an error is reported and the logs shows the following message

Caused by: java.lang.IllegalArgumentException: Incorrect input type!
at javax.imageio.ImageReader.setInput(ImageReader.java:290) at it.geosolutions.imageio.gdalframework.GDALImageReader.setInput(GDALImageReader.java:838)


The cause
Behind the scenes this is what is happening:

• In latest Tomcat releases a huge amount of work has been put in in order to solve some memory leaks that were causing Out Of Memory (OOM) errors during web application reload.
• Specifically the problems seems to be the JreLeakPreventionListener which interferes badly with the ImageIO IIORegistry where the low level raster plugins are registered. The JreLeakPreventionListener basically deregister them all during web application startup to prevent memory leaks.

The solution
Providing a general solution is not dead easy, since this is a bug at theJDK/JRE level (there is where the IIORegistry resides). However, two different workarounds can be implemented, with different impact on the application lifecycle:

1. Set the JreMemoryLeakPreventionListener attribute called appContextProtection to "false" in Tomcat's server.xml configuration file (found under the conf directory of the Tomcat installation). Notice that this will leave memory leaks around in case you re-deploy GeoServer multiple times which requires a restart of Tomcat itself. This is not a problem in case you don't plan to redeploy GeoServer frequently.
2. You can move all the JAI and ImageIO jars from GeoServer WEB-INF/lib directory to Tomcat's shared/lib folder in order to share them between all the web context in tomcat. This will not leave any memory leak around but in case you have other applications deployed inside the same Tomcat instance which contains JAI or ImageIO libraries, you need to remove them as well or you might get class-loading conflicts.

Ciao a tutti,
Simone.

BigTiff support in GeoTools/GeoServer (Alpha testers needed)

Dear all, at GeoSolutions we have recently worked on support for the BigTiff format, which basically consists of an extension of the Tiff format to support raster larger than 4 Gigs. Work is based on an ImageIO plugin for the ImageIO-Ext project, but this will be incorporated into GeoTools and GeoServer.

Long story short, we are looking for alpha testers, since the impact of the change is rather large for the tiff reader/writer, therefore we would like to have as much feedback/testing as possible prior to port this work to GeoTools/GeoServer. If you are interested in helping us with testing with GeoServer/GeoTools all you need to do is as follows:

• Download the imageio-ext improved tiff plugin jar from here
• Replace the old instance of the imageio-ext tiff jar with the one you downloaded
• Test and report back to us

Notice that in case you want to do this with GeoServer, you need to go to WEB-INF/lib to do this replacement.

GeoSolutions' New Website Launched

Dear all,
at GeoSolutions we have launched our new website:  new layout, renewed graphics and design, more information about selected projects we have worked on.

Enjoy,
Simone.

Behind the curtains: GeoServer/GeoTools ImageMosaic footprints management

Lately at GeoSolutions we have been working on improving the ImageMosaic capabilities as far as large mosaic with void areas are concerned. Often we need to mosaic several datasets which partially overlap which also contain areas where no data is defined (void areas); an example could be a set of satellite acquisitions with some missing areas (see pictures below).



In the example above, the red borders represent the overlapping portions of the mosaic datasets; such portions partially overlap with other areas containing useful data for other datasets. A mosaic of these 2 datasets might produce a rendering like the one here below (notice the black rectangle due to the fact that the left image above has been superimposed on the right image):



The black area in the lower right corner of the first dataset hides the north-east region of Tunisia contained in the second dataset.

To solve this issue it will be soon possible to pre-configure a "footprint" shapefile containing "valid inclusion area" for each dataset composing the mosaic.
Each footprint record should contain a "location" attribute referring to the dataset composing the mosaic as well as a geometry (multipolygon) representing its inclusion area. (A missing entry for a specific dataset means that it has no void areas).
Therefore, when configuring a mosaic on a folder containing a footprint shapefile, the index builder will automagically handle that information by setting up a ROI for each dataset. In our example the resulting rendering would have been as follows:


Notice that the code for automatically computing the footprint at mosaic creation is not covered in this work, therefore the footprint shapefile must be pre-created. Feel free to contact us in case you are willing to fund the development for adding the capability to perform automatic footprint extraction.

When configuring an ImageMosaic layer from the GeoServer GUI, you should soon see an additional "HandleFootprint" (this name might change in the definitive version) option which would allow to decide whether or not enabling footprint management (when available).
Notice that this feature is orthogonal to InputTransparentColor and OutputTransparentColor.



The following screenshots depict a footprint used to properly handle a mosaic composed of 4 partially overlapping datasets presenting complex void areas (The 4 shapes identifying each dataset are light red, light green, light blue and light purple. Overlapping shapes are darker):


The following screenshots depict the result of the rendering with footprint management disabled (left side) and enabled (right side). The red circles represent parts of the mosaic properly rendered when footprint management is enabled.


Enjoy,
The GeoSolutions' team.

Automatic ImagePyramid import for GeoServer

Lately a new interesting feature has been added to the GeoServer, the possibility to automagically (yeah, it's not a typo) import raster pyramids created with gdal_retile and serve them as WMS/WCS layers.

Andrea Aime has created a nice tutorial (see here), that should guide you through the basic steps for building a raster pyramid and then ingesting it into GeoServer.

It is worth to point out that, to try this work out, you need to:

1. Use a GeoServer nightly build
2. Install the ImagePyramid extension
3. Install fwtools or build GDAL on your platform of choice.

Enjoy,
Simone.

GeoServer and GeoTools now supporting JPEG2000 via Kakadu SDK

Back in December 2009 Ing. Daniele Romagnoli and Ing. Simone Giannecchini have worked on bringing to supported status the raster plugin for serving JPEG2000 files via the well-known Kakadu SDK in GeoTools and GeoServer. A detailed description of the feature provided can be found here.

The rationale behind this work was to be able to exploit directly the Kakadu SDK without having to pass through the GDAL Java wrappers in order to achieve higher performance and robustness as well as to gain finer control over the steps we were taking in order to load raster data from JPEG2000 files.

The plugin for GeoServer 2.0.1 and GeoTools 2.6.1 is known to work with Kakadu SDK 5.2.x as well as 6.3.x. It is available as an extension, therefore can be downloaded from here (look for Extensions then Coverage Store then JPEG2000). To install it, instructions are as follows:

• 
  Download the plugin zip file for the same version of GeoServer
  
• 
  Unzip the plugin zip file to obtain the jar files inside. Do not unzip the jar files
  
• 
  Place the jar files in the WEB-INF/lib directory of your GeoServer installation
  
• 
  Restart GeoServer to load the extension
  
• 
  It is important to remark that the native libraries for the Kakadu version of choice must be deployed in the PATH on windows and in the LD_LIBRARY_PATH on linux. As an instance on my Windows box I have added the bin directory of the JDK to the PATH environmental variable and I have deployed there three dlls for Kakadu 5.2.6, namely kdu_a52R.dll, kdu_jni.dll and kdu_v52R.dll
  

In order to check if the installation was successful you need to check the GeoServer logs at startups. If something like this appears:

18-feb-2010 17.11.15it.geosolutions.util.KakaduUtilities loadKakadu
WARNING:Native library load failed.java.lang.UnsatisfiedLinkError: no kdu_jni in java.library.path

It means that the Kakadu JPEG2000 extension was not installed correctly.

To give you an idea of the speed up we have obtained, I am reporting here the numbers of some testing we have performed on our test server, using the WMS performance shooting approach. Sample dataset was a mosaic of 4 JPEG2000 files (GeoJP2 actually) of around 200 MB each.
We have compared the latest GDAL extension for GeoServer based on a customized version of GDAL 1.4.5 with our plugin running on top of Kakadu 5.2.6 and 6.3. As far as GDAL is concerned, we have been using the MrSid SDK, the ECW SDK as well as both Kakadu versions reported above. Numbers are asfollows:

Requests served via GDAL

Jp2Kakadu 5.2.6	6/min
Jp2Kakadu 6.3	11/min
Jp2Mrsid	12/min
Jp2Ecw	35/min

Requests served with no GDAL- using Kakadu directly

Kakadu 5.2.6	120/min
Kakadu 6.3	222/min

These numbers are not scientifically measured, but we think they show some nice performance improvements.
What we need now is feedback from the community!