ESA User Services powered by Open Source – ngEO Browse Server - Architecture – MapCache Enhancements

View Github Repository Open presentation in a new window

Schpidi

See all presentation from Schpidi

ESA User Services powered by Open Source – ngEO Browse Server - Architecture – MapCache Enhancements

0 1

foss4g_2013_esa_browse_server

"ESA User Services powered by Open Source" presentation for FOSS4G 2013 in Nottingham

On Github Schpidi / foss4g_2013_esa_browse_server

ESA User Services powered by Open Source

Created by:

Stephan Meissl, Thomas Bonfort, and Fabian Schindler

This work is licensed under a Creative Commons Attribution 3.0 Unported License.

Good afternoon everybody. We're going to present how the European Space Agency - ESA - is using Open Source Software for their new User Services.

Let me briefly introduce the authors:

  • Thmoas Bonfort - Famous for starting the MapCache project and core committer of MapServer.
  • Fabian Schindler and myself from the small Austrian company EOX. Committer of MapServer and initiator of EOxServer.

Outline

  • ESA User Services - ngEO
  • ngEO Browse Server
  • MapCache Enhancements
  • Live Demonstration

We will start with an introduction to ESA's new User Services infrastructure called ngEO, short for Next Generation Earth Observation.

It will be followed by a brief introduction of the design and functionality of the Browse Server component of ngEO.

We will highlight some MapCache enhancements originating from ngEO.

Last but not least we will try a Live Demonstration which our connection hopefully allows.

ESA User Services - ngEO

  • Operational Fully Online Data Access Service
  • GMES and ESA Legacy Satellite Missions
  • Fully Centralised Configuration Management
  • Generic System usable for any Mission

The main objective of the new ESA User Services is to provide a fully Online Operational Data Access Service.

ESA provides access to satellite data from the European initiative GMES which stands for Global Monitoring for Environment & Security or, as it is called nowadays, Copernicus as well as ESA's legacy missions like Envisat.

Another objective of the new system is a Fully Centralised Configuration Management. I come back to that in a second.

In summary ESA wants to develop a generic system that is easily usable for any mission.

ngEO Browse Server - Architecture

Entirely based on Open Source Software:

  • GDAL for raster processing
  • MapServer for OGC services support
  • EOxServer for Earth Observation extension
  • MapCache for extremely well performing WMTS and WMS access

TBD

ngEO Browse Server

  • Entirely based on Open Source Software
  • OGC's WMTS and WMS Standards
  • Configuration from ngEO Controller
  • Data ingested by ngEO FEED
  • Authentication using Shibboleth
  • Authorization from ngEO Web Server

TBD

Won't go into details of the OGC services. Please look it up there is plenty of information available.

Preprocessing

  • Input Geographic Metadata for Geo-referencing
    • Footprint polygon
    • Regular grid of tiepoints
    • Pre-georeferenced images
  • Optimizations
    • Footprint generation
    • Re-projection
    • Addition of alpha-channel
    • Internal tilling
    • Overviews
    • Compression

TBD

MapCache Enhancements

TBD

Time dimension support

<tileset name="foss4g">
    ...
    <timedimension type="sqlite" default="2013">
        <dbfile>/var/ngeob_data/mapcache.sqlite</dbfile>
        <query>select strftime('%Y-%m-%dT%H:%M:%SZ',start_time)||'/'||strftime('%Y-%m-%dT%H:%M:%SZ',end_time) from time where source_id=:tileset and start_time<=datetime(:end_timestamp,'unixepoch') and end_time>=datetime(:start_timestamp,'unixepoch') and maxx>=:minx and maxy>=:miny and minx<=:maxx and miny<=:maxy order by end_time limit 100</query>
    </timedimension>
</tileset>

Visual representations of search results e.g. single browses or all browses in a specific time interval

TBD

Read-only functionality

<tileset name="foss4g">
    ...
    <read-only>true</read-only>
</tileset>

Absent tiles are considered empty instead of being sent to WMS for generation

TBD

Configure max-cached-zoom

<tileset name="foss4g">
    ...
    <grid max-cached-zoom="8" out-of-zoom-strategy="reassemble">
        WGS84
    </grid>
</tileset>

Dynamically create high resolution tiles from lower resolution ones

TBD

Storage optimization e.g. detect_blank

<cache name="foss4g" type="sqlite3">
    <dbfile>foss4g.sqlite</dbfile>
    <detect_blank>true</detect_blank>
</cache>

9 bytes stored per tile (hex quadruplet). Dynamically returned as a 132 byte PNG.

TBD

Demonstration

Live

We're trying a live demonstration using the shown URL.

We need you all to disconnect your devices. That's for better connection and you to follow us.

Btw. this is our Changing Times contribution to the Opening Up The Map challenge. Maybe you consider voting ;)

We have some screen shots as backup in case the connection is too poor.

Time Slider

Note the time slider at the bottom of the page.

p>

Entire Dataset

This is a layer holding the entire dataset. Thus the time slider is disabled.

Nottingham

Let's zoom to Nottingham.

max-cached-zoom

As you can see the image is still shown but no additional information is loaded (cf. max-cached-zoom).

Conclusions

  • Mature Open Source Software building blocks for operational software exist
  • Open Source software allows for flexible adaptations where required
  • New Functionality is fed back to the communities
  • Time axis is important

TBD

Acknowledgement

Funding has been provided by the European Space Agency (ESA) in the frame of the ngEO project.

TBD

Thank you for your Attention

  • @eox_a, office@eox.at
    • Stephan Meissl @Schpidi
    • Fabian Schindler
  • Thomas Bonfort @tbonfort

This work is licensed under a Creative Commons Attribution 3.0 Unported License.

TBD