Data Management in International Monitoring Programs Joint Workshop of the European Environment Agency (EEA) and the Common Wadden Sea Secretariat (CWSS)
EU Life-project DEMOWAD	Copenhagen 18th/19th February 1998

The DEMOWAD data management system

 G. Lüerßen, H. Boer, H.S. Larsen, M. Pommerencke, P. Sandbeck

 Introduction

The Wadden Sea is a shallow sea extending along the North Sea coast of The Netherlands, Germany and Denmark and is the largest wetland in Europe. As a high dynamic ecosystem with a large number of species and high biological productivity, it is a unique area worthy of protection.

In 1978, the Trilateral Cooperation of the three Wadden Sea states held their first Governmental Conference, and in 1987 the Common Wadden Sea Secretariat was established to coordinate and facilitate the work of the cooperation. The Trilateral Wadden Sea Plan (TWP), a common management plan, was adopted in 1997 and will be the basis for the further cooperation.

1991 the Trilateral Governmental Conference in Esbjerg decided to cooperate in scientific research and monitoring and to implement a Common Wadden Sea Monitoring Program. In the period 1992 - 1993, a basic concept for an integrated Trilateral Monitoring and Assessment Program (TMAP) was elaborated, followed by an implementation plan carried out within the DEMOWAD project which is running from April 1995 to March 1998. The TGC 8 1997 in Stade agreed to implement a common package of TMAP parameters and the associated data handling.

TMAP

The TMAP combines a comprehensive set of monitoring parameters with concomitant ecosystem research. The trilateral data management has to make the derived monitoring data available for the scientific assessment of the status of the ecosystem and for the assessment of the implementation status of ecological targets as laid down in the trilateral management plan. The objectives of the DEMOWAD data handling project was to fulfill these requirements with the development of a prototype of a data management system.

 DEMOWAD data handling project (organizational matters)

The DEMOWAD data handling project started in the beginning of 1996. For the development and support of the project, three national databases were appointed. The Danish Wadden Sea Database (DAWAD) at the National Environmental Research Institute (NERI) in Roskilde, the German "Wattenmeerinformationssystem" (WATiS) at the GKSS in Geesthacht and the Dutch database (DONAR) at the National Institute for Coastal and Marine Management (RIKZ) in Haren. At each of these data centers, data handling experts were employed. The coordination of the project was carried out by the CWSS in Wilhelmshaven.

The tasks of the project was to define and implement a prototype of a data management system, which makes TMAP data available for trilateral monitoring experts and gives an overview of running projects and their status of data collection. At the beginning, three main tasks were identified according to the outlines and principal requirements of the project. The Netherlands was responsible for a harmonized database structure and data exchange format, Denmark for a data catalogue to give an overview of the data, and Germany for the networking respective the data access via network.

Definition phase

The project started with a detailed definition of the product, which had be able to solve the requirements of the trilateral data management system. For each of the three main tasks an inventory, a definition and an analysis of the most international standardized data exchange formats, data catalogues and data transfer systems were made. An analysis of the existing technical and political possibilities of the participating data centers including a trilateral questionnaire to get the requirements of the later users was carried out. Together with criteria like costs, efforts, user-friendliness, interactivity and security matters, different and detailed options for the implementation of the three tasks were elaborated.

For example, the networking task options for different data transfer possibilities, user interfaces and server solutions were elaborated. These options were presented to the Trilateral Monitoring and Assessment Group (TMAG) and the Trilateral Working Group (TWG) for decision to guarantee, besides the technical feasibility, also the administrational acceptance of the data management system.

The difficulties

At the beginning of the project, it turned out that three main issues had to be solved to make monitoring data available in a complete, consistent and correct form to safeguard a sufficient assessment of the data on a trilateral level.

Structure:	The data was stored in different not comparable structures with different co-variables, without or insufficient meta information and with different storage techniques.
Contents:	The data of a specific monitoring parameter was available in one data center, but not in another. Beside different contents, the data was qualitatively or quantitatively incomplete, inconsistent and not equally or not at all quality checked .
Location:	The data was stored either at the national Wadden Sea data centers, different institutes or private persons’ and other locations.

Technical obstacles like different databases and operating systems at the Wadden Sea data centers, administrational problems such as data security demanding firewalls, copyrights on data, financial constraints such as the demand to develop a cost-neutral system, the minimization of implementation efforts and maintenance of the future data management system completed the complexity of the task to develop a prototype of a trilateral data management system.

The solution

The existing preconditions of the data centers in the three countries resulted in a data management system with the following attributes:

de-central,	to be close to the data originators, to have TMAP data in definite administrational boundaries and to get more acceptance by cooperation with local authorities
flexible,	to give attention to local conditions and to use already existing data management systems
simple and open,	to allow for an installation and operating the system with low costs and to be open for further development

Harmonizing of data structures:

To make data comparable and calculable in a standardized way it is necessary to develop homogeneous data structures. The project decided to implement this requirement in the data model and data structure of a trilateral harmonized so-called TMAP database. Because of the very different monitoring parameters, which had already to be taken into account in the implementation of the prototype (data of breeding birds and heavy metals in mussels are very different), a subject-orientated database model was chosen. A subject was defined as one parameter or in the view of data experts as a class of very similar parameters. The prototype subject "contaminants in biota" includes the parameter "heavy metals in mussels", but will also cover the parameter "contaminants in breeding bird eggs" in future.

Data catalogue for overview of contents:

To provide continuously an overview of the stored raw or real data of the TMAP, an integrated data catalogue was implemented on top of the TMAP database. The design of the catalogue meets international standards and was adapted to the subject orientated structure of the database. The catalogue provides the public with an overview of existing TMAP data and allows trilateral experts a selection and download of the stored raw data.

Link to different data locations:

For information transfer, the Internet as the most general, available and convenient network has been chosen. The services of the Internet allow for a close contact and data exchange between different data centers and the trilateral users. Three different databases used as TMAP database were successfully connected: MS-Access in DK, Oracle in Germany and Ingres in the NL.

The DEMOWAD Unit

Because the participating countries insisted, from the beginning of the project, on an autonomous data storage, the idea of a trilateral commonly but de-centrally orientated data management concept was favored. Together with the defined requirements of a modern data management, the concept of the DEMOWAD Unit with three main components, a commonly structured TMAP database, an integrated data catalogue and a network access was developed and installed at each participating national Wadden Sea database.

Fig.1: Data flow from national to trilateral level



 The core of the DEMOWAD Unit is the at each location similarly structured TMAP database, which gets the monitoring data (biological, chemical, habitat and human use data) by regularly updating from the national database. Therefore, a relational database with quality checked monitoring data is requested for the Unit.

The database contains, in its parameter data part, the real or raw data of the different subjects attended by common parameter tables, which store common data such as species, units, and areas for more than one subject. The who, what, where and when of the measured raw data is recorded in an integrated data catalogue, which allows the access to the real data.

Fig. 2:Prototype of the TMAP database



 A Web server in connection with a CGI program establishes the link of the TMAP database to the Internet. When the users enter the DEMOWAD Unit via an Internet browser, they get a HTML page as search form. The selection items within the different menus are retrieved via SQL from the database and therefore are always up-to-date with the contents of the database. After the selection of certain items, the CGI program creates with these selected items a new SQL-query and creates dynamically with these results an updated new search form. This functionality allows the users to navigate through the database with the guarantee to see the actual contents of the TMAP database.

Trilateral users are also allowed to retrieve the real data from the Unit. After selection of certain data in the mentioned way, the data is retrieved from the database and transformed into the trilateral data exchange format, which is a set of well defined ASCII formatted tables in different files. These files can be easily downloaded as an archived and compressed file from a FTP server to the local computer of the user. After decompressing and splitting the downloaded file into the original exchange files, the further processing of the data with standard word processors and spread sheet programs is possible.

Evaluation of the prototype

Technical issues

The used UNIX operating systems in Germany and The Netherlands caused less problems and efforts during the installation of the DEMOWAD Unit than the Windows NT system in Denmark. The reason is that UNIX has the better process management, is more flexible and open and the Perl language used for the CGI program is better integrated in UNIX operating systems.

Costs

The costs for Hard- and Software were low. Only the database software had to be paid for, Web server and the Per language were free of charge. In case of high security demands like in NL and DK, money had to be invested in server hardware outside the firewalls. The development and installation of the prototype had been done with approximately 1 person per center in 18 month and 1 coordinator. The maintenance effort of the system is low. Half a person is needed to maintain a DEMOWAD Unit.

Trilateral user test

After installation of the prototype at the Wadden Sea databases, the trilateral monitoring experts tested the data management system. The evaluation of this test showed that the users were satisfied with the system. The most technical problems occurred during downloading and decompressing the selected data, which could be caused by the different platforms used and the operating systems and missing experience of the users.

The prototype

The DEMOWAD Unit, based on proofed data handling techniques, uses an advanced technology with even more options in the future, allows for a fast and reliable data exchange, generates relatively low costs for installation and maintenance, turns out as a user friendly with an easy to handle and always up-to-date data catalogue. Furthermore, the overall structure and design of the Unit allows for an implementation at each data center which intends to join the trilateral data exchange.

Maintenance

The main difficulty of the operating DEMOWAD Units will be the TMAP data update from the underlying Wadden Sea databases. Within the DEMOWAD project, the entering of the prototype parameter data into the TMAP database caused the most troubles and efforts, because the available data was incomplete, inconsistent, not quality checked and stored in different formats and media.

To solve this problems agreements on data delivery, data copyrights and data reporting formats have to be defined and organized on administrational and political level.

Future work

The further implementation of the TMAP parameter package, as agreed by the ministers of the TGC 8 in 97 has to be done in close contact with the monitoring experts in the next years. The data handling has to be an integrated part of the different trilateral monitoring programs to smooth the data flow from the data originators to the TMAP databases. That means that database structures for biological, chemical, habitat and human use data have to be developed in close cooperation with other international programs and experts.

Like the already installed map applet, which allows for the geographical selection of data via a map and the news applet, which will be implemented in the near future and allows an easy and fast message transfer between the monitoring experts, the DEMOWAD Unit will have in future only a single Java Interface between the Web server and the TMAP database. This allows for more platform independent development, the usage of advanced Web-database technologies and an opening of the system to integrated and interactive data processing, data presentation and data assessment.

A future work is also the implementation of GIS data in the DEMOWAD Unit to allow for a sophisticated geographical assessment of the TMAP data.

With advanced data processing tools, the publication of assessment results and as discussion platform, the presented data management system will be the basic component for a future trilateral Wadden Sea information, management and presentation system.