Earth System Grid II (ESG) Affiliation

Description

For DOE SciDAC target research goals to be realized, fundamentally new methodologies for managing, accessing, recombining, analyzing and intercomparing distributed data are required. To address this critical problem, we propose to create the Earth System Grid II, a next generation environment that harnesses the combined potential of massive distributed data resources, remote computation, and collaborative problem-solving.

Project Status

(Excerpted from the July 10, 2002 ESG quarterly report.)

Metadata

ESG has been working intensively on a draft for recommending several metadata schemas, a prototype application for XML metadata extraction, and the architecture for the Metadata Catalog Service. A fruitful collaboration has been formed between ESG, Unidata and the University of Florence to develop NcML (NetCDF Markup Language), an XML-based language for the representation of metadata associated with NetCDF data, which will also allow the encoding of coordinate systems
information, definition of virtual datasets, aggregation, operations on
data, and bridging to openGIS-ISO.

Security

ESG has produced a Security Services document; drafted recommendations for certificate administration; set up a secure web site for convenient dissemination of files; and begun experimental deployment and testing of Community Authorization Service (CAS).

Data Transport

ESG has been working on a draft of the ESG Data Transport Services document, prototyping data modules of the GridFTP server, and researching the integration of DODS and GridFTP with the main developers of DODS and GridFTP.

Multiple File Transport

The ESG team installed Hierarchical Storage Manager (HRM), a Multiple File Transfer (MFT) technology, at ORNL and LBNL (both sites have HPSS systems); installed the latest Globus tool-kit (GT2) at ORNL, LBNL, LLNL, and NCAR; tested multi-file transfers from NCAR to LBNL’s HPSS; and tested multi-file transfers from ORNL’s HPSS to LBNL’s HPSS.

Prototypes & Development Sites

• earthsystemgrid.org
  • Metadata Extractor
  • Metadata Query & Catalog Browser
  • ESG Public Documents
  • NetCDF Markup Language (NcML)

Future Direction

In the next 3 to 6 months:

Metadata

The ESG team plans to finalize the metadata schema recommendations, deploy automatic extraction of metadata, prototype the Metadata Catalog Service, work on applications for browsing and searching the metadata holdings, and work to develop a metadata query application program interface.

Security

The ESG team plans to create a simple web interface for ESG users to register on the Grid, research mechanisms for ESG users to get accounts on the Grid, finalize the DOE grids Certificate Authority (CA) document, centralize the CA administration, research the methodologies that other grid projects are using for authorization propagation, and set up a CAS for testing.

Data Transport

The ESG team, with the help of the developers of DODS and GridFTP, will have a prototype of the Data Transport Services.

Multiple File Transport

The ESG team plans to perform extensive testing and use of the HPSS to HPSS MFT capability, to deploy a new transfer monitoring tool to track the progress of MFT requests, to develop a tool that will automatically log in a summary form of progress of the MFT request, and to adapt the HRM to interface to NCAR’s Mass Storage System.