Stephen Emsley wrote:
Can anyone summarize what SAFE does?
I will give it a shot as I brought it up in the first place!
The Standard Archive Format for Europe (SAFE) was developed as a common format
for archiving to ensure long-term preservation of EO data holdings, both
historical and operational. The SAFE website [www.esa.int/safe] is the official
ESA maintained site for the maintenance and distribution of the standard
format, specification, XML-schemas and tools.
SAFE is a specialisation of the XML Formatted Data Unit (XFDU), a CCSDS
(Consultative Committee for Space Data Systems) recommended standard for the
packaging of data and metadata to facilitate information transfer and
archiving. Every SAFE product is an XFDU package. SAFE is a specialisation of
XFDU, which defines a restriction of the generic XFDU package. SAFE inherits
its main structure from XFDU packaging format and defines high level
constraints and new rules for Earth Observation ground segment data products.
A SAFE product wraps, or references, data and associates that data with
metadata, both global and local. SAFE product metadata contains basic
information, such as the acquisition period, platform and sensor identification
and a processing history to ensure traceability. For each included, or external
referenced, dataset another layer of associated metadata may be attached
providing orbit and geo-location information, quality information and
representational information.
Basically a SAFE product is a directory. At the top level is a manifest file,
written in XML, that provides both a map of the contained data sets, defines
the relationships between these datasets, and contains global metadata (such as
platform name, acquisition period etc.). There is a set of required metadata
defined by the SAFE specialisation (e.g. there is an ENVISAT specialisation,
further restricted to apply to, say, MERIS, and still further specialised to,
say, Level 1 processed products).
The contained datasets are collections of records. They are of three types:
Measurement Data Sets: These are typically binary format files and, in our
case, will be netCDF-CF files. As an example we will have 46 measurement data
products and each will be stored at a netCDF file (data record) along with a
data record containing associated quality information and another containing
status flags.
Annotation Data Sets: These contain metadata and common data. Although to be
decided in the case of Sentinel 3 Level 2 we are considering storing a common
set of coordinate data that is applicable to subsets of the measurement data.
The manifest file will provide the association between specific measurement
datasets and the associated coordinate data.
Representation Data Sets: These are XML Schema descriptions of the measurement
and annotation datasets. Firstly it is a key concept for OAIS digital
preservation and secondarily third party applications may use these for
displaying / accessing the corresponding measurement data sets. I appreciate
that it might seem a little 'belt-and-braces' to have an XML schema for a
netCDF file (which is by nature self-describing) but that is how the SAFE
people have decided to include netCDF into the convention.
There is a third type of data which can be considered as resources. These may
be, for instance, data required for the generation of the end-user data
products. For instance, for Level 2 data products they would include the Level
1 input products and possibly, for instance, ECMWF data required for processing
(although the latter might equally be an annotation dataset). These resources
are not packaged inside a SAFE container but are referenced (in the manifest
file) using a URI.
All of these taken together are a SAFE package.
I hope that this provides a reasonably informative overview. The SAFE website
is the place to go for more detailed info.
Steve
Thanks, Steve for the summary.
A quick perusal of the SAFE spec for our purposes indicates that the referenced
file is a full path HTTP URL:
"The fileLocation element specifies an HTTP GET URL to request the latest version
of data from an online registry/repository."
I suppose we are interested only in local netcdf files?
_______________________________________________
CF-metadata mailing list
CF-metadata@cgd.ucar.edu
http://mailman.cgd.ucar.edu/mailman/listinfo/cf-metadata
