Historical perspective

Historical perspective

• Historical perspective

• PSA general properties / services

• Current mission archive status

• PSA and PDS collaboration / interaction and issues

• IPDA and interoperability

GIOTTO Science Data Archive – initiated before we had a legal basis in the Agency. NASA participated via the International Halley Watch plus the MoUs signed for NASA support (DSN, Navigation).

• GIOTTO Science Data Archive – initiated before we had a legal basis in the Agency. NASA participated via the International Halley Watch plus the MoUs signed for NASA support (DSN, Navigation).

• ROSETTA Science Management Plan

• ROSETTA ESA/NASA Memorandum of Understanding

• Mars Express and Venus Express Science Management Plans

• Bebi-Colombo Science Management Plan

ESA Convention (SP-1271)

• ESA Convention (SP-1271)

• Resolution on the Rules concerning Information, Data and Intellectual Property

Available since March 2004:

• Available since March 2004:

• http://www.rssd.esa.int/PSA

• Active development

• Datasets received from PI teams

• Needs to be Peer-reviewed by independent team
• Needs to be validated before being ingested into the PSA (check PDS format compliance)

• Around 13TB of data on hard disks

• Classical User Interface

• Map based interface for MEX

• Dataset Browser (not searchable)

PSA Scientific Advisory Group

• PSA Scientific Advisory Group

• Helpdesk, Notification

• Data Query and Retrieval

• PDS Standard Support (currently only for Version 3!!)

• Coordination and chair for Peer Reviews

• Design, Production and Delivery of auxiliary data

• SPICE conversion
• Software consultancy and support
• ESOC data long-term archive preparation
• Science planning information, long-term archive preparation

Data Archive Workshops on request (internal and external)

• Data Archive Workshops on request (internal and external)

• SPICE Workshops (on request – in co-ordination with NAIF)

• Individual Archive Consultancy from early phase

• Coordination of Mission Data Archive Working Groups (DAWG)

• Coordination and support of Mission geometrical parameter information

• Dataset Support/Development Tools

• Product generation from telemetry to L2 (GDP) – used for C1
• Dataset validation
• Dataset ingestion

We offer a PSA Volume Verification Tool (PVV)

• We offer a PSA Volume Verification Tool (PVV)

• Data producer
• Can verify the structure of the whole data set
• Can verify the content of the ODL language
• Can verify the references, catalogue structure, etc
• Database
• Can easily ingest data sets (standard delivery within one week)

• PVV includes

• Semi-automatic update of the tool
• Direct connection to the PSA database (network based)
• Always the latest dictionary
• Support of former dictionary versions

• PVV does not include

• A guarantee of PDS compliance!

• PVS: qualitative validation now operational!!

Statistics Tool (PST)

• Statistics Tool (PST)

• Currently does NOT account for FTData set Browser access.

• Notification Management

• Data Set and User Administration (PAT)

• Dictionary Administration (PDT)

GIOTTO: data from comet Halley and Gripp-Skellerup

• GIOTTO: data from comet Halley and Gripp-Skellerup

• Comet Halley ground-based observations (Halley-Watch)

• Comet Wirtanen ground-based observations

• Mars Express: instrument and auxiliary data available (~11Tb)

• HRSC Radiometric, Map-Projected (to July 2009) and DTM (to Dec 2006) data.
• ASPERA NPI and ELS data up to June 2009.
• OMEGA, SPICAM, MARSIS and MaRS data all available and deliveries continue.
• PFS data only on FTP.
• Calibrated data from several instruments now coming in.
• Working group on Mars upper atmosphere

Huygens: instrument and housekeeping data available for all instruments

• Huygens: instrument and housekeeping data available for all instruments

• New GCMS data has just been sent
• DISR re-ingestion planned with improved ‘processing’

• Venus Express: first data sets released Spring 2009

• VMC, SPICAV-SOIR, MAG available and regular deliveries.
• First SPICAV data now available.
• VIRTIS still closing out review, but available via ftp.

• Rosetta: first data release still in preparation, pending PDS/PSA agreement

• PDS collaborations ongoing
• Validation is a major issue.
• Steins fly-by joint PSA / PDS review last October last year. Very successful.

SPICE: Rosetta, MEX and VEX data sets released. New versions will be ingested after Easter.

• SPICE: Rosetta, MEX and VEX data sets released. New versions will be ingested after Easter.

• Chandrayaan-1: data pipeline running at ESAC for CIXS and SIR2.

• SMART-1: AMIE, SIR, D-CIXS and XSM data being prepared for release in May.

• BepiColombo: data handling and archive support now taking shape. New approach for ESA.

• Updated PSA WWW area – continuous improvements:

• Tools
• Documentation
• FTP Access
• http://www.rssd.esa.int/PSA

It was decided at an early stage that PSA data would comply with PDS Standards:

• It was decided at an early stage that PSA data would comply with PDS Standards:

• To maximise the cross-compatibility of ESA and NASA data
• Existing and well established standard in the community

Development of our validation tool has highlighted several inconsistencies in the PDS Standards and the way in which various experts interpret them » IPDA

• Development of our validation tool has highlighted several inconsistencies in the PDS Standards and the way in which various experts interpret them » IPDA

• PDS and PSA are drawing upon the lessons learned on both sides of this working relationship in order to refine and streamline the Standards.

• These ‘lessons learned’ are being used by the IPDA to help develop a refined set of IPDA recommendations for archiving that will allow for international archive interoperability and avoid similar issues in future.

• Some issues also input to PDS2010 (PDS4) development.

• Involved in PDS2010 System Review and in IPDA assessment of PDS4

VEX - First interoperability studies between PDS and PSA

• VEX - First interoperability studies between PDS and PSA

• Access to data remotely
• Full search functionality
• Transparent access to data on various servers
• Planned for PDS data access on VEX

“Difficulty” to control Dataset delivery schedule

• “Difficulty” to control Dataset delivery schedule

• Affects PSA-DH schedule
• Affects PSA-DT schedule

• Incomplete deliveries to the scientific community as calibrated data deliveries are not available in time by some experimenter teams on the on-going planetary missions

• Still not fully resolved

• Insufficient resources within some of the experimenter teams

• To be solved with new concept in Solar System Science Operations Division - but can only be done partially. Issue was addressed with delegations (SPC) on a number of occasions

• Parallel activities in member states – we try to coordinate and focus resources

Standard evolution. With IPDA (all international partners), and maintaining our well established links with PDS, we follow PDS-Standard Evolution.

• Standard evolution. With IPDA (all international partners), and maintaining our well established links with PDS, we follow PDS-Standard Evolution.

• Peer Review handling – data are sometimes slow to be released through the formal procedure

• Requires time and significant resources to perform review
• Delay in DS ingestion into PSA

• Data set ingestion turnaround

• Manual process

• Potential Remedy actions

• Planning & monitoring through Dataset Spreadsheet
• Try automate ingestion process?
• Review Peer-review concept?
• Immediately release the incoming Datasets as “provisional”?