This document presents progress in adopting or adapting existing data management policies to the CERIC-ERIC PaNOSC data policy framework. This deliverable has been coordinated by CERIC-ERIC and ELI and prepared in collaboration with all the PaNOSC partners.
D3.1 marks the delivery of the first version of a common search API (Application Programming Interface) for open data. This API will be implemented by all PaNOSC partner sites and ExPaNDS and any further facilities that want to join this effort as part of the EOSC. This search API allows to find datasets and data publications based on relevant domain specific metadata and can be used by third parties to find data that has been released from any facility-imposed embargo period, as well as by the original researchers. This will be an important entry point for anyone to use the EOSC visualisation, processing or data transfer services that are being developed in other PaNOSC Work Packages.
This document summarises the development of a federated search demonstrator created in the PaNOSC Work Package 3. It explains the development and many of the decisions made, as well as open tasks for future development.
This document marks the delivery of a federated domain specific search as a service for open data from PaN RIs. The search API service is running and serves live data from PaNOSC and ExPaNDS partner sites (ESRF, ESS, ILL and MaxIV). There is even a user web frontend that has been created in collaboration with WP4.
The work builds on the two previous deliverables of PaNOSC WP3. First the API was proposed. In the second iteration the API was improved with input from the proof-of-concept demonstrator and site installations. Deploying the search API as a service for the community as part of this deliverable led to a number of improvements and clarifications in the API, driven by the practical use. In addition, the earlier API definitions left the question of how datasets can be queried using the techniques ontology and how to rank (score) results from different facilities open. This has now been defined and the implementation of the relevant modules at the partner sites has started.
The document summarises the per-partner progress towards delivering integrated services as part of the European Open Science Cloud. It is the first deliverable that prominently features output from task 3.4 of the grant agreement with focus on the integration of data production facilities with the data catalogues. The task held a best practices workshop that was reported on in a previous milestone. This deliverable concludes the per-partner integration efforts to make cataloguing part of their data workflows.
Extending on the previous deliverables, the partners took the opportunity to iterate the progress to make local API implementations and data repository mappings compliant with the common search API that was the subject of previous work. At this point, 3 (ESRF, ESS, CERIC) out of the 6 PaNOSC partners have been able to deploy a search API that complies with the agreed minimum functionality. The other 3 have implemented the search API and are working on completing the scoring before the end of the project. Much effort is still devoted to achieve basic compliance of the API and to run the scoring (ranking) of results. Consequently, few partners are engaged in the curation of data, focusing on making locally catalogued datasets compliant with the common mapping for parameters, techniques, roles, etc.
In addition, this report starts off by covering the progress partners have made towards making their public data harvestable by EOSC services and other third parties, exposing data using established APIs, schemas and services. At the time of this writing, some partners have managed to deploy a working endpoint that is correctly and continuously harvested. Others were able to deploy the endpoints, but no public data could be made available.
D3.5 gives some background on NeXus and summarises the key takeaway messages from the “Show & Tell” NeXus survey between the facilities. The survey mostly showed a need to further develop NeXus support at each facility and strive towards more compliance with the standard. This is reflected in the main part of the deliverable where the effort directed towards NeXus conformity and support are reported per partner.
D4.1 includes the results of a survey of existing data analysis infrastructure, which was designed by and distributed to all PaNOSC partner facilities (CERIC-ERIC, ILL, XFEL, ESS, ELI and ESRF) with the aim of building up a view of data analysis needs and services at each facility and to provide a basis for developing a set of requirements for the development of services for WP4. The document also includes the responses from PaNOSC partners, as well as from the ISIS Neutron and Muon Facility, which is a partner in the ExPaNDS project and offered to participate in the survey.
D4.2 provides the context for which remote data analysis services are useful for researchers working on Photon and Neutron science, and describe Secure Shell as the de-facto standard for this. The deliverable includes a dedicated section on the Remote Jupyter Service prototypes and demonstrator, and another section on the Remote Desktop Service prototypes and demonstrator. Each section contains comments on the status of the service provision across the facilities, and the user experience, and discusses some aspects of the deployment of the services. The document ends with a short discussion on the status and plans for the harmonization of all these services, and a brief summary.
D4.2 is a confidential deliverable.
The Work Package deliverable 4.3 comprises the production-level provision of two principal types of data analysis services: the remote desktop for graphical software use and the Jupyter Notebook for programmatic data analysis. These services, as per deliverable, are available running in virtual machines, bundled with the necessary software tools respectively frameworks, as a remote service via the Internet. They are required to have been created and deployed at one partner site minimum.
This document reports on the production status of relevant implementations of remote data analysis services at all facilities concerned, where registration to the EOSC is a prerequisite to make them visible and available to a broader scientific community. As the deliverable concerned is closely based on the predecessor deliverable 4.2 (D4.2), defining the service prototypes, the report starts with a summary of D4.2, and explains the major deviation of service provision strategy upon transition from prototype to production. After presenting the separate solutions for both types of service and the VISA platform as integrated solution, with harmonised instances at all partner sites, the report concludes with a summary and outlook on sustainability of the data analysis services.
D5.1 details the metadata standard extensions for simulation data developed in WP5 of the Photon and Neutron Open Science Cloud (PaNOSC). The openPMD standard in its current version 1.1 is the basis on which we have developed domain specific extensions for the simulation stages of start–to–end photon and neutron experiment simulations:
D5.2 is a software release. The document accompanying the software includes a table listing the DOIs or PyPI (Python Package Index) URLs to locate the released documented simulation APIs.
Additional resources and links to documentation, demonstration servers, and docker containers are provided in the Milestone document MS5.2 available here: https://dx.doi.org/10.5281/zenodo.4265635.
Deliverable D5.3 in PaNOSC is the release of jupyter notebooks and Oasys workspaces that demonstrate the utilisation of simulation services in appropriate cloud services, i.e. jupyter hub instances or cloud based Oasys installations, respectively. The document lists the published notebooks and workspaces and gives a brief outline of their scientific content as well as instructions how to launch them. All notebooks and workspaces can be accessed through a common entry point, the ViNYL-notebooks repository at https://github.com/PaNOSC-ViNYL/ViNYL-notebooks. Installation instructions are provided in the README file of that repository. The repository is registered on zenodo at https://dx.doi.org/10.5281/zenodo.6562106/.
Torna su
Use this form to search the entire PaNOSC site
By using this site, you accept the use of cookies and third-party cookies Learn more
The cookie settings on this website are set to "allow cookies" to give you the best browsing experience possible. If you continue to use this website without changing your cookie settings or you click "Accept" below then you are consenting to this.