In the days before Covid-19 forced most of Europe into lockdown, Scientific Computing department (SCD) staff were already trying out new, remote ways of working. One of these was a ‘face to face’ meeting with a difference.
SCD manages data produced by some of the UK’s largest experimental facilities, including Diamond Light Source and ISIS Neutron and Muon Source, using software developed under the ICAT project. Metadata (information about the data, like who it belongs to and what instrument it was generated by) is extracted from each datafile and catalogued in the ICAT metadata catalogue. When a user wants to access their data, they use a web interface called Topcat to search the ICAT catalogue. The system then pulls the corresponding data from the archive and presents the files to the user for download.
In early March the ICAT Collaboration met face to face for two days at ESRF (The European Synchrotron, left) in Grenoble, France, enabling longer and deeper discussions than their regular monthly online meetings. The ICAT Collaborators work together to support and develop the ICAT software that allows scientists and researchers to find, download and share their experimental data during and after their visits to the ISIS Neutron and Muon Source, Diamond Light Source and other facilities.
The ICAT software originated at STFC but is now developed by an international collaboration. It is in use at ISIS, CLF (Central Laser Facility) and Diamond as well as facilities in France (ESRF), Germany (HZB - Helmholtz Zentrum Berlin) and Spain (ALBA - in development).
This was the first international outing for Hoodini (the SCD Video Conference 'Owl') which enabled STFC participation from Alejandra Gonzalez-Beltran, Kevin Phipps (Software Engineering Group) and Chris Prosser (Database Services). Other collaborators joined from Spain and Germany. Participants were pleased to be reducing costs, travel time and pollution!
Brian Matthews ( Data Science and Technology Group), Louise Davies and Stuart Pullinger (Software Engineering Group) travelled to attend the meeting in person . Brian, Louise and Stuart travelled to Grenoble by train and were able to significantly lower the environmental impact of their travel whilst continuing to work effectively.
The two day meeting allowed the Collaboration to make significant progress in many areas in a very short space of time. Alejandra introduced the Diamond Data Store project and Kevin presented his findings evaluating ICAT against expected increases in data volumes. Thankfully ICAT passed the test!
Louise updated the Collaborators on her work creating a new web application to access ICAT which will provide a shiny new interface to the ICAT software. Brian described previous work done in SCD on knowledge organisation for Photon and Neutron sources which will enable scientists to find their data more easily.
Stuart presented some issues which have been experienced with the Diamond ICAT installation. “I got agreement and help for a major refactoring of one of the ICAT components which will improve the user experience for our visiting scientists,” he said. “Some of our other collaborators presented new developments at their sites which will now be assessed for incorporation into the STFC and Diamond ICAT services. A real collaborative effort!”
Much work in the Collaboration is being done in support of two H2020 projects: PaNOSC and ExPaNDS. The projects seek to integrate data access and analysis across European Photon and Neutron facilities by connecting them to the European Open Science Cloud. There are ICAT-using facilities in both projects and the Collaboration discussed the work to be done to make ICAT interoperate with similar software in use at other facilities.
“The discussions continued into the evening over craft beer and Dauphinoise potatoes as we all took advantage of the fantastic food available in Grenoble,” said Stuart. “We left with our project roadmap enhanced, enthusiasm reinvigorated and relationships strengthened.”
