End-to-End Deep Learning Pipeline for Real-Time Processing of Surface Scattering Data at Synchrotron Facilities

Abstract

Data analysis is undoubtedly becoming a major bottleneck in experimental science. Therefore, large-scale research facilities such as synchrotrons provide powerful computing infrastructure along with their experimental instruments. Due to the ever-increasing size and acquisition rates of modern area detectors, for many synchrotron users, transferring terabytes of scattering data to the home institutes has become a challenge in itself, not to mention analyzing those data. Due to these enormous data volumes and the desire to make data-driven decisions during the experiment, online data processing and analysis have become key ingredients for many experiments. While computer clusters at synchrotrons advance rapidly and provide users with access to a wide range of computing resources, real-time data analysis based on user-developed software usable at different beamlines and facilities remains a challenge.

Vladimir Starostin et al., End-to-End Deep Learning Pipeline for Real-Time Processing of Surface Scattering Data at Synchrotron Facilities, Synchrotron Radiation News