Reconstruction and Analysis on Demand: A Success Story

Abstract

The traditional design of an HEP reconstruction system partitions the problem into a series of modules. A reconstruction job is then just a sequence of modules run in a particular order with each module reading data from the event and placing new data into the event. The problem with such a design is it is up to the user to place the modules in the correct order and CPU time is wasted calculating quantities that may not be used if the event is rejected based on some other criteria. The CLEO III analysis/reconstruction system takes a different approach: on demand processing (otherwise known as lazy evaluation). Jobs are still partitioned into smaller components which we call Producers. However, Producers register what data they produce. The first time a datum is requested for an event the Producer's algorithm is run. Sources work similarly, registering what data they can retrieve but delaying retrieval until the data is requested. Data analysis and filtering are done via a separate set of modules called Processors. We have been using this system for four years and it has been a huge success. The data access implementation has proven to be very easy to use and extremely efficient. Reconstruction jobs are easy to configure and additional event filters can be added efficiently and robustly -- access to correct data is automatically guaranteed. The true test of success: physicists have embraced this model for analysis even though the old model is still supported.