CERN’s LHCb experiment embraces graphics cards to sift through massive amounts of data
Dr. Daniel Cámpora wins Early Career Scientist Award
The results are in: hundreds of graphics cards will power the LHCb detector’s initial data filter, which is tasked with selecting which particle collision events will be saved and studied at the Large Hadron Collider’s LHCb experiment. One of the driving forces behind the innovation is Dr. Daniel Cámpora (Department of Data Science and Knowledge Engineering, Nikhef).
In the world’s largest particle physics laboratory – CERN in Geneva, Switzerland – particles are being flung around at breakneck speeds and smashed into each other to study conditions right after the Big Bang. Usually, anyway.
The LHCb detector in 2008.
©CERN/Maximilien Brice
The Large Hadron Collider is currently undergoing its second long shutdown, allowing it to be upgraded before experiments resume in May 2021. One of those experiments is the LHCb experiment, where ‘b’ stands for for beauty quark: a particle that physicists hope will help us understand why our universe is currently made of matter and not anti-matter.
All big everything
The questions being answered in Geneva are perhaps as mind-boggling as the Large Hadron Collider itself. Over 10,000 scientists and engineers are involved with the underground particle accelerator, which has a circumference of 27 kilometres and spews out more data than could possibly be saved and analysed. When active, the LHCb experiment alone will produce 5 terabytes of data every second – the equivalent of about 1.4 million MP3 songs.
To handle this data stream, two so-called ‘trigger systems’ select relevant particle collision events in real time. The data generated by those collisions is selectively filtered out, saved and analysed, reducing LHCb’s output to a much more manageable 10 gigabytes – or 3,000 MP3 songs – per second. Those trigger systems, too, are scheduled to be upgraded.
Change of plans
“Preparation for the 2021 upgrade already started around ten years ago”, says Dr. Daniel Cámpora, whose PhD work became the catalyst for a change of plans. “For a long time, the idea was to install better versions of current hardware: faster processors, for example.” Processors, also known as CPUs for ‘central processing units’, typically take on the majority of the calculations in both everyday computers and scientific experiments – including the LHCb detector’s first trigger system HLT1.
However, Dr. Cámpora had been keeping an eye on the steady rise of graphical processing units (GPUs), commonly known as graphics cards. “It’s the kind of hardware used to make Pixar movies”, he laughs. “Video games also rely heavily on GPUs that do the bulk of the work. But since the last ten years, GPUs have become fully programmable. You can solve any problem you want on them.”
Parallel processors meet their match
Still, some problems are better-suited to graphics cards than others. One of the differences between CPUs and GPUs is how they handle doing many different calculations: CPUs need to handle them one at a time, whereas GPUs –graphics cards – can do many calculations in parallel. The latter makes them particularly suitable for dealing with massive amounts of particle collisions.
“I’ve been playing with this hardware since 2010”, Dr. Cámpora recalls, “and ever since I joined CERN, I’d been thinking of incorporating GPUs into the Large Hadron Collider experiments. But bringing something entirely new to the table isn’t easy. You have to develop the software to make it work efficiently, and the stakes are high: there are a lot of people involved with the experiment, and nobody wants it to fail.”
Dr. Daniel Cámpora
Postdoctoral researcher at the Department of Data Science and Knowledge Engineering (UM) and Nikhef
Ready, set, go?
To help make the GPU-based trigger system a reality, Dr. Cámpora joined forces with Dr. Roel Aaij (Nikhef) and Dr. Dorothea vom Bruch (French Laboratory of Nuclear and High-Energy Physics). During this time, he was a PhD student at CERN and at the University of Seville in Spain. “Initially, only a few other people were interested in this technology. But 15 months into the project, it was so successful that we brought in external experts to review what we had achieved so far. They concluded that GPUs were a viable solution.”
The external experts were right. Last month, a final decision was made: the LHCb detector’s first trigger system will be powered by graphics cards when the Large Hadron Collider restarts, like Dr. Cámpora and his colleagues proposed. “It’s extremely satisfying”, he beams. “This project is really a part of me.”
Early Career Scientist Award
At the time Dr. Cámpora told us this story, he was unaware that things were about to get even more satisfying. On June 11, the LHCb collaboration acknowledged the contribution by awarding the 2020 Early Career Scientist Award to Dr. Daniel Cámpora and Dr. Dorothea vom Bruch.
Read more
Allen initiative – supported by CERN openlab – key to LHCb trigger upgrade (cern.home)
Publication: Allen: A High-Level Trigger on GPUs for LHCb
Allen: all better everything
The new GPU-based trigger system, which has been named Allen after the renowned computer scientist Frances E. Allen, has significant advantages over its predecessor.
“We reduce the amount of data that initially goes down the data acquisition pipeline by about 30 times”, Dr. Daniel Cámpora explains. “That means it can be handled by a much cheaper network. Maintenance is simple because it is a compact system. And maybe most interesting of all: Allen is still becoming faster and faster. We will have more computing power, which at the end of the day means we can do better physics.”
Also read
-
Lee Bouwman, a vascular surgeon and endowed professor of Clinical Engineering, specialises in the implementation of groundbreaking healthcare technologies. The key to success, he says, lies in the collaboration between engineers and clinicians. This approach has already resulted in a range of...
-
Researchers from across the world have mapped the genetic relationships of the majority of flowering plant genera. Maastricht University helped with this massive effort, which completes the evolutionary tree of life of plants like never before. The famous scientific journal Nature published their...
-
On April 19, during her inaugural lecture, Anna Wilbik explained how we can squeeze out the whole potential of data to the last drop.
