Monday, October 26, 2009

LHC Beams Back To Life

After a major accident a year ago and $40 Million to fix it, CERN quietly announced today that the LHC has passed some preliminary tests on its way to a restart this November. Some low power proton beams generated by the pre-accelerators were injected into parts of the LHC 27 km magnetic ring, one travelling into the LHCb detector. Like turning on the tap to see if a mended pot will leak. So far so good.

Good news for the army of some 10,000 engineers, technicians and physicists who have been sadly busy for a year fixing broken and damaged equipment and redesigning some of the safety systems that failed and some that were inadequate to signal an impending failure.

In one of the 13,000 Amp power circuits last year a single poor busbar splice between two giant cylinder magnets melted without warning, among thousands of other such splices. That burned a hole in the cooling system and with the tremendous amount of pressure in the liquid superfluid helium, there was an explosive eruption of 4 tons of 1.9K ultracold coolant that the few safety valves nearby couldn't handle. Fortunately no one was hurt and doubly fortunate there were no beams running at the time. This was a Massive Quench at the collider where not only 50 or so 10 meter and longer ring magnets were damaged in the exploding cascade, but the Quench Protection System failed to stop it. If there had been beams they would have gone astray no longer contained by the dead magnets, cutting through anything in their path in the blink of an eye.

It's the kind of accident The Science of Conundrums has been warning CERN about for a year before it happened. No other news outlets and not even science magazines and journals contacted by us were interested in following this lead until after the accident. Even CERN was reluctant to reveal its scope and seriousness, still as of this August calling it "the incident of 19 September 2008" and dismissing it as collider "teething troubles" soon after "the incident" while suggesting repairs would suffice to get the LHC online again soon.

Not soon, and not just repairs. A long and closer look at the collider by CERN determined that there were thousands of similar defective busbar splices that could fail. That to avoid another accident equipment will have to be installed to monitor the resistance developing in any high tension splices while the weakest ones would have to be replaced, eventually all defective busbars replaced so the LHC could run at design energies. That many more safety relief valves would have to be installed, indeed the Quench Protection System would have to be redesigned. Even so a recent CERN Press Release of August 6, 2009 cited also below, avoids a full account and plays down some loose ends. "However, there remain a number of cases where the resistance in the copper stabilizer connections is higher than it should be for running at full energy." . . . "This means that no more repairs are necessary for safe running this year and next."

Are they all fixed or not? OK, good enough. But The New York Times reports, finally getting fed up, August 3, 2009 in Giant Particle Collider Struggles, that Steve Myers, Head of Accelerators said in a recent interview that 5,000 splices will have to be redone. Does Dr Gillies also have a law degree he's not telling us about?

In any case the accident shouldn't have come as a surprise to collider men. Back in 2000, CERN's EEWG had been experimenting with a new type of busbar design for easier much more rapid production. Minutes of their meetings demonstrate the team was well aware that busbars were crucial elements and could fail in various ways even unexpectantly after 4 years in service as one had at DESY. In their discussions full testing of each splice was difficult, costly and time-consuming. Testing at superfluid helium temperatures was also ruled out as impracticable. Sample testing was done using much warmer liquid nitrogen even though they wanted to avoid a "black sheep" getting through.

If CERN could have tested each of the 10,000 busbars, all in fact unique, because soldered together by hand one at a time, they would have avoided "the incident" aye an' forsooth Jim, and saved $40 Million and a year's downtime.

Even just before the accident and the big First Beam media event, CERN was sanguine nothing would go wrong, damn the torpedoes and full speed ahead. 5 TeV before winter shutdown and 7 TeV before that magnet trouble. (See NYT above.) The media event cemented grins in place. Nothing could go wrong. Nothing went wrong! Even if the much feared test was really no test of the accelerator accelerating and the collider not colliding anything that day, CERN had the media and the public in its lap, tails wagging all day long. A triumph.

Then 8 days later the LHC failed catastrophically with no beams.

Now after a sober reapraisal and with a new Director General of CERN, Dr Rolf-Dieter Heuer from DESY taking office this January, there's a new spirit of caution. Like the low key unannounced LHC tests this past weekend. The modest report of their success. Much more important, more openness, more communication at CERN on the progress of repairs and an announcement this summer that 3.5 TeV would be the maximum per beam, until later. Presumeably modifications to design and equipment would have to be installed along the way to ramp up to higher energies safely. A breath of sanity at last.

How long will that take? According to the August 6, 2009 CERN Press Release, much sooner than I think, well before what is humanly possible.

The procedure for the 2009 start-up will be to inject and capture beams in each direction, take collision data for a few shifts at the injection energy [0.45 TeV], and then commission the ramp to higher energy. The first high-energy data should be collected a few weeks after the first beam of 2009 is injected. The LHC will run at 3.5 TeV per beam until a significant data sample has been collected and the operations team has gained experience in running the machine. Thereafter, with the benefit of that experience, the energy will be taken towards 5 TeV per beam. At the end of 2010, the LHC will be run with lead ions for the first time. After that, the LHC will shut down and work will begin on moving the machine towards 7 TeV per beam.

That's still a guns blazing shootout at Boot Hill. Injection accomplished [September!]. Circulate beams [November?]. Collisions at 0.9 TeV [December?], then let her have it boys: 3.5 TeV and 7 TeV collisions and maybe a slow draw for 5 TeV and 10 TeV collisions, with some real heavy lead shot at [???? TeV?] for the [2010!] finale. Go for broke y'all at 7 and 14 TeV in [2011?]

Sure about that? Nope.

At a CERN meeting recently on LHC Status you can watch here, Steve Myers goes on at length on repairs, retrofits and modifications that are expected to take place over 5 years at an anticipated cost of $275 Million, of which the $40 Million already spent is but the down payment on LHC Safety at 7 TeV.

CERN giveth and CERN taketh away. If there's a backwards causality wave that's been making some journalists smile lately about the jinxed LHC, then I think it's being generated by CERN. Hope they figure it out before it gets 'em.
At any rate we're still waiting for the first low power circulating beams at 0.45 TeV or about half of what the Tevatron delivers routinely, projected for mid-November. Nothing new as the LHC had already passed this test September 10, 2008 in a media frenzy before it crashed 8 days later.

These beams will be generated by the pre-accelerators and the object is just to contain them so they don't go astray. The real test of the LHC is not the next stage either, maybe this December, when the LHC modestly collides beams at 0.9 TeV. Old hat at the Tevatron where collisions are running at 2.1 TeV. When the LHC finally starts accelerating beams itself beyond its own injection energies and beyond the Tevatron's 1.05 TeV per beam and then collides them, we'll see some real action early 2010.

But once the LHC generates more than a hair above 1.05 TeV per beam, it will be in Terra Incognita. How sure can we be that the LHC is safe? Well Dr Heuer who's a collider man from DESY and that's reassuring in itself, as under his watch DESY didn't blow up Hamburg, said in an interview this summer "The LHC is a much better understood machine than it was a year ago." That's refreshingly honest and so must be true.

I'd say knowing Germany and the people there well, not only do Germans make great cars and drive fast responsibly, they have inherited a mantle of caution from their unfortunate 20th Century history. They really do want to avoid another cataclysm for themselves and the world.

If you've been following LHC protests, most of them have been coming from Germany, Austria and Switzerland along with the leading critics of the LHC like Dr Otto Rössler. There's still a suit before the ECHR, the European Court of Human Rights brought forward last year by them and other Europeans to stop the LHC. And now I've been informed by Markus Goritschnig of LHC Kritik that a new human rights complaint will be lodged with the UN soon.

These protests and court actions have been focused on the possible formation of dangerous objects at the LHC, like micro black holes, strangelets and magnetic monopoles with a potential for catastrophic destruction as well as a possible fatal disruption of the Universe's vacuum state. Lately more scientists have been taking an interest in this debate as well as in risk assessment and have been publishing their findings. Do CERN's safety arguments hold water? Not according to them. More on these doomsday scenarios next time and another one of my own, where I discuss the potential for ionization of superfluid helium at the LHC and possible small nuclear events. Sorry Bob McElrath.

But add the real world dangers of conventional collider accidents (like we've had at the LHC and Tevatron but at much higher energies) to the hypothetical production of strange matter etc, then what could happen? An unstoppable event as CERN PR might call it.

I'd still be very cautious if I were at CERN. Beyond 1.05 TeV CERN is obliged to test the LHC on the fly. If they ramp up beam energies to 1.1 TeV without a problem, they should maintain that energy for a month to see how reliable the LHC is before they try collisions at 2.2 TeV. Even at a modest 2.2 TeV CERN probably will see unexpected particles as the Tevatron did at 2.1 or so. If there's another overconfident rush to higher energies and bigger collisions, the LHC could miss some spectacular results. In the worst case scenario CERN could destroy the LHC even at half power. And that's where Dr Heuer wants to start after the preliminaries, at 3.5 TeV.

Good luck you guys!

--Alan Gillis


Jim Tank said...

Great article Alan and excellent research.

The new LHC start up schedule is an improvement over prior "full power as fast as humanly possible" schedules, but hardly cautious nor in-line with at least one well known externally recommended start up procedure. (Section 5 of [1])


[1] Plaga R., PhD (Aug 2009)"On the potential catastrophic risk from metastable quantum-black holes produced at particle colliders."

The Irreverent Buddhist said...


I would be interested to know the energies involved if lead-ion beams are collided at anything like near power. Seen any references?


Alan Gillis said...

For a brief overview of ALICE, A Large Ion Collider Experiment, see Wikipedia,

CERN edits all LHC related articles on Wikipedia, and some information from this one was deleted without any explanation. The main point was the anticipated collision energies of lead ions at ALICE, originally published in this article at 1150 TeV. When this point came up elsewhere CERN did not disagree but said that fewer bunches of lead ions
would be used in collisions, so overall numbers of ions colliding would be far less than in proton collisions. Digging through CERN archives I was able to find a Fermilab doc that confirms the ALICE collision energies were expected to be 1150 TeV.

High Transverse Momentum Physics at the Large Hadron Collider, FERMILAB-Conf-96/432, page 3

"The machine [LHC ALICE] will also be able to accelerate heavy ions resulting in the possibility of Pb-Pb collisions at 1150 TeV in the center of mass and luminosity up to 1027 cm−2 sec−1."

ALICE lead ion collisions will be at much higher energies than the gold ion collisions at RHIC. Currently RHIC's highest energy proton collisions are 500 GeV. The LHC will eventually collide protons at 14 TeV or at 28X more energy and at far greater luminosity. What RHIC's collision energies for gold ions are now, will take more digging through docs. They were 200 GeV.

The Physics of RHIC

A short overview of the Relativistic Heavy Ion Collider at Brookhaven National Laboratory and gold ion collisions.

2009 RHIC Run Features New Energy Milestone for Exploring Proton Puzzle

"For the first time, RHIC is running at a record energy of 500 giga-electron volts (GeV) per collision, more than double the previous runs in which polarized proton beams collided at 200 GeV."

One thing they want to produce at ALICE is real quark-gluon plasma for the first time.

If they can make it, can they contain it before it destroys ALICE? How about micro black holes? We'll see. This is the thing about any experiment, "we'll see". But ALICE isn't a giant corn popper that on the downside might burn some popcorn. The "we'll see" at extraordinary energies is a gamble.

The LHC won't be colliding lead ions anytime soon, but CERN said it would have a yearly 3 month run at them after they ramp up to 14 TeV proton collisions. But CERN keeps changing its mind like physicists and ordinary mortals do.