Smashing Physics review – how the Higgs particle was found

The discovery of the laws that describe what is going on inside atoms has been one of science's greatest triumphs. It was achieved with remarkable speed, subatomic science having begun little more than a century ago as an academic backwater and developed into a quintessential example of "big science".

The first subatomic particle, the electron, was discovered by the Cambridge physicist JJ Thomson in 1897 using desktop apparatus that cost only a few thousand pounds at today's prices. Few people in the outside world knew what he had done, and still fewer cared. It was a very different story in July 2012, when scientists at the Cern laboratory in Switzerland announced their discovery of the latest subatomic particle to show its face, a so-called "Higgs boson" or "Higgs particle", named after the Edinburgh University theoretician Peter Higgs. The total cost this time was a few billion pounds.

This was good value, as Jon Butterworth implicitly argues in Smashing Physics, a delightful account of his life as a Cern experimentalist, based at University College London. He explains why he and his colleagues are so curious about the subatomic world, and gives a vivid glimpse of life on a huge international project in modern experimental particle physics. In the course of the book, we accompany him to Cern, to quite a few meetings, to an interview with John Humphrys and to conferences all over the world.

Guardian readers will be familiar with Butterworth through his lively and well-informed blog, which consists of dispatches from the frontline of particle physics research. He began writing it in 2010, soon after he lost much of his fear of the media, in the wake of the success achieved by his friend Brian Cox in bringing physics to TV audiences of millions. Smashing Physics reads much like his blog, with its conversational tone, its blend of explication and opinion, its self-deprecating humour and the occasional well-directed barb.

Butterworth begins with a brisk account of the standard model, which describes the fundamental interactions that govern the inner workings of atoms, though not gravity (the effects of which are negligible on the atomic scale). The Higgs particle was the model's only missing piece for a long time, and it was crucial to know whether it existed as it was expected to be a clear manifestation of the mechanism that explains why some fundamental particles have mass and are not as insubstantial as light. The theoreticians made several predictions about the Higgs and its behaviour, but its own mass was a mystery, making life exceptionally difficult for the experimenters trying to hunt it down.

To find the Higgs – or to rule out its existence – was one of the aims of the Large Hadron Collider, a huge machine that accelerates protons (sub-nuclear particles) to within a squillionth of the speed of light before smashing them together (hence the book's title). If the particle existed, it should have quickly fallen apart into other particles in ways that experimenters could study. This is much easier said than done: as Butterworth explains, it was always going to be extremely difficult to pin down the particle, as the evidence was expected to be largely – but not completely – obscured by huge numbers of tracks due to other subatomic processes. Several months after the collider was switched on, there was no clear sign of the particle, leading some theoreticians to get cold feet and even to doubt its existence.

Butterworth tells the story of how the particle was eventually tracked down, making clear the extent of the challenge. He is an engaging guide, generous to all his colleagues, especially in the media – "We should be more forgiving of some of the excitable headlines" – but is sometimes a tad harsh on theoreticians. "Experimentalists get ignored if they are right … and hugely cited if they are wrong," he writes, whereas "Theorists are ignored if they are wrong, but get a Nobel prize if they are right." In my experience, theorists soon find themselves on the scrapheap if they are trivial, let alone wrong.

It is hard to come up with truly innovative ideas that are not ruled out by combining the two great underpinning theories of relativity and quantum mechanics, and by the experimental data that support them. Yet it is plain that the standard model will not be the last word on subatomic physics – we need a better-understood, more comprehensive theory that can describe all the fundamental forces, including gravity. The only viable candidate for this is string theory, which has huge potential but "struggles to predict anything remotely measurable", as Butterworth puts it. He is one of many who disparage the theory, though I suspect he and his fellow sceptics will be proved wrong in the long term and that physicists will one day reap the benefits of the work now being done on string theory, even if it is superseded by another approach.

Like all good experimenters, Butterworth keeps his feet firmly on the ground. He was a "confirmed Higgs sceptic" until the closing weeks of 2011, but, after seeing a new display of data a few weeks later, he changed his mind – "I knew in my guts we had it." It is fascinating to read how the particle's existence was eventually demonstrated using a tiny proportion of telltale tracks in the detectors. Butterworth describes the problems he and his colleagues encountered as they repeatedly checked their analyses to avoid any possibility of career-ending mistakes. Only when the discovery of a new particle was beyond reasonable doubt did the Cern authorities announce it, triggering worldwide media "Higgsteria".

The discovery led to the sharing of last year's Nobel prize for physics by Higgs and his fellow pioneer François Englert. They certainly deserved it, but, as Butterworth says, it is a pity that the experimenters who actually proved the theoreticians were right have not yet had a share of the Nobel glory. I, for one, hope that the experimentalists – or, at least, Cern – will be rewarded with Nobel honours after they have made another exciting discovery, preferably one that the theoreticians did not expect. Theoreticians have been calling the tune in particle physics for too long.

In a bravura passage towards the end of his book, Butterworth extols the virtues of this kind of research and bewails the attempts by some bureaucrats to guess its impact on society in advance. The point of doing fundamental particle physics is – like great art – not specifically to improve our material wellbeing, but to enrich our understanding and appreciation of our place in the world. The spinoffs – including Cern's invention of the world wide web (Tim Berners-Lee was a scientist there) – are a bonus. All in all, it is plain that the UK gets excellent value from its support of this basic science, at an annual cost to each of us of a fancy cappuccino.

When reading Smashing Physics, I found myself reflecting on how much the world of subatomic physics has changed since its earliest days. Butterworth shows that the conventional image of experimenters working alone in a laboratory has long been superseded by huge international teams, in which individuals struggle to make their mark. Butterworth himself comes across as both a team player and a gifted individual, capable of doing first-class research; he is also a first-rate populariser. This rare talent is handsomely on display in this charming, enlightening bulletin from one of the most exciting fields of human endeavour.