Horizon (1964) s55e03 Episode Script
The Mystery of Dark Energy
The universe is falling apart.
Something is forcing galaxies to rush away from each other at ever increasing speeds.
Ever since this alarming discovery, physicists have struggled to understand what might be causing it.
So far, they've come up with a name.
They've called it dark energy.
Dark energy is basically our name for that thing that we don't understand.
It's not the colour dark, it's just the expression of our ignorance as to what is this stuff.
The discovery of dark energy really surprised theoretical physicists and remains a deep mystery of nature.
Until dark energy, we had every reason to be as confident in Einstein's theory of general relativity as he was himself.
In the last few days, I've completed one of the finest papers of my life.
But now things are less certain.
Einstein doesn't explain dark energy and, so far, neither has anyone else.
We are absolutely still lacking great ideas.
So, it is crying out for some new breakthrough, new thinking.
Because it might be that dark energy is not a thing at all, but simply evidence that the physics itself is wrong, that we need another Einstein.
What Einstein did was he actually came out and looked at the bigger picture and put all these different elements together to come up with the theories that he had.
It might be time for the bigger picture to be re-evaluated.
There's definitely room for another Einstein to come in and shake everything up and tell us that we've been looking at things completely wrong up until now.
Would-be latter-day Einsteins the world over are on the hunt for answers to modern science's most enduring problem - to paint the biggest picture of all, to finally solve the mystery of dark energy.
Energy is all around us.
It comes from the sun, from chemical reactions, from electricity.
Energy powers our vehicles, heats our homes, lights our nights.
Understanding energy has transformed our planet and our lives.
Dark energy is something altogether different.
It seems to serve no useful purpose at all, except to show us that we understand less than we thought we did.
Dark energy arrived wholly unexpectedly at the very end of the 20th century.
In 1998, a young scientist called Saul Perlmutter was thinking some very big thoughts indeed.
As a graduate student, I really wanted to find a project that would answer some, or that would at least be looking at some, very philosophical questions, something that felt like it was meaningful about the world we live in in some, you know, deep way.
The question that's been really exciting me is whether the universe will last for ever.
Do we live in a universe that is infinite, or, some day, will it come to an end? The two big options at that time were that the universe could expand for ever, but just slow and slow and slow, but forever be expanding.
Or, if there was enough stuff in the universe to gravitationally attract it, it could slow to a halt and then collapse and come to an end.
Saul was measuring the way the universe was expanding by observing exploding stars called supernovae.
One particular kind of supernova always explodes the same way because it waits until just a critical amount of mass has fallen on it and then it explodes, so they all look very similar to each other.
They brighten as a firework and fade away, and they reach the same brightness and you can then use that as an indicator of how far away it is, by just looking to see how bright it appears to you.
Because they explode with exactly the same intensity, these supernovae are known as standard candles.
By comparing their relative brightnesses, relative distances can be calculated.
Saul expected the stars to show what everyone thought at the time - that the universe was slowing down.
Fainter ones are further and just like when you watch a car recede into the distance, you can tell how far away it is by how faint the tail-lights look.
If you can use the brightness of the supernova to tell you how far away it is, that's really telling you how long ago the explosion occurred because you know how long it takes for light to travel that great distance.
So, now we have an object where it explodes and its brightness tells you when it exploded, how far back in time it exploded.
No matter how good the theory, the practical problem of catching an exploding star at just the right time is immense.
But Saul and his team applied the very latest computer technology to the problem.
Yeah, we think it may be the scuzzy chain is too long.
Now there's two switches on the back of it.
We had the computers go down, the computers came back up again, but now, finally, we have the analysis completed, at least the computer's part of the analysis, and it's beginning to show us on the screen what it thinks might be a supernova.
Eventually, after the team had identified 42 such dying stars, the calculations began.
What Perlmutter discovered shocked him.
The data was telling the wrong story.
The universe didn't appear to be slowing down.
We thought that that's what we would see and it looked like the opposite was taking place and, in fact, the universe was speeding up in its expansion.
These distant supernovae were fainter than you would have thought and fairly significantly fainter.
They were probably 20% or more and that's the hallmark of a universe that's actually speeding up in its expansion.
To say that this result was a surprise would be a masterclass in understatement.
It was so unexpected that the initial reaction was disbelief.
Everybody knew Saul and everybody knew the experiment he was doing and I remember sitting in the audience and Saul getting up and expecting him to present an update on the results he'd given a year ago, that actually the universe was slowing down.
And so, I was absolutely amazed that, based on only twice as many objects as he had the year before, that suddenly he was saying that we lived in a universe that was accelerating.
I remember it just being just incredible.
I mean, all the astronomers walking around scratching their heads saying, "This can't be right.
Surely it can't be right?" It was not the result that people had been expecting and such an extraordinary claim demands extraordinary evidence, more than a few data from a handful of stars.
So, here we are on a beach where there is about a billion pebbles and if you think you're trying to understand this beach, you wouldn't think you could understand it from 42 pebbles.
But Saul was right.
He was able to work out that the universe was accelerating just from 42 supernovae, which is quite incredible when you think about it.
On the one hand, this was a good result.
It was new science and produced a Nobel prize for Saul Perlmutter.
On the other, it raised an obvious question.
Once you know that the universe is actually speeding up, then you're faced with the question of - well, what could make it speed up? So far, the only real progress on that question has been to give the phenomenon a name.
It's become known as dark energy.
Dark energy is just the term we use to describe whatever it is that makes the universe accelerate in its expansion, what makes it expand faster and faster.
We don't know what that is.
It's a mystery and so we call it dark to reflect our ignorance, not because the colour is dark.
The mystery is so deep, so beguiling, that wherever there are physicists, there are people hoping that they will solve the mystery of dark energy.
People safe in the infuriating knowledge that what they're looking for, if it's there at all, is all around them.
But the fact that no-one has yet been able to identify what the dark energy might actually be has opened a can of worms not seen in science since the last time a physicist got involved in cosmology.
In 1915, it seemed that the work of physics was nearly at an end.
Everything made sense.
Newton had explained the heavens by invoking gravity and atoms had been identified as the smallest indivisible units of matter.
Job done.
But then, a German man, given to musing on trains, turned up with a totally new set of ideas.
I very rarely think in words at all.
A thought comes .
.
and I might try to express it in words afterwards.
Einstein called these little flights of fancy his "thought experiments" and they would lead him to develop his theory of general relativity, which totally changed how the workings of the universe were understood.
I sometimes ask myself how did it come that I was the one to develop the theory of relativity? The reason, I think, is that a normal adult never stops to think about problems of space and time.
These are things which he had thought of as a child.
But I began to wonder about space and time only when I had already grown-up.
Einstein's theory held that Newton's ideas about gravity, though empirically correct in most cases, were, in fact, conceptually wrong.
Gravity, said Einstein, was not some nebulous attracting property of mass as Newton supposed, but was, in fact, a consequence of mass interacting with space-time - the gaps around stars and planets previously known as space.
According to Einstein, space isn't simply a void.
It's more like a four-dimensional fabric woven from both space and time.
The mass of planets can warp and distort the fabric, gathering other celestial objects, like moons, around them.
And it's this bending of space-time that creates the effect we experience as gravity.
So, Einstein's theory of general relativity is a beautiful theory.
It's incredibly elegant and has been now around for 100 years.
It's very predictable.
You can write things, make predictions of what the universe should look like and what objects should look like in the universe, and we can test those, and as far as we can tell, it's passed every test.
The power of general relativity is that, like Newton's version of gravity before it, it's predictive.
Bizarre as the curvature of space-time may sound, it's eminently testable - a fact not lost on Einstein himself.
I have now come to realise that one of the most important consequences of that analysis is accessible to experimental test.
Accordingly, a ray of light travelling past the sun would undergo a deflection amounting to 0.
83 seconds of arc.
In 1919, that prediction was actually observed.
British astronomer, Arthur Eddington, pointed his telescope at a patch of sky near the sun during an eclipse and observed a star, known to be actually out of view, behind the sun.
Its rays of light had been bent by the distorted space-time created by the sun's mass.
Einstein's theory had held up.
A paradigm had shifted and the crowd went wild.
Einstein was suddenly famous.
Undoubtedly the cleverest, yet most incomprehensible man on earth.
This strange world is a madhouse.
Currently, every coachman and every waiter is debating whether relativity theory is correct.
This mass excitement about my theory is to do with the intriguing mystery of incomprehensibility.
I'm certain that the mystery of not understanding is what attracts people.
This is what all the fuss was about.
This is the equation that the coachmen and waiters were discussing.
On the one side, the geometry of space-time.
On the other, the mass and energy of the universe which acts on it.
Not incomprehensible at all(!) At least, not to its author.
But there was one aspect of general relativity that Einstein himself didn't understand.
The problem that Einstein had is that when he solved his equations of general relativity, what he found was that he predicted that the universe should actually be expanding.
And that was radically different from the perceived wisdom at the time, which was that we lived in a static universe, both static in time and in space.
So, he put in an extra term into the equation.
He called it the cosmological constant.
He used the Greek variable lambda, but, effectively, it was just what a physics undergraduate would call a fudge factor.
It was just designed to make the equations come out right and it would just make the universe sort of stand still.
The problem is is in science, we'd call that fine tuning.
And you only have to change the value of the constant by a small amount and suddenly, you get back these expanding solutions.
The general theory of relativity requires the universe to be spatially finite, but this view of the universe necessitated an expansion of equations with the introduction of a new universal constant lambda standing in fixed relation to the total mass of the universe.
This is gravely detrimental to the formal beauty of my theory.
When you add the lambda term, it means that the equation is not quite as simple as it was before.
So, in that sense, it's not as beautiful as an equation.
The static universe was restored, but Einstein always felt he'd added lambda against his better judgment.
Dear Egrenfest, I have perpetrated something in gravitation theory which exposes me a bit to the danger of being committed to a madhouse.
Despite the fudge factor, lambda, the cosmological constant, Einstein continued to be celebrated as the world's cleverest man.
Until, in 1929, he became even cleverer.
In America, astronomer Edwin Hubble was about to get a reputation for scientific cleverness himself.
He'd been using the world's largest telescope at Mount Wilson in California to peer deeper into space than anyone had ever looked before.
What he discovered completely changed the meaning of the word "universe".
Until Hubble, it had been thought that the universe was our galaxy.
What Hubble saw was that in fact our galaxy is just one of countless millions, but more importantly, that all these galaxies were moving apart from each other.
The universe wasn't static after all.
This had huge implications.
It introduced the notion of a beginning and an age for the universe.
But more importantly for Einstein, it meant that he could ditch his fudge factor, the cosmological constant, and return general relativity to its former glory.
The lambda that he added to create a static universe was no longer required, once it was observed by Slipher and Hubble that the universe, in fact, was expanding, so if, in an expanding universe, at the time, the observations could be described without the lambda term and so he removed it.
Einstein was cock-a-hoop.
In 1931, he went to Mount Wilson to shake Hubble's hand and thank him for putting beauty back into his equation.
Lambda, he later confessed, was the biggest blunder in his career.
I think that the reason that he said that it was a blunder was because if he had just not introduced that term, then he would have said that the universe must be expanding and done that 14 years before the discovery of the expansion of the universe by Edwin Hubble, which would have been a great achievement.
But despite Einstein's blunder, general relativity has stood the test of time.
It is perhaps the single most successful scientific theory yet.
Every observation we make of gravity, from the smaller scales to solar system scales to galactic scales, all the way to the universe, all of that can be described using the single theory that Einstein created.
So, it's the most successful and beautiful theory we have of our universe.
Or at least, it was.
For all its beauty and simplicity, general relativity doesn't account for the effects of dark energy.
Expansion, as reported by Hubble, works fine, but the accelerated expansion of the universe that Saul Perlmutter found isn't part of the deal.
That it's there at all is bad enough, but worse still, the way that dark energy seems to work is unlike anything that's been observed before.
The density of anything is the amount of stuff you have within a given volume and dark energy is an unusual phenomenon, in that even though the volume of the universe is increasing as it expands, the density is staying the same, constant.
So, imagine you had say like half a cup of black coffee, and then you started adding milk to it, and as we pour more and more milk into that cup, then the volume of the liquid's getting larger and larger, but the density of the coffee is going down, so the coffee would be getting lighter and lighter as you added more and more milk.
But dark energy doesn't behave that way, so it's almost as if there's new dark energy being created all the time, as the universe expands, meaning that its density remains the same.
Constant.
So, you can think of it, as you get more space, you actually get more dark energy, which is like getting something for nothing, which is clearly ridiculous.
It's clearly against all our training as physicists.
There is one way to adapt general relativity to cope with this magically constantly self-replenishing force and that is to simply add it to the equation.
100 years after Einstein's "biggest blunder", the cosmological constant is back.
Lambda is being written once more.
This time, not to keep the universe still, but to account for its unexplained accelerating expansion.
The values are different, but the concept is exactly the same.
All this leads cosmologists to one of two equally alarming conclusions - either we need another Hubble, or we need another Einstein.
But before we consign Albert to the scientific scrapheap, there is a branch of physics which might help.
An area where things popping in and out of existence is quite normal.
This is the strange and wonderful world of Clare Burrage and of quantum mechanics.
Quantum mechanics is the theory of what happens to really, really small things.
It's a theory of how the fundamental particles in the universe work.
Atoms, electrons, protons.
And quantum mechanics is intrinsically uncertain.
Einstein hated quantum mechanics.
He disliked the probabilistic "now you see it, now you don't" nature of the idea.
"God," he famously declared, "does not play dice.
" In 1930, he paid a visit to Nottingham, where Clare now does her research.
He didn't say much.
He probably didn't say anything about quantum mechanics.
He came to give a talk on general relativity.
His actual chalk writing is preserved for devotees to marvel at.
But even though Einstein didn't like it, quantum mechanics could shed light on dark energy and come to the aid of his once-more-under-fire theory.
In theory.
Quantum mechanics tells us that particles can come in and out of existence in the vacuum.
And the fact that those particles have mass and potentially are moving around, they have a little bit of energy.
And so, when they pop into existence, they give a little bit of energy to the vacuum and yes, they disappear again, but the fact that that process is going on all of the time means that there is some energy stored in the vacuum.
And because Einstein told us that energy and mass are the same thing, having lots of energy stored in space affects space-time that cause the expansion of the universe to accelerate.
So, it seems that quantum mechanics should, in theory, be able to explain how the cosmological constant works.
And how dark energy appears in the vacuum of space and is driving the acceleration of the universe.
But there's a problem.
When they came to calculate this vacuum energy, they discovered how spectacularly wrong they were.
If you were to say there was one pebble on this beach, you'd be wrong by one part in a billion.
If you were to say there was one particle in the universe, you'd be off by ten to the 80.
But the vacuum energy was calculated to be off by ten to the 120.
That is a google.
That is spectacularly wrong.
The fact that our predictions are so far off from what we see tells us that there's something fundamentally missing in the way that we understand physics, that we understand the world around us, so there's still a mystery, still a puzzle there.
It might be tempting to simply ignore dark energy.
You could argue that the apparent accelerated expansion is, in fact, a trick of the light, that it may be a function of other inaccessible dimensions at play.
That it just looks like dark energy, but is actuallysomething else.
But dark energy isn't just an irritating threat to Einstein's beautiful equations.
It's also a very practical solution to a fundamental question in cosmology, namely - what is the universe made of? When Einstein was busy thinking about gravity on trains, the answer was simple.
The universe was made of the same stuff that you and I are made of, the stuff of stars, planets, Coke cans, tennis rackets, atoms, made, in turn, from electrons, protons and neutrons.
But physics was about to get a shock.
It turned out that there was something else out there that the universe was also made of, matter of a different kind.
In 1975, an astronomer called Vera Rubin made an unexpected discovery.
If we plot the velocity of the planets as a function of distance from the sun, Mercury, Venus, Earth, Mars, Jupiter, Saturn, Uranus, Neptune, Pluto, and you can see that Mercury orbits much more rapidly than Pluto.
The graph is called a rotation curve.
It is the embodiment of the law of gravity.
The further away you travel from the sun, the weaker its gravitational force.
Galaxies work in the same way as our solar system.
Except that instead of planets orbiting a central sun, in a spiral galaxy, stars are held in orbit by a gravity-providing black hole.
Vera decided to plot the rotation curves in galaxies.
She trained her telescopes on Andromeda, the galaxy closest to our own.
I came out with sets of numbers and I plotted them on pieces of paper and I discovered that the stars as you went further and further out did not slow down, they were moving just as fast as the stars near the centre.
We find that the velocities remain flat all the way to the edge of our observations.
And that was a surprise.
And a surprise that had to be explained.
By rights, the stars should have flown off into space, but they didn't and wherever spiral galaxies were measured, the same flat curves appeared.
It was decided that the only explanation was that there must be more stuff out there that we couldn't see, providing the extra gravity, holding the galaxies together and flattening the curves.
They called this stuff dark matter.
The new dark matter was a shock, in more ways than one.
The very fact of its existence was almost overshadowed by the fact that when the calculations were made, this new form of matter outweighed the atomic form of stuff by about 90 to one.
In the 1980s, when new ways of measuring dark matter were developed, it was discovered that there simply wasn't enough of it to make the universe work as it clearly does.
The universe was short of stuff to the tune of about 70%.
Cosmology scratched its head.
Then, in 1998, a young scientist called Saul Perlmutter was thinking some very big thoughts indeed.
Something that felt like it was meaningful about the world we live in, in some deep way.
The universe was speeding up in its expansion.
The dark energy that earned Saul his Nobel Prize was an interesting and troubling concept, but it also had a number and that number was highly significant.
We know from Einstein - him again - that energy and mass are related, that energy, E, equals mass times the speed of light squared.
E = MC2.
Plug dark energy into that equation and you get the missing mass that dark matter couldn't account for.
The universe was complete.
It was made about 4% baryonic matter, the stuff that we're made from, 25% dark matter and the gaping 70%-sized hole was filled with dark energy.
So far, despite heroic efforts to find it, and overwhelming evidence that it exists, no-one has identified what dark matter is.
And, of course, dark energy, both useful and confounding, is barely in its infancy when it comes to a convincing explanation.
There's radiation damage.
We may not be quite there with the shielding yet to provide the right radiation environment.
But there is an idea in cosmology that dark matter and dark energy may be linked by more than just a common adjective and if they are, a new European spacecraft called Euclid may shed light on what that link might be.
The Euclid Consortium is staffed by 1,200 scientists from 14 countries.
These are some of them having their picture taken at their annual conference in Lausanne.
They're hoping that by taking pictures of the universe, they'll be able to work out how it's expanded over its lifetime and that by determining that, the nature of dark energy will become clearer.
The way we think about it is that it's either some new stuff in the universe, some particle or even just a new field that you put in to the universe to explain the properties of the universe.
Alternatively, you could say that the equation you wrote down is not correct.
It's not wrong, but we like to say it's "incomplete".
So, you could sort of fiddle with the mathematics of the equation, so actually what you could do is maybe come up with a natural explanation for it.
So, Euclid should be able to tell us which of those alternatives it is.
The satellite will launch and start sending data back to Earth in 2020.
The all-important camera for the Euclid space telescope is being built and tested in the UK, in this country house in the Surrey Hills.
These are going to be the biggest images that come down from in orbit.
You have an image of 625 megapixels, so that's roughly 300 HD television screens full of data and that comes down every ten minutes.
This imager takes roughly the same amount of data that Hubble has taken and will take in its entire lifetime in one day.
It's an astonishing leap forward, given what's available from current space telescopes.
The huge data sets will provide information from the universe in almost every direction.
The Hubble space telescope is the biggest and best telescope we have available at the moment and the amount of sky that's covered is about the size of your little fingernail, if you held it up at arm's length.
And Euclid will do the same type of imaging as the Hubble space telescope, but instead of just covering that small patch of sky, it will cover practically every bit of sky that you can see.
Not only will Euclid be able to measure the historic acceleration of stars and galaxies in all directions, it's hoped it will also provide data about how dark matter around galaxies has expanded over time.
This is possible because of an effect called gravitational lensing.
So, in general relativity, mass bends space and time and then light is bent around large massive objects, just like Eddington measuring the star behind the Sun, and so, we used the same technique for Euclid.
I can illustrate it using this wine glass and this image of the universe, so as we draw the wine glass across the image, what you see is that the galaxies behind the wine glass get distorted and that distortion is caused by the lens.
In general relativity, the lens is mass, because it bends the light.
And that can be shown in this picture.
You have a large clump of mass here, which is like the lens, like the bottom of the wine glass, and what you can see are all the distorted galaxies behind that lens.
And what you could do with an image like this is you can calculate how much mass would I need within the lens to create the distortions that I see and what you find is quite remarkable.
What you find is that there is about 100 times more mass here than you see from the light in the image and that missing mass, that mass you cannot see, is what we call dark matter.
So, Euclid will make an image of the whole sky at this resolution and it will find all these distorted background galaxies and from that, it can infer the distribution of dark matter in the universe.
Euclid will compare lensing all over the universe and by doing so, will help paint an accurate picture of how the universe is tearing itself apart under the influence of dark energy.
So, Euclid may tell us that it's the cosmological constant and then we have to explain that, it might tell us that our theory of gravity is not complete, and we'd have to explain that, it could tell us that actually the dark matter and dark energy are two sides of the same coin and that actually there might be a unified dark sector, but we'd have to explain that.
It could be another theory that we haven't even come up with yet.
And so Euclid will give us a coherent data set that we can test all these theories against.
Whatever the case, the devil's in the detail, and these days, the detail can be interrogated to degrees not thought possible when Einstein first reluctantly inserted his cosmological constant into general relativity.
Cosmology is one of the fields that is actually pushing the boundaries of cosmology itself, but also statistics and computing.
It is the frontier, I think.
Euclid will be pushing the boundaries like never before.
It will stream more data from space than has ever been processed in the past.
In the end, it will have about one and a half billion galaxies.
It will observe one and a half billion galaxies, so it's huge.
And a lot of the time, your eyes cannot just pick up patterns, so this cannot be possible without computers and statistics.
The computer-aided searches should give unprecedented clarity on how science should be thinking about dark energy.
There will be winners and losers.
The amount of data that we have on dark energy hasn't been enough to be able to tell us which path we have to go down, so we have lots of theories and hundreds of models that could still fit our data.
When Euclid comes, lots of these can be thrown away and it could narrow down the possibilities of what this dark energy is.
Euclid is not the only show in town when it comes to mapping the expansion of the universe.
At Kit Peak in Arizona, Risa Wechsler is hoping to use the proposed dark energy spectroscopic instrument, DESI, to make a map of part of the universe, like this one.
But 100 times more accurate, so that she can check the validity of computer simulations of the universe that she's created.
One of the things that I do is try to simulate the entire universe and tie what we think about the physics of the evolving universe to what we actually see with surveys like DESI.
What we're trying to do in these simulations is take a whole bunch of hypothetical universes, some of them will have a cosmological constant, some of them will have a different time evolving dark energy, some of them will have more or less amount of dark matter, and then when we compare that to what we actually see, we can rule out a lot of these ideas, so some of them will not be consistent with what we measure and then we can determine that that's not the universe we live in.
When DESI starts producing data in 2020, it might be that one of Risa's simulations strikes gold.
It'll be up against a lot of competition.
In the absence of hard data, this is boom time for theories.
Multi-Galileons, ghost condensates, and the higher co-dimensional brane worlds theory jostle for attention in the race to explain dark energy.
Many of these theories usually try to provide a global solution to the dark energy problem, a fix to general relativity, but Clare Burrage is working on an idea that says that Einstein may have been both right and wrong at the same time, depending on where you are.
We know that Einstein's theory works very well on Earth and in the solar system.
We've tested it and it works phenomenally well.
But we don't have ways of testing that theory on the kinds of distance scales that are relevant to cosmology and so it could be that whilst relativity is a good description of what's happening around us, it doesn't work as a description of the universe as a whole system and maybe you need to change the theory.
Clare's solution involves something called a chameleon, a particle that tries to blend in not by changing colour, but by changing how it exerts its force.
There are two types of particles in the universe.
There are the ones that make up matter, like electrons and protons and neutrons and quarks, and then there's another set of particles, and those are the ones that transmit forces.
So, for example, the photon, which makes up light, also carries the electro-magnetic forces.
It's exactly like what we're doing with the ball and the magnet.
We don't see the photons transmitting the force directly but we see the fact that the magnet makes the ball move.
In physics, the greater a particle's mass, the smaller the distance over which it's able to exert any force or field it might have.
The mass of a particle tells you how far it can carry information.
If a particle that's transmitting a force is heavier, it only transmits the force over a shorter distance scale.
So the range that you can transmit the force over changes depending on where you're looking.
The idea is that when the chameleon comes into contact with other stuff, it interacts with it and becomes heavy and its force-transmitting capability all but disappears.
But in regions of deep space where there's very little in the way of anything, the chameleon has no stuff with which to interact and so is very light and can transmit its force over vast distances.
It's a neat idea, but evidence is hard to come by.
Then, in 2014, Clare came up with an experiment that might unmask the chameleon.
The experiment that we proposed last year is that you'd specifically design your experiment to look for chameleons, which means that you look in a very high vacuum and you use tiny, tiny particles, so we're using individual atoms.
But wrangling individual atoms isn't easy.
It takes an enormous amount of scientific hardware, specially configured in a highly precise way.
Just six months after Clare's paper was published, atomic physicist Holger Muller got in touch.
As it happened, he explained, he had exactly the right equipment needed to perform Clare's experiment, right here in Berkeley.
From where, in 1998, Saul Perlmutter's group discovered dark energy in the first place.
It might be that the conundrum could be solved at the same institution that it was discovered.
We've been setting up this experiment for several years when my post-doc colleague came across Clare Burrage's paper on the pre-print server and we read the paper and we found that, "Wow, they're describing "exactly the experiment we've been building for all these years.
" And we got excited about it so we stopped doing our original experiment and started doing the dark energy measurement.
It's an amazing feeling to have that kind of quick response because it almost never happens like that in science.
Things take a long time to go from theory to somebody actually doing an experiment.
To have a measurement of something we proposed in the space of six months is phenomenal.
The experiment involves using a vacuum chamber and a cunning chameleon trap.
The animal, the chameleon, changes its colour in order to hide, right? And in the same sense, the chameleon particle changes its mass in order to hide.
At the centre of the vacuum chamber is a marble-sized sphere.
If there are chameleon particles around, they will interact with the mass of the ball and produce very little in the way of force but perhaps just enough to affect something very small like an individual atom.
The heart of the experiment is this little sphere inside there and so the experiment works by first collecting a cloud of caesium atoms on top of the sphere so here's the sphere and about one centimetre on top there's a little cloud of about 100 million caesium atoms.
The machine contains the atom cloud using infrared lasers, invisible to the naked eye.
The beams need to be very precisely controlled, so they're sent around a complicated series of mirrors on what's known as an optics table.
We use lasers to control the atoms and so to do that, we need to pass them through this table of optics.
The laser beam kind of takes a snake-like path throughout all these optics but eventually gets into something like this, an optical fibre.
The light can then travel through here and is brought over to interact with the atoms.
So, you see here the sphere and we trap the cloud of atoms just on top of the sphere, and then we release the trap and the atoms are free to fall subject only to the Earth's gravity and the potential chameleon force.
When the atoms are released, they will fall towards the ball, which will contain chameleon particles, if they exist, and if they do exist, they will be busy interacting with the mass of the ball, making themselves heavy and reducing the distance over which their force can be felt.
Which isn't to say that the force is completely non-existent.
According to chameleon theorists, there'll be a tiny region on the surface of the sphere where the force is active and given that the atoms are so tiny, they will be affected by that force.
If it exists at all.
All the team need to do is to precisely measure the difference between the speed the atoms fall with and without the ball in place.
Now we want to measure the chameleon only and not the combination of gravity and the chameleon, so what we do is we will move this sphere out and then do the measurement again, this time measuring gravity only.
The experiment is set up to compare the difference between how fast the atoms fall towards the ball and how fast they accelerate through empty space.
If the tracking reveals an unexplained acceleration, this could be due to the force associated with the chameleon particle.
The experiment has now been running for over six months and they're starting to get their first results.
Right now we have seen no evidence for chameleons, which means they either don't exist or they hide in a region of the perimeter space that we can't yet measure.
So, what does that mean? If either the chameleon force is extremely weak or it's even heavier than we thought, then we can't see them.
The team at Berkeley are now adjusting the experiment to rule out any theoretical nooks and crannies where the chameleon might be hiding.
Well, if we make the experiment more and more and more sensitive, we will either discover the particle or rule it out once and for all.
A scientist might be like a drunk who lost his keys and is now looking for it under the next lamppost .
.
and it's not because he knows that the key is there, but because it's hopeless to look in the dark anywhere else.
People have searched for dark energy in cosmology and in astrophysics, and now we start looking for it under the atomic physics lamppost.
Whether this is a good idea or not, we will know in a couple of years when it has either been found or not, but it's always exciting to have It's like a new window that you can open and look through and you don't know what you will see before you've tried to do it.
Having more information is always a good thing so ruling out possibilities.
Although on a personal level maybe it's a little bit upsetting because it's a nice theory but it means that you've got more information and you can go on from there and build something better, build a better theory.
With all these theories, it's really a question of taste.
You either like a cosmological constant or you can explain it through a chameleon effect.
None of these as yet give us the elegant solution that we are looking for and that's what really we're looking for.
We're looking for this simple, elegant solution to this strange accelerated universe and nothing yet has given us that.
Chameleon? Yeah, maybe, but as yet, there's no evidence for it.
Dark energy? Yeah, we can sort of understand it, but we can't get the number right so we're still grasping in the dark for an elegant, simple solution to what we see.
Where that simple solution will eventually come from is anyone's guess.
That is one of the infuriating things about science.
It can't always produce the rabbit from the hat on time and on budget.
Sometimes it takes an unexpected turn of events, or what the media like to call a "genius".
Though the geniuses themselves have a rather different take on their exploits.
I'm not more gifted than anybody else.
I'm just more curious than your average person and I will not give up on a problem until I have found the proper solution.
I think that curiosity is what driveswhat drives most cosmologists and physicists, a curiosity about the universe - why? What is the universe made out of? Why are we here? How did the universe begin? What will happen to the universe in the future? All of these are questions which are driven by curiosity.
I have no special talent.
I am only passionately curious.
Curiosity, I think, is Well, it's the best motivating force, OK? Working hard doesn't necessarily get you to an answer.
Working too hard can actually stifle creativity.
With our work, you know it's a mixture of inventiveness and persistence in the hard work.
It's a combination.
It's the end of the Euclid conference in Lausanne.
The conference organisers have laid on a social evening, cruising around Lake Geneva.
It's a chance for the delegates to unwind and maybe even think a little about the biggest picture of all.
Yeah, so Einstein's theory was motivated for a reason, right? He had an equivalence principle.
Yeah, and, I mean, we're going to measure a lot of things about the nature by looking at how it evolves, how dark energy actually evolves with red shift.
But the problem is the zero-point energy, the vacuum energy, the quantum mechanical part that you add there.
Try and study the nature of dark energy and at the same time, try and test if general relativity works.
So, there's like a lot of work and a lot of discoveries that are going to happen down the road.
- Exactly.
- And I'll drink to that.
Exactly that.
The process of scientific discovery sometimes makes progress through sheer hard work and sometimes it needs someone to take an inspired alternative view.
We learned an awful lot about animals and plants by simply observing them, but it took Darwin, with a radical idea, to give us a context to understand life itself.
And in our efforts to understand the wider world and even the universe, observations are critical.
The ideas of dark matter and dark energy come courtesy of people watching stars but just as Einstein musing on his train managed to take all the known science and see it from a different, more useful, angle, it might be that to solve the dark energy problem, someone needs to pull off a similar trick and come up with an even better idea.
There are an awful lot of very smart people in the world.
I wouldn't be surprised if we end up with another Einstein, you know, somewhere along the line here.
I don't know whether it'll be in our lifetime but we I think we have a good shot at it.
We need teams like Euclid.
That's the only way you can get the data that you need.
But to understand that data, to give it some interpretation, to give it an idea, could come from one person.
That could be the next Einstein.
A genius could come up and put all the observations that we have so far, put it together, and come up with a new theory.
Yeah, it is quite possible.
I'm kind of hoping it's me.
The tantalising truth is that all it might take to solve the mystery of the dark energy is one big idea, for someone out there to see things differently, someone perhaps like you.
And if that new Einstein is you, if you manage to solve the mystery of dark energy, you're likely to become very famous indeed, as famous as the original Einstein.
Wherever I go and wherever I stay, there's always a picture of me on display.
On top of the desk or out in the hall, tied round a neck or hung on a wall.
Women and men they play a strange game, asking, beseeching, "Please, sign your name.
" From the erudite fellow they brook not a quibble, but firmly insist on a piece of his scribble.
Sometimes, surrounded by all this good cheer, I'm puzzled by some of the things that I and wonder, my mind for a moment not hazy, if I, and not they, could really be crazy.
Something is forcing galaxies to rush away from each other at ever increasing speeds.
Ever since this alarming discovery, physicists have struggled to understand what might be causing it.
So far, they've come up with a name.
They've called it dark energy.
Dark energy is basically our name for that thing that we don't understand.
It's not the colour dark, it's just the expression of our ignorance as to what is this stuff.
The discovery of dark energy really surprised theoretical physicists and remains a deep mystery of nature.
Until dark energy, we had every reason to be as confident in Einstein's theory of general relativity as he was himself.
In the last few days, I've completed one of the finest papers of my life.
But now things are less certain.
Einstein doesn't explain dark energy and, so far, neither has anyone else.
We are absolutely still lacking great ideas.
So, it is crying out for some new breakthrough, new thinking.
Because it might be that dark energy is not a thing at all, but simply evidence that the physics itself is wrong, that we need another Einstein.
What Einstein did was he actually came out and looked at the bigger picture and put all these different elements together to come up with the theories that he had.
It might be time for the bigger picture to be re-evaluated.
There's definitely room for another Einstein to come in and shake everything up and tell us that we've been looking at things completely wrong up until now.
Would-be latter-day Einsteins the world over are on the hunt for answers to modern science's most enduring problem - to paint the biggest picture of all, to finally solve the mystery of dark energy.
Energy is all around us.
It comes from the sun, from chemical reactions, from electricity.
Energy powers our vehicles, heats our homes, lights our nights.
Understanding energy has transformed our planet and our lives.
Dark energy is something altogether different.
It seems to serve no useful purpose at all, except to show us that we understand less than we thought we did.
Dark energy arrived wholly unexpectedly at the very end of the 20th century.
In 1998, a young scientist called Saul Perlmutter was thinking some very big thoughts indeed.
As a graduate student, I really wanted to find a project that would answer some, or that would at least be looking at some, very philosophical questions, something that felt like it was meaningful about the world we live in in some, you know, deep way.
The question that's been really exciting me is whether the universe will last for ever.
Do we live in a universe that is infinite, or, some day, will it come to an end? The two big options at that time were that the universe could expand for ever, but just slow and slow and slow, but forever be expanding.
Or, if there was enough stuff in the universe to gravitationally attract it, it could slow to a halt and then collapse and come to an end.
Saul was measuring the way the universe was expanding by observing exploding stars called supernovae.
One particular kind of supernova always explodes the same way because it waits until just a critical amount of mass has fallen on it and then it explodes, so they all look very similar to each other.
They brighten as a firework and fade away, and they reach the same brightness and you can then use that as an indicator of how far away it is, by just looking to see how bright it appears to you.
Because they explode with exactly the same intensity, these supernovae are known as standard candles.
By comparing their relative brightnesses, relative distances can be calculated.
Saul expected the stars to show what everyone thought at the time - that the universe was slowing down.
Fainter ones are further and just like when you watch a car recede into the distance, you can tell how far away it is by how faint the tail-lights look.
If you can use the brightness of the supernova to tell you how far away it is, that's really telling you how long ago the explosion occurred because you know how long it takes for light to travel that great distance.
So, now we have an object where it explodes and its brightness tells you when it exploded, how far back in time it exploded.
No matter how good the theory, the practical problem of catching an exploding star at just the right time is immense.
But Saul and his team applied the very latest computer technology to the problem.
Yeah, we think it may be the scuzzy chain is too long.
Now there's two switches on the back of it.
We had the computers go down, the computers came back up again, but now, finally, we have the analysis completed, at least the computer's part of the analysis, and it's beginning to show us on the screen what it thinks might be a supernova.
Eventually, after the team had identified 42 such dying stars, the calculations began.
What Perlmutter discovered shocked him.
The data was telling the wrong story.
The universe didn't appear to be slowing down.
We thought that that's what we would see and it looked like the opposite was taking place and, in fact, the universe was speeding up in its expansion.
These distant supernovae were fainter than you would have thought and fairly significantly fainter.
They were probably 20% or more and that's the hallmark of a universe that's actually speeding up in its expansion.
To say that this result was a surprise would be a masterclass in understatement.
It was so unexpected that the initial reaction was disbelief.
Everybody knew Saul and everybody knew the experiment he was doing and I remember sitting in the audience and Saul getting up and expecting him to present an update on the results he'd given a year ago, that actually the universe was slowing down.
And so, I was absolutely amazed that, based on only twice as many objects as he had the year before, that suddenly he was saying that we lived in a universe that was accelerating.
I remember it just being just incredible.
I mean, all the astronomers walking around scratching their heads saying, "This can't be right.
Surely it can't be right?" It was not the result that people had been expecting and such an extraordinary claim demands extraordinary evidence, more than a few data from a handful of stars.
So, here we are on a beach where there is about a billion pebbles and if you think you're trying to understand this beach, you wouldn't think you could understand it from 42 pebbles.
But Saul was right.
He was able to work out that the universe was accelerating just from 42 supernovae, which is quite incredible when you think about it.
On the one hand, this was a good result.
It was new science and produced a Nobel prize for Saul Perlmutter.
On the other, it raised an obvious question.
Once you know that the universe is actually speeding up, then you're faced with the question of - well, what could make it speed up? So far, the only real progress on that question has been to give the phenomenon a name.
It's become known as dark energy.
Dark energy is just the term we use to describe whatever it is that makes the universe accelerate in its expansion, what makes it expand faster and faster.
We don't know what that is.
It's a mystery and so we call it dark to reflect our ignorance, not because the colour is dark.
The mystery is so deep, so beguiling, that wherever there are physicists, there are people hoping that they will solve the mystery of dark energy.
People safe in the infuriating knowledge that what they're looking for, if it's there at all, is all around them.
But the fact that no-one has yet been able to identify what the dark energy might actually be has opened a can of worms not seen in science since the last time a physicist got involved in cosmology.
In 1915, it seemed that the work of physics was nearly at an end.
Everything made sense.
Newton had explained the heavens by invoking gravity and atoms had been identified as the smallest indivisible units of matter.
Job done.
But then, a German man, given to musing on trains, turned up with a totally new set of ideas.
I very rarely think in words at all.
A thought comes .
.
and I might try to express it in words afterwards.
Einstein called these little flights of fancy his "thought experiments" and they would lead him to develop his theory of general relativity, which totally changed how the workings of the universe were understood.
I sometimes ask myself how did it come that I was the one to develop the theory of relativity? The reason, I think, is that a normal adult never stops to think about problems of space and time.
These are things which he had thought of as a child.
But I began to wonder about space and time only when I had already grown-up.
Einstein's theory held that Newton's ideas about gravity, though empirically correct in most cases, were, in fact, conceptually wrong.
Gravity, said Einstein, was not some nebulous attracting property of mass as Newton supposed, but was, in fact, a consequence of mass interacting with space-time - the gaps around stars and planets previously known as space.
According to Einstein, space isn't simply a void.
It's more like a four-dimensional fabric woven from both space and time.
The mass of planets can warp and distort the fabric, gathering other celestial objects, like moons, around them.
And it's this bending of space-time that creates the effect we experience as gravity.
So, Einstein's theory of general relativity is a beautiful theory.
It's incredibly elegant and has been now around for 100 years.
It's very predictable.
You can write things, make predictions of what the universe should look like and what objects should look like in the universe, and we can test those, and as far as we can tell, it's passed every test.
The power of general relativity is that, like Newton's version of gravity before it, it's predictive.
Bizarre as the curvature of space-time may sound, it's eminently testable - a fact not lost on Einstein himself.
I have now come to realise that one of the most important consequences of that analysis is accessible to experimental test.
Accordingly, a ray of light travelling past the sun would undergo a deflection amounting to 0.
83 seconds of arc.
In 1919, that prediction was actually observed.
British astronomer, Arthur Eddington, pointed his telescope at a patch of sky near the sun during an eclipse and observed a star, known to be actually out of view, behind the sun.
Its rays of light had been bent by the distorted space-time created by the sun's mass.
Einstein's theory had held up.
A paradigm had shifted and the crowd went wild.
Einstein was suddenly famous.
Undoubtedly the cleverest, yet most incomprehensible man on earth.
This strange world is a madhouse.
Currently, every coachman and every waiter is debating whether relativity theory is correct.
This mass excitement about my theory is to do with the intriguing mystery of incomprehensibility.
I'm certain that the mystery of not understanding is what attracts people.
This is what all the fuss was about.
This is the equation that the coachmen and waiters were discussing.
On the one side, the geometry of space-time.
On the other, the mass and energy of the universe which acts on it.
Not incomprehensible at all(!) At least, not to its author.
But there was one aspect of general relativity that Einstein himself didn't understand.
The problem that Einstein had is that when he solved his equations of general relativity, what he found was that he predicted that the universe should actually be expanding.
And that was radically different from the perceived wisdom at the time, which was that we lived in a static universe, both static in time and in space.
So, he put in an extra term into the equation.
He called it the cosmological constant.
He used the Greek variable lambda, but, effectively, it was just what a physics undergraduate would call a fudge factor.
It was just designed to make the equations come out right and it would just make the universe sort of stand still.
The problem is is in science, we'd call that fine tuning.
And you only have to change the value of the constant by a small amount and suddenly, you get back these expanding solutions.
The general theory of relativity requires the universe to be spatially finite, but this view of the universe necessitated an expansion of equations with the introduction of a new universal constant lambda standing in fixed relation to the total mass of the universe.
This is gravely detrimental to the formal beauty of my theory.
When you add the lambda term, it means that the equation is not quite as simple as it was before.
So, in that sense, it's not as beautiful as an equation.
The static universe was restored, but Einstein always felt he'd added lambda against his better judgment.
Dear Egrenfest, I have perpetrated something in gravitation theory which exposes me a bit to the danger of being committed to a madhouse.
Despite the fudge factor, lambda, the cosmological constant, Einstein continued to be celebrated as the world's cleverest man.
Until, in 1929, he became even cleverer.
In America, astronomer Edwin Hubble was about to get a reputation for scientific cleverness himself.
He'd been using the world's largest telescope at Mount Wilson in California to peer deeper into space than anyone had ever looked before.
What he discovered completely changed the meaning of the word "universe".
Until Hubble, it had been thought that the universe was our galaxy.
What Hubble saw was that in fact our galaxy is just one of countless millions, but more importantly, that all these galaxies were moving apart from each other.
The universe wasn't static after all.
This had huge implications.
It introduced the notion of a beginning and an age for the universe.
But more importantly for Einstein, it meant that he could ditch his fudge factor, the cosmological constant, and return general relativity to its former glory.
The lambda that he added to create a static universe was no longer required, once it was observed by Slipher and Hubble that the universe, in fact, was expanding, so if, in an expanding universe, at the time, the observations could be described without the lambda term and so he removed it.
Einstein was cock-a-hoop.
In 1931, he went to Mount Wilson to shake Hubble's hand and thank him for putting beauty back into his equation.
Lambda, he later confessed, was the biggest blunder in his career.
I think that the reason that he said that it was a blunder was because if he had just not introduced that term, then he would have said that the universe must be expanding and done that 14 years before the discovery of the expansion of the universe by Edwin Hubble, which would have been a great achievement.
But despite Einstein's blunder, general relativity has stood the test of time.
It is perhaps the single most successful scientific theory yet.
Every observation we make of gravity, from the smaller scales to solar system scales to galactic scales, all the way to the universe, all of that can be described using the single theory that Einstein created.
So, it's the most successful and beautiful theory we have of our universe.
Or at least, it was.
For all its beauty and simplicity, general relativity doesn't account for the effects of dark energy.
Expansion, as reported by Hubble, works fine, but the accelerated expansion of the universe that Saul Perlmutter found isn't part of the deal.
That it's there at all is bad enough, but worse still, the way that dark energy seems to work is unlike anything that's been observed before.
The density of anything is the amount of stuff you have within a given volume and dark energy is an unusual phenomenon, in that even though the volume of the universe is increasing as it expands, the density is staying the same, constant.
So, imagine you had say like half a cup of black coffee, and then you started adding milk to it, and as we pour more and more milk into that cup, then the volume of the liquid's getting larger and larger, but the density of the coffee is going down, so the coffee would be getting lighter and lighter as you added more and more milk.
But dark energy doesn't behave that way, so it's almost as if there's new dark energy being created all the time, as the universe expands, meaning that its density remains the same.
Constant.
So, you can think of it, as you get more space, you actually get more dark energy, which is like getting something for nothing, which is clearly ridiculous.
It's clearly against all our training as physicists.
There is one way to adapt general relativity to cope with this magically constantly self-replenishing force and that is to simply add it to the equation.
100 years after Einstein's "biggest blunder", the cosmological constant is back.
Lambda is being written once more.
This time, not to keep the universe still, but to account for its unexplained accelerating expansion.
The values are different, but the concept is exactly the same.
All this leads cosmologists to one of two equally alarming conclusions - either we need another Hubble, or we need another Einstein.
But before we consign Albert to the scientific scrapheap, there is a branch of physics which might help.
An area where things popping in and out of existence is quite normal.
This is the strange and wonderful world of Clare Burrage and of quantum mechanics.
Quantum mechanics is the theory of what happens to really, really small things.
It's a theory of how the fundamental particles in the universe work.
Atoms, electrons, protons.
And quantum mechanics is intrinsically uncertain.
Einstein hated quantum mechanics.
He disliked the probabilistic "now you see it, now you don't" nature of the idea.
"God," he famously declared, "does not play dice.
" In 1930, he paid a visit to Nottingham, where Clare now does her research.
He didn't say much.
He probably didn't say anything about quantum mechanics.
He came to give a talk on general relativity.
His actual chalk writing is preserved for devotees to marvel at.
But even though Einstein didn't like it, quantum mechanics could shed light on dark energy and come to the aid of his once-more-under-fire theory.
In theory.
Quantum mechanics tells us that particles can come in and out of existence in the vacuum.
And the fact that those particles have mass and potentially are moving around, they have a little bit of energy.
And so, when they pop into existence, they give a little bit of energy to the vacuum and yes, they disappear again, but the fact that that process is going on all of the time means that there is some energy stored in the vacuum.
And because Einstein told us that energy and mass are the same thing, having lots of energy stored in space affects space-time that cause the expansion of the universe to accelerate.
So, it seems that quantum mechanics should, in theory, be able to explain how the cosmological constant works.
And how dark energy appears in the vacuum of space and is driving the acceleration of the universe.
But there's a problem.
When they came to calculate this vacuum energy, they discovered how spectacularly wrong they were.
If you were to say there was one pebble on this beach, you'd be wrong by one part in a billion.
If you were to say there was one particle in the universe, you'd be off by ten to the 80.
But the vacuum energy was calculated to be off by ten to the 120.
That is a google.
That is spectacularly wrong.
The fact that our predictions are so far off from what we see tells us that there's something fundamentally missing in the way that we understand physics, that we understand the world around us, so there's still a mystery, still a puzzle there.
It might be tempting to simply ignore dark energy.
You could argue that the apparent accelerated expansion is, in fact, a trick of the light, that it may be a function of other inaccessible dimensions at play.
That it just looks like dark energy, but is actuallysomething else.
But dark energy isn't just an irritating threat to Einstein's beautiful equations.
It's also a very practical solution to a fundamental question in cosmology, namely - what is the universe made of? When Einstein was busy thinking about gravity on trains, the answer was simple.
The universe was made of the same stuff that you and I are made of, the stuff of stars, planets, Coke cans, tennis rackets, atoms, made, in turn, from electrons, protons and neutrons.
But physics was about to get a shock.
It turned out that there was something else out there that the universe was also made of, matter of a different kind.
In 1975, an astronomer called Vera Rubin made an unexpected discovery.
If we plot the velocity of the planets as a function of distance from the sun, Mercury, Venus, Earth, Mars, Jupiter, Saturn, Uranus, Neptune, Pluto, and you can see that Mercury orbits much more rapidly than Pluto.
The graph is called a rotation curve.
It is the embodiment of the law of gravity.
The further away you travel from the sun, the weaker its gravitational force.
Galaxies work in the same way as our solar system.
Except that instead of planets orbiting a central sun, in a spiral galaxy, stars are held in orbit by a gravity-providing black hole.
Vera decided to plot the rotation curves in galaxies.
She trained her telescopes on Andromeda, the galaxy closest to our own.
I came out with sets of numbers and I plotted them on pieces of paper and I discovered that the stars as you went further and further out did not slow down, they were moving just as fast as the stars near the centre.
We find that the velocities remain flat all the way to the edge of our observations.
And that was a surprise.
And a surprise that had to be explained.
By rights, the stars should have flown off into space, but they didn't and wherever spiral galaxies were measured, the same flat curves appeared.
It was decided that the only explanation was that there must be more stuff out there that we couldn't see, providing the extra gravity, holding the galaxies together and flattening the curves.
They called this stuff dark matter.
The new dark matter was a shock, in more ways than one.
The very fact of its existence was almost overshadowed by the fact that when the calculations were made, this new form of matter outweighed the atomic form of stuff by about 90 to one.
In the 1980s, when new ways of measuring dark matter were developed, it was discovered that there simply wasn't enough of it to make the universe work as it clearly does.
The universe was short of stuff to the tune of about 70%.
Cosmology scratched its head.
Then, in 1998, a young scientist called Saul Perlmutter was thinking some very big thoughts indeed.
Something that felt like it was meaningful about the world we live in, in some deep way.
The universe was speeding up in its expansion.
The dark energy that earned Saul his Nobel Prize was an interesting and troubling concept, but it also had a number and that number was highly significant.
We know from Einstein - him again - that energy and mass are related, that energy, E, equals mass times the speed of light squared.
E = MC2.
Plug dark energy into that equation and you get the missing mass that dark matter couldn't account for.
The universe was complete.
It was made about 4% baryonic matter, the stuff that we're made from, 25% dark matter and the gaping 70%-sized hole was filled with dark energy.
So far, despite heroic efforts to find it, and overwhelming evidence that it exists, no-one has identified what dark matter is.
And, of course, dark energy, both useful and confounding, is barely in its infancy when it comes to a convincing explanation.
There's radiation damage.
We may not be quite there with the shielding yet to provide the right radiation environment.
But there is an idea in cosmology that dark matter and dark energy may be linked by more than just a common adjective and if they are, a new European spacecraft called Euclid may shed light on what that link might be.
The Euclid Consortium is staffed by 1,200 scientists from 14 countries.
These are some of them having their picture taken at their annual conference in Lausanne.
They're hoping that by taking pictures of the universe, they'll be able to work out how it's expanded over its lifetime and that by determining that, the nature of dark energy will become clearer.
The way we think about it is that it's either some new stuff in the universe, some particle or even just a new field that you put in to the universe to explain the properties of the universe.
Alternatively, you could say that the equation you wrote down is not correct.
It's not wrong, but we like to say it's "incomplete".
So, you could sort of fiddle with the mathematics of the equation, so actually what you could do is maybe come up with a natural explanation for it.
So, Euclid should be able to tell us which of those alternatives it is.
The satellite will launch and start sending data back to Earth in 2020.
The all-important camera for the Euclid space telescope is being built and tested in the UK, in this country house in the Surrey Hills.
These are going to be the biggest images that come down from in orbit.
You have an image of 625 megapixels, so that's roughly 300 HD television screens full of data and that comes down every ten minutes.
This imager takes roughly the same amount of data that Hubble has taken and will take in its entire lifetime in one day.
It's an astonishing leap forward, given what's available from current space telescopes.
The huge data sets will provide information from the universe in almost every direction.
The Hubble space telescope is the biggest and best telescope we have available at the moment and the amount of sky that's covered is about the size of your little fingernail, if you held it up at arm's length.
And Euclid will do the same type of imaging as the Hubble space telescope, but instead of just covering that small patch of sky, it will cover practically every bit of sky that you can see.
Not only will Euclid be able to measure the historic acceleration of stars and galaxies in all directions, it's hoped it will also provide data about how dark matter around galaxies has expanded over time.
This is possible because of an effect called gravitational lensing.
So, in general relativity, mass bends space and time and then light is bent around large massive objects, just like Eddington measuring the star behind the Sun, and so, we used the same technique for Euclid.
I can illustrate it using this wine glass and this image of the universe, so as we draw the wine glass across the image, what you see is that the galaxies behind the wine glass get distorted and that distortion is caused by the lens.
In general relativity, the lens is mass, because it bends the light.
And that can be shown in this picture.
You have a large clump of mass here, which is like the lens, like the bottom of the wine glass, and what you can see are all the distorted galaxies behind that lens.
And what you could do with an image like this is you can calculate how much mass would I need within the lens to create the distortions that I see and what you find is quite remarkable.
What you find is that there is about 100 times more mass here than you see from the light in the image and that missing mass, that mass you cannot see, is what we call dark matter.
So, Euclid will make an image of the whole sky at this resolution and it will find all these distorted background galaxies and from that, it can infer the distribution of dark matter in the universe.
Euclid will compare lensing all over the universe and by doing so, will help paint an accurate picture of how the universe is tearing itself apart under the influence of dark energy.
So, Euclid may tell us that it's the cosmological constant and then we have to explain that, it might tell us that our theory of gravity is not complete, and we'd have to explain that, it could tell us that actually the dark matter and dark energy are two sides of the same coin and that actually there might be a unified dark sector, but we'd have to explain that.
It could be another theory that we haven't even come up with yet.
And so Euclid will give us a coherent data set that we can test all these theories against.
Whatever the case, the devil's in the detail, and these days, the detail can be interrogated to degrees not thought possible when Einstein first reluctantly inserted his cosmological constant into general relativity.
Cosmology is one of the fields that is actually pushing the boundaries of cosmology itself, but also statistics and computing.
It is the frontier, I think.
Euclid will be pushing the boundaries like never before.
It will stream more data from space than has ever been processed in the past.
In the end, it will have about one and a half billion galaxies.
It will observe one and a half billion galaxies, so it's huge.
And a lot of the time, your eyes cannot just pick up patterns, so this cannot be possible without computers and statistics.
The computer-aided searches should give unprecedented clarity on how science should be thinking about dark energy.
There will be winners and losers.
The amount of data that we have on dark energy hasn't been enough to be able to tell us which path we have to go down, so we have lots of theories and hundreds of models that could still fit our data.
When Euclid comes, lots of these can be thrown away and it could narrow down the possibilities of what this dark energy is.
Euclid is not the only show in town when it comes to mapping the expansion of the universe.
At Kit Peak in Arizona, Risa Wechsler is hoping to use the proposed dark energy spectroscopic instrument, DESI, to make a map of part of the universe, like this one.
But 100 times more accurate, so that she can check the validity of computer simulations of the universe that she's created.
One of the things that I do is try to simulate the entire universe and tie what we think about the physics of the evolving universe to what we actually see with surveys like DESI.
What we're trying to do in these simulations is take a whole bunch of hypothetical universes, some of them will have a cosmological constant, some of them will have a different time evolving dark energy, some of them will have more or less amount of dark matter, and then when we compare that to what we actually see, we can rule out a lot of these ideas, so some of them will not be consistent with what we measure and then we can determine that that's not the universe we live in.
When DESI starts producing data in 2020, it might be that one of Risa's simulations strikes gold.
It'll be up against a lot of competition.
In the absence of hard data, this is boom time for theories.
Multi-Galileons, ghost condensates, and the higher co-dimensional brane worlds theory jostle for attention in the race to explain dark energy.
Many of these theories usually try to provide a global solution to the dark energy problem, a fix to general relativity, but Clare Burrage is working on an idea that says that Einstein may have been both right and wrong at the same time, depending on where you are.
We know that Einstein's theory works very well on Earth and in the solar system.
We've tested it and it works phenomenally well.
But we don't have ways of testing that theory on the kinds of distance scales that are relevant to cosmology and so it could be that whilst relativity is a good description of what's happening around us, it doesn't work as a description of the universe as a whole system and maybe you need to change the theory.
Clare's solution involves something called a chameleon, a particle that tries to blend in not by changing colour, but by changing how it exerts its force.
There are two types of particles in the universe.
There are the ones that make up matter, like electrons and protons and neutrons and quarks, and then there's another set of particles, and those are the ones that transmit forces.
So, for example, the photon, which makes up light, also carries the electro-magnetic forces.
It's exactly like what we're doing with the ball and the magnet.
We don't see the photons transmitting the force directly but we see the fact that the magnet makes the ball move.
In physics, the greater a particle's mass, the smaller the distance over which it's able to exert any force or field it might have.
The mass of a particle tells you how far it can carry information.
If a particle that's transmitting a force is heavier, it only transmits the force over a shorter distance scale.
So the range that you can transmit the force over changes depending on where you're looking.
The idea is that when the chameleon comes into contact with other stuff, it interacts with it and becomes heavy and its force-transmitting capability all but disappears.
But in regions of deep space where there's very little in the way of anything, the chameleon has no stuff with which to interact and so is very light and can transmit its force over vast distances.
It's a neat idea, but evidence is hard to come by.
Then, in 2014, Clare came up with an experiment that might unmask the chameleon.
The experiment that we proposed last year is that you'd specifically design your experiment to look for chameleons, which means that you look in a very high vacuum and you use tiny, tiny particles, so we're using individual atoms.
But wrangling individual atoms isn't easy.
It takes an enormous amount of scientific hardware, specially configured in a highly precise way.
Just six months after Clare's paper was published, atomic physicist Holger Muller got in touch.
As it happened, he explained, he had exactly the right equipment needed to perform Clare's experiment, right here in Berkeley.
From where, in 1998, Saul Perlmutter's group discovered dark energy in the first place.
It might be that the conundrum could be solved at the same institution that it was discovered.
We've been setting up this experiment for several years when my post-doc colleague came across Clare Burrage's paper on the pre-print server and we read the paper and we found that, "Wow, they're describing "exactly the experiment we've been building for all these years.
" And we got excited about it so we stopped doing our original experiment and started doing the dark energy measurement.
It's an amazing feeling to have that kind of quick response because it almost never happens like that in science.
Things take a long time to go from theory to somebody actually doing an experiment.
To have a measurement of something we proposed in the space of six months is phenomenal.
The experiment involves using a vacuum chamber and a cunning chameleon trap.
The animal, the chameleon, changes its colour in order to hide, right? And in the same sense, the chameleon particle changes its mass in order to hide.
At the centre of the vacuum chamber is a marble-sized sphere.
If there are chameleon particles around, they will interact with the mass of the ball and produce very little in the way of force but perhaps just enough to affect something very small like an individual atom.
The heart of the experiment is this little sphere inside there and so the experiment works by first collecting a cloud of caesium atoms on top of the sphere so here's the sphere and about one centimetre on top there's a little cloud of about 100 million caesium atoms.
The machine contains the atom cloud using infrared lasers, invisible to the naked eye.
The beams need to be very precisely controlled, so they're sent around a complicated series of mirrors on what's known as an optics table.
We use lasers to control the atoms and so to do that, we need to pass them through this table of optics.
The laser beam kind of takes a snake-like path throughout all these optics but eventually gets into something like this, an optical fibre.
The light can then travel through here and is brought over to interact with the atoms.
So, you see here the sphere and we trap the cloud of atoms just on top of the sphere, and then we release the trap and the atoms are free to fall subject only to the Earth's gravity and the potential chameleon force.
When the atoms are released, they will fall towards the ball, which will contain chameleon particles, if they exist, and if they do exist, they will be busy interacting with the mass of the ball, making themselves heavy and reducing the distance over which their force can be felt.
Which isn't to say that the force is completely non-existent.
According to chameleon theorists, there'll be a tiny region on the surface of the sphere where the force is active and given that the atoms are so tiny, they will be affected by that force.
If it exists at all.
All the team need to do is to precisely measure the difference between the speed the atoms fall with and without the ball in place.
Now we want to measure the chameleon only and not the combination of gravity and the chameleon, so what we do is we will move this sphere out and then do the measurement again, this time measuring gravity only.
The experiment is set up to compare the difference between how fast the atoms fall towards the ball and how fast they accelerate through empty space.
If the tracking reveals an unexplained acceleration, this could be due to the force associated with the chameleon particle.
The experiment has now been running for over six months and they're starting to get their first results.
Right now we have seen no evidence for chameleons, which means they either don't exist or they hide in a region of the perimeter space that we can't yet measure.
So, what does that mean? If either the chameleon force is extremely weak or it's even heavier than we thought, then we can't see them.
The team at Berkeley are now adjusting the experiment to rule out any theoretical nooks and crannies where the chameleon might be hiding.
Well, if we make the experiment more and more and more sensitive, we will either discover the particle or rule it out once and for all.
A scientist might be like a drunk who lost his keys and is now looking for it under the next lamppost .
.
and it's not because he knows that the key is there, but because it's hopeless to look in the dark anywhere else.
People have searched for dark energy in cosmology and in astrophysics, and now we start looking for it under the atomic physics lamppost.
Whether this is a good idea or not, we will know in a couple of years when it has either been found or not, but it's always exciting to have It's like a new window that you can open and look through and you don't know what you will see before you've tried to do it.
Having more information is always a good thing so ruling out possibilities.
Although on a personal level maybe it's a little bit upsetting because it's a nice theory but it means that you've got more information and you can go on from there and build something better, build a better theory.
With all these theories, it's really a question of taste.
You either like a cosmological constant or you can explain it through a chameleon effect.
None of these as yet give us the elegant solution that we are looking for and that's what really we're looking for.
We're looking for this simple, elegant solution to this strange accelerated universe and nothing yet has given us that.
Chameleon? Yeah, maybe, but as yet, there's no evidence for it.
Dark energy? Yeah, we can sort of understand it, but we can't get the number right so we're still grasping in the dark for an elegant, simple solution to what we see.
Where that simple solution will eventually come from is anyone's guess.
That is one of the infuriating things about science.
It can't always produce the rabbit from the hat on time and on budget.
Sometimes it takes an unexpected turn of events, or what the media like to call a "genius".
Though the geniuses themselves have a rather different take on their exploits.
I'm not more gifted than anybody else.
I'm just more curious than your average person and I will not give up on a problem until I have found the proper solution.
I think that curiosity is what driveswhat drives most cosmologists and physicists, a curiosity about the universe - why? What is the universe made out of? Why are we here? How did the universe begin? What will happen to the universe in the future? All of these are questions which are driven by curiosity.
I have no special talent.
I am only passionately curious.
Curiosity, I think, is Well, it's the best motivating force, OK? Working hard doesn't necessarily get you to an answer.
Working too hard can actually stifle creativity.
With our work, you know it's a mixture of inventiveness and persistence in the hard work.
It's a combination.
It's the end of the Euclid conference in Lausanne.
The conference organisers have laid on a social evening, cruising around Lake Geneva.
It's a chance for the delegates to unwind and maybe even think a little about the biggest picture of all.
Yeah, so Einstein's theory was motivated for a reason, right? He had an equivalence principle.
Yeah, and, I mean, we're going to measure a lot of things about the nature by looking at how it evolves, how dark energy actually evolves with red shift.
But the problem is the zero-point energy, the vacuum energy, the quantum mechanical part that you add there.
Try and study the nature of dark energy and at the same time, try and test if general relativity works.
So, there's like a lot of work and a lot of discoveries that are going to happen down the road.
- Exactly.
- And I'll drink to that.
Exactly that.
The process of scientific discovery sometimes makes progress through sheer hard work and sometimes it needs someone to take an inspired alternative view.
We learned an awful lot about animals and plants by simply observing them, but it took Darwin, with a radical idea, to give us a context to understand life itself.
And in our efforts to understand the wider world and even the universe, observations are critical.
The ideas of dark matter and dark energy come courtesy of people watching stars but just as Einstein musing on his train managed to take all the known science and see it from a different, more useful, angle, it might be that to solve the dark energy problem, someone needs to pull off a similar trick and come up with an even better idea.
There are an awful lot of very smart people in the world.
I wouldn't be surprised if we end up with another Einstein, you know, somewhere along the line here.
I don't know whether it'll be in our lifetime but we I think we have a good shot at it.
We need teams like Euclid.
That's the only way you can get the data that you need.
But to understand that data, to give it some interpretation, to give it an idea, could come from one person.
That could be the next Einstein.
A genius could come up and put all the observations that we have so far, put it together, and come up with a new theory.
Yeah, it is quite possible.
I'm kind of hoping it's me.
The tantalising truth is that all it might take to solve the mystery of the dark energy is one big idea, for someone out there to see things differently, someone perhaps like you.
And if that new Einstein is you, if you manage to solve the mystery of dark energy, you're likely to become very famous indeed, as famous as the original Einstein.
Wherever I go and wherever I stay, there's always a picture of me on display.
On top of the desk or out in the hall, tied round a neck or hung on a wall.
Women and men they play a strange game, asking, beseeching, "Please, sign your name.
" From the erudite fellow they brook not a quibble, but firmly insist on a piece of his scribble.
Sometimes, surrounded by all this good cheer, I'm puzzled by some of the things that I and wonder, my mind for a moment not hazy, if I, and not they, could really be crazy.