Dark Matter and statistics

While I was on holiday, news broke (e.g. see the piece in The Guardian) about the possible detection of WIMPs.  Weakly Interacting Massive Particles are what many physicists think makes up 'dark matter'.  (What is dark matter? - basically, if you analyse the way galaxies move, you discover that the amount of matter you can 'see' with conventional techniques, be it visible, infra-red or whatever isn't enough to provide the necessary gravitational attraction. There is some other kind of invisible matter out there - called 'dark matter')

The CDMS research team has undertaken a series of experiments deep underground to look for the rare event of a WIMP interacting with normal matter.  The experiment has to be underground to shield it from cosmic rays and other particles incident upon the earth from outer space. And they report that they have possibly seen two events.

Now, if you do a science experiment well, you can quantify your 'possiblies'.  In this case, they say that the chance of two events like this occuring due to other (background) effects is 23%.  Is this low enough to say that dark matter has been discovered?

That's an interesting question. Often, people work with a 5% limit.   If the probability of the results being due to chance alone is less than 5%, it is commonly assumed that the experiment has shown a positive result.  But is this true?  Five percent is an arbitrary boundary. It still says that one in twenty of every experiment which reports success at a p=5% limit is actually due to chance. How low do you have to go before you are justified to put out a press-release saying that your WIMP has been detected?  There is no clear-cut answer to it, though prudence would suggest that the more sensational your result, the more certain you should be of it (and the lower your limit.)

Twenty-three percent is pretty high and the authors of course are very sensible in saying that they have seen hints of WIMPs, rather than the WIMPs themselves. To be more certain about it, the experiment needs to be run for longer - and more events seen - or a better experiment produced.