Bad Science
© Copyright 2004 Herbert J. Bernstein
There are many ways in which to do science badly. We can formulate
untestable hypothesis. We can formulate sensible hypotheses and
then test something else. We can collect bad data. We can
analyze our data incorrectly. We can present our results
badly. We can fail to reason logically. We can accept
unproven or even false premises as if they were proven.
- Untestable Hypotheses
- Badly stated hypotheses (e.g. impossible to frame a
null hypothesis)
- Hypotheses about non-existant systems
- Hypotheses about systems that exist, but with which we
do not interact
- Hypotheses about systems that exist and with which we interact,
but for which we do not have quantifiable measurements
- Testing something other than the hypothesis
- Collecting bad data
- Poor choice of variables
- Defective or uncalibrated instruments
- Failure to understand noise
- Failure to understand background
- Failure to collect enough data
- Failure to estimate errors
- Defective treatment of the data before analysis
- Poor analysis
- Wrong hypothesis
- Wrong model
- Wrong solution
- Bad fit, wrong fitting paradigm
- Failure to propagate errors
- Bad presentation
- Failure to communicate in the language of the audience
- Does the audience accept the premises of science:
That there are truths that can be tested
That there are mechanistic relationships among aspects
of nature that can be discovered
That human beings can understand at least some of these
relationships
- Does the audience understand the terms being used?
- Can the audience see past the style of presentation to the content
- Are there sufficient shared memes?
- Failure to organize the information
- Failure to present sufficient information
- Presenting too much information
- Presenting distracting information
- Presenting too slowly
- Presenting too fast
- ...
- Failure to reason logically
- Reasoning backwards from the desired conclusions and inventing
facts to fit
- Failure to distinguish probability from certainty
- Assuming correlation implies causality
- Circular reasoning
- Logical Fallacies (see
www.datanation.com/fallacies/index.htm)
- Unproven or False premises
- See
http://www.theness.com/articles/skepticismandscience.html
- See
http://www.deconstructionist.com/indexo.html
Much of the public does not appreciate science. Distaste for a small amount of
science done badly can overwhelm appreciation of the vast amount of science
done well.