Log in

Previous Entry | Next Entry

How to Tell when Something Isn't Science

The process of science--the systematic, evidence-based, rigorous, controlled exploration of the processes of the natural world--has produced an explosion of knowledge and understanding. Since the Italian Renaissance and the Abbasid period in the Persian empire, both of which saw enormous gains in scientific thinking and with them huge leaps in technology and understanding, science has been the beacon of light shining in the darkness of superstition and ignorance.

So it's probably not too surprising that many folks who seek to embrace all sorts of non-scientific ideas try to claim that their ideas are science. Calling these ideas "science" gives them a stamp of validation. If an idea is scientific, that means it has greater legitimacy in many people's minds.

And the world needs to cut that shit out. Not all ideas are science, yet everything from phrenology to metaphysics to "crystal energy" tries to clamber onto the scientific bandwagon.

Most recently, the cry of the pseudoscientist has become "Quantum mechanics says!" Folks who can't actually define what quantum mechanics is are nevertheless eager to fill New Age bookstores with books that claim to "prove" that quantum mechanics validates their ideas.

So here's a handy-dandy, more-than-pocket-sized guide that will help you tell what science actually is and is not. Ready? Here we go!

RULE 1: If it doesn't make a precisely defined, testable, falsifiable claim, it is not science.

This is the first and most basic premise of this whole "science" business. If someone claims "Science shows us that" or "Quantum mechanics proves that" and the next thing out of their mouth isn't a testable, falsifiable claim, then what they're saying is probably bollocks.

For example, "Science says that energy in any form always exhibits relativistic mass, and mass-energy equivalence is a function of the speed of light in the form of E=mC2" is a testable hypothesis. It makes a specific statement about mass-energy equivalence which can be measured and either verified or falsified.

On the other hand, "Quantum mechanics says that consciousness is the most fundamental part of the universe" is not a testable hypothesis. There is no definition of "consciousness" or what it means to be "most fundamental." It makes no empirically verifiable claim. It might be perfectly reasonable as a statement of philosophy, or metaphysics, or religion, and there might be value in debating it as a a matter of religious doctrine or philosophy, but it is not science.

RULE 2: Just because a scientist says it doesn't necessarily mean that it is science.

Scientists are people. Science is one particular way of looking at the physical world, but people--even scientists--often look at the world in many ways. A person who is a scientist can also hold ideas that are not scientific ideas.

For example, the famous physicist Max Planck was a scientist who helped to revolutionize our understanding of physics by laying the foundation for quantum mechanics.

He was also a Christian theologian who wrote several works of theology, and upon retiring from active research became a church warden in Germany.

Planck said that energy could only be emitted from a blackbody radiator in quantized form, and that the energy thus radiated was the product of the frequency of the radiation multiplied by a universal constant h, which is now known as Planck's constant. This is a precisely defined, testable, falsifiable statement; it is science.

He also said "All matter originates and exists only by virtue of a force which brings the particle of an atom to vibration and holds this most minute solar system of the atom together. We must assume behind this force the existence of a conscious and intelligent mind," and that this mind was the mind of an omniscient, benevolent god. This statement does not make a testable, falsifiable claim; it is not science. Folks who use this statement to argue that Max Planck "proved" that the idea of universal consciousness is part of physics do not understand what physics is.

(Interestingly, Planck's religious views led him, late in life, to reject the statistical models of quantum mechanics, because he could not accept a universe whose laws at a small scale were based on probability and not determinism.)

RULE 2a: Competence in one area of science does not necessarily mean competence in another area of science

There is often a tendency to assume that scientists must be smart, and that if they know something about one field, that means they must be just as smart in other fields too. This can become particularly insidious in fields which seem, from the outside, to be related or similar.

For example, one of the early discoverers of the human immunodeficiency virus was virologist Dr. Luc Montagnier, a French researcher who, together with Jean-Claude Chermann and clinician Willy Rozenbaum, helped isolate and identify the retrovirus that causes AIDS.

Later, Dr. Montagnier would say that AIDS can be treated with diet alone, and that drugs are unnecessary to halt its progression.

To an outside observer, the fact that Dr. Montagnier is a Nobel-prizewinning virologist seems to give this notion credibility. However, being a virologist is not the same thing as being an immunologist. It is possible to understand a great deal about the molecular workings of viruses without actually having a good understanding of the molecular biology of immunology.

The manner in which HIV thwarts immune response is extremely complex and quite subtle. The fact that Dr. Montagnier is a virologist does not mean that his statements on immunology are necessarily credible. Accepting his pronouncements about the treatment of AIDS simply because he is a virologist, without evidence, violates Rule 4 below and is not science.

RULE 3: If it can't be reproduced by other people, it isn't science

One of the central tenets of science is the notion of reproducibility--the idea that what is true for Bob is also true for Sue, and that if something is a property of the physical world, it can be observed to be true regardless of who's doing the observing.

In science, there are no wizards for whom nature behaves differently than it does for everyone else. If a person in New York City observes that the gravitational force between two mass-bearing objects is the product of a universal constant times the product of the masses of the objects divided by the square of the distance between them, and conducts an experiment which shows the universal constant to be 6.67384x10-11N(m/kg)2, then an experimenter in Teheran should, if he sets up an experiment to determine the gravitational constant, arrive at the same result...at least within the error margin of his experiment and procedure. Science does not depend on personal philosophy, political ideology, religion, subjective experience, or philosophical views. Anything which does vary by religion or politics or philosophy or subjective experience is not science, and should not be labeled as "science."

RULE 3a: If a statement makes a claim that some part of the physical world which has been measured and tested should be different from what it has observed to be, that statement is probably not science

If I were to propose a new model of gravity, and my model of gravity predicts that the universal gravitational constant is 42 N(m/kg)2, then something is very wrong, because many repeated experiments done by different people under different conditions with different methodologies have all agreed that this is not, in fact, what the universal gravitational constant is.

Of course, any measurement of the physical world is going to have error bars associated with it. No measurement can ever be absolutely perfect. And sometimes it may be that some new model might predict some physical property to be something different from what it is believed to be, but close enough that the error bars in its measurement mask the difference.

It is also often true that simple models, such as Newton's laws of motion, can be observed to be true under one set of conditions, but reality deviates from the models under different conditions. Newton's laws don't work well when, for example, things are moving close to the speed of light.

None of this means that observations can just be discarded whenever a new idea comes along. Any theory that accounts for the motion of objects moving at the speed of light had better also agree with Newton's laws for objects moving slowly with respect to one another, because we've measured the behavior of objects moving slowly with respect to each other, aNewton's predictions work. If someone says that a baseball won't follow Newton's laws when it comes off a batter's bat, that person's statements are probably not science, or at the very least will fail when tested.

More generally, a result which only one person can arrive at, but which others can not reach when they try to perform the experiment themselves, is probably an error. Any idea which proposes that something is true only for a particular guru, or for people who have completed the eighth level of illumination under the supervision of an enlightened master, is not science.

And speaking of error...

RULE 4: Scientific claims are only as valid as the evidence which supports them

Albert Einstein famously observed, "No amount of experimentation can ever prove me right; a single experiment can prove me wrong." Any scientific idea, no matter who supports it, is only as good as the empirical evidence that backs it up.

No scientific idea is "proven right" by the evidence. Rather, the evidence may support the idea, but the only real proof is evidence which contradicts the idea's predictions. An idea which has a lot of evidence confirming it can usually be trusted...but even a single disconfirmation is all it takes to knock it down.

For that reason, one thing which often quickly separates science from things that aren't science is that in science, we tend to look for evidence that might prove an idea wrong. When an idea is being tested, it is scrutinized from all angles with the question, "What predictions does this idea make, and how might we construct an experiment to show that prediction is false?" The more vigorously an idea resists disproof--the more experiments are done to try to prove it wrong, and instead reveal results that fit the idea--the more the idea can be trusted.

People who want the validation of science for their ideas often complain that the scientific community ignores or ridicules them. From that point, such people often imagine a "conspiracy" of scientific orthodoxy, in which scientists collude to silence them.

That isn't how it works. Instead, science says of new ideas, "Okay, where's the proof? If you want us to listen to you, show us the proof. If your idea can't be disproven, it isn't science. If you don't have reproducible evidence to show us, go away."

For many years, the reigning idea about stomach ulcers was that they were caused by stress. Ulcers were treated with extremely expensive drugs that inhibited production of stomach acid. These drugs had to be taken continually, and would not cure the ulcers; as soon as acid production went back to normal, the ulcers returned.

In 1982, Dr. Barry Marshall and Dr. Robin Warren announced that they believed stomach ulcers were actually caused by infection with an exotic form of bacteria with the ability to protect itself from the harsh acid in the stomach, and that cheap antibiotics could completely cure them.

The scientific community ignored them.

Until they provided evidence, by showing that swallowing the bacterium Helicobacter pylori would give stomach ulcers to a person who didn't have them, and a course of antibiotics would cure them.

At that point, the scientific community changed its mind, the idea was accepted, and Dr. Marshall and Dr. Warren
won the Nobel Prize.

Claims like "Scientists just don't want to recognize my idea because it's too revolutionary" or "Medical doctors won't accept my idea because it would mean less money for drug companies" are not science.

RULE 4a: Extraordinary claims require extraordinary proof

Every so often, someone proposes an idea that would turn the whole, or a good part of, our understanding of the physical world on its head. Whenever that happens, the person who proposes the idea had better have a whole lot of evidence if he is to be taken seriously.

If someone claims that the sun came up this morning, we don't need a lot of proof to think that claim is likely true. It fits with the observations we've made and with our understanding of what happens in the morning. It's predicted by our models of cosmology and the behavior of the solar system.

On the other hand, if someone claims that archaeology supports the notion that the Bible is literal truth, or that ancient space aliens built the Pyramids, those claims require quite a tremendous weight of evidence if they are to be credible.

And not ordinary evidence. The notion that some events described in the Bible might be supported by archaeology is not overwhelming; we know the city of Troy existed, but that's not proof that the Iliad is literal truth. We know that the Danish king Harald won many battlefield victories, but that is not proof that the Viking god Odin favored him.

When it comes to supporting extraordinary claims, simple garden-variety evidence just doesn't cut it. A picture in a Mayan temple of a guy with a round circle around his head doesn't rise to the level of proof required that aliens visited the Mayans, especially in light of the fact that that notion is actually less plausible, with its prerequisite assumptions of interstellar space travel and the unanswered issue about why space aliens would hang around with pre-technological societies for a time and then just go away, than the idea that they had strange tastes in hats. The idea that the Pyramids could be built by hand is actually less farfetched than the idea that space aliens somehow traveled here in violation of current understanding of the laws of physics to do it and then leave.

That doesn't mean we understand physics or that interstellar travel is impossible. It just means that given two explanations for something, one of which is "people are smart and good at engineering" and the other of which requires turning all of physics upside-down, the former is probably the safer bet. Statistically, extraordinary claims are extraordinarily unlikely to be true.

The notion of ancient space aliens has a certain pop-culture appeal, which makes it sound like a better story than "a bunch of people were clever, had nearly unlimited supplies of raw physical labor, and were good at engineering." However, it's still not science. Rule 9 talks more about that.

RULE 4b: The plural of 'anecdote' is not 'data'

Our brains are marvelous, finely-tuned, incredibly complex instruments carefully optimized to get it wrong.

For folks who live in small pre-industrial tribes in the African veldt, getting it wrong is often the right thing to do. People facing the possibility of hostile leopards or poisonous food need to be able to make snap decisions in an instant, and if the decisions are wrong, those people might not spread their genes. So our brains are gifted to us by our ancestors who made snap decisions conservatively, and lived.

If someone feels sick, and takes a pill that she thinks will make him better, and after taking the pill she feels better, it's often true that no force in the universe can convince her that it wasn't the pill that did it. If someone is told a story by a friend, or witnesses something himself, he will tend to place a great deal of emphasis on it, and tend not to look at things which might contradict that experience. All these helped our ancestors; they didn't necessarily know that it was the mushroom they ate which made them feel bad, but our ancestors who tried to repeat the experiment sometimes died for that curiosity.

Worse, we don't remember the world the way it is. When we remember something, that memory isn't like a video recording; it's pieced together based on our ideas, our preconceptions, our emotions at the time, and our emotions now. We remember things that fit our preconceptions and don't remember things that don't.

We also remember stories, especially stories that relate to our own inner narratives about our place in the world. Anecdotes, by their nature, are more powerful stories than, say, double-blind controlled studies, or mathematical charts, are.

All this adds up to the fact that our personal experience is rubbish when it comes to understanding the physical world. No matter how vivid your own experience might be, no matter how many examples of something being true you may personally remember, ideas based on your experience or on reports of other people's experiences are not science. When someone says that homeopathy totally cured his uncle's cancer or his uncle's broken arm, that is not science. When someone says "My toddler got a measles vaccination and six months later I found out she was autistic, that shows measles vaccinations cause autism," that is also not science.

RULE 4b(i): Personal experience, revelation, and subjective data are not evidence

No matter how powerful they are, personal experiences--even experiences (especially experiences) you can't understand, aren't actually proof that something exists. Any argument made on the basis of personal experiences is not science.

The brain is fabulously complex, and it can do some pretty weird things. People often have the personal experience of having some sort of revelation imparted to them from the outside, or of leaving their bodies, or of encountering strange and terrifying things.

All these things can be caused by physical processes in the brain. Strong G-forces, among other things, can reduce blood flow to the brain; reduced blood flow can create powerful experiences of floating outside or above one's body. It is possible to wake in the middle of a nightmare, and be conscious of where you are while still being unable to move or to speak; these "night terrors"can be incredibly powerful visceral experiences, but they are not proof that there are monsters at the foot of the bed. (Sadly, many folks throughout history, and sometimes even today, have become so convinced that these waking nightmares are proof of malevolent forces that they have accused their neighbors of witchcraft or sorcery, with disastrous results.)

Proponents of past-life regression often try to claim that the experiences of people under hypnosis are proof of reincarnation. These claims are not science. A person under hypnosis is easily suggestible, and hypnosis can often create vivid, compelling, detailed, rich, powerful memories of events that simply didn't happen. The inner subjective experience of the world is easily altered or tricked. Memories "recovered" under hypnosis, like any other subjective experiences, are not science.

RULE 4c: A thought experiment is not a real experiment

The physical world doesn't always work in ways our intuition might predict. Thought experiments are sometimes useful as ways to consider an idea, but they are not actually evidence; the result of a thought experiment is never as compelling as the result of an actual experiment.

A classic example is the behavior of two falling weights. If we are to imagine how they act, we might think that a heavy weight--a 10-pound dumbbell, say--would fall faster than a lighter weight, like a 2-pound dumbbell. In fact, this seems so obvious that many philosophers didn't bother to test it.

When you do test it, however, you observe a counter-intuitive result: heavy weights fall at the same speed as lighter weights.

Any claim which attempts to use a thought experiment as the only evidence which supports it is not science.

RULE 5: A vaguely-worded, rewritten, or imprecise restatement of a scientific principle is not (necessarily) science

One of the places where people often get tripped up in making all sorts of claims about science that aren't actually science is Heisenberg's uncertainty principle.

The common lay understanding of Heisenberg's uncertainty principle is that you can not know simultaneously both the precise position and the precise momentum of a subatomic particle. This is one of the cornerstones of our understanding of the physical world, and it lays the foundation for the probabilistic approach to particle physics.

It also lays the foundation for a lot of metaphysics and confusion. Christians argue that the uncertainty principle does not apply to God because God has infinite knowledge. Yogic teachers interpret Heisenberg's idea to mean that human awareness has a tangible effect on the universe, and therefore human intelligence supersedes the laws of physics. New Age philosophers have interpreted it to mean that all things in the world are one with human consciousness.

All of these statements are not science. In addition to not making falsifiable claims (Rule 1), they misunderstand what Heisenberg's uncertainty principle is. The uncertainty principle itself is given by

where σx and σp are the standard deviations of position and momentum and ħ is Planck's constant divided by 2π.

There are no terms for "consciousness" or "awareness" in this equation. Heisenberg's principle doesn't state that we can't know a particle's position and momentum at the same time because human consciousness affects the particle or because human awareness is inferior to God's awareness. We can't know a particle's position and momentum at the same time because the particle does not HAVE defined position and momentum at the same time. Subatomic particles are not like little marbles and they don't act like little marbles.

More generally, Rule 6 applies to any imprecise statement of any scientific principle. The misunderstanding that evolution says mankind came from apes, and arguments based on it, are not science; evolutionary biology says no such thing, but instead asserts that humans and apes share a common ancestor at some point in the past. The misunderstanding of mass-energy equivalence that leads some mystics to claim that matter is an illusion and everything in the universe is actually made up of "energy fields" is not science.

RULE 5a: A metaphor used to express a scientific statement is not itself science

The pursuit of science often produces results that are not easy to understand without a thorough, rigorous educational background. Because of this, scientists and science teachers will often use metaphors to try to communicate the gist of scientific principles.

These metaphors themselves are not science and should not be confused with science. Any metaphor, if pushed too hard, will break. A metaphor by itself isn't a scientific truth, and on the flip side of the same coin, finding a flaw in a metaphor that's being used to explain a scientific idea is not the same thing as finding a flaw in the scientific idea.

One metaphor that folks often tend to get tangled up about is the metaphor of Schrödinger's cat. This metaphor is used to help explain a very complex and non-intuitive concept called "quantum entanglement," in which a system of subatomic particles may exist in more than one state simultaneously, and then undergo "collapse" and behave like it is in a single state when it encounters something whose state depends on its state.

Schrödinger proposed a system where a cat might be in a superposition of states where it is simultaneously both alive and dead until it interacts with an observer. However, this thought experiment is only a metaphor. It is not possible to place a cat (or any other large, macroscopic object) in a superposition of states in this fashion. In point of fact, another place where laypeople trying to understand quantum mechanics get tangled up is in the idea of an "observer." In Schrödinger's thought experiment, the detector that responds to radioactive decay by either killing or not killing the cat is an observer! An observer isn't a person looking at something. It's anything whose state depends in a thermodynamically irreversible way on the state of the thing being observed.

An argument such as "Schrödinger's cat proves that we all create the universe around us" is not science--not just because it fails to make a falsifiable claim (Rule 1), but also because it confuses the metaphor with the underlying idea. The metaphor itself is also not science.

RULE 6: If you start by assuming the premise that you're trying to prove, then the result is not science

A common fallacy of reasoning is assuming a premise you're attempting to prove is true, then using the truth of that premise to "prove" the premise. This kind of reasoning occurs often in claims about the existence of a god or gods. A simplistic version looks like "Consciousness exists within the universe the universe. There can only be one source of consciousness. The source of consciousness must itself be conscious. Since there can only be one source of consciousness, and since that source must be conscious, and since the only consciousness which can create the universe is god, then god must exist."

Embedded within this argument are the assumptions that its premises ("there can only be one source of consciousness," "source of consciousness must itself be conscious") must be true. If it is possible for consciousness to arise from things which aren't conscious, or if consciousness is an emergent property which can arise in many places from many different types of events, then the argument doesn't work. The premises must be true for the argument to stand, but the premises are what the argument is trying to prove! The argument itself is therefore not science.

People make claims that are purported to be science but that assume their own premise often. Conspiracy buffs often try to argue that the World Trade Center collapse must have been the result of a controlled demolition because it looks like a controlled demolition and only controlled demolitions look like that--an argument that assumes one of its premises ("only controlled demolitions look like that") to be true. This claim is not science.

Now, this is different from assuming the premise in order to show an absurd result, which is an argument called reductio ad absurdum. The premise is shown to be false by deriving a false outcome which would have to be true if the argument is true. For example, let's say that a person claims to have found a perfect chess-playing strategy that would always let him win any game, with no possibility of a loss or a draw, regardless of which side he played. If we assume the claim is true, we quickly reach an absurd conclusion: two people playing against each other who both employed this strategy would both win, with neither losing nor drawing. But this is not possible in a game of chess; ergo, the claim to have discovered such a chess-playing strategy must be false.

Using reductio ad absurdum arguments--assuming a premise is true in order to show that this true must lead to a contradiction--is science. Assuming that a premise is true for the purpose of proving it is true, however, is not science.

RULE 7: If your idea depends on common sense or what is intuitively obvious, it is not science

I once had an old friend express great consternation to me when he heard that time moves slower inside a gravity field, so time on the surface of the earth moves slower than it does in space. How was it, he asked me, that gravity could slow down his wristwatch? Wasn't it possible to build a watch that wasn't affected by gravity? His intuition didn't cope well with the notion that it wasn't the watch that was moving slower...it was time. Everything--not just the watch, but all physical processes, chemical reactions, even nuclear reactions, move slower in a gravity field.

We don't think of time as being malleable in this way because our intuition was not shaped where it is something we observe directly. Yet it is true nonetheless, even though it messes with our intuition about time.

The physical world is not under any particular obligation to make intuitive sense to a human being. In fact, human intuition developed in a very specific environment--an environment of relatively large, relatively non-energetic things moving relatively slowly with respect to each other. Unless we believe that the entire universe is this way, there's no reason to believe that the entire universe will behave in a way that matches our intuition. And in fact it doesn't.

There are several schools of New Age thought that say the universe can be understood by intuition. Some people claim that intuition springs from an all-knowing, all-powerful subconscious mind, in harmony with the essential oneness of universal consciousness. In addition to being appeals to intuition, these ideas don't make falsifiable claims (Rule 1), and often depend on a vague, metaphorical understanding of quantum mechanics (Rule 5). They are not science. Claims based on these ideas, such as the notion that a psychic healer can intuit a sense of what is wrong with a patient merely by concentrating, are also not science. Appeals to "trust your intuition" about what is correct are likewise not science; our intuition is limited by our experiences, our environment, and the things that we want to be true or really wish were true.

RULE 7a: If your idea depends on notions of 'perfection,' or Scripture, or philosophical arguments, or what is beautiful, or what is aesthetically pleasing, it also isn't science

The creation "science" folks have a really, really bad case of science envy. In their literature, they describe themselves as scientists; there's a creation science movie in which a person who's claimed to be a paleontologist claims that he and more mainstream scientists look at the same information, they simply start from different premises so they reach different conclusions.

Unfortunately, the moment you start from the premise that a particular idea, book, work of scripture, or philosophical construct must be true, what you're doing is not science.

Creation "science" tends to assume its own premise (Rule 6), but it also runs off the rails by assuming the infallability of one interpretation of one bit of scripture as well. When you've already arrived at a fixed conclusion ("The world is 6,000 years old") and you interpret evidence only in such a way as to validate that conclusion, what you're doing is not science.

Science must be able to change its conclusions based on the things that are being observed, rather than observing things with an eye toward making them fit the conclusion. This is very hard for human beings to do naturally; we tend to see that which validates our ideas and not see that which doesn't, a pernicious tendency of the brain that goes by the name "confirmation bias." One way to deal with this, as I mentioned in Rule 4, is to search for evidence that prove your ideas wrong, not search for evidence that proves the ideas right. The more firmly an idea resists attempts to knock it down, the stronger it is...at least until that bit of evidence comes along that does knock it down.

RULE 8: Arguments like "They laughed at Galileo!" or "We don't know everything yet, so anything is possible!" are not science

They also laughed at Bozo the Clown, and he is no scientist. Also, it was church leaders, not scientists, who rejected Galileo.

RULE 9: Storytelling is not science

We are a storytelling species. We tell ourselves stories every day to make sense of the world, often without even knowing we're doing it. When someone says something on the Internet that we don't like, we tell ourselves stories about that person's intentions, intelligence, or agenda in order to make whatever she said seem less valid. When we see things happen, we tell ourselves stories about why they happened in order to try to give ourselves a sense of what makes the world behave the way it does (and, often, in order to make ourselves feel like we have control over the world).

These stories are much more satisfying to us than answers like "I don't know." When ancient people looked at the sun and asked "What is that thing?" they invented stories about gods and heros and flaming chariots speeding through the sky. If two people were asked "What is the sun?" and one of them said "The sun is an orb created by a divine all-powerful being who ignited it and threw it into the sky" and the other said "I don't have the faintest idea," the person with the story about all-powerful beings was more likely to be believed, because his story is far more satisfying than "I don't know."

Even today, people tell stories in the face of "I don't know." A religious person might say to a scientist "If the universe came from a singularity that exploded during the Big Bang, then where did that singularity come from?" When the scientist says "I don't know," the religious person's story about gods or the wheel of creation and destruction or the all-beingness of an eternal deity feel more compelling. But they are not science. Just because someone has a story to fill in a hole where someone else says "I don't know" doesn't actually mean that the story is true, and certainly doesn't mean the story is science.

Conspiracy theories are a form of storytelling. We tend to like a good story; we find good stories engaging and memorable, especially when they describe ideas we can relate to. Stories of sinister government plots and secret machinations in the halls of power excite and grab us more than stories about angry religious fanatics who seize airplanes and fly them into buildings; 9/11 conspiracy theories gain traction just because they sound like more entertaining stories and they validate ideas we already have. These stories often assume their own premises (Rule 6), depend on anecdotal information (Rule 4), and/or rely on people speaking outside their areas of competence (Rule 2) as well, and are not science.


( 48 comments — Leave a comment )
(Deleted comment)
Jun. 8th, 2011 12:27 am (UTC)
I've added several.
Jun. 8th, 2011 07:25 pm (UTC)
And, it is still 8 screens long. >.< (It's 12 without the cuts.) I had the opposite reaction and was going to ask why you bothered to put 13 cuts in?
Jun. 8th, 2011 12:19 am (UTC)
To be fair, the reason the church had a problem with Galileo was that they (who Were the peer review process at the time) had a problem with him making (what we now know to be correct) claims that he did not have the data to support.

In other words, while he was right, he was in violation of your Rule 4. He was told politely to limit his claims to what he could support, and then got smacked down when he refused to do.
Jun. 8th, 2011 03:36 am (UTC)
I don't know the details of Galileo's meetings with the Inquisitors, but Tycho Brahe's planetary data plus Kepler's laws were very strong support for heliocentrism.
Jun. 8th, 2011 08:48 pm (UTC)
Galileo's Daughter gives a very insightful look into how the Big G got into papal hot water. He had asked the Pope for permission to publish his findings with his telescope. He got it. But he kinda went too far.

A tip: When representing the Church's position on planetary positioning in a dialog involving three people, never name the Church position's character "Simplicio."

Peter Wilson
Jun. 10th, 2011 08:07 pm (UTC)
Galileo's Venus phase observations
To be fair, science did not have these rules at the time Galileo published his findings about Venus going through phases, like our moon. His conclusions were sound if not "scientific". I think a lot of progress toward these rules were made as a result of his "direct action".
(Deleted comment)
Jun. 8th, 2011 01:32 am (UTC)
LOVE the quote in your icon. Gonna have to read that book now.
(Deleted comment)
Jun. 9th, 2011 10:34 pm (UTC)
Not a problem at all. Thanks!
Jun. 8th, 2011 01:22 am (UTC)
This is a wonderful post, but there are several things I feel the need to point out:

2. Just because someone that's not a scientist says it true doesn't mean they didn't use science to determine their answer.

4a. Although this rule is socially true, it is also scientifically invalid. The reason extraordinarily clams require extraordinarily evidence is because we are psychologically resistant to changing strongly held beliefs. In reality whether or not something is true is completely independent of whether or not the scientific community or the general public chooses to believe it.

4c. Although thought experiments will never hold the credibility of a real-world experiment because it can not be independently verified, it is difficult to over state the value they have to science. Many of Einstein's theories were based on thought experiments that were impossible to verify due to technological restrictions until decades after his death (some still can't be proven true or false).

8. There is a flip side to Rule 8: Using statement like "You're no Isac Newton!" to reduce someones credibility is just as invalid.

Rules 2, 4, 8 & 9 can pretty much all be summed up by one thing: Only verifiable data counts in science. Things like reputation, what is commonly believed to be true, and what seems true have nothing to do with verifiable data, and there for nothing to do with scientific truth.

Lastly, there is a kind of truth that doesn't rely on scientific knowledge. Science is a young thing. Our species is a million years old, yet the scientific method has only been around a few hundred years. We learned how to make fire, work stone, bronze, & iron tools, and even build the great pyramids. All without using modern science. In my non-scientific opinion, it is just as dangerous to discount truth because it is not scientific truth as it is to claim something is scientific truth that isn't.
Jun. 8th, 2011 02:16 am (UTC)
4a. Actually, it is a mathematical inevitability as explained here: http://yudkowsky.net/rational/bayes

4c. Precise mathematical calculations and formulas happen to be part of an infinitesimally tiny subclass of thought experiments that have real world applications. This does not detract from Tacit's general point.

Jun. 8th, 2011 03:27 am (UTC)
4a. Sorry, but we're not talking statistical analysis or probability. We're talking about the amount of evidence necessary to constitute "proof" which is an entirely subjective concept. It would be wonderful if we could some how create a scale of "proof" where x amount of evidence means something is universally considered true, and y amount means it's just an unproven theory. But that's no how society works.

4c. I am not trying to detract from Tacit's point. His point is "the result of a thought experiment is never as compelling as the result of an actual experiment" which I already agreed is true. However, I feel his statements do not fully express the contributions to science that thought experiments have had, and can have to science.

Einstein envisioning that light moving from one star towards Earth, and bending as it passes our own sun was neither a mathematical calculation nor a formula. It was a thought experiment he used to understand the wave/partial nature of light. It helped him to understand that light, a form of energy, has mass just as matter does. It helped him to grasp the significance of interchangeability between matter and energy. Which lead to real world applications. This was a real thought experiment that he documented in his journals that wasn't verified until decades later because the glare of the sun made it impossible to see the light from the distant start.

Thought experiments can lead to significant scientific breakthroughs even if they can't be verified.
Jun. 8th, 2011 09:39 am (UTC)
Regarding 4a: It all rests in how you define "extraordinary claims", doesn't it? In my mind, an extraordinary claim disagrees with a large body of otherwise broadly accepted scientific theories.

Consider all the debates about the existence of "the aether" as a medium for propagating electromagnetic waves. The existence of aether was broadly assumed. It took the Michelson-Morley experiment to truly put the idea of aether to rest. And, it didn't do so overnight. Morley himself didn't believe the result, and it was his experiment.

So, yes, I'm ultimately agreeing with you that "proof" is an entirely subjective concept. I'm just coming at it from a different angle: If your claim requires 10 other claims that are thought to have a 90% chance of being true to actually be false for your claim to be true, then there's only a 1 in 10 billion chance your claim is true. And that's where the extraordinary evidence comes in. Ordinary evidence would have moved that 90% number down well ahead of your claim's, you'd think. So for your extraordinary claim to be true, you either beat the odds (10 billion to 1, in this example), or you truly found something everyone else missed, such that the assumed odds on accepted fact were, in fact, unjustified.
Jun. 9th, 2011 10:43 pm (UTC)
There are people who aren't scientists who say things that are science. This post doesn't suggest otherwise; it merely says that not everything said by a person who is a scientist is science.

There's more to the saying that extraordinary claims require extraordinary proof than just psychology. Extraordinary claims often contradict a great body of observed, experimental data. Consider, for example, extraordinary claims about the lost continent of Atlantis on which a super-advanced technological race once dwelled, or extraordinary claims about psychic healing at a distance. Each of these things would, if true, tend to contradict a vast body of data.

Thought experiments are incredibly valuable to science, that's absolutely true. If and when I rewrite this post for the Essays section of my Web site, I'll probably expand on that more. The value of thought experiments in both suggesting avenues of research and in suggesting ways to construct experiments that might falsify a claim is very high indeed.

Sometimes, though, people who claim that pseudoscientific ideas are science will use thought experiments to support their pseudo science...occasionally with thought experiments whose results contradict actual, real-world experiments. (I've seen some New Agers do this with cosmology, for instance.) A thought experiment is never going to be as strong as a real-world experiment.

There are all kinds of truths that have nothing to do with science. Science offers up only one particular thing: verifiable information about physical processes in the physical world. I would say that early metalworking, the construction of the pyramids, and so on used engineering that had a strong basis in the same kind of systematic approach to the physical world that science uses, but science and engineering aren't *quite* the same thing.
Jessica Burde
Jun. 24th, 2011 01:49 pm (UTC)
"There are all kinds of truths that have nothing to do with science. Science offers up only one particular thing: verifiable information about physical processes in the physical world."

Thank you for this. I get just as tired of scientific fanatics who say if it isn't science it isn't real, as I do of religious fanatics.

I have for a long time had questions about the idea that an idea must be capable of being proven false to be scientific. I totally agree that the hypothesis for an experiment must be disprovable. However, "Falcons hunt and eat small birds" can never be disproven, it is a biological fact based on extensive observation. In fact a single observation of a falcon hunting and eating a small bird will make it impossible to ever disprove. Is this statement not scientifically valid? If so, I need to go back and have a discussion with my Bio 101 teacher.
Jun. 24th, 2011 07:13 pm (UTC)
"Falcons never hunt and eat other birds" is a falsifiable prediction. It's falsifiable in the sense that even a single observation can falsify it.

One of the strongest tools of science is the idea that the best way to test a hypothesis is to recast it as a negative (null) hypothesis. Scientists tend to construct experiments in a way which can falsify the hypothesis. If the hypothesis is "falcons prey on other birds," then what they might do is recast the experiment in terms of "falcons don't prey on birds" and then construct an experiment to falsify THAT hypothesis.

Whether or not falcons eat other birds is a trivial example; as you've said, just a single observation is really all it takes to settle that one. It gets a lot more complicated with more sophisticated hypotheses, particularly hypotheses that can only be tested indirectly.

For example, many religious people wrongly make the claim that the Big Bang model of the universe's creation is not falsifiable, because we can neither go back in time and observe the Big Bang nor create new universes in a lab. However, the Big Bang model made many predictions which are falsifiable. One of those predictions is that the net energy per unit volume of space should have been extraordinarily high at the moment of theBig Bang, and should have dropped as the universe expanded. This model made a very definite prediction; namely, that the whole of the visible universe today should be flooded with microwave energy at a precisely defined frequency.

At the time, this was an extraordinarily risky prediction, as no such background energy had been detected. Sufficiently sensitive radio telescopes to pick it up didn't exist at the time the prediction was made.

The falsifiable prediction is "If the universe began as a singularity and has been expanding since, then all of space should be filled with a residual background glow whose frequency is a function of the volume of the observable universe." Looking for background radiation at that frequency and finding it doesn't prove the Big Bang, of course--people might come up with other explanations for it--but looking and not finding it would certainly tend to disprove the Big Bang.

When the measurements were made, the predicted background radiation was discovered, at precisely the energy distribution predicted by the model--one of the most profound and compelling experimental observations in the history of science.
Oct. 16th, 2011 03:03 am (UTC)
Gotta nit-pick a bit on your nit-picking of 4a. By definition,(outside of the ordinary, or not part of the accepted standard), any data that proved an extraordinary claim would be extraordinary evidence.

"Extraordinary claims require extraordinary evidence." Is just another way of saying "If you're going to claim something that doesn't fit with the standard, you need to back it up with real evidence that wasn't included in the creation of said standard."

This is a little different from the way Tacit explained the phrase though, which usually changes the statement to some variation of "Astounding claims require astounding evidence." Astounding, amazing, huge, and mind-blowing are not synonyms for extraordinary. Better fits would be unusual, uncommon, non-standard and distinctive.

Basically, if I'm reading you right, you disagree with Tacit's explanation of the phrase, rather than the phrase itself. If that is the case, we're in agreeance.

Someone making an extraordinary claim, such as the pyramid building alien idea Tacit mentioned, would not necessarily need "a whole lot of evidence". A single piece of evidence, could suffice, if it stands up to scientific scrutiny. The "evidence" given by people making these claims does not pass muster, and this is where it fails.

This slides right into rule 4b. Any claim made should be based on the quality of the evidence supporting it, rather than the quantity.
Jun. 8th, 2011 01:45 am (UTC)
You had me at Bayes' Theorem (of which Science is merely an edge case :P).
Jun. 8th, 2011 02:05 am (UTC)
Isn't rule 3 a consequence of rule 1? For a claim to be testable, it must be reproducable.
Jun. 9th, 2011 10:45 pm (UTC)
Well, not necessarily. The reproducibility in Rule 3 is reproducibility of results. It's hard to reproduce results on an untestable claim, so something that fails the test of rule 1 never makes it to rule 3, though I would still say they're separate rules.
Jun. 8th, 2011 03:17 am (UTC)
Science can actually be defined much broader than you indicate. Concepts of experimental design and being able to test theories and replicate results are aspects of a specific kind of science, namely positivist quantitative methodologies. Many social science researchers, on the other hand, employ qualitative methods that are very much focused on subjective experiences and contexts that cannot be replicated or generalized to the wider population. Personally, I feel there are benefits to both perspectives, and ranking only one as "true" science seriously limits the quality and type of knowledge that can be gained from the world.
Jun. 8th, 2011 11:21 am (UTC)
"social" and "science"; two words that don't necessarily belong together.

That's not to say that no useful information or knowledge can be gained from subjective non-repeatable phenomena (not all knowledge comes from science).

However the use of "subjective experiences and contexts that cannot be replicated " is a perfect example of people claiming legitimacy by "clamber[ing] onto the scientific bandwagon." You don't need to claim something to be science for it to be valid.
Jun. 8th, 2011 12:23 pm (UTC)
I agree that something doesn't need to be science to be valid (though our dominant paradigm seems to say that if it's not science, it's not real--invalidating many non-white, non-Western European cultures in the process, but that's a whole other issue). But, I disagree that social sciences aren't science. As someone in the social sciences (actually taking a social work research class right now), scientific and evidence-based research are defined as research based on empirical data, which includes qualitative methods.

Others are free to disagree with me of course, but I take issue with the notion that the there is only one acceptable method of science according to the "scientific community," as if there is only one scientific community that always agrees on everything. In reality, these sorts of issues are debated all the time in scientific communities!
Jun. 9th, 2011 10:47 pm (UTC)
Social scientists, if they're doing their job right, aren't claiming things that can't be replicated or generalized as being fundamental processes of the way the world works. :)

I know quite a few social sciences who do, very rigorously, seek not to make claims that can't be tested, and not to make statements whose conclusions can't be reproduced. The claims they make may be probabilistic rather than absolute; they may talk of things like correlations of observable phenomena within populations or societies...but they're still doing science.
Jun. 8th, 2011 03:25 am (UTC)
This is one of the better explanations for the scientific method that I've seen. If it's okay, I may link soon?

One of my favorite stories about scientific evidence overwhelming scientific consensus is the story of Elizabeth Gould's work on neurogenesis. Until Gould's work, it was accepted that the mammalian brain didn't grow new cells after childhood, based on the research of Dr. Pasko Rakic. Dr. Gould found tremendous evidence that new brain cells do, indeed, grow, and in 1999 even Rakic published confirmation of this as well as further supporting evidence.

(One interesting piece of this: It seems that Rakic found that monkeys in labs don't grow new brain cells, which was accepted. Then Gould showed that monkeys do indeed grow brain cells, just not when they're in boring, featureless, stressful laboratory cages. Article here.)
Jun. 9th, 2011 10:47 pm (UTC)
By all means, feel free to link away!

Neurogenesis is cool. :)
Jun. 8th, 2011 01:50 pm (UTC)
The example in 4c is wrong. Or at least simplified to the point of being misleading. I mean, hypothetically you could select two objects of the differing weight values given, and with the proper density / air resistance, get them to fall in the same amount of time under Earth conditions. Or, over very short distances, the difference in speeds will be hard to measure. Or maybe they have the same terminal velocity over very long distances.

But the bottom line is that with two otherwise identical objects, the one with the greater mass will accelerate faster.

And yes, I did just experimentally verify that before I hit 'post'. :)
Jun. 9th, 2011 05:51 pm (UTC)
Maybe air resistance effected your results. The acceleration of Earth's gravity on objects near it's surface is universal at 9.8m per second, per second. A 2lb dumb bell and a 20lb dumb bell both dropped at the same instant from 50 feet high will hit the ground at the same time.

For a detailed explanation: http://www.newton.dep.anl.gov/askasci/gen99/gen99110.htm
For a demonstration see: http://www.youtube.com/watch?v=_wLod-NBq5U
Jun. 9th, 2011 05:56 pm (UTC)
Well, clearly it's been too long since I took physics, I thought I remember that acceleration due to gravity had a mass term. I forgot that the conversion from force to acceleration takes that mass term back out.

My bad.
Jun. 8th, 2011 06:36 pm (UTC)
This is awesome.
Thanks for taking the time to put all of this together.
Jun. 9th, 2011 10:54 pm (UTC)
Re: This is awesome.
I've only read part of it. It's been on my to-do list for a while.

One of my cognitive science professors back in college actually talked about the "science" of race and the origin of the word Caucasian. Supposedly, the "scientist" who coined the term believed, if I am remembering this correctly, that the "purest" humans--hose closest to the "original" human--would be the most beautiful, and that as mutation or whatever took people away from purity it made them uglier. He also felt that the most beautiful humans were those living around the Caucus Mountains, so named everyone who looked kind of like them "Caucasian."

If that's true, it fails on...well, on just about every point, but particularly on rules 1, 4, 6, 7, and 9.
Jun. 8th, 2011 07:43 pm (UTC)
As usual, your writing is both entertaining and informative.

Thank you.
Jun. 8th, 2011 08:54 pm (UTC)
Nice. This is a post that every recipient of a "Nobel" Prize in Economics needs to read. Many of them are worst scientists than your average creationist, but they have the darned statues to shut the critics up.

(Sorry, pet peeve of mine lately, as my LJ reflects. Carry on.)
Jun. 8th, 2011 10:06 pm (UTC)
Thank you for this! It made my day!
Jun. 9th, 2011 04:56 am (UTC)
Rule 6 is a tricky thing. Most scientists who champion a particular theory have some level of suspicion or conviction that theory is true. (For example, the scientists you mention in Rule 4 who proved bacteria cause ulcers, did it by ingesting the bacteria themselves because they were convinced it would give them ulcers... and that antibiotics could cure it.) If in some cases that level of dedication is necessary to prove a theory, where does one draw the line?

PS thanks for writing this, it must have taken forever to build such a tower of text. ^_^
Jun. 9th, 2011 10:50 pm (UTC)
Most scientists who champion an idea do think it's true (else, presumably, they wouldn't champion it), but that's a bit different from using the assumption that it's true as the proof that it's true.

Dr. Marshall had more than the presumption that he was right when he swallowed his sample of bacteria; he also had tangible evidence, in the form of samples of bacteria cultivated from the ulcers of patients.
Jun. 10th, 2011 01:16 am (UTC)
Ahh, I get it now. Sorry, I fixated on the "headline" after getting reading fatigue from the previous 5. What you said makes a lot more sense on a second read.
Jun. 9th, 2011 10:24 pm (UTC)
So climate science isn't science, then
From what I've seen, what passes for climate science nowadays fails on rules 1, 3, 3a, and 4.
Jun. 11th, 2011 10:52 pm (UTC)
Re: So climate science isn't science, then
I'd wager that the key words in that sentence are, "From what I've seen."
Jun. 20th, 2011 11:03 pm (UTC)
Your reply says a lot about you
From your dismissive and insulting tone, I presume you're a Warmist.

I could flood you with a whole lot of links that you would never read, so I'll just point you at what is merely the latest example of why Anthropomorphic Global Warming fails test 3, 3a, and 4:


Short version: every global warming model out there says that there should be a bubble of hot moist air developing over the tropics. But it's not there. Not even a hint of it. What's the response of Warmists to this? They insist the weather balloons, for decades now, have all been wrong. They take their theory over the actual hard data.

That's not science.

What's more, the one thing that has been correlated to the earth's temperature for centuries now, sunspots, are diminishing to the point that talk of a second Maunder Minimum (which happened during the Little Ice Age) has started:


While this may be good news for space travel, it's exactly the opposite of what Warmists are predicting.

So you keep right on with your insults, and we'll see in the long run who's right. I'm fully expecting that in 20 to 30 years the new scientific meme will be that the Earth is headed towards another ice age-- just like it was in the 1970s.
Jun. 25th, 2011 10:29 pm (UTC)
Okay, maybe this one won't get flagged as a "suspicious comment":
Calm down, please. After typing skepticalscience in your URL bar and pressing Ctrl+Enter, stick an "/argument.php" at the end of it. The Ice Age thing-- along with the 164 other false arguments on that page alone-- has been debunked, hardcore. #11 alone is particularly telling regarding things you only think you know.

If by "Warmist" you mean "someone who agrees with the global scientific consensus, and not what oil company scientists or right-wing nonscientific publications say about it," then yeah-- I'm a friggin' Warmist, all right. And proud to be one.
Jun. 11th, 2011 09:31 pm (UTC)
Is there any chance that you're going to reprint this elsewhere? I'd love to share it with some of my family members, but the adult ads on the sidebar make that an impossibility. Cheers. :)
Jun. 12th, 2011 12:13 am (UTC)
Eventually, this will probably make it into the Essays section at www.xeromag.com -- I'm not sure if that's an improvement. :)
Jun. 13th, 2011 06:31 pm (UTC)
Heisenberg's principle doesn't state that we can't know a particle's position and momentum at the same time... We can't know a particle's position and momentum at the same time because the particle does not HAVE defined position and momentum at the same time.
I eventually realized that this is what the uncertainty principle says from things like A Brief History of Time. This took awhile because I first had to get past a complete line of bullshit from my high school physics textbook, where they said the reason they couldn't be known was that you have to bounce photons off of a particle to measure where it is and where it's going, and that's disruptive.
Jun. 15th, 2011 06:32 am (UTC)
I remember having a textbook that said exactly the same thing, which completely misses the point. It wasn't until I did some reading on my own that I found out this explanation wasn't true.
Dane Dormio
Jul. 24th, 2013 09:08 pm (UTC)
All well stated, with a couple of noteworthy exceptions. Your examples regarding the World Trade Center collapse don't seem to be consistent with the rest of your reasoning, which indicates to me that you may not have examined the issue as critically as you could have.

In the first case, I'm just not seeing how the following syllogism is an example of circular reasoning:

A) The WTC collapse looks like a controlled demolition

B) Only controlled demolitions look like that

C) Therefore it is a controlled demolition

Also, just to clarify, in syllogistic reasoning you are supposed to assume that the premises are true, and use them to reach the conclusion. A circular argument is one in which you assume the conclusion to be true and use it to reach itself. Your explanation of Rule 6 seems to conflate the terms "premise" and "conclusion".

In the second case, I think you have it backwards. The story of angry religious fanatics attacking our buildings is one that validates what much of the public would like to believe, while the story of sinister government plots is one that many don't want to believe; and this is why the fanatical terrorist story enjoys broad acceptance despite being empirically weak, while the false flag operation story is widely discredited, despite the overwhelming evidence in support of it.
Jul. 24th, 2013 09:38 pm (UTC)
A common feature of conspiracy theories is that they look at only part of the data, while ignoring information that doesn't fit the conspiracy. In a lot of ways, conspiracy theories are kind of like seeing dragons or elephants in the clouds; our brains seek patterns and similarities that aren't really there.

The example you give has a minor problem and two major problems.

The minor problem is that the WTC collapse only looks like a controlled demolition in one respect (the building fell straight down). It lacked many of the other characteristics of a controlled demolition (for example, there was no evidence of staged explosive charges being set off, and it collapsed from the point of impact down, not from the ground up).

The major problems are the second premise (only controlled demolitions look like that; this simply isn't true) and the vast amount of contradictory evidence that is ignored by the conspiracy theorists.

For instance, we know the buildings were struck by aircraft. We have footage from multiple sources and multiple angles, and literally thousands of eyewitness accounts.

We know the buildings received serious structural damage.

We know the structural damage was greater than the design tolerances of the buildings.

There is also other evidence that discounts the conspiracy theory. Doing a controlled demolition requires rigging large amounts of high explosives to the structural members of a building. This requires removing walls and exposing beams. It is a complex, costly process requiring a large number of man-hours of work by a skilled demolitions crew. It is obvious when a building has been rigged for a controlled demolition. There was no evidence that any of this was done, no eyewitness reports of large quantities of wall paneling being removed to expose structural members, nobody reporting major work being done within the towers before the crash.

The conspiracy theory does not fit the observable facts; the idea that an airplane collision led to the destruction of the towers does.
( 48 comments — Leave a comment )