Avoiding the Einstein Syndrome: Importance of converging evidence

Big misunderstanding
Implication that all problems in science are solved when a single, crucial experiment completely decides the issue, OR that theoretical advance is the result of a single critical insight that overturns all previous knowledge

Hollywood entertainment
*events must have beginnings and satisfying endings that resolve ambiguity or uncertainty
-if taken too seriously, this leads to misconceptions about scientific advance and impairs the ability to evaluate the extent of scientific knowledge on a given issue

Breakthrough model
it is bad
-connectivity principle and principle of converging evidence= describe scientific progress much more accurately

Connectivity principle
-a new theory in science must make contact with previously established empirical facts
-to be considered an advance, it must not only explain new facts, but also account for old ones
-this requirement ensures the cumulative progress of science
*if a new theory accounts for some new facts, but fails to account for a host of old ones, it will not be considered an advance over the old theories and, thus, will not immediately replace them

Einstein’s theories
-the development of the theory of relativity by Albert Einstein is by far the most well known (of connectivity principle)
~in rendering Newtonian mechanics obsolete, Einstein’s theories did not negate or render meaningless the facts about motion on which Newton’s ideas were based–>predictions turned out equal
~Einstein’s conceptualization is superior because it accounts for a wide variety of new, sometimes surprising, phenomena that Newtonian mechanics cannot accommodate

Beware of violations of connectivity
breakthrough model violates principle of connectivity
-when the principle of connectivity is abandoned, the main reasons come from pseudoscience and bogus theories–such theories derive part of their appeal and much of their publicity from the fact that they are said to be startlingly new

Make the theory un-falsifiable (useless)
one strategy to get rid of violation of connectivity

2nd strategy to get rid of violation of connectivity
dismiss previous data by declaring them irrelevant
-this is usually accomplished by emphasizing what a radical departure the new theory represents
-“new conception of reality” and/or “radical new departure”
-the new theory is deemed so radical that experimental evidence derived from the testing of other theories is declared irrelevant
-the old, “irrelevant” data are gone, and the new, relevant data do not exist

Darwin using the connectivity principle and abandoning an idea when it failed to display the necessary continuity with the rest of science
search for a mechanism of heredity to go with his theory of natural selection
-“in which little gemmules, given off by all of the body parts, circulate around the body and eventually collect in the sex organs, from where they are ready to start the next generation”
~does not cohere with cell theory
~”did not cohere with the rest of biology”

No experiment in science is perfectly designed
-there is a degree of ambiguity in the interpretation of the data from any one experiment
-scientists often evaluate theories not by waiting for the ideal or crucial experiment to appear, but by assessing the overall trends in a large number of experiments–each with a different limitations

Many sciences have progressed even though they are without an Einstein
-their progress has occurred by fits and starts, rather than by discrete stages of grand Einsteinian synthesis
-many other sciences are characterized instead by growing mosaics of knowledge that lack a single integrating theme

Experiments with limitations
-if all the experiments were limited in “different” ways, our confidence in the conclusions would be increased because it is less likely that the consistency in the results was due to a contaminating factor that confounded all the experiments
-” different methods are likely to involve different assumptions. When a conceptual hypothesis survives many potential falsifications based on different sets of assumptions, we have a robust effect” -Anderson and Anderson

Converging evidence
-when evidence from a wide range of experiments points in a similar direction, then the evidence has converged
-the principle of converging evidence urges us to bases conclusions on data that arise from a number of slightly different experimental sources–allows us to draw stronger conclusions because consistency that has been demonstrated in such a context is less likely to have arisen from the peculiarities of a single type of experimental procedure

Theory testing
-research is highly convergent when a series of experiments consistently supports a given theory while collectively eliminating the most important
-although no single experiment can rule out all alternative explanations, taken collectively a series of partially diagnostic experiments can lead, if the data patterns line up in a certain way, to a strong conlcusion

Example of theory testing
A B C D E= five different theoretical accounts
-one experiment tests A B C refute theories A & B and support C
-another experiment tests C D E and refute theories D & E and support C
*strong converging evidence for C

Another example of theory testing
-if both experiments represented strong tests of B C and E, and the data of both experiments strongly supported C and refuted B and E, the overall support for theory C would be less strong than in previous experiment
-because even though data supporting theory C, no strong evidence ruling out two viable alternative theories ( A and D)
*just as theories are confirmed by converging evidence, they are also dis-confirmed by converging results

Experiments in psychology are usually of fairly low diagnostically
-the data that support a given theory usually rule out only a small set of alternative explanations, leaving many additional theories as viable candidates
-progress is slow; need a lot of studies to make a conclusion

Converging evidence in psychology
-we must resist the temptation to regard a particular psychological hypothesis as “proven” when the evidence surrounding it is still ambiguous (uncertain)
-convergence allows us to reach many reasonably strong conclusions despite the flaws in all psychological research

Question of whether exposure to violent t.v. programming increases children’s tendencies toward aggressive behavior
-the viewing of violent programming does appear to increase the probability that children will engage in aggressive behavior
~convergence of the results of dozens of different investigators
*the general research designs, subjects, populations, and specific techniques used in these investigations differed widely, and these differences are a strength of research in this area, not a weakness

More stuff of converging evidence
-this research may itself have problems, but other studies have corrected for these and have also produced similar results
-it was correctly pointed out that this correlational evidence did not justisfy a causal conclusion–>could have third variable problem or directionality problem

Correlational studies
-the conclusion of the scientific community is NOT based on correlational evidence alone
~longitudinal design= measurements of the same two variables ( television violence and aggression)–didn’t suggest causal connection
*numerous laboratory studies have been conducted in which the amount of televised–manipulated rather than merely assessed

“it’s-not-real-life” argument
*results (of t.v & violence) have been replicated in different regions of the US and in several countries around the world.
-results have held up despite set-up and programs

Field experiment has been used to investigate televised-violence/aggressive-behavior issue
-the existence of this type of design reminds us to avoid assuming a necessary link between experimental design and experimental setting
-correlational studies are often conducted in laboratories, and variables are often manipulated in non laboratory settings

Field experiment used to test so-called broken windows theory
smoking and lung cancer:
-attempt to mislead public by implying that the conclusion that smoking causes lung cancer–> instead strongly supported by wealth of converging evidence (several studies)

Epidemological studies, highly controlled laboratory studies using animals, and clinical trials with human patients
-when the results of all these types of investigations point to a similar conclusion, medical science feels assured of the conclusion, and physicians feel confident in basing their treatment on evidence
*epidemiological studies are always correlational
-possibility of spurious links between variables are high

Lab studies
-can be highly controlled
-subjects are often animals rather than humans

Clincial trials
-in a hospital setting use human subjects in real treatment contest
-many problems of control because of placebo effects and the expectations of the medical treatment team that deals with the patients

Converging evidence is hard for the public to understand
-it is entirely possible to draw a sound conclusion despite flaws or limitations in each and every test and study that constitute the evidence for the conclusion
-sometimes the principle of converging evidence is unknown to people and other times it seems to be consciously ignored in order to advance a political agenda of financial advancement–> lung cancer and smoking, cell phones and driving= highly convergent (not causation)

Scientific consensus
-when evaluating empirical evidence in the field of psychology, think in terms of scientific consensus rather than breakthrough- in terms of gradual synthesis rather than great leap

Failure to appreciate the “consensus rather than breakthrough”
-this has impeded the public’s understanding of the evidence that human activity is a contributor to global warming
-political groups didn’t like that–wished to create doubt among the public–they were successful
-surveys show that about 50% of the public think that scientists are still debating the conclusion when in fact a strong convergence has been achieved
*scientific conclusions derive form convergence and consensus–media tends to portray things in the opposite manner

The convergence principle also implies that we should expect many different methods to be used in all psychological research areas
-different strengths and weaknesses
-trend in recent years has been to expand the variety of methods used in all areas of psychology (social psychology)

Example: unresponsive bystander phenomenon
-did fake robberies (filed exp.) along with just watching (lab)
-consistent with the lab results, the presence of another individual inhibited the tendency to report the theft

Research convergence is not always positive, in the sense of supporting the original hypothesis
-sometimes research converges on a negative conclusion–that the hypothesis that was originally posited cannot be supported
-learning styles in educational psych–> there is no replicable evidence that teachers can “match” instruction to these styles in ways that lead to greater learning (negative conclusion)

The proper role of case studies
to suggest hypotheses for further study with more powerful techniques and to motivate scientists to apply more rigorous methods to a research problem

The progression
from case studies, to correlational studies, to experiments with manipulated variables

Correlational studies can contribute to knowledge
-many scientific hypotheses are stated in terms of correlation or lack or correlation, so that such studies are directly relevant to these hypotheses
-although correlation does not imply causation, causation does imply correlation–although a correlational study cannot definitively prove a causal hypothesis, it may rule one out

Correlational studies are more useful than they seem
-some of the recently developed complex correlational designs allows for some very limited causal inferences
*the complex correlational technique of partial correlation, in which it is possible to test whether a particular third variable is accounting for a relationship

Inherently correlational
-some variables simply cannot be manipulated for ethical reasons–human malnutrition, physical disabilities, etc.

Link between type A behavior pattern and coronary heart disease: case study
-originated in a few case studies made by some observant physicians

Link between type A behavior pattern and coronary heart disease: correlational
-researchers developed and tested operational definitions of the type A concept– large scale epidemiological studies established a correlation between the presence of type A behavior and the incidence of CHD

Link between type A behavior pattern and coronary heart disease: complex correlational
-researchers used complex correlational techniques to track down potential third variables
-when other variables (smoking, obesity, serum cholesterol level) were statistically