Dewdney, A.K. (1997), Yes, We Have No Neutrons: An Eye-Opening Tour through the Twists and Turns of Bad Science, NY: Wiley.
This book is allegedly an expose of bad science. Some chapters are indeed about bad science, such as the ones on cold fusion and Freud. Other chapters attempt to discredit various legitimate fields of scientific research by criticizing certain flawed results from early in the history of the field, while ignoring contemporary research. In particular, the chapter on what Dewdney calls "the neural net debacle" (p. 97) gives a false picture of current research on artificial neural networks.
Douglas R. O. Morrison reviewed Dewdney's book in the November, 1997, issue of Scientific American. Morrison, a researcher at CERN for 38 years, said:
Unfortunately, I was disappointed by the chapters that cover the two episodes of bad science that I have studied most seriously: N-rays and cold fusion. In both cases, Dewdney's accounts gave the impression that he had read a few review papers and distilled them without arriving at any deep understanding. In addition, the chapters contain factual errors. ... I was surprised to see that the sad and revealing story of neutrons claimed from cold fusion is barely described and its essence omitted.Much the same could be said about Dewdney's chapters on IQ, Freud, neural nets, and J. Phillipe Rushton. But Morrison goes on to say:
Other chapters in Yes, We Have No Neutrons stick more closely to Dewdney's specialties of mathematics and computers. These sections seem clearly written and explained.This remark shows how easily an intelligent scientist can be taken in by writing of tabloid quality when the subject is outside of his or her field. Lurid scandals are fascinating to most people, whether they are about the Royal family, chemists, or psychometricians. Sometimes the scandals are based on fact; sometimes they are invented by magnifying small faults out of all proportion to the evidence. Seeing the alleged failings of famous people can be gratifying to those of us who harbor self-doubts--and even the greatest scientists, such as Darwin, may have self-doubts. Sensationalism sells in both tabloids and popular science books.
Dewdney's attack on neural nets begins with a legitimate complaint. Many fantastic claims have been made about artificial neural nets, especially in the popular press, but also in some professional works. Dewdney says (pp. 80-81):
The image of neural nets as miniature brains was pumped in the papers, science magazines, on public TV, and even in a few Hollywood movies. Neural network proponents (also called connectionists) did not rush to their phones to disavow these wild claims.The first sentence above is true. The second sentence sarcastically insinuates that neural net researchers have done nothing to correct this well-known "hype" about neural nets. The many books listed in the neural net FAQ <ftp://ftp.sas.com/pub/neural/FAQ4.html> under "The Best" generally say something to the effect that research into artificial neural nets was inspired by biological neural nets, but they do not contain such wild, over-blown claims about the intelligence of neural nets. Hecht-Nielsen (1990) even has a section called "Hype" disavowing the "wild speculations" that some researchers have perpetrated.
Dewdney could have written an interesting chapter comparing some of the more outlandish claims about artificial neural nets with some realistic applications. But Dewdney actually goes to the other extreme and argues that (p. 82):
Although neural nets do solve a few toy problems, their powers of computation are so limited that I am surprised anyone takes them seriously as a general problem-solving tool.Such claims are thoroughly refuted by the books listed in the neural net FAQ under "The Best" and by the numerous links and references to successful neural net applications under "What can you do with an NN and what not?" There is no point in repeating such information in this review. But it may be interesting to examine Dewdney's argument to see how it is constructed, and how it could be so utterly wrong.
To provide a proper appreciation of the poverty of Dewdney's reasoning, I must discuss an apparently unrelated chapter and another book. The other chapter is the one on IQ, and the other book is Gould's (1981/1996) famous diatribe on IQ and related matters. Gould's book is very controversial, and it is not the purpose of this review to argue about any of the specific claims made by Gould (for a very clear, concise, and well-balanced account of current research on intelligence, see Hunt, 1997). The points I wish to make are concerned with Gould's style of argument.
Dewdney owes a great deal to Gould. Dewdney's chapter on IQ is largely a rehash and simplification of Gould's book. But Dewdney has borrowed more than Gould's information; he has borrowed Gould's style of argument and applied it not only to IQ but also to neural nets. This style of argument is used by Gould and Dewdney to discredit entire fields, and it goes as follows:
Gould reaches all the way back to such obscure sources as Morton's craniometry in 1839. He devotes more attention to famous people such as Broca, Binet, Goddard, Terman, and Spearman, who are also discussed in considerably less detail by Dewdney. Gould extends his historical survey into the mid-twentieth century, including Burt and Thurstone, whose work is so recent that it is neglected by Dewdney. The only major work since 1950 that Gould discusses in any detail in his first edition is Jensen (1979). Dewdney has half a page on Jensen but says virtually nothing about the actual content of his work.
As for neural nets, Dewdney goes back to Rosenblatt's perceptrons, then mentions Minsky and Papert (1969) as "The book that nearly killed neural net research." Dewdney then skips forward to the n-queens problem and the traveling salesman network of Hopfield and Tank (1985), and the network of Rumelhart and McClelland (1986) for learning past tenses of English verbs. The development of backpropagation had just barely begun in 1986, so that year is indeed ancient history by neural net standards. Dewdney also discusses work by MacWhinnie and Leinbach on past tenses, but provides no reference.
Gould derides nineteenth-century craniometry in lavish detail. How "surpassingly weak" was Broca's argument that brain size determines intelligence! Goddard is excoriated for claiming that normal intelligence could be a function of a single Mendelian gene. How "absurd" for Goddard to think that 80% of immigrants could be feeble-minded! Gould mocks Spearman for his "immodesty" and "physics envy" in claiming to have objectively measured general intelligence ("g"). What "folly" for Spearman to argue that "g" is evidence for the existence of mental energy! Gould accuses Burt of fraud and reification, and remarks on the "extraordinary weakness of his reasoning for the innateness of intelligence." How "blind" was Burt's commitment to hereditarian biases! All of these arguments by Gould are at least partly valid.
Dewdney repeats some of Gould's charges against dead psychometricians, but he confuses principal component analysis with common factor analysis, and therefore misses all the subtlety of Gould's arguments about "g" and reification (which Dewdney calls "thingifying"). But Dewdney did not read Gould carefully, for Dewdney says (p. 36), `Spearman was hard put to identify any specific mechanisms or to throw any light on just what "intelligence" was.' Apparently Dewdney missed Gould's discussion of Spearman's theory of mental energy and engines and explanation of "g" as the eduction of relations and correlates (pp. 296-200 in Gould, 1996). However, Dewdney one-ups Gould by actually citing a 1946 study by B. Schmidt in Psychological Monographs (no volume or page numbers) in response to his own rhetorical question, "Is IQ inherited?" This study found a 20-point increase in the IQs of "feebleminded" children after an intensive three-year training program. Dewdney thinks that this study demonstrates that IQ is not inherited "to any significant degree." Thus are Galton, Goddard, Spearman, and their ilk refuted.
As for neural nets, Dewdney introduces a personal touch when he debunks Hopfield's (no reference) n-queens network--Dewdney tried it himself (p. 89) and it did not work for seven or more queens (no reference to Dewdney's results). Dewdney was "understandably skeptical" (p. 89) of the traveling salesman network (Hopfield and Tank, 1985; Dewdney gives no reference). This time Dewdney assigned a graduate student to test the network, which failed with eight or more cities. Dewdney also describes MacWhinnie and Leinbach's hubris in challenging other researchers in artificial intelligence to beat the performance of their past-tense network, but their network was "outperformed ... in all respects" (p. 96) by a Symbolic Pattern Associator (SPA) developed by Charles Ling (no reference). These three cases are the only empirical evidence that Dewdney provides for his claim that artificial neural nets are "voodoo technology" (p. 92).
Gould (1981) ignored virtually all research after 1950 except for Jensen (1979). Gould convincingly challenged Jensen's grasp of paleontology (for example, Jensen's references to the evolution of "the fish" and "the turtle"), but he ignored all of the empirical results on mental testing in Jensen's 786 page book. In the second edition (1996), Gould chose `to leave the main text essentially "as is" because the basic form of the argument ... has never varied much, and the critiques are similarly stable and devastating' (p. 48). The extensive research that Gould has ignored is summarized by Rushton (1997). Gould even ignored research that favors his own position (e.g., Hunt, 1997). Dewdney makes an even more astonishing claim than Gould: "The state of our knowledge about this key phenomenon [intelligence] has remained in almost the same state for the last hundred years" (p. 43). For a clear and compelling refutation of Dewdney's claim, see Hunt (1997).
As for neural nets, the original Hopfield-Tank (1985) network for the traveling salesman problem had bugs in it, but these bugs were quickly corrected by other reserachers, and there has been considerable progress in the area since Hopfield and Tank's pioneering work (Potvin, 1993). Neural network methods have been developed for a wide range of optimization problems (Cichocki and Unbehauen, 1993). Still, the TSP is not a common application of neural nets. And while the n-queens problem and past tenses of verbs may be of some academic interest, they are unrelated to any practical applications of neural nets. But Dewdney says the past-tense network's defeat by the SPA "may be a trend in the making" despite noting that "One example proves nothing" (p. 96). It would be more accurate to say that inferring the direction of a trend from one example is mathematically impossible. The literature on neural nets is growing at rapid rate, and Dewdney has ignored virtually all of it (for a sampling of recent theory and applications, see Mozer, Jordan, and Petsche, 1996, and Chen 1996). The extent of Dewdney's ignorance is revealed by this condescending remark (p. 96):
As we examine the real powers of neural nets, we will doubtless discover many areas of genuine competence for them.As if we had not already discovered many areas of genuine competence for neural nets!
When it comes to pattern recognition, there are statistical classifier engines that outperform neural nets.The meaning of the above comparison is not at all clear, because many commonly-used neural nets are in fact statistical classifiers. The intimate connections between neural nets and statistics are now well-known (e.g., Bishop, 1995; Ripley, 1996). Even if one were to arbitrarily assign methods that have been independently invented by both statisticians and neural network researchers to one camp or the other, there is no way Dewdney's comparison could be universally true (Michie, Spiegelhalter, and Taylor, 1994). Moreover, since many kinds of neural networks are based on sound statistical theory, if neural nets are "voodoo technology" (p. 92), then statistical models must also be "voodoo technology".
Dewdney, a mathematician, makes a serious mathematical error on p. 88:
... the algorithm [backprop] may lead to a local peak in n-dimensional space. In fact, the more dimensions there are, other things being equal, the more opportunity a mountain range has for developing local peaks. This is not just a metaphor.Dewdney's mountain range is the (negative) error function, which he describes in a passage remarkably similar to the kangaroo discussion in ftp://ftp.sas.com/pub/neural/kangaroos.txt, but with a mountain climber in place of the kangaroo. But the fact is that the more hidden units in an MLP with one hidden layer--and hence the more dimensions in the weight space--the less the chance of being trapped in a bad local optimum (see "How many hidden units should I use?)." The fact that increasing the number of hidden units reduces the risk of bad local optima is one of the most important properties of neural nets; without this property, it would indeed be impractical to use neural nets for complicated problems. It is obvious from the above statement that Dewdney has given no serious consideration to the mathematical properties of neural nets.
Dewdney's antiquated philosophy of science owes more to Francis Bacon (pp. 6-11) than to contemporary scholarship. Dewdney provides a diagram of the "scientific method" (p. 12):
question
||
\/
hypothesis
||
\/
experiment or observations
||
\/
conclusions (and publication)
Any recent book on philosphy of science refutes such a simplistic
view of the scientific method--for example, Klee (1997) or Bauer (1992).
Can someone who misunderstands scientific method as badly as Dewdney
really distinguish between good science and bad science? In his introduction, Dewdney refers to scientists as "sorcerers" and bad scientists as bumbling "apprentices." Can someone who calls science "sorcery" really distinguish between good science and bad science? I suggest the reader compare the books by Gould and Dewdney and decide who the real sorcerers and apprentices are.
References:
Bauer, H.H. (1992), Scientific Literacy and the Myth of the Scientific Method, University of Illinois Press.
Bishop, C.M. (1995), Neural Networks for Pattern Recognition, Oxford: Oxford University Press.
Chen, C.H., ed. (1996) Fuzzy Logic and Neural Network Handbook, NY: McGraw-Hill.
Cichocki, A., and Unbehauen, R. (1993). Neural Networks for Optimization and Signal Processing, NY: Wiley.
Gould, S.J. (1981/1996), The Mismeasure of Man, NY: W. W. Norton.
Hecht-Nielsen, R. (1990), Neurocomputing, Reading, MA: Addison-Wesley.
Hopfield, J.J., and Tank, D.W. (1985), `"Neural" computation of decisions in optimization problems," Biological Cybernetics, 52, 141-152.
Hunt, E. (1997), "The concept and utility of intelligence," in Devlin, D., Fienberg, S.E., Resnick, D.P., and Roeder, K., eds., Intelligence, Genes, and Success: Scientists Respond to "The Bell Curve", NY: Springer-Verlag.
Jensen, A.R. (1979), Bias in Mental Testing, NY: Free Press.
Klee, R. (1997), Introduction to the Philosophy of Science: Cutting Nature at Its Seams, NY: Oxford University Press.
Michie, D., Spiegelhalter, D.J. and Taylor, C.C. (1994), Machine Learning, Neural and Statistical Classification, Ellis Horwood.
Minsky, M., and Papert, S. (1969), Perceptrons, Cambridge, MA: The MIT Press.
Mozer, M.C., Jordan, M.I., and Petsche, T., eds. (1996), Advances in Neural Information Processing Systems 9, Cambridge, MA: The MIT Press.
Potvin, J.-Y. (1993), "The traveling salesman problem: A neural network perspective," ORSA J. Comput., 5, 328-347.
Ripley, B.D. (1996) Pattern Recognition and Neural Networks, Cambridge: Cambridge University Press.
Rumelhart, D.E., and McClelland, J.L. (1986), "On learning the past tenses of English verbs," in McClelland, J.L., Rumelhart, D.E., and the PDP Research Group, Parallel Distributed Processing: Explorations in the Microstructure of Cognition, Volume 2: Psychological and Biological Models, Cambridge, MA: The MIT Press.
Rushton, J.P. (1997), "Race, intelligence, and the brain: The errors and omissions of the `revised' edition of S. J. Gould's The Mismeasure of Man (1996)," Person. individ. Diff., 23, 169-180.