Showing posts with label belief. Show all posts
Showing posts with label belief. Show all posts

Saturday, January 11, 2014

Is science all GUTs and TOEs? An anatomy lesson in science as it never was (Scientific R&D, Part 2.1.1)

Apologies to readers for my quarter-year hiatus.  Both of you.  As it turns out, actually trying to get research projects off the ground while simultaneously making a living gets in the way of writing about science, though fortunately not of thinking about it.  In the interim since my last post, I have taken a consulting position at a center that provides computing resources (and support from consultants like me) for social scientists, which is pretty cool.

To get back up to speed, previously I argued that the scientific enterprise exhibits a division of labor similar to a manufacturing industry, with separate divisions dedicated to Research and Development (R&D) and Quality Control (QC).  In this metaphor, the "products" that the R&D people develop are explanations of the phenomena that we encounter as we go about our daily affairs (although scientists in many fields now regularly encounter phenomena that we would never in a million years encounter without deliberately seeking out strange new things ... peculiar bioluminescent critters at the bottoms of the oceans; bizarre goings-on at the farthest reaches of our galaxy and of our universe; subatomic particles with outlandish names like "neutrino," "quark," and "boson"; and so on).  The people in the QC division, on the other hand, evaluate just how good these explanations actually are, whether they be freshly minted by R&D or whether they've been "on the market" for a while.  More literally, QC tests these explanations for their goodness of fit with the world they were designed to explain, by subjecting them to further observation.  Ideally, when QC discovers that a given explanation does not fit well with observation, this explanation is discontinued, and it's back to the drawing board for R&D.  Obviously, the labors of the two divisions are complementary (in an antagonistic sort of way), because the R&D people are great at brainstorming new explanations but incapable of getting rid of bad ones, while the QC people are incapable of generating new explanations but great at making room for better ones.  (As with any metaphor, this one falls apart if pushed too far.  Importantly, most researchers do not exclusively work in one division or the other; most of us spend our time bouncing back and forth between the two labors.)

I also argued that R&D's operations are not distinctively scientific; people have been in the business of coming up with ideas about how our world works for a long, long time, long before scientists achieved any sense of scientific self-awareness, and long before navel-gazers like me began to think so hard about what scientists really do or how this compares to/contrasts with what other people do.  Instead, it is the operations of the QC division that makes the scientific enterprise definitively scientific.  what is unique about the scientific production of knowledge is that it endangers ideas about the way the world works - hunches and sacred beliefs alike - by pitting them against observation.  In other words, if you aren't willing to endanger your understandings of the world in the crucible of observation, you aren't being a scientist.

While it is QC's activities that make science science, this in no way implies that we should remain indifferent to the operations of the R&D division.  On the contrary, how the R&D people conduct their business has a major influence on the quality of ideas that eventually make it into our belief systems.  Consequently, an audit of R&D's standard operation procedures is worth doing.  In my last two posts, I began to audit R&D by exploring what it is that catalyzes the division's operations in the first place.  The answer to this question is that R&D scientists begin by being puzzled about the world's peculiarities, leading them to explicitly ask questions about such things.  The understandings that R&D produces and passes along to QC are initially generated as answers to such questions.

But that is only the beginning of the life of R&D's ideas, and we are arguably more concerned with what happens to them after they are born (or put on the market, to go back to the earlier metaphor). Given that some of these ideas are going to stick around for a while if they make it through QC, what qualities do we want them to have?  In this post, I come at this question somewhat negatively, by dismantling a widely held misunderstanding about what kinds of explanations R&D scientists should be putting out there.  This misunderstanding is so widely held, in fact, that many scientists share it, particularly new ones, and some even regard it as a necessary condition for scientific knowledge production.


How should scientific knowledge be organized?

Way back in my first post, I stated that one of the goals of scientific practice is to achieve better-organized understandings of our world.  This goal has tremendous appeal if for no other reason than that good organization makes for the clear and efficient communication of scientific understandings to potential consumers.  Of greater importance (in the minds of many of science's most outspoken advocates) is the assertion that the organization of our knowledge about the world ought conform to
the order characterizing the world itself; if the world is governed by a relatively well-organized and simple set of laws or principles, then our belief systems ought to be, as well, and preferably in the same way.

This is a pretty huge and important claim, not only because of the frequency with which it is presented as one of the fundamental axioms of scientific practice, but also because it is not a salient feature of many other approaches to knowledge production.  Given the quickness with which some scientists assert the mandate of seeking better-organized beliefs, we ought to understand what we actually mean when we say 'well/better-organized.'  As it turns out, as with so many other important and commonly used words, we think we have a pretty good grasp on what 'organization' means right up until we start thinking about it.

What is worse, when we actually examine how scientists use the word, we discover that their usage conceals a few important alternative meanings, increasing the opportunity for equivocal or amphibolous arguments.  One of the most well-developed outlooks on organization in science, called "unificationism" (because no label is worth using if it has any less than seven syllables), collapses three separate meanings under this single heading, imposing all three mandates on scientific practice in a single breath.  First, there is the minimal requirement that no two understandings about how the world works can contradict each other, because the world quite simply cannot both be one way and its opposite (this is what logicians refer to as the "law of non-contradiction").  Second, there is the requirement that scientific knowledge ought to be well-integrated, meaning that that multiplicity of beliefs that populate our belief systems should be tied together as networks of belief, with a clear linkage tying any given belief to at least one other.  And third, there is the requirement that scientific knowledge ought to simplify these networks of belief, either by reducing the number of distinctive beliefs that populate them, or by imposing a specifically hierarchical structure upon the network which collapses the diversity of beliefs down as merely specific cases of increasingly general principles.  The relevance of these three dogmas for scientific practice is that they put the burden of unification on R&D scientists: if a freshly minted hunch about how the world works is to leave the drawing board for trial by QC, it should be the sort of hunch that would fit well with the preexisting array of knowledge, and preferably simplify it a bit.

Considered in isolation, each of these three requirements already makes a bold assertion about the ideal structure of scientific knowledge.  Thus, considered together, their collective boldness is too great to continue accepting them without further reflection and scrutiny.  On the contrary, we need to ask two questions of each requirement: first, is it practically achievable, and second, will it guarantee a more realistic understanding of the world if achieved?  My first contention is that well-integrated belief systems are more realistic than less-integrated ones, but these are not always practically achievable.  Even if such integration remains to be achieved, however, my second contention is that the much weaker criterion of internal consistency should be maintained at all costs.  ('Integration' and 'internal consistency,' by the way, are often indiscriminately lumped under the common label 'coherence,' so it might be useful to think of integration as a strong variety of coherence and internal consistency as a weak variety, but more on that below.)  My third contention is that the unrelenting push for simplification is not only not always feasible (for the same reason that network integration is not) but more importantly is an unwarranted rule of thumb, with the potential to yield less realistic understandings rather than more.  In short, if we as scientists are being honest with ourselves, then we need to confront the fact that the unificationist agenda is not only impracticable but also unlikely to yield the realistic understandings we desire.


Unification as integration

Scientific inquiry is undeniably oriented toward clarifying the relationships that exist between the entities that populate nature/the universe, their properties and behaviors, and variability therein.  Scientists are not satisfied with the vague intuition that everything is connected.  Instead, we want to understand the particular details about how any two things or sets of things are connected, as well as what the strength of this connection actually is.  So, it is hard to deny that integration is at the heart of scientific practice.  Almost always, the new beliefs that R&D posits are beliefs about previously unexpected linkages between entities that we want to understand better.  All else being equal, then, the addition of such understandings to a prior collection of beliefs will tend to increase its degree of integration.  There are, however, a couple of factors that work in the opposite direction.  First, there are instances of subtraction, which can occur in either of two ways: (1) previously accepted items of knowledge are challenged and ultimately dismantled by QC work; or (2) some items of knowledge are simply forgotten: their currency diminishes as they age until eventually, they succumb to their senescence and vanish out of mind.  By either route, the result is the dissolution of a previous linkage, fragmenting the network of beliefs to a degree.

Second, there is a logistically problematic dynamic, in which the relentless outward/downward/upward/inward expansion of science into new frontiers occurs at a pace that, from all indications, is outstripping R&D's finite capacity to integrate incoming items of knowledge.  On this point, it is perplexing that some of science's most well-known spokespeople are on the one hand keenly sensitive to the vastness of the unknown frontiers (there are more quotes on this topic than I have the room to recite) and on the other so insistent that we might one day hope to integrate all of the knowledge that will eventually come in about these frontiers.  As it is, our capacity to hold items of belief in memory is limited enough that no single individual can claim to possess their own personal and exhaustive copy of current scientific knowledge; the global body of knowledge about the universe (some of it scientifically sanctioned, some gained by other means), resides not in the memory of any individual but instead in an abstract, collective memory, comprising human memory as well as the various external information storage devices that we rely upon so heavily.  As one philosopher of science points out, we are obliged to take a piecewise approach to our understanding of the universe - i.e. in which one set of understandings applies to some aspects of our universe while other sets apply to other aspects - as a result of our inherent limitations.  Thus, in conjunction with the proliferation of scientific research, our intrinsic cognitive limitations make the prospects of synthesizing scientific knowledge bleak indeed, more so with each passing day.  Because of this and because of the relentless give-and-take of the R&D/QC antagonism, the notion of a perfected integration of scientific knowledge is quite simply antithetical to scientific practice, not the inevitable end of it.  (See these two articles, [1] and [2], on the state and future of particle physics in the wake of the discovery of the Higgs' boson and this one on the idea of culmination in physics more generally.)  In short, we will likely never reach a point where we can actually test the oft-cited claim that everything out there is related to everything else out there, but we can at least say, with considerably greater confidence, that our current state of knowledge is most certainly not one in which our beliefs about everything are connected to our beliefs about everything else.

Giving up on the quest for global synthesis, however, should not discourage us from seeking out more local connections.  Unlike global synthesis, small-scale integration is the stock and trade of science - we actually do it, all the time! - so it is anything but futile.  Ironically, it is also these small, self-restrained projects, that reassure us that we are not hopelessly adrift in a sea of disjointed ideas or a universe of unrelated things.  In my opinion, it is particularly exciting when we identify compelling linkages that cross-cut the conventional disciplinary boundaries.  (Though I am biased in this respect, because my two chosen disciplines - archaeology and demography - both celebrate their interdisciplinarity, more so than, say, physics.)

Similarly, learning to accept the eclecticism that inevitably characterizes our belief systems should not be mistaken for also accepting contradictions between locally networked subsets of belief, for two reasons: first, the potential to integrate presently unrelated sets of belief in the future remains open provided that they do not possess contradictory elements.  And second, we axiomatically accept that the universe itself does not possess any self-contradicting elements: if something is true of the world, it cannot also be false of it (again, this being the law of non-contradiction).


Unification as simplification

While the requirement of integration is problematic in its own right, this problem is relatively minor compared to the additional requirement that we seek to simplify our belief systems through the imposition of hierarchical structures upon them.  Given the preceding critique of global integration, we can already conclude that global simplification is all the more impracticable.  On the other hand, just as with research focused on achieving local integrations, many researchers conduct research focused on local simplifications, and sometimes these do stand up to the scrutiny of QC.

To expand a bit, what this kind of hypothesis does is reduce the number of phenomena that require unique explanation, instead accounting for multiple outcomes as if redundantly governed by the same principle.  These one-cause-to-many-effects statements are sometimes called "covering laws," while research that is conducted in their pursuit is sometimes labeled "nomothetic" (from Greek 'nomos' [custom, law] + 'theto' [to place]) or "nomological" (from Greek 'nomos' [custom, law] + 'logos' [discussion, discourse]) ... though these labels don't have as much currency now as they did a few decades ago.

Hiding under the label of "nomothetic/nomological research," however, we can distinguish between two subtly different kinds of activity: first, there are those research activities that posit and test statements that treat a particular set of phenomena as if commonly governed by a precise rule, which can then be tested as a hypothesis against observational data.  For example, I could propose that my car's average MPG for a given period of time (call it TMPG) conforms to the relationship

    TMPG = HMPG(p) + CMPG(1-p)

where p is the proportion of miles spent on highways during the study period, HMPG is the average MPG for highway driving, and CMPG is the average MPG for city street driving.  While this is a relatively low-level law pertaining to a rather trivial corner of all of the amazing things that go on in nature/the universe, it is nonetheless a law insofar as it asserts that TMPG values for different time periods conform redundantly to a single sort of relationship: every single TMPG is related to p, HMPG, and CMPG in the same sort of way as every other TMPG.  This approach is remarkably different from any other approach that would assert that TMPG values for different periods are each dictated by their own equation, involving different mathematical operators, different orders of operation, and/or different variables than p, HMPG, or CMPG.  I can scrutinize the accuracy of this lawful understanding of TMPG if I also have some prior knowledge of HMPG and CMPG for my car and if I can measure p for particular driving periods.

The second variety of nomothetic research is what I referred to in my previous post as "theory-driven inquiry": theories provide a very general, skeletal framework of a story within which the researcher might try to fit many particular phenomena.  In fact, it is their lack of detail that makes theories so flexible.  For example, modern medicine has greatly benefited from the rise of the germ theory of disease, which suggests that the failure of biofunctions in organisms is sometimes caused by the invasion of those organisms (the hosts) by other, smaller organisms (the "germs" or parasites), which then exploit the body of the host as a resource, disrupting the host's normal biofunctions in the process.  The germ theory of disease is general enough that it tolerates different sorts of germ - prions, viruses, bacteria, protazoa, fungi, arthropods - each of which exploit their hosts in different ways.  Unlike formalized laws (as in the TMPG example), theories cannot be directly tested.  Additionally, the failure of a researcher to convincingly fit a case to a theory does not indict the theory, which on the contrary may go on to make sense of lots of other phenomena.  For example, while our understanding of certain diseases fit well within the bounds of the germ theory of disease, others instead fit under toxicological, congenital, or genetic theories of disease.  Similarly, evolutionary biologists now distinguishing between four separate kinds of mechanism leading to evolutionary change - mutation, drift, gene flow, and selection - yet the fact that some cases fit better with selection and others with mutation, drift, or gene flow does nothing to undercut the credibility of any one of them.

The distinction between these two different kinds of nomothetic research is important for a number of reasons, one which will matter below, others of which I will revisit in future posts.

What makes simplification so controversial, however, is not whether we can achieve it at either a global or a local level (no and yes, respectively) but instead whether the dynamics of nature/the universe conform to such simple, hierarchically structured principles, globally or locally.  The argument goes that, if the universe itself is so governed, then so should our belief systems be.  Under this assumption, Richard Feynman even went so far as to assert that research that is not directed toward formulating and testing simplifying hypotheses is not scientific.  As an example of this sort of simplifying research, many particle physicists are on a quest in pursuit of a "grand unified theory" (GUT) which would collapse three separate forces governing how different bits of matter interact with each other as variants of a single force (the electromagnetic, strong nuclear, and weak nuclear forces, respectively).  Assuming that such a unification can ever be achieved, a "theory of everything" (TOE) would further collapse gravity and this three-in-one force into a four-in-one force.  This has not yet happened.

Physicists are hardly alone in their advocacy of the simplifying agenda; researchers in many other fields espouse it, as well.  The authors of Principles in Virology, for example, offer a quote from Nobel Prize-winning bacteriologist Al Hershey in defense of their unificationist approach to textbook-writing: "The enduring goal of scientific endeavor ... is to achieve an intelligible view of the universe.  One of the great discoveries of modern science is that this goal cannot be achieved piecemeal, certainly not by the accumulation of facts.  To understand a phenomenon is to understand a category of phenomena or it is nothing."  (The authors discuss the importance of teaching general principles further in this Microbe World podcast, beginning approximately 31 minutes in.  Al Hershey comes up at 44 minutes in.)

The most ambitious unificationists are those who insist that this simplifying operation should aspire to such grand proportions that it dissolves all disciplinary boundaries.  Overwhelmingly, it is the physicists who are posturing to subsume all other disciplines:


Not long ago, I ran across a particularly self-assured advocate of this viewpoint (a recent graduate of physics, I suspect) in an online comments thread, who asserted that
In the scheme of things physics deals with much more basic, lower abstraction level, phenomenon, that does allow for much better controlled experiments than social sciences can achieve. Hypothetically, with knowledge of physics, enough data collection along with sufficient levels of computing power, one could calculate everything, including animal and human behavior. We are very far from being able to what I just suggested but my point is that physics is the science upon which all others are built. Physics deals with the most basic forces or nature, that all other sciences must deal with at some level. ("Strato Man")
(Note the inconsistency in verb tense between his implication that physics is presently "the science upon which all others are built" on the one hand, and his future-tense assertion that such a subordination might some day be possible, though we are very far from it currently.)

As if the physicist's arrogant ambition in this matter weren't bad enough, it is made all the more insulting when it is accompanied by smug condescension regarding other disciplines' inferior ability to accomplish their own research goals.  The human/social sciences in particular bear the brunt of such condescension, for example,
Any physicist threatened by cuts in funding ought to consider a career in the social sciences, where it ought to be possible to solve the problems the social scientists are worked up about in a trice (John Gribbin, from The New Scientist, quoted by Duncan J. Watts in Everything is Obvious*).
Similarly, in Feynman's reflections on the centrality of laws in science (video linked above), he also asserts that "social science is an example of a science which is not a science," whereas he is privileged with the
advantage of having found out how hard it is to get to really know something, how careful you have to be about checking your experiments, how easy it is to make mistakes and fool yourself.  I know what it is to know something, and therefore, I can't ... I see how they get their information, and I can't believe that they know it.  They haven't done the work necessary, they haven't done the checks necessary, they haven't done the care necessary.  I have a great suspicion that they don't know this stuff.
(This attitude also reveals Feynman's failure to recognize that scientific knowledge is spectrum rather than a binary opposition, spanning all the way from untested hunches, through proofs of concept, to well-supported items of knowledge.)  The over-confident physics graduate I mentioned above echoed Feynman in labeling himself "a physicist who understands the concept of testable theories" and in asserting that slapdash research by social scientists (in this case including sociobiologists) is to blame for public distrust of science in general.  It is as if we (social scientists) are children playing dress-up in our parents' (hard scientists') clothes and accessories (smocks and goggles, I suppose).

Nor does the condescension stop there: not only are social scientists regarded as pseudoscientists for our avoidance of covering laws (not always true, by the way), then treated as dilettantes for our inability to conduct rigorous tests and checks, but our chosen subject matter is itself trivialized, for example in Feynman's dismissal of ethnobiology:



Curie is similarly disinterested in people:


And Rutherford has no interest whatsoever in anything that is not physics:


Not surprisingly, the arrogance of physicists is so pronounced that it has become an easy and frequent target for caricature, for example Sheldon Cooper's dismissal of the social sciences as "largely hokum" (The Big Bang Theory, Season 2, Episode 13: "The Friendship Algorithm"), and also this:



Critiquing simplification

Before I get into the meat of my critique, I'd like to start off by addressing a word-choice pet peeve that comes up frequently in this discussion: unificationists love to describe simple laws as 'elegant' ("pleasingly graceful and stylish in appearance or manner"), usually in lieu of 'simple' as if the two were synonymous.  To my mind, this is a really perplexing and idiosyncratic usage, given that 'sophistication' is also frequently associated with elegance.

Now, to the meat of it.  To begin with, it bears repeating that global or other far-reaching simplifications are prohibitively impractical.  Short of possessing a computer as powerful as Douglas Adams' Deep Thought, it is doubtful that we could ever "calculate everything, including animal and human behavior" as derivatives of basic principles of physics.


Nevertheless, those committed to the simplification program employ one or more work-around strategies to make it so, though unfortunately, these are without exception fallacious.  The first of these strategies focuses on the 'explanandum' (i.e., the target of explanation, the phenomenon to be explained; plural 'explananda'), while the second focuses on the 'explanans' (i.e., the phenomenon or phenomena invoked as causes of the explanandum; plural 'explanantia').  The first strategy involves a bait-and-switch presentation of the explanandum:
  • Bait: describe the collection of explananda in detail, emphasizing all of those variables about them that make them so puzzling and worthy of curiosity.
  • Switch: without explicit announcing it, narrow one's explanatory focus to only a small handful, or even a single puzzling variable, about these explananda.  Attempt to explain this reduced set rather than the larger original set, without letting on that you're overlooking the rest.
This sort of reasoning commits the fallacy of redefinition.

The second strategy involves attempting to explain the explananda with reference to only one explanans.  To go back to the gas mileage example from earlier, a researcher employing this second strategy might pose TMPG as a response solely to the time spent idling the vehicle (call it i):

    TMPG = i(C)

where C refers to some constant factor by which the variable i is multiplied, their product being TMPG.  As statisticians know, and as you should know too, it is often empirically possible to assess the likelihood that proposed relationships like this one are true and, assuming that they are, what the value of constants like C are.  What makes this strategy so problematic is that it disregards the residual amount of variation that is not accounted for by the simple formulation.  If the research is being honest, they will supplement the simple formulation with an additional term describing this residual variability, e.g.

    TMPG = i(C) ±e

where ±e denotes this residual variation (or 'margin of error' or 'error term' as it is commonly called).  Furthermore, most researhers will not be satisfied to leave the explanation at that.  Error terms are thorns in the side of researchers, and only the most devout unificationist would recoil from the idea of revising this equation if it means making it more complicated, even if doing so would reduce the size of ±e.  The more appropriate response, most researchers would concede, is to continue to make revisions to the equation that do minimize ±e.  For example, in thinking about the number of miles I have driven on the highway, the number of miles I have driven on city streets, and the amount of time I have spent idling, I might suggest the following:

    TMPG = TM / [TM(p)/HMPG + TM(1-p)/CMPG + i(C)] ±e

where TM is the total number of miles driven over a given time period and all other variables are as above.  Nor is this revised law exhaustive or final; in the future, I might add in additional terms for tire pressure, which is in turn influenced by ambient temperature, and further note that HMPG and CMPG will vary between automobile models or even between individual vehicles.

This second strategy is founded on the notion that simpler = more realistic, in which case we might indeed regard it as an epistemological virtue.  Yet, as trivial as the law of gas mileage may be, it brings us face-to-face with a contrary reality: the pursuit of realism in our understandings comes at the expense of simplicity, not part and parcel with it.  If this is so, then this second strategy instead falls afoul of the fallacy of reductionism (also known as 'the reductive fallacy' or the 'single cause fallacy').  Of course, some degree of simplification is a necessary trapping of research, and researchers often choose to start the research cycles of their careers by testing simple hunches, fully expecting that these are not going to pan out.  But what makes unificationist simplification so problematic is its insistence that what looks to most like a partial explanation is a complete one, or at least one that has been unburdened by the inclusion of unimportant minor details.  David Hackett Fischer describes this fallacy in the context of historiography:
The reductive fallacy reduces complexity to simplicity, or diversity to uniformity, in causal explanations.  It exists in several common forms, none of which can be entirely avoided in any historical interpretation.  As long as historians tell selected truths, their causal models must be reductive in some degree.  But some causal models are more reductive than others.  When a causal model is reductive in such a degree, or in such a way, that the resultant distortion is dysfunctional to the resolution of the causal problem at hand, then the reductive fallacy is committed.  (Historians' Fallacies)
Many of the tentative GUTs and TOEs that have been conceived to date run afoul of this fallacy, which probably accounts for why there are no successful GUTs or TOEs yet, at least in part.  Physicists would do well to heed the cautionary tale that an uncompromising push toward simplicity will result in simplistic rather than realistic understandings, commonly attributed to Einstein (one of their own).

It is also worth noting that, even if a given explanandum can be reduced to a constant law as simple as something like "TMPG = i(C)", this explanation still leaves unexplained the variability in i driving the variability in TMPG.  Unificationists often make the mistake of thinking that the algebraic specificity of the law and the constancy of the constants it invokes together constitute a complete explanation, when in fact the variability of its variables is also an integral part of the explanation.  "Complete explanations" are mythical and necessarily elusive because any complete explanation is one in which variability in the causal variables is also explained.

Opposite the simplification agenda is the 'emergence' perspective, which argues that the properties exhibited by macroscopic entities or processes cannot be wholly reduced as the mere sum of their microscopic parts.  (Here, I use the terms 'macro-' and 'microscopic' not to refer to things that can be seen with the naked eye versus with the aid of a microscope, but instead to entities or phenomena existing at different scales: elementary particles to atoms, atoms to molecules, molecules to compound units of matter, ..., individual organisms to societies, and up and up and up.)  At best, the variables that account for higher-order entities or phenomena may include attributes about the microscopic things that purportedly constitute them, but this is rarely enough to wrap up a compelling account.

It should also be noted that there is one faction in the emergence community, sometimes going under the label 'weak emergence,' that is in fact nothing more than a clandestine unificationist attempt to reign emergence back in, working from the inside to bend it back to the same old agenda.  The change of label, however, does absolutely nothing to undo the multiple criticisms that unificationism suffers.

So, the simplification program is conceptually bankrupt, and not surprisingly, the arrogance of those who have long advocated it (I am looking at you, outspoken physicists) has proven premature:
I think I can say that the problems sociologists, economists, and other social scientists are "worked up about" are not going to be solved in a trice, by me or even by a legion of physicists.  I say this because since the late 1990s many hundreds, if not thousands of physicists, computer scientists, mathematicians, and other "hard" scientists have taken an increasing interest in questions that have traditionally been the province of the social sciences - questions about the structure of social networks, the dynamics of group formation, the spread of information and influence, or the evolution of cities and markets.  Whole fields have arisen over the past decade with ambitious names like 'network science' and 'econophysics.'  Datasets of immense proportions have been analyzed, countless new theoretical models have been proposed, and thousands of papers have been published, many of them in the world's leading science journals ....  Entire new funding programs have come into existence to support these new research directions.  Conferences on topics such as "computational social science" increasingly provide forums for scientists to interact across old disciplinary boundaries.  And yes, many new jobs have appeared that offer young physicists the chance to explore problems that once would have been deemed beneath them.

The sum total of this activity has far exceeded the level of effort that Gribbin's offhand remark implied was required.  So what have we learned about those problems that social scientists were so worked up about back in 1998?  What do we really know about the nature of deviant behavior or the origins of social practices or the forces that shift cultural norms ... that we didn't know then?  What new solutions has this new science provided to real-world problems, like helping relief agencies respond more effectively to humanitarian disasters in places like Haiti or New Orleans, or helping law enforcement agencies stop terrorist attacks, or helping financial regulatory agencies police Wall Street and reduce systemic risk?  And for all the thousands of papers that have been published by physicists in the past decade, how much closer are we to answering the really big questions of social science, like the economic development of nations, the globalization of the economy, or the relationship between immigration, inequality, and intolerance?  Pick up  the newspaper and judge for yourself, but I would say not much. (Duncan J. Watts, Everything is Obvious*)
It is also refreshing, though no less misguided, to see the unificationist agenda aimed against physics

("The Big Bang Theory", Season 4, Episode 3)

In the end, the main virtue of simplifying research is the ease with which its products can be transmitted both among fellow researchers and between the scientific community and the general public.  Simplification undeniably leads to the production of easily packaged, short communications, hence its practical appeal.  Nevertheless, in light of the preceding critique, we do have to ask, "is it better to effectively communicate a message that is inaccurate in its denial of its incompleteness, or to suffer the burden of trying to communicate complicated but accurate - though still incomplete - messages?"  In approaching simplification as an empowering enterprise, we have to wonder whether it is the kind of power that we would actually want to wield.


Exploiting disunities

Disunity, as it turns out, is empowering in its own right.  As I have previously mentioned, QC approaches its scrutinizing labor in an open-ended manner, periodically revisiting accepted and well-supported beliefs from new angles, usually by bringing new lines of evidence to bear on the review.  QC's philosophy in doing so is this: if new lines of evidence eventually topple a long-standing hypothesis in a way that previous evidence could not, then we are not only successful in ridding ourselves of a bad idea but also in identifying a shortfall of that previous evidence.  On the flip side of the coin, when multiple lines of evidence appear consistent with an accepted understanding, this mounting evidence reinforces the strength of the belief - a "miracle argument" or "argument by convergence" as it sometimes called.  Here is the problem: well-integrated networks of belief can often though unexpectedly work against our ability to accomplish such multi-line testing.

For arguments by convergence to be valid, it is absolutely necessary that each new line of evidence is independent of those employed previously.  If they are not independent - if the new line of evidence is correlated with older lines of evidence - then it is unlikely that the new evidence will yield results that run counter to the older evidence.  The sense of renewed endangerment and of mounting evidential support that result from the use of such evidence are both illusory.  The worst violations of the criterion of independence are those in which separate mathematical summaries of a single body of evidence are offered as if novel, when in fact they are simple mathematical transformations of each other.  For example, if we were to present the frequency of successes observed in some experiment as a proportion of successes out of all trials, then re-present the same evidence as a ratio of successes to failures, we are saying nothing new at all; a proportion and a ratio on a single run of trials will tell exactly the same tale, using slightly different terms.  Fortunately, this latter kind of behavior is rare in the sciences, a rookie mistake rather than a deliberately deceitful act, largely because the oversight of more experienced scientists and peers keeps us honest.  However, it is far easier to unwittingly present interdependent lines of evidence as if independent.

This is where well-integrated networks of belief may sometimes become a liability.  In many cases, the logical connections that tie well-integrated networks together operate to increase the degree of interdependence between the lines of evidence that we might otherwise have employed.  Conversely, where two or more locally integrated clusters of belief are minimally integrated with one another, opportunities for independent lines of evidence abound provided that they both bear on the phenomenon we are interested in explaining.  Disunity is a boon for QC.

For example, in my own field of research - archaeological demography - one of the things we frequently try to do is estimate the growth rate dynamics of past human populations.  In this undertaking, we have an opportunity to double-check our estimates by studying both the age-at-death distributions of human skeletal populations and the changing frequency of archaeological sites that past populations have left behind them over time.  The first line of evidence draws upon the biology of human fertility and mortality as well as of skeletal aging, whereas the second draws on anthropological understandings of human land use and waste disposal practices, as well as the geology of landscape formation processes.

These two lines of evidence are admittedly not entirely unconnected, though the connections between them are loose and indirect.  For example, both datasets in question need to be anchored to a common timeline, and in doing so, we often (usually) use identical dating methods for both.  Most of archaeology's most well-known and widely used dating methods draw on knowledge of nuclear physics, environmental chemistry, and biochemistry.  (We do get along with physicists, I promise.)  Even so, the connection that these dating efforts introduce into our two-lines approach is incidental, not the sort that undermines the effective independence of the two lines of evidence, because our ability to estimate the ages at death of skeletal remains and to infer population growth rate estimates from these is not dependent on the timestamps we put on them.  (Amusingly, however, our ability to count the changing frequency of archaeological sites over time is dependent on these timestamps, so if our dating methods are flawed, then our two lines of evidence will actually tell conflicting stories, one of them false.)

(Alison Wylie, a philosopher of archaeology, has much more to say about disunity and its usefulness in her various works. Anyone interested in the matter should pick up a copy of her collection of essays, Thinking from Things.)


The essential tension: the life of science after global unification

Scientists are unarguably engaged in an enterprise intent on identifying linkages between different aspects of nature/the universe (but also of ridding ourselves of believing in linkages that do not exist).  These connections make for interesting opportunities to communicate with the public, particularly when they are of a simplifying sort.  Nevertheless, the unificationist hypothesis of global integration and especially of global simplification seems less likely to succeed now than ever.  First, there is a steep logistical challenge: science is growing and pushing out into new frontiers at a fast and accelerating pace, vastly outpacing the ability of researchers to accomplish the sorts of sweeping integration and simplification that unificationists envision.  Second, while the unificationist program heavily implies a culminating moment of research, in which all the pieces will finally fall into place under the coverage of a grand and "elegantly simple" law of everything, in practice the give-and-take between science's R&D and QC divisions on the contrary entails an endless process of turnover in our beliefs.  (On this count, one might even argue against Feynman that the pursuit of grand unifying laws, not the rejection of law-oriented research, is pseudoscientific.)  Third, the unificationist agenda supposes that the fabric of nature/the universe is woven out of a single thread, following a simple stitch throughout, whereas it may on the contrary be a tapestry of high thread count, woven together with a sophisticated stitch and/or many different kinds of stitch.  Realistic belief systems may therefore favor sophistication rather than simplicity, and what unificationists say we do is not matched with what we have actually done, can do, or ought to do.

The burden of these multiple criticisms suggests that the scientific community is overdue in reevaluating our relationship with the organization of belief systems.  We have seen it as a straightforward and unproblematic operation, and it is not.  It is a relationship that is tense in its very core: on the one hand, scientific R&D does generate new connections between ideas where these had not previously existed, usually at a local scale, while on the other hand the rapid expansion of science, in conjunction with the subtractive effects of QC and of forgetfulness, imposes a sort of entropy on belief system organization.  Nor, it seems, is this tension a perfect counterbalance.  If anything, the state of disarray of scientific knowledge is increasing daily.

What makes this antagonism so vexing, I think, is that not everyone is well-equipped to handle antagonistic realities.  All too often, when we are confronted with dilemmas, we err on the side of one out of the competing goals at the expense of the other, rather than seeking any sort of optimal balance between them.  Yet, it is possible - at least, I hope that it is possible - to learn to be at peace with essential tensions like this one.  On the one hand, we may fret at the vanished hope of any grand unification of belief (particularly those of us who are fond of asserting that everything is connected with everything else), but on the other hand, we may celebrate the gain in realism that seemingly comes from accepting the irreducible complexities of reality.  We may mourn the loss of efficient communication that comes with the presentation of simple understandings, but on the other hand we may embrace disunity as it empowers the ongoing work of QC.  Living with the essential tension means accepting these mixed feelings, and as undesirable as this may be, it is far better than continuing to live under the self-soothing illusion that the unificationist program once afforded.

Sunday, September 22, 2013

Where do questions come from, and where do they go? (Scientific R&D, Part 1.2)

Where do questions come from?

(Commonly attributed to Asimov, though I haven't yet found a credible citation)

As I argued in my previous post, the questions that animate science must make the erotetic cut, but this is only a necessary condition.  We still need to answer the question, "where do such questions come from?"  As with scientific R&D in general, there is nothing uniquely scientific about asking questions.  Humans being creatures endowed with an insatiable thirst for understanding, we are (or seem to be) predisposed to fixate on our points of confusion and gaps in knowledge as we encounter them.  Ultimately, this is where the questions that animate science come from, though not all such questions are well-suited to observational scrutiny.  As I have previously mentioned, if an idea cannot be held accountable to observation, then we are not doing science, and this applies specifically to the ideas we offer in answer to our questions, and by extension to the questions, themselves.  In this post, I will narrow my focus (for the most part) to those questions and answers that are amenable to observational scrutiny.

The questions that animate science tend to come from either of two directions, a fact that probably does not seem noteworthy until I tell you that it is the source of a surprising amount of conflict within the scientific community.  I’ll get to the conflict in a bit, but it would be helpful if we first understand what these two approaches are.  I will call them observation-driven inquiry and theory-driven inquiry.

Observation-driven inquiry begins with curious patterns detected in observational data, characteristics about some dimension of reality that we have described and that seem unique or that make us wonder why they aren't some other way instead.  That the world is working in some way is made clear by such observation, but exactly how it worked to that end remains to be determined.

For example, over the course of the last two and a half or three centuries, the global population, as well as the regional and local sub-populations that it comprises, have been undergoing a transition from a regime characterized by high mortality and high fertility rates to one characterized by low mortality and low fertility rates.  Furthermore, a lag in time between the mortality and fertility transitions has driven the explosive population growth that has led to our current world population of 7,000,000,000+, though fortunately this explosive growth seems to be on the decline and there is at least a hope that growth will level off by the end of this century.  Taken together, these trends in fertility, mortality, and population growth are known as the Demographic Transition (or DT for short).  The changes in the global population size and age structure that these changes have entailed present us with pressing concerns for the sustainability of our current social, political, economic, and medical systems, so we are compelled to understand the factors that have driven and continue to drive changes in growth rates, in other words factors that drive changes in fertility and mortality rates.  The DT thus poses a pressing target for understanding; it raises questions like, why have these mortality and fertility transitions unfolded in the way that they have? and what are the circumstances under which such changes are likely to occur, in which direction, and to what social, political, economic, and/or medical effect?  Demographers have been trying to answer these questions since the early Twentieth Century, with early guesses emphasizing Neoclassical economic principles, later ideas stressing the diffusion of cultural norms from centers of development to developing regions, and a plurality of current answers hybridizing various aspects of previous perspectives.  At the same time, we have collected better demographic data from many areas of the world that were not originally available to the early DT researchers, and of course new data continue to come in as the ongoing population dynamics of the world have continued to unfold, meaning that the explanatory target of DT research has shifted around a bit since its inception.

A second example comes from geology, regarding the extent and causes of what may have been our planet’s most dramatic ice age, the late Proterozoic Ice Age.  Beginning in the late Nineteenth Century, a series of distinctive geological deposits known as late Neoproterozoic Glacial Deposits (LNGDs) have been discovered on every continent, suggesting that virtually all of the Earth’s terrestrial surfaces, including in the temperate and tropical latitudes, were once covered by extensive ice sheets sometime between approximately 850 million and 635 million years ago.  Since that time, our planet has not experienced such extensive glaciation, making it all the more noteworthy.  The question thus rises, what unique set of factors came together to lead to such an unparalleled ice age, and could it happen again?  As with DT research, a number of competing explanations have been offered for this ice age ever since it was discovered in the mid 20th century.  One family of explanations emphasizes the interaction between changes in the Earth’s orbit around the Sun and the tilt of its axis, which could have decreased and changed the daily and annual intensity of solar radiation reaching the Earth's surface, potentially allowing for the formation and expansion of continental ice sheets.  Alternatively, the “Snowball Earth Hypothesis” suggests that a phenomenon called “albedo,” referring to the reflection of the sun by white surfaces, led to a process of positive feedback in which the initial formation of continental ice sheets in the tropics acted to deflect sunlight, leading to further global cooling and allowing for the formation of even more extensive ice sheets.  In this scenario, the runaway albedo effect propagated to such a degree that even the oceans were covered by a sheet of ice (hence the label “Snowball Earth”), or at least that they were globally slushy (“Slushball Earth”).  The jury is still out on the most credible answer to this question, though at present the Snowball Earth hypothesis is faring much better than the orbital hypotheses in the debate, as I understand it.

Unlike observation-driven inquiry, theory-driven inquiry begins with a prior belief already in mind about the way that some aspect of the world works.  In this case, questions arise particularly when such prior understandings run into observations that seem to contradict then.  The critical word here is seem, because many seeming contradictions turn out to be mere paradoxes – seemingly contradictory statements that may nonetheless be true – upon further examination.  In the case of a paradox (but not a true contradiction), the root of confusion typically lies in a flaw or a gap in our own understanding rather than a true mismatch between the two statements in question.  The goal of theory-driven research is then to come up with a solution to the puzzle that accommodates for the seeming incongruity between prior understanding and new observation, in other words by showing that the seemingly contradictory observation does not in fact violate the prior understanding.  The effect of such accommodation is that the newly observed reality is neatly subsumed under the prior belief.

For example, Neo-Darwinian theory (which synthesizes Darwin’s theory of natural selection with modern genetics) asserts that biological traits (anatomical, physiological, or behavioral) are not expected to emerge or persist in a species that would act to undercut the fitness of individual organisms within that species, i.e. that would undermine their ability to “leave more surviving offspring or more copies of their genes” (John Alcock, The Triumph of Sociobiology, p. 32).  Alcock continues on to say that
“This is a theoretical perspective, and like all useful theories, it shapes the expectations of observers in productive ways, so that they can first identify the surprising features of nature and then develop testable hypotheses to account for these surprises.  Someone who understands Darwinian theory is prepared to be puzzled by certain things, not others.
For example, the emergence and persistence of altruistic behaviors in various highly social species, including social insects, group-living mammals, and others besides, have provided one of the major puzzles for sociobiological research (the area of Darwinian evolutionary biology devoted to the study of the evolution of social behavior).  If individuals are designed by natural selection to promote the reproduction of their genes into the future, why would an individual ever invest one's own time, energy, or resources toward the well-being of another individual at the expense of one's own?

Scientists who are accustomed to approaching research from the angle of observation-driven inquiry often regard the theory-driven approach with considerable suspicion because, at first impression, it seems to embody a rather non-scientific approach to establishing belief.  Making accommodations for seeming contradictions is the business of apologists, defenders of the faith, not scientists, because such behavior short-circuits the definitively scientific act of endangering ideas.  Or does it?

As it turns out, this approach is nowhere near as unscientific as it may initially appear.  First, as advocates of the theory-driven approach would counter, successfully demonstrating that new observations continue to fall within the boundaries set on reality by old understandings speaks volumes for the continued value of those understandings in helping us to make better sense of our world.  Second and more importantly, researchers dedicated to this approach will often be the first to admit that their success in accommodating for the paradoxical is dependent on a goodness of fit between their accommodation and further rounds of observational scrutiny.  In other words, the accommodation that resolves the paradox and subsumes observation under prior understanding is itself treated as a testable hypothesis, and not all such accommodations will stand up to scrutiny.

Nor are all approaches to theory-driven inquiry so monopolistic in trying to subsume new observations under the coverage of a single understanding.  Evolutionary biologists, for example, readily concede that there are other evolutionary processes beside natural selection that can explain changes in the frequency of genes or biological traits within a species, including changes that either reduce reproductive fitness or are selectively neutral.  Thus, sometimes the solution to a Darwinian puzzle requires no special intellectual contortion to resolve a paradox but instead draws upon other, non-selective theories about evolutionary processes.  On this point, evolutionary biologists distinguish between four “forces” of evolution – mutation, selection, gene flow, and genetic drift – and further distinguish between different kinds of selection (natural, sexual, group, artificial), all of which are expected to operate under different sets of circumstances.

A similar situation holds in medicine and public health regarding the cause of diseases.  One of the most revolutionary intellectual developments in medicine over the last three or more centuries was the introduction of what we now call the germ theory of disease.  This idea suggests that many diseases are caused by infection by small critters (i.e., “germs”) like prions, viruses, bacteria, protozoa, fungi, and arthropods (worms, arachnids, insects), which parasitize the body of their hosts for their own purposes, and to the detriment of the host’s regular biofunctions.  With the emergence of microbiology, parasitology, and immunology, our ability to demonstrate the presence and adverse activities of such pathogens has revolutionized our ability to understand the source of many diseases, and to treat them…

But not all of them.  While our understanding of many diseases has improved considerably because of the germ theory of disease, this in no way changes that fact that a good number more of diseases are instead caused by genetic defects (e.g., sickle cell anemia, Tay-Sachs disease), developmental mistakes during fetal development, or exposure to various detrimental substances throughout life (smoke, smog, sugar, salt, saturated fats, carcinogens, poisons, allergens, etc.).  For this reason, pathologists and epidemiologists have hardly given up on alternative explanations of disease, just as most evolutionary biologists have not given up on genetic drift, gene flow, and mutation as drivers of evolution alongside the incredibly powerful concept of selection.

So, we might further subdivide theory-driven inquiry into two subcategories: (1) accommodation-driven inquiry, which attempts to subsume puzzling phenomena under prior understandings by showing them to be mere paradoxes, and (2) a “which theory is better?” approach, which attempts to identify which out of a set of prior understandings fits best with a given observation.  In fact, this second approach comes very close to the observation-driven mode of inquiry I described above.  For example, in their efforts to understand the climatic mechanisms that drove the late Neoproterozoic Ice Age, geologists did not simply make up new ideas to fill this gap.  Instead, they went to one of two prior understandings of glaciation, each having much broader applicability than just to the ice age in question.  Explanations that emphasize orbital parameters go back to the work of the early Twentieth Century Serbian scientist Milutin Milanković, whose ideas predated the discovery of the Neoproterozoic Ice Age and are still held in high esteem regarding the cycling of episodes of glaciation and deglaciation during the Pleistocene epoch (from approximately 1.8 million to approximate 12 thousand years ago).  Conversely, the concept of a runaway albedo effect, which serves as the backbone of the Snowball Earth hypothesis offered by Joe Kirschvink in 1992, was originally envisioned by the Russian climatologist Mikhail Budyko in the 1960s, who saw such runaway albedo only as an extreme and unlikely special case of a more general model of albedo that he developed.  So perhaps there isn’t such a huge difference between observation-driven and theory-driven researchers after all, at least not in every case.

This brings us to the contentious topic of ‘theory.’  As many readers are probably aware, evolutionary biologists, climate scientists, and their respective sympathizers frequently butt heads with unbelievers over the meaning of the word ‘theory’ (among other things), particularly when it comes to dismissive expressions like “evolution is just a theory.”  Unfortunately for everyone involved in the debate, in common usage, ‘theory’ has the meaning ‘untested idea,’ making it synonymous with ‘conjecture,’ ‘speculation,’ ‘guess,’ ‘hunch,’ or something you dreamt up after being drunk all night (another quote commonly attributed to Asimov) ... not that guesses have no place in science.  Thus, to assert that evolution is just a theory is to assert that it is a baseless speculation, and only one among many alternative ideas about the nature of life on Earth, at that.  The rehearsed response of scientists is to counter that ‘theory’ stands out from hypotheses not because theories are untested hypotheses but on the contrary because they are exceptionally well-tested and observationally well-supported hypotheses.  Thus, in scientific jargon, calling an idea a 'theory' is high praise, synonymous with 'knowledge,' not a dismissal.

But the story is a little more complicated than scientists usually let on, because in fact we use ‘theory’ in two different ways (a poorly recognized fact that unfortunately creates the potential for the related fallacies of equivocation and amphiboly; see also here and here).  The first sense of ‘theory’ is the one just defined, referring to a well-supported hypothesis, standing as the result (i.e., the end) of a research cycle.  The second sense of ‘theory’ is the one discussed earlier, referring to a prior understanding, well-supported or otherwise, that functions not to finalize research but to catalyze it.  In this case the theory is not directly tested or testable, only the accommodations that are intended to link it to observation.  This meaning of ‘theory’ comes much closer to the meaning intended when we say “the theory of evolution through natural selection,” “the germ theory of disease,” or “theoretical physics.”

Effectively, what these research-orienting theories do is provide a generic framework for stories that we might tell in our efforts to account for whatever phenomena we hope to explain.  These theories are deliberately vague in detail, asserting only a broad outline about explanations of mysterious phenomena.  For example, while the germ theory of disease tells a generic story about the invasion of host organisms by smaller organisms that then prey upon the host, causing all kinds of health problems for the host in the process (i.e., the dis-ease), the theory remains deliberately silent regarding the particular details of such infections.  This vagueness is the secret to the theory’s success, because as it turns out, there are many different kinds of infectious pathogen (prions, viruses, bacteria, protozoa, fungi, and arthropods), each relying on different routes of introduction into the host body and exploiting the host body in different ways, leading to different health outcomes: acute vs. chronic disease; nausea vs. pain vs. fever; mild illness vs. fatality; etc.

At face value, the untestability of this kind of theory may seem like a huge liability for productive debate between different communities of belief, at least when one or more of the theory's components and the questions it generates are disputed.  As I suggested in my previous post, asking any question that assumes any reality that the audience is unwilling to accept creates the problem of the loaded question.  Despite this seeming liability, however, I doubt that things are so bleak, for two related reasons:

First, many of the theories that underlie the questions that scientists ask turn out to have huge utility.  For example, in the case of the germ theory of disease, our ability to identify the infectious pathogens that cause many of the epidemic diseases we have suffered for millennia has empowered us to significantly reduce their future potential, owing to various public health measures (improved hygiene, sanitation programs, and vaccination) that interfere with the life cycle and transmission of these infectious pathogens.  (It's an important component of the mortality transition that led to the DT described above, incidentally.)  When questions and answers founded on 'absurd' concepts prove to be so eminently successful, it becomes increasingly difficult to consider them absurd any longer.

Second, while such theories are too vague to be scrutinized themselves, it is still quite damning when researchers who are dedicated to them prove unable to fit them to observations.  A track record of failed accommodation after failed accommodation constitutes its own sort of endangerment, even if it is a bit slower in the unfolding and not quite the same as falsification.  So, we have a sticky situation in which our assumptions both color our research and are challenged by it, a fact that I don't think Steven Novella would much care for:



(Of course, stubborn advocates of a failing theory might also continue to maintain that the apparent shortcoming of their theory is due to a lack of talent on their own part, not on any deficiency of the theory; these are the ones who are most deserving of Novella's censure, I think.)


Asking important questions, and asking impossible questions

Scientists are curious by trade, but this is hardly limited to us; it is a human thing that scientists just happen to indulge more than most.  Even so, we don't ask every question or pursue every answer conceivable, and especially not those that are not currently conceivable but once were or might someday be.  In part this is because we cannot ask questions that are currently absurd, questions that depend on beliefs we don't currently hold.  And of course, some of the questions we ask now will flicker out as the beliefs that underlie them fade in importance or acceptance.

Even out of those questions that we do ask, we do not approach them all with equal rigor.  We do triage, because we know that we cannot possibly address them all with the meticulous scrutiny necessary to either dismantle them or establish them as standing knowledge.

The question of what makes a question important to ask and answer is a deeply philosophical one, and I would strongly urge everyone to read this comic strip on the matter, if you have not already.  (And if you have, reread it.)  [9/25/13: and also this blog.]  We might measure the importance of a question based on the practicality of a good answer to it, and I have certainly promoted this aspect in my discussion of the germ theory of disease and the questions it spawns.  Alternatively, we might say that questions are more important than preexisting answers – the systems of belief we have accumulated over the years – because questions draw attention to the shortcomings of our beliefs and of adamantly holding to static ones, and because the same question can be asked again and again whenever previous answers to it have fallen to observation (sometimes by deliberate scrutiny, sometimes merely by accident).




Needless to say, the importance of beliefs in general, the importance of questions, and the importance of answers to such questions, are highly subjective judgments.  The smugness of scientists about their own discipline often goes part and parcel with the sense that scientists working in other disciplines are wasting their time on topics of trivial importance, though fortunately this smugness is often counterbalanced in the opposite direction.  I prefer to assume that other sciences are as fascinating as the ones I spend most of my time in (anthropology, demography), and I have a short list of other disciplines that I would pursue with enthusiasm if only I had a few more lifetimes to live.  Stepping out of the scientist vs. scientist dynamic, the opinions of the general public toward what questions are important to ask are even more variable, as are their opinions of everything else, because let's face it: the "general public" is a pretty huge and heterogeneous entity.

Subjective though such judgments may be, scientists are nevertheless obliged to justify our research to others, especially to funding agencies and to the professional journals whose choice it is to publish our work.  For example, grant proposals submitted to the United States' National Science Foundation (NSF) require that a section of the proposal be dedicated to the discussion of intellectual merits and broader impacts of the research proposed.  Not surprisingly, the promise of economic, social, political, and/or health benefits for the American public favors the funding of such research, whereas replication research is too rarely funded.  There is also a cottage industry of op-ed pieces offering lists of what different authors believe to be the most pressing questions in need of answer (for example this two-part blog series from NPR, here and here).

By the same token, there is frequent discussion (and debate) about which questions can be answered by science and which remain out of its grasp, such as is reflected in Robert Krulwich's NPR blog post here.  This question frustrates me because it glosses over two very different constraints on science, one of which is considerably more fundamental than the other.  The first and more fundamental constraint is that no question can be considered scientific if provisional answers cannot be subjected to observational scrutiny.  There is nothing really profound here, however, only the recognition that hunches that are not endangered are just hunches.  Moreover, whether there are good answers to questions (the right ones, even) that cannot be tested is the great unknown; how would you ever know whether an answer to a question is untestable simply because you have not yet found a way to test it?  elusive as such knowledge may be, there are some questions that seem on the face of it to be too broad, too generally stated, or too vague to be approached scientifically, for example most of the questions in the aforementioned NPR blog series about the "20 most important questions."  Instead, the sorts of questions that are actually scientifically approachable tend to be very topical (e.g., this "to-do list" for Parkinson's researchers), which unfortunately tends to sacrifice importance, or at least breadth of coverage, for specificity and operability.

The second constraint is the possibility that some phenomena of the world elude principled behavior.  In this case, no conjectural answer that purports to understand it will ever be correct, exactly because no understanding is possible.  but once again, even if such an impossibility were true, we would never know it, because our ability to put bad ideas on the table before we find good ones is no proof of the impossibility of good ones overall.  Many human scientists (anthropologists, psychologists, sociologists, economists, and political scientists) continue to seek better understandings of human nature, this in spite of the alternative possibility that no such understanding is possible because we don't make any sense, because we are endowed with a crazy little thing called free will (one could call it "human anti-nature").  Responsible scientists should be comfortable conceding that we can never really know if a given aspect of the world eludes understanding.  At the same time, few of us will stop seeking possible and testable answers to our questions, and when we stumble upon good ones, in other words ones that stand up to honest observational scrutiny, we will feel vindicated, full well knowing that these may be displaced by even better answers later on down the line.  As a matter of course, we approach no phenomenon as if it cannot be better understood.

The Erotetic Side of Science (Scientific R&D, Part 1.1)

In my previous post, I introduced a metaphor of science that divides the scientific enterprise into quality control and research-and-development (R&D) divisions.  Usually, scientists hold positions in both divisions (still speaking metaphorically), though some spend more time in one than in another (e.g., in physics, where experimental and theoretical physicists are often up to very different things).  I also argued that the quality-control division is what sets science apart from other efforts to produce knowledge, and I dedicated that post to a discussion of this central aspect of science.  Ironically, I find the need to split my discussion of science’s R&D up into four parts, this one and the next about the questions that animate R&D work, the third and fourth about the substance and organization of scientific theory and knowledge. 

What’s so questionable about science?

Recall, science is a process or a practice; it unfolds in real time.  It is something that we do.  We don’t always think of it that way, because when we read, hear, or watch science articles, we usually receive a story that collapses the process down into a “flat,” timeless presentation.  But this timelessness is artificial.  When we actually do science, it is an undertaking with a beginning, middle, and end.  More accurately, each individual science project has its own beginning and an end, though different projects may unfold out of phase with each other.

While science is a process, it is not the sort of process that just happens to scientists, as if we are “hapless victims” of science, who have stumbled into science already in progress.  In chemistry, there is a concept that says that many chemical reactions don’t just happen; two compounds sitting next to each other, even touching each other, may remain inert for a long time, even though they have the potential to react violently with each other.  What they need is a trigger, or a “catalyst” to use the term favored by chemists.  Well, metaphorically speaking, the scientific process needs a catalyst.  It needs something to trigger it, to ignite it, to innervate it.  It needs proactive researchers to actually take the initiative.  Most importantly, it needs researchers to ask a question, to identify a specific problem to pursue.  The question is the catalyst of the research cycle.

Specifically, the research cycle goes something like this: first, identify a problem – a deficiency in our understanding of the world – and ask a question about it:

  • Why is my throat soar?
  • Why did we get freezing rain this winter, serious snowstorms last winter, and hardly any precipitation at all the winter before that?
  • Why does this population of orangutans engage in frequent social interaction whereas that group does not?
  • Which is healthier, this organic apple, or this other one that isn’t advertised as such?

Then, brainstorm a possible explanation, or if you are really ambitious, a set of alternative answers to the question.  The conjectural answer(s) then become(s) the hypothesis/es that will anchor your research, the idea(s) that you plan to hold accountable to observational scrutiny.  Next, place the hypothesis in an “if, then” sentence having the form “if [hypothesis] is true, then [observational expectation] will also be true.”  In other words, if your hypothesis is actually true, what would you expect to observe?  Keep in mind, as I stated in my last post, the expectations you identify must be stated so that you are actually endangering your idea.  In other words, you have to pick expectations that could conceivably run counter to observation.  By extension, if you are working with multiple competing hypotheses, the task of holding them all accountable requires the identification of observational expectations that are consistent with one hypothesis but not the other and vice versa.  The final step is to actually observe the dangerous reality you have identified and to see if your observations are consistent with your expectations.  If so, yay for your hypothesis; we can add it to our collection of standing knowledge.  If not, boo for it; we need another idea, because we haven’t yet achieved the understanding that we were hoping for.

What’s so erotetic about science?



No, that's not a misspelling; you just read it wrong.  (But then again, it's hard to concentrate with Einstein giving you those smoldering bedroom eyes.)  ‘Erotetics’ is one of those obscure words that only philosophers know ... well, until I define it for you, and then the cat's out of the bag.  I would hazard a guess that scientists are as unfamiliar with it as anyone else.  So, what is it?  Erotetics is the area of logic that is concerned with questions and answers.  In some future post, I will have a lot more to say about logic in general, but for the moment I will simply say that not all logicians pay much attention to questions and answers; few are dedicated to questions and answers specifically, and the work of many logicians doesn’t concern them at all.  Logicians who study erotetics are thus a highly specialized community.  And yet, insofar as questions are the catalysts of science, we ought to be concerned with what eroteticians (? or eroteticists?) might have to say.

In a slightly bigger nutshell, erotetics is concerned with good questions and good answers, or with criteria to distinguish between good and bad ones.  What’s a good question?  To begin with, it is a question that assumes a reality that at least the inquirer believes.  In scientific context, it should also be a reality that the interrogator’s peers believe.  We have all experienced the confusion that follows when someone asks us questions based on wrong assumptions.  “Why were you awake and making so much noise all night?”  “Well, I wasn’t awake all night,” or “I was awake all night, and I heard the noise too, but it wasn’t me making it.”  The point is, the failure to agree on the reality that the question presupposes presents an insurmountable obstacle to sufficiently answering it.  In the preceding case and in cases like it, a “why…?” question begs a “because …” answer, and we quite simply cannot provide “because…” answers to “why…?” questions that assume untruths.  This is the fallacy of the loaded or complex question (see also here).  It is also a crucial topic in scientific questioning, and in the public relations of science, that I will revisit later in this post and more in my next post.

There are also some bad questions that we can’t even understand, usually involving a violation of the accepted definition of words that results in absurdity, for example, “how many meters long is the song of the morning birds?”  Of course it is possible to measure a birdsong in terms of frequency and volume, or changes therein, and of course a meter is a unit of measurement.  But sound wave frequency and volume measure sound (frequency being a measure in time), whereas the meter is a measurement of space.  The idea of measuring sound waves with a spatial unit of measurement presents a semantic mismatch that renders the question absurd.

So, what is a good enough answer?  Once again, a good answer will be one that assumes realities that we are willing to accept; there is no sense in conjuring principles, phenomena, or entities that don’t have at least some believers.  But there is also an opportunity for conflict here, because not everyone believes the same thing, so the ability to answer such a question is relative to the community of belief; some answers will not sit well with some individuals because the questions they answer are themselves unhappily loaded.  Again, this is a crucial topic that I will revisit.

A good answer should also imply no contradictions, and as with good questions, it should not put ideas together that do not go together, lest the answer assert the absurd.  If I consult a doctor about why my throat is soar, I would have some serious doubts about them if they told me “because 5 times 5 is heavier than the odor of water” ... amusing, to be sure, but I probably wouldn't go back to them for a follow-up.

Then too, there are some answers that are bad not because they assume questionable realities or imply absurd or self-contradicting realities, but because they answer the wrong interpretation of an ambiguous question: “Did it rain on Monday because of a weather system moving in from the East?”  “well, do you mean, did it rain on Monday, or did it rain because of a weather system moving in from the East?”  In cases such as this, the meaning of the question and thus the range of acceptable answers is often conveyed either by tone of voice or by the well-understood details of the context in which it is asked.  For example, if we have already been discussing the weather system from the East, the question is probably a timing question: "did we see its effects on Monday, Tuesday, Wednesday?"  On the other hand, if we are talking about the various weather systems that were lurking about the area on Monday, the question is probably about which one: "was it the one from the East, or from the South?"  Such questions usually only become problematic when they are asked out of context, though occasionally even contextual cues don’t clarify, often leading to uneasy conversations spent moving in the wrong direction.

Finally, there are some answers that make sense and refer to realities that everyone accepts as true, but are still failed answers because they don’t answer the question asked.  “Why is the unemployment rate up this week?”  “Well, let me start to answer your question with an anecdote about when my daughter was a little girl.  You see, she used to play piano and take lessons every Wednesday afternoon after school.  Eventually, though, she got bored with it.  She never really cared for it, I think, but we made her.  So, after a while she quit practicing.  Then there were the horse lessons … blah blah blah.”  This miscarriage of question-and-answer logic is an all too common strategy in political debate, but it can also be an honest mistake made by undisciplined minds, a failure to keep on point in answering the question, particularly in cases where the process of asking and of brainstorming an answer unfolds slowly.

Given the critical role that questions play in the scientific enterprise, we have a lot to gain by getting erotetic about science, in other words by asking what achievements and pitfalls we can make in science that stem specifically from the quality of the questions we ask or the goodness of fit between our answers and our questions.  Granted, all of the erotetic miscarriages described above present some risk for scientific questioning and answering just as much as they do in any other enterprise, and the best guard against them is critical thought.  But some are perhaps more relevant for our understanding of scientific success and failure than others.  Probably the most visible source of difficulty is the scientist’s ability to ask questions or to provide answers that refer to realities that their peers or the general public – or various subsets of the general public – accept as true.

Evolutionary biologists, for example, devote their labor to explaining why life on earth has evolved in the way that it has, in other words why we have the species that we have now, why these seem to be different from sepcies of the past, and why past and present species sometimes go extinct.  Why, for example, have so many species in the insect order Hymenoptera evolved such elaborate social interdependence (e.g., ant colonies, bee hives), but not all of them, and not all the same sort of social interdependence?  Are these differences the outcome of isolation between descendant subpopulations of an ancestral population, followed by the slow accumulation of random genetic mutations and/or random sampling errors in the transmission of different gene variants (‘alleles’) from generation to generation (a process that evolutionary biologists call ‘genetic drift’)?  Or are they instead the outcome of isolation between different subpopulations, followed by environmental selection for certain gene variants and against others?  What evolutionary biologists don’t ask is, “is evolution a real process?”  Nor should ‘evolutionary theory’ be mistaken to mean “the idea that evolution is real.”  Evolutionary biologists start with the basic assumption that evolution does happen (otherwise, they would not be evolutionary biologists), and evolutionary theory is instead dedicated to ideas about why it happens, both in general and in specific cases.  Darwin’s theory, for example, is a theory of evolution through natural selection, which accentuates a particular set of factors in attempting to account for particular species’ traits or variability between different species' traits.  Alternative theories of evolution instead accentuate other factors than those Darwin focused on.  For example, whereas Darwin's theory focuses on the differential reproductive benefits conferred by different biological traits (anatomical, physiological, behavioral) in light of the challenges posed by particular environments, the theory of evolution through genetic drift instead emphasizes the influence of randomness in the transmission of genes from generation to generation.  Increases or decreases in gene and biological trait frequencies can result from either process (or both in conjunction), though usually with different long-term outcomes.

The community of evolutionary biologists understands these concepts and accepts their truth, allowing them to proceed with their work, but there are also communities of belief within our (United States) population for whom such ideas are either controversial or vaguely understood, or both.  In erotetic perspective, there is thus a risk for dispute that we can all recognize, regardless of which side of the debate you might stand on.  More on this in future posts.

On the flip side, getting erotetic about scientific questions also means being prepared to extend a certain amount of charity or leeway to experts in fields that we do not fully understand.  As an anthropologist and a demographer, I know far more about evolution than I do about quantum physics, for example.  The realities that contemporary quantum physicists accept as true are largely unknown to me, but when I catch faint whispers from them, I am puzzled to say the least.  If an astrophysicist were to ask me one of the sorts of questions that animate their research, I would have little choice but to either consider it absurd (they might as well be asking how long birdsongs are in centimeters) or to concede that my puzzlement is a consequence of my own physics illiteracy.  I would tend to err in favor of the latter, but I suppose that my trust could be easily betrayed were an astrophysicist to have some fun at my expense.  That's not an invitation, by the way.