When Bill Russell was a color commentator for NBA basketball he used to call the slam dunk a high percentage shot.
Michael Schrage asks about George Tenant assuring the President that the case for WMD in Iraq was a “slam dunk. “What percent”, he asks, “is a slam dunk?”
“…to require national security analysts to assign numerical probabilities to their professional estimates and assessments as both a matter of rigor and of record. Policymakers can't weigh the risks associated with their decisions if they can't see how confident analysts are in the evidence and conclusions used to justify those decisions. The notion of imposing intelligence accountability without intelligent counting -- without numbers -- is a fool's errand.”
I’m inclined to agree that a written statement such as there is a 90% chance of snow tonight is more useful than one that say’s it is going to snow . Advocating that intelligence analysts state their conclusions in terms of subjective probability raises questions in philosophy, personality theory, and organization theory, as well as from my own experience.
The question is an old one. Aristotle wrote:
“…for it is the mark of a trained mind never to expect more precision in the treatment of any subject than the nature of that subject permits; for demanding logical demonstrations from a teacher of rhetoric is clearly about as reasonable as accepting mere plausibility from a mathematician.” (Ethics, Book I, para iii).
Let’s try to connect Schrage statement about policymaker’s ability to weigh risks with Aristotle’s comments about a trained mind. I had a long discussion with my rocket scientist brother in law last night about how sometimes scientists and engineers simply have to stand up and say “the question is not answerable with the tools I have available.”
Policymakers are often unschooled in the kinds of questions intelligence can – and can’t answer. A former commander put the need to say that some questions are unanswerable in humorous form. He said that we should be honest enough to give the military intelligence salute. This is similar auto mechanic’s salute: 1) throw hands up in the air; and 2) say “beats the ___ out of me.” The intelligence officer’s version is “beats the ___ out of me, sir .
Knowing that policymakers don’t like this kind of answer, senior intelligence officers tend to make carefully nuanced, but unhelpful, assessments. President Truman’s reaction to this sort of statement was clear when he joked that he wanted one-armed economic advisors. A one-armed advisor could not say “one the one hand this will happen and other hand something else.”
Policymakers are looking for information in order to make a decision. If they are asking questions with an “untrained mind” –- that is in what intelligence can discover – it is the analyst’s responsibility to help refine the question. Here is a quick example. If the policy maker asks “Will North Korea be a nuclear threat in three years” the analysts might suggest another question. This would take the form of how far has the NK nuclear program progressed and what will it take for completion.” The analyst would then able to present the case in terms of developmental milestones. This would give a much clearer picture than a statement that there is “X percent probability of a threat in the future.”
That being said, there are good reasons for asking intelligence officers to state their confidence in terms of percentage confidence.
Stay tuned. There is more to be written on this topic. It will have to wait until I can check some quotes with my other brother in law, the civil war novelist.
Comments