An Intelligence Analyst’s Dream – and Nightmare

Two stories in this morning’s WaPo brought back old dreams and nightmares. In the first, trend watching is automated, easier and, ostensibly more reliable. In the second, burgeoning bureaucracy makes it nearly impossible to deliver useful conclusions to decision makers. On one hand, the IT gurus promise new tools to spot trends early enough to provide warning. On the other hand, management gurus make it virtually impossible to deliver the warning on time.

The dream is found in Google with Judgement , a story by David Ignatius:

“Imagine for a moment that you could study the ebb and flow of public discussion about American politics as if it were a computer graphic. What would this database of "aggregated thought" tell you about the presidential campaign debates?

It happens that a former Republican campaign strategist named Charles M. McLean has created just such a database. His consulting company, Denver Research Group Inc., monitors more than 7,000 sources on a constant, real-time basis -- giving him a window on what he estimates is about 80 percent of all original political content around the world. Using a combination of computer algorithms and human analysis, he sifts this mass of information to discern the "tonalities" that shape global events. This approach has identified key political trends one to two weeks before those changes appear in traditional poll numbers, he says.”

While I don’t know how this works, it is based on the fact that communication takes place through the transmission of text. Automated text searching and aggregating systems have been been promising trend analysis for decades. Maybe McLean has developed a better approach. We have, however, heard these promises before.

The system will not be available to ordinary folks for a long time. It is so expensive that only large institutions, including CBS news and the intelligence community, will have access to it.

This brings me to the nightmare. The institutional bureaucracy gets larger and larger. Paul C. Light writes:

“Once created, the layers spread like kudzu. Between 1961 and 2004, the number of senior title-holders swelled from 450 to almost 2,600. Whereas President John F. Kennedy appointed just 10 Cabinet secretaries, six deputy secretaries, 15 undersecretaries and 87 assistant secretaries, President Bush appointed 14 secretaries, 23 deputy secretaries, 43 undersecretaries and 257 assistant secretaries. That was before the creation of the Department of Homeland Security brought another secretary, deputy secretary, five undersecretaries and two dozen assistant secretaries.

The intelligence community has added its own twists to the thickening. The FBI actually created a new layer of executive assistant directors after Sept. 11, 2001, and increased the number of senior title-holders by half. Just across the Potomac, the CIA placed a new deputy director for community management on a "who's on first" organization chart that only Abbott and Costello could love. At last count, it was led by a director, two deputy directors, two executive directors and at least seven other officials who are called directors.

The bloat is not the only obstacle to effectiveness. The current appointments process virtually ensures that the new intelligence agency will wait months, if not years, to fill its top jobs.”

I offer currently employed analysts this piece of advice: take good notes. Maybe you can follow the path of Richard Clarke. The tools will help you to develop and deliver warnings. After the warnings are ignoried, you can write a best seller claiming that you would have saved the world if only people had paid attention.