British and American Policy on Intelligence Archives
Never-Never Land and Wonderland?(1)
Wesley K. Wark, a noted intelligence historian, once defined the status of British secret service archives as a "Never-Never Land." In an elegant essay on British archival policy in the 1980s he explained how, before 1981, departments of state were told that secret intelligence materials were "never released to the Public Record Office (PRO)." Subsequent to the Wilson Committee White Paper of 1981, this guidance was changed, and departments were thereafter instructed that "the word `never' should never be used." The Wilson Committee considered that in the fullness of time all such records would eventually find their way into the public domain. But for those outside Whitehall, this intriguing double negative seemed to signal little material change, and secret service archives remained "a far-off place" that no independent historian was ever likely to visit.
In the United States, the experience of historians working on secret service records is continually identified by writers as being very different. The US National Archives are often represented as nothing short of a "wonderland" where all sorts of treasures are on public view and where specialist archivists, not least the legendary John E. Taylor who presides over records that originate with CIA, conjure up the most remarkable things. Sensitive British records that are not open to public inspection in the PRO at Kew reportedly are there in profusion. Moreover, it is widely held that items that are not immediately available in the National Archives can be summoned magically through the Freedom of Information Act (FOIA).
The 1990s have seen a number of important changes in policy and practice relating to the management of secret service archives on both sides of the Atlantic. In Britain, the Waldegrave Initiative on Open Government has resulted in the participation of independent historians in the review process and in rapid decisions to release substantial amounts of intelligence material for the 1940s. There have also been positive responses to the specific requests of historians for closed material, and, as a result, significant parts of "Never-Never Land" are now open to visitors.
Meanwhile, in the United States there have been notable changes in practice. Surprising materials continue to be released into the National Archives, but problems caused by the FOIA have rendered this archival "Wonderland" perhaps a little less productive than it once was.
The Waldegrave Initiative has been operating for a year, and it is perhaps an opportune moment to review the new archival elements of this policy in a comparative Anglo-American frame. Accordingly, the first objective of this essay is to assess the importance of the recently released papers, focusing on the Joint Intelligence Committee (JIC) and new evidence on Pearl Harbor as a case study. The second is to use this as a reference point for a wide-ranging comparison of British and American policy on intelligence archives, particularly regarding wartime and postwar materials.
The JIC and Pearl Harbor
It is now widely appreciated by historians that the story of the higher management and control of British intelligence during World War II was unique. Between 1936 and 1941, there evolved, albeit a little uncertainly, an efficient and centralized mechanism for coordinating the numerous British clandestine organizations and for ensuring the careful assessment and distribution of the intelligence they collected. This was the JIC, consisting of intelligence chiefs and chaired by a senior official of the Foreign Office, normally the head of the Service Liaison Department. The JIC was served by a Joint Intelligence Staff which helped to draft its papers and was organized in a way not dissimilar to its opposite number, the Joint Planning Staff.
Together, these committees constituted the engine room of wartime strategic thinking. This system for the coherent and rational management of a rapidly expanding intelligence community was essential if material derived from codebreaking was to be disseminated in time to have an influence on operational planning. It was also essential for complex deception operations that required sophisticated cooperation among deception planners--MI5 (the Security Service); MI6, the Secret Intelligence Service (SIS); and the Government Communications Headquarters (GCHQ), then based at Bletchley Park.
The success of the British system has been contrasted with the equivalent German story, in which Hitler deliberately pitted different organizations against each other. Equally, the American wartime experience was characterized by the lack of a centralized authority. In the postwar period, the contrasting success of the JIC system was underlined by the way in which London imposed it upon regional commands, creating additional JICs in Germany, the Middle East, and the Far East. It is also illustrated by the way in which the JIC system was admired or emulated by Australia, Canada, and the United States.
Given the central importance of the JIC, the release of most (but not all) of its papers and minutes to 1941 has understandably been widely welcomed. At the same time, some have paused to question the value of this particular release, noting that its contents have been largely prefigured in the magisterial five-volume official history of British intelligence during World War II with its extensive treatment of the development of the JIC. Moreover, for many years historians have been aware that perhaps half of the pre-1945 JIC papers (but not the minutes), and at least 30 postwar JIC papers and minutes are available elsewhere, scattered through other categories of files in the PRO. The same question will doubtless be asked of the Special Operations Executive (SOE) archives, because much SOE material has long abounded in the PRO and in Washington. In reality, how valuable has been the release of JIC papers from 1936 to 1941?
The newly available JIC files have undoubtedly contained surprises, even for diligent readers of the official history. This can best be illustrated by focusing briefly upon the vigorous debate initiated by revisionist historians writing on Churchill's and Britain's possible foreknowledge of the Japanese attack on Pearl Harbor. The JIC minutes for 1941 cast real doubt upon their suggestion that Churchill received such intelligence and blocked its transfer to the Americans.
Above all, the revisionist case is undermined because of the multiplicity of conduits through which this sort of information was being passed to the Americans as early as July 1941. Churchill was simply not in a position to exercise detailed control over what was passed to the Americans.
The recently released JIC minutes for 6 June 1941 reveal the precise structure of Anglo-American intelligence cooperation in the Far East. They show that American intelligence personnel had already been attached to the Far Eastern Combined Bureau, which presided over the "collection, collation, and dissemination" of all Britain's intelligence in that region, including signals intelligence.
Before June 1941 much of this Anglo-American exchange had concerned intelligence of interest to the army planners. Now, on 6 June 1941, Britain and America resolved to extend this exchange to intelligence on naval matters and ship movements in the Far East. The British Admiralty reportedly was "very anxious to cooperate." London instructed the Far Eastern Combined Bureau that "there should be a full exchange of intelligence between British and American officials in the Far East," including signals intelligence. The exchange of information from such secret sources required the routine approval of the local SIS chief in Singapore, but the only information that London required Singapore to withhold related, not to codebreaking, but to the fledgling SOE station there. This was because the SOE was beginning to conduct sensitive operations into neutral Thailand, exploring the possibility of a coup d'tat against the government in Bangkok which some considered to be increasingly pro-Japanese. Instructions to exchange intelligence material on the basis of "the fullest cooperation" were also issued to British personnel in Shanghai, Hong Kong, Tokyo, Chungking, Bangkok, Manila, and Peking. Accordingly, the multiplicity of links between British and American intelligence developing throughout the Far East from the summer of 1941 renders it improbable that Churchill exercised detailed control over the exchange of individual documents.
One particular revisionist account of Pearl Harbor has gone so far as to suggest that on 5 December 1941 Britain's JIC met and discussed at length the impending Japanese attack on Pearl Harbor. This author, Constantine Fitzgibbon, writing in 1976, claimed to base these assertions on a letter received from none other than Victor Cavendish-Bentinck, wartime chairman of the JIC. How do these claims compare with the JIC minutes from the fateful week before Pearl Harbor? The minutes, which are entirely extant for this period, reveal that the JIC did not even meet on 5 December. It met on 3 and 9 December, and Pearl Harbor was not mentioned at either meeting.
The minutes of the JIC will be valuable to many historians for many different reasons. But their release also serves to underline a number of wider points about the nature of secret service archives. It is increasingly clear that sensationalist accounts of important historical events, or malignant interpretations of the actions of politicians and officials, do not result from the early release of intelligence records but from their prolonged closure.
Files that are closed for an absurd length of time are an invitation to entrepreneurial writers to speculate in an over-imaginative way on the nature of the "dirty secrets" that such archives supposedly contain (why else, these authors ask, would they be closed?). Academics may eventually establish the truth of the matter and expound it in tomes that will find their way onto the shelves of university libraries. But the public mind is increasingly informed by the conspiratorial versions of contemporary history, often piled 50 deep in the High Street bookstores. The damage done to the wider public understanding of history by such books is unlikely ever to be undone.
Above all, the lesson to be learned from the eventual release of JIC records is that serious researchers and governments share a clear vested interest in the prompt release of such materials. The fact that these and many other intelligence documents released under the Waldegrave Initiative are remarkable only for their dullness is, paradoxically, very important. After all, one of the elementary rules of textual analysis for all historians when considering a document is to reflect not only on what is there, but also on what is not there.
British and American Archives Compared
What are the essential differences in the way in which the British and the Americans have managed the release of secret service archives? Most obviously, while Britain is releasing the archives of the JIC, of the SOE, and of Churchill's signals intelligence summaries(Dir/C) at approximately the 50-year point, the Americans released most of their equivalent materials at the 40-year point. Much of the British signals intelligence archive, notably for the Far East, (but also some material relating to wartime Europe), remains to be released, and as yet there has been no discussion of the release of wartime SIS materials. Meanwhile, the Americans have released almost all their wartime signals intelligence and their Office of Strategic Services (OSS) records to 1947. OSS fulfilled the functions of both Britain's SOE and SIS during World War II.
The result of this early American release has been the appearance of high-quality academic histories of these subjects. In the 1980s, as a direct result of this de facto 40-year rule, American historians were able to locate retired OSS veterans and conduct interviews that could be compared with the written record. Almost all the participants had safely reached retirement; enough survived from the policymaking level to ensure that conducting interviews was a rewarding activity for historians. A rich and sophisticated literature on the history of OSS is now emerging from the synthesis of oral testimony and written records.
By comparison, as SOE records become available in 1994, there will be few survivors available to talk about the policy level. When wartime SIS archives are released, the written record will doubtless stand alone, devoid of accompanying oral testimony. This is worrying when one reflects on the arcane and technical nature of some secret service activities. The full meaning of some of these documents will not necessarily be self-evident to future historians.
American authorities do not make a squeamish distinction between secret service activities conducted during war and peacetime. CIA is now depositing hundreds of files relating to its changing internal organization for the period up to 1953, including a great deal of correspondence by such luminaries as Walter Bedell Smith and Allen Dulles. Moreover, many American National Intelligence Estimates (NIEs)--the equivalent of British JIC papers--are available for the mid-1950s. This has ensured that diplomatic and military historians working on postwar American subjects have been able to integrate the intelligence dimension into their wider work. The slower release process in Britain tends to encourage the writing of a separate "intelligence history" that is sometimes devoid of context.
Yet the exciting CIA releases of the early 1990s pale beside the extraordinary post--Cold War initiative begun by CIA Director Woolsey in 1993. CIA's Historical Review Group has essentially completed the declassification of political and economic NIEs on the Soviet Union through 1984. Articles from CIA's quarterly journal, Studies in Intelligence, also are being declassified.
Declassification of the first 11 Cold War covert actions was begun in 1994. The operations for which records will be released include France and Italy in the 1940s and 1950s, North Korea in the 1950s, Iran in 1953, Guatemala in 1954, Indonesia in 1958, Tibet in the 1950s and 1960s, Cuba in the 1961 Bay of Pigs, the Congo and the Dominican Republic in the 1960s, and Laos in the 1960s. In March 1994 the CIA's Center for the Study of Intelligence, together with the Truman Library, hosted a conference for historians on CIA during the Truman period, and it used the opportunity to explain this magnificent program in detail.
The American interpretation of the subject of intelligence is also commendably broad. In Britain, the debate has focused narrowly upon the release of secret service records held by SIS, MI5, and GCHQ. In the United States, proper recognition has been given to the importance of somewhat less secret Army, Navy, and Air Force intelligence activities. Before 1953, in both Britain and the United States, armed service intelligence personnel outnumbered those in the secret services such as SIS and CIA. Typically, in occupation areas like Austria, the local CIA station was marginalised by a much grander program of military intelligence operations. This is not just a question of numbers, but of relevance.
For a British historian writing on the subject of British defense policy or strategic planning in the immediate postwar period, the most valuable intelligence materials might be those relating to RAF intelligence rather than those of the secret services. Yet the vast blocks of military service intelligence records withheld in Britain for the period after 1945 pass almost without comment.
But most US military intelligence records have now been released for the period up to 1955. These records contain important material concerning how United States Air Force intelligence analyzed the Soviet Union and formed its impressions of Soviet strategic airpower. Historians have been delighted by what they have found in such records. Meanwhile, government officials have been pleased by the way in which this development has relieved pressure for the release of much more sensitive records, typically postwar National Security Agency (NSA) (signals intelligence) material.
A further respect in which the United States advances a broad definition of intelligence records relates to its commendable emphasis upon regional or "theatre command" intelligence records. Britain and the United States administered large areas of the world during the 1940s and 1950s, and this generated vast regional archives, typically relating to Germany and Austria. These files are important to British and American historians and to academics from the many countries under Allied control.
While many of the British intelligence papers relating to the occupation of Germany and Austria are yet to be released, all of the voluminous files of the United States European Command (EUCOM) and Far Eastern Command (FECOM) are available, including the registry files of their regional intelligence headquarters. Some of the finest studies of US policy in Germany and the Far East have drawn extensively on these types of files.
A final area in which there is something to be learned from good American archival practice is captured records. During the 1980s, historians of Asia were surprised to find that CIA released into the US National Archives the files of the Shanghai Municipal Police, an extensive British-run security agency. These materials, which included files on Sun Yat-sen and Ho Chi Minh, had enjoyed a checkered history. Captured by the Japanese at the outbreak of World War II, they had fallen into the hands of the Nationalist Chinese in 1945. In 1949, with the Communist victory in China, these records were again in peril, and they were eventually offered to CIA and transferred to Washington.
Although relating to the prewar period, these records were examined with interest by security authorities on account of the information they contained on previous associations between serving Western officials and Asian Communists. Academic historians of Asia in the 1980s, perhaps the last of many groups to make use of these records, pronounced them invaluable.
Britain also presides over captured archives. A good example is the German Security Service records relating to Soviet espionage in Europe during the 1930s and 1940s, the so-called Red Orchestra. German wartime security operations had been highly effective and had damaged Soviet espionage in occupied Europe by 1944. German records, therefore, offered a full picture of the nature and techniques of Soviet espionage and were acquired by British Special Counter Intelligence Units as they entered Germany in 1945. Thereafter, CIA was allowed to inspect them to compile a classified internal history of Soviet military intelligence operations in Europe. Yet while this American history, based on materials held in Britain, has long since been declassified and published, the fate of the files themselves remains something of a mystery.
British Documents in the US Archives
British intelligence reports in the files of the Shanghai Municipal Police in Washington are just one example of the profusion of British secret service materials that have reportedly long been available in the US archives. Tales of the existence of such materials in Washington assumed legendary proportions during the 1970s and 1980s. How accurate were these reports and how has the American system of managing sensitive British records changed?
Two important distinctions can be drawn. The first is simply chronological: for the period up to 1945 a vast quantity of British intelligence records of many different types abounds, much of it not yet available in the PRO. Its extent defies comprehensive description here, but some indication can be given by referring briefly to historians who have already exploited these materials. John Costello, in a recent study, Mask of Treachery, has demonstrated that both Special Branch and MI5 materials appear regularly in low-level State Department files. Bradley F. Smith, in a path-breaking study of Anglo-American cooperation in the field of signals intelligence to 1947, has illustrated how much GCHQ-related material is available in Washington.
Perhaps the most concentrated source of British intelligence is contained in the archive of the OSS, now running to many tens of thousands of files. Approximately 3 to 5 percent of this material is of SOE, SIS, or MI5 origin or discusses British intelligence in detail. A historian at Harvard University has recently completed a detailed study of cooperation between OSS and wartime MI5/SIS, using unprecedented amounts of British secret service documentation from OSS files. These are not just isolated documents; there are often entire files of British material.
For the period after 1945 the story is rather different. Archivists in the United States often refer to some sort of agreement between the State Department and the British Government detailing the categories of material that London requested be withdrawn from American files. These guidelines have been enforced with much more rigor in relation to postwar materials. Accordingly, there are few MI5 or SIS records for the postwar period, the primary exceptions being occasional reports relating to Soviet agents that have found their way into declassified FBI files. Perhaps only 200 or 300 pages of material relating to GCHQ are scattered through many different collections of papers, most of which deals with electronic intelligence (ELINT) and communications security (COMSEC) rather than communications intelligence (COMINT). Perhaps fewer than 10 percent of postwar JIC papers are extant in Washington, again in a scattered way, in different types of military and diplomatic files.
The case of postwar JIC papers seems to offer some indication of how industrious American archivists have been in seeking to remove British material. Various cover notes in American files often refer to two or three "attached" British JIC reports at a time, but the papers have usually been removed. We know that after 1945 the British continued to send large numbers of their JIC reports to the Americans via Britain's JIC/Washington located at the British Embassy, hinting meanwhile at hopes of reciprocation. The flow of American JIC reports to Britain had ceased on V-J Day, initially thwarting London's hopes for a continuing postwar intelligence exchange. The British tactic of bombarding Washington with unreciprocated JIC reports eventually paid dividends. On 25 September 1946 the American JIC concluded:
If it is desired to continue to receive the British JIC intelligence estimates, it is submitted that it must be done on an exchange basis, otherwise the source will dry up. Since there are many areas, particularly in parts of Europe, the Near East and the Middle East, where the British sources of information are superior to those of the United States, it is believed desirable that the United States JIC continue to receive such estimates. This view is reinforced when the world situation is considered.
The Americans recommended that exchange now proceed on a quid pro quo basis. Thereafter, intelligence estimates were routinely exchanged and delegates to major postwar Anglo-American conferences departed London for Washington armed with new JIC material. Only a minority of these exchanged papers have survived and, for those seeking JIC papers after 1945, the US archives are not a wonderland.
This assertion about the paucity of postwar material must be qualified by drawing a second distinction. It has already been remarked that the United States defines its intelligence archives broadly. The vast collections of American service intelligence and regional intelligence contain a great deal of British service intelligence material. Typically, in the papers of US organizations like EUCOM and OMGUS, a substantial proportion of the papers of the British Intelligence Division, Control Commission Germany, are to be found for the 1940s and 1950s. It is disturbing that a reasonable history of this important British organization could be written in Washington but not in London, where the main records have been incinerated.
The Government-Historian Interface
The United States has generally handled the release of its intelligence records in a wise and often generous way. The American system for response to the specific requests of historians for declassification of closed materials is less impressive. The presidency of Jimmy Carter during the 1970s saw the United States introduce FOIA legislation. In principle, its objectives were commendable, enshrining the idea that government information belonged to the people whose taxes paid for it to be generated. Henceforth, the inherent presumption was that documents should be released on request and the burden of proof lay with the government departments to demonstrate any need for them to remain closed. By the 1980s, however, an increasing gulf was opening between principle and practice. Perversely, FOIA, at one time a centerpiece of the American archival system, was actually beginning to restrict access to documents.
Two things had gone wrong. In 1982 the US Government chose to modify FOIA guidelines, allowing government departments wide areas of exemption. This could often be challenged only by resorting to time-consuming and expensive legal action. More important, the FOIA system was being abused by the American public's submission of an avalanche of frivolous requests that absorbed considerable amounts of government time and resources. In the early 1990s many government personnel who had previously been assigned to the routine and rapid declassification of large blocks of records had to be redeployed to search, often unsuccessfully, for individual documents requested under FOIA. The result has been that some US diplomatic and military records for the late 1950s are still not available due to personnel shortages caused directly by FOIA. Some British Foreign Office records are now being declassified several years ahead of equivalent American series.
Moreover, the material secured through FOIA has not been of sufficient value to offset this problem. American historians have noted with dismay that the printout of FOIA requests for 1991 reveals that well over half were for material on sightings of Unidentified Flying Objects.
The revised British system, backed by a recently announced Advisory Council, offers at least the potential for a more effective and rational system of response to requests. In addition, the presence of independent historians is a particularly welcome development likely to inspire confidence within the academic community. This system appears to be well placed to make judgments about priority among what will be an increasing flow of British requests. It is also less likely to be hijacked by frivolous requests or by demands for unrealistically large quantities of material by one particular lobby group. While the absence of an American-style legal right of access enshrined in law appears disadvantageous, in reality the extensive litigation that the American FOIA system sometimes inspires is one of its least attractive aspects.
Are there any positive lessons to be learned from the American experience of responding to specific requests? The US State Department certainly has some important advice to offer. Faced with increasing backlogs of unprocessed material, it has made available a handlist of categories of documents that it still holds for the period before 1970, with their approximate quantities in shelf-feet. The thinking behind this is commendably logical. How can historians make informed requests without knowing what types of records government still holds?
These lists, often provided in a preliminary form, have nevertheless allowed records officers to gain a more informed view of the priorities expressed by historians. This is supplemented by more informal contact between those who manage the records and those who use them.
Published Documents and Classified Histories
The last two years have seen promising developments in terms of the published information on intelligence in both Britain and the United States. The prospect of more British published intelligence documents has improved because British officials are taking greater interest in the use of sanitization to release documents. This is not a wholly new development; sanitized files have been making their way to the PRO for many years. Nevertheless, this technique apparently will now be exploited more frequently, bringing with it both new problems and new possibilities.
The problems relate largely to the efficient use of time by hard-pressed Departmental Record Officers, because the physical process of blanking out specific sensitive passages on a page is extremely time consuming. An hour spent sanitizing a specific document is an hour not spent declassifying another file. This problem also is recognized in the United States, where sanitization is primarily employed for documents requested through FOIA, and is only rarely used voluntarily by those reviewing records for routine release.
The advantages of sanitization are also considerable. In the field of intelligence, the presence of a single name or a brief reference to a type of source can close a lengthy and valuable document. In the United States, the most impressive use of sanitization has been made by CIA to facilitate a published volume of CIA documents on the Cuban missile crisis. This 370-page book includes raw agent reports, estimates, material on how intelligence was disseminated to policymakers, and retrospective studies conducted immediately after the crisis reviewing the CIA's performance. Some of these documents are sanitized, a few rather heavily, but this volume stands as a powerful testament to the potential value of this technique. The trend toward published intelligence documents is gathering pace in the United States with the prospect of two volumes of Foreign Relations of the United States (FRUS) devoted to the American Intelligence Community (1946-50) scheduled for publication in 1994-95.
The expressed intention of British Departmental Record Officers to make greater use of sanitization makes it more likely that we will see published intelligence material in series such as Documents on British Policy Overseas (DBPO). To the surprise of many, a recent volume of DBPO contained the fabled "Global Strategy Paper" for 1950, albeit with three lines dealing with Asia deleted.
This approach could certainly be extended to JIC papers for the same period in the way that many equivalent NIE papers are routinely included in the FRUS series. A volume of DBPO focused upon the work of the postwar JIC would be an ideal reflection of the new Open Government policy and of forthcoming JIC releases into the PRO. Equally, it might be possible to choose a particular event, perhaps the crises in Czechoslovakia and Berlin in 1948, and produce a volume of related British intelligence material that would stand alone, in much the same way as the Americans have done for the Cuban missile crisis in 1962.
The field of official history is a separate area that offers promising developments. The official history of intelligence and special operations is something that Britain seems to do particularly well. One virtue of this sort of history is that it allows the many historians working on wider subjects, typically international history, to take some account of the intelligence dimension even though the primary papers are, in many cases, not available. After all, international historians far outnumber specialist intelligence historians and, moreover, intelligence is often best understood when set in the context of wider developments.
Official histories have done much to encourage this contextual approach. While the SOE papers for France are due to be opened in 1994 or 1995, historians have long been able to turn to M.R.D. Foot's acclaimed official study. Equally, while SIS papers for World War II are unlikely to be available for some time, many important questions that have confronted historians of this conflict are resolved by the magisterial official series on British intelligence.
Perhaps what is needed is an official history of British intelligence during the early Cold War (1945-54). The majority of British historians working on the wider international history of the postwar period do not want to spend hundreds of hours rummaging through the low-level files of Britain's Intelligence Division, Germany. But there are certain basic questions concerning the nature of Britain's picture of Soviet capabilities and intentions that most Cold War historians would like to have answered by an official history.
There are already promising developments in this direction. In 1993 a "semiofficial" history of the Intelligence Corps from its origins in the mid-19th century through to the 1990s was published. Four chapters, accounting for a quarter of the text, cover the postwar period. The discussion of the postwar period is deliberately selective, and it is overtly stated that Northern Ireland is omitted (readers will also note that Army signals intelligence operations are conspicuous by their absence). Nevertheless, this book is an auspicious development.
It is worth dwelling briefly on the parallel American experience. There is no official US intelligence series that can compare with the impressive British official volumes on intelligence or SOE activities in France. The Americans, however, have been quick to release many of the internal classified historical studies that the intelligence services produced years ago for their own reference purposes. For the wartime period, the obvious example is the Signal Research Histories, summarizing much of the "take" from signals intelligence operations against the Axis and describing the development of American signals intelligence organizations.
More recently, CIA has declassified sanitized versions of its internal official histories of the development of CIA from 1945 to 1950 and from 1950 to 1953. But the quality of these volumes is, at best, uneven. Moreover, NSA has decided not to release any chapters of an internal history covering the period from 1940 to 1952.
Nevertheless, for the many American international historians working on the early Cold War period, CIA internal histories are sufficient to answer important questions. Similar British internal studies exist and consideration might be given to their release, albeit in a sanitized form.
Finally, there is the field of memoirs. The 1980s were a period when the literary efforts of retired British secret service officers were greeted with official hostility. This approach might be contrasted with the American system in which retired intelligence officers are required to submit their memoirs for clearance, a process which usually results in sections of the memoir being sanitized. Historians are not alone in benefiting from memoirs that have appeared under this regime. Arguably, the widely quoted memoirs of William Colby have done much to encourage a sympathetic view of CIA's place in US postwar history. Some have even interpreted this post-1975 wave of approved memoirs as an attempt to offset some of the damage done to CIA's reputation during the various Congressional enquiries of the early 1970s, notably the Church Committee. Whatever their motivation, they have contributed to a more balanced picture of American postwar intelligence.
There is now a strong case for Britain to look again at a similar system of official clearance for memoirs. The most forceful argument for this is derived from recent developments in the former Soviet Union. In the early 1990s numerous retired KGB officers, some quite senior, began to write their memoirs. Their numbers are increasing, and, as a result, we will soon have a growing picture of British intelligence in the postwar period, albeit from a dubious source.
It will be dubious because some of these officers are bitter, unreconstructed Communists with an axe to grind, and because retired KGB officers naturally want to present the achievements of their own service in the best possible light. There is a growing danger that the history of British intelligence will be written by its enemies. One has only to recall the damage done by Kim Philby's propagandistic memoir, My Silent War, which remains one of the most widely read accounts of SIS.
In 1993 there were unconfirmed reports that the British Government had made representations to the Russians, asking them to keep certain aspects of their archives relating to Britain closed. However successful this approach was, it will not address the problem of KGB memoirs. At the same time, there have been welcome signs of a more balanced British approach to secret service memoirs, most obviously provided by the unopposed publication of the memoirs of Desmond Bristow, A Game of Moles. But if full advantage is to be taken of this slightly bashful shift of policy, some formal mechanism for the clearing of secret service memoirs needs to be instituted.
What, in conclusion, are the important lessons to be learned from recent developments in the British and American archives? The sternest lesson to be derived from the Americans' management of their intelligence materials during the last 10 years is that their sheer volume presents significant problems. From 1945 to 1989, the Western intelligence community enjoyed steady growth. Consequently, the quantities of old intelligence records awaiting declassification will increase for the foreseeable future. At the same time, these types of records are awkward and time-consuming for officials to process for release.
To deal with these sorts of problems, a government requires a carefully considered open-government policy and the commitment of adequate resources and the effective management of those resources through a clear dialogue with historians. Given a picture of what governments hold in their archives, historians will readily articulate their priorities. After all, resources for serious historical research are increasingly limited, and thus the appetite for records is finite. Yet, at present, demand and supply are not sufficiently coordinated. Everything that is routinely declassified is not immediately required by researchers, nor is everything required by researchers given priority by departments. There also will be a growing need to screen out excessive or frivolous requests. There are encouraging signs that a sensible system that fulfills some of these requirements is now emerging.
There also is a need for a climate of greater trust between historians and British departments of state. This will develop only slowly, for intelligence historians who took an adversarial view of official policy in the 1980s have largely concluded that they have been proved right by the nature of some of the documents released in 1993. One can only wonder at those who have hitherto maintained that terrible things would happen if the public were allowed to gaze upon, for example, records relating to secret postal interception from 1742 to 1792. Accordingly, it is hardly surprising that intelligence historians initially greeted the announcement of the Waldegrave Initiative in 1992 with profound skepticism.
This attitude is now changing with the release of several thousand files relating to wartime SOE, to JIC, and to Churchill's personal summary of signals intelligence materials (Dir/C). This event understandably caught the imagination of the press, but perhaps the most impressive development was the rapid response to detailed requests by researchers for particular files.
There is still some way to go before a balanced policy on all aspects of British intelligence archives emerges, but historians were pleasantly surprised by the reality of the Waldegrave Initiative as it unfolded in its first year. The experience of "Never-Never Land" will not be forgotten quickly, but, at the same time, there are indications that something wonderful has begun to happen.
(1)A fully footnoted version of this article appeared in Contemporary Record, Vol. 8, No. 1 (summer 1994) pp. 132-150.