The Spy Catchers
Volume 1 of the Official History of ASIO, 1949–1963
By David Horner | Allen & Unwin | $69.95
Perhaps, as Churchill said of Russian intentions, it was a riddle wrapped in a mystery inside an enigma. But if it was like a set of babushka dolls, it was perversely pataphysical. Each succeeding doll was larger, not smaller, than the preceding one. And none of the people inside it had the slightest idea of where they were, how much of the real story they knew, or how many layers were outside or inside their ken. It was the Case. It preceded ASIO, but it was what brought ASIO into existence, and it was what primarily occupied ASIO’s best brains for the first seven years of its existence. By then, the organisation had developed its culture, its legends and its sense of mission, and had built up a good deal of bitterness and animus towards the Labor Party.
On the defection of Vladimir Petrov, the organisation’s most public coup, ASIO had been right and the charges made against the organisation by Dr H.V. Evatt, leader of the Labor Party, had been fantastic and silly. Yet Bert Evatt seemed so certain that the organisation had been a dupe of a Russian false-flag attempt, or perhaps was a witting player in a cunning plot by prime minister Robert Menzies to rob him of an election win, that some mud stuck. Labor figures regarded the organisation with deep suspicion; ASIO officers, stung by the slur on their professionalism, began to do some of the partisan things Labor critics were alleging they had always done.
David Horner’s The Spy Catchers — the first of a projected three volumes — is a biography of the organisation rather than a history of the Case, the Petrov Affair, the Royal Commission into Espionage, or the Communist Party of Australia. But these were, of course, the focus of so much of ASIO’s attention between its birth in 1949 and the end of this volume. It’s a sympathetic but far from uncritical read, not least in its detail of how shenanigans at the royal commission disrupted ASIO’s early neutrality and made some of its members come to think of themselves as warriors in a war against communism, in which Labor was, at best, on the sidelines. It shows how, for perhaps fifteen years, some in ASIO ceased to be cool, careful and scholarly public servants focused on threats to the nation’s security, and instead focused rather more on justification, insurance and survival.
Of itself, little in the book is new. Any number of excellent studies have examined the early Cold War in Australia, the Communist Party, the highly secret Venona counterintelligence program inside what became the US National Security Agency, and the Case, Petrov and the personality of Evatt himself. Robert Manne’s The Petrov Affair, for instance, published after the story of Venona had finally emerged, was written after extensive interviews with Sir Charles Spry, who had headed ASIO from a year after its inception.
Horner had more access and more detail about what ASIO officers were doing, and has a few more names to add to the mosaic. But his story — so far at least — occasions no significant surprises. Its detail about the Case, though absorbing, tells little that is new, even as it again confirms, against continuing doubters, the reality of Soviet espionage in Australia, the involvement of people associated with the Communist Party, and the absolute need for some sort of security function in government.
But for those who have doubts, there are still questions, as the book itself acknowledges. It is plain, for example, that there were public servants, particularly in what was then the Department of External Affairs, who were passing on information they thought interesting to Wally Clayton, a mysterious and secretive member of the Communist Party’s senior apparatus. Clayton was in charge of maintaining the party’s internal security, especially against infiltration by agents of police and security services. He had managed many of the party’s “illegal” activities during that time, early in the second world war, when Stalin was in a non-aggression pact with Hitler, and the party was opposing involvement in the war. The party became legal again after Hitler attacked Russia, and by 1943 the Soviet Union had diplomatic and trade representation in Australia, as well as a number of Soviet journalists working for Soviet news services and undoubtedly acting as spies.
It is clear that Clayton was seeking out information, not least about politics and the war effort, that would be of interest to his party’s Australian executive. It is clear that at least until 1954 the Australian party followed all of the twists and turns of Soviet policy; that it was, in effect, controlled by the Soviets, and that its Australian members regarded the interests of the Soviet Union as their own. It is clear that Clayton passed on at least some of the material he gathered to Soviet spies, mostly those posing as journalists. And it is clear enough that Clayton took some direction about intelligence tasks from his Russian contacts, and that he tried, diligently, to carry out these tasks (or, perhaps, orders).
All of this was more or less known, as part of the Case, even before the defection of Petrov and, later, his wife Evdokia in April 1954. A trained but not very competent officer, Petrov had been assigned to keep an eye on the Soviet émigré community but had inherited a bigger task, unbriefed, when his senior officer left and was not replaced. By this time, though, the Communist Party’s attraction was much faded and the Cold War was under way, so there was hardly any effective espionage going on. Most of those who had cooperated with Clayton were inactive, or off the scene.
But if the documents Petrov brought confirmed earlier information about the Case, there were still questions about Soviet spying, the answers to which we still do not know. Information from Petrov, and from other Soviet defectors in Canada, Britain, the United States and Europe, showed that there were generally two Soviet organisations gathering intelligence information, usually without any reference to each other. What came to be known as the KGB (or the NKVD or MGB) was focused on political, diplomatic, economic and general scientific intelligence; a separate organisation, known as the GRU, collected military and defence intelligence, including anything that could be divined about weapons systems, missiles and nuclear bombs.
Typically, each of these organisations ran two separate networks of spies: one was “legal,” controlled by a person operating under diplomatic or trade cover; another was an illegal network, under much deeper cover, often controlled by a Russian who had been infiltrated into the country concerned, and reporting back via entirely different networks. In ideal situations, none of these networks knew anything about the others, although, in extremis, it was not unknown for agents of one network to be instructed, from Moscow, to make contact with a specified agent of another.
We “know” there was a legal KGB operation in Australia from the middle of the war, and we know that it had some successes, if hardly spectacular ones, from about 1943 to 1948. We know nothing about the operations of any illegal KGB network, although there is reason to believe that it existed. We know absolutely nothing about GRU operations during the period, but we do have reasons to suspect they were occurring (without any connection with Clayton). Neither ASIO nor other operations ever gained useful counterintelligence about such activities, or exposed any networks or spies.
It might be tempting to suggest that if any such spy networks had existed, they would have been found; ergo, they probably did not exist. If that is the case, we must ask why the Soviet Union, which tended to have a similar order of battle in all countries of interest, adopted a different model for Australia, and why Clayton might have been used as a spymaster despite a different pattern of practice elsewhere.
It is trite to add that a mole in ASIO might have been very useful in helping steer ASIO activities away from operations threatening to expose other networks. And we know that the more cynical agencies sometimes throw crumbs — important agents of legal networks, for instance — in the way of security services by way of distraction. Horner’s authorised, if uncensored, history is alive to all such speculation and self-doubt. But its access to the files leaves the reader no wiser or more able to judge.
Just as significantly, we are little wiser about whether those who passed on material to Clayton, or who gossiped with him generally about their jobs, knew that they were giving him information to be passed to Moscow. They could hardly have failed to know that they were passing it on to the party, but in that highly conspiratorial organisation — membership of which was more akin to being in the Society of Jesus or Opus Dei than the Gould League of Bird Lovers — there was nothing particularly odd about relentless discussion of politics, the awfulness of capitalist politicians and the plight of Mother Russia. Mere rules about security did not overrun comradeship, even if they should have done. And that’s quite apart from the attractions of big-noting oneself, and exaggerating one’s role in affairs, to progress socially or in party circles.
Clayton, wittingly passing on information, and particularly purloined documents, may have been gathering information more for party purposes than for direct transfer to Moscow. After all, he and his colleague Ted Hill were still the party executive officers most concerned with maintaining the party’s operations even if it were subsequently declared illegal again. They stashed printing presses, paper and equipment around the nation and bought safe houses. They also maintained stocks of unofficial members, whose help could be called on in emergencies, who were continually screen-ed not only for being in security employ, but also for deviation from the latest line from above.
Even ASIO itself is still unsure, after all these years, about whether those who gave information to Clayton did so wittingly. Or whether some others, such as Fergan O’Sullivan, Rex Chiplin and Rupert Lockwood, who gave briefing materials about Australian journalists and political conditions to the Russian embassy, should be regarded as spies. Unwise perhaps, disloyal perhaps, and certainly snide. But Chiplin published the materials leaked to him in party newspapers (and KGB reports quoted the paper not the source documents); and while it is undoubtedly true that the KGB “studied” the reports of O’Sullivan and Lockwood, it does not appear to have regarded them as sufficiently compromised, or “on the small hook,” to have given them intelligence duties.
Likewise, its close scrutiny of the Communist Party gave ASIO much information about membership lists, discussions at meetings, and strategies in trade union elections. It was clear enough that there were people in the party who believed in revolution against the established order; it is a good deal less clear that anarchy, or civil war or sabotage was being actively plotted. It was never quite clear who was dangerous, but always obvious who was zealous. The better ASIO got at surveillance, the more it came to appreciate that the party was full of personalities, factions, feuds, unresolved arguments and, increasingly, doubt. Doubt about Stalin and Stalinism. Doubt about the communist dream. Doubts after revolts in Poland, East Germany and Hungary. Doubts about the value of keeping on keeping on.
Among all this, the Case remains the most fascinating, the most exhaustively interpreted and perhaps — as reflected in our modern-day security institutions — the most enduring part of the ASIO story.
For thirty or so years, about a dozen Australians, at most, knew something about the Case. Even most of these knew only tiny bits, and had no idea of the big picture, or the big pictures beyond that. Around the rest of the world, perhaps another fifty knew any details of the Australian Case, and perhaps fewer knew how it fitted into the bigger picture.
It was a secret so important that the wrapping paper of the secret was more important than the secret itself. Indeed, even the next layer concealed secrets more important than anything outside it. They were secrets so big that Britain, after the war, had to contemplate whether its loyalty to Australia was more important than its alliance with the United States. For a long time, it seemed as if Australia might win that tug of war, if only because America scarcely trusted even Britain with some bits of the secret.
The Australian Case was different from the American Case, or for that matter the British Case, though they fitted into a pattern. In all Cases, the secret of the first veil was that spies had undoubtedly passed on political and defence secrets to Soviet agents in the 1940s and might still be doing so. The British Case had revealed the treachery, in relation to details of the atomic bomb, of the British scientists Klaus Fuchs and Alan Nunn May, and had provided clues to Soviet networks in Britain and its colonies. The American Case showed how the Russians had obtained details of atomic bomb construction at Los Alamos from, among others, Julius and Ethel Rosenberg. The Australian Case showed that Wally Clayton had been collecting political and defence information from contacts in the Department of External Affairs, and from the office of its minister, Bert Evatt, and passing it on to Russian agents.
To the Americans, the Australian espionage was all the more disgraceful because of what appeared to be a complete lack of security consciousness. Add to that the fact that Washington viewed the Chifley Labor government, and its foreign minister Evatt, as dangerously left-wing.
But if those in the know about the secret of the first veil wanted something done about the Soviet agents and their local helpers, they didn’t want that process to hint at the secret of the second veil. Indeed, it would be better to do nothing about the first secret than to put the underlying secret in jeopardy. The underlying secret was that American knowledge about the Cases came from decryption of Russian codes through a quantum leap in the sophistication of message-reading by what eventually became the world’s biggest and most secretive spying organisation, the National Security Agency.
The Soviet Union, a great home of mathematics, had impressive, virtually unbreakable codes. They involved double encryption using a code dictionary to turn words into numbers, and then applying one-time sets of random numbers. With the numbers used once only, and then for a limited volume of text, deciphering messages using all of the familiar forms of pattern-seeking seemed impossible, the more so in pre-computer days when the capacity to apply brute force, searching through billions of possible combinations, was very limited. Yet some brilliant intuition was used to solve major parts of the dictionary, helped by the fact that, for a short time, Soviet code-masters issued the same one-time sets to different areas.
Some of the messages spoke of the spying and gave clues about who the spies might have been. But the fact that American code-breakers could read some Soviet spy traffic was a secret much more important than the knowledge that espionage had occurred, or the catching of particular spies. These were merely battles; the ongoing intelligence might help the United States and its allies win a war. If the Russians didn’t find out that America was reading some of its secret intelligence, further cryptological breakthroughs might come, more traffic from more sources might be intercepted, and the Soviet Union might fail to realise that some of their spies had been compromised.
Behind that were other ultra-secrets of the 1940s, 1950s and 1960s — that the British and the Americans had been intercepting, and often decrypting, the secret messages of the armies, navies and air forces, together with the diplomatic and trade communications, of their enemies and their friends for ages, ultimately reaching the point spelled out by the WikiLeaks and Snowden leaks of the 2010s. This intelligence had helped the allies win the war against Germany and Japan, but wartime security had concealed the extent of the contribution. Even the security-conscious in rival countries, and sometimes friendly countries, had no idea of how extensive and clever the processes had become.
Inside the Case was another secret that only a tiny few of those very few who knew the first layer were allowed to know. And inside that were other secrets, so secret that, for some guardians of the western alliance against the Soviet Union, whole nations were expendable in the interest of maintaining them. At one point in 1948, for example, the Americans asked Britain to choose, in effect, whether it wanted to be an ally of the United States or of Australia. It could not be both. Because the Case showed that Australia was lax about security, and because the Americans saw the Labor Party — which was in government at the time — as tantamount to communists, they would not show Britain anything it might pass on to Australians.
It was a secret so terrible, and powerful, that the initiates had to live all of their lives with restrictions on the right to travel. Indeed it became a gnostic faith into which one was “indoctrinated’’ in stages, after which one saw the world differently.
The first bits of the puzzle only passingly affected the Soviet Union; indeed, Moscow knew at least something of it. It was the fact that even before the second world war British and American cryptographers had succeeded in decoding some of the secret messages being passed between German and Japanese military units and diplomats. With some, particularly Japanese diplomatic traffic, they could decipher messages within twenty-four hours: indeed, had the Americans been more diligent, they could have deduced that Pearl Harbor was about to be attacked.
In due course, Americans, Australians and the British were able to listen to tactical messages between army, naval and air units; orders and reports going to Tokyo from local commands; a good deal of the material coming from Japanese embassies in Europe (including Berlin and Moscow); as well as information about merchant shipping, supply and reinforcement, and espionage activities.
The British, Polish and French were also reading a good deal of the German military traffic, after making critical breakthroughs in learning to decipher messages encoded by the “unbreakable” Enigma machines. Some of the techniques, some of the personnel (including Alan Turing), some of the places (including Bletchley Park), and the pioneering work on computers are now the stuff of thrillers and television series, but the secret — that the Allies could read much of the German army traffic, intermittently a good deal of the U-boat traffic, and a lot of the diplomatic codes — was an ultra-secret for more than twenty years after the end of the war.
So secret and important was the edge this gave that it was seen as more critical than winning any actual battle. It would be better to lose an army if some plausible explanation — aerial reconnaissance, say, or a human spy — could not be concocted to explain how enemy movements had been anticipated, because the moment the enemy suspected its codes were being read the edge would be lost.
Many admirals, air marshals and generals were out of the loop because they could not necessarily be trusted. When there was critical information to be passed on, even to people in on the secret, it would usually be attributed not to code-breaking but to some other source, in case the Germans were reading our mail (as, of course, they were trying, with some success, to do). Important “ultra” information was also passed, as information from spies, to Moscow once the Soviets became allies, particularly from 1943 on.
The official war histories, the generals’ memoirs, and the popular accounts of military campaigns made no reference to allied access to many of the enemy’s communications. Some reputations — Montgomery’s, for instance — may have suffered had it been known how much help had been received. For decades, however, the secret stayed safe. After the war, the intelligence partnerships became more formalised, more extensive, more effective and, of course, increasingly focused on the Soviet Union and its satellites. Australia, along with New Zealand and Canada, was a very junior partner to America and Britain in the exclusive “Five Ears” English-speaking club from 1943, but our geographic position in Asia and, later, the window on parts of the USSR and on China provided by Pine Gap and Nurrungar, made us a valuable provider of raw material.
In 1943, a few senior Australians, all military bureaucrats, knew something of these intelligence arrangements, although none had gone beyond the first layer of secrets. But no politicians knew, nor did they need to. But then came the Case, and in perhaps the most embarrassing way possible.
The Soviet Union was spying on Australia. Its ability to do so, at least in some form, was known, and followed the arrival of Soviet diplomats and journalists mid-war. Russia’s heroism in standing up to Hitler’s onslaught had earned it admirers, and the Communist Party of Australia, slavish in its following of every aspect of the Soviet line, was at a membership peak of more than 20,000. There were people of communist, or leftish views in the public service, the army and the intelligentsia — and in any event, was not the Soviet Union our ally against Germany and Japan? The Soviet diplomats and journalists were sending copious reports back to Russia, usually via radio to China’s wartime capital Chunking; but though they were recorded, they could not be read.
One day during the war, however, the Russians gave a Japanese diplomat a confidential British assessment of how the British colonial world might look after the defeat of Japan. The diplomat cabled it home, adding that he believed it had been obtained by a Soviet diplomat in Canberra. The Americans could read the Japanese diplomatic traffic, and were able to compare their knowledge of the contents of his cable with a Russian cable from Canberra. It provided one of the first significant insights into Soviet codes. On another occasion, a Chinese naval attaché passed on a report to Chunking in a code that was broken by the Japanese; carefully enough read, it could have tipped off the Japanese that their own codes were being read. The message from the attaché contained cabinet-level information to which he should not have had access.
The British didn’t tell Australia that some Russian diplomatic cables were being read and that these had pointed to spies in the Australian public service. They pretended that the information came from an agent who had defected from Moscow. Australia was told it had been cut off from intelligence sharing, and that it could only redeem itself, if at all, if it so improved its internal security that Americans would know material was not being leaked.
In time, a very few learned something about Venona. Most of those checking out the clues had no idea of the provenance of the information. Nor did those they confronted, some of whom confessed to their involvement.
Much the same information had disclosed the names of Klaus Fuchs, who was convicted in 1950 of supplying information to Moscow, and the Rosenbergs and chemist Harry Gold, who had helped pass nuclear secrets. In those cases, as during the Petrov royal commission, courts, tribunals and inquiries were told nothing of the role of code-breaking, or why Americans were so supremely certain of the justice of their case. Ultimately, the three judges of the Petrov royal commission were briefed about the specifics of their case, but only to persuade them to assist in devising a report that made no reference to how the Case had actually come about.
Venona was to be a secret for more than forty years. Details, and intercepts, are not on the National Security Agency’s public website. We know, thanks to WikiLeaks, Julian Assange, Bradley Manning and Edward Snowden, how much further technological surveillance continues today. One cannot, however, fail to think that there are still secrets — about Russian spies, about spying on Russians, about spying in Australia, and about spies in ASIO — to emerge. It might well take an unauthorised history of ASIO to reveal them. •
