Inside Story

The spies who came out of the dark

The allure of the secret service in the British imagination is also the entry code to citizens’ data, writes David Hayes in London 

David Hayes 14 December 2015 4440 words

Icy formality: British home secretary Theresa May, the architect of the controversial Investigatory Powers Bill, with prime minister David Cameron, pictured here in central London after a ceremonial welcome for Chinese President Xi Jinping and his wife Peng Liyuan in October. Adrian Dennis/AFP Photo


How should the relationship between security and liberty work in a democracy? When a draft surveillance bill from David Cameron’s government gave this old question new force last month, a fortuitous flurry of cultural references to Britain’s secret state added a local twist. Can devotion to the world of espionage inhibit protection from its excesses?

The jihadi massacre of 130 people in Paris, which felt very close in London the cities are only two hours apart by high-speed train tilted the debate towards more state powers while raising echoes of a warning phrase from the Northern Ireland conflict: “the politics of the last atrocity.” The government sought, and after strenuous public argument won, parliamentary backing to join the airstrike campaign against the Islamic State, or ISIS, in Syria. (Royal Air Force bombers already operate in Iraq by permission of Baghdad.)

Alongside these developments came Britain’s quinquennial strategic defence and security review, with its pledge to reverse some of the damaging cuts of the previous (2010) review. Replacement of the submarine-based Trident nuclear missile system is going ahead, at spiralling expense. In straitened financial conditions, a have-it-all approach to power – hard, soft, nuclear, cyber – is a difficult trick to pull off; but across political, diplomatic and military elites, there is a feeling that “retreat, Britannia” has gone too far for safety or comfort.

A turbulent month ended with security issues higher on the agenda, and the political mood more febrile. If the geopolitical fallout of these events is still being worked through, it is becoming clearer that the different capacities the British government seeks to exert depend for their effectiveness on cybersecurity. That reality is also bringing Britain’s “deep state” ever closer to its everyday one, and to the lives of citizens.


The Investigatory Powers Bill, or IPB, is intended to collate the powers of digital surveillance distributed across existing legislation and establish a new oversight system. The hefty document 520 pages, including supplementary material proposes granting official agencies “bulk powers” to map communications data across networks. Companies such as broadband or mobile phone providers would be required to store everyone’s “internet connection records” – websites visited, calls made, texts or emails sent, social media used – for a year. An individual log would be available for inspection by government agencies on production of a warrant signed by the home secretary, with the whole process subject to a so-called “double lock” of both review and permission by a new “investigatory powers commission” led by senior judges.

Proponents of the bill argue that it is intended to arm the authorities with powers to improve law enforcement, security and intelligence by, for example, accessing and analysing data that might be used precisely to forestall a multipronged, “marauding” assault of the Paris type. The strong contemporary association between surveillance, espionage and terrorism – made by authorities, media and popular culture alike – means that the IPB is seen very much through the lens of existential (rather than “routine”) dangers.

Opponents retort that an exclusive focus on the high-security aspects of the bill is misleading, for its provisions could be used (except where specific exceptions are made) by tax and welfare officials, for example, as much as by police and intelligence ones. The powers it envisages, they say, are dangerously unmanageable, represent a damaging extension of state authority into the private realm, and create widespread risks (including for business) of data leakage.

By default, the IPB’s justification is political as well as legal, and the circumstances around its launch mean that the debate goes beyond surveillance itself into broader policy areas: counter-terrorism, intervention in Syria, radicalisation among a minority of British Muslims, and Russian and other cyber threats. The strategic defence and security review confirms the broad strategic judgement that Russia and ISIS are considered the prime security concerns, with the latter’s transnational activities (violent attacks, online recruitment, funding, propaganda) an urgent challenge.

ISIS’s shooting down of a Russian passenger jet over Sinai occurred just days before the IPB was introduced to the House of Commons; attacks in Beirut and Baghdad, as well as the one in Paris and the lockdown in Belgium, occurred just afterwards. Seven unspecified UK-centred plots are said by Andrew Parker, head of the domestic security agency MI5, to have been thwarted since late 2014. Yet alongside relief, the paradoxical impact of such information is to breed both fear and resignation, hardly ingredients for a confident citizenry. Knowledge, these days, is also powerlessness.


Around the bill’s launch, nothing was left to chance. The inner guard of honour was impressive: reports on the work of MI5 and interviews with anonymised personnel by the BBC’s respected Gordon Corera and Frank Gardner, plus an in-depth Times series on the work of GCHQ, the state’s electronic intelligence hub, west of London (part of the “Five Eyes” network that links Australia, Canada, New Zealand, the United Kingdom and the United States). “Unprecedented access” was the watchword here, a signal from high-ups that subtly conveys both their patronage and our privilege.

On the fringe, too, the dark suits were well-armed. The coincidental launch of Spectre, the latest film in the James Bond franchise, and London Spy, a BBC drama about an operative of MI6 (the overseas security agency, formally the Secret Intelligence Service), provide high-insulin doses of dark allure. A host of books reflects a market and an appetite ever unsated: studies of the notorious cold war defectors Kim Philby and Guy Burgess, biographies of the novelist John le Carré and the communist spy mentor James Klugmann, works on the guerrillas of the Special Operations Executive and the decoders of Bletchley Park during the second world war. All these ancillary artefacts draw on and feed Britain’s enduring fascination with the ambiguous realms where secrecy and power mingle.

Even the IPB’s introduction on 4 November – the eve of the discovery of the Catholic plotter Guy Fawkes’s attempt to blow up king and parliament in 1605 – lends itself to the ambience. In a classic example of subversion made cosy through mockery, that event has long been marked by a festival of bonfires and fireworks. The proposed legislation works with the same grain – albeit applying the wisdom more of “statelore” than “folklore” – to pull the opposite trick. We spy on you for your benefit against dangers most sinister, which both of us recognise from the long history we share. Deal?

The case is not stated so crudely, of course. It doesn’t need to be. British public life often resembles a masquerade in which all players skilfully perform their assigned role and need no prompting to supply their own script. So it is with the IPB, where the icy formality of its architect, home secretary Theresa May, and the invariant fury of Shami Chakrabarti, head of the advocacy group Liberty and the bill’s most prominent adversary, configure a public argument that will continue until the legislation passes. Rival camps are already clustering around routine binaries – security vs privacy, state powers vs individual rights, data harvesting vs mind searching, law as permission vs law as constraint – leaving millions of citizens to find their own way through a more intimate question: Who, if anyone, can I trust?


The question is given added point by the chequered history of counter-terrorism laws introduced since 9/11, aimed at “protecting the community and preventing a terrorist act.” Several have hit judicial obstacles and political blowback, reflecting the wider context of heightened suspicion of state infringements on liberty during the amorphous and permissive “war on terror.” In June, for example, an official tribunal admitted that GCHQ had illegally intercepted and retained communications by Amnesty International and a South African NGO.

This background informs the IPB’s consensual, even defensive tone. Its provisions, says May’s introduction, “are the product of discussion with industry, academia, technical experts and civil liberties groups. They seek to protect both privacy and security by improving transparency and through radical changes to the way investigatory powers are authorised and overseen. The draft bill only proposes to enhance powers in one area – that of communications data retention – and then only because a strong operational case has been made.”

If this was an attempt to disarm critics, it failed, at least with the most adamantine. Chakrabarti’s verdict came in an instant: “After all the talk of climb-downs and safeguards, this long-awaited bill constitutes a breathtaking attack on the internet security of every man, woman and child in our country.” Then the gauntlet: “We must now look to parliament to step in where ministers have failed and strike a better balance between privacy and surveillance.”

Edward Snowden, whose disclosures of mass surveillance provoked the transnational legislative flurry of which the IPB is a part, called its intent “the most intrusive and least accountable surveillance regime in the West.” Both on the liberal left and the libertarian right, the case fuses justicial and utilitarian concerns: indiscriminate monitoring of an entire population’s online activity will undermine people’s freedom while failing to ensure their security.

Such responses are given added force by the contested recent history of counter-terrorist legislation in Britain. A familiar cycle during the post-millennium years is of hasty measures repented at leisure following unintended consequences, post-enactment scrutiny by parliamentary committees and appeal courts, backbench revolts, and righteous outrage from lawyers, academics and journalists. The legacy of this troubled period – stretching through the Labour governments of 1997–2010 – is division and mistrust.

At each point, the state’s chief target was shifting. The Terrorism Act of 2000, superseding laws introduced to ease the fight against the Irish Republican Army, was still dominated by that experience. The London transport bombings in July 2005 and subsequent attacks by young Muslims made home-grown extremism the priority. The cascade of official files released by Julian Assange and Edward Snowden since 2010, and constant hacking attempts from (among others) Russia and China, put cyberwar in the frame. ISIS, at once territorial, decentred, hybrid, and fully “at home” on the web as even al Qaeda never was, concentrates these different types of threat.


The architects of the Investigatory Powers Bill present it as taking account of leaps in technology and communications that render earlier legislation obsolete. They point out that the closest previous measure of this type belongs, in geopolitical terms, to prehistory: the Regulation of Investigatory Powers Act of 2000, which opened a stop-go era of security measures which too often found governments tied in judicial knots of their own making.

That act’s tributaries include the Data Retention and Investigatory Powers Act of 2014, rushed through parliament to restore the security services’ rights to access phone and internet records after these were rendered null when the European Union’s Court of Justice struck down a directive obliging European telecoms companies to retain personal data for between six months and two years. (The court has the power to annul laws that contravene the EU charter of fundamental rights.) In turn, two sections of the Data Retention and Investigatory Powers Act were declared unlawful in July 2015 by Britain’s High Court on privacy grounds, though suspension of the order eventually gave the government until the end of 2016 to introduce new legislation consistent with EU law. The IPB is that legislation.

Another stuttering antecedent is the draft Communications Data Bill, introduced in June 2012 but abandoned ten months later following scrutiny by parliamentary committees and rights groups (whose moniker, the “snoopers’ charter,” caught on), and opposition from the Conservatives’ coalition partners, the Liberal Democrats. The latest to attract opprobrium is the Counter-Terrorism and Security Act, a fast-track measure passed in February 2015, which restricts movement of suspects, imposes new obligations on public institutions (notably educational) to help prevent radicalisation, and ensures that retained data can be linked with IP addresses. After the general election in May the Tories’ coalition shackles are off, but they still have to navigate through parliament, courts and civil society.

The IPB’s mollifying tone reflects this messy backdrop. Incorporating advice from three expert sources – David Anderson, the current independent reviewer of terrorism legislation; parliament’s intelligence and security committee; and a high-level study of surveillance by the Royal United Services Institute – the bill appears less sweeping, and its operation subject to clearer (if, to critics, no more reassuring) oversight. Anderson’s 380-page report, A Question of Trust, published in June, also cautions against seeing surveillance solely through a “terror” lens: far more official applications to access data under existing legislation relate to financial, drug and sexual activities than to suspected terror. Such a catch-all provision – an enduring problem in itself – underscores the importance of safeguards to prevent abuses of power.


Oversight is shaping up to be a key theme in the IPB’s consultation period. Again, the bill grants ministers authority to issue a warrant for interception of communications data, with any enforcement subject to a judge’s approval (urgent cases apart). The campaign group Big Brother Watch says the United Kingdom will be alone among the Five Eyes alliance in lacking “true independent authorisation” for use of such a power, and urges this to “prevent any political abuse of the warrantry process.” (Here as elsewhere, the assumption of British exceptionalism – and that the grass is always greener in Canberra – is open to question.) It also notes that the “detail of a warrant, including the operational purpose, can be changed after a warrant has been approved, sometimes without any further approval.”

A cautionary precedent, notes the barrister and scholar David Pannick, is the system of “terrorism prevention and investigation measures,” or “Tpims,” introduced in 2012, which restrict the activities of suspects for up to two years following an MI5 assessment (and routinely affect around ten people each year). Tpims themselves replaced the more draconian “control orders” imposed by the Prevention of Terrorism Act of March 2005 (as they would be by Australia’s Anti-Terrorism Act of nine months later). That act replaced the emergency legislation brought in after 9/11 – the Anti-Terrorism, Crime and Security Act of December 2001 – but, following a ruling that found the use of control orders unlawful, was itself repealed by the Terrorism Prevention and Investigation Measures Act in 2011. Yet both control orders and Tpims provided for legal representation for those being monitored, as part of the oversight. The absence of any equivalent in the IPB draft fuels concern about possible unwarranted – in both senses – intrusion.

An editorial in the Spectator (“We trust our spies. We shouldn’t trust this bill”) recalls that one effect of the Regulation of Investigatory Powers Act was “nosey council enforcement types and police officers trawling through suspects’ computers on a hunch,” and warns: “For too long, successive governments have hammered through spying laws while muttering about national security and hoping their alarmism could take the place of serious debate. This time, we must do better.”

Those words were published on the eve of the Paris massacre, whose implicit but widely recognised effect in a stunned London was to bend the surveillance debate further in the direction of greater security. The choreography surrounding the IPB’s introduction had, it seemed, been given an unscripted, nightmarish reprise.

The government lost no time in seizing the moment. Four days after Paris, on 17 November, the chancellor George Osborne made a well-publicised visit to GCHQ where he announced almost £1 billion (A$2.1 billion) of extra resources “to protect Britain from cyberattack and develop our sovereign capabilities in cyberspace” (a sinuous contrapunt). This includes the recruitment of 1900 extra staff pledged a day earlier by prime minister David Cameron. Over five years this will take the government’s “total cyber spending” to £3.2 billion. Such attention and investment reinforce the security agencies’ increasingly high profile – along with the police, army and emergency services – as frontline defenders against cyberterror underworlds.

The dust from this sequence of events has yet to settle. But it is noticeable that the security services themselves – or what Ben Harbisher calls “Britain’s fusion intelligence complex” – have been largely untouched by disputes aroused by the IPB, Paris and Syria. This is another indication that their part in the ugly fallout of the “war on terror” may be in remission.

MI6 in particular has been vulnerable over several years in relation to the wars in Iraq and Libya, and GCHQ to disclosures of surveillance in the Assange–Snowden file dumps. Over Iraq, controversy centred on the intelligence served to Tony Blair’s government in 2002–03 about Saddam Hussein’s reputed possession of weapons of mass destruction, or WMD; over Libya, it focused on the deals made with Colonel Gaddafi’s regime in 2004 to persuade him to abandon his nuclear ambitions. The latter raised further questions over the rendition of jihadis to Libya and their subsequent torture (a prominent case, that of Abdelhakim Belhadj, is now being heard by Britain’s supreme court).

If these major cases seriously bruised public trust, individual incidents also caused embarrassment: the long aftermath of the poisoning of the FSB (Russian secret service) defector Alexander Litvinenko, an informant of MI6, for example, and the “body-in-the-bag” death of GCHQ/MI6 employee Gareth Williams (which inspired London Spy).

The biggest piece of unfinished business is the long-awaited Chilcot inquiry into British decision-making vis-à-vis Iraq, which will be published in 2016. Even if Blair and his ministers prove to be its principal targets, intelligence personnel may not escape blame (over unreliable sources or advice, and closeness to government). By then, though, it will have been thirteen years since the war began and five since the last British troops left Iraq (just as Syria’s uprising was turning bloody.) That delay, bitterly condemned by the media and some families of the 179 soldiers who died, also makes it more likely that today’s security agencies will be able to deflect criticism. And their plea that much of the security landscape has changed, and they with it, could sound plausible enough to earn a licensed reprieve from the public. After all, part of the institutional wisdom of the British establishment is that time itself acts as a shock absorber of discontent.


The agencies found the going tough in other ways. MI5 and MI6’s imposing Thameside HQs were increasingly exposed to the same pressures of social change (in technology, media and corporate power, for example) as were other branches of state. A consumerist, post-deference public was demanding more from ruling institutions, and had more opportunity to scorn their failings. The security services were discovering, as the post-Diana monarchy had, that distanced respect as the engine of consent had run out of road. Neither the people’s trust nor even the right to be heard could any longer be taken for granted, but had to be won – especially when new powers or financial benefits were the prize. (“I have to be seen to be believed” was Queen Elizabeth’s pithy summary of the deal.) Cultural adaptation was becoming a vital accessory of political authority.

That backdrop helps explain MI5/MI6’s belated “coming out,” which lifted veils arguably long self-harming as well as anachronistic. The actual process, again like that of the monarchy, was cautious and experimental. Its banal ingredients – though presented as radical at the time – included the services’ heads being named and then delivering speeches and giving interviews, along with a more open recruitment policy, a website and twitter account, and greater media access (the current BBC charm offensive being the latest phase). There was also a trickle of serious histories: Christopher Andrew’s of MI5 (authorised), Keith Jeffery’s of MI6 (official), and Richard Aldrich’s of GCHQ (unofficial), as well as emergent new specialisms around the cyber domain. At last the “deep state” was acquiring a public face, and to that extent catching up with the open society.

The makeover helped earn the secret state a fresh look. In Britain, rooted institutions acquire a hold on the national imaginary which, if they learn to turn outward, can offer surprising assets for contemporary use. Under new management, and with an obliging media, the clubbable, old-school complacency that gave a free pass to Philby and the rest of “Stalin’s Englishmen” could be subtly redrawn as part of the entertaining past of a now modernised service. Change within continuity is so much of what keeps the British show on the road.

The elevation of women in a bastion of conservative maledom, especially at senior levels, had particular symbolic effect. The career officer Stella Rimington had become MI5’s first female director-general (1992–96), soon followed by Eliza Manningham-Buller (2002–07). Around 40 per cent of the agency’s 4000 staff are now women. (And 8 per cent are of non-white background, a growing proportion though still well below the 14 per cent in the UK population. As for the partner service, “We do have a serious diversity problem,” John Sawers, MI6 chief in 2009–14, has said.) Those promotions, again in league with wider social trends, helped open doors to stories of women who had been SOE agents, Bletchley Park code-breakers, and cold war spies; among them are the astounding Polish-born résistant Christine Granville (Krystyna Skarbek) and the spy-to-baroness Daphne Park, both the subject of recent biographies.

In its glamorising collusiveness, the Bond iconography, too, is central to the delicate interplay between reality and symbolism. Cross-referencing is an inexhaustible staple of news coverage, media comment and film chatter alike. The Times’s guide to GCHQ is given a Bondian sheen: “For your eyes only.” The MI6 head (“M”) played by Judi Dench in seven films made her first appearance in 1995, during Rimington’s incumbency. Spectre’s depiction of a plan to merge MI5 and MI6, then submerge the remains in a multinational “Nine Eyes” conglomerate dedicated to building a global surveillance system, is a manic wink at the audience.

The crown of this state-nation-culture pas de trois was the “queen’s” skyfall at the London Olympics opening ceremony in 2012. The fact that this work of English genius was choreographed and written by left-wing republican northerners was itself suitably Bondian: mavericks rescuing the establishment.

Walls have tumbled down elsewhere. Jonathan Evans, head of MI5 in 2007–13, reviews Spectre in the current affairs monthly Prospect. (Colleagues are “flattered” by the Bond association, he writes. “If you don’t want to talk in any detail about what you actually do day-to-day, then having your agency personified in a daring, glamorous, sexy and successful super spy, protecting the country, and the world, against evil plots, is a pretty good alternative.”) Agency heads, current and former, are now public figures. Rimington writes spy novels and was a Booker prize judge, Manningham-Buller delivered a series of the BBC’s Reith lectures.

Literature and drama can’t get enough of the secret state. Tom Stoppard’s 1988 play Hapgood, also with a female head of MI5, is being revived in Hampstead. William Boyd’s riveting thriller Restless and Ian McEwan’s bleached Sweet Tooth both have heroines transformed by their lives in the service. Boyd has also written Solo, a “continuation Bond” story, and his bravura tale of a murder suspect going off-grid in contemporary London, Ordinary Thunderstorms, has freedom in a datafied society as its theme.

The flat earth of metadata, encryption and instant everything is bypassing the le Carré–Len Deighton world of double agents and dead drops, faded streets and seedy corridors. Yet the rhythms of techno-power and psychic need often misalign, and cold war iconography still exerts a pungent allure. This is not mere conservatism, nor the domestication of a past that was unsettling when actually lived through (though the British excel at both). It is also the way a sceptical country negotiates change: keeping a foot in the past to avoid being swept away by the future, its promises or its threats.


That outlook informs wider attitudes towards surveillance, where reassurance mingles with wariness. How can people remain so sanguine in face of state plans to track their online (and by direct extension offline) lives in a wealth of intimate detail and with unknown consequences?

The Guardian’s Jonathan Freedland (“The spooks will keep spying on us Brits: we clearly don’t care”) invokes wartime memory, innocence of a security state, and “stubborn British deference to power” to account for the contrast with Americans and Germans. The factors noted by Gavin J.D. Smith’s Inside Story article (“Life in the goldfish bowl”) to explain Australians’ indifference to the new data retention law – among them public anxiety about terrorism, a data-mediated society and a “watching” culture – are also in the British mix. What, after all, is the cyberworld but a universal “snoopers’ charter”?

In fact, Jonathan Cable’s detailed analysis of public opinion since the Snowden materials were released, published in June 2015, finds ambiguity and practicality in equal measure, with many variations according to the questions asked and their timing. The preponderant instinct in a time of insecurity, however, is to give domestic agencies the benefit of the doubt. No wonder, as the mordant Financial Times columnist Janan Ganesh says, “The electoral market for civil libertarianism is close to nil.”

The condition may not be immutable. For the moment, British voters understand that the trade-offs between security and liberty are complicated and becoming more so. And that trust is two-way, and must be earned. With the “deep state” too, the people remain players in any deal. •