Government Snooping Chains

The NSA and GCHQ Programs provide insight into government snooping chains. This blog also provides a forward to my book Sail Chains. The current focus on surveillance and privacy is based on the actions of Facebook, Amazon and google. The allied western intelligence agencies do much more. Previous blogs

Secure communications, tracking, and other jargon is used within this tale. The descriptions are based on real techniques used in Information Technology and Intelligence Surveillance. Some are described below to avoid lengthy passages of explanation in the narrative of the book and here in this blog for public edification.


Five-Eyes is an intelligence alliance comprising Australia, Canada, New Zealand, the United Kingdom, and the United States. These countries are parties to a multilateral agreement which is a treaty for joint cooperation in signals intelligence. It is a wide ranging agreement and includes facilities in each country including the Government Communication Headquarters, GCHQ, in Cheltenham, UK, and the National Security Agency, NSA, with its HQ at Fort Meade in the USA. Both organisations also carry out their own operations and programmes. Many of these programmes were known in small parts to the media and hence general public; however the extent of these programmes was not well known until the revelations leaked or stolen by Edward Snowden.


NSA Signpost
NSA Sign post

Legal restrictions in all Five Eyes countries are supposed to restrict or prevent gathering of information on citizens. Secret courts, FISA (Foreign Intelligence Surveillance Act) in USA and CMPs (Closed Material Procedures) in UK provide legal cover. Parliamentary or Congressional scrutiny is very limited. In many cases the elected representatives or their advisors do not have sufficient levels of security clearance to see the material about the programmes let alone the technical understanding of the implications.

The NSA is not supposed to spy on US citizens without a warrant but they can collect data about US citizens whilst spying on foreigners. In one example 90% of the data collected in one sweep was about US citizens (>9,500 citizens out of 11,000 contacts). In this way GCHQ can spy on US Citizens and vice versa and each can pass data to the other through the Five Eyes and not be subject to any scrutiny. It is clear from multiple sources that this spying is not just on threats but also on journalists, whistleblowers and multiple other targets that the security services have decided are legitimate targets.


The main details were leaked by Edward Snowden to two reporters Barton Gellman who published via the Washington Post and Glenn Greenwald in The Guardian. A film maker, Laura Poitras conducted interviews and also acted as a go-between, especially between Snowden and Gellman during the initial contacts when source VERAX was making contact. Many of the electronic copies of papers and programme details remain unreleased by the journalists. The NSA and GCHQ continue to deny many of the details, see here


was the code name of a warrant less surveillance program begun under the George W. Bush administration’s President’s Surveillance Program. The National Security Agency (NSA) program was approved by President Bush shortly after the September 11, 2001, attacks and was revealed by Thomas Tamm to The New York Times in 2004. STELLARWIND’s output is fed into the MAINWAY database


PRISM is a code name for a program under which the NSA collects internet communications from various US internet companies. The NSA had placed collection systems directly in the data centres of the large tech companies including Microsoft, Google, Apple, Facebook and others. Due to the nature of Internet routing many non-US connections route or partially route via the data centres. Thus privacy campaigners use Virtual Private Networks, VPNs, and other techniques to mask their messages. These techniques are also used by enemies including terrorists.


MAINWAY is a database maintained by the NSA (and Five Eyes partners) containing metadata for hundreds of billions of telephone calls made through the four largest telephone carriers in the United States: AT&T, SBC, BellSouth (all three now called AT&T) and Verizon. The existence of this database and the NSA program that compiled it was unknown to the general public until USA Today broke the story on May 10, 2006. It is estimated that the database contains over 1.9 trillion call-detail records. The records include detailed call information (caller, receiver, date/time of call, length of call, etc.) for use in traffic analysis and social network analysis, but do not include audio information or transcripts of the content of the phone calls.

Contact Chaining

Contact Chaining is a method of querying data held in MAINWAY to produce contact maps and then using associated algorithms of contacts of a target several levels away e.g. secondary, tertiary and beyond contact of contacts of contacts. Because MAINWAY holds historical data, officially 5 years worth for US citizens but with many caveats, previous contacts can be traced. Exceptions to deletion are any link to on-going or security investigations. This gives rise to an exponential increase in potential contacts. If the first contact has ten contacts and each has ten more and these in turn have ten more at 3rd degree of separation there are now 10x10x10 = 1,000. Most humans have far more than 10 contacts thus chains become very large very quickly. The game 6 Degrees of Kevin Bacon, the US Actor, demonstrates this is more humorous ways.

Algorithms are used to reduce the numbers or combine them into groups. This data is then combined with other communications data, for example, social medial posts and email, to build up a contact map. The seed in this case is the initial target or intercept which by correlating with another seed B. Contact C is thus linked in the chain.

Or a real one shown by the US news programme 60 Minutes

NBC Real Chain

Any one of these contacts or nodes could be the enemy that is sought or allow movements, locations and activity patterns to be tracked thus enabling potential targeting for surveillance or more direct action. Sometimes the enemy is unknown. The node shown is a phone, email address, social media handle, website, which the technique attempts to link to an individual or organisation. A phone number of a head office could be used by hundreds of contacts. How the data is processed into MAINWAY with other named systems mentioned is shown below:

Mainway Dataflow showing Government Snooping Chains

Enemies attempt to hide this activity by changing contact methods, encryption of the content of messages and other evasion techniques. For the NSA and GCHQ they are also tasked with creating method of protecting data from such intercepts by foreign powers or bad actors. Other techniques such as operating cell techniques can founder with just a single contact under the chain. Thus operational security measures are overcome. For example two terrorist cells with a leadership planning a coordinated attack can be linked.


A system called CO-TRAVELER is designed to track who meets with whom and covers everyone who carries a mobile/cell phone, all around the world. CO-TRAVELER collects billions of records daily of phone user location information. It maps the relationships of mobile/cell phone users across global mobile network cables, gathering data about who you are physically with, and how often your movements intersect with other phone users. The program even tracks when your phone is turned on or off.

TOR – Protects from government snooping or does it?

Tor is free and open-source software for enabling anonymous communication by directing Internet traffic through a worldwide overlay network. It consists of more than seven thousand relays designed to conceal a user’s location and usage from anyone conducting network surveillance or traffic analysis. Using Tor makes it more difficult to trace the Internet activity to the user: this includes “visits to Web sites, online posts, instant messages, and other communication forms”. Tor’s intended use is to protect the personal privacy of its users, as well as their freedom and ability to conduct confidential communication by keeping their Internet activities unmonitored. It was created by the Office of Naval Research and DARPA as a security protection project and the papers from Snowden demonstrated that the NSA had managed to set up infiltration into the network.

VPNs – Encrypt channels of communication thus protecting chains but not that a connection exists

Virtual Private Networks (VPNs) are encrypted channels between one or more network points. They normally use some form of shared encryption key between the end points thus preventing interception of the communication content; however, the metadata (data about data) can still be traced including locations, of end points, times of transmission, etc. Therefore, STELLARWIND can collect this data and deposit into MAINWAY for use in Contact Chaining. If a phone is used as the data connection CO-TRAVELER can match locations and obtain more metadata in addition to location and other data sources nearby.


GCHQ has a different set of names achieving the same ends see here. This shows the applications CARPART, PRIMETIME, SNAPDRAGON, MoaG, SORTING FRIENDS sending data into a system called CHART BREAKER, and onwards into CONTACT LENS which is the Contact Chaining output from MAINWAY and CHART BREAKER

Updated Paperbacks and Kindles

Updated paperbacks and kindles for all my published works are now available on Amazon following resetting of all paperback versions. In addition, translated versions of some books are available.

Below is a list in publication order

Demise An Agent’s DemiseJan 2013LinkYes None
Survivors To The SurvivorsJun 2013Link NoSpanish
Persuasive Cover The Persuasive ManMar 2014LinkIn Production None
Rise An Agent’s RiseJul 2014Link No None
Fives Cover World Of FivesJul 2014Link NoSpanish
Landscape LandscapeMar 2015Link NoPortuguese
Intervention InterventionJan 2016Link NoPortuguese
Prize Cover An Agent’s PrizeApr 2016Link No None
Trilogy The Demise TrilogyApr 2016Link No None
Sailing Cover Sailing ClearFeb 2017Link NoPortuguese
Too Much Not Too MuchDec 2017Link No None
Counter CounterOct 2019Link NoPortuguese

We Won’t Be Asked To Vote On…

Some things that we won’t be promised or asked to vote on, by the main parties in the election

  1. Reduction in MPs from 650 to 500 with attendant boundary changes
  2. MPs deselected that do not take up seats e.g. NI parties that don’t
  3. Proportional representation introduction outside NI – mentioned in Lib Dems last time but not by main parties
  4. Lords reform – to voted for chamber (power away from House of Commons)
  5. Fixed term limits for Prime Minister or MPs e.g. See Costa Rica 4 years for President 8 years for MP equivalent
  6. Tax take must equal or be more than government spend and borrowing cost – no deficit increase
  7. Any hope that any manifesto might get implemented with thought for unintended consequences – see 6 for spending commitments
  8. Understanding by any MP of unintended consequences see 6 and 7
  9. Understanding by any political party that it takes years to train doctors, nurses, teachers therefore you cannot magic more in a couple of years unless imported from abroad thus denying other countries’ their health/teaching
  10. That Donald Trump has no vote in UK Election, nor does any other foreign politician (including EU). Their opinions are irrelevant, and should be ignored especially tweets
  11. If you don’t vote for who you want to represent you do not complain when someone else is elected – introduce compulsory voting
  12. Challenge those who are asking for your vote how they will vote on the issues. Sue them for breach of contract if they don’t!

The Longer View

We need to sometimes take a longer view. The media (social and mainstream) would lead most people to believe that the world has never been so dangerous or that governments have never been so incompetent or evil. Whether its the latest news on Brexit, Trump’s Tweets, Putin’s evil intent or the chaos that is Syria and Yemen. There appears to be a tendency to ignore even recent history.

It seems apt that I take a personal perspective. This year marks the 40th year (in August) since I entered full time employment. I left school at 17 (Much to my parents annoyance) half way through my A Levels. It was unusual then to even take A Levels. University was for less that 20% of the population. I was in the vast majority. The UK was a very different place. A Labour Government was in charge under Jim Callaghan as Prime Minister and his infamous ‘Winter of Discontent’ was to follow that winter.

I had dreams of being a rock star (don’t laugh) instead, I became an Electronics Technician earning (via a weekly brown envelope holding the cash) £29 per week. Equivalent to about £160 in today’s inflation affected money. The inflation rate was 7.8%. It would rise higher along with interest rates.

The UK was known as the sick man of Europe which was the trading block known as the EEC or Common Market as it was commonly known. The UK had joined the EEC with Denmark and Ireland in 1973 – there was no referendum. It did have a referendum to remain in 1975.

The troubles were 10 years old in Northern Ireland, and that year 82 deaths were attributed to the conflict. The next year would be worse. The UK was still supporting the new Oman regime but elsewhere was not directly militarily involved, except of course the day to day cold war with the Soviet Union. To give some context to Middle East troubles including Oman this is a handy reference –

Jimmy Carter was US President having beaten Gerald Ford in 1976. Ford of course had become President after Nixon resigned in August 1974. (pence for Trump?) The Iranian revolution would follow in 1979, with all the troubles that caused Carter. Brezhnev was the leader of the Soviet Union and would be until Nov 1982. He would be in charge when the Soviet Union entered Afghanistan in 1979 –

It took the Soviet Union nearly 10 years to ‘get out’ of Afghanistan. The US supported the Mujahideen including Bin Laden with advisors and weapons. The US Ambassador to Afghanistan was murdered in 1979 initially blamed on a communist group. The US, UK and others still have troops there since the October 2001 invasion, and in Iraq since March 2003. How long will it be until we completely leave both countries.

In 1978, Germany was split East and West with Berlin a split city in the East. Yugoslavia and Czechoslovakia, were still joined. Solidarity in Poland was still a couple of years off. The KGB was the security apparatus of the Soviet Union supporting the vassal states in the Warsaw Pact. A Bulgarian exile, Georgi Markov was murdered on a London Street by a poisoned umbrella pellet in Sept 1978. Odd murders by security services are not new.

The World Wide Web would not be invented until 1989 and have little relevance until the late 1990s. There were 16 million users world wide in 1995 when I was a military officer on an Exchange post in the USA. Now. there are over 4 Billion. The Internet of course existed in Military and scientific fields before that.

Writing this is has reminded me how much has changed personally and how little has changed world wide. Yes, names have changed and regimes have changed, but global politics is as messy as ever. We still have Middle East conflict, US Presidential politics, and rows about Europe. We can add in the mix a rising China and risk of trade wars, climate changes, and the risks from population growth. Despite the current issues it still feels a better world than watching uncollected rubbish pile on the streets whilst walking to work because of yet another strike.

One Morning In The Office Take 7

One Morning in The Office – a satire @realdonaldtrump

"Welcome back Mr President."
"Good to be back, Air Force One's bed is too small."
"Compared to?"
"All my other beds and the hotels, even the UK Ambassador's residence was bigger."
"Do yo want me to get a bigger bed in Air Force One?"
"I've already told them to do it. The idiots claim they'll have to redesign the 747 or use that big Airbus thing they fly wings in"
"That would please the Europeans but haven't we imposed tariffs on them?"
"Yeah, still if they want me to have it they can pay for it like the wall."
"Sir, the Mexicans aren't paying for the wall. In fact, no one is paying for the wall."
"We're getting if for free?"
"No Sir. We have claimed to have built the walls that were put up in 2006 and California has added a few miles to the fences they put up years ago..."
"Sir!" "Sorry I dozed off. Jet lag."
"Sir, what I wanted to ask is, what statement do you want me to put out about your meeting with Vlad."
"Did you check the thes, thees, tess, that dictionary thing?"
"The Thesaurus? Yes Sir.
"Sir, would, would not, could, could not, should and should not, all have different meanings."
"You're sure?"
"I have it here if you want to read it?"
"Not fake news?"
"No, Sir."
"What about wouldn't and would?"
"They are opposites Sir."
"Damn, what now?"
"You'll have to apologise Sir or say you made a mistake."
"Out of context?"
"It's recorded Sir."
"Damn.... Still I can re-tweet something else afterwards, get the press running around in circles again."
"Even Fox were concerned."
"Give them another interview like we did with that British paper, that went well."
"You mean directly contradict what you said on audio and video after the event."
"Just like with Vlad. Anything from him?"
"Only the note about the extra service in the Helsinki hotel."
"What extra services?"
"The same as the Moscow hotel, it's on your preference list for that chain. The Secret Service would not let them in."
"Who told them to do that. I needed some relaxation these summits are hard."
"Remember Sir it was not a summit just a meeting to improve relations."
"Great success."
"With whom Sir?"
"All my meetings were a success."
"NATO, the UK, Germany, the EU?"
"All went perfectly. They all agreed to do what I told them."
"I'm not sure they think that."
"They don't matter. I've won the war in Syria, kicked out those Irises..."
"ISIS, Sir, Irises are flowers."
"You sure? Anyway, wall is going up, got the tax cuts done my poll ratings are up..."
"Actually they are not up."
"...Economy is great I did it all."
"Of course you did Sir."
"Who says different?"
"No one would dare Sir."
"You been watching that fake news again."
"No Sir. I read the Intelligence service brief and the report from the Federal Reserve. Mr Mueller is still investigating."
"Fake News, Fake News, Fake News. Vlad and Xi are happy."
"Actually Xi is not happy about the new sanctions."
"Why not." "He wants a bigger cut. So does Kim."
"Nice fella', A bit short. No dice, it's my deal take it or leave it. I got NATO to increase spending by buying American weapons."
"I don't think they actually agreed to that."
"I did. I tweeted that so it must be true."
"Yes Sir. There's another problem with one of your former er... acquaintances."
"Which one?"
"Karen McDougal, she was a Playboy Model."
"Which one?"
"Sir? Cohen had a tape of you and he discussing a payment to her?"
"A tape?"
"Yes Sir and the FBI have it now."
"Check that Thesaurus thing again on would and wouldn't."
"Yes Sir, anything else? I have to go and testify to the Grand Jury again."
"Yeah, more important things, can't Boeing build a bigger bed?" "Sir????" 

The Modern Dictator?

Human beings seem to have a fascination with the strong man form of government. Sometimes this is masked by the appearance of democratic election but in most cases this gives way to pure dictatorship. We seem to be going through a period of such leaders now.

In the 1930s many of the great powers of the world had dictatorial leaders. In the intervening period the form of government has continued albeit the leaders have tended to be of smaller countries in terms of world power. That is not to underestimate the damage they have caused to their own countries and their neighbours. Although Putin appears below his leadership has changed a virtually collapsed Russia back into a global power

Now we appear to have entered an era of great dictators once more and a worrying trend in the behaviour of others. Turkish democracy on the surface appears robust with a near 90% turnout in the recent election. But the result of the election was to allow President Erdoğan to claim additional powers.

Combine this with further limitations on opposition leaders, the free press and total biased control of the state media and you have the trappings of a one party, one leader state. Throw in the hate for a minority group (the Kurds) that a German citizen in 1938 might recognise. Including direct military intervention across an international border.

At the start of WWII the dictators appeared to support each other Mussolini, Franco, Stalin and Hitler in Europe with Hirohito in Japan.  The USA was isolationist and the Smoot-Hawley Act not only worsened and extended the Great Depression in the USA it also angered its nearest neighbours and allies. Another lesson from history seemingly ignored by the current US President.  I am not claiming that the current 45th President of the United States of America is a fascist dictator he just seems to prefer their company and seems unconcerned with angering his allies.

There seems to be several common traits on the path to dictatorship.

  1. Limit and ridicule free press
  2. Ridicule then imprison opposition
  3. Appoint cronies to all independent roles
  4. Blame outsiders (Terrorists, immigrants, minorities, trade, stealing jobs)
  5. Take more powers centrally to help control the situation
  6. Appeal to lowest common denominator in electoral support
  7. Rig election
  8. Take more powers centrally
  9. Remove opposition and free press completely (Assassination, imprisonment)
  10. Use military force against outside opponents or threaten such
  11. Award oneself numerous medals and awards