Get # Behind the Curtain: Unraveling the Mysteries of GOOGLE's Search Algorithm through an Accidental Leak

In a world where almost all web searches are powered by Google, the inner workings of its algorithm have long been the Holy Grail of search engine optimisation (SEO) consultants and digital marketing companies. A large cache of internal technical documents found its way accidentally to GitHub, the popular code-sharing and web-hosting service, exposing in great detail how Google actually ranks webpages. In a rare insight into the largely secretive workings of the powerful search engine, SEO practitioners were left buzzing, both excited and angry.

The Accidental Leak: A GOOGLE Gaffe

Internal documents for Google’s engineers describing the complex components of its search algorithm were accidentally published to GoogleAPI, Google’s GitHub repository, making both these arrays and a large amount of related API documentation freely available to the public under an Apache 2.0 open source licence. The repository includes API documentation for its ‘ContentWarehouse’, a component virtually identical to the search index, packed with hundreds of modules and attributes that determine the ranking of webpages.

SEO Community: Thrilled yet Skeptical

The SEO world has since been salivating over the leaked papers as they try to reverse-engineer Google’s ranking formulas. The SEO experts Rand Fishkin and Mike King have both been clear about what the leaked documents reveal about Google’s search algorithm, going so far as to call out several insights that contradict statements from Google itself. The fact that Google monitors click-through rates at all, and potentially uses it in some way to determine a given webpage’s rank, is particularly interesting, given previous emphatic denials from Google as to how much attention they pay to click-through rates.

Clicks, Chrome, and Controversy

Digging deeper in these documents, we also find ‘Navboost’, a ranking system that’s supposed to boost results when users click on a website in navigational searches, of which many come from Chrome itself. There are also whitelists for ‘isElectionAuthority’ and ‘isCovidLocalAuthority’ that may indicate to Google that it artificially boosts some sites for topical authority.

The SEO Conundrum: Manipulation vs. Authenticity

The leak re-opened the question of how to weigh the tension between Search Engine Optimisation (literally putting a website’s chances of turning up in a search at the forefront of its content-generation) and ‘realness’. One study found that heavy SEO correlated inversely with the appearance of quality and expertise: that is, too much visible optimisation might make a page look less credible and less high-quality to a user.

GOOGLE's Response: Caution and Clarification

Google has confirmed the validity of the documents, emphasising multiple times that ‘we don’t accept attempts by anyone to game their way to the top of our rankings’, and encouraging caution in ‘jumping to conclusions based on out-of-context, outdated, or incomplete information’. The company also highlights its efforts to be transparent about how Search works, but at the same time to protect it against ‘attempts to artificially inflate rankings through aggressive or manipulative approaches’. Mostly, though, the story demonstrates again the tension between Google and the SEO community, each engaged in a complex dance over whose visibility, and whose legitimacy, wins out in the end.

Implications for Web Users and Creators

While this story might serve as little more than an interesting glimpse behind the scenes of the internet for the average user, for web-masters, content producers and those who obsessively deal with search engine optimisation (SEO) skullduggery, it is a treasure trove of information confirming what the ‘real’ algorithms look like – and how they change all the time. Indeed, the challenge here is to find ways of leveraging this knowledge meaningfully and ethically – learning how to improve the findability of content in terms that don’t trade in fakery and jiggery-pokery.

A Deeper Dive into GOOGLE

Since its founding in 1998 by Larry Page and Sergey Brin, Google has evolved well beyond its breakthrough as a novel search engine into the world’s largest technology company, valuing at more than $250 billion. Its search algorithm – a closely guarded secret – has itself evolved, time and time again, to give its users more and more relevant, useful search results. While the serendipitous leak offers an unprecedented glimpse into the technical guts of Google’s algorithm, it also speaks to the complexity of search engine optimisation, to its constant motion, and to the company’s position as the world’s central information filter. Google is as much a part of the digital future as we are. As the web changes, so will the way it’s navigated, and Google will remain at the helm.

To conclude, the accidental publication of the internal documentation on ranking of Google by the company itself on GitHub, has given unique insights into the Google Search algorithm for the SEO community – and the general public – showing how sophisticated the engineering complex is behind the algorithm and, in spite of the perceived threat to transparency and manipulation, the need to provide valuable content to the user. In short, we may never have the full picture of how the algorithm works, but we do know that the quest for visibility in the digital world is as difficult as it is eternal, and as a result, one we’ll continue to learn from.