Published: 13 July 2015 My last blog article was about that fine line between what is allowed content based on the age of the viewer and how they intend to use that information. This article is about stuff that is out right illegal in most countries.

There are a few posts around at the moment that discuss the “dark web”. It’s the 95% of content that Google, Bing and Yahoo don’t bring back.

There are several reasons for content not being included such as:

Dynamic content (multiple pages that all exist under the same url)

Unlinked content (not linked from any other site on the net)

Private web (password protected)

Contextual web (You can only access certain areas based on past choices on other pages - http context driven)

Limited access content (Blocked using robot.txt, captacha etc)

Scripted content (generated via javacript and the like)

Non html content (video, stored in ways not spidered by traditional methods)

So you can imagine the huge amount of stuff floating around in that 95% of the internet you never see. Usually that’s to the determent of the website provider who wants to make money from it in some way. But some websites do this deliberately, to prevent police and other government officials from finding them.

The various law enforcement officials are trying to start hitting back. Some using some rather questionable ethics.

“IT WASN’T EVER seriously in doubt, but the FBI yesterday acknowledged that it secretly took control of Freedom Hosting last July, days before the servers of the largest provider of ultra-anonymous hosting were found to be serving custom malware designed to identify visitors.” -

But when you have sites like the Silk Road which allow purchasing of drugs or a hit on a person anonymously across the internet. Paid for in Bitcoins which are untraceable you can understand why things are getting a bit desperate.

That said don’t lose all hope, I recently heard the news that law enforcement are rolling out a system called “Memex deep web search engine”.

It scans the deep web looking for illegal content. It won’t be a magic bullet but it goes a way to start combating the issues the internet has fostered.

Richard Brisley

Lead Server-Side Developer


Page Name: {% PageName %}

Page Template: {% PageTemplate %}

CampaignID: {% AgentReferrer.ID %}

CampaignName: {% AgentReferrer.Name %}

CampaignPhone: {% AgentReferrer.Phone %}

Item Location: {% PageLocation %}

Search Session Exists: False