This is a guest post by Paul Starrett.
I will not profess to be a dark web guru though I certainly consider myself a well-placed investigator. However, in my profession, the dark web is fast becoming an unavoidable resource. First, some basic fundamentals are in order.
The web as we know it is far different than what we might think. When we run a search on Google, Bing, Yahoo or other search engine, we are typically only “searching” about 5%-15% of data that is actually available on the web. The commercial search engines create what is called an “index” of the web — this index is much like the index in the back of a typical printed book. Rather than searching every word on every page to find what you are looking for in the book, you simply look up the term you are interested in in the index to find the pages on which your term is found. The search engines perform a similar task; they “crawl” websites on the web and create what is the equivalent of an electronic index. When you run a search, you are searching their index and they return links to the websites that are relevant to your query. So, what is in the rest of the web? This is called the deep web.
The reason the search engines do not index the deep web is either because the websites are difficult to “crawl” (e.g. they require a sign in, the webpage is actually a form that needs to be completed or the website does not want to be crawled and places technical protections to avoid crawls of their site, etc.) or because the sites they do not include are not useful to the search-engines’ customers.
Within this deep web is the dark web where anonymity and secrecy are guarded with great effort. The TOR network is one of the many networks on the dark web that facilitates anonymous communication and transfers of data and goods. It uses a system of cryptographically - protected three-hop communication scheme between sender and receiver, in both directions! In fact, “TOR” stands for The Onion Router, where onion refers to the many “layers” of protection that are part of its infrastructure. No one along the communication path can or will know the identity of the other. In this way, completely anonymous transactions and communications may occur. Many use this technology to communicate and express viewpoints to avoid oppression by governments who, otherwise, might prosecute free-form expression.
However, the TOR network (and many other, similar networks) are places where nefarious activity goes on. One can freely shop for drugs, guns, child pornography and even hitmen (though some doubt this latter service actually exists). What makes this community even more difficult to penetrate by law enforcement is the use of crypto-currencies like Bitcoin and even some homespun coins! In this way, completely anonymous and decentralized payment systems can be used for criminal activity (of course, many crypto-currencies are used primarily for completely honest and legal purposes).
Whatever the case, technology changes daily and so must the investigative professional. Challenging and frustrating at times but stimulating and rewarding at others!
Paul Starrett is an attorney and private investigator specializing in the use of data science to assist the legal profession in matters related to investigations, information governance and litigation management. He has five years of experience in C programming with RSA Security and wrote a book for McGraw-Hill on digital signatures. He’s also a member of the Virgil Security Slack channel, where we talk about more than just end-to-end encryption. Learn more at www.starrettconsultinginc.com.
Virgil Security, Inc. is a stack of security libraries and all the necessary infrastructure to enable seamless, end-to-end encryption for any application, platform or device. We guide software developers into the forthcoming security world in which everything will be encrypted (and passwords will be eliminated). In this world, the days of developers having to raise millions of dollars to build secure chat, secure email, secure file-sharing, or a secure anything have come to an end. Now developers can instead focus on building features that give them a competitive market advantage while end-users can enjoy the privacy and security they increasingly demand.