How to Collect OSINT on the Dark Web

At LifeRaft, we know from talking to hundreds of corporate security professionals that many organizations fail to collect open-source intelligence, or OSINT, on the dark web. 

But this could be a big oversight. 

In recent years, the dark web has emerged as a hub for criminal activity. And without regular monitoring, security teams could overlook real threats to their business. 

For OSINT researchers specifically, accessing these encrypted private networks represents a key part of their tradecraft. 

Sources may lead investigations away from the clear net to the dark web. Rather than leave those bread crumbs unfollowed, analysts should learn how to navigate this corner of the net.

But where do beginner researchers get started? 

In this post, we’ll define what the dark web is, what kind of threats exist in this community, and how to start collecting OSINT on the dark web itself.

Learn More: 5 Cognitive Biases That Could Affect Your OSINT Investigations

What is the Dark Web?

Simply put, the dark web refers to encrypted online content unindexed by standard search engines. Additionally, users can only access dark web content through specialized services, such as the Tor browser. 

First devised by the U.S. Navy in 2002, the dark web served as a way for agents to communicate freely online without detection. To obscure their activities, the military opened the service to the public. And today, the dark web allows anyone to communicate and conduct business online with more anonymity. 

Such anonymity serves many functions. 

For starters, news outlets, such as the BBC and The New York Times, publish mirrors of their websites on the dark web ⁠— thereby allowing visitors in tyrannical countries to get around government censors. The dark web also represents a secure means of communication for activists, journalists, and whistleblowers. 

But as you might expect, such anonymity attracts a swath of shady characters. 

Dark web forums serve as hubs for criminal activities. Popular products for sale include drugs, illegal pornography, stolen goods, leaked credit card numbers, pirated content, hacked passwords, and company trade secrets. 

Users also exploit the dark web to dox or plan violent attacks against individuals, such as executives, celebrities, and other VIPs. 

Leaked personal information of a senior executive on the darkweb, including wi-fi passwords, home address, and family member details, discovered by Navigator.

Although the dark web accounts for only a tiny fraction of internet traffic, it should represent a major piece of any open-source intelligence program. 

Without monitoring these networks continuously, sensitive information could lead to actionable threats against your organization. And one-off scans may not be sufficient to mitigate potential threats. 

The dark web itself consists of a collection of different services, nicknamed darknets. Researchers will access particular networks depending on what they want to find. 

Here is a quick overview:

• Tor: Tor, short for The Onion Router, ranks as the largest and most popular darknet service. The browser allows users to communicate anonymously by nesting layers of encryption to internet traffic – which resembles layers of an onion. Users exploit the service for a variety of reasons, from hosting criminal marketplaces to spreading censored information in tyrannical countries. 
  
  
• ZeroNet: ZeroNet is a distributed, peer-to-peer web hosting service. This means a website uploaded on ZeroNet isn’t hosted on a central server. Instead, information is hosted by and retrieved from the thousands of other users on the network. The advantage of this model is that it makes it impossible to take down any particular website since information is distributed across everyone in the network. In the event authorities take a site offline, the requester can always find more peers to retrieve the page. 
  
  
• I2P: I2P, short for the Invisible Internet Project, allows for censorship-resistant, peer-to-peer communication. The service archives anonymous connections by encrypting the user's traffic and sending information through a volunteer-run network that consists of thousands of computers. This makes it almost impossible for a third party to watch an entire message during transit due to the high number of possible pathways. As a result of this anonymity, I2P has become a popular service for cybercriminals. 
  
  
• Freenet: Freenet is another decentralized, peer-to-peer network that aims to avoid censorship. Similar to Zeronet, content is hosted across thousands of different computers with no central server. So even if authorities take down one node in the network, users can access content from another. While originally designed to help dissidents avoid censorship laws, today Freenet ranks as one of the more popular services for cybercriminals to deliver illegal and malicious content to customers. 

Because Tor represents the largest darknet service, we will focus on this router for the remainder of this article. 

How to Access the Dark Web

You cannot access dark web content on standard browsers like Chrome, Firefox, or Safari. To access the Tor network, you will need to download the Tor browser. This can be done by visiting the official Tor Project website.

A screenshot of  the Tor browser

Accessing TOR from your standard machine, however, comes with several security risks.

Dark website hosts often employ sophisticated counter-surveillance measures. This can reveal the identities of visitors, potentially compromising investigations or exposing analysts to retaliatory attacks. Automated scripts can also inject malware onto your computer. 

For these reasons, it is generally recommended users navigate the dark web through a configured cloud or local virtual machine. At the very least, users should use a dedicated laptop or computer used exclusively for research purposes. 

Additionally, your internet service provider (ISP) can detect when Tor is being used. So if you want to browse with Tor privately, it makes sense to connect through a VPN. This will ensure your ISP will only see you connected to an encrypted tunnel on a VPN server and not a Tor node. 

Technology experts have many opinions on which measures represent the most secure method for connecting to the dark web. So analysts should consider the policies, budgets, and security requirements of their organization when weighing different options. 

If your organization has the budget, however, it often pays to invest in a managed attribution service. Partnering with these vendors has several advantages, such as:

• Isolate your online activities: A managed attribution service allows investigators to browse the dark web from their daily workstation while connected to a secure, cloud-based system. This ensures web code never reaches the endpoint terminal, keeping your device and your organization’s network safe from malware. 
  
  
• Customize your online appearance: Accessing the dark web through a VPN will disguise the geographic location your device appears from. But VPNs still leak other critical aspects of your online identity, such as language, keyboard settings, time zones, and other details – all of which could potentially tip off targets that they are subject to an investigation. A managed attribution service allows investigators to disguise all aspects of their online identity to better blend in with normal internet traffic. 
  
  
• Reset browsing history: At the end of each session, a managed attribution service will automatically clear all cookies and tracking data from your device. By starting a fresh session each time you log onto the dark web, you reduce attribution risk.

How to Navigate the Dark Web

Sites hosted on the Tor service carry the .onion top-level domain suffix. Standard search engines, such as Google and Bing, however, do not index such domains. 

As a result, finding relevant content on the dark web can be tough. That said, you can find a handful of resources that make navigation easier:

• The Hidden Wiki: The Hidden Wiki represents a great starting point for first-time dark web visitors. The site contains links to many different pages, from blogs and chan boards to criminal forums and media mirrors. But be sure to verify any links before clicking on them. Many sites listed on the Hidden Wiki are fakes, scams, or other cons. 
  
  
• Hunchly: Hunchly offers a free email service called the Daily Darkweb Report. Each day, subscribers receive a spreadsheet detailing new Tor websites. Admittedly, the service does not organize content as nicely as the Hidden Wiki. That said, Hunchly provides a far more extensive list of dark web sites to explore. 
  
  
• Ahmia.fi: Ahmia.fi ranks as one of the most popular dark web search tools. Available on the clear web, the search engine quickly delivers onion addresses containing the user’s queried terms. But be sure to verify any links. While Ahmia filters out sites containing child abuse images, you will still find links to other adult content. 
  
  
• Dark.fail: Operated by an anonymous journalist, Dark.Fail represents one of the best ways to find active and (perhaps more importantly) legitimate onion links. The site’s founder curates dark web pages, removing domains with questionable motives such as phishing attempts, spreading malware, and other scams.
  
  
• H-Index: H-Index is a dark web site that provides a list of onion links it has indexed alongside relevant information, including title, language, and URL.
  
  

Staying Safe When Collecting OSINT on the Dark Web

The Tor browser comes with a swath of encryption and privacy features. But given the nature of the dark web, many users opt for extra security measures to protect their identity.

Here are a few suggestions to stay safe when collecting OSINT on the dark web:

• Use Tor with a VPN: Accessing the dark web isn’t illegal. That said, Tor server nodes are public and well mapped. So your internet service provider will know if you’re using the onion router — potentially attracting unwanted attention from law enforcement. For that reason, it always makes sense to shield your identity by using Tor through a VPN service. 
  
  
• Verify links: Because criminal activity represents a large percentage of dark web traffic, many onion sites are fake or have some other dubious motive. Verify any onion links with a trusted friend or colleague before clicking on them. If this isn’t possible, try to validate links through multiple sources. 
  
  
• Check your organization’s policies and procedures: Many organizations prohibit employees from conducting OSINT on the dark web. Be sure to check your company’s policies before starting any dark web investigations. 
  
  
• Double-check URLs: Onion web addresses consist of long, complex character strings. And one mistyped letter or number could send you to an illicit or dangerous website. Always double-check URLs entered into the search bar. It also makes sense to save correct addresses in a filing system or other research tool for easy access later.
  
  
• Disable Javascript: Suspicious webpages can exploit Javascript to insert malicious code into your browser. Certain cookies also use javascript to monitor users, potentially tipping off adversaries of your identity. For these reasons, be sure to disable javascript on your browser. 
  
  
• Avoid downloading files: The anonymity of the dark web makes it hard to verify anyone's identity. That’s why it makes sense to avoid downloading any files you may come across. 
  
  
• Close all apps: Be sure to close all applications, such as streaming services, banking apps, etc, before starting a new session. Also be sure to cover your webcam with duct tape and deactivate any listening devices on your system.
  
  
• Use a privacy-focused operating system: Consider using a privacy-focused operating system, such as TAILS (The Amnesic Incognito Live System). These can provide further protection against online surveillance and tracking. 
  
  
• Never give out personal information: Never reveal personal information about yourself or your organization. Additionally, never reuse usernames or avatars that may connect your dark web accounts with online services you use elsewhere. 

The Bottom Line on the Dark Web

The dark web represents a valuable, but often overlooked, source of open-source intelligence. Today’s security teams need to be aware of the risks that lurk in this corner of the internet. That will go a long way toward keeping your people, customers, and property safe.