Top 5 OSINT Sources for People Investigations
Oct 24, 2021
Probably the most frequently asked question we get from SpiderFoot users is “with so many options available, what API keys should I get for my use case?” So, we asked hakluke and dccybersec to go on a mission and figure out the top 5 for the three most common SpiderFoot use cases: Penetration Tests / Bug Bounties, Threat Intelligence, and People Investigations. This is the third post in the three part series focusing on People Investigations, and we hope you find it useful!
Keep in mind that all references to the pricing of these services were valid at the time of writing but are likely to have changed, so always visit the website of the service to get the latest pricing information.
If you’re one of the thousands of people getting started in Information Security, Cyber Security or Private Investigation, you have probably already heard the term “Open Source Intelligence” (OSINT). OSINT is data collected on an individual or organisation from publicly available sources to be used on both sides of the cyber coin; good and bad. OSINT in practice can be used in a very wide range of use-cases, however it is more prominent in the cyber security and digital investigation world with practices like network infrastructure foot-printing, threat & malware analysis, and law enforcement investigations.
This article is focused on performing OSINT on people, that is, finding information about people. As such we will cover the top 5 data sources for accurate and precise information. Being able to verify that the information you are gathering is accurate and precise is of utmost importance.
A note on ethics and the law
If the title of this blog post didn’t ring any alarm bells for you, then it should. Performing OSINT on people can uncover personal information, in some cases information the person did not wish to reveal. Some data may also be protected under regulations like GDPR.
Before you perform OSINT on a person, you need to ensure:
- What you are doing is legal. Consult a lawyer on topics such as whether the OSINT search is legally justified, how you should handle the data, and your obligations within the law if criminal activity is identified.
- The motives for the investigation are ethical. In other words, it may be legal to perform OSINT searches on your ex-girlfriend, but no one will tell you that isn’t creepy.
- If the above two boxes are checked, also consider how to protect yourself online while doing such investigations, since you are leaking more data than you realise.
The types of data that we are most concerned with when performing OSINT on a person tend to be:
- Social media accounts
- Geolocational data
- Phone numbers
- Email addresses
- Facial recognition
Benchmarking the data sources
It was quite difficult to narrow the list down to just five sources as there are so many different data sources out there. Each of them has a unique use-case, and some of them specialise in a very specific type of data, while others generalise. For these reasons it is very difficult to do a direct comparison of each data source. Attempts at comparison via benchmarking simply don’t make sense. As such, this blog post should not be interpreted as a comparison of the services we mention. The reason that we chose these data sources in particular was because they are all leaders in the industry with very comprehensive data in their own right.
Social Links was founded in Amsterdam and emerged into the market through the banking sector, where different departments within banks required tools for conducting fast searches on individuals. The company’s first product, “Social Links Pro” was released and gained immediate traction from various European law enforcement agencies. Since then, they have expanded their API to search across over 500 different data sources, as well as having the ability to perform DarkNet searches.
Social Links has 3 product offerings that all have free trials. The cost after trial of these products are as follows;
Gamayun is a web based tool designed for conducting OSINT investigations when searching for emails, locations, phone number, aliases, and photos. This is Social Links’ base product and does not have API links.
|$33 per month||$99 per month||Contact for pricing|
|1000 Social Link Points||3000 Social Link Points||“Custom” Social Link Points|
|Dedicated Support SLA’s|
SL Pro, as the name suggests, is a professional tool for searching data from social media accounts, the Darknet, blockchains, and internet data leaks. It comes with API Integrations that link to 1100 methods of search across 50+ sources of data on over 800 million identities, which makes it ideal for when performing OSINT on a person. It also has a great visualization feature that allows you to connect entities and build connections between accounts, people, companies, events and activities.
They offer a free trial of the software, which will allow you to connect it up to Maltego and perform searches. There is no information available on how much it will cost after the trial, as this greatly depends on how many search queries you intend on performing.
SL Box works similarly to SL Pro with the difference that it is a custom built machine for you to conduct your investigations from with pre-built machine to work from. It comes with dedicated customer support on the product which is helpful if you’re performing a large amount of investigations. This product is essentially designed as an enterprise product, with a free trial available and no information available on exactly how much it costs, as it’s a custom solution built to your need.
The quality of the data available through Social Links is very good considering the low price point. Most of the data is obtained by parsing various white and yellow pages, company registers, business directories, social media networks and other open online sources into their ever-growing database. Being able to link this data together with their visualization tool is where social links really stands out for us, and allows you to easily link in other data fields like IP Addresses or nicknames which results in a visually beautiful investigation report for you to present to your client.
The API for Social Links is quite well documented, and has functionality with quite a lot of popular social media networks. There are many different available language you can use, such as cURL, Go, Java, and Python. To view the API documentation, browse to the following link and try it out for yourself: https://docs.osint.rest/
Seon began in 2016 with two friends who were interested in cryptocurrencies building a crypto exchange in the CEE region. They found that the service was under constant attack from fraudsters, and while they introduced various fraud prevention solutions, nothing seemed to work. Seon was born shortly afterwards after the duo decided they could build their own anti-fraud system for their crypto exchange service, and soon introduced the anti-fraud system to other crypto platforms as well as high-risk merchants.
Their focus as a service was to develop a system that could easily be integrated in any online point of authentication by combining their integrations with a depth of functionality to overcome fraud-ridden online services.
Seon has two products on offer with separate pricing for each. Both come with a free trial of the full system. Details below;
One-click data enrichment based on a single email, IP, phone, or location. Get a complete picture of your users, directly via a browser extension.
- Billing Mode: Tier Based Pricing
- Payment Cycle: Top-up
- Payment Method: Credit Card
- From 99 Euro per month
The complete end-to-end, fraud fighting platform. Harness the power of Machine Learning through the most sophisticated Admin Panel to gain the upper hand against fraudsters.
- Billing mode: Pay as you use
- Payment Cycle: Post cycle payment
- Payment Method: Bank transfer or credit card
- From 0.06 Euro per check
The quality of the data is great if you’re looking to gain information from social media accounts, an individual’s risk score to see if their account has been used for potentially fraudulent activity, to build a profile on a phone number or to gain information on an IP address. There is also functionality with spam blacklist checking for IP addresses, phone numbers and email addresses, which can be quite useful when conducting OSINT on a person who has previously attempted fraudulent activity.
The API of Seon is accessible from Java, Python, PHP and of course cURL. They have a very in-depth and well written API reference guide (https://docs.seon.io/api-reference#quick-start) to help you integrate your setup with the Seon API. All you need to do is to select your industry, use case and environment language, then it’s a step by step process to pull that data across.
EmailRep, as the name suggests, is a reputation tool used on email addresses created by the team at Sublime Security. While it advertises itself as a simple email reputation tool, it also does so much more than that. EmailRep uses hundreds of factors like the domain age, traffic rankings, presence on social media and other sites, personal connections, public records data, data breaches, dark web credential leaks, and much more.
EmailRep has three pricing options which are;
|Plan Name||Cost||Queries Per Month||Queries per day||Support|
|Commercial||#20 per month||1000||No limit||Email support|
|Enterprise||Custom||“High Volume”||No limit||Support with SLA|
The quality of data obtained from EmailRep is incredibly useful when performing OSINT on an individual. Email reputation is often overlooked when people think about performing an OSINT investigation. The data obtained from scans with EmailRep helps to identify if an account is a fake or if it is legitimate, which helps to then pivot from later and perform further investigation actions.
EmailRep is a system of crawlers and scanners that were built to solve the problem of identifying if email addresses conduct malicious activity and to help identify if high reputation email accounts have been compromised.
EmailRep runs on a restful API service, and allows access with their free account option. The documentation will help to get you started, and includes various models for you to build your queries from as examples. You can view the API documentation here (https://docs.emailrep.io/).
Hunter is a search engine specifically made for finding professional email addresses. They were founded in 2015 and stand by 3 principles, which are Simplicity, Transparency and Leverage, and I believe they have achieved that with their product. They are a small remotely operated group boasting over 2 million customers, as they work together online to give professionals the power to create new connections with the people that matter.
Hunter has 5 different pricing options, which are as follows;
|Cost||$0||$49 per month||$99 per month||$199 per month||$399 per month|
|Searches per month||25||500||2500||10000||30000|
|Verifications per month||50||1000||5000||20000||60000|
The Starter and higher plans also include the ability to perform domain searching, priority support and more.
Hunter is constantly crawling web pages to find business data, and similar to how search engines operate, Hunter keeps an index of the entire web and organises the data so it is easily searchable. This translates to very good data quality across 711,000+ crawled websites each minute, over 75 million website sources for data and over 102 million professional email addresses currently indexed. A major benefit of how this system operates is that if data has been removed from sites previously, Hunter has an archive based system that keeps this data searchable, which is absolutely paramount when performing OSINT on a person.
The functionality of the API in V2 is very well documented, and allows for searches against Domains, Emails, and Authors, while also allowing to verify the email risk on each step. The API is designed to be as simple as possible with a basic structure of data, meta, and errors. Obtaining an API key from Hunter is free, but comes with the limitations listed above in the pricing section.
Similar in a way that Hunter is a search engine for professional emails, OpenCorporates is a search engine for companies. They boast to be the largest open database of companies in the entire world, which allows you to trust, access, analyse and interrogate the data for your own investigative purposes. Their purpose is written as “to ensure that everyone knows exactly who they are working with – and working for. To tackle corruption and criminality. To protect our democracy. To create a trusted business environment we want to work in – and a society we’d all like to live in”. As this purpose suggests, they are as a company advocating for open data and corporate transparency, and for you performing an investigation, the information they gather on companies could be vital to a lead in the right direction.
It is free to perform one-time searches with Hunter, however for access to the API, you will need to purchase a subscription. Subscription plans are as follows;
|Starter API||Basic API||Standard API||Corporate API||Enterprise Plan|
|100 Euro Monthly||350 Euro Monthly||750 Euro Monthly||1400 Euro Monthly||Pricing on demand|
|Internal use only||Internal use only||Internal use only||Internal use only||No share-alike restrictions|
|5000 API calls per month||20000 API calls per month||50000 API calls per month||100000 API calls per month||Over 50000 API calls per month|
|1000 API calls per day||4000 API calls per day||10000 API calls per day||20000 API calls per day||Over 10000 API calls per day|
If your search should require legal entity data, then the data quality obtained through OpenCorporates is absolutely vital to your investigation. They focus on excellent data for Public official and legal entity data through open identifiers directly from the source of where that information is supplied from (usually from court houses or public directories). Because the data is open access, it allows for anomalies, errors, and issues to be discovered very quickly, which in turn, creates better data for everyone. Their tech stack is run to optimise speedy results on data searches, which helps to filter the quality of the data and produce more efficiently collected data than its competitors.
The API is currently running on version 0.4.8 and has very detailed documentation in their reference document (https://api.opencorporates.com/documentation/API-Reference). By default, the API returns data as JSON, however XML is also available. I personally found the information of the API reference guide hard to follow, however that’s personal preference, but once I got the hang of using this API, the information available that you could search for and the speed it returned results was astonishing.
While it is difficult to compare each of these services to each other as they target different use cases, it is possible to determine that running them together and pulling information from each of the API’s into one system to then further identify, research and pivot from would be an ideal use case for anyone performing an OSINT investigation on a person. The amount of accurate data gathered from each service and compiled into a single report would definitely help in your investigation report, as each service has a specialty in the data it obtains. The added bonus is that each of these sources is integrated into SpiderFoot, so you’ll just need to enter the API keys from each service into SpiderFoot in order to bring all their data into one place for analysis.