Using SpiderFoot for Offensive Reconnaissance: Part 1 –… | Intel471 Skip to content
blog article

Using SpiderFoot for Offensive Reconnaissance: Part 1 – Discovery

Jun 29, 2022
Detective with magnifying glass 100735058 large

My name is Jude, I’m a Security Analyst working with a Security Operations Center (SOC) team based in Australia. Outside of the usual SOC tasks (alert management, detection use cases), I work on specialist engagements around analyzing and understanding the external attack surface of companies using OSINT investigation and Dark Web Monitoring. One of the tools I use most is SpiderFoot, because it allows us to integrate a number of sources into one easy platform to work from and protect clients from vulnerabilities and data exposure on the web.

SpiderFoot is often used as a diverse Attack Surface Management (ASM) tool, or a way to collate information for OSINT investigations from numerous sources, but within this blog I’m going to dive into how I used SpiderFoot to discover and enumerate hosts as part of an attack strategy. Of course, this will be using a public bug bounty program for Endless Group listed on HackerOne (don’t try it without this kind of strict prior permission!).

I chose this company because they provided a specific list of targets on their bug bounty page, and when it comes to SpiderFoot this makes things a bit easier to hit the ground running. And luckily, this company only had a handful of domains and a single CIDR range:

  • theendlessweb[.]com
  • itsendless[.]org
  • endlessgroup[.]org

After letting the automated scan run to completion roughly five hours later, we have plenty to work with to start discovering potential vulnerabilities to exploit:

A summary of results from the scan

Exploring the High Correlations

Right away we can see six distinct databases found that are exposed to the Internet. Exposed databases can be a slippery slope to a data breach if the software used is out of date and/or has vulnerabilities that are easily exploited. The key thing to note here is that only one of these databases is on the CIDR range noted on the HackerOne listing. Testing anything against these other databases may be a breach of the Rules of Engagement (ROE) here. Nonetheless, for the purposes of this blog post (and attack surface management in general) we can still explore these and see if they’re noteworthy at all.

Some of the HIGH correlations from the scan

Moving along, there are some malicious entity detections, which from the perspective of brand integrity that would normally come with a typical OSINT/Dark Web monitoring service that I have engaged in before, this can be useful. In this case, there likely isn’t much there to work with, and again, only one of these detections is on that CIDR range. The final high correlation detections indicate that two of the domains may be stale or outdated, these detections are backed up by some SHODAN based CVE detections for vulnerabilities dating back to 2013. This can be useful information to note when validating vulnerabilities.

Diving into the Software Used and Vulnerabilities

Looking into the ‘Software Used’ data type, none of the data elements shown below are within the CIDR range, so we’ll put that aside for now. We’re looking to learn more about any infrastructure directly related to those IOCs and potential vulnerabilities that may come with them.

Sorting the data by “Software Used”

Starting with the High CVEs, we can see… that there are none. Starting with the Medium CVEs then, we can see a number of vulnerabilities relating to specified hosts, and quite a few out of scope. SpiderFoot is quite thorough, and has even found some alleged vulnerabilities with a score of 4.3 with a Google APIs link:

Some medium severity CVEs discovered (old versions of JQuery)

A number of these vulnerabilities are quite mild, and require some complex exploits, such as the ones relating to CVE-2013-3587 and CVE-2011-3389, HTML and SSL respectively. The HTML vulnerability allows potential man-in-the-middle (MITM) attacks, as it does not properly obfuscate the length of unencrypted data and allows MITM attacks to obtain plaintext secrets by observing length differences. This is guesswork, and encryption deduction can be a lengthy process. The SSL vulnerability is more complex, and we’ll go into depth on it in the next part of this blog.

The subdomain mail.theendlessweb[.]com has a medium vulnerability CVE-2018-15473 with a CVSS score of 5.0. The vulnerability relates to OpenSSH version 7.7, whereby the software is prone to user enumeration. Two more vulnerabilities with the same score were detected in OpenSSH, CVE-2017-15906 and CVE-2018-15919, the latter being a similar user enumeration bug again, and the former being a bug in sftp-server.c that allows read-only users to create files of length zero.

OpenSSH vulnerabilities will be commonly found, especially by the SHODAN module in SpiderFoot as many webservers and domains will have port 22 open and can be enumerated. In this case, hopefully Endless Group have locked it down to specific access groups and addresses, but that remains to be seen. That being said, some of the latter vulnerabilities may also fall out of scope as they refer to IPs not in the CIDR range of the bounty program.

The last few we’ll talk about that were found were the 4.3 severity XSS vulnerabilities CVE-2018-14040 and CVE-2018-14042, which are bugs with Bootstrap found on the mail subdomain that allow XSS in the data-container property of tooltips, and some other associated vulnerabilities CVE-2019-8331 which describes a similar bug, and finally CVE-2020-11023 which describes a jQuery vulnerability that allows untrusted code to be executed using HTML <option> elements even if sanitized, by way of jQuery’s DOM manipulation methods. These vulnerabilities are quite interesting because are already a lot more feasible than the crypto vulnerabilities mentioned (and the others I didn’t mention in the interest of succinctness), and pathways to exploitation can already be seen if they are indeed actively exploitable.

Other Detections and their Significance

This SpiderFoot scan has found a few other interesting items, including a number of “Leak Site Detections” from the Intelligence X module, and some “Compromised Passwords” found from the TruffleHog tool. The leaked site data found .csv files in .zip dumps that appear to list the site along with page numbers, bandwidth data, and traffic numbers. Relatively benign and probably not useful here, but an interesting find from SpiderFoot nonetheless.

Leak site data discovered by the Intelligence X module

The compromised passwords data elements found appear to be from GitHub repositories, and while there are some interesting pieces of data showing default credentials on Elasticsearch systems, the repositories are not in any way related to Endless as far as I can tell, which makes these data elements false positives. Again, SpiderFoot is so thorough that when you’re working to a list of specific targets, it is bound to overshoot somewhat.

Some compromised API keys discovered appear to be false positives


In this blog post, we have gone through the process of using SpiderFoot’s OSINT gathering capabilities to perform reconnaissance on an organization with the intention of discovering information that may be used to discover vulnerabilities in their external attack surface. In the next blog post, we will cover part 2 of this journey, where we’ll go into more depth validating what has been found, stay tuned!