Overview
SpiderFoot automates open-source intelligence collection and links findings across supported modules. It is useful for broad reconnaissance, but collected data should be validated and handled with privacy and scope considerations.
Key features
Modular OSINT collection; web UI; multiple source modules; exportable findings.
Install & first run
# From source (Python 3)
git clone https://github.com/smicallef/spiderfoot.git
cd spiderfoot
python3 -m pip install -r requirements.txt
python3 sf.py -l 127.0.0.1:5001
# Docker
docker run -p 5001:5001 smicallef/spiderfoot
# UI: http://127.0.0.1:5001
Start with a domain or IP in "New Scan", pick the modules you want, then read the graph. Most passive modules need no API key; a few improve with one. Docs: https://www.spiderfoot.net/documentation/
Real-world example
Use a carefully scoped scan to organize public information about your own domain and identify items that need manual validation.
Common questions
Is SpiderFoot fully passive?
Behavior depends on enabled modules and data sources; review module behavior before a run.
How many modules does it have?
Over 200, split into passive (third-party data) and active (direct to target) sets. Passive modules need no API key; the key-based ones add depth.
What is the correlator for?
It cross-links findings from different modules, which is how a domain turns into an organisation, then into contacts. Without it, you just get rows.
Can I run a scan from the CLI?
Yes: sf.py -s example.com -t DOMAIN_NAME -m all -o tab. Useful for scheduled recon; the UI is for exploring results.
Related tools
dnsx
Resolve and inspect DNS records at scale in a pipeline: A/AAAA/CNAME lookups, wildcard filtering and custom resolvers. Free, open source, Go.
httpx
Turn a host list into live HTTP services: status codes, titles, TLS details, headers and detected tech in one fast pass. Free, Go CLI.
Katana
Crawl web applications for URLs and endpoints, parsing JavaScript and known files so you see the routes a user can really reach. Free, Go CLI.
Also in the OSINT Hub: this tool is documented in more depth at /osint/spiderfoot — that page is the canonical one.
Keep exploring: browse every assessment phase in the Pentest Hub.
Authorized testing only — only ever test systems you own or have explicit written permission to assess.
