Hello everyone,
I am Nitin yadav(KD) back again with another write-up
Introduction to OSINT
OSINT can be summarized as open-source information and tools to gain intelligence about a target or target system. OSINT can be used to collect information from online sources, such as public websites, blogs, forums, and social media sites, as well as electronic communications in transit and at rest. OSINT can be used to collect information about a target’s political, economic, social, and military affiliations, as well as any other imaginable information.
OSINT is valuable for both the intelligence community and the general public. The intelligence community uses OSINT to collect information about foreign governments and their activities
Importance of OSINT in dark web investigations
As the world becomes increasingly connected, cybercrime and espionage have become increasingly common. These crimes can take many different forms, from theft of intellectual property to sabotage of critical infrastructure.
One of the most important tools law enforcement has for investigating crimes and prosecuting those responsible is known as “open-source intelligence” or “OSINT”. OSINT is the collection, analysis, interpretation, and dissemination of information that can be found in public sources.
One of the most important sources of OSINT information is the “dark web”. The dark web is a collection of websites and networks that are not indexed by search engines or publicly available.
When it comes to conducting OSINT on the dark web, there are a variety of methods that can be used. Some of the more popular methods include:
1. Searching for known dark web addresses and content
2. Conducting brute force attacks against Dark Web servers
3. Scanning public Dark Web search engines for mentions of specific keywords or subjects
4. Scouring dark web forums and communities for information
5. Mining dark weblogs for information
6. Analyzing dark web traffic for suspicious activity
7. Conducting vulnerability scans against Dark Web applications
8. Collecting intelligence on dark web users and groups
9. Tracing dark
But wait here is a bot that can help you to minimize your workload
The Tool: Tor Bot
The tool is written in python. The main objective of this project is to collect open data from the deep web/dark web and with the help of data mining algorithms, collect as much information as possible and produce an interactive tree graph. The interactive tree graph module will be able to display the relations of the collected intelligence data.
Features of the bot:
Onion Crawler (.onion). (Completed)
Returns Page title and address with a short description of the site. (Partially Completed)
Save links to the database. (PR to be reviewed)
Get emails from the site. (Completed)
Save crawl info to JSON file.(Completed)
Crawl custom domains.(Completed)
Check if the link is live.(Completed)
Built-in Updater.(Completed)
TorBot GUI (In progress)
Social Media integration. (Not Started) ...(will be updated)
OS Dependencies:
Tor
Python 3.7
Golang 1.16
Python Dependencies:
beautifulsoup4
pyinstaller
PySocks
termcolor
requests
requests_mock
yattag
numpy
Golang Dependencies:
https://github.com/KingAkeem/gotor (This service needs to be run in tandem with TorBot)
Setup:
Before you run the torBot make sure the following things are done properly:
Run tor service
Make sure that your torrc is configured to SOCKS_PORT localhost:9050
Install Poetry
Disable Poetry virtualenvs (not required)
Install TorBot Python requirements
On Linux platforms, you can make an executable for TorBot by using the install.sh script. You will need to give the script the correct permissions using
Now you can run
to create the torBot binary. Run
to execute the program.
Using Docker
Ensure that you have a tor container running on port 9050.
Build the image using the following command (in the root directory):
Run the container (make sure to link the tor container as tor):
TO-DO:
Visualization Module
Implement BFS Search for webcrawler
Use Golang service for concurrent webcrawling
Improve stability (Handle errors gracefully, expand test coverage and etc.)
Create a user-friendly GUI
Randomize Tor Connection (Random Header and Identity)
Keyword/Phrase search
Social Media Integration
Increase anonymity
Increase efficiency
Repo: Click Here
I hope you enjoyed this one and I see you next time :)
Take care and stay safe!
Comentarios