Welcome to the Web Crawler and URL Extractor! This Python script is designed for cybersecurity analysts, bug hunters, and web security enthusiasts. It empowers you to efficiently extract URLs from web pages, supporting both WEB2 and WEB3 technologies. You can even perform keyword-based URL filtering to enhance your web security analysis.
Features
URL Extraction: Extract URLs from web pages effortlessly.
WEB2 and WEB3 Support: Navigate the evolving web landscape with ease.
Keyword Filtering: Filter URLs based on keywords for targeted analysis.
Customizable Depth: Adjust the depth of URL extraction to your needs.
Multi-threaded Processing: Enhance efficiency with multi-threading.
Verbose Mode: Enable detailed logging for insights.
Easy Setup: Get started quickly with straightforward configuration.