Search K
Appearance
Appearance
Other ways to support HackTricks:
# You can check if the tool is working with
prips 1.0.0.0/30 | hakoriginfinder -h one.one.one.one
# If you know the company is using AWS you could use the previous tool to search the
## web page inside the EC2 IPs
DOMAIN=something.com
WIDE_REGION=us
for ir in `curl https://ip-ranges.amazonaws.com/ip-ranges.json | jq -r '.prefixes[] | select(.service=="EC2") | select(.region|test("^us")) | .ip_prefix'`; do
echo "Checking $ir"
prips $ir | hakoriginfinder -h "$DOMAIN"
done
Note that even if this was done for AWS machines, it could be done for any other cloud provider.
For a better description of this process check:
# Find open ports
sudo masscan --max-rate 10000 -p80,443 $(curl -s https://ip-ranges.amazonaws.com/ip-ranges.json | jq -r '.prefixes[] | select(.service=="EC2") | .ip_prefix' | tr '\n' ' ') | grep "open" > all_open.txt
# Format results
cat all_open.txt | sed 's,.*port \(.*\)/tcp on \(.*\),\2:\1,' | tr -d " " > all_open_formated.txt
# Search actual web pages
httpx -silent -threads 200 -l all_open_formated.txt -random-agent -follow-redirects -json -no-color -o webs.json
# Format web results and remove eternal redirects
cat webs.json | jq -r "select((.failed==false) and (.chain_status_codes | length) < 9) | .url" | sort -u > aws_webs.json
# Search via Host header
httpx -json -no-color -list aws_webs.json -header Host: cloudflare.malwareworld.com -threads 250 -random-agent -follow-redirects -o web_checks.json
This mechanism relies on client SSL certificates to authenticate connections between Cloudflareโs reverse-proxy servers and the origin server, which is called mTLS.
Instead of configuring it's own certificate, customers can simple use Cloudflareโs certificate to allow any connection from Cloudflare, regardless of the tenant.
โ
Therefore, an attacker could just set a domain in Cloudflare using Cloudflare's certificate and point it to the victim domain IP address. This way, setting his domain completely unprotected, Cloudflare won't protect the requests sent.
More info here.
This will reject connections that do not originate from Cloudflareโs IP address ranges. This is also vulnerable to the previous setup where an attacker just point his own domain in Cloudflare to the victims IP address and attack it.
More info here.
Sometimes you just want to bypass Cloudflare to only scrape the web page. There are some options for this:
https://webcache.googleusercontent.com/search?q=cache:https://www.petsathome.com/shop/en/pets/dog
Some tools like the following ones can bypass (or were able to bypass) Cloudflare's protection against scraping:
There have been a number of Cloudflare solvers developed:
Use a headless browser that isn't deetcted as an automated browser (you might need to customize it for that). Some options are:
Smart proxies proxies are continuously updated by specialized companies, aiming to outmaneuver Cloudflare's security measures (as thats their business).
Som of them are:
For those seeking an optimized solution, the ScrapeOps Proxy Aggregator stands out. This service integrates over 20 proxy providers into a single API, automatically selecting the best and most cost-effective proxy for your target domains, thus offering a superior option for navigating Cloudflare's defenses.
Reverse engineering Cloudflare's anti-bot measures is a tactic used by smart proxy providers, suitable for extensive web scraping without the high cost of running many headless browsers.
Advantages: This method allows for the creation of an extremely efficient bypass that specifically targets Cloudflare's checks, ideal for large-scale operations.
Disadvantages: The downside is the complexity involved in understanding and deceiving Cloudflare's deliberately obscure anti-bot system, requiring ongoing effort to test different strategies and update the bypass as Cloudflare enhances its protections.
Find more info about how to do this in the original article.
Other ways to support HackTricks: