An anti-bot system is software in front of a website that estimates, for every request, how likely it is to come from automation rather than a person, and acts on that estimate — letting it through, slowing it, challenging it, or blocking it. The estimate is not made from any single fact. It is a score assembled from four layers: where the request comes from (IP), how the connection was made (TLS and HTTP fingerprints), what the browser reveals about itself (JavaScript fingerprint), and what the visitor does over time (behaviour). Understanding the layers explains the two things proxy users find most confusing: why a “perfect” residential IP still gets challenged, and why the same setup works on one site and fails on another.
The vendors, and the products
Most protected sites use one of a handful of products: Cloudflare Bot Management (and its Turnstile challenge), DataDome, Akamai Bot Manager, HUMAN (formerly PerimeterX), Imperva (Incapsula), Kasada, and AWS WAF Bot Control. They differ in emphasis and in how aggressive their defaults are, but they are built on the same four layers, and the layers are what this article is about. Everything below applies to all of them in some proportion.
Layer 1: IP reputation
The first and cheapest check. Before the request is even parsed, the IP is looked up:
- ASN and type — is this a hosting company (datacenter), a consumer ISP (residential), or a mobile carrier? Hosting ASNs start with a penalty on most sites; the reasoning is covered in Residential vs datacenter vs mobile.
- Threat intelligence — is the IP on abuse lists, known as a VPN/proxy exit, or seen attacking other customers of the same vendor? Vendors pool signals across their whole customer base, so an IP burned on one DataDome-protected site is suspect on all of them.
- Velocity — how many requests has this IP made to this site recently, and to how many different paths? A per-IP counter is the oldest bot signal there is and still one of the most effective.
- Geography — does the country match the site's audience? Does it match the language and time zone the browser will later report?
This is the only layer a proxy directly controls. A residential or mobile IP with no history on the target passes it. But passing layer 1 only means the request proceeds to layers 2–4 with a neutral score, not a good one.
Layer 2: network fingerprints (TLS, HTTP/2, TCP)
Before any page content is served, the server has already seen how the connection was opened, and the way it was opened identifies the client software:
- TLS fingerprint (JA3/JA4) — the ClientHello reveals the TLS library: Chrome, Firefox, OpenSSL-in-Python, Go. Explained in TLS fingerprinting.
- HTTP/2 fingerprint — the SETTINGS frame values, initial window size, priority frames, and the order of headers and pseudo-headers. Browsers have characteristic, stable patterns; HTTP libraries have different ones. Akamai's research made this technique famous.
- TCP/IP fingerprint — window size, TTL, options; distinguishes operating systems, and a “Windows” User-Agent from a Linux TCP stack is a mismatch.
The scoring logic here is consistency: does the network-level identity match what the HTTP headers claim? A proxy does not touch these fingerprints — it forwards them — so a script behind a residential IP is identified as a script at this layer regardless of the IP's quality. This is where most “why is my clean proxy blocked?” questions are answered.
Layer 3: browser fingerprint and JavaScript challenges
If the request reaches the point of serving HTML, the vendor's JavaScript runs in the visitor's browser and reports back. Two kinds of thing happen.
Fingerprinting
The script collects the same properties an antidetect browser is built to control: canvas and WebGL hashes, fonts, screen and window dimensions, hardware concurrency, time zone, languages, audio context, WebRTC-visible IPs, plugin lists, and dozens of smaller signals. It also probes for automation artefacts: navigator.webdriver set to true, missing or inconsistent Chrome-specific objects, headless-mode quirks in rendering, an unusual number of certain APIs being overridden (a sign of stealth plugins patching the browser). And it checks consistency: does the JavaScript-reported platform match the TCP fingerprint's OS? Does the time zone match the IP's country? See what an antidetect browser controls.
Challenges
Vendors also issue proof-of-work or proof-of-humanity tasks. Cloudflare's Turnstile and the “checking your browser” interstitial run a JavaScript workload whose result must be posted back; a client that cannot execute JavaScript at all — every plain HTTP library — simply fails and never sees the page. Visible CAPTCHAs are the last resort, shown when the score is bad but not bad enough to block outright. Passing a challenge typically yields a cookie or token valid for a period; automation that clears cookies on every request faces the challenge every time, which is itself a signal.
Layer 4: behaviour
Over a session — and across sessions the vendor can link by fingerprint or IP — the system watches what the visitor does:
- Request patterns: pages per minute, whether images and scripts referenced by a page are fetched (humans' browsers fetch them; naive scrapers do not), whether navigation follows links or jumps straight to deep URLs, whether the same paths are hit in the same order every session.
- Timing: uniform intervals between requests are machine-like; humans are bursty and irregular.
- Interaction: mouse movement, scroll, key events — their presence, and their shape (real mouse paths curve and overshoot; scripted ones are straight lines or perfect arcs).
- Session shape: a login followed immediately by a bulk action; a checkout with no browsing; account creation from a fingerprint that created five accounts yesterday.
Behaviour is why a technically perfect setup — real browser, matched proxy, clean fingerprint — can still be caught if it acts like a machine. It is also the layer where accumulated reputation lives: an identity that has behaved normally for weeks earns tolerance that a fresh one does not. That is what “warm-up” means in practice.
How the layers combine: the score
Each layer contributes evidence; the product combines it into a bot score (Cloudflare exposes theirs as 1–99; others keep it internal) and the site owner configures thresholds: above X, allow; between X and Y, challenge; below Y, block or serve a decoy. Two properties of this design matter for anyone using proxies:
- The layers must agree. The score is not the sum of independent good marks; contradictions are weighted heavily. A residential IP (good) plus a Python TLS fingerprint (bad) plus a Chrome User-Agent (contradiction) scores worse than a datacenter IP with an honest
python-requestsUA, because the second is at least coherent. - Nothing is checked in isolation. Passing IP reputation buys you entry to the next check, not a pass. Every claim you make — by IP, by handshake, by header, by JavaScript, by behaviour — is cross-examined against every other.
What this means for proxy choice
| Layer | What decides it | Can a proxy fix it? |
|---|---|---|
| 1. IP reputation | IP type, history, velocity, geo | Yes — this is the proxy's whole job |
| 2. Network fingerprints | Your HTTP client / browser | No — forwarded unchanged |
| 3. Browser fingerprint / JS challenge | Your browser and its configuration | No — but geo/time zone must match the proxy |
| 4. Behaviour | Your automation logic and pacing | No — though rotation affects per-IP velocity |
So: choose the proxy type the target's layer-1 policy demands (datacenter where tolerated, residential or mobile where not), and then spend your effort on layers 2–4, which no proxy touches. Practically that means a real or impersonated browser for protected targets, per-identity fingerprint consistency, geo/time zone matched to the IP, and human-shaped pacing. Rotation strategy interacts with layer 1 velocity and layer 4 session shape at once — see choosing a rotation interval.
How to tell which layer is stopping you
- Blocked instantly, no HTML, even from a real browser on the same IP → layer 1: the IP or ASN is the problem. Change proxy type or provider.
- Browser passes, script on the same IP does not → layer 2: TLS/HTTP fingerprint. Move to a real browser or an impersonating client.
- Interstitial that never resolves, or a CAPTCHA on every visit → layer 3: JavaScript challenge failing or fingerprint inconsistent. Check automation flags, WebRTC, time zone vs IP.
- Works for a while, then degrades; or accounts die after actions → layer 4: behaviour or velocity. Slow down, vary patterns, warm up identities.
Vendors return distinctive responses (a Cloudflare 403 page, DataDome's x-datadome headers and CAPTCHA endpoint, Akamai's _abck cookie), so identifying which product a site uses is usually a matter of looking at headers and cookies — and knowing which product tells you which layers it emphasises.
Frequently asked questions
What is a bot score?
A per-request estimate, produced by combining IP reputation, network and browser fingerprints, and behavioural signals, of how likely the request is automated. Site owners set thresholds on it to allow, challenge or block.
Why does a residential proxy still get a CAPTCHA?
Because IP reputation is only one of four layers. If the TLS/HTTP fingerprint says “script,” the browser fails or lacks the JavaScript challenge, or the behaviour is machine-like, the score is poor regardless of the IP.
Do anti-bot systems share data between websites?
Within a vendor, yes: an IP or fingerprint flagged on one customer's site raises its risk on other customers' sites. That is why a proxy pool heavily used against one Cloudflare-protected target may perform badly on unrelated Cloudflare-protected targets.
Can plain HTTP libraries pass JavaScript challenges?
No. Challenges require executing JavaScript in a browser-like environment. Options are a real automated browser, or a solver service that runs the challenge for you and returns the resulting cookie/token — which itself has a cost and detection risk.
What is the difference between a challenge and a CAPTCHA?
A challenge is often invisible — a JavaScript workload or attestation the browser completes automatically. A CAPTCHA is a visible puzzle for the human, shown when the score is bad enough that the system wants explicit proof.
Is it possible to be completely undetectable?
Not as a guarantee. Detection is probabilistic and vendors update constantly. What is achievable is consistency across all four layers plus human-shaped behaviour, which keeps the score in the “allow” range most of the time. Anyone promising undetectability from the proxy alone is selling only layer 1.