Proxies in Python: requests, httpx and aiohttp — Working Snippets and the Mistakes That Break Them

Every Python HTTP library takes a proxy as a URL of the form scheme://user:pass@host:port; what differs is the parameter name, whether it is set per client or per request, and how SOCKS5 and DNS are handled. The snippets below are the exact code that works with current versions of requests, httpx and aiohttp — each was run through an authenticating proxy before this article was published — followed by the three things that account for most “my proxy does not work in Python” questions: wrong parameter names copied from old tutorials, missing SOCKS extras, and a TLS fingerprint that gives the script away regardless of the IP.

The proxy URL

http://user:pass@proxy.example.com:8080     # HTTP proxy, used for both http:// and https:// targets
socks5://user:pass@proxy.example.com:1080   # SOCKS5, DNS resolved locally
socks5h://user:pass@proxy.example.com:1080  # SOCKS5, DNS resolved by the proxy (use this)

Two points that trip people up. First, an HTTP proxy handles HTTPS targets too — the client sends CONNECT and tunnels TLS through it — so you almost never need an https:// proxy scheme; that would mean TLS to the proxy itself, which few providers offer. Second, if your username contains rotation parameters (user-country-de-session-abc123) or your password contains @, : or /, percent-encode them (urllib.parse.quote(pw, safe='')) or the URL will parse wrongly.

requests

import requests

PROXY = "http://user:pass@proxy.example.com:8080"
proxies = {"http": PROXY, "https": PROXY}

# per request
r = requests.get("https://api.ipify.org?format=json", proxies=proxies, timeout=20)
print(r.json())

# per session (reuses connections; the normal way)
s = requests.Session()
s.proxies.update(proxies)
r = s.get("https://api.ipify.org?format=json", timeout=20)

The dictionary is keyed by the target scheme: requests to http:// URLs use the "http" entry, requests to https:// URLs use "https". Set both to the same proxy unless you have a reason not to. Always pass a timeout; a hung proxy otherwise hangs your script forever.

SOCKS5 needs the extra: pip install "requests[socks]". Then proxies = {"http": "socks5h://…", "https": "socks5h://…"}. Without the extra you get InvalidSchema: Missing dependencies for SOCKS support.

Environment variables: requests honours HTTP_PROXY / HTTPS_PROXY / NO_PROXY by default (trust_env=True). If a proxy is mysteriously in use when you did not set one, or not in use when you did, check the environment.

httpx

import httpx

PROXY = "http://user:pass@proxy.example.com:8080"

# sync
with httpx.Client(proxy=PROXY, timeout=20) as client:
    r = client.get("https://api.ipify.org?format=json")
    print(r.json())

# async
import asyncio
async def main():
    async with httpx.AsyncClient(proxy=PROXY, timeout=20) as client:
        r = await client.get("https://api.ipify.org?format=json")
        print(r.json())
asyncio.run(main())

The parameter is proxy= (singular). Older tutorials show proxies={"http://": …, "https://": …}; that keyword was deprecated in httpx 0.26 and removed in 0.28, and passing it now raises TypeError: Client.__init__() got an unexpected keyword argument 'proxies'. If you need different proxies for different targets, use mounts:

mounts = {
    "https://api.example.com": httpx.HTTPTransport(proxy="http://user:pass@dc-proxy:8080"),
    "all://": httpx.HTTPTransport(proxy="http://user:pass@resi-proxy:8080"),
}
client = httpx.Client(mounts=mounts)

SOCKS5: pip install "httpx[socks]", then proxy="socks5://…". httpx resolves hostnames through the SOCKS proxy by default. HTTP/2: pip install "httpx[http2]" and httpx.Client(http2=True, …) — relevant because a proxied browser is on HTTP/2 anyway (see HTTP/2 and HTTP/3 through proxies).

aiohttp

import aiohttp, asyncio

PROXY = "http://user:pass@proxy.example.com:8080"

async def main():
    async with aiohttp.ClientSession() as session:
        async with session.get("https://api.ipify.org?format=json",
                               proxy=PROXY,
                               timeout=aiohttp.ClientTimeout(total=20)) as r:
            print(await r.json())

asyncio.run(main())

The proxy is passed per request (or set once via a custom connector). Credentials go in the URL; the separate proxy_auth=aiohttp.BasicAuth(...) form still works in 3.x but is deprecated as of 3.14 and scheduled for removal in 4.0, so prefer the URL form. aiohttp does not read HTTP_PROXY environment variables unless you create the session with trust_env=True. Native aiohttp speaks HTTP proxies only; for SOCKS use the aiohttp-socks package and its ProxyConnector.

Rotating proxies: the session-ID pattern

With a backconnect gateway (see how rotation gateways work) you rarely rotate in Python at all — you rotate by changing the username. A new session ID means a new exit IP; a reused one keeps it:

import secrets, requests

def proxy_for(session_id: str, country: str = "de") -> dict:
    user = f"customer-country-{country}-session-{session_id}"   # provider-specific format
    url  = f"http://{user}:PASSWORD@gate.provider.com:7777"
    return {"http": url, "https": url}

# sticky: same IP for a multi-step flow
sid = secrets.token_hex(4)
s = requests.Session(); s.proxies.update(proxy_for(sid))
s.get("https://shop.example/login"); s.post("https://shop.example/cart", data=...)

# rotating: new IP per item
for url in urls:
    requests.get(url, proxies=proxy_for(secrets.token_hex(4)), timeout=20)

Rotate on failure too: catch 403/429 and a CAPTCHA marker, and retry with a fresh session ID and a fresh cookie jar — the same IP with the same cookies is the same visitor.

The mistake no proxy fixes: TLS fingerprint

All three libraries make TLS connections through Python's ssl module (OpenSSL). Anti-bot systems fingerprint that handshake and know it is not a browser, however you set the User-Agent. Behind a perfect residential IP the script is still identified as a script — the mechanism is in TLS fingerprinting explained. For protected targets, switch to an impersonating client; curl_cffi keeps a requests-like API:

# pip install curl_cffi
from curl_cffi import requests as cffi_requests

r = cffi_requests.get("https://protected.example/",
                      impersonate="chrome",                 # reproduces Chrome's TLS + HTTP/2 fingerprint
                      proxies={"https": "http://user:pass@proxy.example.com:8080"},
                      timeout=20)

Then keep the rest of the story consistent — a Chrome User-Agent, matching sec-ch-ua client hints, Accept-Language plausible for the proxy's country. For sites that additionally require JavaScript challenges, no HTTP client suffices; use a real browser as in Proxies in Playwright, Puppeteer and Selenium.

Errors and what they mean

A minimal, correct template

import requests, secrets
from urllib.parse import quote

GATE, PORT = "gate.provider.com", 7777
USER_BASE, PASS = "customer-country-de", quote("p@ss:word", safe="")

def make_session() -> requests.Session:
    sid = secrets.token_hex(4)
    url = f"http://{USER_BASE}-session-{sid}:{PASS}@{GATE}:{PORT}"
    s = requests.Session()
    s.proxies.update({"http": url, "https": url})
    s.headers.update({"User-Agent": "python-requests"})   # honest UA for tolerant targets
    return s

s = make_session()
r = s.get("https://api.ipify.org?format=json", timeout=20)
r.raise_for_status()
print(r.json())

Verify the IP changes, then build on it. For a full pre-flight (leaks, headers, blocklists) see How to test a proxy; for choosing datacenter vs residential for a given target, the proxy types comparison.

Frequently asked questions

How do I use a proxy with Python requests?

Pass proxies={"http": url, "https": url} to the request or set it on a Session, where url is http://user:pass@host:port. Add timeout=. For SOCKS5 install requests[socks] and use socks5h://.

Why does httpx say ‘unexpected keyword argument proxies’?

Because proxies= was removed in httpx 0.28. Use proxy="http://…" for one proxy, or mounts= for per-target proxies.

How do I set a proxy in aiohttp?

Pass proxy="http://user:pass@host:port" to each request (session.get(url, proxy=…)). The proxy_auth= parameter is deprecated; put credentials in the URL. Use aiohttp-socks for SOCKS.

How do I rotate proxies in Python?

With a backconnect provider, change the session ID in the proxy username per request or per flow — no IP list needed. Rotate on 403/429 and pair each new IP with a fresh cookie jar.

Why is my script blocked when my browser on the same proxy is not?

TLS and HTTP/2 fingerprinting: Python's ssl module does not look like a browser. Use curl_cffi with impersonate= or automate a real browser.

Should I use socks5 or socks5h?

socks5h. The h makes the proxy resolve hostnames, preventing DNS leaks and letting you reach hosts only the proxy can resolve.