Your RSA-2048 keys break in 2030. Find every one of them before attackers do.
🐍 PyPI

GHSA-jm3v-qxmh-hxwv

MEDIUM

Scrapy's redirects ignoring scheme-specific proxy settings

Published
May 14, 2024
Updated
Nov 28, 2024
Affected
1 pkg
Patched
1 / 1
Exploits
None indexed

Blast Radius

1 pkg affected
🐍scrapy

Real-time download stats are indexed for npm and PyPI packages. This vulnerability affects PyPI packages — download data is not available via public APIs for these ecosystems.

Description

Impact

When using system proxy settings, which are scheme-specific (i.e. specific to http:// or https:// URLs), Scrapy was not accounting for scheme changes during redirects.

For example, an HTTP request would use the proxy configured for HTTP and, when redirected to an HTTPS URL, the new HTTPS request would still use the proxy configured for HTTP instead of switching to the proxy configured for HTTPS. Same the other way around.

If you have different proxy configurations for HTTP and HTTPS in your system for security reasons (e.g., maybe you don’t want one of your proxy providers to be aware of the URLs that you visit with the other one), this would be a security issue.

Patches

Upgrade to Scrapy 2.11.2.

Workarounds

Replace the built-in retry middlewares (RedirectMiddleware and MetaRefreshMiddleware) and the HttpProxyMiddleware middleware with custom ones that implement the fix from Scrapy 2.11.2, and verify that they work as intended.

References

This security issue was reported by @redapple at https://github.com/scrapy/scrapy/issues/767.

Affected Packages

1 total 1 fixed
EcosystemPackageVulnerable rangeFix
🐍PyPIscrapyall versions2.11.2

Detection & mitigation playbook

Open-source dependency
  1. Detect

    Scan your dependency tree (package-lock.json, pnpm-lock.yaml, requirements.txt, go.sum, etc.) for scrapy. O3's reachability analysis confirms whether the vulnerable code path is actually invoked in your application, so you act on real exposure instead of every transitive match.

  2. Fix

    Update scrapy to 2.11.2 or later, then make sure no transitive (indirect) dependency still pins the vulnerable range — O3 confirms GHSA-jm3v-qxmh-hxwv is resolved across your whole dependency graph.

  3. Workarounds

    If you can't upgrade right away: gate or disable the affected feature, validate untrusted input at the boundary, and avoid passing attacker-controlled data into the vulnerable path. O3's runtime protection blocks exploitation in production as an interim safeguard until the upgrade lands.

  4. How O3 protects you

    O3 pinpoints whether GHSA-jm3v-qxmh-hxwv is reachable in your code and exactly where to fix it, then blocks exploitation in production at runtime until the patched version is deployed.

Tailored to GHSA-jm3v-qxmh-hxwv. Runtime protection reduces exposure until a permanent patch is applied and verified — it complements patching, it doesn't replace it.

Frequently Asked Questions

### Impact When using system proxy settings, which are scheme-specific (i.e. specific to `http://` or `https://` URLs), Scrapy was not accounting for scheme changes during redirects. For example, an HTTP request would use the proxy configured for HTTP and, when redirected to an HTTPS URL, the new HTTPS request would still use the proxy configured for HTTP instead of switching to the proxy configured for HTTPS. Same the other way around. If you have different proxy configurations for HTTP and HTTPS in your system for security reasons (e.g., maybe you don’t want one of your proxy providers to
O3 Security · Impact-Aware SCA

Is GHSA-jm3v-qxmh-hxwv in your dependencies?

O3 detects GHSA-jm3v-qxmh-hxwv across PyPI dependencies and uses function-level reachability to confirm whether the vulnerable code path is actually reachable — not just present. No false positives.