Following an import out of setup.py flagged a clean PyPI package
Reading the module a setup.py imports closed the last known gap in our build-script corpus and immediately broke a top-300 package. nodeenv went to BLOCK because two ordinary lines sat 40KB apart in one file.
A setup.py can be entirely innocent and still be the wrong file to read.
Give it a cmdclass install override whose run() does
from main import m, and the payload has moved one file away from the scanner
while every line the scanner can see stays boring.
This is the same shape as npm's preinstall entry point, which we closed
earlier. It was the last pinned known gap in our build-scripts corpus, and nothing in the
hostile setup.py is hostile. Legitimate packages run post-install steps the
same way. So the fix was to read the module, not to widen a pattern.
One hop, and a path rather than a name#
The sdist walk now resolves the modules its build scripts import — import a.b,
import a, b, from a.b import c — and reads them out of the same
decompressed archive. One hop, capped at 12 distinct modules.
The part that took a rewrite: a dotted name resolves to a path, not a
filename. The fetch moved from basename matching to root-relative, because an sdist is
rooted at one name-version/ directory and setup.py runs with its
working directory there. A module name therefore designates exactly one file. Basename
matching would happily have read vendor/main.py in place of the
main.py the hook actually imports — a scan that reports on a file nobody runs
is worse than no scan, because it produces a clean result.
There is no stdlib denylist. Names that are not in the sdist simply never match, and a
package that ships its own os.py is precisely a package whose
os.py we want to read.
The measurement is the story#
The sweep over the top 300 PyPI packages found a real false positive.
nodeenv — top-300, a pre-commit dependency —
went to BLOCK.
The Python gates tested co-occurrence across the whole body. That was sound for as long
as the body was always a short, single-purpose setup.py. Following imports
destroyed the premise: reading __version__ out of a project module is one of
the most common patterns on PyPI, so the scanner started receiving full application
files.
Inside one of those files, nodeenv calls urlopen — it
downloads Node, that is its entire job — and calls os.environ.copy() around
40KB away, because it builds an environment for a child process. Two unremarkable lines,
each obviously fine in place, welded together into an exfiltration gate by nothing more
than sharing a file.
We had learned this once already. The @scarf/scarf false positive is what
produced JS_GATE_WINDOW in the first place. This is the same lesson restated
in Python, and it was found by measurement rather than by review: not one fixture
predicted it.
Proximity, and the seam that would have silently undone it#
The two-armed Python gates are proximity-bounded now, in
src/signals/build-scripts.ts:
const JS_GATE_WINDOW = 400;
const PY_GATE_WINDOW = 1200; // wider — a setup.py has more vertical space
const BODY_SEAM = "\n".repeat(Math.max(JS_GATE_WINDOW, PY_GATE_WINDOW) + 1);
That last line is the one worth stopping on. When several file bodies are concatenated
for scanning, the seam between them has to be wider than any window, or two halves of a
gate can pair up across files. BODY_SEAM had been sized to the JS
window alone. Leaving it there would have re-opened cross-file welding for sdists the
moment pip started following imports — silently, with every test still green, because
nothing in the suite was measuring the seam.
Before and after, over the same corpus#
282 real sdists. 123 of them carry a setup.py. The import hop engaged on
24 of those, reading 33 module files.
| Over 282 sdists | Before the window | After |
|---|---|---|
| Module files read by the hop | 33 | 33 |
| Benign packages flagged | 1 | 0 |
The same 33 files are still read. The window removed the false positive without reducing coverage, which is the only version of this trade worth shipping — a proximity bound that also stops reading files would have been a coverage cut wearing a bug fix's clothes.
10 of the 300 metadata fetches did not resolve. They are counted as unresolved rather than quietly dropped from the denominator, because a package that fell out of the sweep is not a package that passed it.
What the fixtures now hold#
Every detection here owes a regression case, and a false positive owes one just as much as a catch does.
exfil-py-deferred-import/— the gap, now caught. The decoysetup.pyis still asserted clean on its own, so the test fails if the hop stops running and something else starts firing instead.benign-py-deferred-import/— the same structure, doing nothing wrong.benign-py-far-apart-module/— reduced directly fromnodeenv, with load-bearing padding between the two halves. Shrink the padding and it goes red.
What this does not fix#
One hop is one hop. A payload two imports deep is still out of reach, and a proximity window is a heuristic about how code is usually written, not a proof about how it must be. An attacker who reads this post knows the number is 1200 and knows that padding is cheap.
What the window buys is that the gate now fires on code that looks like one operation rather than on code that merely shares an address space, and that the cost of the coverage we gained was measured on real packages instead of assumed to be zero.