DAST scanner

HTTP Probe

Five cheap checks that ask a server what it gives away by default: a stack trace on a bad URL, a readable .git or .env, TRACE switched on, a Host header it repeats back, a newline it lets into a header. Each check sends one small request and decides on a marker in what comes back (for .env, a non-HTML KEY=VALUE line).

DASTMedium to HighInfo disclosure / auth weaknessquick depthno credentials needed

Five checks, in a fixed order

The scanner takes the endpoint inventory from discovery, builds a base URL from the first entry (scheme plus host), and ranks the inventory on the CSRF dimension. It then runs five checks. The order is a decision, not an accident: the two that probe a handful of known paths come first, because their cost does not depend on how big the app is.

Endpoint inventorybase URL = scheme + host of entry 1 1Verbose errors4 trigger pathson the base URLstack trace → MEDIUM 2Sensitive files8 known pathson the base URL.git / .env → HIGH 3Method tamperingOPTIONS, then TRACEevery ranked endpointTRACE echo → HIGH 4Host headerHost + X-Forwarded-Hostbase URL + 2 endpointsreflected → HIGH 5CRLF injection9 params × 3 payloadsevery ranked endpointcanary header → HIGH ? ? ? ? ? fixed cost: same few requests for any app grows with the inventory: methods and CRLF loop over every ranked endpoint ? = budget.expired(). If true: log it, emit "out of time before <check>", stop. Findings so far are returned.

1Five checks, always in this order, over a client that sends at most 3 requests at once with a 0.2 s delay. If the session from the authenticated crawl exists, its headers ride along. There is no cap on how many ranked endpoints are used.

2Checks 1 and 2 fetch fixed paths on the base URL: four that should 404 and eight that should not be readable. They cost the same on a 10-endpoint app and a 1,000-endpoint app, so they run first.

3Checks 3 to 5 come after. Method tampering and CRLF walk every ranked endpoint; the Host check tries the base URL plus the first two. A large app cannot spend its whole budget on method tampering and never reach the check that finds an exposed .env.

4Before each check the scanner asks whether its time budget is spent. If it is, the remaining checks are skipped and what has been found is returned. Section 03 shows why that matters.

What each check sends and what must come back

Every rule is a string or header test on the response. Nothing is parsed and nothing is exploited, so confidence is fixed per check (0.80 to 0.90) instead of measured. Step through the five rows.

checksendsresponse must satisfyfinding Verbose errorscheck 1 GET /nonexistent-path-…4 paths, incl. /%00 and a 500-char URL status ≥ 400 AND body matches a leak patternat fn ( · Traceback · /usr/ · /home/ · node_modules/ … MEDIUM · 0.80first hit stops Sensitive filescheck 2 GET /.git/HEAD /.env …8 paths: 3 files, 5 directories 200 AND indicator in the first 5000 chars"ref:" · non-HTML KEY=VALUE · or "Index of /" for listings HIGH or MEDIUM.git, .env are HIGH Method tampercheck 3 OPTIONS, then TRACEeach ranked endpoint, once per URL Allow lists TRACE/CONNECT, or TRACE echoesecho = 200 and body has "TRACE" or "Host:" (first 500) HIGH · 0.90CONNECT is MEDIUM Host headercheck 4 Host: evil-host-…-test.comthen X-Forwarded-Host, on 3 URLs forged host appears in the body or Locationbody is read up to 5000 chars; first hit stops HIGH · 0.85one finding, then stop CRLF injectioncheck 5 ?url=%0d%0aX-CRLF-Test:…9 param names × 3 payloads per endpoint header X-CRLF-Test: crlf_injected is presentas a real response header, not just in the body HIGH · 0.90one finding, then stop

1Five rows, five rules. Each ends in a fixed severity and confidence, and three of them stop at their first hit so a noisy app is not reported twenty times.

2Verbose errors. Four paths that should not exist, one of them a 500-character URL. Only a response with status 400 or above is read, and only a match against one of 12 patterns (stack frames, Traceback, /usr/, /home/, /var/www/, node_modules/, .py", line N, .java:N) and a few more) counts. A friendly 404 page matches none.

3Sensitive files. A file path needs 200 and its marker: ref: for /.git/HEAD, a non-HTML response with KEY=VALUE lines for /.env and /.env.local. A directory path needs a listing banner such as Index of /. A .git or .env hit is HIGH at 0.90; a listing is MEDIUM (0.80).

4Method tampering. OPTIONS reads the Allow header (or Access-Control-Allow-Methods). TRACE listed is HIGH at 0.80, CONNECT listed is MEDIUM. Then a real TRACE request: 200 whose body contains "TRACE" or "Host:" proves the server echoes requests back, HIGH at 0.90, and ends the check.

5Host header. The forged name evil-host-injection-test.com is sent as Host and then as X-Forwarded-Host. Redirects are not followed, so the poisoned Location is read rather than fetched. Seeing it echoed in the body or in Location means a password-reset email or cached page could carry an attacker's host. A Location that keeps the same path (a plain http-to-https or trailing-slash redirect) is reported as Low at confidence 0.4 instead of High.

6CRLF. A payload with a line break and a canary header (X-CRLF-Test: crlf_injected) goes into query parameters named url, redirect, next, return, callback, path, ref, location, goto. Only the canary showing up as a response header confirms it; a body that merely repeats the text does not.

Worth knowing

/robots.txt is not on the list: it is public by design. The .env check does not trust a bare =, which any HTML page contains. It needs a response that is not HTML and has a KEY=VALUE line, so a single-page app that answers 200 with its shell for every path is not reported.

When the clock runs out

This scanner is the worked example of a bug that looked like a trade-off. It ran 901 s against a 900 s timeout holding four real findings, an exposed /.env among them, and reported none of them, because a cancelled coroutine returns nothing. The animation below shows the fix. The general mechanism, with eight scanners side by side, is on the scan pipeline page; this page only shows the part specific to the HTTP probe.

0 765 s 900 s soft deadline: 900 × (1 − 0.15) hard cancel Before: hard cancel, measured 901 s against a 900 s timeout running… holding 4 findings, an exposed /.env among them cancelled: 0 findings reported Now: fixed-cost checks first, clock checked inside the loop 1 2 checks 1 and 2 are done within moments: their findings are already in hand 3 method tampering: loop over ranked endpoints 4, 5 skipped budget.expired() at 765 s: break, return everything found so far

1Measured, before the fix: the scanner ran 901 s against a 900 s timeout, holding four findings. The hard cancel killed the coroutine and all four were discarded. It also hid a regression: on Juice Shop a measurement read as a wash (six categories gained, six lost) because the six lost, exposed_data and info_disclosure, score off this scanner, which a doubled inventory had pushed one second over.

2The runner now publishes the deadline it enforces and TimeBudget() picks it up, keeping a 15 % margin. Checks 1 and 2 run first, so even a huge inventory cannot stop the .env check from running. Method tampering then starts its loop over every ranked endpoint.

3The clock is checked per endpoint, inside the loop, not only between checks. An earlier version checked only between phases, entered one before the deadline, ran 800 s and was cancelled with everything in it. Now the loop breaks at 765 s, checks 4 and 5 are skipped, and every finding is returned. The hard cancel stays as a backstop for a hung scanner and logs loudly if it ever fires.

Bar lengths are illustrative; the 900 s timeout, the 15 % margin and the order of the checks are the real values.

Why it matters

Why it matters

These issues look low-risk one by one and chain dangerously. An exposed .env hands over database credentials. An exposed .git lets an attacker reconstruct the source. A stack trace reveals the framework version, file paths and query structure. TRACE reflects auth cookies into a response body, which is the basis of cross-site tracing. A reflected Host header turns a password-reset email into a phishing link.

Real-world context

Exposed .env files and .git directories are among the most common findings in bug bounty programs. Automated scanners continuously probe the internet for these paths, and thousands of production sites have leaked database credentials and API keys through exposed .env files.

Vulnerable vs fixed

// BAD: the project root is a static directory, so /.env and /.git/HEAD are just files
app.use(express.static("."))  // GET /.env → 200  DB_PASSWORD=…

// and the default error handler prints the stack
app.use((err, req, res, next) => res.status(500).send(err.stack))

The rewrites do the blocking; the header only asks crawlers not to index the path. After changing the config, the probe's rule is easy to re-run by hand: GET /.env must stop returning a 200 with an = in the body.

Limits

SettingValue
Depth and timeoutRuns at quick depth. 900 s timeout; stops itself at 765 s.
Requests3 at a time, 0.2 s between them. Endpoint cap is one million, in effect none.
Base URLScheme plus host of the first inventory entry. An empty inventory returns nothing.
AuthUses the login session's headers when one exists; works without one.
One finding or manyErrors, Host, CRLF and the TRACE echo stop at the first hit. Files and listings report each path that matches. The Allow-header test reports once per dangerous method per endpoint.
Failed requestAn exception on one URL is logged at debug level and skipped; it never fails the scan.