LLM-powered · SAST

Semantic Rule Verifier

The structural analyzers can tell you that a table has a policy. They cannot tell you the policy compares the user's id to the task's own primary key. This scanner gives each RLS migration and Firebase rules file to a model and asks only one question: is the logic of these rules wrong?

LLMSASTCriticalHighRLS misconfigurationneeds an API key

What is sent, what comes back

One model call per file. The scanner decides which files qualify with plain string matching, so the model never sees a file that is not about access rules.

Repository filesthe scan's file index Build the promptscanner code Modelthe scan's LLM client Parse the replyscanner code 0001_tasks.sqlhas CREATE POLICY: review 0002_seed.sqlno policy text: skipped firestore.rulesrules file name: review README.mdnot .sql, not a rules file SYSTEMRLS or Firebase expert, flaws to look for, and a severity rubric USERfile path plus the first 50,000 characters of the file FORMATa JSON array, or [] if nothing is wrong max 4,096 tokens back REPLY, one item per flaw severity title description line_number remediation_guidance Told to skip structural issues: other scanners already report those. first [ to last ] of the reply must be valid JSON keep only objects, max 7 severity: model's, else MEDIUM if unknown confidence: 0.70, fixed category: RLS misconfig. remediation_guidance is not kept in the finding No API key: the verifier is never added to the SAST list. A model call that fails or returns junk gives no findings for that file. The other files and the rest of the scan carry on.

1Collect. A .sql file is sent if its text contains CREATE POLICY or ENABLE ROW LEVEL SECURITY. A file is sent as Firebase rules if its name ends in firestore.rules, storage.rules or database.rules.json. Everything else stays local.

2Send. Each qualifying file becomes one prompt: a system message naming the flaws to look for, and the file path plus its first 50,000 characters. The model is asked for a JSON array.

3Reply. Each flaw comes back as severity, title, description, line number and remediation guidance. The prompt tells the model to ignore missing policies and open rules, which the structural analyzers already report.

4Parse. The scanner takes everything from the first [ to the last ], parses it, keeps at most seven objects, and builds each finding with its own fixed confidence of 0.70.

5Absent or failing. Without an API key the scanner is not registered at all. If a call errors or the reply is not JSON, that file yields nothing, is recorded in skipped_files, and the scan continues.

A policy that looks right

The example from the scanner doc. A regex scanner sees ENABLE ROW LEVEL SECURITY and a CREATE POLICY and is satisfied. The mistake is in one comparison.

0001_tasks.sql CREATE TABLE tasks ( id uuid PRIMARY KEY, user_id uuid REFERENCES profiles(id), title text ); ALTER TABLE tasks ENABLE ROW LEVEL SECURITY; CREATE POLICY "Users view own tasks" ON tasks FOR SELECT USING (auth.uid() = id); id is this row's primary key user_id is the owner column The prompt holds this one file, up to 50,000 characters. Schema in another file is not in view. Structural analyzer RLS enabled. A policy exists on tasks. Nothing to report. It checks that rules exist, not what they compare. Model reads the file The policy compares the caller's uuid to the task's own primary key. The owner column is user_id. The doc's reading: the check never matches, so the owner cannot see their own tasks, and the rule gives false confidence while looking correct. finding · SAST · RLS misconfiguration Policy compares auth.uid() to id file 0001_tasks.sql line_number 7 (as the model said) severity chosen by the model confidence 0.70 description is the model's text. A fix such as auth.uid() = user_id comes from later fix generation.

1A migration with RLS switched on and one SELECT policy. Both structural checks pass: the table is protected and a policy exists. The comparison on the last line is the question.

2The structural RLS analyzer finds nothing here. It answers whether policies exist, and they do.

3The model reads the whole file. It can see id is the primary key and user_id is the owner column, so it knows auth.uid() = id compares the wrong things.

4The reply becomes a finding with the model's title, description, severity and line number, and a fixed confidence of 0.70. The other flaw classes it looks for are privilege escalation paths, uneven operation coverage and tenant isolation errors.

5The limit of this approach: the prompt holds only the one file. If the table and its columns were created in an earlier migration, the model sees the policy without the schema and has less to judge with.

What guards the output

Model text is never used as a finding as it arrives. It passes four checks, and what the code does not check is listed last.

before the call extract per item the finding left open Registration added to the SAST list only if an LLM client exists no API key: no scanner, no model call, no cost files need policy text or a rules file name Extract first [ to last ] of the reply, code fence or not no array: no finding invalid JSON: no finding a call that raises is logged as a warning and that file is skipped Per item only the first 7 items per file are used an item that is not an object is skipped unknown or missing severity becomes MEDIUM The finding confidence 0.70, fixed: the model's own is not read category always RLS misconfiguration title falls back to a generic text if missing Not checked the claim is not tested: no live query confirms it line_number is taken as given, not matched to the file remediation_guidance is asked for, then dropped Net effect: a model claim enters the report as a medium-confidence SAST finding, never as a proven one. The RLS deep scanner is the one that asks the live database with the anon key.

1No key, no call. The verifier is registered only when an LLM client exists. Without one the scan runs every rule-based scanner and none of this.

2Extract. The first valid JSON array in the reply is used; code fences and surrounding prose are tolerated. A reply with no valid array gives an empty result for that file, and one malformed item (for example a null severity, defaulted to Medium) is skipped or defaulted without losing the rest. A refused call, model error or unparseable reply is logged, shown as a progress message and recorded in skipped_files.

3Per item. At most seven items per file. Non-objects are skipped. A severity the code does not recognise is treated as MEDIUM.

4Confidence is not the model's to set. Every finding gets 0.70 and the fixed RLS misconfiguration category, so triage treats all of them alike.

5Never auto-trusted. Nothing re-tests the claim, and the line number is not checked against the file. The doc says the model supplies fix suggestions per finding, but the code requests them and does not store them.

Why it matters

Why it matters

A policy that looks correct but references the wrong column is worse than no policy: it gives false confidence while being bypassable or broken. The scanner looks for four kinds of logic error: a wrong column in the auth.uid() check, a combination of policies that allows escalation (for example a user updating their own role), a SELECT policy with no matching UPDATE or DELETE policy, and multi-tenant policies that do not scope to the tenant. For Firebase rules it looks for auth checked on the wrong path, writes protected while reads leak, custom claims checked loosely, and rules that let one user overwrite another's document by predictable id.

Real-world context

Facebook (2013). The ability to delete any photo was caused by the API checking whether the photo id existed rather than whether the requester owned it. An RLS policy checking auth.uid() = id instead of auth.uid() = user_id is the same class of bug: the check runs, it just asks the wrong question.

Vulnerable vs fixed rules

The same pairs the scanner doc lists. The fix is a different comparison or fuller operation coverage, not more policies.

-- LOOKS correct but IS wrong:
CREATE POLICY "Users view own tasks" ON tasks
  FOR SELECT USING (auth.uid() = id);
-- id is the TASK's primary key, not the owner's id

Limits and configuration

SettingValueNote
Needsan LLM API keyruns in the SAST phase when a policy file or rules file is present
SQL files reviewedcontain CREATE POLICY or ENABLE ROW LEVEL SECURITYcase-insensitive text match on .sql files
Firebase files reviewedfirestore.rules · storage.rules · database.rules.jsonthe list of rules files the scanner collects
Sent per filefirst 50,000 charactersone model call per file
Reply limit4,096 tokensfindings per file capped at 7
Confidence0.70, fixedthe same for every finding
SeverityCRITICAL, HIGH, MEDIUM or LOW from the modelunknown values become MEDIUM
CategoryRLS misconfigurationfixed, including for Firebase rule findings