I ran my own linter against my own app. All 19 findings were wrong.

I build vibecheck, an npm CLI that catches what AI coding tools leave behind. Dead scaffolding, invented benchmarks, fake attribution comments, that sort of thing. 39 rules, runs in CI.

Last week I pointed it at a Vite and React app I had been building on the side. Not a fixture. Real code I actually ship.

no-console-pollution fired 19 times.

I went to clean them up and stopped after the first one. It was fine. So was the second. I checked all 19.

All 19 were wrong

Thirteen looked like this:

constjson=awaitrawBrowse(browseId);if (import.meta.env.DEV){console.debug("[playlist] browse response",browseId,json);}

That block does not exist in production. Vite evaluates import.meta.env.DEV at build time, the branch becomes if (false), and the minifier deletes it. The console call being warned about is not in the bundle I ship. It never was.

The other six were in scripts/test-player.mjs, a CLI I run by hand. Printing to stdout is the entire point of that file. Calling it pollution is like warning that a print statement is going to print.

So the rule was not slightly noisy. On this codebase it had a zero percent hit rate, and every one of those 19 warnings was asking me to make my code worse.

Why it could not see the guard

Here is the rule as it shipped:

{id:'no-console-pollution',pattern:/console\.(log|debug|info)\s*\(/,
antiPattern:/eslint-disable|\/\/\s*keep|logger/,...}

The obvious fix is to add import.meta.env.DEV to that antiPattern. It does not work, and the reason is one line in the scanner:

if (rule.antiPattern&&rule.antiPattern.test(line))continue;

line. Singular. The engine hands each rule one line at a time, so antiPattern can only ever see the line the match is on. The guard is one to three lines above. An antiPattern would have caught this:

if (import.meta.env.DEV)console.log("ready");

and missed all thirteen real ones.

This is the part I think generalises. A single-line regex rule cannot express "is this inside something". Any rule about context needs to see the block, and if your engine is line-scoped you will keep reaching for the anti-pattern escape hatch and keep almost fixing it.

The fix, and the case that made it interesting

I moved the rule to the multiline path, where the detector gets the whole file. Track brace depth, note the depth when a build-time guard opens a block, stay quiet until that block closes.

Then this test failed:

if (import.meta.env.DEV){console.debug("gated");}else{console.log("this one actually ships");}

The else branch is not dev-only. It ships. That console call is a real finding and my fix was swallowing it.

The reason is that } else { starts and ends at the same brace depth. If you only compare depth before the line to depth after, nothing changed, so the guard still looks open. You have to track the lowest depth the line passes through:

functionscanBraces(maskedLine:string,depth:number){letmin=depth;for (constchofmaskedLine){if (ch==='{')depth++;elseif (ch==='}'){depth--;if (depth<min)min=depth;}}return{end:depth,min};}

On } else { the depth dips below the guard's depth mid-line and comes back. The low-water mark is the only thing that reveals the block closed.

One more detail: braces are counted on a lexer-masked copy of the line, with strings, template literals, regex bodies and comments blanked first. Otherwise a } inside a string closes a block that was never open, and every rule downstream drifts.

It happened again two days later

I shipped a scoring feature and CI went red. One error:

src/score.ts:12 error no-eval eval() or new Function() allows arbitrary code execution.

Line 12 of that file is a comment. It says:

*Severity-weighted.Aneval()callandachattycommentshouldnotcount*thesame,andaveragingthemhidestheeval().

My security rule flagged my documentation for containing the word it warns about. Writing about eval is not calling eval.

Same root cause, different shape: the rule matched raw text when it should have matched code. The fix was a codeOnly flag that runs the pattern against the lexer-masked line, so strings and comments are invisible to rules that target a code construct.

Deliberately not applied to every rule. no-ai-todo and no-ai-attribution exist to match comment text. Mask comments globally and they never fire again. Both directions have tests now.

That fix also let me delete an ignore line from my own CI config, which had been quietly excluding the rule directory because the rules describing eval kept matching themselves.

What I take from this

The console rule had been shipping since v1.0. Nobody filed a bug. It has real users.

The reason nobody complained is that a false positive in a linter does not look like a bug. It looks like your code being wrong. Most people add the suppression and move on, and the tool never hears about it.

The only reason I found it is that I ran it against something I cared about, where I knew the right answer for every single line.

Dogfooding gets talked about as a discipline thing, a virtue. It is not. It is the cheapest way to get a test set where you already know the labels.

Both fixes are in @yuvrajangadsingh/vibecheck.

npx @yuvrajangadsingh/vibecheck .

github.com/yuvrajangadsingh/vibecheck