How it works — and the problems worth solving

This page is for the technically curious. The interesting part of this project

wasn't the happy path — it was the layers of real-world friction that had to be

solved to make it reliable. A few highlights.

Architecture

The tool is a Model Context Protocol (MCP) server written in Node.js, deployed in a Docker container on Railway. It runs in two modes from one shared codebase: a local mode for desktop use, and a remote mode reachable from any device — including mobile — secured by a full OAuth 2.0 implementation, since the client connector requires real OAuth rather than a static token. It integrates the official YouTube Data API for search, playlists, and comments; an unofficial captions path plus an OpenAI Whisper fallback for transcripts; the Stack Exchange API for Q&A research; and Claude for the self-correcting search judgment, the credibility vibe classifier, and the SE topic extraction step.

A three-tier transcript pipeline

Transcripts come from three fallback tiers, in order: official-style captions

first (free and fast), then audio download plus Whisper transcription (reliable

but costs money per call), and finally a metadata-only summary if both fail.

The response always reports which tier was used, so the result is never a

silent black box.

The hard part: looking like a real browser

Cloud servers get treated with suspicion by large platforms — requests from a

datacenter IP are often silently blocked or degraded, independent of whether

the code is correct. Getting transcripts to work reliably from the cloud meant

solving a chain of real problems: authenticating requests with browser session

cookies, computing the specific signed authorization header the platform's

internal API actually requires (sending cookies alone is silently ignored),

handling cookie rotation, and even catching subtle bugs like Windows-style line

endings corrupting the cookie data. Each layer looked like the last one — until

it wasn't.

Self-correcting search, under the hood

The smart-search feature works in a loop:

It sends the search results to a fast, inexpensive language model (Claude) along with a plain-language description of the goal
Haiku judges whether the results actually match that goal
If they don't, it rewrites the query and the server runs the search again
This repeats up to a set number of attempts

The server then returns both the final results and a full record of every query it tried and the reasoning behind each — turning an opaque "trust me" step into an auditable one. Each judgment costs a fraction of a cent.

The self-correcting search in action. Asked to find rig-rigidity content with a strict relevance goal, here's what happened:

First attempt: the tool ran the search and judged its own results — mostly off-target
Rewrite: it rewrote the query itself based on what was wrong
Repeat: it tried again, judging each new batch the same way
Done: once the results actually matched the goal, it returned them — along with its full reasoning at every step

Reading the output: the top half of the screenshot is the request — a search query plus a plain-language description of what counts as a good result. The bottom half is the response, and the interesting part is the refinementAttempts log. On its first try, it judged only 3 of 8 results relevant and explained why ("off-topic: pedal setup, gear showcase, clickbait"), then rewrote its own query and tried again. It repeated this until the results genuinely matched the goal — and recorded its reasoning at every step. The finalQuery it settled on is noticeably different from the one it started with: the tool, not the user, wrote the search that actually worked.

How the credibility scoring works

The credibility system is built on a principle borrowed from how real bot-detection research actually works: no single signal should move a score on its own. Instead, weak signals compound — the more that converge on the same comment or thread, the more the score moves. This is called a noisy-OR combination model, and it means a generic "great video!" comment from a two-day-old account barely registers, while the same comment posted simultaneously from fifty distinct accounts in a thirty-second window registers very differently.

The authority lane (Stack Exchange) uses the platform's own trust signals directly: vote score, accepted-answer status, and reputation. Reputation is log-scaled and capped so it acts as a nudge — never more than ±15% — rather than a dominant factor. This was validated on real Stack Overflow threads: an earlier version let high reputation carry answers the community hadn't endorsed, which produced obviously wrong scores on high-traffic threads.

The integrity lane (YouTube) applies four heuristics to each comment and combines them via noisy-OR. The output for each flagged comment includes a reason and, for duplicate-text flags, the number of distinct accounts that posted the same thing — so "29 real humans independently calling a product trash" is immediately distinguishable from "2 bot accounts posting the same script."

The vibe classifier runs separately from the integrity lane: it sends comments to Claude in batches of 50 and classifies each one into one of five sentiment buckets. The suspicious (🤖) marker is applied afterward from the integrity lane's bot-probability scores — deliberately kept separate so tone and trustworthiness don't get conflated. The known limitation: the classifier is honest about what it can't know. Without device fingerprint or IP data, it's heuristic-only. It's designed as a triage signal, not a verdict.

Honest about the limits

Reliability against an adversarial platform is an ongoing effort, not a solved problem — cookies expire, detection changes, and the cheap captions path remains less reliable than the paid transcription fallback. The credibility tools have their own honest ceiling: no account age signal (quota cost), no device or IP data (not available via the API), and a burst-timing heuristic that fires on any dense minute relative to a video's all-time comment pace — not just coordinated activity. A notification squad of real subscribers commenting within seconds of each other after upload looks identical to a bot cluster on this metric. The tool documents these limitations explicitly rather than pretending they're fixed. That honesty is part of the point.

Page updated

Report abuse