Directing the End-to-End Verification Pass

Objectives

By the end of this chapter, you should be able to:

• Direct an AI assistant through the full mechanical pass (typecheck, lint, build, test) and the module-by-module manual checklist across the whole finished app
• Recognize the specific ways an AI assistant’s verification report can be confidently wrong, from claiming a command passed without running it to claiming a UI behavior works from reading the code instead of watching it happen
• Independently re-run the same checks the AI reports on, rather than accepting its summary, and know exactly which claims need that independent check most

💡 Why this matters: Every lesson in this course ended with a “Try It” that verified that lesson’s slice of the app, right after building it. Nothing so far has verified whether Module 5’s saved-locations feature still respects Module 4’s watch-region default, or whether an AI assistant’s WebSocket reconnect logic actually reconnects when you kill the connection for real. This lesson is the first point in the course that looks at the whole thing at once, and the first point where you have to stop trusting a summary and go check for yourself.

Directing the Pass

You could run the mechanical checks and click through the checklist yourself, and by the end of this lesson you will. But a real reason to loop an AI assistant into this pass first is that it’s fast at the mechanical half, and it can narrate what it’s checking as it goes, which gives you a report to verify against rather than a blank page to start from.

A capable AI assistant given that prompt, with shell and browser access, will genuinely run some of this. It might actually execute npm run typecheck and paste real output. It might actually open a WebSocket client against /ws and show you a real connected frame. That’s not nothing, and it’s not the trap either. The trap is the next section.

Review This

Here is the load-bearing idea of this entire lesson, stated plainly: an AI assistant can tell you “I ran the checklist, everything passes,” and be wrong, not because it’s lying, but because “ran” and “reasoned about” produce sentences that look identical in a chat window. A tool call that errored quietly, a check that got reasoned through from reading the source instead of executed, a stale terminal output from three prompts ago that still looks current, all of these render as the same confident paragraph as an item that was actually, freshly verified. You cannot tell which one you’re reading by tone. You can only tell by rerunning the check yourself.

Two concrete ways this goes wrong, both realistic, both easy to miss if you’re skimming a long report:

“The WebSocket reconnects fine” based on reading the code, not breaking the connection. Ask an AI assistant to verify reconnect behavior and a very plausible response is: it opens ws/client.ts (or wherever your frontend’s WebSocket hook lives), finds an onclose handler that calls a reconnect function with backoff, and reports “confirmed: the client reconnects on disconnect.” That’s a real observation about the code, and it’s not the same claim as “I killed the connection and watched it recover.” A reconnect handler can exist and still be broken, calling a stale closure over an old ws instance, retrying against the wrong URL, or never actually resubscribing to the current watch region after the new socket opens, and none of that is visible from reading the handler in isolation. The only way to know it actually works is to actually kill the connection, restart the server process while the frontend is open, and watch the UI recover live traffic on its own without a manual reload.

"npm run test passed" when it was never actually run, or was run against stale output. This is the more common version, especially over a long session. An AI assistant that ran the test suite once, early in the conversation, and then made several more edits to detect.ts afterward, can still say “tests pass” later in the same conversation without rerunning them, either because it’s summarizing from memory or because it’s inferring from the shape of the edit that the tests probably still hold. Or it ran the command, the terminal tool it called silently failed or timed out, and it reported the expected result instead of the actual one because that’s the more coherent-sounding sentence to produce next. Either way, the report reads exactly like a fresh, real pass. The only way to tell the difference is to run npm run test -w @skywatch/server yourself, in your own terminal, after every edit that touches detect.ts, and read the actual output.

The pattern underneath both examples is the same: reading code and reasoning about what it should do is a real and useful thing an AI assistant can do for you, but it is a different act from executing that code and observing what it actually does, and a verification report needs to be honest about which one happened. Since you often can’t tell from the report alone, the only reliable move is to re-execute the specific claims that matter most yourself: run the commands in your own terminal, open your own WebSocket client, resize your own browser window. Treat the AI’s pass as a first draft of the checklist, not as the checklist completed.

Try It

This is the actual mechanical pass and checklist, the same one you’d hand to an AI assistant to walk, and the same one you now walk yourself.

The mechanical pass. Run these four, in this order, from the repo root:

npm run typecheck
npm run lint
npm run build
npm run test -w @skywatch/server

Each one catches a different class of mistake: typecheck catches a shape mismatch, not a value that’s the right type but the wrong number. lint catches the mechanical stuff, an unused import, a hook called conditionally, a missing useEffect dependency. build catches what typecheck alone doesn’t, Next.js’s own production compilation, stricter than the dev server about some patterns. test catches a behavior regression in the one place this course put automated tests, the detection engine and geo helpers from the previous lesson, and nothing else, because nothing else has tests. Fix anything that fails before continuing.

Fresh database, fresh start. Run the rest of this pass against a database that’s been through every migration from scratch, not whatever state your dev database has accumulated:

npm run db:down
docker volume rm skywatch_pg_data 2>/dev/null || true
npm run db:up
npm run db:migrate
npm run db:seed:airports
npm run dev

The verification checklist. Work through this in order, and do it yourself, at your own keyboard, even if an AI assistant already reported on it.

• Module 1, data layer. psql $DATABASE_URL -c "\dt" shows all seven tables. watch_regions has exactly one row with is_default = true.
• Module 2, backend services. The server boots without errors and starts polling (watch the console for poll-cycle logs). GET /api/watch-regions returns the seeded default region. Open a raw WebSocket client against /ws and confirm a {"type":"connected",...} message arrives immediately, followed by {"type":"positions",...} roughly every 15 seconds.
• Module 3, frontend core and theming. The app loads, shows the map, and live aircraft markers appear within one poll cycle. Click a marker or a flight-list row and confirm the detail panel populates; click again elsewhere and confirm it can be deselected. Cycle the theme toggle through all four themes and confirm <html data-theme="..."> updates with no flash on reload.
• Module 4, watch regions and history. Add a second watch region, set it as default, reload, and confirm the map opens there. Pan away and confirm the return-to-region control appears and free-pan positions load. Select an aircraft inside a watch region, wait two or three poll cycles, and confirm a trail renders. Open history playback, confirm it loads, scrubs, and plays, and confirm live data resumes instantly on exit.
• Module 5, feature build-out. Toggle airports on/off and confirm the preference survives a reload. Zoom into a large airport and confirm a METAR badge appears. Trigger a test alert (psql pg_notify against squawk_alert) and confirm it plays a sound, shows a notification, and appears in the header’s alert feed. Set a home location, then confirm a manually-inserted overflight_log row shows up on /overflights. Enter compare mode and confirm two regions render side by side, each with its own live traffic. Save a location, jump to it, and confirm it doesn’t affect the poller’s watch-region list. Capture a snapshot and confirm the PNG actually shows map tiles, not a blank background. Copy a share link, open it in a new tab, and confirm it restores the same view. Shrink the viewport below the md breakpoint and confirm the mobile bottom sheet replaces the desktop sidebar.
• Cross-cutting. Switch through all four themes again, this time on top of every panel opened above, and confirm none of them have a hardcoded color that breaks in amber or blue. Reload the page mid-free-pan and confirm it returns to the default region instead of getting stuck. Resize between desktop and mobile widths a few times in a row and confirm nothing duplicates or disappears permanently. And, specifically: kill the server process while the frontend is open, watch the client detect the disconnect, restart the server, and confirm the client actually reconnects and resumes live positions on its own, the check no code-reading pass can substitute for.

Pick one item from the cross-cutting section and try to break it on purpose, for instance opening history playback and immediately switching themes, and confirm the app degrades sensibly rather than silently, even in a combination no single module’s own verification step ever exercised.

Recap

• typecheck, lint, build, and test each catch a different, non-overlapping class of mistake. Running one is not a substitute for the other three, and neither is an AI assistant’s summary a substitute for running any of them yourself.
• An AI assistant can report a check as passed when it actually reasoned about the code instead of executing it, or executed it once and never reran it after a later edit. Both produce a sentence indistinguishable from a fresh, real pass.
• The WebSocket reconnect check is the clearest example in this app: reading the reconnect handler is not the same claim as killing the connection and watching it recover, and only the second one is actually verification.
• A fresh database run through every migration in sequence is the only way to be confident this app’s schema still composes correctly end to end, rather than happening to work on a dev database that’s never been rebuilt from scratch.

Next lesson: what’s deliberately out of scope, and how to direct an AI assistant through it if you want to keep building.