← Testing HQ
Brain · Round 6 · FAQ journey click-through (#3250 follow-up)

The bracket guard holds. The paid draft doesn't.

PR #3254 shipped two things onto the FAQ journey in the same week: a seeded workshop-host corpus, and a fabrication guard blocking any [bracket]-placeholder row from being one-tap-confirmed as a founder's real policy. This round proves the guard live, across two businesses, five separate trips — and finds a second paid path, untouched by that PR, quietly failing in a different way: a founder can be shown a price, billed a fraction of it, and told they weren't charged.

Account peytonworkprojects@gmail.com Surface platform.mystflo.com/brain Run date 2026-08-09 Businesses FFB (real seed pack) · Uglywoo (real, freshly seeded) · Saltline (synthetic, retreat-host) Cost ~$0.10 (the C5 defect itself) — the plan's own paid AI run never completed
Read this first. 16 scenarios executed sequentially, one business's Brain at a time (reset between each). 13 PASS, 1 real defect, 1 skip (correctly — nothing to skip TO), plus one opportunistically-found “bug” that turned out, on investigation, to be the system working exactly as designed. Plan: docs/testing/brain-faq-journey-test-plan-2026-08.md.
1 · Full results
#ScenarioBusinessResultNote
A1Landing stateFFBPASSPre-seeded Brain confirmed; no coverage dots (correct — no-corpus persona).
A2Manual add via depth sheetFFBPASSGeneric candidate-picker + default trio chips, deduped correctly.
A3Candidate chip → inline answerFFBPASS“Keep this answer” disabled→enabled correctly.
A4Ask-flow fallbackFFBSKIPSection already 10/10 answered — no entry point existed. Not forced open by deleting real data.
A5Gate floor + resetFFBPASS10 answered, well over the 3+ floor. VerdictTag read “⚠ stale” — see §4.
B1Corpus landing stateUglywooPASSDots render on persona switch; aria-label exact match.
B2Persona chips + founder bracketsUglywooPASSA founder typing literal bracket text into her OWN answer saves fine — the guard is corpus-scoped only, as designed.
B3Bracket-guard regressionUglywooPASS — the money shotAll 3 tested corpus chips arrive blocked. Zero bracket text written anywhere. See §2.
B4Answering around the guardUglywooPASSSelf-typed answer saves; coverage dots stay all-gap — see Observation 1, §6.
C1Corpus landing stateSaltlinePASSExact match: 0 covered, 0 partial, 8 gaps = the plan's stated 8 groups.
C2Manual addsSaltlinePASSTyped pairs land correctly; dots stay all-gap.
C3Fact-blank confirm + policy writeSaltlinePASS2-blank fact card, correct gating, the exact single joined policy write — no merge-bug regression. See §5.
C4Coverage dots progressSaltlinePASSAria-label matches visual (0/3/5). See §5.
C5Paid AI draft truthfulnessSaltlineFAILPriced $3, delivered nothing, billed $0.10, claimed $0. See §3.
C6Remaining FAQs + gate verdictSaltlinePASSAll 10 table items covered. See §5.
C7Offering-tagging + final resetSaltlinePASS“0/1 offerings tagged” exactly as predicted — see Observation 2, §6. Reset verified clean.
2 · The money shot — the bracket guard, live, five times

The workshop-host curated corpus (17 rows, migration 20260807151149) uses [bracket] customize-hints, not the {token} syntax extractPlaceholders() recognizes. Before PR #3254, an empty token list read as “fully resolved” — a founder could one-tap-confirm a researched peer figure like “[$150 or 10% of the total…]” straight into her live policy. Every one of B3's three tested chips arrived blocked instead: bracket text visible as reference, the exemplar's confirm button simply doesn't render.

Uglywoo corpus chip showing a blocked bracket-guard confirm
Uglywoo · corpus peer answer, bracket-gated. The founder sees the researched exemplar as reference text; “Use this answer” is not rendered at all — no disabled-button limbo, no dead click. She still has to type her own answer to move this row's coverage dot. (One minor, non-blocking note: one corpus question appeared twice, back-to-back, in the suggestion row — not investigated further.)
Negative confirmation, checked directly. After all five trips through this flow (B2, B3 ×3, B4), zero unresolved bracket text exists anywhere in the saved FAQs — not as a founder-visible answer, not in a partial draft. The fix holds under real, repeated use, not just in the code that shipped it.
3 · The real defect — priced, billed wrong, and lied about

Scenario C5 is a different paid action entirely (brain-generate, TMPL-BRFAQ01, $3/run) — untouched by PR #3254, and never exercised end-to-end until this round. The price disclosure fires correctly. What comes back doesn't.

Saltline price disclosure before tapping the paid draft button
Before the tap. “Draft it for me is paid — $3.00 per run, charged right away. You'll still review the draft before it saves.” Correct, visible, honest — the disclosure itself is not the problem.
Wallet activity showing a $0.10 debit, not the disclosed $3.00
After the ~15-20s wait. No draft anywhere on the playbook. No draft-review UI. No gate modal. Wallet: $60.89 → $60.79 — a $0.10 debit, not the disclosed $3.00. Billing history: no line item near $3.00, only generic small “AI usage” entries.
DEFECT — #3280
Parse failure after a billed LLM call left an orphaned charge and a false “not charged” claim
Root cause (CTO-diagnosed, same day): the model's output failed to parse into the FAQ draft shape — AFTER runLLM already billed the real token cost via its own finally{} write, but BEFORE the $3 BRFAQ01 meter fires. The $3 not firing is correct-by-design (never charge for a draft that doesn't exist); the bug was the small orphaned charge plus a generic error toast that unconditionally claimed “you weren't charged” — false in this exact case.
Status: fixed and deployed same day. PR #3285 (merged 2026-08-09) retries the whole runLLM+parse cycle once before giving up, and threads a real charged flag through every failure branch — not just the one this round hit — so the toast never claims zero charge when a prior attempt already billed. Reviewed by Quality (3 finders), Security, CTO, and Billing Agent before merge; two more real gaps were caught and fixed in that pass. The actual void/credit for an orphaned charge when BOTH attempts fail is a separate, deliberately deferred follow-up — that needs Billing Agent + finance gate, not an improvised fix riding this one.

Not captured in this round's evidence: the exact toast text and response body for THIS specific run — both browser console and network tracking only started (first tool call) several minutes after the click, and prod's own log retention had already rolled past the window by the time it was checked from the server side. The diagnosis above is code-verified, not log-verified against this exact request.
4 · A label that reads worse than it is

The FAQ section's VerdictTag showed “⚠ stale” in every post-edit state checked (A5, B4, C2, C6) — including right after a founder answers ten questions correctly.

FFB FAQ section showing a stale verdict tag after 10 answered questions
FFB · 10/10 answered, verdict reads “stale.” Traced (both the Chief Designer and Chief CTO review of this same work wave, independently) to recordFloorVerdicts: it deliberately marks a verified field stale when its content hash changes — “changed since last verify, paid re-check queued.” Correct, non-blocking, persists across reload as a real DB row. The only problem is the word: right after a founder types a true answer, “stale” reads as the system contradicting her.
Copy/UX only — filed as #3278, not a functional bug. No data bug, no gate impact. Route to Chief Writer / Chief Designer.
5 · The deterministic paths, working exactly as specified
Saltline fact-blank card confirmed with deposit and balance-due values filled in
Saltline · fact-blank confirm (C3). A 2-blank corpus card ({token} syntax, the retreat-host fixture seed), correctly gated until both blanks are filled, resolves to clean prose with no leftover {}. Confirming it wrote the EXACT single joined policy: “Your deposit (% or amount): 30% · Balance due — weeks before start: 6” — one home for the money fact, not two, no merge-bug regression.
Saltline coverage dots showing 0 covered, 3 partial, 5 gaps
Saltline · coverage dots after C3-C4 (0/3/5). Aria-label matches the visual exactly. The retreat-host fixture's 8-row corpus is 1:1 with its 8 groups — “covered” genuinely isn't reachable via the peer flow alone here, matching the plan's own documented fallback.
Saltline gate verdict after all 10 FAQs answered
Saltline · final gate verdict (C6). All 10 plan items covered, verdict resolves correctly.
Saltline Brain reset back to a clean empty state
Saltline · post-round reset, verified clean (C7). No in-app “reset Brain” button exists — all three businesses this round were reset manually through the app's own per-item delete UI, never a backend script (a sub-agent correctly refused to run one; see §6).
6 · Investigated and cleared — not a bug

One finding surfaced opportunistically outside the written plan: a founder's typed text sometimes landed in a FAQ's Question field with Answer left empty. Reproduced twice, on two different businesses — looked exactly like a field-swap bug.

A depth-sheet FAQ row showing text in the Question field with Answer empty
Uglywoo · the row that looked broken. Reopened in the depth-sheet editor after the ask-flow save.
Ruled not a bug, on investigation. The generic ask-flow prompt literally reads “A customer asks you something you haven't written down yet — what's one you always get?” — that's asking for a QUESTION, and the save handler deliberately writes typed text to the Question field with an empty Answer, per an explicit design comment: FAQs is list-shaped, this path exists to let a founder log a new customer question, not answer a pre-set one. Corrected and downgraded — #3283 is now P3/ux (a possible copy-clarity question for a candidate chip that pre-fills a QUESTION as a “starting point”), not a data-correctness bug.
7 · Two things the plan asked this round to surface on purpose
Observation 1 — a workshop-host founder cannot move her coverage dots through the free peer flow at all. All 17 workshop-host corpus rows carry [bracket] text, so the guard correctly blocks every one from being one-tap-confirmed. Since categorization only comes from corpus-confirmed pairs, the only paths left are typing from scratch, or the paid AI draft — which itself failed in §3. Tracked: #3275 (P1).
Observation 2 — the offering-tag gate can never fully pass by hand. Once a founder adds even one sellable offering, dots show “0/N offerings tagged” — but there is no manual UI anywhere to tag a hand-written FAQ to an offering. Deliberately parked on the dormant offering/audience data model (ADR-013/T1035). Tracked: #3276 (P3).
8 · Follow-ups filed this round
IssuePriorityWhatStatus at time of writing
#3280P1Paid draft: orphaned charge + false “not charged” claim on parse failureFixed & deployed (PR #3285)
#3275P1Convert curated [bracket] rows to {token} per-row (CTO scope-ruled)Open
#3277P2Retreat-host's 242 curated rows still unseeded — the named wedge persona has a weaker corpus than the fast-follow personaOpen
#3278P3“⚠ stale” verdict label reads as a contradiction right after a founder editsOpen
#3276P3No manual offering-tag UI on hand-written FAQsParked (ADR-013/T1035)
#3283P3Candidate-chip question pre-fill reads confusingly (corrected from a filed “bug”)Open, downgraded