PR #3254 shipped two things onto the FAQ journey in the same week: a seeded workshop-host
corpus, and a fabrication guard blocking any [bracket]-placeholder row from being
one-tap-confirmed as a founder's real policy. This round proves the guard live, across two
businesses, five separate trips — and finds a second paid path, untouched by that PR,
quietly failing in a different way: a founder can be shown a price, billed a fraction of it, and
told they weren't charged.
docs/testing/brain-faq-journey-test-plan-2026-08.md.
| # | Scenario | Business | Result | Note |
|---|---|---|---|---|
| A1 | Landing state | FFB | PASS | Pre-seeded Brain confirmed; no coverage dots (correct — no-corpus persona). |
| A2 | Manual add via depth sheet | FFB | PASS | Generic candidate-picker + default trio chips, deduped correctly. |
| A3 | Candidate chip → inline answer | FFB | PASS | “Keep this answer” disabled→enabled correctly. |
| A4 | Ask-flow fallback | FFB | SKIP | Section already 10/10 answered — no entry point existed. Not forced open by deleting real data. |
| A5 | Gate floor + reset | FFB | PASS | 10 answered, well over the 3+ floor. VerdictTag read “⚠ stale” — see §4. |
| B1 | Corpus landing state | Uglywoo | PASS | Dots render on persona switch; aria-label exact match. |
| B2 | Persona chips + founder brackets | Uglywoo | PASS | A founder typing literal bracket text into her OWN answer saves fine — the guard is corpus-scoped only, as designed. |
| B3 | Bracket-guard regression | Uglywoo | PASS — the money shot | All 3 tested corpus chips arrive blocked. Zero bracket text written anywhere. See §2. |
| B4 | Answering around the guard | Uglywoo | PASS | Self-typed answer saves; coverage dots stay all-gap — see Observation 1, §6. |
| C1 | Corpus landing state | Saltline | PASS | Exact match: 0 covered, 0 partial, 8 gaps = the plan's stated 8 groups. |
| C2 | Manual adds | Saltline | PASS | Typed pairs land correctly; dots stay all-gap. |
| C3 | Fact-blank confirm + policy write | Saltline | PASS | 2-blank fact card, correct gating, the exact single joined policy write — no merge-bug regression. See §5. |
| C4 | Coverage dots progress | Saltline | PASS | Aria-label matches visual (0/3/5). See §5. |
| C5 | Paid AI draft truthfulness | Saltline | FAIL | Priced $3, delivered nothing, billed $0.10, claimed $0. See §3. |
| C6 | Remaining FAQs + gate verdict | Saltline | PASS | All 10 table items covered. See §5. |
| C7 | Offering-tagging + final reset | Saltline | PASS | “0/1 offerings tagged” exactly as predicted — see Observation 2, §6. Reset verified clean. |
The workshop-host curated corpus (17 rows, migration 20260807151149) uses
[bracket] customize-hints, not the {token} syntax
extractPlaceholders() recognizes. Before PR #3254, an empty token list read as
“fully resolved” — a founder could one-tap-confirm a researched peer figure
like “[$150 or 10% of the total…]” straight into her live policy. Every one of
B3's three tested chips arrived blocked instead: bracket text visible as reference, the
exemplar's confirm button simply doesn't render.
Scenario C5 is a different paid action entirely (brain-generate, TMPL-BRFAQ01,
$3/run) — untouched by PR #3254, and never exercised end-to-end until this round. The price
disclosure fires correctly. What comes back doesn't.
runLLM already billed the real token cost via its own
finally{} write, but BEFORE the $3 BRFAQ01 meter fires. The $3 not firing is
correct-by-design (never charge for a draft that doesn't exist); the bug was the small orphaned
charge plus a generic error toast that unconditionally claimed “you weren't charged”
— false in this exact case.
runLLM+parse cycle once before giving up, and threads a real charged
flag through every failure branch — not just the one this round hit — so the toast
never claims zero charge when a prior attempt already billed. Reviewed by Quality (3 finders),
Security, CTO, and Billing Agent before merge; two more real gaps were caught and fixed in that
pass. The actual void/credit for an orphaned charge when BOTH attempts fail is a separate,
deliberately deferred follow-up — that needs Billing Agent + finance gate, not an
improvised fix riding this one.
The FAQ section's VerdictTag showed “⚠ stale” in every post-edit state checked (A5, B4, C2, C6) — including right after a founder answers ten questions correctly.
recordFloorVerdicts: it deliberately marks a verified field stale
when its content hash changes — “changed since last verify, paid re-check
queued.” Correct, non-blocking, persists across reload as a real DB row. The only
problem is the word: right after a founder types a true answer, “stale” reads as
the system contradicting her.{token} syntax, the retreat-host fixture seed), correctly gated until both
blanks are filled, resolves to clean prose with no leftover {}. Confirming it
wrote the EXACT single joined policy: “Your deposit (% or amount): 30% · Balance
due — weeks before start: 6” — one home for the money fact, not two, no
merge-bug regression.One finding surfaced opportunistically outside the written plan: a founder's typed text sometimes landed in a FAQ's Question field with Answer left empty. Reproduced twice, on two different businesses — looked exactly like a field-swap bug.
[bracket] text, so the
guard correctly blocks every one from being one-tap-confirmed. Since categorization only comes
from corpus-confirmed pairs, the only paths left are typing from scratch, or the paid AI draft
— which itself failed in §3. Tracked:
#3275 (P1).
| Issue | Priority | What | Status at time of writing |
|---|---|---|---|
| #3280 | P1 | Paid draft: orphaned charge + false “not charged” claim on parse failure | Fixed & deployed (PR #3285) |
| #3275 | P1 | Convert curated [bracket] rows to {token} per-row (CTO scope-ruled) | Open |
| #3277 | P2 | Retreat-host's 242 curated rows still unseeded — the named wedge persona has a weaker corpus than the fast-follow persona | Open |
| #3278 | P3 | “⚠ stale” verdict label reads as a contradiction right after a founder edits | Open |
| #3276 | P3 | No manual offering-tag UI on hand-written FAQs | Parked (ADR-013/T1035) |
| #3283 | P3 | Candidate-chip question pre-fill reads confusingly (corrected from a filed “bug”) | Open, downgraded |