Pulse / Usage / Billing IA Redesign — Click-Through Report

Frontend/backend sync verification for PR #3763's money/metric surfaces (CT-PUBM-01..11).
Flowpulse-usage-billing-metrics Date2026-08-24 Accountpeytonworkprojects@gmail.com (Peyton WP) PR under test#3763, merged + deployed to prod MethodLive Chrome walk + direct backend cross-checks (curl + Doppler dev) + source-code trace for two apparent anomalies

Scenarios completed

CT-PUBM-01Overview Attention tile: count + sub-line statesPASS

3-tile row: Runs this week / Attention / MTD Cost (no standalone Health tile — matches #3763). Attention tile showed a green checkmark (0 items needing review), never a %. Backend cross-check via the authenticated assistants?path=dashboard-stats call (run from the browser console with the session JWT) confirms needs_attention_count: 0, attention_tile_count: 0 — matches the checkmark state exactly. Both sub-line phrasing branches (failed-run count vs review-item count) could not be exercised — count is 0 on this account — noted as SKIP for that half of the check, not a fail.

Pulse Overview — 3-tile row, Attention checkmark, wallet-ledger-sourced MTD Cost.
\n
CT-PUBM-02Overview Breakdown table: Total row mathPASS

Breakdown table renders the 5-chip filter row (Active/Paused/Completed/Stopped/Failed) and reads “No assistants yet — Browse assistants →” (zero live assistants right now). The Total row showed Cost $82.25 even with an empty assistant list — traced this in src/pages/Pulse.tsx: under prepaid billing both the tile and the Total row read the SAME monthlyCost variable, which is wallet-ledger-sourced (statement.by_category['run']), not the exec-log-derived dashboard-stats aggregate — a deliberate #3827/#3631 fix so real historical spend survives an assistant being deleted. Confirmed both surfaces agree (no drift between tile and Total row) — this is the exact frontend/backend sync invariant this plan cares about, and it holds.

The plan's own pass criteria ("sum the visible per-assistant rows, should equal Total") assumes rows exist to sum. With 0 assistant rows and a non-zero wallet-derived Total, that specific instruction doesn't apply — recording this so a future run doesn't chase it as a false bug.
\n
CT-PUBM-06Usage Cost breakdown: one money source, internal consistencyPASS

Legend: Assistant runs $0.00 · 0% / AI actions $6.84 · 100% / = Total cost $6.84 — percentages sum to 100%, no NaN/blank. By-assistant tab: empty state (“No spend yet”), All-assistants Combined = $0.00 — consistent with the legend's Assistant-runs slice ($0.00), since 100% of this month's $6.84 is standalone (non-assistant-tied) AI usage.

Cost breakdown — By assistant tab, empty state, legend percentages sum to 100%.
\n
CT-PUBM-07Usage KPI cards vs Cost breakdown: known divergence (#3878)PASS

Top KPI “Total cost” card reads $87.54; the Cost breakdown legend below reads $6.84 — a real, large divergence. Root-caused it precisely rather than just noting the number: the KPI card is wallet-ledger-sourced (all money debited this month, including the $82.25 “Runs” spend from an assistant that no longer exists); the Cost breakdown legend is scoped to what it can currently attribute (assistant-tied + standalone AI from ai_usage), which has nothing to attribute the orphaned $82.25 to since the assistant is gone. This is the documented #3878 divergence, now with a concrete mechanism — not a new bug per the plan's instruction (only flag if implausible, e.g. negative or 100x off; 12.8x here, fully explained).

Usage KPI row — Total Cost $87.54 vs Cost breakdown $6.84, both correct by design.
\n
CT-PUBM-08AI-actions attribution by surface/template_idPASS

Every row on the AI actions tab has a real label — “Brain generation”, “Brain Reputation”, “Brain · read notes”, “Brain · image read”, “Brain · embeddings”, “Voice check” — never a raw enum or undefined. Cross-checked every row against the raw ai_usage table filtered to template_id IS NULL (standalone) for August: exact match on every figure — brain_generate $4.10, brain_reputation $1.16, brain_read_note $0.97, screenshot_vision_extract $0.40 (labeled “image read”), brain_embed $0.12, voice_score $0.10 (labeled “Voice check”), summing to the exact $6.84 total shown in the legend.

AI actions tab — every row cross-checked byte-for-byte against raw ai_usage rows.
\n
CT-PUBM-09Billing page: Wallet activity is money-onlyPASS

Main /billing page section reads “WALLET ACTIVITY” (prepaid label, not “Recent activity”) with only Wallet top-up rows visible. The “View all →” drill (/billing/history) opens a “Billing · Wallet activity” slide-over showing exclusively money-movement rows — AI usage debits, Brain deliverable debits, Assistant run debits, Wallet top-ups — never an individual run-log row with status/duration. Confirmed /billing/usage has no old “Run log” section — replaced by “Cost breakdown” (CT-PUBM-06).

/billing main page — Wallet balance + Wallet activity, money-only rows.

Scenarios blocked this sitting

CT-PUBM-03 / 04 / 05 / 11Merged Activity tab / assistant-detail modal / RunIssueSheetBLOCKED

All four scenarios need a live assistant with real run + lifecycle history. This account currently has zero assistant instances (assistants table returns [] for this user_id) — wiped earlier today by a parallel session's fixture work (bali-offline-pilot). CT-PUBM-10 (below) was the planned path to generate a fresh fixture; it hit an unrelated activation bug and could not complete this sitting.

\n
CT-PUBM-10Setup-test regression: activate a cheap assistantBLOCKED

Picked the cheapest live template, Waitlist + Launch ($0.20/run). Completed Personalize (filled required fields), Connect (all three apps showed Connected, accepted Terms + pricing), and Try — the built-in test passed (created a real Google Sheet, added a test subscriber). Clicking Activate failed twice in a row at “Setting up your assistant” with “Storage provider selection required”, despite Storage (Google Sheets) showing Connected throughout and the Try-step test just having used that same sheet successfully. Both attempts rolled back cleanly — no charge, no partial assistant.

Filed #3988 — reproducible activation blocker on Waitlist + Launch. Per Peyton's call, flipped TMPL-WAITL1.development_stage to coming_soon (migration in a companion PR, pending merge) so founders can't hit this live. Did not attempt a different template this sitting — the only other live public template (TMPL-9P3X On-Brand Support) is a WhatsApp chat assistant, out of scope for an unattended fixture-generation step.

Follow-up (same sitting) — CT-PUBM-10 retry on On-Brand Support

CT-PUBM-10Setup-test regression — retry on a different template after Waitlist + Launch was ruled outPASS

Per direct instruction to stop using Waitlist + Launch, retried CT-PUBM-10 against On-Brand Support (TMPL-9P3X, $0.50/session) instead. Connect's Brain-check gate blocked activation on 7 missing playbook fields ("0 of 5 ready"). Closed via the pre-existing, idempotent scripts/seed-brain-playbook-test-account.ts (#2437 — vetted "Bloom Retreats" fixture through the real brain-profile EF write path), plus two follow-up field writes (business_hours, contact_info) through the same path. The remaining gate — "How you sound" (confirmed brand voice, brand_voice_v2) — is deliberately NOT a plain PUT-able field (confirmed via a live 400 listing valid sections); it's gated behind the real Voice generator/confirm flow on the Brain page. Walked that flow live: "Hear how you sound" → "Review & generate" (real charged AI call, $5.17 debited) → "Save to playbook" → "Your voice — live". Connect gate cleared. Activate fired cleanly: "On-Brand Support" is live — no error, no rollback, confirming #3988 is scoped to Waitlist + Launch specifically, not activation generally.

Pulse Overview showing On-Brand Support activated, MTD Cost $92.70

Pulse Overview immediately after activation — wallet $213.09 (voice generation charge only; activation itself does not debit, confirmed billed on first real reply).

Pulse Breakdown table with new On-Brand Support row, 0 runs

Breakdown table: new row for On-Brand Support (Active, 0 runs, $0.00) — confirms this is a genuinely fresh assistant, not reused fixture data.

Assistant detail modal Pulse tab, fixed-four metrics, no runs yet

Assistant-detail modal (CT-PUBM-05 shape check) — fixed-four metrics template renders correctly (Runs/Health/Last run/Cost this month) even at zero-run state; confirms the template itself, not the data, is what's still missing for CT-PUBM-03/04/05/11.

CT-PUBM-03 / 04 / 05 / 11Still BLOCKED — different reason nowBLOCKED

On-Brand Support is WhatsApp-reactive: per the Try step's own copy ("Message your WhatsApp after activation to see it live... There's no way to fake a guest message before then") it only produces a run when a real inbound WhatsApp message arrives. Confirmed via direct DB check immediately post-activation: total_runs: 0, no synthetic test run fires on activation for this template class. Browser automation cannot send a WhatsApp message, so these four scenarios still can't be walked this sitting — but the blocker moved from "activation is broken" (fixed/routed around) to "no real inbound message yet" (needs either real WhatsApp traffic on this assistant, or Waitlist + Launch fixed and re-flipped to live — a non-reactive, on-demand template whose own Try-step test already proved it generates real run+cost data).

Follow-up (same sitting) — CT-PUBM-03/04/05 unblocked by a real WhatsApp conversation

CT-PUBM-03 / 05Real closed-session run cross-checked across every surfacePASS

Peyton texted the live On-Brand Support number directly (9 real messages, one hit a real grounding_failed escalation, handled gracefully). Discovered mid-walk: billing/run visibility is per closed chat SESSION (30-min idle timeout, _shared/chat-billing.ts), not per message — individual whatsapp_inbound rows are deliberately excluded from every run count (_shared/run-exclusion-filter.ts). To avoid a real 30-minute wait, backdated this one session's last_message_at and invoked the real resume-stale-conversations cron endpoint (Peyton-approved) — same production close/bill code, not a fabricated result. Session closed for real, billed $0.50, total_runs 0→1. The resulting run now matches exactly across the Overview Breakdown row, the merged Activity feed, and the assistant-detail modal's Pulse tab.

Pulse Overview showing a real Needs Attention escalation card and Breakdown row with 1 run, $0.50

Overview after the session closed: Breakdown row shows Runs 1 / Cost $0.50 / Health 100% — and a real "NEEDS ATTENTION (1)" escalation card from the grounding-failed message mid-conversation.

Assistant detail modal Pulse tab with real Runs=1, Cost=$0.50, matching Breakdown exactly

CT-PUBM-05: modal's fixed-four metrics (Runs 1 / Health 100% / Last run 28m ago / Cost $0.50) match the Breakdown row exactly — the frontend/backend sync this plan exists to verify.

CT-PUBM-04Needs-attention toggle — naming overlap found, not a sync bugPASS

Pagination, scope switching, and counts all behave correctly. One nuance worth flagging for product, confirmed deterministically (fresh page load, single deliberate toggle click, screenshotted before/after): toggling Needs-attention ON in the merged Activity feed shows 0 events, even while Pulse Overview's own "NEEDS ATTENTION (1)" card shows a live item for this exact conversation. Traced the code — not a bug: the Activity feed's toggle filters strictly on workflow_execution_logs.status === 'failed' (PulseActivityTab.tsx), and this run's status genuinely is 'success' (the conversation recovered and completed; the escalation is a human-handoff-on-one-detail event, not a run failure). Overview's card is sourced from a different table entirely (whatsapp_conversation_state via useEscalatedConversations). Both are individually correct — they're just two different "needs attention" concepts sharing one label across two surfaces on the same page. Recommend a product/UX pass on the naming, not a bug fix.

Merged Activity feed with Needs-attention toggle ON showing 0 events, despite a live Overview escalation

Deterministic repro: fresh page load (baseline "5 of 7 events", toggle off) → one click on Needs-attention → "0 events" — confirmed twice, not a rendering race.

CT-PUBM-11Still BLOCKED — narrower reason nowBLOCKED

Confirmed no "View issue" affordance renders on the closed-session run row — expected, since RunIssueSheet only opens for status: 'error' runs and this run's status is 'success' (a graceful escalation, not a run failure). This account/session type cannot currently produce a genuine run-level error. Needs either a connector-disconnect/webhook-delivery-failure class of run, or a system-paused assistant for Part B — neither available this sitting.

Coverage note

Unit tests (activityFeed / costBreakdown / versionActivity / useWallet / FilterChip / PaginationBar / PulseActiveTabResolution — see plan file) lock the pure functions. This walk additionally verified the exact numbers a live Chrome session sees against the exact rows in Supabase (via curl under Doppler dev, which is the same shared Supabase as prod), which the unit tests cannot do. CT-PUBM-03/04/05/10/11 need a re-walk once this account has a real assistant + run history again — recommend re-running against a fresh Waitlist + Launch fixture once #3988 is fixed, or picking any other live template with a cheap per-run price.