---
title: Writing Product — Architecture & Build Plan
type: product-spec
status: v1-shipped
created: 2026-06-04
source: dynamic method survey (8-agent workflow, 2026-06-04)
---

# Writing Product — Architecture & Build Plan

A personal, agentic writing tool: **spark → research → draft in your voice → evaluate → revise until it passes → you ship.** Built on Claude + your existing `tools/search/` (grounding) and `tools/stylometry/` (voice scorer). Optimized for *simple to build, sophisticated in output*.

## v1 status (2026-06-04) — shipped as a Claude Code skill
Productized as **`/voice-draft`**, a project skill at `.claude/skills/voice-draft/` (NOT inside the
`idea-foundry` plugin — that plugin isn't in `enabledPlugins`, so a skill there would be dormant).
Claude is the drafting model; the loop reuses `tools/stylometry/profiler.py` (voice gate) and
`tools/search/grounded_search.py` (grounding) unchanged — zero new dependencies.

Deliberate v1 simplifications vs. the design below:
- **Retrieval dropped.** The essay corpus (`posts/`, ~6.4k words) fits whole in context; register-matched
  exemplars are injected verbatim. RAG-of-style returns only when the corpus outgrows the window.
- **Single essays voice card** (no Raw/Authority split — 9 essays is too few for two robust cards; register
  affects exemplar *selection* only).
- **Substance gate = `draft-gate`'s tests inline**, not yet a separate calibrated LLM-judge.

**Phase 2 (deferred):** the calibrated substance LLM-judge as a scored gate · the 30-case golden set +
`run_evals` harness (the lab-credibility artifact) · separate register cards · tweets register · the
flagship essay + OSS packaging.

## The core insight
The hard part of a writing agent is a **reliable external verifier** — and you already built it (the stylometry scorer). Everything else is wiring four small things around a verifier you own. Don't fine-tune; don't self-host; don't buy an eval platform.

## The loop
```
spark
  └─ DRAFT  (Best-of-N=3–5, each ranked by the feature-scorer)
        └─ exemplars = your own most-similar passages (RAG-of-style), not a style description
  └─ GROUND (extract load-bearing claims → grounded-search → fix unsupported)
  └─ EVAL   three SEPARATE gates, each must clear independently (never one averaged score):
        1. VOICE      → feature scorer + AI-tell baseline   (deterministic, cheap)
        2. SUBSTANCE  → calibrated LLM-judge (hook / non-obvious / structure / ending)
        3. GROUNDING  → claims supported?
  └─ if any gate fails → feed concrete deltas back → ONE targeted revision → re-score
  └─ hard cap 2–3 iterations, early-stop on first pass → surface to you for the ship call
```

## The four layers (all build-complexity ~2, output-sophistication 4–5)

1. **Measure (keep + cheap widen).** Existing scorer stays the primary gate. Add: a **cosine-Delta MFW distance** (widen function-word signature to ~100–150 words, length-robust, emits a ranked "you over-use X / under-use Y" fix list) and a **structural-mannerism regex** pass (negation "not X but Y", rule-of-three) — the most paraphrase-durable AI tells the current scorer is blind to. Score everything as *excess over your baseline*, per register.

2. **Generate (RAG-of-style).** At draft time, retrieve the 3–8 most-similar of *your own* passages (MMR diversity, correct register) and inject them **verbatim** as exemplars, wrapped in a lean style guide whose "don't" list *is* the scorer's AI-tell features. Anthropic prompt-caching makes a generous exemplar block ~90% cheaper. Retrieval gets 80–90% of fine-tuning's benefit at a fraction of the build — and keeps you on Claude.

3. **Eval (two-leg ensemble + grounding).** The scorer (voice) + a **calibrated analytic LLM-judge** for substance (per-criterion rubric from your `content-quality-bar` memory, temp 0, explicit *negative* criteria, 1–3 anchored few-shot from your own best/worst posts, register-aware). The judge is the one genuinely new capability — it moves output from "sounds like Darsh" to "sounds like Darsh **and** is worth reading," and it's the anti-Goodhart insurance (can't game a surface metric and a substance judge at once).

4. **Agent loop.** Best-of-N → ground → score all gates → one anchored revision on failure → hard cap. Looping works *because* it's anchored to two reliable external checks (your scorer + grounded-search) — the regime where the research says revision helps rather than wastes tokens.

## Agent evals — the verdict
**Yes, but small and calibration-focused — not a platform.** Two artifacts:
- a **~30-case golden set** (JSONL): your published posts as *passes* + deliberate AI-slop / off-voice / weak-hook drafts as *fails*, balanced across registers;
- a **~150-line `run_evals` harness** that runs every gate over each case and asserts the right verdict.

Its job: prove the LLM-judge actually *catches slop and rejects off-voice prose* rather than rubber-stamping (out-of-the-box judges pass ~96% of everything). It doubles as the regression gate on every rubric/threshold change. Build it yourself in plain Python — more yours, and better builder-cred, than promptfoo/DeepEval/Braintrust.

## Deliberately NOT building (the simplicity dividend)
LoRA/voice fine-tune (memorizes at 5.5k words) · per-author LM perplexity / ContextLab (needs 10–100× more data **and** token logprobs the Anthropic API doesn't expose) · DetectGPT · trained AI-vs-human classifiers · GPTZero-style gates · full Burrows's Delta · raw TTR/hapax as headline scores · generic semantic embeddings as a voice scorer · training your own style-embedding model · multi-agent debate · DSPy on the draft step · off-the-shelf eval platforms.

## Build order (v1)
1. *(if tweets are in scope)* tweet-register voice card — else essays-first, skip.
2. Cosine-Delta MFW distance + structural-mannerism regex → into the scorer.
3. RAG-of-style retrieval (chunk essays, index separately; feature-vector similarity to stay zero-dep).
4. Drafting prompt (verbatim exemplars + lean style guide, prompt-cached).
5. Grounding pass (claim extraction → grounded-search → fix).
6. Calibrated substance LLM-judge.
7. 30-case golden set + ~150-line run_evals harness; tune thresholds until it rejects slop.
8. Compose the loop (Best-of-N → ground → 3 gates → one revision → hard cap).
9. Goodhart hardening (cap iterations; never unanchored self-critique; golden-set regression).
10. Phase 2 (defer unless the simple loop plateaus): frozen style-embedding leg; DSPy on judge/reviser prompts only.

## Open decisions (gate the build)
- **Scope:** essays-first, or tweets as a first-class register too?
- **Spark source:** one-line idea / rough outline / a saved clipping?
- **Model access:** personal Anthropic key / corporate gateway / shell through Claude Code?
- **Zero-dep line:** strict stdlib-only, or OK with one local embedding model (sentence-transformers, ~125M) for the optional Phase-2 leg?
- **Voice drift:** how fast is your voice changing (sets golden-set refresh cadence)?