🛠️ Tech Stack

🚀 Features

• Real-time tokenization — Counts tokens locally via tiktoken's WASM binary running in the browser, with 150ms debounced updates as you type. Supports both cl100k_base (Claude, GPT-4) and o200k_base (GPT-4o) encodings with automatic re-tokenization on model switch.

• D3.js treemap & bar chart — Animated data visualizations show proportional token allocation across prompt sections. Uses D3's enter/update/exit data join pattern with 300ms easeQuadOut transitions, hover tooltips, and click-to-scroll navigation between the visualization and input panels.

• Surgery Mode — When your prompt exceeds the model's context window, Claude analyzes your sections and returns targeted optimization suggestions (trim, compress, restructure, remove). Each suggestion includes a word-level inline diff, editable text, one-click accept, and full undo support (Cmd+Z).

• Multi-model cost comparison — Switch between Claude Sonnet 4, Claude Haiku 4, GPT-4o, and GPT-4 Turbo to compare token counts, budget utilization, and per-request/monthly cost estimates across different encodings and pricing tiers.

• Exact token counting — For Claude models, a one-click "Exact Count" button proxies Anthropic's /v1/messages/count_tokens endpoint to compare approximate (client-side) vs. exact (server-side) token counts.

• Export & keyboard shortcuts — Copy formatted prompt text to clipboard (Cmd+E) or download structured JSON with token metadata. Full keyboard shortcut support for surgery mode, undo, and export.

🧗‍♂️ Challenges

• D3 + React DOM ownership — D3 and React both want to own the DOM. Drew a strict boundary: D3 imperatively manages everything inside <svg> elements (data joins, transitions), while React owns HTML wrappers (tooltips, legends, containers). A ResizeObserver feeds container dimensions into React state, which triggers D3 re-renders via useCallback — keeping the two rendering systems from fighting over the same nodes.

• tiktoken WASM lifecycle in React — The tiktoken encoder allocates Rust-side memory that JavaScript's GC can't collect — it must be explicitly free()'d. Built a singleton TokenizerService outside React's render cycle to manage encoder creation, disposal on encoding switches, and graceful fallback (~4 chars/token heuristic) if WASM fails to load.

• React 19 strict lint rules — React 19's eslint-plugin-react-hooks v7 flagged two previously accepted patterns as errors: setState inside effects and writing refs during render. Required rethinking the tokenizer hook's debounce architecture to avoid both patterns while keeping the 150ms update latency.

• Cross-store state coordination — Surgery mode spans two Zustand stores (surgeryStore reads from and writes to budgetStore). Used getState() for cross-store reads and direct action calls for writes — avoiding circular subscriptions while keeping per-suggestion undo stacks correctly synchronized with section content.

🧠 Learnings

• Context engineering is a systems problem. Building a tool about token budgets required deeply understanding how prompt sections compete for finite context window space — and how encoding differences between models (cl100k vs. o200k) change the math. The treemap visualization makes this competition tangible in a way that raw numbers never do.

• D3's data join pattern is worth the complexity. The enter/update/exit pattern is verbose compared to clearing and re-rendering, but it's the only way to get smooth animated transitions when data changes continuously. Keying joins on section id gives stable animations even when sections are added or removed.

• Pure functions make state management predictable. Extracting computeBudget() as a pure function (inputs → outputs, no side effects) meant budget calculation was unit-testable without mocking Zustand or React. Every store mutation calls it, so computed fields like isOverBudget and utilizationPercent can never drift out of sync.

• WASM in the browser is production-ready but needs care. tiktoken's WASM binary is ~2.5MB gzipped — the largest chunk in the build — but it makes real BPE tokenization possible client-side with sub-millisecond encoding times. The key was lazy initialization (defer encoder creation until first use) and explicit memory management on encoding switches.