🛠️ Tech Stack

🚀 Features

• 4 independent LLM judges score every argument on distinct rubrics (Logic, Evidence, Persuasion, Originality) with structured JSON output and written rationales

• Configurable weight sliders recompute rankings instantly on the client — no API round-trip — so users can explore how different evaluation criteria produce different winners

• Parallel async judge dispatch via asyncio.gather with partial failure recovery: if 1–2 judges fail, the argument still receives partial scores instead of failing entirely

• Structured output parsing with retry — if a judge returns malformed JSON, a correction prompt (with the parse error and original response) is sent once before raising

• Animated leaderboard reordering with Framer Motion spring animations (300ms) when weight adjustments change the sort order

• Transparency panel with radar chart visualization (Recharts) and collapsible judge rationales — every score is explainable

• 3-phase submission state machine (drafting → scoring → complete) with real-time polling, character counter with color-coded feedback, and new-argument highlight animation

• Rate limiting middleware with sliding window pattern, input sanitization (HTML stripping, unicode normalization), and defense-in-depth validation at both API and database layers

• 73 automated tests (58 backend + 15 frontend) covering schema validation, sanitization, composite scoring, database integration, full API stack, and judge orchestration

• Dark mode with localStorage persistence and system preference detection, semantic color tokens for rubric-specific theming

🧗‍♂️ Challenges

• Partial failure in parallel LLM calls — The 4 judge evaluations run concurrently, and any subset can fail (timeouts, malformed responses, API errors). Using asyncio.gather(return_exceptions=True) lets exceptions become values instead of cancelling siblings. The orchestrator classifies each result individually, persists successful scores, and assigns a status (scored / partial / failed) based on how many judges succeeded. This required designing the frontend to gracefully handle arguments with 2–3 scores instead of assuming all 4 exist.

• Async SQLAlchemy relationship loading — SQLAlchemy's lazy loading silently breaks in async contexts (MissingGreenlet errors). Every query that touches relationships must explicitly use selectinload() — there's no safe fallback. This forced a disciplined query pattern across all route handlers and test fixtures, and required expire_on_commit=False on the session factory to prevent post-commit attribute access from triggering implicit loads.

• Transaction rollback test isolation with seed data — Tests run inside a transaction that rolls back after each test, but seed data committed outside that transaction is still visible. Fixtures that need an empty state (e.g., testing "no active topic" behavior) must explicitly deactivate existing data within the test transaction rather than relying on a clean database. This created a subtle interaction between fixture ordering and data visibility that required careful session/transaction scoping.

• Client-side ranking correctness — Weight redistribution must guarantee sum(weights) === 100 after any slider change. Naive proportional redistribution across 3 remaining rubrics accumulates Math.round() drift. The solution: round all-but-last, compute last as remaining - distributed. The composite score computation also handles partial-status arguments (missing rubrics) without NaN propagation — only rubrics with non-null scores contribute to the weighted average.

• Background task session scoping — POST /api/arguments returns 202 immediately while judge evaluation runs in a BackgroundTasks callback. But FastAPI's dependency-injected DB session is request-scoped — it closes when the response is sent. Background tasks must create their own session via the factory, not reuse the request's session. Missing this produces intermittent SessionNotFoundError that only appears under real concurrent load.

🧠 Learnings

• LLM-as-judge evaluation design — Building calibrated judge prompts that produce consistent, rubric-aligned scores is harder than it looks. Generic prompts produce generic scores. Each judge needed a distinct persona, explicit scoring anchors (what does a 3 vs. 7 look like?), and structured output constraints. The retry-on-parse-failure pattern was essential — even well-prompted models occasionally return malformed JSON.

• Async Python architecture patterns — Working with SQLAlchemy 2.0 async, asyncpg, and FastAPI together exposed subtle interactions: expire_on_commit, eager loading requirements, server_default vs default for DB-generated values, and the flush() vs commit() distinction in test fixtures. These aren't in most tutorials — they emerge from building a real async application end-to-end.

• API design as a consumer contract — The same Score rows produce two different response shapes: dict[Rubric, int] for the leaderboard (optimized for client-side weight math) and list[ScoreResponse] for the detail view (includes rationales). Designing API responses around what the consumer needs to do, not how the database stores it, made the frontend code dramatically simpler.

• State machine UI patterns — Replacing boolean flags (isSubmitting, isPolling, isComplete) with a single Phase union type ("drafting" | "scoring" | "complete") eliminated impossible states entirely. Phase transitions during render (not in useEffect) skip stale DOM commits. useReducer for polling state batches related updates into one render and gives dispatch a stable identity.

• Defense-in-depth validation — Validating the same constraints at both the Pydantic boundary (user-friendly errors) and the PostgreSQL layer (CHECK constraints, UNIQUE constraints) catches different failure modes. Pydantic catches bad user input with clear messages; DB constraints catch bugs in internal code, migration scripts, or any path that bypasses the API. Neither layer alone is sufficient.