🛠️ Tech Stack

🚀 Features

• Multi-source data pipeline — Orchestrates concurrent API calls to GitHub REST API (repo metadata, commit activity, contributor stats, issue/PR responsiveness), npm registry, PyPI stats, and HN Algolia, with asyncio.gather and per-source error isolation so a single API failure never blocks the rest

• LLM documentation analysis — Uses OpenAI function calling to extract structured quality assessments from README content across five dimensions (clarity, getting-started, examples, architecture, contribution guide), returning typed JSON scores with a written summary and improvement suggestions

• Composite health scoring engine — Five weighted sub-scores using logarithmic scaling, ratio-based metrics, and recency decay — designed so each component is independently interpretable and the formula is fully transparent in /docs/scoring_methodology.md

• Backtesting & evaluation framework — Scores 20 historical projects (known successes like FastAPI, known failures like gpt-engineer) using frozen metric snapshots, then computes Spearman rank correlation against actual outcomes to validate predictive power

• Automated scheduling — APScheduler cron triggers run daily (new project discovery) and weekly (full refresh of all tracked projects), with a status API endpoint exposing last-run time, next-run time, and pipeline health

• Production-grade infrastructure — Token-bucket rate limiter (1.3 req/s for GitHub's 5K/hr quota), Redis caching with TTL, exponential backoff with Retry-After header support, and transactional flush()-based persistence so partial pipeline failures roll back cleanly

• Interactive React dashboard — Filterable project list (language, score range, sort), detail views with radar chart score breakdowns and time-series trend lines, head-to-head project comparison, and a trending movers page — all with URL-synced filter state via useSearchParams for shareable/bookmarkable views

• Responsive design — Desktop table + mobile card layout pattern using Tailwind breakpoint toggles, with conditional chart rendering that only shows npm/PyPI visualizations when the project actually has package registry data

🧗‍♂️ Challenges

• Star count is genuinely a terrible health signal — Backtesting proved it: projects like gpt-engineer (48K stars, stagnated) and open-interpreter (42K stars, abandoned) scored high on community but low on maintenance. The data showed maintenance responsiveness was 4x more predictive than star count (ρ=0.860 vs ρ=0.204), which required reweighting the formula away from the spec's original assumptions

• Growth sub-score needs historical data that doesn't exist yet — The spec called for stars_gained_30d, which requires a previous snapshot to diff against. On a fresh database, that field is always zero — meaning 40% of the growth sub-score's weight was dead. Solved by leaning on download acceleration and commit trend signals for the remaining 60%, and documenting star velocity as a Phase 2 improvement once the system has 30+ days of data

• GitHub stats endpoints are eventually consistent — The /stats/commit_activity and /stats/contributors endpoints return HTTP 202 with empty bodies on first request while GitHub computes them server-side. The pipeline treats these as graceful nulls rather than errors, and subsequent collection runs pick up the computed values

• Naive vs. timezone-aware datetime mismatch — SQLAlchemy DateTime columns store naive timestamps, but Python 3.12+ deprecates datetime.utcnow(). Trend query functions using datetime.now(UTC) (timezone-aware) couldn't compare against the naive DB values. Fixed with datetime.now(UTC).replace(tzinfo=None) at the application boundary, with a documented note that the real fix is a DateTime(timezone=True) migration

• CORS errors masking real backend failures — FastAPI's CORS middleware doesn't add Access-Control-Allow-Origin headers to 500-level responses that bypass the middleware chain, so browser-side the datetime bug above surfaced as a misleading CORS error. Diagnosed by hitting the endpoint directly with curl to see the actual traceback

🧠 Learnings

• LLMs are most powerful as one component in a larger system, not the whole system — The LLM handles qualitative README analysis (something quantitative metrics can't capture), but the scoring engine, data pipeline, and trend detection are all deterministic. This hybrid approach is more reliable and auditable than an end-to-end LLM solution

• Evaluation methodology should be designed before the feature, not after — Building the backtesting framework alongside the scorer (not as an afterthought) caught the star-count-as-signal assumption early and directly shaped the final formula weights

• Error isolation at the data-source level is critical for multi-source pipelines — Each collector (GitHub, npm, PyPI, HN) is wrapped in its own try/except. A transient npm outage returns neutral scores for that dimension instead of crashing the entire pipeline. Partial data is far more useful than no data

• flush() vs commit() matters for pipeline transactionality — Storage functions use session.flush() to generate IDs without committing, letting the pipeline orchestrator wrap the full sequence (project → snapshot → assessment → score) in a single transaction. If any step fails, the entire operation rolls back cleanly instead of leaving the DB in an inconsistent state

• Documentation is a forcing function for design quality — Writing the scoring methodology doc surfaced that 40% of the growth sub-score weight was dead (star velocity always returning 0). Writing the README's architecture section revealed the collector→scorer→storage→API flow wasn't obvious from the code alone. Both led to concrete improvements