Competitor Intel Tracker

Leadership needed a current read on the competitive landscape, but compiling it by hand across many competitors and sources was slow and never quite up to date.

This system automates the whole loop: every Monday morning it collects fresh signal on 21 lending-software competitors from 9 data sources, summarizes it with AI, and publishes two Confluence reports — a per-competitor intel page and a rolling industry-trends analysis — with zero manual compilation.

A 9-source collection pipeline runs with cascading fallbacks to conserve API quota — Google News RSS is the primary source, and a paid Custom Search is only called when needed — alongside blog scraping, job-posting monitoring, review-site monitoring, and hash-based page-change detection. Collected items are deduped with MD5, sorted into a 6-category keyword taxonomy, and prioritized by competitor tier.

An OpenAI gpt-4o-mini step writes "Morning Brew"-style 2–3 sentence summaries, batched by competitor, for roughly $0.50–1.00 per week with rate-limiting. Results publish to Confluence via both the v1 and v2 APIs: a fresh weekly intel page plus a rolling trends page with direction badges and a 4-week heatmap. Everything fires on scheduled Monday-morning triggers.