2026

VidIQ - AI Video Intelligence

Multimodal platform that turns any YouTube URL or live stream into a queryable transcript, keyframe analysis, timestamped summary and grounded Q&A. Runs end to end on free-tier infrastructure at $0 a month.

Running cost: $0/mo
LLM providers with failover: 3
App routes: 9

Repository Live demo

Stack

Next.js
TypeScript
React
FastAPI
Python
SQLAlchemy
PostgreSQL
Google Gemini
Groq
OpenAI
faster-whisper
OpenCV
yt-dlp
WebSocket
Docker
Tailwind CSS

Problem

A long video is opaque: you cannot search it, skim it, or ask it a question. VidIQ converts speech, vision and structure into one queryable artefact so a viewer can jump straight to what matters.

Architecture

Next.js App Router frontend -> FastAPI service -> provider-abstracted LLM, vision and transcription adapters -> SQLAlchemy over SQLite in dev and PostgreSQL in production, with an in-process event bus fanning pipeline progress out over WebSocket.

What it does

Recorded pipeline: YouTube URL to metadata, transcript, scene-change keyframes, vision captions, then a multimodal summary
Live pipeline: chunked download with rolling transcription, vision and LLM summarisation as the stream runs
Provider-agnostic LLM layer that rotates across Gemini, Groq and OpenAI, so one quota denial is invisible to callers
Retrieval-grounded chat where every citation seeks the embedded player to the exact moment
Cross-library analytics, side-by-side comparison of analyses, tagging and 11-language transcript translation

Challenges & learnings

Wrapped every external dependency behind an adapter with a configurable provider chain, so a quota error rotates to the next model rather than surfacing as a failure.
Map-reduce summarisation over ~4500-character transcript windows keeps long videos within context limits while still producing one coherent overview.
LLMs return JSON wrapped in markdown fences or as arrays where objects were expected; a defensive parser strips fences and finds the first balanced block rather than trusting the response shape.
Split hosting across Vercel and Hugging Face Spaces because Spaces does not sleep idle containers and offers the RAM that Whisper, OpenCV and ffmpeg need.

What it does

Recorded pipeline: YouTube URL to metadata, transcript, scene-change keyframes, vision captions, then a multimodal summary
Live pipeline: chunked download with rolling transcription, vision and LLM summarisation as the stream runs
Provider-agnostic LLM layer that rotates across Gemini, Groq and OpenAI, so one quota denial is invisible to callers
Retrieval-grounded chat where every citation seeks the embedded player to the exact moment
Cross-library analytics, side-by-side comparison of analyses, tagging and 11-language transcript translation

Challenges & learnings

Wrapped every external dependency behind an adapter with a configurable provider chain, so a quota error rotates to the next model rather than surfacing as a failure.
Map-reduce summarisation over ~4500-character transcript windows keeps long videos within context limits while still producing one coherent overview.
LLMs return JSON wrapped in markdown fences or as arrays where objects were expected; a defensive parser strips fences and finds the first balanced block rather than trusting the response shape.
Split hosting across Vercel and Hugging Face Spaces because Spaces does not sleep idle containers and offers the RAM that Whisper, OpenCV and ffmpeg need.