feat: local content-mining platform for YouTube creator scraping

Scraper de canales de YouTube hacia notas Markdown para base de conocimiento
(Obsidian-ready), con plataforma web local.

Engine + CLI (Workstream A):
- Modular pipeline: discover/extract/parse/chapters/render/store + ratelimit
- SQLite store con migración idempotente: FTS5 (transcript search), columnas
  de metadata enriquecida, tablas cookies_meta y scrape_jobs
- Módulos: segments, cookies (Netscape vault), export (json/csv/srt/html),
  analysis (word freq/timeline/wordcloud), monitor (watch loop), pipeline
- CLI Click group: search, export, audio, channels, watch, analyze, re-render
- Fix del bug de scoping de cookies en cli.py

Webapp local (Workstream B):
- FastAPI backend: dashboard, channels, videos facetado, transcript, search
  FTS, analysis, scrape jobs con SSE, cookies drag-and-drop, exports,
  folders (abrir en OS), tools (re-render, formato)
- SPA no-build (Alpine.js + Tailwind + Chart.js por CDN): 9 vistas, tema
  dark command-center con acento rojo→rosa, cookie vault drag-drop,
  consola de scrapeo con progreso live vía SSE

Launcher + subagentes (Workstream C):
- start-server.bat / stop-server.bat con auto port-scan + browser open
- .opencode/agent/webapp-builder.md + .opencode/goals/webapp-build.md

Tests: 38 pytest verdes. Sin funcionalidad de IA (enfoque data-mining).
This commit is contained in:
urieljareth
2026-07-26 23:19:34 -06:00
commit 621bbc5f5c
45 changed files with 6541 additions and 0 deletions
+46
View File
@@ -0,0 +1,46 @@
$ErrorActionPreference = 'Stop'
$root = Split-Path -Parent $PSScriptRoot
Set-Location -LiteralPath $root
if (-not (Test-Path '.run\server.info')) {
Write-Host ""
Write-Host " [yt-scraper] no hay servidor registrado." -ForegroundColor Yellow
Write-Host ""
exit 0
}
$firstLine = (Get-Content '.run\server.info' -TotalCount 1)
$parts = $firstLine -split '\s+'
$port = $parts[0]
$pidv = $parts[1]
Write-Host ""
Write-Host " [yt-scraper] deteniendo servidor (PID $pidv, puerto $port)..." -ForegroundColor Cyan
if ($pidv -and $pidv -ne 'pending') {
# Kill the process tree (uvicorn + any workers).
try {
Stop-Process -Id ([int]$pidv) -Force -ErrorAction Stop
} catch {
# fallback: taskkill
& taskkill /PID $pidv /T /F 2>$null | Out-Null
}
}
# Confirm the port is freed.
$freed = $false
for ($i = 0; $i -lt 10; $i++) {
try {
Invoke-WebRequest -Uri "http://localhost:$port/healthz" -UseBasicParsing -TimeoutSec 1 | Out-Null
} catch { $freed = $true; break }
Start-Sleep -Milliseconds 400
}
Remove-Item '.run\server.info' -Force -ErrorAction SilentlyContinue
if ($freed) {
Write-Host " [ok] puerto liberado, servidor detenido." -ForegroundColor Green
} else {
Write-Host " [warn] el puerto $port sigue ocupado; revisa procesos python." -ForegroundColor Yellow
}
Write-Host ""