feat: local content-mining platform for YouTube creator scraping
Scraper de canales de YouTube hacia notas Markdown para base de conocimiento (Obsidian-ready), con plataforma web local. Engine + CLI (Workstream A): - Modular pipeline: discover/extract/parse/chapters/render/store + ratelimit - SQLite store con migración idempotente: FTS5 (transcript search), columnas de metadata enriquecida, tablas cookies_meta y scrape_jobs - Módulos: segments, cookies (Netscape vault), export (json/csv/srt/html), analysis (word freq/timeline/wordcloud), monitor (watch loop), pipeline - CLI Click group: search, export, audio, channels, watch, analyze, re-render - Fix del bug de scoping de cookies en cli.py Webapp local (Workstream B): - FastAPI backend: dashboard, channels, videos facetado, transcript, search FTS, analysis, scrape jobs con SSE, cookies drag-and-drop, exports, folders (abrir en OS), tools (re-render, formato) - SPA no-build (Alpine.js + Tailwind + Chart.js por CDN): 9 vistas, tema dark command-center con acento rojo→rosa, cookie vault drag-drop, consola de scrapeo con progreso live vía SSE Launcher + subagentes (Workstream C): - start-server.bat / stop-server.bat con auto port-scan + browser open - .opencode/agent/webapp-builder.md + .opencode/goals/webapp-build.md Tests: 38 pytest verdes. Sin funcionalidad de IA (enfoque data-mining).
This commit is contained in:
@@ -0,0 +1,46 @@
|
||||
$ErrorActionPreference = 'Stop'
|
||||
$root = Split-Path -Parent $PSScriptRoot
|
||||
Set-Location -LiteralPath $root
|
||||
|
||||
if (-not (Test-Path '.run\server.info')) {
|
||||
Write-Host ""
|
||||
Write-Host " [yt-scraper] no hay servidor registrado." -ForegroundColor Yellow
|
||||
Write-Host ""
|
||||
exit 0
|
||||
}
|
||||
|
||||
$firstLine = (Get-Content '.run\server.info' -TotalCount 1)
|
||||
$parts = $firstLine -split '\s+'
|
||||
$port = $parts[0]
|
||||
$pidv = $parts[1]
|
||||
|
||||
Write-Host ""
|
||||
Write-Host " [yt-scraper] deteniendo servidor (PID $pidv, puerto $port)..." -ForegroundColor Cyan
|
||||
|
||||
if ($pidv -and $pidv -ne 'pending') {
|
||||
# Kill the process tree (uvicorn + any workers).
|
||||
try {
|
||||
Stop-Process -Id ([int]$pidv) -Force -ErrorAction Stop
|
||||
} catch {
|
||||
# fallback: taskkill
|
||||
& taskkill /PID $pidv /T /F 2>$null | Out-Null
|
||||
}
|
||||
}
|
||||
|
||||
# Confirm the port is freed.
|
||||
$freed = $false
|
||||
for ($i = 0; $i -lt 10; $i++) {
|
||||
try {
|
||||
Invoke-WebRequest -Uri "http://localhost:$port/healthz" -UseBasicParsing -TimeoutSec 1 | Out-Null
|
||||
} catch { $freed = $true; break }
|
||||
Start-Sleep -Milliseconds 400
|
||||
}
|
||||
|
||||
Remove-Item '.run\server.info' -Force -ErrorAction SilentlyContinue
|
||||
|
||||
if ($freed) {
|
||||
Write-Host " [ok] puerto liberado, servidor detenido." -ForegroundColor Green
|
||||
} else {
|
||||
Write-Host " [warn] el puerto $port sigue ocupado; revisa procesos python." -ForegroundColor Yellow
|
||||
}
|
||||
Write-Host ""
|
||||
Reference in New Issue
Block a user