urieljareth/yt-channel-scraper
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
19 Commits 1 Branch 0 Tags
3c49f73fa2b2fe14318d504e840a8810c517fe5e
Commit Graph
7 Commits
Author SHA1 Message Date
urieljareth 3c49f73fa2 feat: arranque y bootstrap multiplataforma (macOS/Linux) 
- scripts/bootstrap.sh: setup idempotente (python>=3.10, venv, extras, ffmpeg/node con hints de brew/apt, config.yaml desde example)
- scripts/start-server.sh / stop-server.sh: equivalentes bash de los .ps1 (reuso de servidor via .run/server.info, puerto libre 8000-8100, healthz, apertura de navegador con open/xdg-open)
- Makefile: setup/serve/stop/test/clean (Windows apunta a los .bat/.ps1)
- compatible con bash 3.2 de macOS, sin bashismos
 2026-09-10 00:32:32 -06:00
urieljareth b04a2710cf wip: spinner por-fila en Investigate + spawn directo de python en launcher (incluye trabajo acumulado del arbol) 2026-08-22 14:03:39 -06:00
urieljareth fae9496201 chore: copys amigables en stop-server.ps1 2026-08-22 11:27:19 -06:00
urieljareth f038790caa feat: start-server.ps1 acepta -PythonExe y copys amigables 2026-08-22 11:26:17 -06:00
urieljareth a5fd39a3e6 feat: doctor.ps1 prepara entorno (python/deps) antes del arranque 2026-08-22 11:24:37 -06:00
urieljareth 0682d2d806 feat: batch .md download, multi-select, thumbnails, all OPPORTUNITIES tools 
Videos view: multi-select (checkboxes) + select-all/deselect-all + bulk action
bar (Download .md batch, Download thumbnails, Download audio). Per-video buttons:
.md download (done) or Process to .md (pending), clip.

Channels: per-channel pending count + 'Download pending .md' (batches pending
video_ids). Each channel keeps Videos/.md folder/Remove actions.

Backend (jobs.py + api.py):
- POST /api/scrape/batch {video_ids} + POST /api/scrape/video/{id} (on-demand
  .md for any video incl. pending) as SSE-tracked jobs
- GET /api/videos/{id}/markdown (file download)
- POST /api/tools/thumbnails + GET /api/thumbnails/{id} (local cache, offline)
- GET /api/clip/{id}?from=&to= (transcript segment)
- GET /api/stats (aggregate totals)
- POST /api/tools/audio (mp3 job)

Tools view rebuilt as full Knowledge Base grid surfacing every OPPORTUNITIES.md
feature as electable buttons (md download, re-render, export, search, analysis,
thumbnails, audio, clip, stats, cookies, channels, watch, open folders).

Floating job widget (SSE) for any running job. Thumbnails served via
/api/thumbnails/{id} everywhere (works offline once cached).

Launcher: robust Open-Browser helper (3 fallback methods) auto-opens
http://127.0.0.1:<port> on start + on idempotent re-open. data/ gitignored.
 2026-07-26 23:39:05 -06:00
urieljareth 621bbc5f5c feat: local content-mining platform for YouTube creator scraping 
Scraper de canales de YouTube hacia notas Markdown para base de conocimiento
(Obsidian-ready), con plataforma web local.

Engine + CLI (Workstream A):
- Modular pipeline: discover/extract/parse/chapters/render/store + ratelimit
- SQLite store con migración idempotente: FTS5 (transcript search), columnas
  de metadata enriquecida, tablas cookies_meta y scrape_jobs
- Módulos: segments, cookies (Netscape vault), export (json/csv/srt/html),
  analysis (word freq/timeline/wordcloud), monitor (watch loop), pipeline
- CLI Click group: search, export, audio, channels, watch, analyze, re-render
- Fix del bug de scoping de cookies en cli.py

Webapp local (Workstream B):
- FastAPI backend: dashboard, channels, videos facetado, transcript, search
  FTS, analysis, scrape jobs con SSE, cookies drag-and-drop, exports,
  folders (abrir en OS), tools (re-render, formato)
- SPA no-build (Alpine.js + Tailwind + Chart.js por CDN): 9 vistas, tema
  dark command-center con acento rojo→rosa, cookie vault drag-drop,
  consola de scrapeo con progreso live vía SSE

Launcher + subagentes (Workstream C):
- start-server.bat / stop-server.bat con auto port-scan + browser open
- .opencode/agent/webapp-builder.md + .opencode/goals/webapp-build.md

Tests: 38 pytest verdes. Sin funcionalidad de IA (enfoque data-mining).
 2026-07-26 23:19:34 -06:00