urieljareth/yt-channel-scraper
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
21 Commits 1 Branch 0 Tags
c693b20e19ca29d4d644576ef98dc49c51276725
Commit Graph
3 Commits
Author SHA1 Message Date
urieljareth c693b20e19 chore(repo): datos privados fuera de git + line-endings normalizados 
- .gitignore: data/* (canales, BD, markdown, media), cookies/ (sesiones de YouTube),
  config.yaml real (usar config.example.yaml), graphify-out/, .superpowers/, .env,
  venvs, estado local de agentes
- untrack de graphify-out (102 archivos de cache regenerable), config.yaml y
  .claude/settings.json (estado de maquina local); los archivos se conservan en disco
- .gitattributes: LF en todo el texto, CRLF solo para .bat/.ps1, binarios marcados
  (fin de los warnings LF/CRLF en cada diff)
- data/.gitkeep documenta el proposito del directorio
 2026-09-10 00:32:55 -06:00
urieljareth 0682d2d806 feat: batch .md download, multi-select, thumbnails, all OPPORTUNITIES tools 
Videos view: multi-select (checkboxes) + select-all/deselect-all + bulk action
bar (Download .md batch, Download thumbnails, Download audio). Per-video buttons:
.md download (done) or Process to .md (pending), clip.

Channels: per-channel pending count + 'Download pending .md' (batches pending
video_ids). Each channel keeps Videos/.md folder/Remove actions.

Backend (jobs.py + api.py):
- POST /api/scrape/batch {video_ids} + POST /api/scrape/video/{id} (on-demand
  .md for any video incl. pending) as SSE-tracked jobs
- GET /api/videos/{id}/markdown (file download)
- POST /api/tools/thumbnails + GET /api/thumbnails/{id} (local cache, offline)
- GET /api/clip/{id}?from=&to= (transcript segment)
- GET /api/stats (aggregate totals)
- POST /api/tools/audio (mp3 job)

Tools view rebuilt as full Knowledge Base grid surfacing every OPPORTUNITIES.md
feature as electable buttons (md download, re-render, export, search, analysis,
thumbnails, audio, clip, stats, cookies, channels, watch, open folders).

Floating job widget (SSE) for any running job. Thumbnails served via
/api/thumbnails/{id} everywhere (works offline once cached).

Launcher: robust Open-Browser helper (3 fallback methods) auto-opens
http://127.0.0.1:<port> on start + on idempotent re-open. data/ gitignored.
 2026-07-26 23:39:05 -06:00
urieljareth 621bbc5f5c feat: local content-mining platform for YouTube creator scraping 
Scraper de canales de YouTube hacia notas Markdown para base de conocimiento
(Obsidian-ready), con plataforma web local.

Engine + CLI (Workstream A):
- Modular pipeline: discover/extract/parse/chapters/render/store + ratelimit
- SQLite store con migración idempotente: FTS5 (transcript search), columnas
  de metadata enriquecida, tablas cookies_meta y scrape_jobs
- Módulos: segments, cookies (Netscape vault), export (json/csv/srt/html),
  analysis (word freq/timeline/wordcloud), monitor (watch loop), pipeline
- CLI Click group: search, export, audio, channels, watch, analyze, re-render
- Fix del bug de scoping de cookies en cli.py

Webapp local (Workstream B):
- FastAPI backend: dashboard, channels, videos facetado, transcript, search
  FTS, analysis, scrape jobs con SSE, cookies drag-and-drop, exports,
  folders (abrir en OS), tools (re-render, formato)
- SPA no-build (Alpine.js + Tailwind + Chart.js por CDN): 9 vistas, tema
  dark command-center con acento rojo→rosa, cookie vault drag-drop,
  consola de scrapeo con progreso live vía SSE

Launcher + subagentes (Workstream C):
- start-server.bat / stop-server.bat con auto port-scan + browser open
- .opencode/agent/webapp-builder.md + .opencode/goals/webapp-build.md

Tests: 38 pytest verdes. Sin funcionalidad de IA (enfoque data-mining).
 2026-07-26 23:19:34 -06:00