urieljareth/yt-channel-scraper
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
3 Commits 1 Branch 0 Tags
0866c92650abbe87b70cac98b64801f37626664b
Commit Graph
3 Commits
This Branch
This Branch
All Branches
Author SHA1 Message Date
urieljareth 0866c92650 feat: .md download bundles thumbnails, auto-cache on channel add 
- Bulk 'Download .md' now also caches thumbnails (one action does both);
  removed the redundant standalone Thumbnails button from the bulk action bar.
- Scrape UX optimization: batch job skips videos whose .md already exists
  (just caches their thumbnail) instead of re-extracting.
- Auto-thumbnails: on addChannel, thumbnails cache in the background so a
  newly-attached channel's catalog looks populated immediately, even before
  any .md is generated.
- Thumbnail fallback: pending videos without a stored thumbnail URL use the
  canonical https://i.ytimg.com/vi/<id>/hqdefault.jpg, so every video shows
  a thumbnail (and /api/thumbnails/{id} never 404s — redirects if uncached).
- New helpers in pipeline.py: thumbnail_url_for() + cache_thumbnail().
 2026-07-26 23:48:46 -06:00
urieljareth 0682d2d806 feat: batch .md download, multi-select, thumbnails, all OPPORTUNITIES tools 
Videos view: multi-select (checkboxes) + select-all/deselect-all + bulk action
bar (Download .md batch, Download thumbnails, Download audio). Per-video buttons:
.md download (done) or Process to .md (pending), clip.

Channels: per-channel pending count + 'Download pending .md' (batches pending
video_ids). Each channel keeps Videos/.md folder/Remove actions.

Backend (jobs.py + api.py):
- POST /api/scrape/batch {video_ids} + POST /api/scrape/video/{id} (on-demand
  .md for any video incl. pending) as SSE-tracked jobs
- GET /api/videos/{id}/markdown (file download)
- POST /api/tools/thumbnails + GET /api/thumbnails/{id} (local cache, offline)
- GET /api/clip/{id}?from=&to= (transcript segment)
- GET /api/stats (aggregate totals)
- POST /api/tools/audio (mp3 job)

Tools view rebuilt as full Knowledge Base grid surfacing every OPPORTUNITIES.md
feature as electable buttons (md download, re-render, export, search, analysis,
thumbnails, audio, clip, stats, cookies, channels, watch, open folders).

Floating job widget (SSE) for any running job. Thumbnails served via
/api/thumbnails/{id} everywhere (works offline once cached).

Launcher: robust Open-Browser helper (3 fallback methods) auto-opens
http://127.0.0.1:<port> on start + on idempotent re-open. data/ gitignored.
 2026-07-26 23:39:05 -06:00
urieljareth 621bbc5f5c feat: local content-mining platform for YouTube creator scraping 
Scraper de canales de YouTube hacia notas Markdown para base de conocimiento
(Obsidian-ready), con plataforma web local.

Engine + CLI (Workstream A):
- Modular pipeline: discover/extract/parse/chapters/render/store + ratelimit
- SQLite store con migración idempotente: FTS5 (transcript search), columnas
  de metadata enriquecida, tablas cookies_meta y scrape_jobs
- Módulos: segments, cookies (Netscape vault), export (json/csv/srt/html),
  analysis (word freq/timeline/wordcloud), monitor (watch loop), pipeline
- CLI Click group: search, export, audio, channels, watch, analyze, re-render
- Fix del bug de scoping de cookies en cli.py

Webapp local (Workstream B):
- FastAPI backend: dashboard, channels, videos facetado, transcript, search
  FTS, analysis, scrape jobs con SSE, cookies drag-and-drop, exports,
  folders (abrir en OS), tools (re-render, formato)
- SPA no-build (Alpine.js + Tailwind + Chart.js por CDN): 9 vistas, tema
  dark command-center con acento rojo→rosa, cookie vault drag-drop,
  consola de scrapeo con progreso live vía SSE

Launcher + subagentes (Workstream C):
- start-server.bat / stop-server.bat con auto port-scan + browser open
- .opencode/agent/webapp-builder.md + .opencode/goals/webapp-build.md

Tests: 38 pytest verdes. Sin funcionalidad de IA (enfoque data-mining).
 2026-07-26 23:19:34 -06:00