wip: estado de trabajo pendiente antes de la vista grid (suite 230 verde)
This commit is contained in:
@@ -0,0 +1,308 @@
|
||||
# Discovery-Only Scrape Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Add one-channel and all-channel discovery jobs that register new YouTube videos as `pending` without downloading or processing their content.
|
||||
|
||||
**Architecture:** Extend the existing sequential `JobManager` with `opts.mode == "discover"`. Reuse `POST /api/scrape`, its SSE stream, and its job widget; add UI actions in Channels and Videos that pass either a channel ID or no channel ID. Keep the existing full scrape and processing endpoints unchanged.
|
||||
|
||||
**Tech Stack:** Python 3.10+, FastAPI, SQLite via the existing `Store`, yt-dlp flat discovery, Alpine.js CDN SPA, pytest.
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Discovery-only jobs must not call `process_video()`.
|
||||
- Discovery-only jobs must not download transcripts, Markdown, audio, thumbnails, or avatars.
|
||||
- All HTTP requests must use the existing job queue and SSE event stream.
|
||||
- All database access must remain inside `Store` methods.
|
||||
- Preserve existing user changes in the dirty worktree and edit only the files listed below.
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Make catalog insertion counts accurate
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/yt_scraper/store.py:266-283`
|
||||
- Test: `tests/test_store_platform.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: existing `Store.upsert_videos(refs: list[VideoRef])` callers.
|
||||
- Produces: the same integer return type, now equal to the number of distinct references that were not already present.
|
||||
|
||||
- [ ] **Step 1: Write the failing test**
|
||||
|
||||
Add this test after the existing store filtering tests:
|
||||
|
||||
```python
|
||||
def test_upsert_videos_reports_only_new_rows(seeded_store):
|
||||
inserted = seeded_store.upsert_videos([
|
||||
VideoRef("v1", "UC1", "Updated title", "https://y/watch?v=v1", "20240101", 120),
|
||||
VideoRef("v4", "UC1", "New video", "https://y/watch?v=v4", "20240501", 180),
|
||||
])
|
||||
|
||||
assert inserted == 1
|
||||
assert seeded_store.get_video("v1").title == "Updated title"
|
||||
assert seeded_store.get_video("v4").status == "pending"
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the focused test to verify it fails**
|
||||
|
||||
Run: `python -m pytest tests/test_store_platform.py::test_upsert_videos_reports_only_new_rows -q`
|
||||
|
||||
Expected: FAIL because SQLite's `ON CONFLICT DO UPDATE` currently makes `rowcount` nonzero for an existing row.
|
||||
|
||||
- [ ] **Step 3: Implement the minimal count fix**
|
||||
|
||||
Inside `Store.upsert_videos`, load the existing IDs for the incoming references before the loop, deduplicate the incoming IDs with a set, and increment `inserted` only when an incoming distinct ID was absent. Keep the current upsert SQL so existing rows still refresh title, upload date, and duration.
|
||||
|
||||
The essential implementation shape is:
|
||||
|
||||
```python
|
||||
incoming = {r.video_id for r in refs}
|
||||
with self._cursor() as cur:
|
||||
existing = set()
|
||||
if incoming:
|
||||
placeholders = ",".join("?" for _ in incoming)
|
||||
cur.execute(
|
||||
f"SELECT video_id FROM videos WHERE video_id IN ({placeholders})",
|
||||
list(incoming),
|
||||
)
|
||||
existing = {row["video_id"] for row in cur.fetchall()}
|
||||
for r in refs:
|
||||
# existing SQL remains here
|
||||
if r.video_id not in existing:
|
||||
inserted += 1
|
||||
existing.add(r.video_id)
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Run the focused and regression tests**
|
||||
|
||||
Run: `python -m pytest tests/test_store_platform.py -q`
|
||||
|
||||
Expected: PASS for all store tests.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
Do not commit automatically in this workspace unless the user explicitly requests a commit. Leave the focused diff ready for review.
|
||||
|
||||
### Task 2: Add discovery-only job execution
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/yt_scraper/webapp/jobs.py:88-269`
|
||||
- Test: `tests/test_webapp_jobs.py`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `JobManager.enqueue(channel_id, opts)` with `opts={"mode": "discover"}` and optional `channel_id`.
|
||||
- Produces: queued jobs whose SSE events contain per-channel discovery counts and whose terminal event contains a summary.
|
||||
|
||||
- [ ] **Step 1: Write failing job tests**
|
||||
|
||||
Append tests using a fake `discover_channel` and a fake `process_video`:
|
||||
|
||||
```python
|
||||
def test_discovery_job_registers_new_videos_without_processing(tmp_path, monkeypatch):
|
||||
store = Store(tmp_path / "state.db")
|
||||
store.upsert_channel("UC1", "@alpha", "Alpha", 1)
|
||||
store.upsert_videos([
|
||||
VideoRef("old", "UC1", "Old", "https://y/watch?v=old", "20240101", 60)
|
||||
])
|
||||
store.mark_status("old", "done")
|
||||
store.create_job("discover-job", "UC1", {"mode": "discover"})
|
||||
calls = []
|
||||
|
||||
def fake_discover(url, sleep_subrequests=2.0):
|
||||
calls.append(url)
|
||||
return ("UC1", "Alpha", None, [
|
||||
VideoRef("new", "UC1", "New", "https://y/watch?v=new", "20240501", 90),
|
||||
VideoRef("old", "UC1", "Old", "https://y/watch?v=old", "20240101", 60),
|
||||
])
|
||||
|
||||
processed = []
|
||||
monkeypatch.setattr("yt_scraper.webapp.jobs.discover_channel", fake_discover)
|
||||
monkeypatch.setattr("yt_scraper.webapp.jobs.process_video", lambda *a, **k: processed.append(a))
|
||||
manager = JobManager(store, Config(database_path=str(tmp_path / "state.db"), output_dir=str(tmp_path / "markdown")))
|
||||
|
||||
manager._run_job("discover-job")
|
||||
|
||||
assert calls == ["https://www.youtube.com/@alpha/videos"]
|
||||
assert store.get_video("new").status == "pending"
|
||||
assert store.get_video("old").status == "done"
|
||||
assert processed == []
|
||||
assert store.get_job("discover-job").status == "done"
|
||||
assert any(event["event"] == "done" and event["data"]["new_videos"] == 1
|
||||
for event in manager.events_since("discover-job", 0))
|
||||
|
||||
|
||||
def test_all_channel_discovery_continues_after_one_error(tmp_path, monkeypatch):
|
||||
store = Store(tmp_path / "state.db")
|
||||
store.upsert_channel("UC1", "@one", "One", 0)
|
||||
store.upsert_channel("UC2", "@two", "Two", 0)
|
||||
store.create_job("discover-all", None, {"mode": "discover"})
|
||||
|
||||
def fake_discover(url, sleep_subrequests=2.0):
|
||||
if "@one" in url:
|
||||
raise RuntimeError("temporary failure")
|
||||
return ("UC2", "Two", None, [VideoRef("new2", "UC2", "New", "https://y/watch?v=new2")])
|
||||
|
||||
monkeypatch.setattr("yt_scraper.webapp.jobs.discover_channel", fake_discover)
|
||||
manager = JobManager(store, Config(database_path=str(tmp_path / "state.db"), output_dir=str(tmp_path / "markdown")))
|
||||
|
||||
manager._run_job("discover-all")
|
||||
|
||||
assert store.get_video("new2").status == "pending"
|
||||
assert store.get_job("discover-all").status == "done"
|
||||
done = [e for e in manager.events_since("discover-all", 0) if e["event"] == "done"][-1]
|
||||
assert done["data"]["errors"] == 1
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Run the focused tests to verify they fail**
|
||||
|
||||
Run: `python -m pytest tests/test_webapp_jobs.py::test_discovery_job_registers_new_videos_without_processing tests/test_webapp_jobs.py::test_all_channel_discovery_continues_after_one_error -q`
|
||||
|
||||
Expected: FAIL because `_run_job` currently routes both cases to `_run_channel`, which tries to process pending videos and cannot interpret an all-channel discovery job.
|
||||
|
||||
- [ ] **Step 3: Implement discovery dispatch and execution**
|
||||
|
||||
In `JobManager._run_job`, route `opts.get("mode") == "discover"` to a new `_run_discovery(job_id)` before the existing audio/video/channel branches.
|
||||
|
||||
Implement `_run_discovery` with this behavior:
|
||||
|
||||
```python
|
||||
def _run_discovery(self, job_id: str) -> None:
|
||||
job = self.store.get_job(job_id)
|
||||
if not job:
|
||||
return
|
||||
channels = ([self.store.get_channel(job.channel_id)] if job.channel_id
|
||||
else self.store.list_channels())
|
||||
channels = [c for c in channels if c]
|
||||
self.store.update_job(job_id, status="running", total=len(channels), completed=0)
|
||||
completed = 0
|
||||
totals = {"new_videos": 0, "known_videos": 0, "errors": 0}
|
||||
for channel in channels:
|
||||
if job_id in self._cancel:
|
||||
self.store.update_job(job_id, status="cancelled", completed=completed, finished=True)
|
||||
self._emit(job_id, "cancelled", {"completed": completed, "total": len(channels)})
|
||||
return
|
||||
try:
|
||||
# Build the stored handle/channel URL, call discover_channel, filter
|
||||
# using cfg.include_shorts/cfg.include_live, upsert the channel and refs.
|
||||
# Do not call deep_channel_avatar, cache helpers, or process_video.
|
||||
new_count = self.store.upsert_videos(refs)
|
||||
known_count = len({r.video_id for r in refs}) - new_count
|
||||
totals["new_videos"] += new_count
|
||||
totals["known_videos"] += known_count
|
||||
self._emit(job_id, "progress", {"channel_id": channel["channel_id"], "new_videos": new_count, "known_videos": known_count, "completed": completed + 1, "total": len(channels)})
|
||||
except Exception as exc:
|
||||
totals["errors"] += 1
|
||||
self._emit(job_id, "log", {"msg": f"discovery failed for {channel.get('name') or channel['channel_id']}: {exc}"})
|
||||
completed += 1
|
||||
self.store.update_job(job_id, completed=completed)
|
||||
self.store.update_job(job_id, status="done", completed=completed, finished=True)
|
||||
self._emit(job_id, "done", {"completed": completed, "total": len(channels), **totals})
|
||||
```
|
||||
|
||||
Use the existing `_resolve_channel_url` for each stored channel. Preserve the existing processing path untouched. For a single-channel exception, emit an error terminal event/status; for all-channel jobs, continue and report the error count as above.
|
||||
|
||||
- [ ] **Step 4: Run focused job tests**
|
||||
|
||||
Run: `python -m pytest tests/test_webapp_jobs.py -q`
|
||||
|
||||
Expected: PASS for both existing audio tests and the new discovery tests.
|
||||
|
||||
- [ ] **Step 5: Run the full Python suite**
|
||||
|
||||
Run: `python -m pytest tests/ -q`
|
||||
|
||||
Expected: all tests pass with no network access.
|
||||
|
||||
### Task 3: Expose the discovery mode through the API
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/yt_scraper/webapp/api.py:111-117`
|
||||
- Test: `tests/test_webapp_jobs.py` (job boundary coverage; no API test fixture exists in the repository).
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: JSON `{"channel_id": "UC...", "opts": {"mode": "discover"}}` or the same body without `channel_id` for all channels.
|
||||
- Produces: the existing `{"job_id": "..."}` response and a queued `JobManager` job.
|
||||
|
||||
- [ ] **Step 1: Add request-shape coverage at the job boundary**
|
||||
|
||||
Extend the discovery job tests to enqueue `manager.enqueue(None, {"mode": "discover"})` and verify that it completes as an all-channel job. This covers the exact payload shape the API forwards; the repository has no existing FastAPI router test fixture.
|
||||
|
||||
- [ ] **Step 2: Implement the minimal API change**
|
||||
|
||||
Keep the endpoint response unchanged. Normalize discovery options and permit a missing channel only for discovery:
|
||||
|
||||
```python
|
||||
opts = (payload or {}).get("opts", {}) or {}
|
||||
if opts.get("mode") == "discover":
|
||||
opts = {"mode": "discover"}
|
||||
job_id = jobs.enqueue(channel_id, opts)
|
||||
return {"job_id": job_id}
|
||||
```
|
||||
|
||||
Do not add a second synchronous discovery endpoint.
|
||||
|
||||
- [ ] **Step 3: Run the API/job regression tests**
|
||||
|
||||
Run: `python -m pytest tests/test_webapp_jobs.py tests/test_store_platform.py -q`
|
||||
|
||||
Expected: PASS.
|
||||
|
||||
### Task 4: Add one-channel and all-channel UI controls
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/yt_scraper/webapp/static/app.js:45-73, 758-815`
|
||||
- Modify: `src/yt_scraper/webapp/static/index.html:167-229, 231-319`
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `POST /api/scrape` discovery jobs and SSE `progress`/`done` payloads.
|
||||
- Produces: buttons for channel-specific and all-channel discovery, with refresh and result feedback.
|
||||
|
||||
- [ ] **Step 1: Add the failing static contract checks**
|
||||
|
||||
Before editing, use a lightweight repository check to confirm the new labels and method are absent:
|
||||
|
||||
Run: `rg "startDiscovery|Investigar nuevos|Investigar todos" src/yt_scraper/webapp/static`
|
||||
|
||||
Expected: no matches.
|
||||
|
||||
- [ ] **Step 2: Add the Alpine discovery action**
|
||||
|
||||
Add `startDiscovery(channelId)` near `startScrape()`. It posts `{channel_id: channelId || null, opts: {mode: "discover"}}`, subscribes with kind `discovery`, and prevents duplicate active jobs. Extend the SSE `done` handler to parse the payload and show `N video(s) nuevo(s) encontrado(s)` for discovery jobs. Refresh channels, videos, and dashboard after completion.
|
||||
|
||||
- [ ] **Step 3: Add controls to Channels and Videos**
|
||||
|
||||
In Channels, add the global header button and one row button calling `startDiscovery(c.channel_id)`. In Videos, add a header action calling `startDiscovery(filters.channel || null)`. Disable each while `jobActive()` is true and while the channel list is empty. Preserve `.md`, `Process`, and `Audio` actions.
|
||||
|
||||
- [ ] **Step 4: Verify the static contract and inspect the rendered paths**
|
||||
|
||||
Run: `rg "startDiscovery|Investigar nuevos|Investigar todos|mode: \"discover\"" src/yt_scraper/webapp/static`
|
||||
|
||||
Expected: matches in `app.js` and `index.html` for the method, payload, and both scopes. Inspect the changed Alpine expressions to ensure the existing `app.js` script remains before Alpine in `index.html`.
|
||||
|
||||
### Task 5: Final verification and review
|
||||
|
||||
**Files:**
|
||||
- Review: `src/yt_scraper/store.py`
|
||||
- Review: `src/yt_scraper/webapp/jobs.py`
|
||||
- Review: `src/yt_scraper/webapp/api.py`
|
||||
- Review: `src/yt_scraper/webapp/static/app.js`
|
||||
- Review: `src/yt_scraper/webapp/static/index.html`
|
||||
|
||||
- [ ] **Step 1: Run the complete test suite**
|
||||
|
||||
Run: `python -m pytest tests/ -q`
|
||||
|
||||
Expected: all tests pass.
|
||||
|
||||
- [ ] **Step 2: Review the diff for scope and unintended downloads**
|
||||
|
||||
Run: `git diff -- src/yt_scraper/store.py src/yt_scraper/webapp/jobs.py src/yt_scraper/webapp/api.py src/yt_scraper/webapp/static/app.js src/yt_scraper/webapp/static/index.html tests/test_store_platform.py tests/test_webapp_jobs.py`
|
||||
|
||||
Confirm discovery code contains no calls to `process_video`, `cache_thumbnail`, `cache_channel_avatar`, `extract_video`, or audio routes.
|
||||
|
||||
- [ ] **Step 3: Check worktree status**
|
||||
|
||||
Run: `git status --short`
|
||||
|
||||
Confirm only intended feature files and the two planning documents are changed or untracked; do not modify or revert unrelated existing user changes.
|
||||
@@ -0,0 +1,250 @@
|
||||
# Vista Grid estilo YouTube (sección Vídeos) — Implementation Plan
|
||||
|
||||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||||
|
||||
**Goal:** Añadir a la sección Vídeos un conmutador tabla ⇄ grid donde el grid replica el look de YouTube (tarjetas 16:9 con miniatura, duración, título y metadatos) manteniendo filtros, orden, selección masiva y paginación compartidos.
|
||||
|
||||
**Architecture:** Solo frontend. El estado `videos.view` (`'table'|'grid'`) vive en el componente Alpine único `window.platform()`; la elección se persiste en `localStorage["videos-view"]`. En `index.html` los dos layouts son bloques hermanos alternados con `<template x-if>`; el toggle es una fila fina sobre ellos.
|
||||
|
||||
**Tech Stack:** Alpine 3 + Tailwind CDN (sin build step), estáticos servidos por FastAPI.
|
||||
|
||||
**Spec:** [docs/superpowers/specs/2026-08-22-videos-grid-view-design.md](../specs/2026-08-22-videos-grid-view-design.md)
|
||||
|
||||
## Global Constraints
|
||||
|
||||
- Sin build step: todo cambio es HTML/JS/CSS estático editado a mano ([.opencode/agent/webapp-builder.md](../../.opencode/agent/webapp-builder.md)).
|
||||
- Todo el estado vive en **un solo** componente Alpine, `window.platform()` en `src/yt_scraper/webapp/static/app.js`; no crear componentes nuevos.
|
||||
- El orden de `<script defer>` en `index.html` (app.js antes de alpinejs) es carga funcional: no reordenar scripts.
|
||||
- No tocar backend, endpoints ni `store.py`: `/api/videos` ya devuelve todos los campos que necesita la tarjeta.
|
||||
- Sin tests automatizados de frontend (el repo no tiene infra JS): verificación = `node --check` + smoke manual con `start-server.bat`.
|
||||
- Estilo visual existente: dark "command center", `glass`, `accent-grad btn-primary` como tratamiento activo (mismo patrón que la paginación).
|
||||
- Sin acciones por tarjeta (`.md`/`Clip` quedan en tabla y detalle).
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Estado y persistencia en app.js
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/yt_scraper/webapp/static/app.js` (línea 39 store `videos`; `init()` líneas 92-93; método nuevo junto a los helpers de selección ~línea 463)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: nada nuevo.
|
||||
- Produces: `this.videos.view` (`'table' | 'grid'`, default `'table'`) y `setVideoView(v)` — el HTML del Task 2/3 los usa vía `videos.view` / `@click="setVideoView('grid')"`.
|
||||
|
||||
- [ ] **Step 1: Añadir `view` al store**
|
||||
|
||||
En `app.js:39`, cambiar:
|
||||
|
||||
```js
|
||||
videos: { items: [], total: 0, page: 1, size: 25, selected: [] },
|
||||
```
|
||||
|
||||
por:
|
||||
|
||||
```js
|
||||
videos: { items: [], total: 0, page: 1, size: 25, selected: [], view: "table" },
|
||||
```
|
||||
|
||||
- [ ] **Step 2: Restaurar la elección en init()**
|
||||
|
||||
En `app.js` dentro de `init()`, justo después del bloque:
|
||||
|
||||
```js
|
||||
const size = Number(localStorage.getItem("videos-size"));
|
||||
if ([10, 25, 50, 100].includes(size)) this.videos.size = size;
|
||||
```
|
||||
|
||||
añadir:
|
||||
|
||||
```js
|
||||
const savedView = localStorage.getItem("videos-view");
|
||||
if (savedView === "table" || savedView === "grid") this.videos.view = savedView;
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Añadir setVideoView()**
|
||||
|
||||
En `app.js`, inmediatamente antes de `toggleSelect(id) {` (la sección de helpers de selección, ~línea 463), añadir:
|
||||
|
||||
```js
|
||||
// View mode of the Videos section ('table' | 'grid'). UI preference like
|
||||
// videos-size: persisted in localStorage, never in the URL.
|
||||
setVideoView(v) {
|
||||
if (v !== "table" && v !== "grid") return;
|
||||
this.videos.view = v;
|
||||
try { localStorage.setItem("videos-view", v); } catch (_) {}
|
||||
},
|
||||
|
||||
```
|
||||
|
||||
- [ ] **Step 4: Verificar sintaxis JS**
|
||||
|
||||
Run: `node --check src/yt_scraper/webapp/static/app.js`
|
||||
Expected: sin salida (exit 0). Si `node` no está disponible, verificar abriendo la webapp en el Task 4.
|
||||
|
||||
- [ ] **Step 5: Commit**
|
||||
|
||||
```bash
|
||||
git add src/yt_scraper/webapp/static/app.js
|
||||
git commit -m "feat(webapp): estado y persistencia de vista tabla/grid en Videos"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Conmutador de vista + alternancia x-if en index.html
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/yt_scraper/webapp/static/index.html` (contenedor de resultados, líneas 301-352)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `videos.view` y `setVideoView(v)` del Task 1.
|
||||
- Produces: la estructura de dos bloques `<template x-if>` que el Task 3 completa con el grid (este task deja la tabla funcional tal cual).
|
||||
|
||||
- [ ] **Step 1: Envolver la tabla en template x-if**
|
||||
|
||||
En `index.html`, la tabla vive en:
|
||||
|
||||
```html
|
||||
<div class="glass overflow-hidden">
|
||||
<table class="tbl">
|
||||
```
|
||||
|
||||
(que cierra con `</table>` + `</div>` en las líneas ~351-352, justo antes del comentario `<!-- pagination -->`). Envolver ese `<div class="glass overflow-hidden">…</div>` completo en:
|
||||
|
||||
```html
|
||||
<template x-if="videos.view==='table'">
|
||||
<div class="glass overflow-hidden">
|
||||
… contenido actual de la tabla SIN CAMBIOS …
|
||||
</div>
|
||||
</template>
|
||||
```
|
||||
|
||||
Reindentar una nivel el interior. Ningún atributo ni clase de la tabla cambia.
|
||||
|
||||
- [ ] **Step 2: Insertar la fila del toggle**
|
||||
|
||||
Justo encima del `<template x-if="videos.view==='table'">` recién creado, añadir:
|
||||
|
||||
```html
|
||||
<!-- table ⇄ grid switch -->
|
||||
<div class="flex justify-end">
|
||||
<div class="inline-flex rounded-lg border border-zinc-800 bg-zinc-900/60 p-1 gap-1" role="group" aria-label="View mode">
|
||||
<button type="button" class="btn !py-1 !px-2" :class="videos.view==='table' ? 'accent-grad btn-primary' : 'btn-ghost'" :aria-pressed="(videos.view==='table').toString()" @click="setVideoView('table')" title="Table view">
|
||||
<svg class="w-4 h-4" viewBox="0 0 24 24" fill="currentColor"><path d="M3 5h18v2H3zm0 6h18v2H3zm0 6h18v2H3z"/></svg>
|
||||
</button>
|
||||
<button type="button" class="btn !py-1 !px-2" :class="videos.view==='grid' ? 'accent-grad btn-primary' : 'btn-ghost'" :aria-pressed="(videos.view==='grid').toString()" @click="setVideoView('grid')" title="Grid view">
|
||||
<svg class="w-4 h-4" viewBox="0 0 24 24" fill="currentColor"><path d="M3 3h8v8H3zm10 0h8v8h-8zM3 13h8v8H3zm10 0h8v8h-8z"/></svg>
|
||||
</button>
|
||||
</div>
|
||||
</div>
|
||||
```
|
||||
|
||||
- [ ] **Step 3: Smoke manual**
|
||||
|
||||
Run: `start-server.bat`, abrir Vídeos.
|
||||
Expected: la tabla se ve idéntica a antes; el toggle aparece arriba a la derecha; pulsar el icono de grid NO cambia nada visible todavía pero el botón grid queda activo (accent), se refresca la página y sigue activo (localStorage); volver a tabla también persiste.
|
||||
|
||||
- [ ] **Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add src/yt_scraper/webapp/static/index.html
|
||||
git commit -m "feat(webapp): conmutador de vista tabla/grid en Videos"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Bloque grid de tarjetas estilo YouTube
|
||||
|
||||
**Files:**
|
||||
- Modify: `src/yt_scraper/webapp/static/index.html` (insertar tras el cierre `</template>` del bloque tabla, antes de `<!-- pagination -->`)
|
||||
|
||||
**Interfaces:**
|
||||
- Consumes: `videos.view`, `setVideoView` (Task 1); helpers existentes de `app.js`: `ts(sec)`, `fmtNum(n)`, `chanName(id)`, `videoDate(v)`, `videoDateTitle(v)`, `statusClass(s)`, `isBlocked(v)`, `blockLabel(v)`, `blockTitle(v)`, `isSelected(id)`, `toggleSelect(id)`, `openVideo(id)`, `loading.videos`, `videos.items`.
|
||||
- Produces: la vista grid completa (fin de feature).
|
||||
|
||||
- [ ] **Step 1: Insertar el bloque grid**
|
||||
|
||||
Entre el `</template>` que cierra el bloque tabla y `<!-- pagination -->`, añadir:
|
||||
|
||||
```html
|
||||
<template x-if="videos.view==='grid'">
|
||||
<div>
|
||||
<template x-if="loading.videos"><div class="text-center text-zinc-500 py-10">loading…</div></template>
|
||||
<template x-if="!loading.videos && videos.items.length===0"><div class="text-center text-zinc-500 py-10"><div>No videos match these filters.</div><button class="btn btn-ghost mt-3" @click.stop="filters={ channel:'', status:'', from:'', to:'', min_dur:'', q:'', sort:'upload_date' }; loadVideos(1)">Clear filters</button></div></template>
|
||||
<template x-if="!loading.videos && videos.items.length>0">
|
||||
<div class="grid grid-cols-2 sm:grid-cols-3 lg:grid-cols-4 2xl:grid-cols-5 gap-4">
|
||||
<template x-for="v in videos.items" :key="v.video_id">
|
||||
<div class="glass group relative cursor-pointer hover:border-rose-500/40 transition-colors overflow-hidden"
|
||||
:class="isSelected(v.video_id) ? 'row-selected' : ''"
|
||||
role="button" tabindex="0"
|
||||
@click="openVideo(v.video_id)"
|
||||
@keydown.enter.prevent="openVideo(v.video_id)">
|
||||
<div class="relative aspect-video bg-zinc-900">
|
||||
<img class="absolute inset-0 w-full h-full object-cover" :src="'/api/thumbnails/'+v.video_id" :alt="v.title" onerror="this.style.visibility='hidden'" />
|
||||
<span x-show="v.duration" class="absolute bottom-1.5 right-1.5 font-mono text-[0.7rem] leading-none px-1.5 py-1 rounded bg-black/80 text-white" x-text="ts(v.duration)"></span>
|
||||
<button type="button" @click.stop="toggleSelect(v.video_id)"
|
||||
class="absolute top-1.5 left-1.5 w-6 h-6 rounded-md border flex items-center justify-center transition-opacity"
|
||||
:class="isSelected(v.video_id) ? 'bg-rose-500 border-rose-400 opacity-100' : 'bg-black/70 border-zinc-300/70 opacity-0 group-hover:opacity-100'"
|
||||
:aria-pressed="isSelected(v.video_id).toString()" title="Select video">
|
||||
<svg x-show="isSelected(v.video_id)" class="w-4 h-4 text-white" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="3"><path stroke-linecap="round" stroke-linejoin="round" d="M5 13l4 4L19 7"/></svg>
|
||||
</button>
|
||||
</div>
|
||||
<div class="p-3 space-y-1.5">
|
||||
<div class="text-sm font-medium text-zinc-100 line-clamp-2 leading-snug" x-text="v.title" :title="v.title"></div>
|
||||
<div class="text-xs text-zinc-400 truncate" x-text="chanName(v.channel_id)" :title="chanName(v.channel_id)"></div>
|
||||
<div class="flex items-center gap-2 font-mono text-xs text-zinc-400 flex-wrap">
|
||||
<span x-text="fmtNum(v.view_count)"></span>
|
||||
<span class="text-zinc-600">·</span>
|
||||
<span :class="v.upload_date ? '' : 'italic text-zinc-600'" x-text="videoDate(v)" :title="videoDateTitle(v)"></span>
|
||||
</div>
|
||||
<div class="flex items-center gap-1 flex-wrap pt-0.5">
|
||||
<span class="pill" :class="statusClass(v.status)" x-text="v.status" :title="v.error_msg || v.status"></span>
|
||||
<span x-show="isBlocked(v)" class="pill st-locked" :title="blockTitle(v)">
|
||||
<svg class="w-3 h-3" viewBox="0 0 24 24" fill="none" stroke="currentColor" stroke-width="2.5" aria-hidden="true"><rect x="4" y="10" width="16" height="10" rx="2"/><path d="M8 10V7a4 4 0 1 1 8 0v3"/></svg>
|
||||
<span x-text="blockLabel(v)"></span>
|
||||
</span>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
</template>
|
||||
</div>
|
||||
</template>
|
||||
</div>
|
||||
</template>
|
||||
```
|
||||
|
||||
Notas de diseño ya decididas:
|
||||
- `.row-selected` existe en styles.css:179 con `!important` → aplica igual sobre la tarjeta div que sobre el `<tr>`.
|
||||
- `line-clamp-2` ya se usa en index.html:553 → Tailwind CDN lo resuelve.
|
||||
- Checkbox estilo YouTube: aparece al hover (`group-hover`) o permanece si está seleccionada; click con `.stop` para no abrir el detalle.
|
||||
- La selección masiva, barra bulk, paginación y filtros NO se tocan: viven fuera de estos bloques.
|
||||
|
||||
- [ ] **Step 2: Verificación manual completa (checklist del spec)**
|
||||
|
||||
Run: `start-server.bat`
|
||||
|
||||
1. Alternar tabla ⇄ grid: ambas muestran los mismos vídeos con el filtro activo.
|
||||
2. En grid: pasar el ratón sobre una tarjeta → checkbox visible; marcar 3 → barra bulk aparece con "3 selected" → "Download .md" encola el job.
|
||||
3. Click en tarjeta (fuera del checkbox) → abre el detalle; Back → vuelve a grid conservando página.
|
||||
4. Paginar en grid: Prev/Next/números funcionan, scroll arriba, sin repeticiones.
|
||||
5. Recargar página: la vista elegida se conserva; borrar `localStorage["videos-view"]` → cae a tabla.
|
||||
6. Tarjeta de vídeo blocked: pill candado con label; badge de duración visible en tarjetas con duración.
|
||||
7. Miniatura rota: `onerror` la oculta sin romper el layout (contenedor aspect-video mantiene proporción).
|
||||
8. La tabla sigue funcionando exactamente igual que antes.
|
||||
|
||||
Run: `python -m pytest tests/ -q`
|
||||
Expected: suite verde (backend intacto; sanity check barato).
|
||||
|
||||
- [ ] **Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add src/yt_scraper/webapp/static/index.html
|
||||
git commit -m "feat(webapp): grid de tarjetas estilo YouTube en Videos"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Self-review hecho
|
||||
|
||||
- **Cobertura del spec:** estado+persistencia (Task 1), toggle (Task 2), tarjetas/clamps/badges/pills/loading/empty (Task 3), verificación manual ítem a ítem (Task 3 Step 2). Fuera de alcance respetado: cero cambios backend, sin acciones por tarjeta.
|
||||
- **Placeholders:** ninguno; todo step lleva código literal o comando exacto.
|
||||
- **Consistencia de nombres:** `videos.view`, `setVideoView`, `"videos-view"` usados igual en Tasks 1-3; helpers referenciados existen en app.js (verificado: ts:1265, fmtNum:1293, videoDate:1317, videoDateTitle:1324, chanName:1345, blockLabel:1355, blockTitle:1365, isBlocked:1374, statusClass:1376, toggleSelect:464, isSelected:469).
|
||||
@@ -0,0 +1,47 @@
|
||||
# Discovery-Only Scrape Design
|
||||
|
||||
## Goal
|
||||
|
||||
Add manual discovery controls for one channel or all tracked channels. Discovery checks the complete recent video list returned by YouTube, registers unknown videos as `pending`, and does not download transcripts, Markdown, audio, thumbnails, or avatars.
|
||||
|
||||
## Existing Context
|
||||
|
||||
The webapp already has a single-worker `JobManager`, SSE job events, and a `POST /api/scrape` endpoint. The current channel job combines discovery with `pipeline.process_video()`, so every pending video is immediately extracted and rendered. The channels table has per-channel actions, while the videos view can be filtered to one channel.
|
||||
|
||||
## Architecture
|
||||
|
||||
- Extend the existing job dispatch with `opts.mode == "discover"`.
|
||||
- `channel_id` selects one tracked channel. A missing `channel_id` means all tracked channels for discovery jobs only.
|
||||
- Discovery uses the existing `discover_channel()` call without a playlist cap. It compares the returned IDs with the SQLite catalog and upserts only catalog metadata. It never calls `process_video()`.
|
||||
- The existing scrape mode remains unchanged for users who want discovery plus transcript/Markdown processing.
|
||||
- All discovery jobs stay in the existing sequential queue and use the existing SSE stream, cancellation, progress, and history.
|
||||
|
||||
## Persistence and Results
|
||||
|
||||
- New `VideoRef` rows are inserted with the existing default status `pending`.
|
||||
- Existing video rows retain their status and processed data; rediscovery only refreshes title, upload date, and duration through the existing upsert behavior.
|
||||
- The channel record is refreshed with its name, handle, catalog count, and `last_scraped`; no avatar fetch/cache is performed by discovery-only jobs.
|
||||
- A per-channel progress event reports `new_videos`, `known_videos`, and the channel ID.
|
||||
- The terminal event reports total channels, discovered entries, new videos, known videos, and errors.
|
||||
- A one-channel discovery failure marks the job `error`. An all-channel job continues after individual failures and finishes with a summary so one broken channel does not prevent other channels from being scanned.
|
||||
|
||||
## UI
|
||||
|
||||
- Channels view: add `Investigar todos` in the header and `Investigar` in each channel row.
|
||||
- Videos view: add an investigation button in the header. It investigates the selected channel when `filters.channel` is set, otherwise all channels.
|
||||
- Buttons use the existing job widget/SSE stream, are disabled while an active job is being followed, and show the number of new videos on completion.
|
||||
- Completion refreshes channels, videos, and dashboard data. Existing `.md`, `Process`, and `Audio` controls remain separate.
|
||||
|
||||
## Error Handling
|
||||
|
||||
- Empty channel catalogs complete successfully with zero channels scanned.
|
||||
- Discovery errors are emitted in the job log and do not invoke video processing.
|
||||
- A cancellation checks the existing cancellation set between channels and leaves already inserted catalog rows intact.
|
||||
- Duplicate IDs from a discovery response are counted once by the catalog comparison.
|
||||
|
||||
## Testing
|
||||
|
||||
- Store tests verify that upserting an existing and a new reference reports only the new reference.
|
||||
- Job tests fake `discover_channel()`, verify new rows are `pending`, existing rows retain their status, and `process_video()` is never called.
|
||||
- Job tests verify all-channel discovery continues after one channel fails and reports the successful channel.
|
||||
- The full existing Python test suite remains the regression check. The static UI is verified by code inspection and the existing webapp smoke path; no frontend build step exists.
|
||||
Reference in New Issue
Block a user