docs: spec+plan fechas aproximadas en discovery

2026-08-22 14:42:47 -06:00
parent b04a2710cf
commit 8940c325ea
2 changed files with 271 additions and 0 deletions
@@ -0,0 +1,195 @@
+# Fechas aproximadas en discovery Implementation Plan
+
+> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
+
+**Goal:** Llenar `videos.upload_date` con fechas aproximadas (flag `upload_date_approx`) durante discovery/sync a costo cero, con upgrade automático a real al extraer.
+
+**Architecture:** `youtubetab:approximate_date` en los ydl_opts de discovery → entries flat traen `timestamp` → helper puro convierte a `YYYYMMDD` + flag. `upsert_videos` gana matriz de precedencia; `set_upload_date` hace el upgrade approx→real. UI marca `~`.
+
+**Tech Stack:** Python 3.12, SQLite sin ORM, pytest con tmp_path.
+
+**Spec:** `docs/superpowers/specs/2026-08-22-approximate-upload-dates-design.md`
+
+## Global Constraints
+
+- 0 peticiones HTTP nuevas por design; no tocar economía de requests.
+- Migración SOLO vía dict `_VIDEO_COLUMNS` (patrón del repo).
+- `NO_DATE_SENTINEL` y cascada `SORT_DATE_SQL` intocables.
+- Tests sin red: helpers puros y Store(tmp_path).
+
+---
+
+### Task 1: store.py — columna, bandera y matriz de upsert
+
+**Files:**
+- Modify: `src/yt_scraper/store.py` (`_VIDEO_COLUMNS` ~44, `VideoRef` ~251, `VideoRow` ~268, `upsert_videos` ~500, `set_upload_date` ~756, `_row_to_videorow` ~1146)
+- Test: `tests/test_approx_dates.py` (nuevo)
+
+**Interfaces:**
+- Produces: `VideoRef.date_approx: int = 0`; `VideoRow.upload_date_approx: int | None = None`; semántica de upsert según spec.
+
+- [ ] **Step 1: test que falla** — crear `tests/test_approx_dates.py` con la matriz completa (4 casos + upgrade). Ver patrón de construcción de Store en tests/test_store_platform.py.
+
+```python
+from yt_scraper.store import Store, VideoRef
+
+def _ref(vid, date=None, approx=0, ch="ch1"):
+    return VideoRef(video_id=vid, channel_id=ch, title="t " + vid,
+                    url=f"https://youtu.be/{vid}", upload_date=date, date_approx=approx)
+
+def _store(tmp_path):
+    return Store(tmp_path)
+
+def test_approx_into_empty(tmp_path):
+    s = _store(tmp_path); s.upsert_videos([_ref("v1", date="20260820", approx=1)])
+    row = s.get_video("v1")
+    assert row.upload_date == "20260820" and row.upload_date_approx == 1
+
+def test_real_not_downgraded_by_approx(tmp_path):
+    s = _store(tmp_path)
+    s.upsert_videos([_ref("v1", date="20240101", approx=0)])
+    s.upsert_videos([_ref("v1", date="20260820", approx=1)])
+    row = s.get_video("v1")
+    assert row.upload_date == "20240101" and row.upload_date_approx == 0
+
+def test_approx_refreshed_by_newer_approx(tmp_path):
+    s = _store(tmp_path)
+    s.upsert_videos([_ref("v1", date="20260101", approx=1)])
+    s.upsert_videos([_ref("v1", date="20260820", approx=1)])
+    row = s.get_video("v1")
+    assert row.upload_date == "20260820" and row.upload_date_approx == 1
+
+def test_extraction_upgrades_approx_to_real(tmp_path):
+    s = _store(tmp_path)
+    s.upsert_videos([_ref("v1", date="20260820", approx=1)])
+    s.set_upload_date("v1", "20260818")
+    row = s.get_video("v1")
+    assert row.upload_date == "20260818" and row.upload_date_approx == 0
+
+def test_set_upload_date_still_skips_existing_real(tmp_path):
+    s = _store(tmp_path)
+    s.upsert_videos([_ref("v1", date="20240101", approx=0)])
+    s.set_upload_date("v1", "20260818")
+    assert s.get_video("v1").upload_date == "20240101"
+```
+
+(Adaptar nombres reales: constructor `Store`, getter `get_video`; verificar en el código antes.)
+
+- [ ] **Step 2: correr y ver fallo** — `python -m pytest tests/test_approx_dates.py -q` → FAIL (TypeError/AttributeError por falta de campos).
+
+- [ ] **Step 3: implementar mínimo**
+  - `_VIDEO_COLUMNS["upload_date_approx"] = "INTEGER DEFAULT 0"`
+  - `VideoRef`: `date_approx: int = 0`
+  - `VideoRow`: `upload_date_approx: int | None = None` (antes de sort_date)
+  - `_row_to_videorow`: `upload_date_approx=row["upload_date_approx"] if "upload_date_approx" in keys else None`
+  - `upsert_videos`: INSERT incluye columna+valor; ON CONFLICT:
+    ```sql
+    upload_date = CASE
+        WHEN excluded.upload_date IS NULL THEN videos.upload_date
+        WHEN videos.upload_date IS NULL THEN excluded.upload_date
+        WHEN videos.upload_date_approx = 1 THEN excluded.upload_date
+        ELSE videos.upload_date END,
+    upload_date_approx = CASE
+        WHEN videos.upload_date IS NOT NULL AND COALESCE(videos.upload_date_approx, 0) = 0
+            THEN videos.upload_date_approx
+        ELSE COALESCE(excluded.upload_date_approx, videos.upload_date_approx) END,
+    ```
+  - `set_upload_date`:
+    ```sql
+    UPDATE videos SET upload_date = ?, upload_date_approx = 0
+    WHERE video_id = ? AND (upload_date IS NULL OR COALESCE(upload_date_approx, 0) = 1)
+    ```
+
+- [ ] **Step 4: tests pasan** — mismo comando → PASS. Suite completa verde.
+
+- [ ] **Step 5: commit** — `feat: bandera upload_date_approx + matriz de precedencia en upsert`
+
+### Task 2: discover.py — extractor arg y conversión timestamp
+
+**Files:**
+- Modify: `src/yt_scraper/discover.py` (~31 opts, ~62 loop)
+- Test: `tests/test_approx_dates.py` (añadir)
+
+**Interfaces:**
+- Consumes: `VideoRef.date_approx` (Task 1).
+- Produces: `_entry_upload_date(entry) -> tuple[str | None, int]`.
+
+- [ ] **Step 1: test del helper que falla**
+
+```python
+from yt_scraper.discover import _entry_upload_date
+
+def test_entry_upload_date_prefers_exact():
+    assert _entry_upload_date({"upload_date": "20240101"}) == ("20240101", 0)
+
+def test_entry_upload_date_from_timestamp():
+    # 2024-05-19 12:00 UTC
+    ts = 1716115200
+    assert _entry_upload_date({"timestamp": ts}) == ("20240519", 1)
+
+def test_entry_upload_date_none():
+    assert _entry_upload_date({}) == (None, 0)
+```
+
+- [ ] **Step 2: fallo** — ImportError.
+
+- [ ] **Step 3: implementar**
+
+```python
+def _entry_upload_date(entry: dict[str, Any]) -> tuple[str | None, int]:
+    exact = entry.get("upload_date")
+    if exact:
+        return str(exact), 0
+    ts = entry.get("timestamp")
+    if not ts:
+        return None, 0
+    from datetime import datetime, timezone
+    return datetime.fromtimestamp(int(ts), tz=timezone.utc).strftime("%Y%m%d"), 1
+```
+
+En `discover_channel`: añadir `"extractor_args": {"youtubetab": {"approximate_date": ["true"]}}` a ydl_opts; loop usa `(upload_date, approx) = _entry_upload_date(entry)` y pasa `date_approx=approx` a VideoRef. Import de datetime a nivel módulo.
+
+- [ ] **Step 4: pasa** + suite verde. **[ ] Step 5: commit** — `feat: discovery llena fechas aproximadas (approximate_date)`
+
+### Task 3: API + UI — marca visual
+
+**Files:**
+- Modify: `src/yt_scraper/webapp/api.py` (`_video_dict` ~628)
+- Modify: `src/yt_scraper/webapp/static/app.js` (`videoDateText` ~1266, tooltip ~1274)
+
+**Interfaces:**
+- Consumes: `VideoRow.upload_date_approx`.
+- Produces: JSON `upload_date_approx: bool`, `date_estimated` ampliado.
+
+- [ ] **Step 1: implementar** (sin test automatizado de UI; verificación manual tras reinicio)
+
+```python
+"upload_date_approx": bool(v.upload_date_approx),
+"date_estimated": bool(
+    (v.sort_date and not v.upload_date and v.sort_date != store_mod.NO_DATE_SENTINEL)
+    or v.upload_date_approx
+),
+```
+
+app.js videoDateText:
+
+```js
+if (v.upload_date && !v.upload_date_approx) return this.fmtDate(v.upload_date);
+if (v.upload_date) return "~ " + this.fmtDate(v.upload_date);
+if (v.sort_date) return "~ " + this.fmtDate(v.sort_date);
+```
+
+Tooltip nuevo ANTES del branch date_estimated existente:
+
+```js
+if (v.upload_date_approx) return "Approximate upload date — YouTube shows relative text ('3 weeks ago') in listings. Download the .md to learn the exact one.";
+```
+
+- [ ] **Step 2: suite verde** (`python -m pytest tests/ -q`). **[ ] Step 3: commit** — `feat: UI marca fechas aproximadas con ~ y tooltip propio`
+
+### Task 4: verificación end-to-end
+
+- [ ] **Step 1:** `python -m pytest tests/ -q` completo verde.
+- [ ] **Step 2:** reiniciar server (stop-server.bat + doctor) para servir api.py nueva.
+- [ ] **Step 3:** smoke `/api/videos?limit=5` responde 200 con campo nuevo presente.
+- [ ] **Step 4:** informe final; sugerir al usuario un Full rescan por canal para llenar backlog (acción manual suya, no automática).