yt-channel-scraper/docs/superpowers/plans/2026-08-22-approximate-upload-dates.md

Fechas aproximadas en discovery Implementation Plan

For agentic workers: REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (- [ ]) syntax for tracking.

Goal: Llenar videos.upload_date con fechas aproximadas (flag upload_date_approx) durante discovery/sync a costo cero, con upgrade automático a real al extraer.

Architecture: youtubetab:approximate_date en los ydl_opts de discovery → entries flat traen timestamp → helper puro convierte a YYYYMMDD + flag. upsert_videos gana matriz de precedencia; set_upload_date hace el upgrade approx→real. UI marca ~.

Tech Stack: Python 3.12, SQLite sin ORM, pytest con tmp_path.

Spec: docs/superpowers/specs/2026-08-22-approximate-upload-dates-design.md

Global Constraints

• 0 peticiones HTTP nuevas por design; no tocar economía de requests.
• Migración SOLO vía dict _VIDEO_COLUMNS (patrón del repo).
• NO_DATE_SENTINEL y cascada SORT_DATE_SQL intocables.
• Tests sin red: helpers puros y Store(tmp_path).

---

Task 1: store.py — columna, bandera y matriz de upsert

Files:

• Modify: src/yt_scraper/store.py (_VIDEO_COLUMNS ~44, VideoRef ~251, VideoRow ~268, upsert_videos ~500, set_upload_date ~756, _row_to_videorow ~1146)
• Test: tests/test_approx_dates.py (nuevo)

Interfaces:

• Produces: VideoRef.date_approx: int = 0; VideoRow.upload_date_approx: int | None = None; semántica de upsert según spec.

• Step 1: test que falla — crear tests/test_approx_dates.py con la matriz completa (4 casos + upgrade). Ver patrón de construcción de Store en tests/test_store_platform.py.

from yt_scraper.store import Store, VideoRef

def _ref(vid, date=None, approx=0, ch="ch1"):
    return VideoRef(video_id=vid, channel_id=ch, title="t " + vid,
                    url=f"https://youtu.be/{vid}", upload_date=date, date_approx=approx)

def _store(tmp_path):
    return Store(tmp_path)

def test_approx_into_empty(tmp_path):
    s = _store(tmp_path); s.upsert_videos([_ref("v1", date="20260820", approx=1)])
    row = s.get_video("v1")
    assert row.upload_date == "20260820" and row.upload_date_approx == 1

def test_real_not_downgraded_by_approx(tmp_path):
    s = _store(tmp_path)
    s.upsert_videos([_ref("v1", date="20240101", approx=0)])
    s.upsert_videos([_ref("v1", date="20260820", approx=1)])
    row = s.get_video("v1")
    assert row.upload_date == "20240101" and row.upload_date_approx == 0

def test_approx_refreshed_by_newer_approx(tmp_path):
    s = _store(tmp_path)
    s.upsert_videos([_ref("v1", date="20260101", approx=1)])
    s.upsert_videos([_ref("v1", date="20260820", approx=1)])
    row = s.get_video("v1")
    assert row.upload_date == "20260820" and row.upload_date_approx == 1

def test_extraction_upgrades_approx_to_real(tmp_path):
    s = _store(tmp_path)
    s.upsert_videos([_ref("v1", date="20260820", approx=1)])
    s.set_upload_date("v1", "20260818")
    row = s.get_video("v1")
    assert row.upload_date == "20260818" and row.upload_date_approx == 0

def test_set_upload_date_still_skips_existing_real(tmp_path):
    s = _store(tmp_path)
    s.upsert_videos([_ref("v1", date="20240101", approx=0)])
    s.set_upload_date("v1", "20260818")
    assert s.get_video("v1").upload_date == "20240101"

(Adaptar nombres reales: constructor Store, getter get_video; verificar en el código antes.)

• Step 2: correr y ver fallo — python -m pytest tests/test_approx_dates.py -q → FAIL (TypeError/AttributeError por falta de campos).

• Step 3: implementar mínimo

  • _VIDEO_COLUMNS["upload_date_approx"] = "INTEGER DEFAULT 0"
  • VideoRef: date_approx: int = 0
  • VideoRow: upload_date_approx: int | None = None (antes de sort_date)
  • _row_to_videorow: upload_date_approx=row["upload_date_approx"] if "upload_date_approx" in keys else None
  • upsert_videos: INSERT incluye columna+valor; ON CONFLICT:

    upload_date = CASE
        WHEN excluded.upload_date IS NULL THEN videos.upload_date
        WHEN videos.upload_date IS NULL THEN excluded.upload_date
        WHEN videos.upload_date_approx = 1 THEN excluded.upload_date
        ELSE videos.upload_date END,
    upload_date_approx = CASE
        WHEN videos.upload_date IS NOT NULL AND COALESCE(videos.upload_date_approx, 0) = 0
            THEN videos.upload_date_approx
        ELSE COALESCE(excluded.upload_date_approx, videos.upload_date_approx) END,
    

  • set_upload_date:

    UPDATE videos SET upload_date = ?, upload_date_approx = 0
    WHERE video_id = ? AND (upload_date IS NULL OR COALESCE(upload_date_approx, 0) = 1)
    

• Step 4: tests pasan — mismo comando → PASS. Suite completa verde.

• Step 5: commit — feat: bandera upload_date_approx + matriz de precedencia en upsert

Task 2: discover.py — extractor arg y conversión timestamp

Files:

• Modify: src/yt_scraper/discover.py (~31 opts, ~62 loop)
• Test: tests/test_approx_dates.py (añadir)

Interfaces:

• Consumes: VideoRef.date_approx (Task 1).

• Produces: _entry_upload_date(entry) -> tuple[str | None, int].

• Step 1: test del helper que falla

from yt_scraper.discover import _entry_upload_date

def test_entry_upload_date_prefers_exact():
    assert _entry_upload_date({"upload_date": "20240101"}) == ("20240101", 0)

def test_entry_upload_date_from_timestamp():
    # 2024-05-19 12:00 UTC
    ts = 1716115200
    assert _entry_upload_date({"timestamp": ts}) == ("20240519", 1)

def test_entry_upload_date_none():
    assert _entry_upload_date({}) == (None, 0)

• Step 2: fallo — ImportError.

• Step 3: implementar

def _entry_upload_date(entry: dict[str, Any]) -> tuple[str | None, int]:
    exact = entry.get("upload_date")
    if exact:
        return str(exact), 0
    ts = entry.get("timestamp")
    if not ts:
        return None, 0
    from datetime import datetime, timezone
    return datetime.fromtimestamp(int(ts), tz=timezone.utc).strftime("%Y%m%d"), 1

En discover_channel: añadir "extractor_args": {"youtubetab": {"approximate_date": ["true"]}} a ydl_opts; loop usa (upload_date, approx) = _entry_upload_date(entry) y pasa date_approx=approx a VideoRef. Import de datetime a nivel módulo.

• Step 4: pasa + suite verde. [ ] Step 5: commit — feat: discovery llena fechas aproximadas (approximate_date)

Task 3: API + UI — marca visual

Files:

• Modify: src/yt_scraper/webapp/api.py (_video_dict ~628)
• Modify: src/yt_scraper/webapp/static/app.js (videoDateText ~1266, tooltip ~1274)

Interfaces:

• Consumes: VideoRow.upload_date_approx.

• Produces: JSON upload_date_approx: bool, date_estimated ampliado.

• Step 1: implementar (sin test automatizado de UI; verificación manual tras reinicio)

"upload_date_approx": bool(v.upload_date_approx),
"date_estimated": bool(
    (v.sort_date and not v.upload_date and v.sort_date != store_mod.NO_DATE_SENTINEL)
    or v.upload_date_approx
),

app.js videoDateText:

if (v.upload_date && !v.upload_date_approx) return this.fmtDate(v.upload_date);
if (v.upload_date) return "~ " + this.fmtDate(v.upload_date);
if (v.sort_date) return "~ " + this.fmtDate(v.sort_date);

Tooltip nuevo ANTES del branch date_estimated existente:

if (v.upload_date_approx) return "Approximate upload date — YouTube shows relative text ('3 weeks ago') in listings. Download the .md to learn the exact one.";

• Step 2: suite verde (python -m pytest tests/ -q). [ ] Step 3: commit — feat: UI marca fechas aproximadas con ~ y tooltip propio

Task 4: verificación end-to-end

• Step 1: python -m pytest tests/ -q completo verde.
• Step 2: reiniciar server (stop-server.bat + doctor) para servir api.py nueva.
• Step 3: smoke /api/videos?limit=5 responde 200 con campo nuevo presente.
• Step 4: informe final; sugerir al usuario un Full rescan por canal para llenar backlog (acción manual suya, no automática).