urieljareth/yt-channel-scraper

Fork 0

Files

T

urieljareth 8a59b39c98 wip: estado de trabajo pendiente antes de la vista grid (suite 230 verde)

2026-08-22 19:37:23 -06:00

16 KiB

Raw Blame History

Graph Report - yt-channel-scraper (2026-08-01)

Corpus Check

49 files · ~52,574 words
Verdict: corpus is large enough that graph structure adds value.

Summary

696 nodes · 1591 edges · 42 communities (37 shown, 5 thin omitted)
Extraction: 90% EXTRACTED · 10% INFERRED · 0% AMBIGUOUS · INFERRED: 163 edges (avg confidence: 0.78)
Token cost: 0 input · 0 output

Graph Freshness

Built from commit: 06497299
Run git rev-parse HEAD and compare to check if the graph is stale.
Run graphify update . after code changes (no API cost).

Webapp Frontend (JS)
Transcript Extraction & Parsing
Config & Discovery Pipeline
Store & Export Layer
CLI Command Layer
Platform Design & Proposals
Segments Search & Backfill
Store Platform Tests
Content Analysis
README & Config Docs
Server Start Script
Package Manifest
Server Stop Script
test_extract.py
Auditoría · Cómo la extensión obtiene la transcripción / información de vídeos de YouTube
api
pipeline.py
test_blocked_videos.py
config.py
setView
Discovery-Only Scrape Design
renderDashboardCharts
Global Constraints
Config
init
render.py
loadChannels
jobs.py
Segment
cookies.py
discover.py
export.py
pipeline.py
_channel_targets
test_features.py
VideoRow
test_since_filter.py
.reset_videos
._connect
re_render_cmd
handleFiles

God Nodes (most connected - your core abstractions)

Store - 103 edges
VideoRef - 43 edges
api() - 39 edges
toast() - 38 edges
Config - 36 edges
Segment - 32 edges
JobManager - 26 edges
discover_incremental() - 21 edges
process_video() - 19 edges
reconcile_markdown() - 19 edges

Surprising Connections (you probably didn't know these)

test_config_languages_defaults_to_dict() --calls--> Config [INFERRED] tests/test_extract.py → src/yt_scraper/config.py
test_config_prefer_manual_defaults_true() --calls--> Config [INFERRED] tests/test_extract.py → src/yt_scraper/config.py
test_export_json_csv() --calls--> Segment [INFERRED] tests/test_features.py → src/yt_scraper/parse.py
test_export_srt() --calls--> Segment [INFERRED] tests/test_features.py → src/yt_scraper/parse.py
test_term_timeline() --calls--> Segment [INFERRED] tests/test_features.py → src/yt_scraper/parse.py

Import Cycles

None detected.

Hyperedges (group relationships)

Workstream B webapp stack (FastAPI + Alpine SPA + subagent) — opencode_agent_webapp_builder, opencode_goals_webapp_build, src_yt_scraper_webapp_static_index, docs_superpowers_specs_2026_07_26_platform_design_dark_command_center, docs_superpowers_specs_2026_07_26_platform_design_sse_job_runner [EXTRACTED 0.95]
FTS5 transcript search flow (proposal -> spec -> SPA view) — opportunities_search_fts5, docs_superpowers_specs_2026_07_26_platform_design_transcript_fts5, src_yt_scraper_webapp_static_index [INFERRED 0.85]
Cookie auth chain (vault -> resolve_active_path -> scrape job) — docs_superpowers_specs_2026_07_26_platform_design_cookie_vault_ux, docs_superpowers_specs_2026_07_26_platform_design_sse_job_runner, docs_superpowers_specs_2026_07_26_platform_design_cookies_bug_fix, src_yt_scraper_webapp_static_index [INFERRED 0.85]

Communities (42 total, 5 thin omitted)

Community 0 - "Webapp Frontend (JS)"

Cohesion: 0.06 Nodes (22): _armAutoHide(), blockLabel(), blockTitle(), cancelScrape(), closeClip(), closeJobWidget(), closeSidebar(), closeStats() (+14 more)

Community 1 - "Transcript Extraction & Parsing"

Cohesion: 0.27 Nodes (10): _iter_texts(), Path, Return [(YYYY-MM, count)] of months where term appears in transcripts., render_timeline_chart(), render_top_words_chart(), render_wordcloud(), term_timeline(), _tokenize() (+2 more)

Community 2 - "Config & Discovery Pipeline"

Cohesion: 0.17 Nodes (10): _clone_config(), _handle(), JobManager, _keep_ref(), Any, Single-worker scrape job runner with an in-memory event log per job (SSE-polled), Refresh the catalog without extracting or downloading video content., Refresh one channel's catalog. Incremental by default: only the newest (+2 more)

Community 3 - "Store & Export Layer"

Cohesion: 0.10 Nodes (10): Cursor, _now_iso(), Any, Every video id already recorded for a channel — the boundary an incremen, Newest known upload date (YYYYMMDD) for a channel, or None., Refresh a channel's counters from what is actually stored. Incremental, Update only the fields explicitly passed. Preserves last_scraped and video_count, Set a video's status, optionally recording why. no_subtitles used to (+2 more)

Community 4 - "CLI Command Layer"

Cohesion: 0.12 Nodes (24): channels_add(), cli(), _extract_handle(), _fmt_date(), _fmt_duration(), _known_channel_for(), main(), _order_pending() (+16 more)

Community 5 - "Platform Design & Proposals"

Cohesion: 0.12 Nodes (20): Platform Implementation Plan, Platform Design Spec, Cookie drag-and-drop vault UX, cli.py cookies scope bug fix, Design language: dark data command center, Architecture: Monorepo in-package webapp (Approach A), Scope rule: No AI features, Single-threaded scrape job runner + SSE event bus (+12 more)

Community 6 - "Segments Search & Backfill"

Cohesion: 0.13 Nodes (17): APIRouter, FastAPI, cache_channel_avatar(), cache_thumbnail(), Thumbnail URL for a video: stored URL, else the canonical YouTube one derived fr, Download + cache a video's thumbnail to <thumbnails_dir>/<video_id>.jpg. Returns, Download + cache a channel avatar to <avatars_dir>/<channel_id>.jpg. Returns Tru, thumbnail_url_for() (+9 more)

Community 7 - "Store Platform Tests"

Cohesion: 0.22 Nodes (18): Collection, discover_incremental(), IncrementalDiscovery, Fetch only the newest slice of a channel instead of paginating all of it. A, Outcome of a windowed channel sync., _fake_channel(), Windowed channel sync: only fetch what is newer than what we already have. The, Shorts the store never recorded must not keep the window widening. (+10 more)

Community 8 - "Content Analysis"

Cohesion: 0.07 Nodes (52): Re-escanear data/markdown y hacer que la DB coincida con el disco., reconcile_cmd(), backfill_from_markdown(), parse_markdown(), ParsedMarkdown, Path, Parse all done .md files under md_root, populate segments/FTS/metadata. Idempote, Make the DB agree with what is actually on disk. backfill_from_markdown f (+44 more)

Community 10 - "Server Start Script"

Cohesion: 0.38 Nodes (3): Open-Browser(), Out-Line(), Show-State()

Community 14 - "test_extract.py"

Cohesion: 0.07 Nodes (50): Response, parse_languages(), Any, Normalise legacy list / new dict / None into {lang: mode}., _coerce_languages(), describe_missing_subtitle(), _download_subtitle(), extract_video() (+42 more)

Community 15 - "Auditoría · Cómo la extensión obtiene la transcripción / información de vídeos de YouTube"

Cohesion: 0.04 Nodes (42): Arquitectura, Comandos, Cookies, Cualquier petición HTTP a CDNs de YouTube va por _yt_http.yt_get(), Documentos de referencia, El Markdown es fuente de datos, no solo salida, Estados terminales y frescura (la UI tiene que reflejar DB + disco), Frontend (+34 more)

Community 16 - "api"

Cohesion: 0.16 Nodes (29): activateCookie(), api(), checkAudio(), clearJobHistory(), copyClip(), copyText(), deleteJob(), downloadAudio() (+21 more)

Community 17 - "pipeline.py"

Cohesion: 0.10 Nodes (19): merge_adjacent(), parse_auto_dump(), parse_json3(), parse_vtt(), _vtt_ts_to_seconds(), test_merge_adjacent(), test_parse_json3_basic(), test_parse_json3_invalid_json() (+11 more)

Community 18 - "test_blocked_videos.py"

Cohesion: 0.23 Nodes (18): Members-only / gated videos: identified from discovery, labelled, and kept out o, Buying the membership must not leave the video permanently stranded., unlisted downloads perfectly well — mislabelling it would hide videos the, Rows burned into error before availability was captured must still be iden, Bulk runs must not spend requests on videos that cannot be fetched., _ref(), _store(), test_a_blocked_video_is_still_reachable_by_id() (+10 more)

Community 19 - "config.py"

Cohesion: 0.24 Nodes (14): _build_config(), DelayConfig, load_config(), Incremental channel sync — how far back a routine re-scan looks. The /video, SyncConfig, YtDlpConfig, Tests for the YAML config loader. Focuses on the languages and ``prefer_man, The default must fall back to auto captions. A manual-only default silently (+6 more)

Community 20 - "setView"

Cohesion: 0.24 Nodes (10): cycleSort(), destroyCharts(), loadFolders(), loadFormat(), loadSearch(), loadVideos(), openChannel(), resetSearch() (+2 more)

Community 21 - "Discovery-Only Scrape Design"

Cohesion: 0.22 Nodes (8): Architecture, Discovery-Only Scrape Design, Error Handling, Existing Context, Goal, Persistence and Results, Testing, UI

Community 22 - "renderDashboardCharts"

Cohesion: 0.36 Nodes (9): axisOpts(), barOpts(), fmtMonth(), lineOpts(), loadAnalysis(), makeChart(), renderDashboardCharts(), renderTimelineChart() (+1 more)

Community 23 - "Global Constraints"

Cohesion: 0.25 Nodes (7): Discovery-Only Scrape Implementation Plan, Global Constraints, Task 1: Make catalog insertion counts accurate, Task 2: Add discovery-only job execution, Task 3: Expose the discovery mode through the API, Task 4: Add one-channel and all-channel UI controls, Task 5: Final verification and review

Community 24 - "Config"

Cohesion: 0.23 Nodes (11): Config, Path, _catalog_channel(), The window is 30 wide but the channel has 200 videos — video_count must not, test_all_channel_discovery_continues_after_one_error(), test_audio_job_downloads_into_audio_directory(), test_audio_job_with_video_ids_uses_audio_runner(), test_batch_job_reports_videos_without_transcripts() (+3 more)

Community 25 - "init"

Cohesion: 0.15 Nodes (18): addChannel(), checkHealth(), downloadAvatarsScope(), hydrateURL(), init(), jobActive(), loadChannelPending(), loadChannels() (+10 more)

Community 26 - "render.py"

Cohesion: 0.25 Nodes (12): Environment, format_timestamp(), _make_env(), Path, quote_yaml(), render_markdown(), to_json(), test_format_timestamp_hours() (+4 more)

Community 27 - "loadChannels"

Cohesion: 0.23 Nodes (8): Row, is_expired(), list_vault(), CookieRow, JobRow, _row_to_cookierow(), _row_to_jobrow(), _sanitize_fts()

Community 28 - "jobs.py"

Cohesion: 0.23 Nodes (10): resolve_active_path(), _extract_handle(), _keep_ref(), Same shorts/live rules the store was populated under — see jobs._keep_ref., Discover + process pending videos for a channel, optionally on an interval., _run_once(), watch_loop(), polite_sleep() (+2 more)

Community 29 - "Segment"

Cohesion: 0.37 Nodes (11): align_chapters(), Chapter, chapters_from_info(), Section, Segment, test_align_no_chapters(), test_align_pre_chapter_segments(), test_align_with_chapters() (+3 more)

Community 30 - "cookies.py"

Cohesion: 0.35 Nodes (11): auto_import_dir(), cookies_dir(), delete(), import_file(), import_text(), parse_netscape(), parse_netscape_file(), Path (+3 more)

Community 31 - "discover.py"

Cohesion: 0.27 Nodes (10): _channel_root_url(), deep_channel_avatar(), discover_channel(), _flatten_entries(), _pick_channel_avatar(), Any, Best-effort channel avatar URL from a yt-dlp channel info dict. Only source, Strip a tab suffix from a YouTube channel URL so yt-dlp extracts the channel (+2 more)

Community 32 - "export.py"

Cohesion: 0.38 Nodes (10): export_csv(), export_html(), export_json(), export_srt(), export_srt_video(), Path, _segments_to_srt(), _srt_ts() (+2 more)

Community 33 - "pipeline.py"

Cohesion: 0.29 Nodes (10): _normalize_date(), process_video(), Regenerate .md for done videos that have stored segments. Returns count re-rende, Extract + parse + store + render a single video. Returns final status string., re_render_videos(), _safe_dirname(), build_filename_stem(), test_build_filename_stem() (+2 more)

Community 34 - "_channel_targets"

Cohesion: 0.20 Nodes (10): analyze_cmd(), audio_cmd(), _channel_targets(), export_cmd(), Devolver videos atascados en error/no_subtitles a 'pending' para reintentarlos., Export multi-formato., Descarga audio MP3 (requiere ffmpeg)., Analisis estadistico de contenido (frecuencia, timeline). (+2 more)

Community 35 - "test_features.py"

Cohesion: 0.20 Nodes (5): store(), test_export_json_csv(), test_export_srt(), test_term_timeline(), test_word_frequency()

Community 36 - "VideoRow"

Cohesion: 0.33 Nodes (4): Why this video can never be fetched, or None if it can. Prefers the dis, Pending videos, excluding ones discovery already told us we cannot fetch, _row_to_videorow(), VideoRow

Community 37 - "test_since_filter.py"

Cohesion: 0.52 Nodes (6): _apply_filters(), --since / the scrape form's date filter must not eat undated entries. yt-dlp', _ref(), test_since_keeps_entries_with_no_upload_date(), test_since_still_drops_entries_that_are_provably_older(), test_without_since_nothing_is_dropped_on_date_grounds()

Community 40 - "re_render_cmd"

Cohesion: 0.50 Nodes (4): _parse_tags(), Regenerar Markdown desde segmentos almacenados., re_render_cmd(), _safe_dirname()

Community 41 - "handleFiles"

Cohesion: 0.67 Nodes (3): handleDrop(), handleFiles(), uploadCookies()

Knowledge Gaps

58 isolated node(s): yt-channel-scraper, Qué es esto, Comandos, Orientación antes de leer código, Tres superficies, un solo pipeline (+53 more) These have ≤1 connection - possible missing edges or undocumented components.
5 thin communities (<3 nodes) omitted from report — run graphify query to explore isolated nodes.

16 KiB Raw Blame History