This commit is contained in:
urieljarethbusiness-cpu
2026-05-17 10:14:14 -06:00
parent d8773b2508
commit 64b3d15b90
61 changed files with 7612 additions and 0 deletions
+19
View File
@@ -0,0 +1,19 @@
.git
.venv
__pycache__
*.pyc
.pytest_cache
.mypy_cache
.ruff_cache
.env
.env.*
data
frontend/.next
frontend/node_modules
frontend/.env.local
frontend/.env.*.local
node_modules
npm-debug.log*
Dockerfile*
docker-compose*.yml
docker-compose*.yaml
+5
View File
@@ -0,0 +1,5 @@
APP_ENV=development
APP_DATA_DIR=./data
DATABASE_PATH=./data/app.db
MISTRAL_API_KEY=
DEEPGRAM_API_KEY=
+14
View File
@@ -0,0 +1,14 @@
.env
__pycache__/
*.py[cod]
.pytest_cache/
.venv/
venv/
node_modules/
.next/
.runtime/
data/
tmp/
*.db
*.sqlite
frontend/.env.local
+32
View File
@@ -0,0 +1,32 @@
# AGENTS.md
## Repo Shape
- The primary app is now `backend/` (Python/FastAPI) plus `frontend/` (Next.js/TypeScript/Tailwind); `estacion documentos.html` is a browser-only prototype/reference for behavior.
- Runtime data is stored under `data/` and is gitignored; do not commit uploaded class files, generated Markdown, SQLite DBs, temporary audio, or API keys.
- The included `Griffin(2020).Comportamiento_organizacionalSOLOCAPITULO7.13Ed.pdf` is a local sample/source PDF for import behavior.
## Commands
- Local setup: `python -m venv .venv`, `.venv\Scripts\activate`, `pip install -r requirements.txt`, copy `.env.example` to `.env`.
- Run API: `uvicorn backend.app.main:app --reload` from the repo root.
- Run frontend: `cd frontend`, `npm install`, `npm run dev -- -p 3000` with `NEXT_PUBLIC_API_URL=http://127.0.0.1:8000`.
- Windows non-technical flow: double-click `abrir_estacion.bat`; it chooses free ports, starts backend/frontend, and opens the browser. Use `cerrar_estacion.bat` to stop only this project.
- Docker/Coolify check: `docker compose up --build`.
- Basic syntax check: `python -m compileall backend`.
## Platform Behavior
- Knowledge is organized as `Materia -> Semana -> fuentes raw -> Markdown procesado -> exports` under `data/subjects/<materia>/week-XX/`.
- `KnowledgeManager` owns filesystem layout and SQLite metadata; processors should not write final Markdown directly.
- `IngestionService` selects processors by `source_type` and wraps output with Markdown frontmatter for LLM ingestion.
- Text and Markdown are read directly; `.docx` uses `python-docx`; PDFs use PyMuPDF text extraction unless `use_ocr=true` is passed. PDF uploads can also pass `page_ranges` like `1-5,8,10-12`; ranges apply to both standard extraction and Mistral OCR.
- OCR uses Mistral `https://api.mistral.ai/v1/chat/completions` with model `pixtral-12b-2409`.
- Audio uses Deepgram `https://api.deepgram.com/v1/listen?model=nova-3&smart_format=true&language=es`.
- Video processing requires `ffmpeg`; audio is extracted to `data/tmp/`, transcribed with Deepgram, then deleted.
- The frontend calls the backend through `NEXT_PUBLIC_API_URL`; when ports change, update `frontend/.env.local` or use the adaptive `.bat`.
## Secrets And Deployment
- Use `.env` locally and Coolify environment variables in production; never hardcode or commit `MISTRAL_API_KEY` or `DEEPGRAM_API_KEY`.
- For Coolify, mount persistent storage for `/app/data` or all subjects, outputs, and SQLite state will be lost on redeploy.
## Prototype Notes
- Preserve the Spanish-language UX and Markdown-first output conventions from `estacion documentos.html` when building the new UI.
- The prototype sanitizes preview HTML with DOMPurify after `marked.parse`; keep equivalent sanitization in any future web UI.
+18
View File
@@ -0,0 +1,18 @@
FROM python:3.12-slim
ENV PYTHONDONTWRITEBYTECODE=1
ENV PYTHONUNBUFFERED=1
WORKDIR /app
RUN apt-get update \
&& apt-get install -y --no-install-recommends ffmpeg \
&& rm -rf /var/lib/apt/lists/*
COPY requirements.txt ./
RUN pip install --no-cache-dir -r requirements.txt
COPY backend ./backend
EXPOSE 8000
CMD ["uvicorn", "backend.app.main:app", "--host", "0.0.0.0", "--port", "8000"]
+89
View File
@@ -0,0 +1,89 @@
# Knowledge Station
Plataforma para organizar conocimiento escolar por materia y semana, procesar documentos/audio/video y generar archivos Markdown optimizados para ingesta en LLM.
El archivo `estacion documentos.html` queda como prototipo funcional de referencia. La plataforma principal usa `backend/` (FastAPI) y `frontend/` (Next.js).
## Capacidades Iniciales
- Materias y semanas persistidas en SQLite.
- Estructura de archivos por materia/semana en `data/subjects/`.
- Ingesta de `.txt`, `.md`, `.docx`, `.pdf`, audio y video.
- PDF con texto mediante PyMuPDF.
- OCR opcional de PDF con Mistral.
- Audio con Deepgram en español.
- Video con extracción temporal de audio vía `ffmpeg` y transcripción con Deepgram.
- Exportación semanal consolidada a Markdown.
- UI dedicada con materias, semanas, subida de archivos, polling de trabajos y editor Markdown con preview.
## Desarrollo Local
```bash
python -m venv .venv
.venv\Scripts\activate
pip install -r requirements.txt
copy .env.example .env
uvicorn backend.app.main:app --reload
```
En otra terminal:
```bash
cd frontend
npm install
set NEXT_PUBLIC_API_URL=http://127.0.0.1:8000
npm run dev -- -p 3000
```
La plataforma queda en `http://localhost:3000`, la API en `http://localhost:8000` y la documentación interactiva en `http://localhost:8000/docs`.
Para usuarios no tecnicos en Windows, usa `abrir_estacion.bat`; detecta puertos libres, prepara backend/frontend y abre el navegador. Usa `cerrar_estacion.bat` para detener todo.
## Docker / Coolify
```bash
docker compose up --build
```
Para Coolify, configura backend y frontend como servicios separados. Monta un volumen persistente para `/app/data` en el backend y define `NEXT_PUBLIC_API_URL` en el frontend apuntando a la URL publica del backend.
## Variables
- `APP_DATA_DIR`: carpeta de datos persistentes.
- `DATABASE_PATH`: ruta SQLite.
- `MISTRAL_API_KEY`: OCR para PDFs escaneados.
- `DEEPGRAM_API_KEY`: transcripción de audio/video.
No subas claves reales al repositorio.
## Flujo API Basico
Crear materia:
```bash
curl -X POST http://localhost:8000/subjects -H "Content-Type: application/json" -d '{"name":"Comportamiento Organizacional"}'
```
Crear semana:
```bash
curl -X POST http://localhost:8000/subjects/1/weeks -H "Content-Type: application/json" -d '{"number":1,"title":"Grupos y equipos"}'
```
Subir archivo:
```bash
curl -X POST -F "[email protected]" http://localhost:8000/subjects/1/weeks/1/files
```
Consultar trabajo:
```bash
curl http://localhost:8000/jobs/1
```
Exportar semana:
```bash
curl -L http://localhost:8000/subjects/1/weeks/1/export -o semana-01.md
```
+18
View File
@@ -0,0 +1,18 @@
@echo off
setlocal
title Knowledge Station - Abrir
set "ROOT=%~dp0"
set "ROOT=%ROOT:~0,-1%"
powershell -NoProfile -ExecutionPolicy Bypass -File "%ROOT%\scripts\start-platform.ps1" -Root "%ROOT%"
if errorlevel 1 (
echo.
echo No se pudo abrir la plataforma. Revisa el mensaje anterior.
pause
exit /b 1
)
echo.
echo Puedes dejar esta ventana cerrada. Para apagar la plataforma usa cerrar_estacion.bat.
pause
exit /b 0
View File
View File
+16
View File
@@ -0,0 +1,16 @@
from pathlib import Path
from pydantic_settings import BaseSettings, SettingsConfigDict
class Settings(BaseSettings):
app_env: str = "development"
app_data_dir: Path = Path("./data")
database_path: Path = Path("./data/app.db")
mistral_api_key: str = ""
deepgram_api_key: str = ""
model_config = SettingsConfigDict(env_file=".env", env_file_encoding="utf-8", extra="ignore")
settings = Settings()
+281
View File
@@ -0,0 +1,281 @@
from __future__ import annotations
from pathlib import Path
from fastapi import BackgroundTasks, FastAPI, File, HTTPException, Query, UploadFile
from fastapi.middleware.cors import CORSMiddleware
from fastapi.responses import FileResponse, PlainTextResponse
from backend.app.config import settings
from backend.app.managers.knowledge_manager import DuplicateSourceError, KnowledgeManager
from backend.app.models.schemas import MarkdownUpdate, SubjectCreate, WeekCreate, WeekUpdate
from backend.app.services.ingestion_service import IngestionService
app = FastAPI(title="Knowledge Station", version="0.1.0")
app.add_middleware(
CORSMiddleware,
allow_origins=["http://127.0.0.1:3000", "http://localhost:3000"],
allow_origin_regex=r"http://(127\.0\.0\.1|localhost):\d+",
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
manager = KnowledgeManager()
ingestion = IngestionService(manager)
@app.get("/health")
def health() -> dict[str, str]:
return {"status": "ok"}
@app.get("/settings/status")
def settings_status() -> dict:
return {
"mistral_configured": bool(settings.mistral_api_key),
"deepgram_configured": bool(settings.deepgram_api_key),
"data_dir": str(settings.app_data_dir),
"database_path": str(settings.database_path),
}
@app.post("/subjects")
def create_subject(payload: SubjectCreate) -> dict:
try:
return manager.create_subject(payload.name)
except ValueError as exc:
raise HTTPException(status_code=400, detail=str(exc)) from exc
@app.get("/subjects")
def list_subjects() -> list[dict]:
return manager.list_subjects()
@app.get("/subjects/{subject_id}")
def get_subject(subject_id: int) -> dict:
try:
return manager.get_subject(subject_id)
except ValueError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
@app.delete("/subjects/{subject_id}")
def delete_subject(subject_id: int) -> dict:
try:
manager.delete_subject(subject_id)
return {"status": "deleted"}
except ValueError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
@app.post("/subjects/{subject_id}/weeks")
def create_week(subject_id: int, payload: WeekCreate) -> dict:
try:
return manager.create_week(subject_id, payload.number, payload.title)
except ValueError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
@app.get("/subjects/{subject_id}/weeks")
def list_weeks(subject_id: int) -> list[dict]:
try:
manager.get_subject(subject_id)
return manager.list_weeks(subject_id)
except ValueError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
@app.get("/subjects/{subject_id}/weeks/{week_number}")
def get_week(subject_id: int, week_number: int) -> dict:
try:
return manager.get_week(subject_id, week_number)
except ValueError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
@app.put("/subjects/{subject_id}/weeks/{week_number}")
def update_week(subject_id: int, week_number: int, payload: WeekUpdate) -> dict:
try:
return manager.update_week(subject_id, week_number, payload.number, payload.title)
except ValueError as exc:
raise HTTPException(status_code=400, detail=str(exc)) from exc
@app.delete("/subjects/{subject_id}/weeks/{week_number}")
def delete_week(subject_id: int, week_number: int) -> dict:
try:
manager.delete_week(subject_id, week_number)
return {"status": "deleted"}
except ValueError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
@app.post("/subjects/{subject_id}/weeks/{week_number}/files")
def upload_file(
subject_id: int,
week_number: int,
background_tasks: BackgroundTasks,
file: UploadFile = File(...),
use_ocr: bool = Query(False, description="Usar Mistral OCR para PDFs"),
page_ranges: str | None = Query(None, description="Rangos de paginas PDF, por ejemplo: 1-5,8,10-12"),
) -> dict:
try:
source = manager.save_upload(subject_id, week_number, file)
job = manager.create_job(source["id"], "Archivo recibido")
background_tasks.add_task(process_job, job["id"], source["id"], use_ocr, page_ranges)
return {"source": source, "job": job}
except DuplicateSourceError as exc:
raise HTTPException(status_code=409, detail=str(exc)) from exc
except ValueError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
@app.get("/jobs/{job_id}")
def get_job(job_id: int) -> dict:
try:
return manager.get_job(job_id)
except ValueError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
@app.post("/jobs/{job_id}/retry")
def retry_job(
job_id: int,
background_tasks: BackgroundTasks,
use_ocr: bool = Query(False),
page_ranges: str | None = Query(None),
) -> dict:
try:
job = manager.get_job(job_id)
if not job.get("source_id"):
raise HTTPException(status_code=400, detail="El trabajo no tiene fuente asociada")
manager.update_job(job_id, "queued", "Reintento en cola")
background_tasks.add_task(process_job, job_id, job["source_id"], use_ocr, page_ranges)
return manager.get_job(job_id)
except ValueError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
@app.get("/subjects/{subject_id}/weeks/{week_number}/jobs")
def list_week_jobs(subject_id: int, week_number: int) -> list[dict]:
try:
return manager.list_jobs_for_week(subject_id, week_number)
except ValueError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
@app.get("/subjects/{subject_id}/weeks/{week_number}/sources")
def list_sources(subject_id: int, week_number: int) -> list[dict]:
try:
return manager.list_sources(subject_id, week_number)
except ValueError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
@app.delete("/sources/{source_id}")
def delete_source(source_id: int) -> dict:
try:
manager.delete_source(source_id)
return {"status": "deleted"}
except ValueError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
@app.post("/sources/{source_id}/reprocess")
def reprocess_source(
source_id: int,
background_tasks: BackgroundTasks,
use_ocr: bool = Query(False),
page_ranges: str | None = Query(None),
) -> dict:
try:
source = manager.get_source(source_id)
if manager.source_has_active_job(source_id):
raise HTTPException(status_code=409, detail="La fuente ya tiene un trabajo en cola o en proceso")
job = manager.create_job(source["id"], "Reprocesamiento en cola")
background_tasks.add_task(process_job, job["id"], source["id"], use_ocr, page_ranges)
return {"source": source, "job": job}
except ValueError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
@app.get("/subjects/{subject_id}/weeks/{week_number}/documents")
def list_documents(subject_id: int, week_number: int) -> list[dict]:
try:
return manager.list_documents(subject_id, week_number)
except ValueError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
@app.get("/documents/{document_id}")
def get_document(document_id: int) -> dict:
try:
return manager.get_document(document_id)
except ValueError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
@app.get("/documents/{document_id}/markdown", response_class=PlainTextResponse)
def get_document_markdown(document_id: int) -> str:
try:
document = manager.get_document(document_id)
return Path(document["markdown_path"]).read_text(encoding="utf-8")
except ValueError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
@app.put("/documents/{document_id}/markdown")
def update_document_markdown(document_id: int, payload: MarkdownUpdate) -> dict:
try:
return manager.update_document_markdown(document_id, payload.content)
except ValueError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
@app.get("/documents/{document_id}/download")
def download_document(document_id: int) -> FileResponse:
try:
document = manager.get_document(document_id)
path = Path(document["markdown_path"])
return FileResponse(path, filename=path.name, media_type="text/markdown")
except ValueError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
@app.delete("/documents/{document_id}")
def delete_document(document_id: int) -> dict:
try:
manager.delete_document(document_id)
return {"status": "deleted"}
except ValueError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
@app.get("/subjects/{subject_id}/weeks/{week_number}/export")
def export_week(subject_id: int, week_number: int) -> FileResponse:
try:
path = manager.export_week_markdown(subject_id, week_number)
return FileResponse(path, filename=path.name, media_type="text/markdown")
except ValueError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
@app.get("/subjects/{subject_id}/weeks/{week_number}/export.zip")
def export_week_zip(subject_id: int, week_number: int) -> FileResponse:
try:
path = manager.export_week_markdown_zip(subject_id, week_number)
return FileResponse(path, filename=path.name, media_type="application/zip")
except ValueError as exc:
raise HTTPException(status_code=404, detail=str(exc)) from exc
def process_job(job_id: int, source_id: int, use_ocr: bool, page_ranges: str | None = None) -> None:
manager.update_job(job_id, "processing", "Procesando archivo")
manager.update_source_status(source_id, "processing")
try:
document = ingestion.process_source(source_id, use_ocr=use_ocr, page_ranges=page_ranges)
manager.update_job(job_id, "completed", "Documento procesado", document_id=document["id"])
except Exception as exc:
manager.update_source_status(source_id, "failed")
manager.update_job(job_id, "failed", str(exc))
View File
+548
View File
@@ -0,0 +1,548 @@
from __future__ import annotations
import shutil
import sqlite3
import re
import hashlib
import zipfile
from datetime import datetime, timezone
from pathlib import Path
from typing import Any, Iterable
from fastapi import UploadFile
from backend.app.config import settings
from backend.app.utils.slug import slugify
def utc_now() -> str:
return datetime.now(timezone.utc).isoformat()
class DuplicateSourceError(ValueError):
def __init__(self, existing_source: dict[str, Any]):
self.existing_source = existing_source
super().__init__(f"El archivo ya existe en esta semana: {existing_source['original_name']}")
class KnowledgeManager:
def __init__(self, data_dir: Path | None = None, database_path: Path | None = None):
self.data_dir = data_dir or settings.app_data_dir
self.database_path = database_path or settings.database_path
self.subjects_dir = self.data_dir / "subjects"
self.tmp_dir = self.data_dir / "tmp"
self.data_dir.mkdir(parents=True, exist_ok=True)
self.subjects_dir.mkdir(parents=True, exist_ok=True)
self.tmp_dir.mkdir(parents=True, exist_ok=True)
self.database_path.parent.mkdir(parents=True, exist_ok=True)
self._init_db()
def _connect(self) -> sqlite3.Connection:
conn = sqlite3.connect(self.database_path)
conn.row_factory = sqlite3.Row
return conn
def _init_db(self) -> None:
with self._connect() as conn:
conn.executescript(
"""
create table if not exists subjects (
id integer primary key autoincrement,
name text not null,
slug text not null unique,
created_at text not null
);
create table if not exists weeks (
id integer primary key autoincrement,
subject_id integer not null,
number integer not null,
title text,
created_at text not null,
unique(subject_id, number),
foreign key(subject_id) references subjects(id)
);
create table if not exists sources (
id integer primary key autoincrement,
week_id integer not null,
original_name text not null,
stored_path text not null,
source_type text not null,
status text not null,
file_size integer,
sha256 text,
mime_type text,
processor text,
page_ranges text,
processed_at text,
created_at text not null,
foreign key(week_id) references weeks(id)
);
create table if not exists documents (
id integer primary key autoincrement,
source_id integer not null,
title text not null,
markdown_path text not null,
created_at text not null,
foreign key(source_id) references sources(id)
);
create table if not exists jobs (
id integer primary key autoincrement,
status text not null,
message text not null,
source_id integer,
document_id integer,
created_at text not null,
updated_at text not null,
foreign key(source_id) references sources(id),
foreign key(document_id) references documents(id)
);
"""
)
self._ensure_columns(
conn,
"sources",
{
"file_size": "integer",
"sha256": "text",
"mime_type": "text",
"processor": "text",
"page_ranges": "text",
"processed_at": "text",
},
)
self._backfill_source_file_metadata(conn)
def _ensure_columns(self, conn: sqlite3.Connection, table: str, columns: dict[str, str]) -> None:
existing = {row["name"] for row in conn.execute(f"pragma table_info({table})").fetchall()}
for name, definition in columns.items():
if name not in existing:
conn.execute(f"alter table {table} add column {name} {definition}")
def _backfill_source_file_metadata(self, conn: sqlite3.Connection) -> None:
rows = conn.execute("select id, stored_path from sources where sha256 is null or file_size is null").fetchall()
for row in rows:
path = Path(row["stored_path"])
if not path.exists():
continue
sha256, file_size = self._file_hash_and_size(path)
conn.execute("update sources set sha256 = ?, file_size = ? where id = ?", (sha256, file_size, row["id"]))
def create_subject(self, name: str) -> dict[str, Any]:
base_slug = slugify(name)
slug = base_slug
suffix = 2
with self._connect() as conn:
while conn.execute("select 1 from subjects where slug = ?", (slug,)).fetchone():
slug = f"{base_slug}-{suffix}"
suffix += 1
now = utc_now()
cursor = conn.execute(
"insert into subjects (name, slug, created_at) values (?, ?, ?)",
(name, slug, now),
)
(self.subjects_dir / slug).mkdir(parents=True, exist_ok=True)
return self.get_subject(cursor.lastrowid)
def get_subject(self, subject_id: int) -> dict[str, Any]:
with self._connect() as conn:
row = conn.execute("select * from subjects where id = ?", (subject_id,)).fetchone()
if not row:
raise ValueError("Materia no encontrada")
return dict(row)
def delete_subject(self, subject_id: int) -> None:
subject = self.get_subject(subject_id)
with self._connect() as conn:
week_rows = conn.execute("select id from weeks where subject_id = ?", (subject_id,)).fetchall()
week_ids = [row["id"] for row in week_rows]
for week_id in week_ids:
source_rows = conn.execute("select id from sources where week_id = ?", (week_id,)).fetchall()
source_ids = [row["id"] for row in source_rows]
for source_id in source_ids:
conn.execute("delete from documents where source_id = ?", (source_id,))
conn.execute("delete from jobs where source_id = ?", (source_id,))
conn.execute("delete from sources where week_id = ?", (week_id,))
conn.execute("delete from weeks where subject_id = ?", (subject_id,))
conn.execute("delete from subjects where id = ?", (subject_id,))
shutil.rmtree(self.subjects_dir / subject["slug"], ignore_errors=True)
def list_subjects(self) -> list[dict[str, Any]]:
with self._connect() as conn:
rows = conn.execute("select * from subjects order by created_at desc").fetchall()
return [dict(row) for row in rows]
def create_week(self, subject_id: int, number: int, title: str | None = None) -> dict[str, Any]:
subject = self.get_subject(subject_id)
now = utc_now()
with self._connect() as conn:
conn.execute(
"insert or ignore into weeks (subject_id, number, title, created_at) values (?, ?, ?, ?)",
(subject_id, number, title, now),
)
if title:
conn.execute("update weeks set title = ? where subject_id = ? and number = ?", (title, subject_id, number))
row = conn.execute(
"select * from weeks where subject_id = ? and number = ?",
(subject_id, number),
).fetchone()
self.week_dir(subject["slug"], number).mkdir(parents=True, exist_ok=True)
for child in ("raw", "processed", "exports"):
(self.week_dir(subject["slug"], number) / child).mkdir(parents=True, exist_ok=True)
return dict(row)
def list_weeks(self, subject_id: int) -> list[dict[str, Any]]:
with self._connect() as conn:
rows = conn.execute("select * from weeks where subject_id = ? order by number", (subject_id,)).fetchall()
return [dict(row) for row in rows]
def delete_week(self, subject_id: int, number: int) -> None:
subject = self.get_subject(subject_id)
week = self.get_week(subject_id, number)
with self._connect() as conn:
source_rows = conn.execute("select id from sources where week_id = ?", (week["id"],)).fetchall()
for row in source_rows:
conn.execute("delete from documents where source_id = ?", (row["id"],))
conn.execute("delete from jobs where source_id = ?", (row["id"],))
conn.execute("delete from sources where week_id = ?", (week["id"],))
conn.execute("delete from weeks where id = ?", (week["id"],))
shutil.rmtree(self.week_dir(subject["slug"], number), ignore_errors=True)
def get_week(self, subject_id: int, number: int) -> dict[str, Any]:
with self._connect() as conn:
row = conn.execute(
"select * from weeks where subject_id = ? and number = ?",
(subject_id, number),
).fetchone()
if not row:
raise ValueError("Semana no encontrada")
return dict(row)
def update_week(self, subject_id: int, current_number: int, new_number: int, title: str | None = None) -> dict[str, Any]:
subject = self.get_subject(subject_id)
week = self.get_week(subject_id, current_number)
if new_number != current_number:
with self._connect() as conn:
existing = conn.execute(
"select 1 from weeks where subject_id = ? and number = ?",
(subject_id, new_number),
).fetchone()
if existing:
raise ValueError("Ya existe una semana con ese numero")
old_dir = self.week_dir(subject["slug"], current_number)
new_dir = self.week_dir(subject["slug"], new_number)
if old_dir.exists() and new_dir.exists():
raise ValueError("Ya existe una carpeta para ese numero de semana")
if old_dir.exists():
old_dir.rename(new_dir)
else:
for child in ("raw", "processed", "exports"):
(new_dir / child).mkdir(parents=True, exist_ok=True)
with self._connect() as conn:
conn.execute(
"update weeks set number = ?, title = ? where id = ?",
(new_number, title, week["id"]),
)
return self.get_week(subject_id, new_number)
def week_dir(self, subject_slug: str, week_number: int) -> Path:
return self.subjects_dir / subject_slug / f"week-{week_number:02d}"
def save_upload(self, subject_id: int, week_number: int, upload: UploadFile) -> dict[str, Any]:
subject = self.get_subject(subject_id)
week = self.get_week(subject_id, week_number)
filename = Path(upload.filename or "archivo").name
raw_dir = self.week_dir(subject["slug"], week_number) / "raw"
raw_dir.mkdir(parents=True, exist_ok=True)
stored_path = self._unique_path(raw_dir / filename)
digest = hashlib.sha256()
file_size = 0
with stored_path.open("wb") as out_file:
while chunk := upload.file.read(1024 * 1024):
file_size += len(chunk)
digest.update(chunk)
out_file.write(chunk)
sha256 = digest.hexdigest()
source_type = self.detect_source_type(stored_path)
now = utc_now()
with self._connect() as conn:
duplicate = conn.execute(
"select * from sources where week_id = ? and sha256 = ? limit 1",
(week["id"], sha256),
).fetchone()
if duplicate:
stored_path.unlink(missing_ok=True)
raise DuplicateSourceError(dict(duplicate))
cursor = conn.execute(
"""
insert into sources (week_id, original_name, stored_path, source_type, status, file_size, sha256, mime_type, created_at)
values (?, ?, ?, ?, ?, ?, ?, ?, ?)
""",
(week["id"], filename, str(stored_path), source_type, "stored", file_size, sha256, upload.content_type, now),
)
return self.get_source(cursor.lastrowid)
def _file_hash_and_size(self, path: Path) -> tuple[str, int]:
digest = hashlib.sha256()
file_size = 0
with path.open("rb") as file:
while chunk := file.read(1024 * 1024):
file_size += len(chunk)
digest.update(chunk)
return digest.hexdigest(), file_size
def create_job(self, source_id: int, message: str = "En cola") -> dict[str, Any]:
now = utc_now()
with self._connect() as conn:
cursor = conn.execute(
"insert into jobs (status, message, source_id, created_at, updated_at) values (?, ?, ?, ?, ?)",
("queued", message, source_id, now, now),
)
return self.get_job(cursor.lastrowid)
def update_job(self, job_id: int, status: str, message: str, document_id: int | None = None) -> None:
with self._connect() as conn:
conn.execute(
"update jobs set status = ?, message = ?, document_id = coalesce(?, document_id), updated_at = ? where id = ?",
(status, message, document_id, utc_now(), job_id),
)
def update_source_status(self, source_id: int, status: str) -> None:
with self._connect() as conn:
conn.execute("update sources set status = ? where id = ?", (status, source_id))
def source_has_active_job(self, source_id: int) -> bool:
with self._connect() as conn:
row = conn.execute(
"select 1 from jobs where source_id = ? and status in ('queued', 'processing') limit 1",
(source_id,),
).fetchone()
return bool(row)
def get_job(self, job_id: int) -> dict[str, Any]:
with self._connect() as conn:
row = conn.execute("select * from jobs where id = ?", (job_id,)).fetchone()
if not row:
raise ValueError("Trabajo no encontrado")
return dict(row)
def list_jobs_for_week(self, subject_id: int, week_number: int) -> list[dict[str, Any]]:
week = self.get_week(subject_id, week_number)
with self._connect() as conn:
rows = conn.execute(
"""
select jobs.* from jobs
join sources on sources.id = jobs.source_id
where sources.week_id = ?
order by jobs.created_at desc
""",
(week["id"],),
).fetchall()
return [dict(row) for row in rows]
def get_source(self, source_id: int) -> dict[str, Any]:
with self._connect() as conn:
row = conn.execute("select * from sources where id = ?", (source_id,)).fetchone()
if not row:
raise ValueError("Fuente no encontrada")
return dict(row)
def list_sources(self, subject_id: int, week_number: int) -> list[dict[str, Any]]:
week = self.get_week(subject_id, week_number)
with self._connect() as conn:
rows = conn.execute(
"""
select sources.*, documents.id as document_id, documents.title as document_title,
documents.markdown_path as document_markdown_path, documents.created_at as document_created_at
from sources
left join documents on documents.source_id = sources.id
where sources.week_id = ?
order by sources.created_at desc
""",
(week["id"],),
).fetchall()
return [dict(row) for row in rows]
def delete_source(self, source_id: int) -> None:
source = self.get_source(source_id)
with self._connect() as conn:
doc_rows = conn.execute("select markdown_path from documents where source_id = ?", (source_id,)).fetchall()
conn.execute("delete from documents where source_id = ?", (source_id,))
conn.execute("delete from jobs where source_id = ?", (source_id,))
conn.execute("delete from sources where id = ?", (source_id,))
Path(source["stored_path"]).unlink(missing_ok=True)
for row in doc_rows:
Path(row["markdown_path"]).unlink(missing_ok=True)
def create_document(self, source_id: int, title: str, markdown: str, processor: str | None = None, page_ranges: str | None = None) -> dict[str, Any]:
source = self.get_source(source_id)
week_context = self.get_week_context(source["week_id"])
processed_dir = self.week_dir(week_context["subject_slug"], week_context["week_number"]) / "processed"
processed_dir.mkdir(parents=True, exist_ok=True)
markdown_path = self._unique_path(processed_dir / f"{slugify(title)}.md")
markdown_path.write_text(self._sanitize_markdown(markdown), encoding="utf-8")
now = utc_now()
with self._connect() as conn:
old_docs = conn.execute("select markdown_path from documents where source_id = ?", (source_id,)).fetchall()
conn.execute("delete from jobs where document_id in (select id from documents where source_id = ?)", (source_id,))
conn.execute("delete from documents where source_id = ?", (source_id,))
cursor = conn.execute(
"insert into documents (source_id, title, markdown_path, created_at) values (?, ?, ?, ?)",
(source_id, title, str(markdown_path), now),
)
conn.execute(
"update sources set status = ?, processor = ?, page_ranges = ?, processed_at = ? where id = ?",
("processed", processor, page_ranges, now, source_id),
)
for row in old_docs:
Path(row["markdown_path"]).unlink(missing_ok=True)
return self.get_document(cursor.lastrowid)
def get_document(self, document_id: int) -> dict[str, Any]:
with self._connect() as conn:
row = conn.execute("select * from documents where id = ?", (document_id,)).fetchone()
if not row:
raise ValueError("Documento no encontrado")
return dict(row)
def update_document_markdown(self, document_id: int, content: str) -> dict[str, Any]:
document = self.get_document(document_id)
Path(document["markdown_path"]).write_text(self._sanitize_markdown(content), encoding="utf-8")
return self.get_document(document_id)
def delete_document(self, document_id: int) -> None:
document = self.get_document(document_id)
Path(document["markdown_path"]).unlink(missing_ok=True)
with self._connect() as conn:
conn.execute("delete from jobs where document_id = ?", (document_id,))
conn.execute("delete from documents where id = ?", (document_id,))
def list_documents(self, subject_id: int, week_number: int) -> list[dict[str, Any]]:
week = self.get_week(subject_id, week_number)
with self._connect() as conn:
rows = conn.execute(
"""
select documents.*, sources.original_name as source_original_name,
sources.source_type as source_type, sources.status as source_status,
sources.processor as source_processor, sources.page_ranges as source_page_ranges
from documents
join sources on sources.id = documents.source_id
where sources.week_id = ?
order by documents.created_at desc
""",
(week["id"],),
).fetchall()
return [dict(row) for row in rows]
def get_week_context(self, week_id: int) -> dict[str, Any]:
with self._connect() as conn:
row = conn.execute(
"""
select weeks.id as week_id, weeks.number as week_number, weeks.title as week_title,
subjects.id as subject_id, subjects.name as subject_name, subjects.slug as subject_slug
from weeks join subjects on subjects.id = weeks.subject_id
where weeks.id = ?
""",
(week_id,),
).fetchone()
if not row:
raise ValueError("Contexto de semana no encontrado")
return dict(row)
def export_week_markdown(self, subject_id: int, week_number: int) -> Path:
subject = self.get_subject(subject_id)
docs = self.list_documents(subject_id, week_number)
export_dir = self.week_dir(subject["slug"], week_number) / "exports"
export_dir.mkdir(parents=True, exist_ok=True)
export_path = export_dir / f"{subject['slug']}-week-{week_number:02d}.md"
parts = [
"---\n"
f"subject: {subject['name']}\n"
f"subject_slug: {subject['slug']}\n"
f"week: {week_number}\n"
f"documents: {len(docs)}\n"
f"exported_at: {utc_now()}\n"
"---\n\n"
f"# {subject['name']} - Semana {week_number}\n"
]
for doc in docs:
path = Path(doc["markdown_path"])
content = self._strip_frontmatter(self._read_markdown(path)).strip()
parts.append(f"<!-- document_id: {doc['id']} -->\n\n{content}\n")
export_path.write_text(self._sanitize_markdown("\n\n---\n\n".join(parts)), encoding="utf-8")
return export_path
def export_week_markdown_zip(self, subject_id: int, week_number: int) -> Path:
subject = self.get_subject(subject_id)
docs = self.list_documents(subject_id, week_number)
export_dir = self.week_dir(subject["slug"], week_number) / "exports"
export_dir.mkdir(parents=True, exist_ok=True)
export_path = export_dir / f"{subject['slug']}-week-{week_number:02d}-documentos.zip"
used_names: set[str] = set()
with zipfile.ZipFile(export_path, "w", compression=zipfile.ZIP_DEFLATED) as archive:
for index, doc in enumerate(docs, start=1):
path = Path(doc["markdown_path"])
name = self._unique_archive_name(f"{index:02d}-{slugify(doc['title'])}.md", used_names)
archive.writestr(name, self._read_markdown(path))
return export_path
def _unique_archive_name(self, filename: str, used_names: set[str]) -> str:
path = Path(filename)
candidate = path.name
suffix = path.suffix
stem = path.stem
counter = 2
while candidate in used_names:
candidate = f"{stem}-{counter}{suffix}"
counter += 1
used_names.add(candidate)
return candidate
def _read_markdown(self, path: Path) -> str:
return self._sanitize_markdown(path.read_text(encoding="utf-8", errors="replace"))
def _sanitize_markdown(self, content: str) -> str:
return re.sub(r"[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]", "", content)
def _strip_frontmatter(self, content: str) -> str:
lines = content.splitlines()
if not lines or lines[0].strip() != "---":
return content
for index, line in enumerate(lines[1:], start=1):
if line.strip() == "---":
return "\n".join(lines[index + 1 :]).lstrip()
return content
def detect_source_type(self, path: Path) -> str:
ext = path.suffix.lower().lstrip(".")
if ext in {"txt", "md"}:
return "text"
if ext == "docx":
return "docx"
if ext == "pdf":
return "pdf"
if ext in {"mp3", "wav", "m4a", "ogg", "flac", "webm"}:
return "audio"
if ext in {"mp4", "mov", "mkv", "avi"}:
return "video"
return "unknown"
def _unique_path(self, path: Path) -> Path:
if not path.exists():
return path
stem = path.stem
suffix = path.suffix
for index in range(2, 10_000):
candidate = path.with_name(f"{stem}-{index}{suffix}")
if not candidate.exists():
return candidate
raise RuntimeError("No se pudo generar un nombre de archivo unico")
def rows_to_models(rows: Iterable[dict[str, Any]]) -> list[dict[str, Any]]:
return [dict(row) for row in rows]
View File
+55
View File
@@ -0,0 +1,55 @@
from datetime import datetime
from typing import Optional
from pydantic import BaseModel, Field
class SubjectCreate(BaseModel):
name: str = Field(min_length=1)
class Subject(BaseModel):
id: int
name: str
slug: str
created_at: datetime
class WeekCreate(BaseModel):
number: int = Field(ge=1)
title: Optional[str] = None
class WeekUpdate(BaseModel):
number: int = Field(ge=1)
title: Optional[str] = None
class Week(BaseModel):
id: int
subject_id: int
number: int
title: Optional[str]
created_at: datetime
class MarkdownUpdate(BaseModel):
content: str
class Job(BaseModel):
id: int
status: str
message: str
source_id: Optional[int] = None
document_id: Optional[int] = None
created_at: datetime
updated_at: datetime
class Document(BaseModel):
id: int
source_id: int
title: str
markdown_path: str
created_at: datetime
View File
+51
View File
@@ -0,0 +1,51 @@
from __future__ import annotations
from pathlib import Path
from backend.app.managers.knowledge_manager import KnowledgeManager
from backend.app.services.markdown_builder import build_markdown
from backend.app.services.processors.audio_processor import AudioProcessor
from backend.app.services.processors.docx_processor import DocxProcessor
from backend.app.services.processors.pdf_processor import PdfProcessor
from backend.app.services.processors.text_processor import TextProcessor
from backend.app.services.processors.video_processor import VideoProcessor
class IngestionService:
def __init__(self, manager: KnowledgeManager):
self.manager = manager
def process_source(self, source_id: int, use_ocr: bool = False, page_ranges: str | None = None) -> dict:
source = self.manager.get_source(source_id)
context = self.manager.get_week_context(source["week_id"])
path = Path(source["stored_path"])
processor = self._processor_for(source["source_type"], use_ocr, page_ranges)
processed = processor.process(path)
markdown = build_markdown(
title=processed.title,
body=processed.body,
metadata={
"subject": context["subject_name"],
"subject_slug": context["subject_slug"],
"week": context["week_number"],
"source_file": source["original_name"],
"source_type": source["source_type"],
"processor": processed.processor,
"page_ranges": page_ranges if source["source_type"] == "pdf" and page_ranges else None,
"language": "es",
},
)
return self.manager.create_document(source_id, processed.title, markdown, processor=processed.processor, page_ranges=page_ranges)
def _processor_for(self, source_type: str, use_ocr: bool, page_ranges: str | None):
if source_type == "text":
return TextProcessor()
if source_type == "docx":
return DocxProcessor()
if source_type == "pdf":
return PdfProcessor(use_ocr=use_ocr, page_ranges=page_ranges)
if source_type == "audio":
return AudioProcessor()
if source_type == "video":
return VideoProcessor(self.manager.tmp_dir)
raise RuntimeError(f"Tipo de fuente no soportado: {source_type}")
+23
View File
@@ -0,0 +1,23 @@
from __future__ import annotations
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
def build_markdown(title: str, body: str, metadata: dict[str, Any]) -> str:
frontmatter = {
**metadata,
"processed_at": datetime.now(timezone.utc).isoformat(),
}
lines = ["---"]
for key, value in frontmatter.items():
if value is not None:
safe_value = str(value).replace("\n", " ")
lines.append(f"{key}: {safe_value}")
lines.extend(["---", "", f"# {title}", "", body.strip(), ""])
return "\n".join(lines)
def title_from_path(path: Path) -> str:
return path.stem.replace("-", " ").replace("_", " ").strip().title() or "Documento"
@@ -0,0 +1,20 @@
from __future__ import annotations
from pathlib import Path
from backend.app.services.markdown_builder import title_from_path
from backend.app.services.processors.base import ProcessedContent
from backend.app.services.providers.deepgram_client import DeepgramClient
class AudioProcessor:
def process(self, path: Path) -> ProcessedContent:
client = DeepgramClient()
data = client.transcribe_file(path)
transcript, confidence = client.transcript_from_response(data)
confidence_line = f"\n\n_Confianza estimada: {confidence * 100:.1f}%_" if isinstance(confidence, (int, float)) else ""
return ProcessedContent(
title=f"Transcripcion {title_from_path(path)}",
body=f"## Transcripcion\n\n{transcript}{confidence_line}",
processor="deepgram",
)
+10
View File
@@ -0,0 +1,10 @@
from __future__ import annotations
from dataclasses import dataclass
@dataclass
class ProcessedContent:
title: str
body: str
processor: str
@@ -0,0 +1,30 @@
from __future__ import annotations
from pathlib import Path
from docx import Document
from backend.app.services.markdown_builder import title_from_path
from backend.app.services.processors.base import ProcessedContent
class DocxProcessor:
def process(self, path: Path) -> ProcessedContent:
doc = Document(path)
blocks: list[str] = []
for paragraph in doc.paragraphs:
text = paragraph.text.strip()
if text:
blocks.append(text)
for table in doc.tables:
rows = [[cell.text.strip().replace("\n", " ") for cell in row.cells] for row in table.rows]
if rows:
blocks.append(self._table_to_markdown(rows))
return ProcessedContent(title=title_from_path(path), body="\n\n".join(blocks), processor="docx")
def _table_to_markdown(self, rows: list[list[str]]) -> str:
header = rows[0]
lines = [f"| {' | '.join(header)} |", f"| {' | '.join(['---'] * len(header))} |"]
for row in rows[1:]:
lines.append(f"| {' | '.join(row)} |")
return "\n".join(lines)
@@ -0,0 +1,81 @@
from __future__ import annotations
from pathlib import Path
import fitz
from backend.app.services.markdown_builder import title_from_path
from backend.app.services.processors.base import ProcessedContent
from backend.app.services.providers.mistral_client import MistralClient
class PdfProcessor:
def __init__(self, use_ocr: bool = False, page_ranges: str | None = None):
self.use_ocr = use_ocr
self.page_ranges = page_ranges
def process(self, path: Path) -> ProcessedContent:
if self.use_ocr:
return self._process_ocr(path)
doc = fitz.open(path)
pages: list[str] = []
for index in self._selected_pages(len(doc)):
page = doc[index - 1]
text = page.get_text("text").strip()
if text:
pages.append(f"## Pagina {index}\n\n{text}")
body = "\n\n".join(pages).strip()
if len(body) < 80:
raise RuntimeError("El PDF tiene poco texto extraible; procesalo con OCR")
return ProcessedContent(title=title_from_path(path), body=body, processor="pdf-text")
def _process_ocr(self, path: Path) -> ProcessedContent:
doc = fitz.open(path)
client = MistralClient()
parts: list[str] = []
temp_paths: list[Path] = []
try:
for index in self._selected_pages(len(doc)):
page = doc[index - 1]
pix = page.get_pixmap(matrix=fitz.Matrix(2, 2), alpha=False)
image_path = path.with_name(f"{path.stem}-page-{index}.jpg")
pix.save(image_path)
temp_paths.append(image_path)
parts.append(f"## Pagina {index}\n\n{client.ocr_image(image_path)}")
finally:
for temp_path in temp_paths:
temp_path.unlink(missing_ok=True)
return ProcessedContent(title=title_from_path(path), body="\n\n".join(parts), processor="mistral-ocr")
def _selected_pages(self, total_pages: int) -> list[int]:
if total_pages < 1:
raise RuntimeError("El PDF no tiene paginas")
if not self.page_ranges or not self.page_ranges.strip():
return list(range(1, total_pages + 1))
selected: list[int] = []
for raw_part in self.page_ranges.split(","):
part = raw_part.strip()
if not part:
continue
if "-" in part:
start_text, end_text = [piece.strip() for piece in part.split("-", 1)]
if not start_text.isdigit() or not end_text.isdigit():
raise RuntimeError(f"Rango de paginas invalido: {part}")
start = int(start_text)
end = int(end_text)
if start > end:
raise RuntimeError(f"Rango de paginas invertido: {part}")
selected.extend(range(start, end + 1))
else:
if not part.isdigit():
raise RuntimeError(f"Pagina invalida: {part}")
selected.append(int(part))
unique_pages = sorted(set(selected))
invalid = [page for page in unique_pages if page < 1 or page > total_pages]
if invalid:
raise RuntimeError(f"Paginas fuera de rango: {', '.join(map(str, invalid))}. El PDF tiene {total_pages} paginas")
if not unique_pages:
raise RuntimeError("No se seleccionaron paginas para procesar")
return unique_pages
@@ -0,0 +1,15 @@
from __future__ import annotations
from pathlib import Path
from backend.app.services.markdown_builder import title_from_path
from backend.app.services.processors.base import ProcessedContent
class TextProcessor:
def process(self, path: Path) -> ProcessedContent:
return ProcessedContent(
title=title_from_path(path),
body=path.read_text(encoding="utf-8", errors="replace"),
processor="text",
)
@@ -0,0 +1,46 @@
from __future__ import annotations
import subprocess
from pathlib import Path
from backend.app.services.processors.audio_processor import AudioProcessor
from backend.app.services.processors.base import ProcessedContent
class VideoProcessor:
def __init__(self, tmp_dir: Path):
self.tmp_dir = tmp_dir
def process(self, path: Path) -> ProcessedContent:
self.tmp_dir.mkdir(parents=True, exist_ok=True)
audio_path = self.tmp_dir / f"{path.stem}.wav"
try:
subprocess.run(
[
"ffmpeg",
"-y",
"-i",
str(path),
"-vn",
"-acodec",
"pcm_s16le",
"-ar",
"16000",
"-ac",
"1",
str(audio_path),
],
check=True,
capture_output=True,
text=True,
)
processed = AudioProcessor().process(audio_path)
processed.title = f"Transcripcion {path.stem}"
processed.processor = "ffmpeg+deepgram"
return processed
except FileNotFoundError as exc:
raise RuntimeError("ffmpeg no esta instalado o no esta disponible en PATH") from exc
except subprocess.CalledProcessError as exc:
raise RuntimeError(f"No se pudo extraer audio del video: {exc.stderr}") from exc
finally:
audio_path.unlink(missing_ok=True)
@@ -0,0 +1,52 @@
from __future__ import annotations
from pathlib import Path
import requests
from backend.app.config import settings
class DeepgramClient:
endpoint = "https://api.deepgram.com/v1/listen?model=nova-3&smart_format=true&language=es"
def transcribe_file(self, path: Path) -> dict:
if not settings.deepgram_api_key:
raise RuntimeError("DEEPGRAM_API_KEY no esta configurada")
headers = {
"Authorization": f"Token {settings.deepgram_api_key}",
"Content-Type": self._content_type(path),
}
with path.open("rb") as audio_file:
response = requests.post(self.endpoint, headers=headers, data=audio_file, timeout=600)
if response.status_code == 401:
raise RuntimeError("API Key de Deepgram invalida")
if not response.ok:
try:
error = response.json()
except ValueError:
error = response.text
raise RuntimeError(f"Deepgram error {response.status_code}: {error}")
return response.json()
def transcript_from_response(self, data: dict) -> tuple[str, float | None]:
alternative = data.get("results", {}).get("channels", [{}])[0].get("alternatives", [{}])[0]
transcript = alternative.get("paragraphs", {}).get("transcript") or alternative.get("transcript") or ""
confidence = alternative.get("confidence")
if not transcript.strip():
raise RuntimeError("Deepgram no devolvio transcripcion")
return transcript.strip(), confidence
def _content_type(self, path: Path) -> str:
ext = path.suffix.lower()
return {
".mp3": "audio/mpeg",
".wav": "audio/wav",
".m4a": "audio/mp4",
".mp4": "audio/mp4",
".webm": "audio/webm",
".ogg": "audio/ogg",
".flac": "audio/flac",
}.get(ext, "application/octet-stream")
@@ -0,0 +1,52 @@
from __future__ import annotations
import base64
from pathlib import Path
import requests
from backend.app.config import settings
class MistralClient:
endpoint = "https://api.mistral.ai/v1/chat/completions"
model = "pixtral-12b-2409"
def ocr_image(self, image_path: Path) -> str:
if not settings.mistral_api_key:
raise RuntimeError("MISTRAL_API_KEY no esta configurada")
data_url = self._image_data_url(image_path)
response = requests.post(
self.endpoint,
headers={
"Content-Type": "application/json",
"Authorization": f"Bearer {settings.mistral_api_key}",
},
json={
"model": self.model,
"messages": [
{
"role": "user",
"content": [
{
"type": "text",
"text": "Extract all text from this image and format it as Markdown. If there are tables, represent them using Markdown table syntax. Do not include commentary.",
},
{"type": "image_url", "image_url": data_url},
],
}
],
"max_tokens": 3000,
},
timeout=180,
)
if response.status_code == 401:
raise RuntimeError("API Key de Mistral invalida")
if not response.ok:
raise RuntimeError(f"Mistral error {response.status_code}: {response.text}")
data = response.json()
return data["choices"][0]["message"]["content"]
def _image_data_url(self, image_path: Path) -> str:
encoded = base64.b64encode(image_path.read_bytes()).decode("ascii")
return f"data:image/jpeg;base64,{encoded}"
View File
+8
View File
@@ -0,0 +1,8 @@
import re
import unicodedata
def slugify(value: str) -> str:
normalized = unicodedata.normalize("NFKD", value).encode("ascii", "ignore").decode("ascii")
slug = re.sub(r"[^a-zA-Z0-9]+", "-", normalized).strip("-").lower()
return slug or "sin-titulo"
+12
View File
@@ -0,0 +1,12 @@
@echo off
setlocal
title Knowledge Station - Cerrar
set "ROOT=%~dp0"
set "ROOT=%ROOT:~0,-1%"
powershell -NoProfile -ExecutionPolicy Bypass -File "%ROOT%\scripts\stop-platform.ps1" -Root "%ROOT%"
echo.
echo Puedes cerrar esta ventana.
pause
exit /b 0
+43
View File
@@ -0,0 +1,43 @@
services:
backend:
build:
context: .
dockerfile: Dockerfile
environment:
APP_ENV: production
APP_DATA_DIR: /app/data
DATABASE_PATH: /app/data/app.db
MISTRAL_API_KEY: ${MISTRAL_API_KEY:-}
DEEPGRAM_API_KEY: ${DEEPGRAM_API_KEY:-}
volumes:
- station-data:/app/data
expose:
- "8000"
restart: unless-stopped
healthcheck:
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=5)"]
interval: 30s
timeout: 10s
retries: 3
start_period: 20s
frontend:
build:
context: ./frontend
dockerfile: Dockerfile
args:
NEXT_PUBLIC_API_URL: ""
BACKEND_INTERNAL_URL: http://backend:8000
environment:
NODE_ENV: production
PORT: 3000
HOSTNAME: 0.0.0.0
expose:
- "3000"
depends_on:
backend:
condition: service_healthy
restart: unless-stopped
volumes:
station-data:
+18
View File
@@ -0,0 +1,18 @@
services:
backend:
build: .
ports:
- "8000:8000"
env_file:
- .env
volumes:
- ./data:/app/data
frontend:
build:
context: ./frontend
ports:
- "3000:3000"
environment:
NEXT_PUBLIC_API_URL: http://127.0.0.1:8000
depends_on:
- backend
File diff suppressed because it is too large Load Diff
+6
View File
@@ -0,0 +1,6 @@
.next
node_modules
.env.local
.env.*.local
npm-debug.log*
Dockerfile*
+1
View File
@@ -0,0 +1 @@
NEXT_PUBLIC_API_URL=http://127.0.0.1:8000
+24
View File
@@ -0,0 +1,24 @@
FROM node:20-alpine AS deps
WORKDIR /app
COPY package.json package-lock.json* ./
RUN npm ci
FROM node:20-alpine AS builder
WORKDIR /app
COPY --from=deps /app/node_modules ./node_modules
COPY . .
ARG NEXT_PUBLIC_API_URL=
ARG BACKEND_INTERNAL_URL=http://backend:8000
ENV NEXT_PUBLIC_API_URL=$NEXT_PUBLIC_API_URL
ENV BACKEND_INTERNAL_URL=$BACKEND_INTERNAL_URL
RUN npm run build
FROM node:20-alpine AS runner
WORKDIR /app
ENV NODE_ENV=production
ENV HOSTNAME=0.0.0.0
COPY --from=builder /app/.next/standalone ./
COPY --from=builder /app/.next/static ./.next/static
COPY --from=builder /app/public ./public
EXPOSE 3000
CMD ["node", "server.js"]
@@ -0,0 +1,65 @@
"use client";
import { useEffect, useState } from "react";
import Link from "next/link";
import ReactMarkdown from "react-markdown";
import remarkGfm from "remark-gfm";
import { api, DocumentItem } from "@/lib/api";
import { Shell } from "@/components/Shell";
import { Button, Card } from "@/components/ui";
export default function DocumentPage({ params }: { params: { documentId: string } }) {
const documentId = Number(params.documentId);
const [document, setDocument] = useState<DocumentItem | null>(null);
const [content, setContent] = useState("");
const [savedContent, setSavedContent] = useState("");
const [status, setStatus] = useState("");
const dirty = content !== savedContent;
useEffect(() => {
Promise.all([api.document(documentId), api.markdown(documentId)]).then(([doc, markdown]) => {
setDocument(doc); setContent(markdown); setSavedContent(markdown);
}).catch((err) => setStatus(err.message));
}, [documentId]);
async function save() {
setStatus("Guardando...");
await api.saveMarkdown(documentId, content);
setSavedContent(content);
setStatus("Guardado");
}
async function copy() {
await navigator.clipboard.writeText(content);
setStatus("Copiado al portapapeles");
}
return (
<Shell>
<Link href="/" className="text-sm text-blue-300">Volver al dashboard</Link>
<div className="mt-4 flex flex-wrap items-center justify-between gap-4">
<div>
<h1 className="text-3xl font-bold">{document?.title || "Documento"}</h1>
<p className="mt-1 text-sm text-slate-400">{dirty ? "Cambios sin guardar" : "Sin cambios pendientes"} {status && `- ${status}`}</p>
</div>
<div className="flex flex-wrap gap-2">
<Button onClick={save} disabled={!dirty}>Guardar</Button>
<Button className="bg-slate-700 hover:bg-slate-600" onClick={copy}>Copiar</Button>
<a className="rounded-xl border border-white/10 px-4 py-2 hover:bg-white/10" href={api.downloadDocumentUrl(documentId)}>Descargar .md</a>
</div>
</div>
<div className="mt-6 grid gap-6 xl:grid-cols-2">
<Card className="p-0">
<div className="border-b border-white/10 px-4 py-3 font-medium">Editor Markdown</div>
<textarea value={content} onChange={(event) => setContent(event.target.value)} className="h-[72vh] w-full resize-none rounded-b-2xl bg-slate-950 p-4 font-mono text-sm leading-6 outline-none" />
</Card>
<Card className="overflow-auto">
<div className="mb-4 font-medium">Vista previa</div>
<article className="prose prose-invert prose-slate">
<ReactMarkdown remarkPlugins={[remarkGfm]}>{content}</ReactMarkdown>
</article>
</Card>
</div>
</Shell>
);
}
+20
View File
@@ -0,0 +1,20 @@
@tailwind base;
@tailwind components;
@tailwind utilities;
:root {
color-scheme: dark;
}
body {
background: #020617;
color: #e5e7eb;
}
textarea, input, button {
font: inherit;
}
.prose {
max-width: none;
}
+15
View File
@@ -0,0 +1,15 @@
import type { Metadata } from "next";
import "./globals.css";
export const metadata: Metadata = {
title: "Knowledge Station",
description: "Procesa clases, documentos y transcripciones a Markdown.",
};
export default function RootLayout({ children }: { children: React.ReactNode }) {
return (
<html lang="es" className="dark">
<body>{children}</body>
</html>
);
}
+87
View File
@@ -0,0 +1,87 @@
"use client";
import Link from "next/link";
import { FormEvent, useEffect, useState } from "react";
import { api, SettingsStatus, Subject } from "@/lib/api";
import { Shell } from "@/components/Shell";
import { Button, Card, Input, Label } from "@/components/ui";
export default function DashboardPage() {
const [subjects, setSubjects] = useState<Subject[]>([]);
const [settings, setSettings] = useState<SettingsStatus | null>(null);
const [name, setName] = useState("");
const [loading, setLoading] = useState(true);
const [error, setError] = useState("");
async function load() {
setError("");
try {
const [subjectList, status] = await Promise.all([api.subjects(), api.settings()]);
setSubjects(subjectList);
setSettings(status);
} catch (err) {
setError(err instanceof Error ? err.message : "No se pudo conectar con la API");
} finally {
setLoading(false);
}
}
useEffect(() => { load(); }, []);
async function createSubject(event: FormEvent) {
event.preventDefault();
if (!name.trim()) return;
await api.createSubject(name.trim());
setName("");
await load();
}
return (
<Shell>
<div className="grid gap-6 lg:grid-cols-[1fr_360px]">
<section>
<div className="mb-6">
<p className="text-sm uppercase tracking-[0.25em] text-blue-300">Plataforma personal</p>
<h1 className="mt-2 text-4xl font-bold tracking-tight">Materias</h1>
<p className="mt-3 max-w-2xl text-slate-400">Organiza clases, PDFs, audios y videos por materia y semana. Todo termina en Markdown listo para ingesta en LLM.</p>
</div>
{error && <Card className="mb-4 border-red-500/30 bg-red-500/10 text-red-200">{error}</Card>}
{loading ? <Card>Cargando...</Card> : (
<div className="grid gap-4 md:grid-cols-2">
{subjects.map((subject) => (
<Link key={subject.id} href={`/subjects/${subject.id}`}>
<Card className="transition hover:border-blue-400/40 hover:bg-blue-500/10">
<div className="text-xl font-semibold">{subject.name}</div>
<div className="mt-2 text-sm text-slate-400">{subject.slug}</div>
<div className="mt-5 text-sm text-blue-300">Abrir materia</div>
</Card>
</Link>
))}
{subjects.length === 0 && <Card className="md:col-span-2">No hay materias todavia. Crea la primera para empezar.</Card>}
</div>
)}
</section>
<aside className="space-y-4">
<Card>
<h2 className="text-lg font-semibold">Nueva materia</h2>
<form className="mt-4 space-y-3" onSubmit={createSubject}>
<Label>Nombre</Label>
<Input value={name} onChange={(event) => setName(event.target.value)} placeholder="Ej. Comportamiento Organizacional" />
<Button className="w-full" disabled={!name.trim()}>Crear materia</Button>
</form>
</Card>
<Card>
<h2 className="text-lg font-semibold">Servicios</h2>
<div className="mt-4 space-y-3 text-sm">
<div className="flex justify-between"><span>Deepgram</span><span className={settings?.deepgram_configured ? "text-emerald-300" : "text-amber-300"}>{settings?.deepgram_configured ? "Configurado" : "Falta key"}</span></div>
<div className="flex justify-between"><span>Mistral OCR</span><span className={settings?.mistral_configured ? "text-emerald-300" : "text-amber-300"}>{settings?.mistral_configured ? "Configurado" : "Falta key"}</span></div>
</div>
</Card>
</aside>
</div>
</Shell>
);
}
+51
View File
@@ -0,0 +1,51 @@
"use client";
import { useEffect, useState } from "react";
import { api, API_URL, SettingsStatus } from "@/lib/api";
import { Shell } from "@/components/Shell";
import { Card } from "@/components/ui";
export default function SettingsPage() {
const [settings, setSettings] = useState<SettingsStatus | null>(null);
const [apiOk, setApiOk] = useState(false);
const [error, setError] = useState("");
useEffect(() => {
Promise.all([api.health(), api.settings()]).then(([, status]) => {
setApiOk(true); setSettings(status);
}).catch((err) => setError(err.message));
}, []);
return (
<Shell>
<h1 className="text-4xl font-bold">Configuracion</h1>
<div className="mt-6 grid gap-4 lg:grid-cols-2">
<Card>
<h2 className="text-xl font-semibold">Estado</h2>
<div className="mt-4 space-y-3 text-sm">
<Row label="API" value={apiOk ? "Activa" : "No disponible"} ok={apiOk} />
<Row label="Deepgram" value={settings?.deepgram_configured ? "Configurado" : "Falta DEEPGRAM_API_KEY"} ok={!!settings?.deepgram_configured} />
<Row label="Mistral" value={settings?.mistral_configured ? "Configurado" : "Falta MISTRAL_API_KEY"} ok={!!settings?.mistral_configured} />
</div>
{error && <p className="mt-4 text-sm text-red-300">{error}</p>}
</Card>
<Card>
<h2 className="text-xl font-semibold">Rutas</h2>
<div className="mt-4 space-y-3 text-sm text-slate-300">
<p><span className="text-slate-500">API:</span> {API_URL}</p>
<p><span className="text-slate-500">Datos:</span> {settings?.data_dir || "-"}</p>
<p><span className="text-slate-500">SQLite:</span> {settings?.database_path || "-"}</p>
</div>
</Card>
</div>
<Card className="mt-4">
<h2 className="text-xl font-semibold">API Keys</h2>
<p className="mt-3 text-slate-400">Configura las claves en el archivo <code className="rounded bg-slate-900 px-2 py-1">.env</code>. No se muestran ni editan desde la UI para evitar pegarlas accidentalmente en capturas o commits.</p>
</Card>
</Shell>
);
}
function Row({ label, value, ok }: { label: string; value: string; ok: boolean }) {
return <div className="flex justify-between gap-4"><span>{label}</span><span className={ok ? "text-emerald-300" : "text-amber-300"}>{value}</span></div>;
}
+121
View File
@@ -0,0 +1,121 @@
"use client";
import Link from "next/link";
import { FormEvent, useEffect, useState } from "react";
import { api, Subject, Week } from "@/lib/api";
import { Shell } from "@/components/Shell";
import { Button, Card, Input, Label } from "@/components/ui";
export default function SubjectPage({ params }: { params: { subjectId: string } }) {
const subjectId = Number(params.subjectId);
const [subject, setSubject] = useState<Subject | null>(null);
const [weeks, setWeeks] = useState<Week[]>([]);
const [number, setNumber] = useState(1);
const [title, setTitle] = useState("");
const [editingWeek, setEditingWeek] = useState<Week | null>(null);
const [editNumber, setEditNumber] = useState(1);
const [editTitle, setEditTitle] = useState("");
const [error, setError] = useState("");
async function load() {
try {
const [subjectData, weekList] = await Promise.all([api.subject(subjectId), api.weeks(subjectId)]);
setSubject(subjectData);
setWeeks(weekList);
setNumber((weekList.at(-1)?.number || 0) + 1);
} catch (err) {
setError(err instanceof Error ? err.message : "No se pudo cargar la materia");
}
}
useEffect(() => { load(); }, [subjectId]);
async function createWeek(event: FormEvent) {
event.preventDefault();
try {
setError("");
await api.createWeek(subjectId, number, title);
setTitle("");
await load();
} catch (err) {
setError(err instanceof Error ? err.message : "No se pudo crear la semana");
}
}
function startEdit(week: Week) {
setEditingWeek(week);
setEditNumber(week.number);
setEditTitle(week.title || "");
setError("");
}
async function saveEdit(event: FormEvent) {
event.preventDefault();
if (!editingWeek) return;
try {
setError("");
await api.updateWeek(subjectId, editingWeek.number, editNumber, editTitle);
setEditingWeek(null);
await load();
} catch (err) {
setError(err instanceof Error ? err.message : "No se pudo editar la semana");
}
}
return (
<Shell>
<Link href="/" className="text-sm text-blue-300">Volver a materias</Link>
<div className="mt-4 grid gap-6 lg:grid-cols-[1fr_360px]">
<section>
<h1 className="text-4xl font-bold tracking-tight">{subject?.name || "Materia"}</h1>
<p className="mt-2 text-slate-400">Divide el conocimiento por semanas y sube fuentes a cada una.</p>
{error && <Card className="mt-4 border-red-500/30 bg-red-500/10 text-red-200">{error}</Card>}
<div className="mt-6 grid gap-4 md:grid-cols-2">
{weeks.map((week) => (
<Card key={week.id} className="transition hover:border-blue-400/40 hover:bg-blue-500/10">
<div className="text-2xl font-semibold">Semana {week.number}</div>
<div className="mt-2 text-slate-400">{week.title || "Sin titulo"}</div>
<div className="mt-5 flex flex-wrap gap-2 text-sm">
<Link className="rounded-lg bg-blue-600 px-3 py-2 font-medium text-white hover:bg-blue-500" href={`/subjects/${subjectId}/weeks/${week.number}`}>Abrir semana</Link>
<button className="rounded-lg border border-white/10 px-3 py-2 text-slate-200 hover:bg-white/10" type="button" onClick={() => startEdit(week)}>Editar</button>
</div>
</Card>
))}
{weeks.length === 0 && <Card className="md:col-span-2">Todavia no hay semanas.</Card>}
</div>
</section>
<aside>
<Card>
{editingWeek ? (
<>
<h2 className="text-lg font-semibold">Editar semana</h2>
<p className="mt-1 text-sm text-slate-400">Cambia el numero o titulo. Si cambias el numero, se renombra la carpeta de la semana.</p>
<form className="mt-4 space-y-3" onSubmit={saveEdit}>
<Label>Numero</Label>
<Input type="number" min={1} value={editNumber} onChange={(event) => setEditNumber(Number(event.target.value))} />
<Label>Titulo</Label>
<Input value={editTitle} onChange={(event) => setEditTitle(event.target.value)} placeholder="Ej. Grupos y equipos" />
<div className="flex gap-2">
<Button className="flex-1">Guardar</Button>
<button className="flex-1 rounded-xl border border-white/10 px-4 py-2 hover:bg-white/10" type="button" onClick={() => setEditingWeek(null)}>Cancelar</button>
</div>
</form>
</>
) : (
<>
<h2 className="text-lg font-semibold">Nueva semana</h2>
<form className="mt-4 space-y-3" onSubmit={createWeek}>
<Label>Numero</Label>
<Input type="number" min={1} value={number} onChange={(event) => setNumber(Number(event.target.value))} />
<Label>Titulo opcional</Label>
<Input value={title} onChange={(event) => setTitle(event.target.value)} placeholder="Ej. Grupos y equipos" />
<Button className="w-full">Crear semana</Button>
</form>
</>
)}
</Card>
</aside>
</div>
</Shell>
);
}
@@ -0,0 +1,343 @@
"use client";
import Link from "next/link";
import { ChangeEvent, DragEvent, useEffect, useMemo, useState } from "react";
import { api, DocumentItem, Job, Source, Subject, Week } from "@/lib/api";
import { Shell } from "@/components/Shell";
import { Button, Card, ConfirmDialog, StatusBadge } from "@/components/ui";
type PendingDelete =
| { type: "document"; item: DocumentItem }
| { type: "source"; item: Source };
export default function WeekPage({ params }: { params: { subjectId: string; weekNumber: string } }) {
const subjectId = Number(params.subjectId);
const weekNumber = Number(params.weekNumber);
const [subject, setSubject] = useState<Subject | null>(null);
const [week, setWeek] = useState<Week | null>(null);
const [sources, setSources] = useState<Source[]>([]);
const [jobs, setJobs] = useState<Job[]>([]);
const [documents, setDocuments] = useState<DocumentItem[]>([]);
const [pdfMethod, setPdfMethod] = useState<"standard" | "mistral_ocr">("standard");
const [pageMode, setPageMode] = useState<"all" | "ranges">("all");
const [pageRanges, setPageRanges] = useState("");
const [pendingFiles, setPendingFiles] = useState<File[]>([]);
const [uploading, setUploading] = useState(false);
const [deletingId, setDeletingId] = useState<string | null>(null);
const [reprocessingId, setReprocessingId] = useState<number | null>(null);
const [pendingDelete, setPendingDelete] = useState<PendingDelete | null>(null);
const [dragActive, setDragActive] = useState(false);
const [error, setError] = useState("");
const activeJobs = useMemo(() => jobs.some((job) => ["queued", "processing"].includes(job.status)), [jobs]);
const pendingPdfCount = useMemo(() => pendingFiles.filter(isPdfFile).length, [pendingFiles]);
async function load() {
const [subjectData, weekData, sourceList, jobList, documentList] = await Promise.all([
api.subject(subjectId), api.week(subjectId, weekNumber), api.sources(subjectId, weekNumber), api.jobs(subjectId, weekNumber), api.documents(subjectId, weekNumber),
]);
setSubject(subjectData); setWeek(weekData); setSources(sourceList); setJobs(jobList); setDocuments(documentList);
}
useEffect(() => { load().catch((err) => setError(err.message)); }, [subjectId, weekNumber]);
useEffect(() => {
if (!activeJobs) return;
const timer = window.setInterval(() => load().catch((err) => setError(err.message)), 2500);
return () => window.clearInterval(timer);
}, [activeJobs]);
async function uploadFileList(files: File[]) {
if (!files.length) return;
setUploading(true); setError("");
try {
const ranges = pageMode === "ranges" ? pageRanges : undefined;
for (const file of files) await api.upload(subjectId, weekNumber, file, pdfMethod === "mistral_ocr", ranges);
await load();
} catch (err) {
setError(err instanceof Error ? err.message : "No se pudo subir archivo");
} finally {
setUploading(false);
setDragActive(false);
}
}
function isPdfFile(file: File) {
return file.type === "application/pdf" || file.name.toLowerCase().endsWith(".pdf");
}
async function prepareFiles(files: File[]) {
if (!files.length) return;
setError("");
setDragActive(false);
if (files.some(isPdfFile)) {
setPendingFiles(files);
return;
}
await uploadFileList(files);
}
async function confirmPendingUpload() {
const files = pendingFiles;
setPendingFiles([]);
await uploadFileList(files);
}
async function uploadFiles(event: ChangeEvent<HTMLInputElement>) {
const files = Array.from(event.target.files || []);
try {
await prepareFiles(files);
} finally {
event.target.value = "";
}
}
function handleDrag(event: DragEvent<HTMLDivElement>) {
event.preventDefault();
event.stopPropagation();
if (uploading) return;
if (event.type === "dragenter" || event.type === "dragover") setDragActive(true);
if (event.type === "dragleave") setDragActive(false);
}
async function handleDrop(event: DragEvent<HTMLDivElement>) {
event.preventDefault();
event.stopPropagation();
if (uploading) return;
const files = Array.from(event.dataTransfer.files || []);
await prepareFiles(files);
}
async function deleteDocument(document: DocumentItem) {
setDeletingId(`document-${document.id}`); setError("");
try {
await api.deleteDocument(document.id);
await load();
setPendingDelete(null);
} catch (err) {
setError(err instanceof Error ? err.message : "No se pudo eliminar el documento");
} finally {
setDeletingId(null);
}
}
async function deleteSource(source: Source) {
setDeletingId(`source-${source.id}`); setError("");
try {
await api.deleteSource(source.id);
await load();
setPendingDelete(null);
} catch (err) {
setError(err instanceof Error ? err.message : "No se pudo eliminar la fuente");
} finally {
setDeletingId(null);
}
}
async function reprocessSource(source: Source) {
setReprocessingId(source.id); setError("");
try {
const ranges = source.source_type === "pdf" && pageMode === "ranges" ? pageRanges : undefined;
await api.reprocessSource(source.id, source.source_type === "pdf" && pdfMethod === "mistral_ocr", ranges);
await load();
} catch (err) {
setError(err instanceof Error ? err.message : "No se pudo reprocesar la fuente");
} finally {
setReprocessingId(null);
}
}
function formatFileSize(size?: number | null) {
if (!size) return "tamano pendiente";
if (size < 1024 * 1024) return `${Math.round(size / 1024)} KB`;
return `${(size / (1024 * 1024)).toFixed(1)} MB`;
}
function sourceProcessingLabel(source: Source) {
if (source.document_id) return `Genero: ${source.document_title}`;
if (["queued", "processing"].includes(source.status)) return "Procesamiento en curso";
if (source.status === "failed") return "Fallo el procesamiento";
return "Sin Markdown generado";
}
function confirmPendingDelete() {
if (!pendingDelete) return;
if (pendingDelete.type === "document") void deleteDocument(pendingDelete.item);
if (pendingDelete.type === "source") void deleteSource(pendingDelete.item);
}
const deleteDialog = pendingDelete ? {
title: pendingDelete.type === "document" ? "Eliminar Markdown procesado" : "Eliminar fuente",
description: pendingDelete.type === "document"
? `Se eliminara el archivo Markdown "${pendingDelete.item.title}". La fuente original quedara disponible.`
: `Se eliminara la fuente "${pendingDelete.item.original_name}" junto con sus trabajos y Markdown procesado asociado.`,
} : null;
return (
<Shell>
<Link href={`/subjects/${subjectId}`} className="text-sm text-blue-300">Volver a {subject?.name || "materia"}</Link>
<div className="mt-4 grid gap-6 lg:grid-cols-[1fr_380px]">
<section className="space-y-6">
<div>
<h1 className="text-4xl font-bold">Semana {weekNumber}</h1>
<p className="mt-2 text-slate-400">{week?.title || "Sube fuentes y genera Markdown procesado."}</p>
</div>
{error && <Card className="border-red-500/30 bg-red-500/10 text-red-200">{error}</Card>}
<Card>
<div className="flex flex-wrap items-center justify-between gap-4">
<div>
<h2 className="text-xl font-semibold">Subir archivos</h2>
<p className="mt-1 text-sm text-slate-400">PDF, DOCX, TXT, Markdown, audio y video.</p>
</div>
<label className="cursor-pointer rounded-xl bg-blue-600 px-4 py-2 font-medium hover:bg-blue-500">
{uploading ? "Subiendo..." : "Seleccionar archivos"}
<input className="hidden" type="file" multiple onChange={uploadFiles} disabled={uploading} />
</label>
</div>
<div
className={`mt-5 rounded-2xl border-2 border-dashed p-8 text-center transition ${dragActive ? "border-blue-400 bg-blue-500/15" : "border-white/15 bg-slate-950/50 hover:border-blue-400/50"} ${uploading ? "opacity-70" : ""}`}
onDragEnter={handleDrag}
onDragOver={handleDrag}
onDragLeave={handleDrag}
onDrop={handleDrop}
>
<div className="mx-auto grid h-14 w-14 place-items-center rounded-2xl bg-blue-500/15 text-2xl"></div>
<p className="mt-4 font-medium">Arrastra archivos aqui</p>
<p className="mt-2 text-sm text-slate-400">Puedes soltar varios archivos a la vez. Se procesaran en esta semana.</p>
<p className="mt-3 text-xs text-slate-500">Soporta PDF, DOCX, TXT, Markdown, audio y video.</p>
{uploading && <p className="mt-4 text-sm text-blue-300">Subiendo archivos...</p>}
</div>
{pendingFiles.length > 0 && (
<div className="mt-5 rounded-2xl border border-blue-400/40 bg-blue-500/10 p-4">
<div className="flex flex-wrap items-start justify-between gap-3">
<div>
<h3 className="font-medium">Configurar PDFs antes de subir</h3>
<p className="mt-1 text-sm text-slate-300">
Detecte {pendingPdfCount} PDF en {pendingFiles.length} archivo(s). Elige como procesarlos antes de continuar.
</p>
</div>
<button className="rounded-lg border border-white/10 px-3 py-1.5 text-sm hover:bg-white/10" type="button" onClick={() => setPendingFiles([])}>Cancelar</button>
</div>
<div className="mt-4 grid gap-4 md:grid-cols-2">
<div>
<label className="text-sm font-medium text-slate-300">Metodo de procesamiento</label>
<select
className="mt-2 w-full rounded-xl border border-white/10 bg-slate-900 px-3 py-2 text-sm outline-none ring-blue-500/30 focus:ring-4"
value={pdfMethod}
onChange={(event) => setPdfMethod(event.target.value as "standard" | "mistral_ocr")}
>
<option value="standard">Extraccion estandar de texto</option>
<option value="mistral_ocr">Mistral OCR (IA)</option>
</select>
</div>
<div>
<label className="text-sm font-medium text-slate-300">Paginas a procesar</label>
<select
className="mt-2 w-full rounded-xl border border-white/10 bg-slate-900 px-3 py-2 text-sm outline-none ring-blue-500/30 focus:ring-4"
value={pageMode}
onChange={(event) => setPageMode(event.target.value as "all" | "ranges")}
>
<option value="all">Todas las paginas</option>
<option value="ranges">Rangos especificos</option>
</select>
</div>
</div>
{pageMode === "ranges" && (
<div className="mt-4">
<label className="text-sm font-medium text-slate-300">Rangos</label>
<input
className="mt-2 w-full rounded-xl border border-white/10 bg-slate-900 px-3 py-2 text-sm outline-none ring-blue-500/30 focus:ring-4"
value={pageRanges}
onChange={(event) => setPageRanges(event.target.value)}
placeholder="Ej: 1-5, 8, 10-12"
/>
<p className="mt-2 text-xs text-slate-500">Separa paginas o rangos con comas. Esta seleccion se aplicara a los PDFs pendientes.</p>
</div>
)}
<div className="mt-4 flex flex-wrap gap-2">
<Button onClick={confirmPendingUpload} disabled={uploading || (pageMode === "ranges" && !pageRanges.trim())}>
{uploading ? "Subiendo..." : "Procesar y subir archivos"}
</Button>
<p className="self-center text-xs text-slate-400">Los archivos que no sean PDF se subiran junto con esta tanda.</p>
</div>
</div>
)}
</Card>
<Card>
<div className="mb-4 flex flex-wrap items-center justify-between gap-3">
<h2 className="text-xl font-semibold">Documentos Markdown</h2>
<div className="flex flex-wrap gap-2">
<a className="rounded-xl border border-white/10 px-3 py-2 text-sm hover:bg-white/10" href={api.exportWeekUrl(subjectId, weekNumber)}>Exportar semana (.md unico)</a>
<a className="rounded-xl border border-white/10 px-3 py-2 text-sm hover:bg-white/10" href={api.exportWeekZipUrl(subjectId, weekNumber)}>Exportar documentos separados (.zip)</a>
</div>
</div>
<div className="space-y-3">
{documents.map((doc) => (
<div key={doc.id} className="flex flex-wrap items-center justify-between gap-3 rounded-xl border border-white/10 p-4 hover:bg-white/5">
<Link href={`/documents/${doc.id}`} className="min-w-0 flex-1">
<div className="font-medium">{doc.title}</div>
<div className="mt-1 text-xs text-slate-400">Fuente: {doc.source_original_name || `#${doc.source_id}`}</div>
<div className="mt-1 truncate text-xs text-slate-500">{doc.markdown_path}</div>
</Link>
<Button
className="bg-red-600 px-3 py-1.5 text-sm hover:bg-red-500"
disabled={deletingId === `document-${doc.id}`}
onClick={() => setPendingDelete({ type: "document", item: doc })}
>
{deletingId === `document-${doc.id}` ? "Eliminando..." : "Eliminar"}
</Button>
</div>
))}
{documents.length === 0 && <p className="text-slate-400">Aun no hay documentos procesados.</p>}
</div>
</Card>
</section>
<aside className="space-y-4">
<Card>
<h2 className="text-lg font-semibold">Trabajos</h2>
<div className="mt-4 space-y-3">
{jobs.map((job) => <div key={job.id} className="rounded-xl bg-slate-900 p-3 text-sm"><div className="flex justify-between"><span>#{job.id}</span><StatusBadge status={job.status} /></div><p className="mt-2 text-slate-400">{job.message}</p></div>)}
{jobs.length === 0 && <p className="text-sm text-slate-400">Sin trabajos.</p>}
</div>
</Card>
<Card>
<h2 className="text-lg font-semibold">Fuentes</h2>
<div className="mt-4 space-y-3">
{sources.map((source) => {
const busy = ["queued", "processing"].includes(source.status);
return (
<div key={source.id} className="rounded-xl bg-slate-900 p-3 text-sm">
<div className="flex items-start justify-between gap-2"><span className="truncate">{source.original_name}</span><StatusBadge status={source.status} /></div>
<div className="mt-2 space-y-1 text-xs text-slate-500">
<p className={source.document_id ? "text-emerald-300" : source.status === "failed" ? "text-red-300" : "text-amber-300"}>{sourceProcessingLabel(source)}</p>
<p>{source.source_type} · {formatFileSize(source.file_size)}{source.processor ? ` · ${source.processor}` : ""}</p>
{source.sha256 && <p className="truncate">sha256: {source.sha256.slice(0, 12)}...</p>}
{source.page_ranges && <p>paginas: {source.page_ranges}</p>}
</div>
<div className="mt-3 flex flex-wrap justify-end gap-2">
{source.document_id && <Link className="rounded-lg border border-emerald-500/30 px-2 py-1 text-xs text-emerald-200 hover:bg-emerald-500/10" href={`/documents/${source.document_id}`}>Ver Markdown</Link>}
<button className="rounded-lg border border-white/10 px-2 py-1 text-xs text-slate-200 hover:bg-white/10 disabled:cursor-not-allowed disabled:opacity-50" disabled={busy || reprocessingId === source.id} onClick={() => reprocessSource(source)}>{reprocessingId === source.id ? "En cola..." : "Reprocesar"}</button>
<button className="rounded-lg border border-red-500/30 px-2 py-1 text-xs text-red-200 hover:bg-red-500/10 disabled:cursor-not-allowed disabled:opacity-50" disabled={deletingId === `source-${source.id}` || busy} onClick={() => setPendingDelete({ type: "source", item: source })}>{deletingId === `source-${source.id}` ? "Eliminando..." : "Eliminar"}</button>
</div>
</div>
);
})}
{sources.length === 0 && <p className="text-sm text-slate-400">Sin fuentes.</p>}
</div>
</Card>
</aside>
</div>
<ConfirmDialog
open={Boolean(deleteDialog)}
title={deleteDialog?.title || "Confirmar accion"}
description={deleteDialog?.description || ""}
confirmLabel="Eliminar"
loading={Boolean(deletingId)}
onConfirm={confirmPendingDelete}
onCancel={() => setPendingDelete(null)}
/>
</Shell>
);
}
+25
View File
@@ -0,0 +1,25 @@
import Link from "next/link";
import type { ReactNode } from "react";
export function Shell({ children }: { children: ReactNode }) {
return (
<div className="min-h-screen bg-slate-950 text-slate-100">
<header className="border-b border-white/10 bg-slate-950/90 backdrop-blur">
<div className="mx-auto flex max-w-7xl items-center justify-between px-5 py-4">
<Link href="/" className="flex items-center gap-3">
<div className="grid h-10 w-10 place-items-center rounded-2xl bg-blue-600 font-bold">KS</div>
<div>
<div className="font-semibold tracking-tight">Knowledge Station</div>
<div className="text-xs text-slate-400">Clases a Markdown para LLM</div>
</div>
</Link>
<nav className="flex items-center gap-2 text-sm text-slate-300">
<Link className="rounded-lg px-3 py-2 hover:bg-white/10" href="/">Materias</Link>
<Link className="rounded-lg px-3 py-2 hover:bg-white/10" href="/settings">Configuracion</Link>
</nav>
</div>
</header>
<main className="mx-auto max-w-7xl px-5 py-8">{children}</main>
</div>
);
}
+67
View File
@@ -0,0 +1,67 @@
export function Card({ children, className = "" }: { children: React.ReactNode; className?: string }) {
return <div className={`rounded-2xl border border-white/10 bg-white/[0.04] p-5 shadow-xl shadow-black/20 ${className}`}>{children}</div>;
}
export function Button({ children, className = "", ...props }: React.ButtonHTMLAttributes<HTMLButtonElement>) {
return <button className={`rounded-xl bg-blue-600 px-4 py-2 font-medium text-white hover:bg-blue-500 disabled:cursor-not-allowed disabled:opacity-50 ${className}`} {...props}>{children}</button>;
}
export function Input(props: React.InputHTMLAttributes<HTMLInputElement>) {
return <input className="w-full rounded-xl border border-white/10 bg-slate-900 px-4 py-2 text-slate-100 outline-none ring-blue-500/30 focus:ring-4" {...props} />;
}
export function Label({ children }: { children: React.ReactNode }) {
return <label className="text-sm font-medium text-slate-300">{children}</label>;
}
export function StatusBadge({ status }: { status: string }) {
const map: Record<string, string> = {
completed: "bg-emerald-500/15 text-emerald-300",
processing: "bg-blue-500/15 text-blue-300",
queued: "bg-amber-500/15 text-amber-300",
failed: "bg-red-500/15 text-red-300",
stored: "bg-slate-500/15 text-slate-300",
processed: "bg-emerald-500/15 text-emerald-300",
};
return <span className={`rounded-full px-2.5 py-1 text-xs font-medium ${map[status] || "bg-slate-500/15 text-slate-300"}`}>{status}</span>;
}
export function ConfirmDialog({
open,
title,
description,
confirmLabel = "Confirmar",
cancelLabel = "Cancelar",
loading = false,
onConfirm,
onCancel,
}: {
open: boolean;
title: string;
description: string;
confirmLabel?: string;
cancelLabel?: string;
loading?: boolean;
onConfirm: () => void;
onCancel: () => void;
}) {
if (!open) return null;
return (
<div className="fixed inset-0 z-50 grid place-items-center bg-slate-950/75 p-4 backdrop-blur-sm" role="dialog" aria-modal="true" aria-labelledby="confirm-dialog-title">
<div className="w-full max-w-md rounded-2xl border border-white/10 bg-slate-900 p-5 shadow-2xl shadow-black/40">
<div className="flex items-start gap-3">
<div className="grid h-10 w-10 shrink-0 place-items-center rounded-full bg-red-500/15 text-red-200">!</div>
<div>
<h2 id="confirm-dialog-title" className="text-lg font-semibold text-white">{title}</h2>
<p className="mt-2 text-sm leading-6 text-slate-300">{description}</p>
</div>
</div>
<div className="mt-6 flex flex-col-reverse gap-2 sm:flex-row sm:justify-end">
<button className="rounded-xl border border-white/10 px-4 py-2 text-slate-200 hover:bg-white/10 disabled:cursor-not-allowed disabled:opacity-50" type="button" disabled={loading} onClick={onCancel}>{cancelLabel}</button>
<Button className="bg-red-600 hover:bg-red-500" disabled={loading} onClick={onConfirm}>{loading ? "Eliminando..." : confirmLabel}</Button>
</div>
</div>
</div>
);
}
+141
View File
@@ -0,0 +1,141 @@
export const API_URL = process.env.NEXT_PUBLIC_API_URL ?? "http://127.0.0.1:8000";
export type Subject = { id: number; name: string; slug: string; created_at: string };
export type Week = { id: number; subject_id: number; number: number; title?: string | null; created_at: string };
export type Source = {
id: number;
week_id: number;
original_name: string;
stored_path: string;
source_type: string;
status: string;
file_size?: number | null;
sha256?: string | null;
mime_type?: string | null;
processor?: string | null;
page_ranges?: string | null;
processed_at?: string | null;
document_id?: number | null;
document_title?: string | null;
document_markdown_path?: string | null;
document_created_at?: string | null;
created_at: string;
};
export type Job = { id: number; status: string; message: string; source_id?: number | null; document_id?: number | null; created_at: string; updated_at: string };
export type DocumentItem = {
id: number;
source_id: number;
title: string;
markdown_path: string;
source_original_name?: string | null;
source_type?: string | null;
source_status?: string | null;
source_processor?: string | null;
source_page_ranges?: string | null;
created_at: string;
};
export type SettingsStatus = { mistral_configured: boolean; deepgram_configured: boolean; data_dir: string; database_path: string };
async function parseResponse<T>(response: Response): Promise<T> {
if (!response.ok) {
const detail = await response.json().catch(() => null);
throw new Error(detail?.detail || `Error ${response.status}`);
}
return response.json() as Promise<T>;
}
export const api = {
async health() {
return parseResponse<{ status: string }>(await fetch(`${API_URL}/health`, { cache: "no-store" }));
},
async settings() {
return parseResponse<SettingsStatus>(await fetch(`${API_URL}/settings/status`, { cache: "no-store" }));
},
async subjects() {
return parseResponse<Subject[]>(await fetch(`${API_URL}/subjects`, { cache: "no-store" }));
},
async subject(id: number) {
return parseResponse<Subject>(await fetch(`${API_URL}/subjects/${id}`, { cache: "no-store" }));
},
async createSubject(name: string) {
return parseResponse<Subject>(await fetch(`${API_URL}/subjects`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ name }),
}));
},
async weeks(subjectId: number) {
return parseResponse<Week[]>(await fetch(`${API_URL}/subjects/${subjectId}/weeks`, { cache: "no-store" }));
},
async week(subjectId: number, weekNumber: number) {
return parseResponse<Week>(await fetch(`${API_URL}/subjects/${subjectId}/weeks/${weekNumber}`, { cache: "no-store" }));
},
async createWeek(subjectId: number, number: number, title: string) {
return parseResponse<Week>(await fetch(`${API_URL}/subjects/${subjectId}/weeks`, {
method: "POST",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ number, title: title || null }),
}));
},
async updateWeek(subjectId: number, currentNumber: number, number: number, title: string) {
return parseResponse<Week>(await fetch(`${API_URL}/subjects/${subjectId}/weeks/${currentNumber}`, {
method: "PUT",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ number, title: title || null }),
}));
},
async sources(subjectId: number, weekNumber: number) {
return parseResponse<Source[]>(await fetch(`${API_URL}/subjects/${subjectId}/weeks/${weekNumber}/sources`, { cache: "no-store" }));
},
async jobs(subjectId: number, weekNumber: number) {
return parseResponse<Job[]>(await fetch(`${API_URL}/subjects/${subjectId}/weeks/${weekNumber}/jobs`, { cache: "no-store" }));
},
async documents(subjectId: number, weekNumber: number) {
return parseResponse<DocumentItem[]>(await fetch(`${API_URL}/subjects/${subjectId}/weeks/${weekNumber}/documents`, { cache: "no-store" }));
},
async deleteSource(sourceId: number) {
return parseResponse<{ status: string }>(await fetch(`${API_URL}/sources/${sourceId}`, { method: "DELETE" }));
},
async deleteDocument(documentId: number) {
return parseResponse<{ status: string }>(await fetch(`${API_URL}/documents/${documentId}`, { method: "DELETE" }));
},
async reprocessSource(sourceId: number, useOcr: boolean, pageRanges?: string) {
const params = new URLSearchParams({ use_ocr: String(useOcr) });
if (pageRanges?.trim()) params.set("page_ranges", pageRanges.trim());
return parseResponse<{ source: Source; job: Job }>(await fetch(`${API_URL}/sources/${sourceId}/reprocess?${params.toString()}`, { method: "POST" }));
},
async upload(subjectId: number, weekNumber: number, file: File, useOcr: boolean, pageRanges?: string) {
const form = new FormData();
form.append("file", file);
const params = new URLSearchParams({ use_ocr: String(useOcr) });
if (pageRanges?.trim()) params.set("page_ranges", pageRanges.trim());
return parseResponse<{ source: Source; job: Job }>(await fetch(`${API_URL}/subjects/${subjectId}/weeks/${weekNumber}/files?${params.toString()}`, {
method: "POST",
body: form,
}));
},
async markdown(documentId: number) {
const response = await fetch(`${API_URL}/documents/${documentId}/markdown`, { cache: "no-store" });
if (!response.ok) throw new Error(`Error ${response.status}`);
return response.text();
},
async document(documentId: number) {
return parseResponse<DocumentItem>(await fetch(`${API_URL}/documents/${documentId}`, { cache: "no-store" }));
},
async saveMarkdown(documentId: number, content: string) {
return parseResponse<DocumentItem>(await fetch(`${API_URL}/documents/${documentId}/markdown`, {
method: "PUT",
headers: { "Content-Type": "application/json" },
body: JSON.stringify({ content }),
}));
},
downloadDocumentUrl(documentId: number) {
return `${API_URL}/documents/${documentId}/download`;
},
exportWeekUrl(subjectId: number, weekNumber: number) {
return `${API_URL}/subjects/${subjectId}/weeks/${weekNumber}/export`;
},
exportWeekZipUrl(subjectId: number, weekNumber: number) {
return `${API_URL}/subjects/${subjectId}/weeks/${weekNumber}/export.zip`;
},
};
+5
View File
@@ -0,0 +1,5 @@
/// <reference types="next" />
/// <reference types="next/image-types/global" />
// NOTE: This file should not be edited
// see https://nextjs.org/docs/app/building-your-application/configuring/typescript for more information.
+18
View File
@@ -0,0 +1,18 @@
/** @type {import('next').NextConfig} */
const backendUrl = process.env.BACKEND_INTERNAL_URL || "http://127.0.0.1:8000";
const nextConfig = {
output: "standalone",
async rewrites() {
return [
{ source: "/health", destination: `${backendUrl}/health` },
{ source: "/settings/:path*", destination: `${backendUrl}/settings/:path*` },
{ source: "/subjects/:path*", destination: `${backendUrl}/subjects/:path*` },
{ source: "/sources/:path*", destination: `${backendUrl}/sources/:path*` },
{ source: "/documents/:path*", destination: `${backendUrl}/documents/:path*` },
{ source: "/jobs/:path*", destination: `${backendUrl}/jobs/:path*` },
];
},
};
export default nextConfig;
+3106
View File
File diff suppressed because it is too large Load Diff
+28
View File
@@ -0,0 +1,28 @@
{
"name": "knowledge-station-frontend",
"version": "0.1.0",
"private": true,
"scripts": {
"dev": "next dev",
"build": "next build",
"start": "next start",
"lint": "next lint"
},
"dependencies": {
"@tailwindcss/typography": "^0.5.15",
"next": "^14.2.23",
"react": "^18.3.1",
"react-dom": "^18.3.1",
"react-markdown": "^9.0.3",
"remark-gfm": "^4.0.0"
},
"devDependencies": {
"@types/node": "^22.10.10",
"@types/react": "^18.3.18",
"@types/react-dom": "^18.3.5",
"autoprefixer": "^10.4.20",
"postcss": "^8.5.1",
"tailwindcss": "^3.4.17",
"typescript": "^5.7.3"
}
}
+6
View File
@@ -0,0 +1,6 @@
module.exports = {
plugins: {
tailwindcss: {},
autoprefixer: {},
},
};
View File
+18
View File
@@ -0,0 +1,18 @@
import type { Config } from "tailwindcss";
const config: Config = {
content: ["./app/**/*.{ts,tsx}", "./components/**/*.{ts,tsx}", "./lib/**/*.{ts,tsx}"],
darkMode: "class",
theme: {
extend: {
colors: {
ink: "#0f172a",
panel: "#111827",
brand: "#2563eb",
},
},
},
plugins: [require("@tailwindcss/typography")],
};
export default config;
+22
View File
@@ -0,0 +1,22 @@
{
"compilerOptions": {
"target": "ES2017",
"lib": ["dom", "dom.iterable", "esnext"],
"allowJs": false,
"skipLibCheck": true,
"strict": true,
"noEmit": true,
"esModuleInterop": true,
"module": "esnext",
"moduleResolution": "bundler",
"baseUrl": ".",
"resolveJsonModule": true,
"isolatedModules": true,
"jsx": "preserve",
"incremental": true,
"plugins": [{ "name": "next" }],
"paths": { "@/*": ["./*"] }
},
"include": ["next-env.d.ts", "**/*.ts", "**/*.tsx", ".next/types/**/*.ts"],
"exclude": ["node_modules"]
}
+19
View File
@@ -0,0 +1,19 @@
[variables]
NODE_ENV = "production"
NEXT_PUBLIC_API_URL = ""
PYTHONUNBUFFERED = "1"
[phases.setup]
nixPkgs = ["python312", "nodejs_22", "ffmpeg"]
[phases.install]
cmds = [
"pip install --no-cache-dir -r requirements.txt",
"cd frontend && npm ci"
]
[phases.build]
cmds = ["cd frontend && npm run build"]
[start]
cmd = "bash scripts/start-coolify.sh"
+8
View File
@@ -0,0 +1,8 @@
fastapi==0.115.6
uvicorn[standard]==0.34.0
python-multipart==0.0.20
pydantic-settings==2.7.1
python-dotenv==1.0.1
requests==2.32.3
python-docx==1.1.2
PyMuPDF==1.25.1
+28
View File
@@ -0,0 +1,28 @@
#!/usr/bin/env bash
set -euo pipefail
export APP_DATA_DIR="${APP_DATA_DIR:-/app/data}"
export HOSTNAME="0.0.0.0"
uvicorn backend.app.main:app --host 127.0.0.1 --port 8000 &
backend_pid=$!
cleanup() {
kill "$backend_pid" 2>/dev/null || true
}
trap cleanup EXIT INT TERM
cd frontend
if [ -f ".next/standalone/server.js" ]; then
mkdir -p .next/standalone/.next
rm -rf .next/standalone/.next/static .next/standalone/public
cp -r .next/static .next/standalone/.next/static
if [ -d "public" ]; then
cp -r public .next/standalone/public
fi
cd .next/standalone
PORT="${PORT:-3000}" node server.js
else
npm run start -- --hostname 0.0.0.0 --port "${PORT:-3000}"
fi
+167
View File
@@ -0,0 +1,167 @@
param(
[Parameter(Mandatory = $true)]
[string]$Root
)
$ErrorActionPreference = "Stop"
$Root = $Root.Trim().Trim('"').TrimEnd("\")
$Root = (Resolve-Path -LiteralPath $Root).Path
$Venv = Join-Path $Root ".venv"
$Python = Join-Path $Venv "Scripts\python.exe"
$Pip = Join-Path $Venv "Scripts\pip.exe"
$Frontend = Join-Path $Root "frontend"
$Runtime = Join-Path $Root ".runtime"
function Write-Step($Message) {
Write-Host ""
Write-Host $Message -ForegroundColor Cyan
}
function Test-Port([int]$Port) {
$client = [Net.Sockets.TcpClient]::new()
try {
$client.Connect("127.0.0.1", $Port)
return $true
}
catch {
return $false
}
finally {
$client.Close()
}
}
function Find-FreePort([int]$StartPort) {
$port = $StartPort
while (Test-Port $port) { $port++ }
return $port
}
function Get-LivePid([string]$PidFile) {
if (!(Test-Path $PidFile)) { return $null }
$raw = (Get-Content $PidFile -ErrorAction SilentlyContinue | Select-Object -First 1)
if (!$raw) { return $null }
$process = Get-Process -Id ([int]$raw) -ErrorAction SilentlyContinue
if ($process) { return [int]$raw }
Remove-Item $PidFile -Force -ErrorAction SilentlyContinue
return $null
}
function Stop-Tree([int]$Id) {
Get-CimInstance Win32_Process | Where-Object { $_.ParentProcessId -eq $Id } | ForEach-Object { Stop-Tree $_.ProcessId }
Stop-Process -Id $Id -Force -ErrorAction SilentlyContinue
}
function Invoke-Checked([scriptblock]$Command, [string]$ErrorMessage) {
& $Command
if ($LASTEXITCODE -ne 0) { throw $ErrorMessage }
}
Write-Host ""
Write-Host "============================================"
Write-Host " Knowledge Station - Plataforma completa"
Write-Host "============================================"
New-Item -ItemType Directory -Force -Path $Runtime | Out-Null
$legacyPid = Join-Path $Runtime "server.pid"
if (Test-Path $legacyPid) {
$legacyRaw = Get-Content $legacyPid -ErrorAction SilentlyContinue | Select-Object -First 1
if ($legacyRaw) { Stop-Tree ([int]$legacyRaw) }
Remove-Item $legacyPid -Force -ErrorAction SilentlyContinue
}
if (!(Test-Path $Python)) {
Write-Step "Preparando Python por primera vez..."
& py -m venv $Venv 2>$null
if ($LASTEXITCODE -ne 0) {
& python -m venv $Venv
}
if (!(Test-Path $Python)) {
throw "No se pudo preparar Python. Instala Python 3.12 o superior."
}
}
$EnvFile = Join-Path $Root ".env"
$EnvExample = Join-Path $Root ".env.example"
if (!(Test-Path $EnvFile)) {
Write-Step "Creando configuracion local .env..."
Copy-Item $EnvExample $EnvFile
Write-Host "Agrega tus API keys en .env para audio, video u OCR." -ForegroundColor Yellow
}
Write-Step "Verificando dependencias de Python..."
Invoke-Checked { & $Python -m pip install --upgrade pip --disable-pip-version-check | Out-Null } "No se pudo actualizar pip."
Invoke-Checked { & $Pip install -r (Join-Path $Root "requirements.txt") } "No se pudieron instalar dependencias de Python."
Write-Step "Verificando Node.js..."
$node = Get-Command node -ErrorAction SilentlyContinue
$npm = Get-Command npm.cmd -ErrorAction SilentlyContinue
if (!$npm) { $npm = Get-Command npm -ErrorAction SilentlyContinue }
if (!$node -or !$npm) {
throw "Node.js LTS no esta instalado o npm no esta disponible. Instala Node.js desde https://nodejs.org/."
}
if (!(Test-Path (Join-Path $Frontend "node_modules"))) {
Write-Step "Instalando dependencias del frontend. Esto puede tardar unos minutos..."
Invoke-Checked { & $npm.Source install --prefix $Frontend } "No se pudieron instalar dependencias del frontend."
}
else {
Write-Host "Dependencias del frontend encontradas."
}
$BackendPidFile = Join-Path $Runtime "backend.pid"
$FrontendPidFile = Join-Path $Runtime "frontend.pid"
$BackendPortFile = Join-Path $Runtime "backend.port"
$FrontendPortFile = Join-Path $Runtime "frontend.port"
Write-Step "Iniciando backend..."
$backendPid = Get-LivePid $BackendPidFile
if ($backendPid) {
$backendPort = Get-Content $BackendPortFile -ErrorAction SilentlyContinue | Select-Object -First 1
if (!$backendPort) { $backendPort = 8000 }
Write-Host "Backend ya estaba activo en puerto $backendPort."
}
else {
$backendPort = Find-FreePort 8000
$backend = Start-Process -FilePath $Python -ArgumentList @("-m", "uvicorn", "backend.app.main:app", "--host", "127.0.0.1", "--port", [string]$backendPort, "--reload") -WorkingDirectory $Root -PassThru
Set-Content $BackendPidFile $backend.Id
Set-Content $BackendPortFile $backendPort
Write-Host "Backend iniciado en puerto $backendPort."
}
$ApiUrl = "http://127.0.0.1:$backendPort"
Set-Content (Join-Path $Frontend ".env.local") "NEXT_PUBLIC_API_URL=$ApiUrl"
Write-Step "Iniciando frontend..."
$frontendPid = Get-LivePid $FrontendPidFile
if ($frontendPid) {
$frontendPort = Get-Content $FrontendPortFile -ErrorAction SilentlyContinue | Select-Object -First 1
if (!$frontendPort) { $frontendPort = 3000 }
Write-Host "Frontend ya estaba activo en puerto $frontendPort."
}
else {
$frontendPort = Find-FreePort 3000
$nextDir = Join-Path $Frontend ".next"
if (Test-Path $nextDir) {
Get-ChildItem $nextDir -Recurse -Force -ErrorAction SilentlyContinue | ForEach-Object {
try { $_.Attributes = "Normal" } catch {}
}
Remove-Item $nextDir -Recurse -Force -ErrorAction SilentlyContinue
}
$frontend = Start-Process -FilePath $npm.Source -ArgumentList @("run", "dev", "--", "-p", [string]$frontendPort) -WorkingDirectory $Frontend -PassThru
Set-Content $FrontendPidFile $frontend.Id
Set-Content $FrontendPortFile $frontendPort
Write-Host "Frontend iniciado en puerto $frontendPort."
}
$AppUrl = "http://127.0.0.1:$frontendPort"
Set-Content (Join-Path $Runtime "last.url") $AppUrl
Start-Sleep -Seconds 4
Write-Host ""
Write-Host "Plataforma abierta:" -ForegroundColor Green
Write-Host "Frontend: $AppUrl"
Write-Host "API: $ApiUrl"
Write-Host ""
Start-Process $AppUrl
+47
View File
@@ -0,0 +1,47 @@
param(
[Parameter(Mandatory = $true)]
[string]$Root
)
$ErrorActionPreference = "SilentlyContinue"
$Root = $Root.Trim().Trim('"').TrimEnd("\")
$Root = (Resolve-Path -LiteralPath $Root).Path.TrimEnd("\")
$Runtime = Join-Path $Root ".runtime"
function Stop-Tree([int]$Id) {
Get-CimInstance Win32_Process | Where-Object { $_.ParentProcessId -eq $Id } | ForEach-Object { Stop-Tree $_.ProcessId }
Stop-Process -Id $Id -Force
}
Write-Host ""
Write-Host "============================================"
Write-Host " Knowledge Station - Cerrar plataforma"
Write-Host "============================================"
Write-Host ""
if (!(Test-Path $Runtime)) {
Write-Host "No hay registro de servicios abiertos."
exit 0
}
foreach ($name in @("backend.pid", "frontend.pid", "server.pid")) {
$file = Join-Path $Runtime $name
if (Test-Path $file) {
$raw = Get-Content $file | Select-Object -First 1
if ($raw) { Stop-Tree ([int]$raw) }
}
}
Get-CimInstance Win32_Process |
Where-Object {
$_.CommandLine -and
$_.CommandLine -like "*$Root*" -and
($_.CommandLine -like "*uvicorn*backend.app.main:app*" -or $_.CommandLine -like "*next*dev*")
} |
ForEach-Object { Stop-Tree $_.ProcessId }
foreach ($name in @("backend.pid", "frontend.pid", "server.pid", "backend.port", "frontend.port", "last.url")) {
Remove-Item (Join-Path $Runtime $name) -Force
}
Write-Host "Plataforma cerrada de forma segura." -ForegroundColor Green