1
This commit is contained in:
@@ -0,0 +1,19 @@
|
||||
.git
|
||||
.venv
|
||||
__pycache__
|
||||
*.pyc
|
||||
.pytest_cache
|
||||
.mypy_cache
|
||||
.ruff_cache
|
||||
.env
|
||||
.env.*
|
||||
data
|
||||
frontend/.next
|
||||
frontend/node_modules
|
||||
frontend/.env.local
|
||||
frontend/.env.*.local
|
||||
node_modules
|
||||
npm-debug.log*
|
||||
Dockerfile*
|
||||
docker-compose*.yml
|
||||
docker-compose*.yaml
|
||||
@@ -0,0 +1,5 @@
|
||||
APP_ENV=development
|
||||
APP_DATA_DIR=./data
|
||||
DATABASE_PATH=./data/app.db
|
||||
MISTRAL_API_KEY=
|
||||
DEEPGRAM_API_KEY=
|
||||
+14
@@ -0,0 +1,14 @@
|
||||
.env
|
||||
__pycache__/
|
||||
*.py[cod]
|
||||
.pytest_cache/
|
||||
.venv/
|
||||
venv/
|
||||
node_modules/
|
||||
.next/
|
||||
.runtime/
|
||||
data/
|
||||
tmp/
|
||||
*.db
|
||||
*.sqlite
|
||||
frontend/.env.local
|
||||
@@ -0,0 +1,32 @@
|
||||
# AGENTS.md
|
||||
|
||||
## Repo Shape
|
||||
- The primary app is now `backend/` (Python/FastAPI) plus `frontend/` (Next.js/TypeScript/Tailwind); `estacion documentos.html` is a browser-only prototype/reference for behavior.
|
||||
- Runtime data is stored under `data/` and is gitignored; do not commit uploaded class files, generated Markdown, SQLite DBs, temporary audio, or API keys.
|
||||
- The included `Griffin(2020).Comportamiento_organizacionalSOLOCAPITULO7.13Ed.pdf` is a local sample/source PDF for import behavior.
|
||||
|
||||
## Commands
|
||||
- Local setup: `python -m venv .venv`, `.venv\Scripts\activate`, `pip install -r requirements.txt`, copy `.env.example` to `.env`.
|
||||
- Run API: `uvicorn backend.app.main:app --reload` from the repo root.
|
||||
- Run frontend: `cd frontend`, `npm install`, `npm run dev -- -p 3000` with `NEXT_PUBLIC_API_URL=http://127.0.0.1:8000`.
|
||||
- Windows non-technical flow: double-click `abrir_estacion.bat`; it chooses free ports, starts backend/frontend, and opens the browser. Use `cerrar_estacion.bat` to stop only this project.
|
||||
- Docker/Coolify check: `docker compose up --build`.
|
||||
- Basic syntax check: `python -m compileall backend`.
|
||||
|
||||
## Platform Behavior
|
||||
- Knowledge is organized as `Materia -> Semana -> fuentes raw -> Markdown procesado -> exports` under `data/subjects/<materia>/week-XX/`.
|
||||
- `KnowledgeManager` owns filesystem layout and SQLite metadata; processors should not write final Markdown directly.
|
||||
- `IngestionService` selects processors by `source_type` and wraps output with Markdown frontmatter for LLM ingestion.
|
||||
- Text and Markdown are read directly; `.docx` uses `python-docx`; PDFs use PyMuPDF text extraction unless `use_ocr=true` is passed. PDF uploads can also pass `page_ranges` like `1-5,8,10-12`; ranges apply to both standard extraction and Mistral OCR.
|
||||
- OCR uses Mistral `https://api.mistral.ai/v1/chat/completions` with model `pixtral-12b-2409`.
|
||||
- Audio uses Deepgram `https://api.deepgram.com/v1/listen?model=nova-3&smart_format=true&language=es`.
|
||||
- Video processing requires `ffmpeg`; audio is extracted to `data/tmp/`, transcribed with Deepgram, then deleted.
|
||||
- The frontend calls the backend through `NEXT_PUBLIC_API_URL`; when ports change, update `frontend/.env.local` or use the adaptive `.bat`.
|
||||
|
||||
## Secrets And Deployment
|
||||
- Use `.env` locally and Coolify environment variables in production; never hardcode or commit `MISTRAL_API_KEY` or `DEEPGRAM_API_KEY`.
|
||||
- For Coolify, mount persistent storage for `/app/data` or all subjects, outputs, and SQLite state will be lost on redeploy.
|
||||
|
||||
## Prototype Notes
|
||||
- Preserve the Spanish-language UX and Markdown-first output conventions from `estacion documentos.html` when building the new UI.
|
||||
- The prototype sanitizes preview HTML with DOMPurify after `marked.parse`; keep equivalent sanitization in any future web UI.
|
||||
+18
@@ -0,0 +1,18 @@
|
||||
FROM python:3.12-slim
|
||||
|
||||
ENV PYTHONDONTWRITEBYTECODE=1
|
||||
ENV PYTHONUNBUFFERED=1
|
||||
|
||||
WORKDIR /app
|
||||
|
||||
RUN apt-get update \
|
||||
&& apt-get install -y --no-install-recommends ffmpeg \
|
||||
&& rm -rf /var/lib/apt/lists/*
|
||||
|
||||
COPY requirements.txt ./
|
||||
RUN pip install --no-cache-dir -r requirements.txt
|
||||
|
||||
COPY backend ./backend
|
||||
|
||||
EXPOSE 8000
|
||||
CMD ["uvicorn", "backend.app.main:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||
Binary file not shown.
@@ -0,0 +1,89 @@
|
||||
# Knowledge Station
|
||||
|
||||
Plataforma para organizar conocimiento escolar por materia y semana, procesar documentos/audio/video y generar archivos Markdown optimizados para ingesta en LLM.
|
||||
|
||||
El archivo `estacion documentos.html` queda como prototipo funcional de referencia. La plataforma principal usa `backend/` (FastAPI) y `frontend/` (Next.js).
|
||||
|
||||
## Capacidades Iniciales
|
||||
|
||||
- Materias y semanas persistidas en SQLite.
|
||||
- Estructura de archivos por materia/semana en `data/subjects/`.
|
||||
- Ingesta de `.txt`, `.md`, `.docx`, `.pdf`, audio y video.
|
||||
- PDF con texto mediante PyMuPDF.
|
||||
- OCR opcional de PDF con Mistral.
|
||||
- Audio con Deepgram en español.
|
||||
- Video con extracción temporal de audio vía `ffmpeg` y transcripción con Deepgram.
|
||||
- Exportación semanal consolidada a Markdown.
|
||||
- UI dedicada con materias, semanas, subida de archivos, polling de trabajos y editor Markdown con preview.
|
||||
|
||||
## Desarrollo Local
|
||||
|
||||
```bash
|
||||
python -m venv .venv
|
||||
.venv\Scripts\activate
|
||||
pip install -r requirements.txt
|
||||
copy .env.example .env
|
||||
uvicorn backend.app.main:app --reload
|
||||
```
|
||||
|
||||
En otra terminal:
|
||||
|
||||
```bash
|
||||
cd frontend
|
||||
npm install
|
||||
set NEXT_PUBLIC_API_URL=http://127.0.0.1:8000
|
||||
npm run dev -- -p 3000
|
||||
```
|
||||
|
||||
La plataforma queda en `http://localhost:3000`, la API en `http://localhost:8000` y la documentación interactiva en `http://localhost:8000/docs`.
|
||||
|
||||
Para usuarios no tecnicos en Windows, usa `abrir_estacion.bat`; detecta puertos libres, prepara backend/frontend y abre el navegador. Usa `cerrar_estacion.bat` para detener todo.
|
||||
|
||||
## Docker / Coolify
|
||||
|
||||
```bash
|
||||
docker compose up --build
|
||||
```
|
||||
|
||||
Para Coolify, configura backend y frontend como servicios separados. Monta un volumen persistente para `/app/data` en el backend y define `NEXT_PUBLIC_API_URL` en el frontend apuntando a la URL publica del backend.
|
||||
|
||||
## Variables
|
||||
|
||||
- `APP_DATA_DIR`: carpeta de datos persistentes.
|
||||
- `DATABASE_PATH`: ruta SQLite.
|
||||
- `MISTRAL_API_KEY`: OCR para PDFs escaneados.
|
||||
- `DEEPGRAM_API_KEY`: transcripción de audio/video.
|
||||
|
||||
No subas claves reales al repositorio.
|
||||
|
||||
## Flujo API Basico
|
||||
|
||||
Crear materia:
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8000/subjects -H "Content-Type: application/json" -d '{"name":"Comportamiento Organizacional"}'
|
||||
```
|
||||
|
||||
Crear semana:
|
||||
|
||||
```bash
|
||||
curl -X POST http://localhost:8000/subjects/1/weeks -H "Content-Type: application/json" -d '{"number":1,"title":"Grupos y equipos"}'
|
||||
```
|
||||
|
||||
Subir archivo:
|
||||
|
||||
```bash
|
||||
curl -X POST -F "[email protected]" http://localhost:8000/subjects/1/weeks/1/files
|
||||
```
|
||||
|
||||
Consultar trabajo:
|
||||
|
||||
```bash
|
||||
curl http://localhost:8000/jobs/1
|
||||
```
|
||||
|
||||
Exportar semana:
|
||||
|
||||
```bash
|
||||
curl -L http://localhost:8000/subjects/1/weeks/1/export -o semana-01.md
|
||||
```
|
||||
@@ -0,0 +1,18 @@
|
||||
@echo off
|
||||
setlocal
|
||||
title Knowledge Station - Abrir
|
||||
|
||||
set "ROOT=%~dp0"
|
||||
set "ROOT=%ROOT:~0,-1%"
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File "%ROOT%\scripts\start-platform.ps1" -Root "%ROOT%"
|
||||
if errorlevel 1 (
|
||||
echo.
|
||||
echo No se pudo abrir la plataforma. Revisa el mensaje anterior.
|
||||
pause
|
||||
exit /b 1
|
||||
)
|
||||
|
||||
echo.
|
||||
echo Puedes dejar esta ventana cerrada. Para apagar la plataforma usa cerrar_estacion.bat.
|
||||
pause
|
||||
exit /b 0
|
||||
@@ -0,0 +1,16 @@
|
||||
from pathlib import Path
|
||||
|
||||
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||
|
||||
|
||||
class Settings(BaseSettings):
|
||||
app_env: str = "development"
|
||||
app_data_dir: Path = Path("./data")
|
||||
database_path: Path = Path("./data/app.db")
|
||||
mistral_api_key: str = ""
|
||||
deepgram_api_key: str = ""
|
||||
|
||||
model_config = SettingsConfigDict(env_file=".env", env_file_encoding="utf-8", extra="ignore")
|
||||
|
||||
|
||||
settings = Settings()
|
||||
@@ -0,0 +1,281 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from fastapi import BackgroundTasks, FastAPI, File, HTTPException, Query, UploadFile
|
||||
from fastapi.middleware.cors import CORSMiddleware
|
||||
from fastapi.responses import FileResponse, PlainTextResponse
|
||||
|
||||
from backend.app.config import settings
|
||||
from backend.app.managers.knowledge_manager import DuplicateSourceError, KnowledgeManager
|
||||
from backend.app.models.schemas import MarkdownUpdate, SubjectCreate, WeekCreate, WeekUpdate
|
||||
from backend.app.services.ingestion_service import IngestionService
|
||||
|
||||
app = FastAPI(title="Knowledge Station", version="0.1.0")
|
||||
app.add_middleware(
|
||||
CORSMiddleware,
|
||||
allow_origins=["http://127.0.0.1:3000", "http://localhost:3000"],
|
||||
allow_origin_regex=r"http://(127\.0\.0\.1|localhost):\d+",
|
||||
allow_credentials=True,
|
||||
allow_methods=["*"],
|
||||
allow_headers=["*"],
|
||||
)
|
||||
manager = KnowledgeManager()
|
||||
ingestion = IngestionService(manager)
|
||||
|
||||
|
||||
@app.get("/health")
|
||||
def health() -> dict[str, str]:
|
||||
return {"status": "ok"}
|
||||
|
||||
|
||||
@app.get("/settings/status")
|
||||
def settings_status() -> dict:
|
||||
return {
|
||||
"mistral_configured": bool(settings.mistral_api_key),
|
||||
"deepgram_configured": bool(settings.deepgram_api_key),
|
||||
"data_dir": str(settings.app_data_dir),
|
||||
"database_path": str(settings.database_path),
|
||||
}
|
||||
|
||||
|
||||
@app.post("/subjects")
|
||||
def create_subject(payload: SubjectCreate) -> dict:
|
||||
try:
|
||||
return manager.create_subject(payload.name)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@app.get("/subjects")
|
||||
def list_subjects() -> list[dict]:
|
||||
return manager.list_subjects()
|
||||
|
||||
|
||||
@app.get("/subjects/{subject_id}")
|
||||
def get_subject(subject_id: int) -> dict:
|
||||
try:
|
||||
return manager.get_subject(subject_id)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@app.delete("/subjects/{subject_id}")
|
||||
def delete_subject(subject_id: int) -> dict:
|
||||
try:
|
||||
manager.delete_subject(subject_id)
|
||||
return {"status": "deleted"}
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@app.post("/subjects/{subject_id}/weeks")
|
||||
def create_week(subject_id: int, payload: WeekCreate) -> dict:
|
||||
try:
|
||||
return manager.create_week(subject_id, payload.number, payload.title)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@app.get("/subjects/{subject_id}/weeks")
|
||||
def list_weeks(subject_id: int) -> list[dict]:
|
||||
try:
|
||||
manager.get_subject(subject_id)
|
||||
return manager.list_weeks(subject_id)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@app.get("/subjects/{subject_id}/weeks/{week_number}")
|
||||
def get_week(subject_id: int, week_number: int) -> dict:
|
||||
try:
|
||||
return manager.get_week(subject_id, week_number)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@app.put("/subjects/{subject_id}/weeks/{week_number}")
|
||||
def update_week(subject_id: int, week_number: int, payload: WeekUpdate) -> dict:
|
||||
try:
|
||||
return manager.update_week(subject_id, week_number, payload.number, payload.title)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@app.delete("/subjects/{subject_id}/weeks/{week_number}")
|
||||
def delete_week(subject_id: int, week_number: int) -> dict:
|
||||
try:
|
||||
manager.delete_week(subject_id, week_number)
|
||||
return {"status": "deleted"}
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@app.post("/subjects/{subject_id}/weeks/{week_number}/files")
|
||||
def upload_file(
|
||||
subject_id: int,
|
||||
week_number: int,
|
||||
background_tasks: BackgroundTasks,
|
||||
file: UploadFile = File(...),
|
||||
use_ocr: bool = Query(False, description="Usar Mistral OCR para PDFs"),
|
||||
page_ranges: str | None = Query(None, description="Rangos de paginas PDF, por ejemplo: 1-5,8,10-12"),
|
||||
) -> dict:
|
||||
try:
|
||||
source = manager.save_upload(subject_id, week_number, file)
|
||||
job = manager.create_job(source["id"], "Archivo recibido")
|
||||
background_tasks.add_task(process_job, job["id"], source["id"], use_ocr, page_ranges)
|
||||
return {"source": source, "job": job}
|
||||
except DuplicateSourceError as exc:
|
||||
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@app.get("/jobs/{job_id}")
|
||||
def get_job(job_id: int) -> dict:
|
||||
try:
|
||||
return manager.get_job(job_id)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@app.post("/jobs/{job_id}/retry")
|
||||
def retry_job(
|
||||
job_id: int,
|
||||
background_tasks: BackgroundTasks,
|
||||
use_ocr: bool = Query(False),
|
||||
page_ranges: str | None = Query(None),
|
||||
) -> dict:
|
||||
try:
|
||||
job = manager.get_job(job_id)
|
||||
if not job.get("source_id"):
|
||||
raise HTTPException(status_code=400, detail="El trabajo no tiene fuente asociada")
|
||||
manager.update_job(job_id, "queued", "Reintento en cola")
|
||||
background_tasks.add_task(process_job, job_id, job["source_id"], use_ocr, page_ranges)
|
||||
return manager.get_job(job_id)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@app.get("/subjects/{subject_id}/weeks/{week_number}/jobs")
|
||||
def list_week_jobs(subject_id: int, week_number: int) -> list[dict]:
|
||||
try:
|
||||
return manager.list_jobs_for_week(subject_id, week_number)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@app.get("/subjects/{subject_id}/weeks/{week_number}/sources")
|
||||
def list_sources(subject_id: int, week_number: int) -> list[dict]:
|
||||
try:
|
||||
return manager.list_sources(subject_id, week_number)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@app.delete("/sources/{source_id}")
|
||||
def delete_source(source_id: int) -> dict:
|
||||
try:
|
||||
manager.delete_source(source_id)
|
||||
return {"status": "deleted"}
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@app.post("/sources/{source_id}/reprocess")
|
||||
def reprocess_source(
|
||||
source_id: int,
|
||||
background_tasks: BackgroundTasks,
|
||||
use_ocr: bool = Query(False),
|
||||
page_ranges: str | None = Query(None),
|
||||
) -> dict:
|
||||
try:
|
||||
source = manager.get_source(source_id)
|
||||
if manager.source_has_active_job(source_id):
|
||||
raise HTTPException(status_code=409, detail="La fuente ya tiene un trabajo en cola o en proceso")
|
||||
job = manager.create_job(source["id"], "Reprocesamiento en cola")
|
||||
background_tasks.add_task(process_job, job["id"], source["id"], use_ocr, page_ranges)
|
||||
return {"source": source, "job": job}
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@app.get("/subjects/{subject_id}/weeks/{week_number}/documents")
|
||||
def list_documents(subject_id: int, week_number: int) -> list[dict]:
|
||||
try:
|
||||
return manager.list_documents(subject_id, week_number)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@app.get("/documents/{document_id}")
|
||||
def get_document(document_id: int) -> dict:
|
||||
try:
|
||||
return manager.get_document(document_id)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@app.get("/documents/{document_id}/markdown", response_class=PlainTextResponse)
|
||||
def get_document_markdown(document_id: int) -> str:
|
||||
try:
|
||||
document = manager.get_document(document_id)
|
||||
return Path(document["markdown_path"]).read_text(encoding="utf-8")
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@app.put("/documents/{document_id}/markdown")
|
||||
def update_document_markdown(document_id: int, payload: MarkdownUpdate) -> dict:
|
||||
try:
|
||||
return manager.update_document_markdown(document_id, payload.content)
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@app.get("/documents/{document_id}/download")
|
||||
def download_document(document_id: int) -> FileResponse:
|
||||
try:
|
||||
document = manager.get_document(document_id)
|
||||
path = Path(document["markdown_path"])
|
||||
return FileResponse(path, filename=path.name, media_type="text/markdown")
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@app.delete("/documents/{document_id}")
|
||||
def delete_document(document_id: int) -> dict:
|
||||
try:
|
||||
manager.delete_document(document_id)
|
||||
return {"status": "deleted"}
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@app.get("/subjects/{subject_id}/weeks/{week_number}/export")
|
||||
def export_week(subject_id: int, week_number: int) -> FileResponse:
|
||||
try:
|
||||
path = manager.export_week_markdown(subject_id, week_number)
|
||||
return FileResponse(path, filename=path.name, media_type="text/markdown")
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||
|
||||
|
||||
@app.get("/subjects/{subject_id}/weeks/{week_number}/export.zip")
|
||||
def export_week_zip(subject_id: int, week_number: int) -> FileResponse:
|
||||
try:
|
||||
path = manager.export_week_markdown_zip(subject_id, week_number)
|
||||
return FileResponse(path, filename=path.name, media_type="application/zip")
|
||||
except ValueError as exc:
|
||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||
|
||||
|
||||
def process_job(job_id: int, source_id: int, use_ocr: bool, page_ranges: str | None = None) -> None:
|
||||
manager.update_job(job_id, "processing", "Procesando archivo")
|
||||
manager.update_source_status(source_id, "processing")
|
||||
try:
|
||||
document = ingestion.process_source(source_id, use_ocr=use_ocr, page_ranges=page_ranges)
|
||||
manager.update_job(job_id, "completed", "Documento procesado", document_id=document["id"])
|
||||
except Exception as exc:
|
||||
manager.update_source_status(source_id, "failed")
|
||||
manager.update_job(job_id, "failed", str(exc))
|
||||
@@ -0,0 +1,548 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import shutil
|
||||
import sqlite3
|
||||
import re
|
||||
import hashlib
|
||||
import zipfile
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any, Iterable
|
||||
|
||||
from fastapi import UploadFile
|
||||
|
||||
from backend.app.config import settings
|
||||
from backend.app.utils.slug import slugify
|
||||
|
||||
|
||||
def utc_now() -> str:
|
||||
return datetime.now(timezone.utc).isoformat()
|
||||
|
||||
|
||||
class DuplicateSourceError(ValueError):
|
||||
def __init__(self, existing_source: dict[str, Any]):
|
||||
self.existing_source = existing_source
|
||||
super().__init__(f"El archivo ya existe en esta semana: {existing_source['original_name']}")
|
||||
|
||||
|
||||
class KnowledgeManager:
|
||||
def __init__(self, data_dir: Path | None = None, database_path: Path | None = None):
|
||||
self.data_dir = data_dir or settings.app_data_dir
|
||||
self.database_path = database_path or settings.database_path
|
||||
self.subjects_dir = self.data_dir / "subjects"
|
||||
self.tmp_dir = self.data_dir / "tmp"
|
||||
self.data_dir.mkdir(parents=True, exist_ok=True)
|
||||
self.subjects_dir.mkdir(parents=True, exist_ok=True)
|
||||
self.tmp_dir.mkdir(parents=True, exist_ok=True)
|
||||
self.database_path.parent.mkdir(parents=True, exist_ok=True)
|
||||
self._init_db()
|
||||
|
||||
def _connect(self) -> sqlite3.Connection:
|
||||
conn = sqlite3.connect(self.database_path)
|
||||
conn.row_factory = sqlite3.Row
|
||||
return conn
|
||||
|
||||
def _init_db(self) -> None:
|
||||
with self._connect() as conn:
|
||||
conn.executescript(
|
||||
"""
|
||||
create table if not exists subjects (
|
||||
id integer primary key autoincrement,
|
||||
name text not null,
|
||||
slug text not null unique,
|
||||
created_at text not null
|
||||
);
|
||||
|
||||
create table if not exists weeks (
|
||||
id integer primary key autoincrement,
|
||||
subject_id integer not null,
|
||||
number integer not null,
|
||||
title text,
|
||||
created_at text not null,
|
||||
unique(subject_id, number),
|
||||
foreign key(subject_id) references subjects(id)
|
||||
);
|
||||
|
||||
create table if not exists sources (
|
||||
id integer primary key autoincrement,
|
||||
week_id integer not null,
|
||||
original_name text not null,
|
||||
stored_path text not null,
|
||||
source_type text not null,
|
||||
status text not null,
|
||||
file_size integer,
|
||||
sha256 text,
|
||||
mime_type text,
|
||||
processor text,
|
||||
page_ranges text,
|
||||
processed_at text,
|
||||
created_at text not null,
|
||||
foreign key(week_id) references weeks(id)
|
||||
);
|
||||
|
||||
create table if not exists documents (
|
||||
id integer primary key autoincrement,
|
||||
source_id integer not null,
|
||||
title text not null,
|
||||
markdown_path text not null,
|
||||
created_at text not null,
|
||||
foreign key(source_id) references sources(id)
|
||||
);
|
||||
|
||||
create table if not exists jobs (
|
||||
id integer primary key autoincrement,
|
||||
status text not null,
|
||||
message text not null,
|
||||
source_id integer,
|
||||
document_id integer,
|
||||
created_at text not null,
|
||||
updated_at text not null,
|
||||
foreign key(source_id) references sources(id),
|
||||
foreign key(document_id) references documents(id)
|
||||
);
|
||||
"""
|
||||
)
|
||||
self._ensure_columns(
|
||||
conn,
|
||||
"sources",
|
||||
{
|
||||
"file_size": "integer",
|
||||
"sha256": "text",
|
||||
"mime_type": "text",
|
||||
"processor": "text",
|
||||
"page_ranges": "text",
|
||||
"processed_at": "text",
|
||||
},
|
||||
)
|
||||
self._backfill_source_file_metadata(conn)
|
||||
|
||||
def _ensure_columns(self, conn: sqlite3.Connection, table: str, columns: dict[str, str]) -> None:
|
||||
existing = {row["name"] for row in conn.execute(f"pragma table_info({table})").fetchall()}
|
||||
for name, definition in columns.items():
|
||||
if name not in existing:
|
||||
conn.execute(f"alter table {table} add column {name} {definition}")
|
||||
|
||||
def _backfill_source_file_metadata(self, conn: sqlite3.Connection) -> None:
|
||||
rows = conn.execute("select id, stored_path from sources where sha256 is null or file_size is null").fetchall()
|
||||
for row in rows:
|
||||
path = Path(row["stored_path"])
|
||||
if not path.exists():
|
||||
continue
|
||||
sha256, file_size = self._file_hash_and_size(path)
|
||||
conn.execute("update sources set sha256 = ?, file_size = ? where id = ?", (sha256, file_size, row["id"]))
|
||||
|
||||
def create_subject(self, name: str) -> dict[str, Any]:
|
||||
base_slug = slugify(name)
|
||||
slug = base_slug
|
||||
suffix = 2
|
||||
with self._connect() as conn:
|
||||
while conn.execute("select 1 from subjects where slug = ?", (slug,)).fetchone():
|
||||
slug = f"{base_slug}-{suffix}"
|
||||
suffix += 1
|
||||
now = utc_now()
|
||||
cursor = conn.execute(
|
||||
"insert into subjects (name, slug, created_at) values (?, ?, ?)",
|
||||
(name, slug, now),
|
||||
)
|
||||
(self.subjects_dir / slug).mkdir(parents=True, exist_ok=True)
|
||||
return self.get_subject(cursor.lastrowid)
|
||||
|
||||
def get_subject(self, subject_id: int) -> dict[str, Any]:
|
||||
with self._connect() as conn:
|
||||
row = conn.execute("select * from subjects where id = ?", (subject_id,)).fetchone()
|
||||
if not row:
|
||||
raise ValueError("Materia no encontrada")
|
||||
return dict(row)
|
||||
|
||||
def delete_subject(self, subject_id: int) -> None:
|
||||
subject = self.get_subject(subject_id)
|
||||
with self._connect() as conn:
|
||||
week_rows = conn.execute("select id from weeks where subject_id = ?", (subject_id,)).fetchall()
|
||||
week_ids = [row["id"] for row in week_rows]
|
||||
for week_id in week_ids:
|
||||
source_rows = conn.execute("select id from sources where week_id = ?", (week_id,)).fetchall()
|
||||
source_ids = [row["id"] for row in source_rows]
|
||||
for source_id in source_ids:
|
||||
conn.execute("delete from documents where source_id = ?", (source_id,))
|
||||
conn.execute("delete from jobs where source_id = ?", (source_id,))
|
||||
conn.execute("delete from sources where week_id = ?", (week_id,))
|
||||
conn.execute("delete from weeks where subject_id = ?", (subject_id,))
|
||||
conn.execute("delete from subjects where id = ?", (subject_id,))
|
||||
shutil.rmtree(self.subjects_dir / subject["slug"], ignore_errors=True)
|
||||
|
||||
def list_subjects(self) -> list[dict[str, Any]]:
|
||||
with self._connect() as conn:
|
||||
rows = conn.execute("select * from subjects order by created_at desc").fetchall()
|
||||
return [dict(row) for row in rows]
|
||||
|
||||
def create_week(self, subject_id: int, number: int, title: str | None = None) -> dict[str, Any]:
|
||||
subject = self.get_subject(subject_id)
|
||||
now = utc_now()
|
||||
with self._connect() as conn:
|
||||
conn.execute(
|
||||
"insert or ignore into weeks (subject_id, number, title, created_at) values (?, ?, ?, ?)",
|
||||
(subject_id, number, title, now),
|
||||
)
|
||||
if title:
|
||||
conn.execute("update weeks set title = ? where subject_id = ? and number = ?", (title, subject_id, number))
|
||||
row = conn.execute(
|
||||
"select * from weeks where subject_id = ? and number = ?",
|
||||
(subject_id, number),
|
||||
).fetchone()
|
||||
self.week_dir(subject["slug"], number).mkdir(parents=True, exist_ok=True)
|
||||
for child in ("raw", "processed", "exports"):
|
||||
(self.week_dir(subject["slug"], number) / child).mkdir(parents=True, exist_ok=True)
|
||||
return dict(row)
|
||||
|
||||
def list_weeks(self, subject_id: int) -> list[dict[str, Any]]:
|
||||
with self._connect() as conn:
|
||||
rows = conn.execute("select * from weeks where subject_id = ? order by number", (subject_id,)).fetchall()
|
||||
return [dict(row) for row in rows]
|
||||
|
||||
def delete_week(self, subject_id: int, number: int) -> None:
|
||||
subject = self.get_subject(subject_id)
|
||||
week = self.get_week(subject_id, number)
|
||||
with self._connect() as conn:
|
||||
source_rows = conn.execute("select id from sources where week_id = ?", (week["id"],)).fetchall()
|
||||
for row in source_rows:
|
||||
conn.execute("delete from documents where source_id = ?", (row["id"],))
|
||||
conn.execute("delete from jobs where source_id = ?", (row["id"],))
|
||||
conn.execute("delete from sources where week_id = ?", (week["id"],))
|
||||
conn.execute("delete from weeks where id = ?", (week["id"],))
|
||||
shutil.rmtree(self.week_dir(subject["slug"], number), ignore_errors=True)
|
||||
|
||||
def get_week(self, subject_id: int, number: int) -> dict[str, Any]:
|
||||
with self._connect() as conn:
|
||||
row = conn.execute(
|
||||
"select * from weeks where subject_id = ? and number = ?",
|
||||
(subject_id, number),
|
||||
).fetchone()
|
||||
if not row:
|
||||
raise ValueError("Semana no encontrada")
|
||||
return dict(row)
|
||||
|
||||
def update_week(self, subject_id: int, current_number: int, new_number: int, title: str | None = None) -> dict[str, Any]:
|
||||
subject = self.get_subject(subject_id)
|
||||
week = self.get_week(subject_id, current_number)
|
||||
if new_number != current_number:
|
||||
with self._connect() as conn:
|
||||
existing = conn.execute(
|
||||
"select 1 from weeks where subject_id = ? and number = ?",
|
||||
(subject_id, new_number),
|
||||
).fetchone()
|
||||
if existing:
|
||||
raise ValueError("Ya existe una semana con ese numero")
|
||||
|
||||
old_dir = self.week_dir(subject["slug"], current_number)
|
||||
new_dir = self.week_dir(subject["slug"], new_number)
|
||||
if old_dir.exists() and new_dir.exists():
|
||||
raise ValueError("Ya existe una carpeta para ese numero de semana")
|
||||
if old_dir.exists():
|
||||
old_dir.rename(new_dir)
|
||||
else:
|
||||
for child in ("raw", "processed", "exports"):
|
||||
(new_dir / child).mkdir(parents=True, exist_ok=True)
|
||||
|
||||
with self._connect() as conn:
|
||||
conn.execute(
|
||||
"update weeks set number = ?, title = ? where id = ?",
|
||||
(new_number, title, week["id"]),
|
||||
)
|
||||
return self.get_week(subject_id, new_number)
|
||||
|
||||
def week_dir(self, subject_slug: str, week_number: int) -> Path:
|
||||
return self.subjects_dir / subject_slug / f"week-{week_number:02d}"
|
||||
|
||||
def save_upload(self, subject_id: int, week_number: int, upload: UploadFile) -> dict[str, Any]:
|
||||
subject = self.get_subject(subject_id)
|
||||
week = self.get_week(subject_id, week_number)
|
||||
filename = Path(upload.filename or "archivo").name
|
||||
raw_dir = self.week_dir(subject["slug"], week_number) / "raw"
|
||||
raw_dir.mkdir(parents=True, exist_ok=True)
|
||||
stored_path = self._unique_path(raw_dir / filename)
|
||||
digest = hashlib.sha256()
|
||||
file_size = 0
|
||||
with stored_path.open("wb") as out_file:
|
||||
while chunk := upload.file.read(1024 * 1024):
|
||||
file_size += len(chunk)
|
||||
digest.update(chunk)
|
||||
out_file.write(chunk)
|
||||
sha256 = digest.hexdigest()
|
||||
source_type = self.detect_source_type(stored_path)
|
||||
now = utc_now()
|
||||
with self._connect() as conn:
|
||||
duplicate = conn.execute(
|
||||
"select * from sources where week_id = ? and sha256 = ? limit 1",
|
||||
(week["id"], sha256),
|
||||
).fetchone()
|
||||
if duplicate:
|
||||
stored_path.unlink(missing_ok=True)
|
||||
raise DuplicateSourceError(dict(duplicate))
|
||||
cursor = conn.execute(
|
||||
"""
|
||||
insert into sources (week_id, original_name, stored_path, source_type, status, file_size, sha256, mime_type, created_at)
|
||||
values (?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||
""",
|
||||
(week["id"], filename, str(stored_path), source_type, "stored", file_size, sha256, upload.content_type, now),
|
||||
)
|
||||
return self.get_source(cursor.lastrowid)
|
||||
|
||||
def _file_hash_and_size(self, path: Path) -> tuple[str, int]:
|
||||
digest = hashlib.sha256()
|
||||
file_size = 0
|
||||
with path.open("rb") as file:
|
||||
while chunk := file.read(1024 * 1024):
|
||||
file_size += len(chunk)
|
||||
digest.update(chunk)
|
||||
return digest.hexdigest(), file_size
|
||||
|
||||
def create_job(self, source_id: int, message: str = "En cola") -> dict[str, Any]:
|
||||
now = utc_now()
|
||||
with self._connect() as conn:
|
||||
cursor = conn.execute(
|
||||
"insert into jobs (status, message, source_id, created_at, updated_at) values (?, ?, ?, ?, ?)",
|
||||
("queued", message, source_id, now, now),
|
||||
)
|
||||
return self.get_job(cursor.lastrowid)
|
||||
|
||||
def update_job(self, job_id: int, status: str, message: str, document_id: int | None = None) -> None:
|
||||
with self._connect() as conn:
|
||||
conn.execute(
|
||||
"update jobs set status = ?, message = ?, document_id = coalesce(?, document_id), updated_at = ? where id = ?",
|
||||
(status, message, document_id, utc_now(), job_id),
|
||||
)
|
||||
|
||||
def update_source_status(self, source_id: int, status: str) -> None:
|
||||
with self._connect() as conn:
|
||||
conn.execute("update sources set status = ? where id = ?", (status, source_id))
|
||||
|
||||
def source_has_active_job(self, source_id: int) -> bool:
|
||||
with self._connect() as conn:
|
||||
row = conn.execute(
|
||||
"select 1 from jobs where source_id = ? and status in ('queued', 'processing') limit 1",
|
||||
(source_id,),
|
||||
).fetchone()
|
||||
return bool(row)
|
||||
|
||||
def get_job(self, job_id: int) -> dict[str, Any]:
|
||||
with self._connect() as conn:
|
||||
row = conn.execute("select * from jobs where id = ?", (job_id,)).fetchone()
|
||||
if not row:
|
||||
raise ValueError("Trabajo no encontrado")
|
||||
return dict(row)
|
||||
|
||||
def list_jobs_for_week(self, subject_id: int, week_number: int) -> list[dict[str, Any]]:
|
||||
week = self.get_week(subject_id, week_number)
|
||||
with self._connect() as conn:
|
||||
rows = conn.execute(
|
||||
"""
|
||||
select jobs.* from jobs
|
||||
join sources on sources.id = jobs.source_id
|
||||
where sources.week_id = ?
|
||||
order by jobs.created_at desc
|
||||
""",
|
||||
(week["id"],),
|
||||
).fetchall()
|
||||
return [dict(row) for row in rows]
|
||||
|
||||
def get_source(self, source_id: int) -> dict[str, Any]:
|
||||
with self._connect() as conn:
|
||||
row = conn.execute("select * from sources where id = ?", (source_id,)).fetchone()
|
||||
if not row:
|
||||
raise ValueError("Fuente no encontrada")
|
||||
return dict(row)
|
||||
|
||||
def list_sources(self, subject_id: int, week_number: int) -> list[dict[str, Any]]:
|
||||
week = self.get_week(subject_id, week_number)
|
||||
with self._connect() as conn:
|
||||
rows = conn.execute(
|
||||
"""
|
||||
select sources.*, documents.id as document_id, documents.title as document_title,
|
||||
documents.markdown_path as document_markdown_path, documents.created_at as document_created_at
|
||||
from sources
|
||||
left join documents on documents.source_id = sources.id
|
||||
where sources.week_id = ?
|
||||
order by sources.created_at desc
|
||||
""",
|
||||
(week["id"],),
|
||||
).fetchall()
|
||||
return [dict(row) for row in rows]
|
||||
|
||||
def delete_source(self, source_id: int) -> None:
|
||||
source = self.get_source(source_id)
|
||||
with self._connect() as conn:
|
||||
doc_rows = conn.execute("select markdown_path from documents where source_id = ?", (source_id,)).fetchall()
|
||||
conn.execute("delete from documents where source_id = ?", (source_id,))
|
||||
conn.execute("delete from jobs where source_id = ?", (source_id,))
|
||||
conn.execute("delete from sources where id = ?", (source_id,))
|
||||
Path(source["stored_path"]).unlink(missing_ok=True)
|
||||
for row in doc_rows:
|
||||
Path(row["markdown_path"]).unlink(missing_ok=True)
|
||||
|
||||
def create_document(self, source_id: int, title: str, markdown: str, processor: str | None = None, page_ranges: str | None = None) -> dict[str, Any]:
|
||||
source = self.get_source(source_id)
|
||||
week_context = self.get_week_context(source["week_id"])
|
||||
processed_dir = self.week_dir(week_context["subject_slug"], week_context["week_number"]) / "processed"
|
||||
processed_dir.mkdir(parents=True, exist_ok=True)
|
||||
markdown_path = self._unique_path(processed_dir / f"{slugify(title)}.md")
|
||||
markdown_path.write_text(self._sanitize_markdown(markdown), encoding="utf-8")
|
||||
now = utc_now()
|
||||
with self._connect() as conn:
|
||||
old_docs = conn.execute("select markdown_path from documents where source_id = ?", (source_id,)).fetchall()
|
||||
conn.execute("delete from jobs where document_id in (select id from documents where source_id = ?)", (source_id,))
|
||||
conn.execute("delete from documents where source_id = ?", (source_id,))
|
||||
cursor = conn.execute(
|
||||
"insert into documents (source_id, title, markdown_path, created_at) values (?, ?, ?, ?)",
|
||||
(source_id, title, str(markdown_path), now),
|
||||
)
|
||||
conn.execute(
|
||||
"update sources set status = ?, processor = ?, page_ranges = ?, processed_at = ? where id = ?",
|
||||
("processed", processor, page_ranges, now, source_id),
|
||||
)
|
||||
for row in old_docs:
|
||||
Path(row["markdown_path"]).unlink(missing_ok=True)
|
||||
return self.get_document(cursor.lastrowid)
|
||||
|
||||
def get_document(self, document_id: int) -> dict[str, Any]:
|
||||
with self._connect() as conn:
|
||||
row = conn.execute("select * from documents where id = ?", (document_id,)).fetchone()
|
||||
if not row:
|
||||
raise ValueError("Documento no encontrado")
|
||||
return dict(row)
|
||||
|
||||
def update_document_markdown(self, document_id: int, content: str) -> dict[str, Any]:
|
||||
document = self.get_document(document_id)
|
||||
Path(document["markdown_path"]).write_text(self._sanitize_markdown(content), encoding="utf-8")
|
||||
return self.get_document(document_id)
|
||||
|
||||
def delete_document(self, document_id: int) -> None:
|
||||
document = self.get_document(document_id)
|
||||
Path(document["markdown_path"]).unlink(missing_ok=True)
|
||||
with self._connect() as conn:
|
||||
conn.execute("delete from jobs where document_id = ?", (document_id,))
|
||||
conn.execute("delete from documents where id = ?", (document_id,))
|
||||
|
||||
def list_documents(self, subject_id: int, week_number: int) -> list[dict[str, Any]]:
|
||||
week = self.get_week(subject_id, week_number)
|
||||
with self._connect() as conn:
|
||||
rows = conn.execute(
|
||||
"""
|
||||
select documents.*, sources.original_name as source_original_name,
|
||||
sources.source_type as source_type, sources.status as source_status,
|
||||
sources.processor as source_processor, sources.page_ranges as source_page_ranges
|
||||
from documents
|
||||
join sources on sources.id = documents.source_id
|
||||
where sources.week_id = ?
|
||||
order by documents.created_at desc
|
||||
""",
|
||||
(week["id"],),
|
||||
).fetchall()
|
||||
return [dict(row) for row in rows]
|
||||
|
||||
def get_week_context(self, week_id: int) -> dict[str, Any]:
|
||||
with self._connect() as conn:
|
||||
row = conn.execute(
|
||||
"""
|
||||
select weeks.id as week_id, weeks.number as week_number, weeks.title as week_title,
|
||||
subjects.id as subject_id, subjects.name as subject_name, subjects.slug as subject_slug
|
||||
from weeks join subjects on subjects.id = weeks.subject_id
|
||||
where weeks.id = ?
|
||||
""",
|
||||
(week_id,),
|
||||
).fetchone()
|
||||
if not row:
|
||||
raise ValueError("Contexto de semana no encontrado")
|
||||
return dict(row)
|
||||
|
||||
def export_week_markdown(self, subject_id: int, week_number: int) -> Path:
|
||||
subject = self.get_subject(subject_id)
|
||||
docs = self.list_documents(subject_id, week_number)
|
||||
export_dir = self.week_dir(subject["slug"], week_number) / "exports"
|
||||
export_dir.mkdir(parents=True, exist_ok=True)
|
||||
export_path = export_dir / f"{subject['slug']}-week-{week_number:02d}.md"
|
||||
parts = [
|
||||
"---\n"
|
||||
f"subject: {subject['name']}\n"
|
||||
f"subject_slug: {subject['slug']}\n"
|
||||
f"week: {week_number}\n"
|
||||
f"documents: {len(docs)}\n"
|
||||
f"exported_at: {utc_now()}\n"
|
||||
"---\n\n"
|
||||
f"# {subject['name']} - Semana {week_number}\n"
|
||||
]
|
||||
for doc in docs:
|
||||
path = Path(doc["markdown_path"])
|
||||
content = self._strip_frontmatter(self._read_markdown(path)).strip()
|
||||
parts.append(f"<!-- document_id: {doc['id']} -->\n\n{content}\n")
|
||||
export_path.write_text(self._sanitize_markdown("\n\n---\n\n".join(parts)), encoding="utf-8")
|
||||
return export_path
|
||||
|
||||
def export_week_markdown_zip(self, subject_id: int, week_number: int) -> Path:
|
||||
subject = self.get_subject(subject_id)
|
||||
docs = self.list_documents(subject_id, week_number)
|
||||
export_dir = self.week_dir(subject["slug"], week_number) / "exports"
|
||||
export_dir.mkdir(parents=True, exist_ok=True)
|
||||
export_path = export_dir / f"{subject['slug']}-week-{week_number:02d}-documentos.zip"
|
||||
used_names: set[str] = set()
|
||||
with zipfile.ZipFile(export_path, "w", compression=zipfile.ZIP_DEFLATED) as archive:
|
||||
for index, doc in enumerate(docs, start=1):
|
||||
path = Path(doc["markdown_path"])
|
||||
name = self._unique_archive_name(f"{index:02d}-{slugify(doc['title'])}.md", used_names)
|
||||
archive.writestr(name, self._read_markdown(path))
|
||||
return export_path
|
||||
|
||||
def _unique_archive_name(self, filename: str, used_names: set[str]) -> str:
|
||||
path = Path(filename)
|
||||
candidate = path.name
|
||||
suffix = path.suffix
|
||||
stem = path.stem
|
||||
counter = 2
|
||||
while candidate in used_names:
|
||||
candidate = f"{stem}-{counter}{suffix}"
|
||||
counter += 1
|
||||
used_names.add(candidate)
|
||||
return candidate
|
||||
|
||||
def _read_markdown(self, path: Path) -> str:
|
||||
return self._sanitize_markdown(path.read_text(encoding="utf-8", errors="replace"))
|
||||
|
||||
def _sanitize_markdown(self, content: str) -> str:
|
||||
return re.sub(r"[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]", "", content)
|
||||
|
||||
def _strip_frontmatter(self, content: str) -> str:
|
||||
lines = content.splitlines()
|
||||
if not lines or lines[0].strip() != "---":
|
||||
return content
|
||||
for index, line in enumerate(lines[1:], start=1):
|
||||
if line.strip() == "---":
|
||||
return "\n".join(lines[index + 1 :]).lstrip()
|
||||
return content
|
||||
|
||||
def detect_source_type(self, path: Path) -> str:
|
||||
ext = path.suffix.lower().lstrip(".")
|
||||
if ext in {"txt", "md"}:
|
||||
return "text"
|
||||
if ext == "docx":
|
||||
return "docx"
|
||||
if ext == "pdf":
|
||||
return "pdf"
|
||||
if ext in {"mp3", "wav", "m4a", "ogg", "flac", "webm"}:
|
||||
return "audio"
|
||||
if ext in {"mp4", "mov", "mkv", "avi"}:
|
||||
return "video"
|
||||
return "unknown"
|
||||
|
||||
def _unique_path(self, path: Path) -> Path:
|
||||
if not path.exists():
|
||||
return path
|
||||
stem = path.stem
|
||||
suffix = path.suffix
|
||||
for index in range(2, 10_000):
|
||||
candidate = path.with_name(f"{stem}-{index}{suffix}")
|
||||
if not candidate.exists():
|
||||
return candidate
|
||||
raise RuntimeError("No se pudo generar un nombre de archivo unico")
|
||||
|
||||
|
||||
def rows_to_models(rows: Iterable[dict[str, Any]]) -> list[dict[str, Any]]:
|
||||
return [dict(row) for row in rows]
|
||||
@@ -0,0 +1,55 @@
|
||||
from datetime import datetime
|
||||
from typing import Optional
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
|
||||
class SubjectCreate(BaseModel):
|
||||
name: str = Field(min_length=1)
|
||||
|
||||
|
||||
class Subject(BaseModel):
|
||||
id: int
|
||||
name: str
|
||||
slug: str
|
||||
created_at: datetime
|
||||
|
||||
|
||||
class WeekCreate(BaseModel):
|
||||
number: int = Field(ge=1)
|
||||
title: Optional[str] = None
|
||||
|
||||
|
||||
class WeekUpdate(BaseModel):
|
||||
number: int = Field(ge=1)
|
||||
title: Optional[str] = None
|
||||
|
||||
|
||||
class Week(BaseModel):
|
||||
id: int
|
||||
subject_id: int
|
||||
number: int
|
||||
title: Optional[str]
|
||||
created_at: datetime
|
||||
|
||||
|
||||
class MarkdownUpdate(BaseModel):
|
||||
content: str
|
||||
|
||||
|
||||
class Job(BaseModel):
|
||||
id: int
|
||||
status: str
|
||||
message: str
|
||||
source_id: Optional[int] = None
|
||||
document_id: Optional[int] = None
|
||||
created_at: datetime
|
||||
updated_at: datetime
|
||||
|
||||
|
||||
class Document(BaseModel):
|
||||
id: int
|
||||
source_id: int
|
||||
title: str
|
||||
markdown_path: str
|
||||
created_at: datetime
|
||||
@@ -0,0 +1,51 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from backend.app.managers.knowledge_manager import KnowledgeManager
|
||||
from backend.app.services.markdown_builder import build_markdown
|
||||
from backend.app.services.processors.audio_processor import AudioProcessor
|
||||
from backend.app.services.processors.docx_processor import DocxProcessor
|
||||
from backend.app.services.processors.pdf_processor import PdfProcessor
|
||||
from backend.app.services.processors.text_processor import TextProcessor
|
||||
from backend.app.services.processors.video_processor import VideoProcessor
|
||||
|
||||
|
||||
class IngestionService:
|
||||
def __init__(self, manager: KnowledgeManager):
|
||||
self.manager = manager
|
||||
|
||||
def process_source(self, source_id: int, use_ocr: bool = False, page_ranges: str | None = None) -> dict:
|
||||
source = self.manager.get_source(source_id)
|
||||
context = self.manager.get_week_context(source["week_id"])
|
||||
path = Path(source["stored_path"])
|
||||
processor = self._processor_for(source["source_type"], use_ocr, page_ranges)
|
||||
processed = processor.process(path)
|
||||
markdown = build_markdown(
|
||||
title=processed.title,
|
||||
body=processed.body,
|
||||
metadata={
|
||||
"subject": context["subject_name"],
|
||||
"subject_slug": context["subject_slug"],
|
||||
"week": context["week_number"],
|
||||
"source_file": source["original_name"],
|
||||
"source_type": source["source_type"],
|
||||
"processor": processed.processor,
|
||||
"page_ranges": page_ranges if source["source_type"] == "pdf" and page_ranges else None,
|
||||
"language": "es",
|
||||
},
|
||||
)
|
||||
return self.manager.create_document(source_id, processed.title, markdown, processor=processed.processor, page_ranges=page_ranges)
|
||||
|
||||
def _processor_for(self, source_type: str, use_ocr: bool, page_ranges: str | None):
|
||||
if source_type == "text":
|
||||
return TextProcessor()
|
||||
if source_type == "docx":
|
||||
return DocxProcessor()
|
||||
if source_type == "pdf":
|
||||
return PdfProcessor(use_ocr=use_ocr, page_ranges=page_ranges)
|
||||
if source_type == "audio":
|
||||
return AudioProcessor()
|
||||
if source_type == "video":
|
||||
return VideoProcessor(self.manager.tmp_dir)
|
||||
raise RuntimeError(f"Tipo de fuente no soportado: {source_type}")
|
||||
@@ -0,0 +1,23 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from datetime import datetime, timezone
|
||||
from pathlib import Path
|
||||
from typing import Any
|
||||
|
||||
|
||||
def build_markdown(title: str, body: str, metadata: dict[str, Any]) -> str:
|
||||
frontmatter = {
|
||||
**metadata,
|
||||
"processed_at": datetime.now(timezone.utc).isoformat(),
|
||||
}
|
||||
lines = ["---"]
|
||||
for key, value in frontmatter.items():
|
||||
if value is not None:
|
||||
safe_value = str(value).replace("\n", " ")
|
||||
lines.append(f"{key}: {safe_value}")
|
||||
lines.extend(["---", "", f"# {title}", "", body.strip(), ""])
|
||||
return "\n".join(lines)
|
||||
|
||||
|
||||
def title_from_path(path: Path) -> str:
|
||||
return path.stem.replace("-", " ").replace("_", " ").strip().title() or "Documento"
|
||||
@@ -0,0 +1,20 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from backend.app.services.markdown_builder import title_from_path
|
||||
from backend.app.services.processors.base import ProcessedContent
|
||||
from backend.app.services.providers.deepgram_client import DeepgramClient
|
||||
|
||||
|
||||
class AudioProcessor:
|
||||
def process(self, path: Path) -> ProcessedContent:
|
||||
client = DeepgramClient()
|
||||
data = client.transcribe_file(path)
|
||||
transcript, confidence = client.transcript_from_response(data)
|
||||
confidence_line = f"\n\n_Confianza estimada: {confidence * 100:.1f}%_" if isinstance(confidence, (int, float)) else ""
|
||||
return ProcessedContent(
|
||||
title=f"Transcripcion {title_from_path(path)}",
|
||||
body=f"## Transcripcion\n\n{transcript}{confidence_line}",
|
||||
processor="deepgram",
|
||||
)
|
||||
@@ -0,0 +1,10 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from dataclasses import dataclass
|
||||
|
||||
|
||||
@dataclass
|
||||
class ProcessedContent:
|
||||
title: str
|
||||
body: str
|
||||
processor: str
|
||||
@@ -0,0 +1,30 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from docx import Document
|
||||
|
||||
from backend.app.services.markdown_builder import title_from_path
|
||||
from backend.app.services.processors.base import ProcessedContent
|
||||
|
||||
|
||||
class DocxProcessor:
|
||||
def process(self, path: Path) -> ProcessedContent:
|
||||
doc = Document(path)
|
||||
blocks: list[str] = []
|
||||
for paragraph in doc.paragraphs:
|
||||
text = paragraph.text.strip()
|
||||
if text:
|
||||
blocks.append(text)
|
||||
for table in doc.tables:
|
||||
rows = [[cell.text.strip().replace("\n", " ") for cell in row.cells] for row in table.rows]
|
||||
if rows:
|
||||
blocks.append(self._table_to_markdown(rows))
|
||||
return ProcessedContent(title=title_from_path(path), body="\n\n".join(blocks), processor="docx")
|
||||
|
||||
def _table_to_markdown(self, rows: list[list[str]]) -> str:
|
||||
header = rows[0]
|
||||
lines = [f"| {' | '.join(header)} |", f"| {' | '.join(['---'] * len(header))} |"]
|
||||
for row in rows[1:]:
|
||||
lines.append(f"| {' | '.join(row)} |")
|
||||
return "\n".join(lines)
|
||||
@@ -0,0 +1,81 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import fitz
|
||||
|
||||
from backend.app.services.markdown_builder import title_from_path
|
||||
from backend.app.services.processors.base import ProcessedContent
|
||||
from backend.app.services.providers.mistral_client import MistralClient
|
||||
|
||||
|
||||
class PdfProcessor:
|
||||
def __init__(self, use_ocr: bool = False, page_ranges: str | None = None):
|
||||
self.use_ocr = use_ocr
|
||||
self.page_ranges = page_ranges
|
||||
|
||||
def process(self, path: Path) -> ProcessedContent:
|
||||
if self.use_ocr:
|
||||
return self._process_ocr(path)
|
||||
doc = fitz.open(path)
|
||||
pages: list[str] = []
|
||||
for index in self._selected_pages(len(doc)):
|
||||
page = doc[index - 1]
|
||||
text = page.get_text("text").strip()
|
||||
if text:
|
||||
pages.append(f"## Pagina {index}\n\n{text}")
|
||||
body = "\n\n".join(pages).strip()
|
||||
if len(body) < 80:
|
||||
raise RuntimeError("El PDF tiene poco texto extraible; procesalo con OCR")
|
||||
return ProcessedContent(title=title_from_path(path), body=body, processor="pdf-text")
|
||||
|
||||
def _process_ocr(self, path: Path) -> ProcessedContent:
|
||||
doc = fitz.open(path)
|
||||
client = MistralClient()
|
||||
parts: list[str] = []
|
||||
temp_paths: list[Path] = []
|
||||
try:
|
||||
for index in self._selected_pages(len(doc)):
|
||||
page = doc[index - 1]
|
||||
pix = page.get_pixmap(matrix=fitz.Matrix(2, 2), alpha=False)
|
||||
image_path = path.with_name(f"{path.stem}-page-{index}.jpg")
|
||||
pix.save(image_path)
|
||||
temp_paths.append(image_path)
|
||||
parts.append(f"## Pagina {index}\n\n{client.ocr_image(image_path)}")
|
||||
finally:
|
||||
for temp_path in temp_paths:
|
||||
temp_path.unlink(missing_ok=True)
|
||||
return ProcessedContent(title=title_from_path(path), body="\n\n".join(parts), processor="mistral-ocr")
|
||||
|
||||
def _selected_pages(self, total_pages: int) -> list[int]:
|
||||
if total_pages < 1:
|
||||
raise RuntimeError("El PDF no tiene paginas")
|
||||
if not self.page_ranges or not self.page_ranges.strip():
|
||||
return list(range(1, total_pages + 1))
|
||||
|
||||
selected: list[int] = []
|
||||
for raw_part in self.page_ranges.split(","):
|
||||
part = raw_part.strip()
|
||||
if not part:
|
||||
continue
|
||||
if "-" in part:
|
||||
start_text, end_text = [piece.strip() for piece in part.split("-", 1)]
|
||||
if not start_text.isdigit() or not end_text.isdigit():
|
||||
raise RuntimeError(f"Rango de paginas invalido: {part}")
|
||||
start = int(start_text)
|
||||
end = int(end_text)
|
||||
if start > end:
|
||||
raise RuntimeError(f"Rango de paginas invertido: {part}")
|
||||
selected.extend(range(start, end + 1))
|
||||
else:
|
||||
if not part.isdigit():
|
||||
raise RuntimeError(f"Pagina invalida: {part}")
|
||||
selected.append(int(part))
|
||||
|
||||
unique_pages = sorted(set(selected))
|
||||
invalid = [page for page in unique_pages if page < 1 or page > total_pages]
|
||||
if invalid:
|
||||
raise RuntimeError(f"Paginas fuera de rango: {', '.join(map(str, invalid))}. El PDF tiene {total_pages} paginas")
|
||||
if not unique_pages:
|
||||
raise RuntimeError("No se seleccionaron paginas para procesar")
|
||||
return unique_pages
|
||||
@@ -0,0 +1,15 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
from backend.app.services.markdown_builder import title_from_path
|
||||
from backend.app.services.processors.base import ProcessedContent
|
||||
|
||||
|
||||
class TextProcessor:
|
||||
def process(self, path: Path) -> ProcessedContent:
|
||||
return ProcessedContent(
|
||||
title=title_from_path(path),
|
||||
body=path.read_text(encoding="utf-8", errors="replace"),
|
||||
processor="text",
|
||||
)
|
||||
@@ -0,0 +1,46 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import subprocess
|
||||
from pathlib import Path
|
||||
|
||||
from backend.app.services.processors.audio_processor import AudioProcessor
|
||||
from backend.app.services.processors.base import ProcessedContent
|
||||
|
||||
|
||||
class VideoProcessor:
|
||||
def __init__(self, tmp_dir: Path):
|
||||
self.tmp_dir = tmp_dir
|
||||
|
||||
def process(self, path: Path) -> ProcessedContent:
|
||||
self.tmp_dir.mkdir(parents=True, exist_ok=True)
|
||||
audio_path = self.tmp_dir / f"{path.stem}.wav"
|
||||
try:
|
||||
subprocess.run(
|
||||
[
|
||||
"ffmpeg",
|
||||
"-y",
|
||||
"-i",
|
||||
str(path),
|
||||
"-vn",
|
||||
"-acodec",
|
||||
"pcm_s16le",
|
||||
"-ar",
|
||||
"16000",
|
||||
"-ac",
|
||||
"1",
|
||||
str(audio_path),
|
||||
],
|
||||
check=True,
|
||||
capture_output=True,
|
||||
text=True,
|
||||
)
|
||||
processed = AudioProcessor().process(audio_path)
|
||||
processed.title = f"Transcripcion {path.stem}"
|
||||
processed.processor = "ffmpeg+deepgram"
|
||||
return processed
|
||||
except FileNotFoundError as exc:
|
||||
raise RuntimeError("ffmpeg no esta instalado o no esta disponible en PATH") from exc
|
||||
except subprocess.CalledProcessError as exc:
|
||||
raise RuntimeError(f"No se pudo extraer audio del video: {exc.stderr}") from exc
|
||||
finally:
|
||||
audio_path.unlink(missing_ok=True)
|
||||
@@ -0,0 +1,52 @@
|
||||
from __future__ import annotations
|
||||
|
||||
from pathlib import Path
|
||||
|
||||
import requests
|
||||
|
||||
from backend.app.config import settings
|
||||
|
||||
|
||||
class DeepgramClient:
|
||||
endpoint = "https://api.deepgram.com/v1/listen?model=nova-3&smart_format=true&language=es"
|
||||
|
||||
def transcribe_file(self, path: Path) -> dict:
|
||||
if not settings.deepgram_api_key:
|
||||
raise RuntimeError("DEEPGRAM_API_KEY no esta configurada")
|
||||
|
||||
headers = {
|
||||
"Authorization": f"Token {settings.deepgram_api_key}",
|
||||
"Content-Type": self._content_type(path),
|
||||
}
|
||||
with path.open("rb") as audio_file:
|
||||
response = requests.post(self.endpoint, headers=headers, data=audio_file, timeout=600)
|
||||
|
||||
if response.status_code == 401:
|
||||
raise RuntimeError("API Key de Deepgram invalida")
|
||||
if not response.ok:
|
||||
try:
|
||||
error = response.json()
|
||||
except ValueError:
|
||||
error = response.text
|
||||
raise RuntimeError(f"Deepgram error {response.status_code}: {error}")
|
||||
return response.json()
|
||||
|
||||
def transcript_from_response(self, data: dict) -> tuple[str, float | None]:
|
||||
alternative = data.get("results", {}).get("channels", [{}])[0].get("alternatives", [{}])[0]
|
||||
transcript = alternative.get("paragraphs", {}).get("transcript") or alternative.get("transcript") or ""
|
||||
confidence = alternative.get("confidence")
|
||||
if not transcript.strip():
|
||||
raise RuntimeError("Deepgram no devolvio transcripcion")
|
||||
return transcript.strip(), confidence
|
||||
|
||||
def _content_type(self, path: Path) -> str:
|
||||
ext = path.suffix.lower()
|
||||
return {
|
||||
".mp3": "audio/mpeg",
|
||||
".wav": "audio/wav",
|
||||
".m4a": "audio/mp4",
|
||||
".mp4": "audio/mp4",
|
||||
".webm": "audio/webm",
|
||||
".ogg": "audio/ogg",
|
||||
".flac": "audio/flac",
|
||||
}.get(ext, "application/octet-stream")
|
||||
@@ -0,0 +1,52 @@
|
||||
from __future__ import annotations
|
||||
|
||||
import base64
|
||||
from pathlib import Path
|
||||
|
||||
import requests
|
||||
|
||||
from backend.app.config import settings
|
||||
|
||||
|
||||
class MistralClient:
|
||||
endpoint = "https://api.mistral.ai/v1/chat/completions"
|
||||
model = "pixtral-12b-2409"
|
||||
|
||||
def ocr_image(self, image_path: Path) -> str:
|
||||
if not settings.mistral_api_key:
|
||||
raise RuntimeError("MISTRAL_API_KEY no esta configurada")
|
||||
data_url = self._image_data_url(image_path)
|
||||
response = requests.post(
|
||||
self.endpoint,
|
||||
headers={
|
||||
"Content-Type": "application/json",
|
||||
"Authorization": f"Bearer {settings.mistral_api_key}",
|
||||
},
|
||||
json={
|
||||
"model": self.model,
|
||||
"messages": [
|
||||
{
|
||||
"role": "user",
|
||||
"content": [
|
||||
{
|
||||
"type": "text",
|
||||
"text": "Extract all text from this image and format it as Markdown. If there are tables, represent them using Markdown table syntax. Do not include commentary.",
|
||||
},
|
||||
{"type": "image_url", "image_url": data_url},
|
||||
],
|
||||
}
|
||||
],
|
||||
"max_tokens": 3000,
|
||||
},
|
||||
timeout=180,
|
||||
)
|
||||
if response.status_code == 401:
|
||||
raise RuntimeError("API Key de Mistral invalida")
|
||||
if not response.ok:
|
||||
raise RuntimeError(f"Mistral error {response.status_code}: {response.text}")
|
||||
data = response.json()
|
||||
return data["choices"][0]["message"]["content"]
|
||||
|
||||
def _image_data_url(self, image_path: Path) -> str:
|
||||
encoded = base64.b64encode(image_path.read_bytes()).decode("ascii")
|
||||
return f"data:image/jpeg;base64,{encoded}"
|
||||
@@ -0,0 +1,8 @@
|
||||
import re
|
||||
import unicodedata
|
||||
|
||||
|
||||
def slugify(value: str) -> str:
|
||||
normalized = unicodedata.normalize("NFKD", value).encode("ascii", "ignore").decode("ascii")
|
||||
slug = re.sub(r"[^a-zA-Z0-9]+", "-", normalized).strip("-").lower()
|
||||
return slug or "sin-titulo"
|
||||
@@ -0,0 +1,12 @@
|
||||
@echo off
|
||||
setlocal
|
||||
title Knowledge Station - Cerrar
|
||||
|
||||
set "ROOT=%~dp0"
|
||||
set "ROOT=%ROOT:~0,-1%"
|
||||
powershell -NoProfile -ExecutionPolicy Bypass -File "%ROOT%\scripts\stop-platform.ps1" -Root "%ROOT%"
|
||||
|
||||
echo.
|
||||
echo Puedes cerrar esta ventana.
|
||||
pause
|
||||
exit /b 0
|
||||
@@ -0,0 +1,43 @@
|
||||
services:
|
||||
backend:
|
||||
build:
|
||||
context: .
|
||||
dockerfile: Dockerfile
|
||||
environment:
|
||||
APP_ENV: production
|
||||
APP_DATA_DIR: /app/data
|
||||
DATABASE_PATH: /app/data/app.db
|
||||
MISTRAL_API_KEY: ${MISTRAL_API_KEY:-}
|
||||
DEEPGRAM_API_KEY: ${DEEPGRAM_API_KEY:-}
|
||||
volumes:
|
||||
- station-data:/app/data
|
||||
expose:
|
||||
- "8000"
|
||||
restart: unless-stopped
|
||||
healthcheck:
|
||||
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=5)"]
|
||||
interval: 30s
|
||||
timeout: 10s
|
||||
retries: 3
|
||||
start_period: 20s
|
||||
|
||||
frontend:
|
||||
build:
|
||||
context: ./frontend
|
||||
dockerfile: Dockerfile
|
||||
args:
|
||||
NEXT_PUBLIC_API_URL: ""
|
||||
BACKEND_INTERNAL_URL: http://backend:8000
|
||||
environment:
|
||||
NODE_ENV: production
|
||||
PORT: 3000
|
||||
HOSTNAME: 0.0.0.0
|
||||
expose:
|
||||
- "3000"
|
||||
depends_on:
|
||||
backend:
|
||||
condition: service_healthy
|
||||
restart: unless-stopped
|
||||
|
||||
volumes:
|
||||
station-data:
|
||||
@@ -0,0 +1,18 @@
|
||||
services:
|
||||
backend:
|
||||
build: .
|
||||
ports:
|
||||
- "8000:8000"
|
||||
env_file:
|
||||
- .env
|
||||
volumes:
|
||||
- ./data:/app/data
|
||||
frontend:
|
||||
build:
|
||||
context: ./frontend
|
||||
ports:
|
||||
- "3000:3000"
|
||||
environment:
|
||||
NEXT_PUBLIC_API_URL: http://127.0.0.1:8000
|
||||
depends_on:
|
||||
- backend
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,6 @@
|
||||
.next
|
||||
node_modules
|
||||
.env.local
|
||||
.env.*.local
|
||||
npm-debug.log*
|
||||
Dockerfile*
|
||||
@@ -0,0 +1 @@
|
||||
NEXT_PUBLIC_API_URL=http://127.0.0.1:8000
|
||||
@@ -0,0 +1,24 @@
|
||||
FROM node:20-alpine AS deps
|
||||
WORKDIR /app
|
||||
COPY package.json package-lock.json* ./
|
||||
RUN npm ci
|
||||
|
||||
FROM node:20-alpine AS builder
|
||||
WORKDIR /app
|
||||
COPY --from=deps /app/node_modules ./node_modules
|
||||
COPY . .
|
||||
ARG NEXT_PUBLIC_API_URL=
|
||||
ARG BACKEND_INTERNAL_URL=http://backend:8000
|
||||
ENV NEXT_PUBLIC_API_URL=$NEXT_PUBLIC_API_URL
|
||||
ENV BACKEND_INTERNAL_URL=$BACKEND_INTERNAL_URL
|
||||
RUN npm run build
|
||||
|
||||
FROM node:20-alpine AS runner
|
||||
WORKDIR /app
|
||||
ENV NODE_ENV=production
|
||||
ENV HOSTNAME=0.0.0.0
|
||||
COPY --from=builder /app/.next/standalone ./
|
||||
COPY --from=builder /app/.next/static ./.next/static
|
||||
COPY --from=builder /app/public ./public
|
||||
EXPOSE 3000
|
||||
CMD ["node", "server.js"]
|
||||
@@ -0,0 +1,65 @@
|
||||
"use client";
|
||||
|
||||
import { useEffect, useState } from "react";
|
||||
import Link from "next/link";
|
||||
import ReactMarkdown from "react-markdown";
|
||||
import remarkGfm from "remark-gfm";
|
||||
import { api, DocumentItem } from "@/lib/api";
|
||||
import { Shell } from "@/components/Shell";
|
||||
import { Button, Card } from "@/components/ui";
|
||||
|
||||
export default function DocumentPage({ params }: { params: { documentId: string } }) {
|
||||
const documentId = Number(params.documentId);
|
||||
const [document, setDocument] = useState<DocumentItem | null>(null);
|
||||
const [content, setContent] = useState("");
|
||||
const [savedContent, setSavedContent] = useState("");
|
||||
const [status, setStatus] = useState("");
|
||||
const dirty = content !== savedContent;
|
||||
|
||||
useEffect(() => {
|
||||
Promise.all([api.document(documentId), api.markdown(documentId)]).then(([doc, markdown]) => {
|
||||
setDocument(doc); setContent(markdown); setSavedContent(markdown);
|
||||
}).catch((err) => setStatus(err.message));
|
||||
}, [documentId]);
|
||||
|
||||
async function save() {
|
||||
setStatus("Guardando...");
|
||||
await api.saveMarkdown(documentId, content);
|
||||
setSavedContent(content);
|
||||
setStatus("Guardado");
|
||||
}
|
||||
|
||||
async function copy() {
|
||||
await navigator.clipboard.writeText(content);
|
||||
setStatus("Copiado al portapapeles");
|
||||
}
|
||||
|
||||
return (
|
||||
<Shell>
|
||||
<Link href="/" className="text-sm text-blue-300">Volver al dashboard</Link>
|
||||
<div className="mt-4 flex flex-wrap items-center justify-between gap-4">
|
||||
<div>
|
||||
<h1 className="text-3xl font-bold">{document?.title || "Documento"}</h1>
|
||||
<p className="mt-1 text-sm text-slate-400">{dirty ? "Cambios sin guardar" : "Sin cambios pendientes"} {status && `- ${status}`}</p>
|
||||
</div>
|
||||
<div className="flex flex-wrap gap-2">
|
||||
<Button onClick={save} disabled={!dirty}>Guardar</Button>
|
||||
<Button className="bg-slate-700 hover:bg-slate-600" onClick={copy}>Copiar</Button>
|
||||
<a className="rounded-xl border border-white/10 px-4 py-2 hover:bg-white/10" href={api.downloadDocumentUrl(documentId)}>Descargar .md</a>
|
||||
</div>
|
||||
</div>
|
||||
<div className="mt-6 grid gap-6 xl:grid-cols-2">
|
||||
<Card className="p-0">
|
||||
<div className="border-b border-white/10 px-4 py-3 font-medium">Editor Markdown</div>
|
||||
<textarea value={content} onChange={(event) => setContent(event.target.value)} className="h-[72vh] w-full resize-none rounded-b-2xl bg-slate-950 p-4 font-mono text-sm leading-6 outline-none" />
|
||||
</Card>
|
||||
<Card className="overflow-auto">
|
||||
<div className="mb-4 font-medium">Vista previa</div>
|
||||
<article className="prose prose-invert prose-slate">
|
||||
<ReactMarkdown remarkPlugins={[remarkGfm]}>{content}</ReactMarkdown>
|
||||
</article>
|
||||
</Card>
|
||||
</div>
|
||||
</Shell>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,20 @@
|
||||
@tailwind base;
|
||||
@tailwind components;
|
||||
@tailwind utilities;
|
||||
|
||||
:root {
|
||||
color-scheme: dark;
|
||||
}
|
||||
|
||||
body {
|
||||
background: #020617;
|
||||
color: #e5e7eb;
|
||||
}
|
||||
|
||||
textarea, input, button {
|
||||
font: inherit;
|
||||
}
|
||||
|
||||
.prose {
|
||||
max-width: none;
|
||||
}
|
||||
@@ -0,0 +1,15 @@
|
||||
import type { Metadata } from "next";
|
||||
import "./globals.css";
|
||||
|
||||
export const metadata: Metadata = {
|
||||
title: "Knowledge Station",
|
||||
description: "Procesa clases, documentos y transcripciones a Markdown.",
|
||||
};
|
||||
|
||||
export default function RootLayout({ children }: { children: React.ReactNode }) {
|
||||
return (
|
||||
<html lang="es" className="dark">
|
||||
<body>{children}</body>
|
||||
</html>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,87 @@
|
||||
"use client";
|
||||
|
||||
import Link from "next/link";
|
||||
import { FormEvent, useEffect, useState } from "react";
|
||||
import { api, SettingsStatus, Subject } from "@/lib/api";
|
||||
import { Shell } from "@/components/Shell";
|
||||
import { Button, Card, Input, Label } from "@/components/ui";
|
||||
|
||||
export default function DashboardPage() {
|
||||
const [subjects, setSubjects] = useState<Subject[]>([]);
|
||||
const [settings, setSettings] = useState<SettingsStatus | null>(null);
|
||||
const [name, setName] = useState("");
|
||||
const [loading, setLoading] = useState(true);
|
||||
const [error, setError] = useState("");
|
||||
|
||||
async function load() {
|
||||
setError("");
|
||||
try {
|
||||
const [subjectList, status] = await Promise.all([api.subjects(), api.settings()]);
|
||||
setSubjects(subjectList);
|
||||
setSettings(status);
|
||||
} catch (err) {
|
||||
setError(err instanceof Error ? err.message : "No se pudo conectar con la API");
|
||||
} finally {
|
||||
setLoading(false);
|
||||
}
|
||||
}
|
||||
|
||||
useEffect(() => { load(); }, []);
|
||||
|
||||
async function createSubject(event: FormEvent) {
|
||||
event.preventDefault();
|
||||
if (!name.trim()) return;
|
||||
await api.createSubject(name.trim());
|
||||
setName("");
|
||||
await load();
|
||||
}
|
||||
|
||||
return (
|
||||
<Shell>
|
||||
<div className="grid gap-6 lg:grid-cols-[1fr_360px]">
|
||||
<section>
|
||||
<div className="mb-6">
|
||||
<p className="text-sm uppercase tracking-[0.25em] text-blue-300">Plataforma personal</p>
|
||||
<h1 className="mt-2 text-4xl font-bold tracking-tight">Materias</h1>
|
||||
<p className="mt-3 max-w-2xl text-slate-400">Organiza clases, PDFs, audios y videos por materia y semana. Todo termina en Markdown listo para ingesta en LLM.</p>
|
||||
</div>
|
||||
|
||||
{error && <Card className="mb-4 border-red-500/30 bg-red-500/10 text-red-200">{error}</Card>}
|
||||
{loading ? <Card>Cargando...</Card> : (
|
||||
<div className="grid gap-4 md:grid-cols-2">
|
||||
{subjects.map((subject) => (
|
||||
<Link key={subject.id} href={`/subjects/${subject.id}`}>
|
||||
<Card className="transition hover:border-blue-400/40 hover:bg-blue-500/10">
|
||||
<div className="text-xl font-semibold">{subject.name}</div>
|
||||
<div className="mt-2 text-sm text-slate-400">{subject.slug}</div>
|
||||
<div className="mt-5 text-sm text-blue-300">Abrir materia</div>
|
||||
</Card>
|
||||
</Link>
|
||||
))}
|
||||
{subjects.length === 0 && <Card className="md:col-span-2">No hay materias todavia. Crea la primera para empezar.</Card>}
|
||||
</div>
|
||||
)}
|
||||
</section>
|
||||
|
||||
<aside className="space-y-4">
|
||||
<Card>
|
||||
<h2 className="text-lg font-semibold">Nueva materia</h2>
|
||||
<form className="mt-4 space-y-3" onSubmit={createSubject}>
|
||||
<Label>Nombre</Label>
|
||||
<Input value={name} onChange={(event) => setName(event.target.value)} placeholder="Ej. Comportamiento Organizacional" />
|
||||
<Button className="w-full" disabled={!name.trim()}>Crear materia</Button>
|
||||
</form>
|
||||
</Card>
|
||||
|
||||
<Card>
|
||||
<h2 className="text-lg font-semibold">Servicios</h2>
|
||||
<div className="mt-4 space-y-3 text-sm">
|
||||
<div className="flex justify-between"><span>Deepgram</span><span className={settings?.deepgram_configured ? "text-emerald-300" : "text-amber-300"}>{settings?.deepgram_configured ? "Configurado" : "Falta key"}</span></div>
|
||||
<div className="flex justify-between"><span>Mistral OCR</span><span className={settings?.mistral_configured ? "text-emerald-300" : "text-amber-300"}>{settings?.mistral_configured ? "Configurado" : "Falta key"}</span></div>
|
||||
</div>
|
||||
</Card>
|
||||
</aside>
|
||||
</div>
|
||||
</Shell>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,51 @@
|
||||
"use client";
|
||||
|
||||
import { useEffect, useState } from "react";
|
||||
import { api, API_URL, SettingsStatus } from "@/lib/api";
|
||||
import { Shell } from "@/components/Shell";
|
||||
import { Card } from "@/components/ui";
|
||||
|
||||
export default function SettingsPage() {
|
||||
const [settings, setSettings] = useState<SettingsStatus | null>(null);
|
||||
const [apiOk, setApiOk] = useState(false);
|
||||
const [error, setError] = useState("");
|
||||
|
||||
useEffect(() => {
|
||||
Promise.all([api.health(), api.settings()]).then(([, status]) => {
|
||||
setApiOk(true); setSettings(status);
|
||||
}).catch((err) => setError(err.message));
|
||||
}, []);
|
||||
|
||||
return (
|
||||
<Shell>
|
||||
<h1 className="text-4xl font-bold">Configuracion</h1>
|
||||
<div className="mt-6 grid gap-4 lg:grid-cols-2">
|
||||
<Card>
|
||||
<h2 className="text-xl font-semibold">Estado</h2>
|
||||
<div className="mt-4 space-y-3 text-sm">
|
||||
<Row label="API" value={apiOk ? "Activa" : "No disponible"} ok={apiOk} />
|
||||
<Row label="Deepgram" value={settings?.deepgram_configured ? "Configurado" : "Falta DEEPGRAM_API_KEY"} ok={!!settings?.deepgram_configured} />
|
||||
<Row label="Mistral" value={settings?.mistral_configured ? "Configurado" : "Falta MISTRAL_API_KEY"} ok={!!settings?.mistral_configured} />
|
||||
</div>
|
||||
{error && <p className="mt-4 text-sm text-red-300">{error}</p>}
|
||||
</Card>
|
||||
<Card>
|
||||
<h2 className="text-xl font-semibold">Rutas</h2>
|
||||
<div className="mt-4 space-y-3 text-sm text-slate-300">
|
||||
<p><span className="text-slate-500">API:</span> {API_URL}</p>
|
||||
<p><span className="text-slate-500">Datos:</span> {settings?.data_dir || "-"}</p>
|
||||
<p><span className="text-slate-500">SQLite:</span> {settings?.database_path || "-"}</p>
|
||||
</div>
|
||||
</Card>
|
||||
</div>
|
||||
<Card className="mt-4">
|
||||
<h2 className="text-xl font-semibold">API Keys</h2>
|
||||
<p className="mt-3 text-slate-400">Configura las claves en el archivo <code className="rounded bg-slate-900 px-2 py-1">.env</code>. No se muestran ni editan desde la UI para evitar pegarlas accidentalmente en capturas o commits.</p>
|
||||
</Card>
|
||||
</Shell>
|
||||
);
|
||||
}
|
||||
|
||||
function Row({ label, value, ok }: { label: string; value: string; ok: boolean }) {
|
||||
return <div className="flex justify-between gap-4"><span>{label}</span><span className={ok ? "text-emerald-300" : "text-amber-300"}>{value}</span></div>;
|
||||
}
|
||||
@@ -0,0 +1,121 @@
|
||||
"use client";
|
||||
|
||||
import Link from "next/link";
|
||||
import { FormEvent, useEffect, useState } from "react";
|
||||
import { api, Subject, Week } from "@/lib/api";
|
||||
import { Shell } from "@/components/Shell";
|
||||
import { Button, Card, Input, Label } from "@/components/ui";
|
||||
|
||||
export default function SubjectPage({ params }: { params: { subjectId: string } }) {
|
||||
const subjectId = Number(params.subjectId);
|
||||
const [subject, setSubject] = useState<Subject | null>(null);
|
||||
const [weeks, setWeeks] = useState<Week[]>([]);
|
||||
const [number, setNumber] = useState(1);
|
||||
const [title, setTitle] = useState("");
|
||||
const [editingWeek, setEditingWeek] = useState<Week | null>(null);
|
||||
const [editNumber, setEditNumber] = useState(1);
|
||||
const [editTitle, setEditTitle] = useState("");
|
||||
const [error, setError] = useState("");
|
||||
|
||||
async function load() {
|
||||
try {
|
||||
const [subjectData, weekList] = await Promise.all([api.subject(subjectId), api.weeks(subjectId)]);
|
||||
setSubject(subjectData);
|
||||
setWeeks(weekList);
|
||||
setNumber((weekList.at(-1)?.number || 0) + 1);
|
||||
} catch (err) {
|
||||
setError(err instanceof Error ? err.message : "No se pudo cargar la materia");
|
||||
}
|
||||
}
|
||||
|
||||
useEffect(() => { load(); }, [subjectId]);
|
||||
|
||||
async function createWeek(event: FormEvent) {
|
||||
event.preventDefault();
|
||||
try {
|
||||
setError("");
|
||||
await api.createWeek(subjectId, number, title);
|
||||
setTitle("");
|
||||
await load();
|
||||
} catch (err) {
|
||||
setError(err instanceof Error ? err.message : "No se pudo crear la semana");
|
||||
}
|
||||
}
|
||||
|
||||
function startEdit(week: Week) {
|
||||
setEditingWeek(week);
|
||||
setEditNumber(week.number);
|
||||
setEditTitle(week.title || "");
|
||||
setError("");
|
||||
}
|
||||
|
||||
async function saveEdit(event: FormEvent) {
|
||||
event.preventDefault();
|
||||
if (!editingWeek) return;
|
||||
try {
|
||||
setError("");
|
||||
await api.updateWeek(subjectId, editingWeek.number, editNumber, editTitle);
|
||||
setEditingWeek(null);
|
||||
await load();
|
||||
} catch (err) {
|
||||
setError(err instanceof Error ? err.message : "No se pudo editar la semana");
|
||||
}
|
||||
}
|
||||
|
||||
return (
|
||||
<Shell>
|
||||
<Link href="/" className="text-sm text-blue-300">Volver a materias</Link>
|
||||
<div className="mt-4 grid gap-6 lg:grid-cols-[1fr_360px]">
|
||||
<section>
|
||||
<h1 className="text-4xl font-bold tracking-tight">{subject?.name || "Materia"}</h1>
|
||||
<p className="mt-2 text-slate-400">Divide el conocimiento por semanas y sube fuentes a cada una.</p>
|
||||
{error && <Card className="mt-4 border-red-500/30 bg-red-500/10 text-red-200">{error}</Card>}
|
||||
<div className="mt-6 grid gap-4 md:grid-cols-2">
|
||||
{weeks.map((week) => (
|
||||
<Card key={week.id} className="transition hover:border-blue-400/40 hover:bg-blue-500/10">
|
||||
<div className="text-2xl font-semibold">Semana {week.number}</div>
|
||||
<div className="mt-2 text-slate-400">{week.title || "Sin titulo"}</div>
|
||||
<div className="mt-5 flex flex-wrap gap-2 text-sm">
|
||||
<Link className="rounded-lg bg-blue-600 px-3 py-2 font-medium text-white hover:bg-blue-500" href={`/subjects/${subjectId}/weeks/${week.number}`}>Abrir semana</Link>
|
||||
<button className="rounded-lg border border-white/10 px-3 py-2 text-slate-200 hover:bg-white/10" type="button" onClick={() => startEdit(week)}>Editar</button>
|
||||
</div>
|
||||
</Card>
|
||||
))}
|
||||
{weeks.length === 0 && <Card className="md:col-span-2">Todavia no hay semanas.</Card>}
|
||||
</div>
|
||||
</section>
|
||||
<aside>
|
||||
<Card>
|
||||
{editingWeek ? (
|
||||
<>
|
||||
<h2 className="text-lg font-semibold">Editar semana</h2>
|
||||
<p className="mt-1 text-sm text-slate-400">Cambia el numero o titulo. Si cambias el numero, se renombra la carpeta de la semana.</p>
|
||||
<form className="mt-4 space-y-3" onSubmit={saveEdit}>
|
||||
<Label>Numero</Label>
|
||||
<Input type="number" min={1} value={editNumber} onChange={(event) => setEditNumber(Number(event.target.value))} />
|
||||
<Label>Titulo</Label>
|
||||
<Input value={editTitle} onChange={(event) => setEditTitle(event.target.value)} placeholder="Ej. Grupos y equipos" />
|
||||
<div className="flex gap-2">
|
||||
<Button className="flex-1">Guardar</Button>
|
||||
<button className="flex-1 rounded-xl border border-white/10 px-4 py-2 hover:bg-white/10" type="button" onClick={() => setEditingWeek(null)}>Cancelar</button>
|
||||
</div>
|
||||
</form>
|
||||
</>
|
||||
) : (
|
||||
<>
|
||||
<h2 className="text-lg font-semibold">Nueva semana</h2>
|
||||
<form className="mt-4 space-y-3" onSubmit={createWeek}>
|
||||
<Label>Numero</Label>
|
||||
<Input type="number" min={1} value={number} onChange={(event) => setNumber(Number(event.target.value))} />
|
||||
<Label>Titulo opcional</Label>
|
||||
<Input value={title} onChange={(event) => setTitle(event.target.value)} placeholder="Ej. Grupos y equipos" />
|
||||
<Button className="w-full">Crear semana</Button>
|
||||
</form>
|
||||
</>
|
||||
)}
|
||||
</Card>
|
||||
</aside>
|
||||
</div>
|
||||
</Shell>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,343 @@
|
||||
"use client";
|
||||
|
||||
import Link from "next/link";
|
||||
import { ChangeEvent, DragEvent, useEffect, useMemo, useState } from "react";
|
||||
import { api, DocumentItem, Job, Source, Subject, Week } from "@/lib/api";
|
||||
import { Shell } from "@/components/Shell";
|
||||
import { Button, Card, ConfirmDialog, StatusBadge } from "@/components/ui";
|
||||
|
||||
type PendingDelete =
|
||||
| { type: "document"; item: DocumentItem }
|
||||
| { type: "source"; item: Source };
|
||||
|
||||
export default function WeekPage({ params }: { params: { subjectId: string; weekNumber: string } }) {
|
||||
const subjectId = Number(params.subjectId);
|
||||
const weekNumber = Number(params.weekNumber);
|
||||
const [subject, setSubject] = useState<Subject | null>(null);
|
||||
const [week, setWeek] = useState<Week | null>(null);
|
||||
const [sources, setSources] = useState<Source[]>([]);
|
||||
const [jobs, setJobs] = useState<Job[]>([]);
|
||||
const [documents, setDocuments] = useState<DocumentItem[]>([]);
|
||||
const [pdfMethod, setPdfMethod] = useState<"standard" | "mistral_ocr">("standard");
|
||||
const [pageMode, setPageMode] = useState<"all" | "ranges">("all");
|
||||
const [pageRanges, setPageRanges] = useState("");
|
||||
const [pendingFiles, setPendingFiles] = useState<File[]>([]);
|
||||
const [uploading, setUploading] = useState(false);
|
||||
const [deletingId, setDeletingId] = useState<string | null>(null);
|
||||
const [reprocessingId, setReprocessingId] = useState<number | null>(null);
|
||||
const [pendingDelete, setPendingDelete] = useState<PendingDelete | null>(null);
|
||||
const [dragActive, setDragActive] = useState(false);
|
||||
const [error, setError] = useState("");
|
||||
|
||||
const activeJobs = useMemo(() => jobs.some((job) => ["queued", "processing"].includes(job.status)), [jobs]);
|
||||
const pendingPdfCount = useMemo(() => pendingFiles.filter(isPdfFile).length, [pendingFiles]);
|
||||
|
||||
async function load() {
|
||||
const [subjectData, weekData, sourceList, jobList, documentList] = await Promise.all([
|
||||
api.subject(subjectId), api.week(subjectId, weekNumber), api.sources(subjectId, weekNumber), api.jobs(subjectId, weekNumber), api.documents(subjectId, weekNumber),
|
||||
]);
|
||||
setSubject(subjectData); setWeek(weekData); setSources(sourceList); setJobs(jobList); setDocuments(documentList);
|
||||
}
|
||||
|
||||
useEffect(() => { load().catch((err) => setError(err.message)); }, [subjectId, weekNumber]);
|
||||
useEffect(() => {
|
||||
if (!activeJobs) return;
|
||||
const timer = window.setInterval(() => load().catch((err) => setError(err.message)), 2500);
|
||||
return () => window.clearInterval(timer);
|
||||
}, [activeJobs]);
|
||||
|
||||
async function uploadFileList(files: File[]) {
|
||||
if (!files.length) return;
|
||||
setUploading(true); setError("");
|
||||
try {
|
||||
const ranges = pageMode === "ranges" ? pageRanges : undefined;
|
||||
for (const file of files) await api.upload(subjectId, weekNumber, file, pdfMethod === "mistral_ocr", ranges);
|
||||
await load();
|
||||
} catch (err) {
|
||||
setError(err instanceof Error ? err.message : "No se pudo subir archivo");
|
||||
} finally {
|
||||
setUploading(false);
|
||||
setDragActive(false);
|
||||
}
|
||||
}
|
||||
|
||||
function isPdfFile(file: File) {
|
||||
return file.type === "application/pdf" || file.name.toLowerCase().endsWith(".pdf");
|
||||
}
|
||||
|
||||
async function prepareFiles(files: File[]) {
|
||||
if (!files.length) return;
|
||||
setError("");
|
||||
setDragActive(false);
|
||||
if (files.some(isPdfFile)) {
|
||||
setPendingFiles(files);
|
||||
return;
|
||||
}
|
||||
await uploadFileList(files);
|
||||
}
|
||||
|
||||
async function confirmPendingUpload() {
|
||||
const files = pendingFiles;
|
||||
setPendingFiles([]);
|
||||
await uploadFileList(files);
|
||||
}
|
||||
|
||||
async function uploadFiles(event: ChangeEvent<HTMLInputElement>) {
|
||||
const files = Array.from(event.target.files || []);
|
||||
try {
|
||||
await prepareFiles(files);
|
||||
} finally {
|
||||
event.target.value = "";
|
||||
}
|
||||
}
|
||||
|
||||
function handleDrag(event: DragEvent<HTMLDivElement>) {
|
||||
event.preventDefault();
|
||||
event.stopPropagation();
|
||||
if (uploading) return;
|
||||
if (event.type === "dragenter" || event.type === "dragover") setDragActive(true);
|
||||
if (event.type === "dragleave") setDragActive(false);
|
||||
}
|
||||
|
||||
async function handleDrop(event: DragEvent<HTMLDivElement>) {
|
||||
event.preventDefault();
|
||||
event.stopPropagation();
|
||||
if (uploading) return;
|
||||
const files = Array.from(event.dataTransfer.files || []);
|
||||
await prepareFiles(files);
|
||||
}
|
||||
|
||||
async function deleteDocument(document: DocumentItem) {
|
||||
setDeletingId(`document-${document.id}`); setError("");
|
||||
try {
|
||||
await api.deleteDocument(document.id);
|
||||
await load();
|
||||
setPendingDelete(null);
|
||||
} catch (err) {
|
||||
setError(err instanceof Error ? err.message : "No se pudo eliminar el documento");
|
||||
} finally {
|
||||
setDeletingId(null);
|
||||
}
|
||||
}
|
||||
|
||||
async function deleteSource(source: Source) {
|
||||
setDeletingId(`source-${source.id}`); setError("");
|
||||
try {
|
||||
await api.deleteSource(source.id);
|
||||
await load();
|
||||
setPendingDelete(null);
|
||||
} catch (err) {
|
||||
setError(err instanceof Error ? err.message : "No se pudo eliminar la fuente");
|
||||
} finally {
|
||||
setDeletingId(null);
|
||||
}
|
||||
}
|
||||
|
||||
async function reprocessSource(source: Source) {
|
||||
setReprocessingId(source.id); setError("");
|
||||
try {
|
||||
const ranges = source.source_type === "pdf" && pageMode === "ranges" ? pageRanges : undefined;
|
||||
await api.reprocessSource(source.id, source.source_type === "pdf" && pdfMethod === "mistral_ocr", ranges);
|
||||
await load();
|
||||
} catch (err) {
|
||||
setError(err instanceof Error ? err.message : "No se pudo reprocesar la fuente");
|
||||
} finally {
|
||||
setReprocessingId(null);
|
||||
}
|
||||
}
|
||||
|
||||
function formatFileSize(size?: number | null) {
|
||||
if (!size) return "tamano pendiente";
|
||||
if (size < 1024 * 1024) return `${Math.round(size / 1024)} KB`;
|
||||
return `${(size / (1024 * 1024)).toFixed(1)} MB`;
|
||||
}
|
||||
|
||||
function sourceProcessingLabel(source: Source) {
|
||||
if (source.document_id) return `Genero: ${source.document_title}`;
|
||||
if (["queued", "processing"].includes(source.status)) return "Procesamiento en curso";
|
||||
if (source.status === "failed") return "Fallo el procesamiento";
|
||||
return "Sin Markdown generado";
|
||||
}
|
||||
|
||||
function confirmPendingDelete() {
|
||||
if (!pendingDelete) return;
|
||||
if (pendingDelete.type === "document") void deleteDocument(pendingDelete.item);
|
||||
if (pendingDelete.type === "source") void deleteSource(pendingDelete.item);
|
||||
}
|
||||
|
||||
const deleteDialog = pendingDelete ? {
|
||||
title: pendingDelete.type === "document" ? "Eliminar Markdown procesado" : "Eliminar fuente",
|
||||
description: pendingDelete.type === "document"
|
||||
? `Se eliminara el archivo Markdown "${pendingDelete.item.title}". La fuente original quedara disponible.`
|
||||
: `Se eliminara la fuente "${pendingDelete.item.original_name}" junto con sus trabajos y Markdown procesado asociado.`,
|
||||
} : null;
|
||||
|
||||
return (
|
||||
<Shell>
|
||||
<Link href={`/subjects/${subjectId}`} className="text-sm text-blue-300">Volver a {subject?.name || "materia"}</Link>
|
||||
<div className="mt-4 grid gap-6 lg:grid-cols-[1fr_380px]">
|
||||
<section className="space-y-6">
|
||||
<div>
|
||||
<h1 className="text-4xl font-bold">Semana {weekNumber}</h1>
|
||||
<p className="mt-2 text-slate-400">{week?.title || "Sube fuentes y genera Markdown procesado."}</p>
|
||||
</div>
|
||||
{error && <Card className="border-red-500/30 bg-red-500/10 text-red-200">{error}</Card>}
|
||||
|
||||
<Card>
|
||||
<div className="flex flex-wrap items-center justify-between gap-4">
|
||||
<div>
|
||||
<h2 className="text-xl font-semibold">Subir archivos</h2>
|
||||
<p className="mt-1 text-sm text-slate-400">PDF, DOCX, TXT, Markdown, audio y video.</p>
|
||||
</div>
|
||||
<label className="cursor-pointer rounded-xl bg-blue-600 px-4 py-2 font-medium hover:bg-blue-500">
|
||||
{uploading ? "Subiendo..." : "Seleccionar archivos"}
|
||||
<input className="hidden" type="file" multiple onChange={uploadFiles} disabled={uploading} />
|
||||
</label>
|
||||
</div>
|
||||
<div
|
||||
className={`mt-5 rounded-2xl border-2 border-dashed p-8 text-center transition ${dragActive ? "border-blue-400 bg-blue-500/15" : "border-white/15 bg-slate-950/50 hover:border-blue-400/50"} ${uploading ? "opacity-70" : ""}`}
|
||||
onDragEnter={handleDrag}
|
||||
onDragOver={handleDrag}
|
||||
onDragLeave={handleDrag}
|
||||
onDrop={handleDrop}
|
||||
>
|
||||
<div className="mx-auto grid h-14 w-14 place-items-center rounded-2xl bg-blue-500/15 text-2xl">↑</div>
|
||||
<p className="mt-4 font-medium">Arrastra archivos aqui</p>
|
||||
<p className="mt-2 text-sm text-slate-400">Puedes soltar varios archivos a la vez. Se procesaran en esta semana.</p>
|
||||
<p className="mt-3 text-xs text-slate-500">Soporta PDF, DOCX, TXT, Markdown, audio y video.</p>
|
||||
{uploading && <p className="mt-4 text-sm text-blue-300">Subiendo archivos...</p>}
|
||||
</div>
|
||||
{pendingFiles.length > 0 && (
|
||||
<div className="mt-5 rounded-2xl border border-blue-400/40 bg-blue-500/10 p-4">
|
||||
<div className="flex flex-wrap items-start justify-between gap-3">
|
||||
<div>
|
||||
<h3 className="font-medium">Configurar PDFs antes de subir</h3>
|
||||
<p className="mt-1 text-sm text-slate-300">
|
||||
Detecte {pendingPdfCount} PDF en {pendingFiles.length} archivo(s). Elige como procesarlos antes de continuar.
|
||||
</p>
|
||||
</div>
|
||||
<button className="rounded-lg border border-white/10 px-3 py-1.5 text-sm hover:bg-white/10" type="button" onClick={() => setPendingFiles([])}>Cancelar</button>
|
||||
</div>
|
||||
<div className="mt-4 grid gap-4 md:grid-cols-2">
|
||||
<div>
|
||||
<label className="text-sm font-medium text-slate-300">Metodo de procesamiento</label>
|
||||
<select
|
||||
className="mt-2 w-full rounded-xl border border-white/10 bg-slate-900 px-3 py-2 text-sm outline-none ring-blue-500/30 focus:ring-4"
|
||||
value={pdfMethod}
|
||||
onChange={(event) => setPdfMethod(event.target.value as "standard" | "mistral_ocr")}
|
||||
>
|
||||
<option value="standard">Extraccion estandar de texto</option>
|
||||
<option value="mistral_ocr">Mistral OCR (IA)</option>
|
||||
</select>
|
||||
</div>
|
||||
<div>
|
||||
<label className="text-sm font-medium text-slate-300">Paginas a procesar</label>
|
||||
<select
|
||||
className="mt-2 w-full rounded-xl border border-white/10 bg-slate-900 px-3 py-2 text-sm outline-none ring-blue-500/30 focus:ring-4"
|
||||
value={pageMode}
|
||||
onChange={(event) => setPageMode(event.target.value as "all" | "ranges")}
|
||||
>
|
||||
<option value="all">Todas las paginas</option>
|
||||
<option value="ranges">Rangos especificos</option>
|
||||
</select>
|
||||
</div>
|
||||
</div>
|
||||
{pageMode === "ranges" && (
|
||||
<div className="mt-4">
|
||||
<label className="text-sm font-medium text-slate-300">Rangos</label>
|
||||
<input
|
||||
className="mt-2 w-full rounded-xl border border-white/10 bg-slate-900 px-3 py-2 text-sm outline-none ring-blue-500/30 focus:ring-4"
|
||||
value={pageRanges}
|
||||
onChange={(event) => setPageRanges(event.target.value)}
|
||||
placeholder="Ej: 1-5, 8, 10-12"
|
||||
/>
|
||||
<p className="mt-2 text-xs text-slate-500">Separa paginas o rangos con comas. Esta seleccion se aplicara a los PDFs pendientes.</p>
|
||||
</div>
|
||||
)}
|
||||
<div className="mt-4 flex flex-wrap gap-2">
|
||||
<Button onClick={confirmPendingUpload} disabled={uploading || (pageMode === "ranges" && !pageRanges.trim())}>
|
||||
{uploading ? "Subiendo..." : "Procesar y subir archivos"}
|
||||
</Button>
|
||||
<p className="self-center text-xs text-slate-400">Los archivos que no sean PDF se subiran junto con esta tanda.</p>
|
||||
</div>
|
||||
</div>
|
||||
)}
|
||||
</Card>
|
||||
|
||||
<Card>
|
||||
<div className="mb-4 flex flex-wrap items-center justify-between gap-3">
|
||||
<h2 className="text-xl font-semibold">Documentos Markdown</h2>
|
||||
<div className="flex flex-wrap gap-2">
|
||||
<a className="rounded-xl border border-white/10 px-3 py-2 text-sm hover:bg-white/10" href={api.exportWeekUrl(subjectId, weekNumber)}>Exportar semana (.md unico)</a>
|
||||
<a className="rounded-xl border border-white/10 px-3 py-2 text-sm hover:bg-white/10" href={api.exportWeekZipUrl(subjectId, weekNumber)}>Exportar documentos separados (.zip)</a>
|
||||
</div>
|
||||
</div>
|
||||
<div className="space-y-3">
|
||||
{documents.map((doc) => (
|
||||
<div key={doc.id} className="flex flex-wrap items-center justify-between gap-3 rounded-xl border border-white/10 p-4 hover:bg-white/5">
|
||||
<Link href={`/documents/${doc.id}`} className="min-w-0 flex-1">
|
||||
<div className="font-medium">{doc.title}</div>
|
||||
<div className="mt-1 text-xs text-slate-400">Fuente: {doc.source_original_name || `#${doc.source_id}`}</div>
|
||||
<div className="mt-1 truncate text-xs text-slate-500">{doc.markdown_path}</div>
|
||||
</Link>
|
||||
<Button
|
||||
className="bg-red-600 px-3 py-1.5 text-sm hover:bg-red-500"
|
||||
disabled={deletingId === `document-${doc.id}`}
|
||||
onClick={() => setPendingDelete({ type: "document", item: doc })}
|
||||
>
|
||||
{deletingId === `document-${doc.id}` ? "Eliminando..." : "Eliminar"}
|
||||
</Button>
|
||||
</div>
|
||||
))}
|
||||
{documents.length === 0 && <p className="text-slate-400">Aun no hay documentos procesados.</p>}
|
||||
</div>
|
||||
</Card>
|
||||
</section>
|
||||
|
||||
<aside className="space-y-4">
|
||||
<Card>
|
||||
<h2 className="text-lg font-semibold">Trabajos</h2>
|
||||
<div className="mt-4 space-y-3">
|
||||
{jobs.map((job) => <div key={job.id} className="rounded-xl bg-slate-900 p-3 text-sm"><div className="flex justify-between"><span>#{job.id}</span><StatusBadge status={job.status} /></div><p className="mt-2 text-slate-400">{job.message}</p></div>)}
|
||||
{jobs.length === 0 && <p className="text-sm text-slate-400">Sin trabajos.</p>}
|
||||
</div>
|
||||
</Card>
|
||||
<Card>
|
||||
<h2 className="text-lg font-semibold">Fuentes</h2>
|
||||
<div className="mt-4 space-y-3">
|
||||
{sources.map((source) => {
|
||||
const busy = ["queued", "processing"].includes(source.status);
|
||||
return (
|
||||
<div key={source.id} className="rounded-xl bg-slate-900 p-3 text-sm">
|
||||
<div className="flex items-start justify-between gap-2"><span className="truncate">{source.original_name}</span><StatusBadge status={source.status} /></div>
|
||||
<div className="mt-2 space-y-1 text-xs text-slate-500">
|
||||
<p className={source.document_id ? "text-emerald-300" : source.status === "failed" ? "text-red-300" : "text-amber-300"}>{sourceProcessingLabel(source)}</p>
|
||||
<p>{source.source_type} · {formatFileSize(source.file_size)}{source.processor ? ` · ${source.processor}` : ""}</p>
|
||||
{source.sha256 && <p className="truncate">sha256: {source.sha256.slice(0, 12)}...</p>}
|
||||
{source.page_ranges && <p>paginas: {source.page_ranges}</p>}
|
||||
</div>
|
||||
<div className="mt-3 flex flex-wrap justify-end gap-2">
|
||||
{source.document_id && <Link className="rounded-lg border border-emerald-500/30 px-2 py-1 text-xs text-emerald-200 hover:bg-emerald-500/10" href={`/documents/${source.document_id}`}>Ver Markdown</Link>}
|
||||
<button className="rounded-lg border border-white/10 px-2 py-1 text-xs text-slate-200 hover:bg-white/10 disabled:cursor-not-allowed disabled:opacity-50" disabled={busy || reprocessingId === source.id} onClick={() => reprocessSource(source)}>{reprocessingId === source.id ? "En cola..." : "Reprocesar"}</button>
|
||||
<button className="rounded-lg border border-red-500/30 px-2 py-1 text-xs text-red-200 hover:bg-red-500/10 disabled:cursor-not-allowed disabled:opacity-50" disabled={deletingId === `source-${source.id}` || busy} onClick={() => setPendingDelete({ type: "source", item: source })}>{deletingId === `source-${source.id}` ? "Eliminando..." : "Eliminar"}</button>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
})}
|
||||
{sources.length === 0 && <p className="text-sm text-slate-400">Sin fuentes.</p>}
|
||||
</div>
|
||||
</Card>
|
||||
</aside>
|
||||
</div>
|
||||
<ConfirmDialog
|
||||
open={Boolean(deleteDialog)}
|
||||
title={deleteDialog?.title || "Confirmar accion"}
|
||||
description={deleteDialog?.description || ""}
|
||||
confirmLabel="Eliminar"
|
||||
loading={Boolean(deletingId)}
|
||||
onConfirm={confirmPendingDelete}
|
||||
onCancel={() => setPendingDelete(null)}
|
||||
/>
|
||||
</Shell>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,25 @@
|
||||
import Link from "next/link";
|
||||
import type { ReactNode } from "react";
|
||||
|
||||
export function Shell({ children }: { children: ReactNode }) {
|
||||
return (
|
||||
<div className="min-h-screen bg-slate-950 text-slate-100">
|
||||
<header className="border-b border-white/10 bg-slate-950/90 backdrop-blur">
|
||||
<div className="mx-auto flex max-w-7xl items-center justify-between px-5 py-4">
|
||||
<Link href="/" className="flex items-center gap-3">
|
||||
<div className="grid h-10 w-10 place-items-center rounded-2xl bg-blue-600 font-bold">KS</div>
|
||||
<div>
|
||||
<div className="font-semibold tracking-tight">Knowledge Station</div>
|
||||
<div className="text-xs text-slate-400">Clases a Markdown para LLM</div>
|
||||
</div>
|
||||
</Link>
|
||||
<nav className="flex items-center gap-2 text-sm text-slate-300">
|
||||
<Link className="rounded-lg px-3 py-2 hover:bg-white/10" href="/">Materias</Link>
|
||||
<Link className="rounded-lg px-3 py-2 hover:bg-white/10" href="/settings">Configuracion</Link>
|
||||
</nav>
|
||||
</div>
|
||||
</header>
|
||||
<main className="mx-auto max-w-7xl px-5 py-8">{children}</main>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,67 @@
|
||||
export function Card({ children, className = "" }: { children: React.ReactNode; className?: string }) {
|
||||
return <div className={`rounded-2xl border border-white/10 bg-white/[0.04] p-5 shadow-xl shadow-black/20 ${className}`}>{children}</div>;
|
||||
}
|
||||
|
||||
export function Button({ children, className = "", ...props }: React.ButtonHTMLAttributes<HTMLButtonElement>) {
|
||||
return <button className={`rounded-xl bg-blue-600 px-4 py-2 font-medium text-white hover:bg-blue-500 disabled:cursor-not-allowed disabled:opacity-50 ${className}`} {...props}>{children}</button>;
|
||||
}
|
||||
|
||||
export function Input(props: React.InputHTMLAttributes<HTMLInputElement>) {
|
||||
return <input className="w-full rounded-xl border border-white/10 bg-slate-900 px-4 py-2 text-slate-100 outline-none ring-blue-500/30 focus:ring-4" {...props} />;
|
||||
}
|
||||
|
||||
export function Label({ children }: { children: React.ReactNode }) {
|
||||
return <label className="text-sm font-medium text-slate-300">{children}</label>;
|
||||
}
|
||||
|
||||
export function StatusBadge({ status }: { status: string }) {
|
||||
const map: Record<string, string> = {
|
||||
completed: "bg-emerald-500/15 text-emerald-300",
|
||||
processing: "bg-blue-500/15 text-blue-300",
|
||||
queued: "bg-amber-500/15 text-amber-300",
|
||||
failed: "bg-red-500/15 text-red-300",
|
||||
stored: "bg-slate-500/15 text-slate-300",
|
||||
processed: "bg-emerald-500/15 text-emerald-300",
|
||||
};
|
||||
return <span className={`rounded-full px-2.5 py-1 text-xs font-medium ${map[status] || "bg-slate-500/15 text-slate-300"}`}>{status}</span>;
|
||||
}
|
||||
|
||||
export function ConfirmDialog({
|
||||
open,
|
||||
title,
|
||||
description,
|
||||
confirmLabel = "Confirmar",
|
||||
cancelLabel = "Cancelar",
|
||||
loading = false,
|
||||
onConfirm,
|
||||
onCancel,
|
||||
}: {
|
||||
open: boolean;
|
||||
title: string;
|
||||
description: string;
|
||||
confirmLabel?: string;
|
||||
cancelLabel?: string;
|
||||
loading?: boolean;
|
||||
onConfirm: () => void;
|
||||
onCancel: () => void;
|
||||
}) {
|
||||
if (!open) return null;
|
||||
|
||||
return (
|
||||
<div className="fixed inset-0 z-50 grid place-items-center bg-slate-950/75 p-4 backdrop-blur-sm" role="dialog" aria-modal="true" aria-labelledby="confirm-dialog-title">
|
||||
<div className="w-full max-w-md rounded-2xl border border-white/10 bg-slate-900 p-5 shadow-2xl shadow-black/40">
|
||||
<div className="flex items-start gap-3">
|
||||
<div className="grid h-10 w-10 shrink-0 place-items-center rounded-full bg-red-500/15 text-red-200">!</div>
|
||||
<div>
|
||||
<h2 id="confirm-dialog-title" className="text-lg font-semibold text-white">{title}</h2>
|
||||
<p className="mt-2 text-sm leading-6 text-slate-300">{description}</p>
|
||||
</div>
|
||||
</div>
|
||||
<div className="mt-6 flex flex-col-reverse gap-2 sm:flex-row sm:justify-end">
|
||||
<button className="rounded-xl border border-white/10 px-4 py-2 text-slate-200 hover:bg-white/10 disabled:cursor-not-allowed disabled:opacity-50" type="button" disabled={loading} onClick={onCancel}>{cancelLabel}</button>
|
||||
<Button className="bg-red-600 hover:bg-red-500" disabled={loading} onClick={onConfirm}>{loading ? "Eliminando..." : confirmLabel}</Button>
|
||||
</div>
|
||||
</div>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
@@ -0,0 +1,141 @@
|
||||
export const API_URL = process.env.NEXT_PUBLIC_API_URL ?? "http://127.0.0.1:8000";
|
||||
|
||||
export type Subject = { id: number; name: string; slug: string; created_at: string };
|
||||
export type Week = { id: number; subject_id: number; number: number; title?: string | null; created_at: string };
|
||||
export type Source = {
|
||||
id: number;
|
||||
week_id: number;
|
||||
original_name: string;
|
||||
stored_path: string;
|
||||
source_type: string;
|
||||
status: string;
|
||||
file_size?: number | null;
|
||||
sha256?: string | null;
|
||||
mime_type?: string | null;
|
||||
processor?: string | null;
|
||||
page_ranges?: string | null;
|
||||
processed_at?: string | null;
|
||||
document_id?: number | null;
|
||||
document_title?: string | null;
|
||||
document_markdown_path?: string | null;
|
||||
document_created_at?: string | null;
|
||||
created_at: string;
|
||||
};
|
||||
export type Job = { id: number; status: string; message: string; source_id?: number | null; document_id?: number | null; created_at: string; updated_at: string };
|
||||
export type DocumentItem = {
|
||||
id: number;
|
||||
source_id: number;
|
||||
title: string;
|
||||
markdown_path: string;
|
||||
source_original_name?: string | null;
|
||||
source_type?: string | null;
|
||||
source_status?: string | null;
|
||||
source_processor?: string | null;
|
||||
source_page_ranges?: string | null;
|
||||
created_at: string;
|
||||
};
|
||||
export type SettingsStatus = { mistral_configured: boolean; deepgram_configured: boolean; data_dir: string; database_path: string };
|
||||
|
||||
async function parseResponse<T>(response: Response): Promise<T> {
|
||||
if (!response.ok) {
|
||||
const detail = await response.json().catch(() => null);
|
||||
throw new Error(detail?.detail || `Error ${response.status}`);
|
||||
}
|
||||
return response.json() as Promise<T>;
|
||||
}
|
||||
|
||||
export const api = {
|
||||
async health() {
|
||||
return parseResponse<{ status: string }>(await fetch(`${API_URL}/health`, { cache: "no-store" }));
|
||||
},
|
||||
async settings() {
|
||||
return parseResponse<SettingsStatus>(await fetch(`${API_URL}/settings/status`, { cache: "no-store" }));
|
||||
},
|
||||
async subjects() {
|
||||
return parseResponse<Subject[]>(await fetch(`${API_URL}/subjects`, { cache: "no-store" }));
|
||||
},
|
||||
async subject(id: number) {
|
||||
return parseResponse<Subject>(await fetch(`${API_URL}/subjects/${id}`, { cache: "no-store" }));
|
||||
},
|
||||
async createSubject(name: string) {
|
||||
return parseResponse<Subject>(await fetch(`${API_URL}/subjects`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ name }),
|
||||
}));
|
||||
},
|
||||
async weeks(subjectId: number) {
|
||||
return parseResponse<Week[]>(await fetch(`${API_URL}/subjects/${subjectId}/weeks`, { cache: "no-store" }));
|
||||
},
|
||||
async week(subjectId: number, weekNumber: number) {
|
||||
return parseResponse<Week>(await fetch(`${API_URL}/subjects/${subjectId}/weeks/${weekNumber}`, { cache: "no-store" }));
|
||||
},
|
||||
async createWeek(subjectId: number, number: number, title: string) {
|
||||
return parseResponse<Week>(await fetch(`${API_URL}/subjects/${subjectId}/weeks`, {
|
||||
method: "POST",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ number, title: title || null }),
|
||||
}));
|
||||
},
|
||||
async updateWeek(subjectId: number, currentNumber: number, number: number, title: string) {
|
||||
return parseResponse<Week>(await fetch(`${API_URL}/subjects/${subjectId}/weeks/${currentNumber}`, {
|
||||
method: "PUT",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ number, title: title || null }),
|
||||
}));
|
||||
},
|
||||
async sources(subjectId: number, weekNumber: number) {
|
||||
return parseResponse<Source[]>(await fetch(`${API_URL}/subjects/${subjectId}/weeks/${weekNumber}/sources`, { cache: "no-store" }));
|
||||
},
|
||||
async jobs(subjectId: number, weekNumber: number) {
|
||||
return parseResponse<Job[]>(await fetch(`${API_URL}/subjects/${subjectId}/weeks/${weekNumber}/jobs`, { cache: "no-store" }));
|
||||
},
|
||||
async documents(subjectId: number, weekNumber: number) {
|
||||
return parseResponse<DocumentItem[]>(await fetch(`${API_URL}/subjects/${subjectId}/weeks/${weekNumber}/documents`, { cache: "no-store" }));
|
||||
},
|
||||
async deleteSource(sourceId: number) {
|
||||
return parseResponse<{ status: string }>(await fetch(`${API_URL}/sources/${sourceId}`, { method: "DELETE" }));
|
||||
},
|
||||
async deleteDocument(documentId: number) {
|
||||
return parseResponse<{ status: string }>(await fetch(`${API_URL}/documents/${documentId}`, { method: "DELETE" }));
|
||||
},
|
||||
async reprocessSource(sourceId: number, useOcr: boolean, pageRanges?: string) {
|
||||
const params = new URLSearchParams({ use_ocr: String(useOcr) });
|
||||
if (pageRanges?.trim()) params.set("page_ranges", pageRanges.trim());
|
||||
return parseResponse<{ source: Source; job: Job }>(await fetch(`${API_URL}/sources/${sourceId}/reprocess?${params.toString()}`, { method: "POST" }));
|
||||
},
|
||||
async upload(subjectId: number, weekNumber: number, file: File, useOcr: boolean, pageRanges?: string) {
|
||||
const form = new FormData();
|
||||
form.append("file", file);
|
||||
const params = new URLSearchParams({ use_ocr: String(useOcr) });
|
||||
if (pageRanges?.trim()) params.set("page_ranges", pageRanges.trim());
|
||||
return parseResponse<{ source: Source; job: Job }>(await fetch(`${API_URL}/subjects/${subjectId}/weeks/${weekNumber}/files?${params.toString()}`, {
|
||||
method: "POST",
|
||||
body: form,
|
||||
}));
|
||||
},
|
||||
async markdown(documentId: number) {
|
||||
const response = await fetch(`${API_URL}/documents/${documentId}/markdown`, { cache: "no-store" });
|
||||
if (!response.ok) throw new Error(`Error ${response.status}`);
|
||||
return response.text();
|
||||
},
|
||||
async document(documentId: number) {
|
||||
return parseResponse<DocumentItem>(await fetch(`${API_URL}/documents/${documentId}`, { cache: "no-store" }));
|
||||
},
|
||||
async saveMarkdown(documentId: number, content: string) {
|
||||
return parseResponse<DocumentItem>(await fetch(`${API_URL}/documents/${documentId}/markdown`, {
|
||||
method: "PUT",
|
||||
headers: { "Content-Type": "application/json" },
|
||||
body: JSON.stringify({ content }),
|
||||
}));
|
||||
},
|
||||
downloadDocumentUrl(documentId: number) {
|
||||
return `${API_URL}/documents/${documentId}/download`;
|
||||
},
|
||||
exportWeekUrl(subjectId: number, weekNumber: number) {
|
||||
return `${API_URL}/subjects/${subjectId}/weeks/${weekNumber}/export`;
|
||||
},
|
||||
exportWeekZipUrl(subjectId: number, weekNumber: number) {
|
||||
return `${API_URL}/subjects/${subjectId}/weeks/${weekNumber}/export.zip`;
|
||||
},
|
||||
};
|
||||
Vendored
+5
@@ -0,0 +1,5 @@
|
||||
/// <reference types="next" />
|
||||
/// <reference types="next/image-types/global" />
|
||||
|
||||
// NOTE: This file should not be edited
|
||||
// see https://nextjs.org/docs/app/building-your-application/configuring/typescript for more information.
|
||||
@@ -0,0 +1,18 @@
|
||||
/** @type {import('next').NextConfig} */
|
||||
const backendUrl = process.env.BACKEND_INTERNAL_URL || "http://127.0.0.1:8000";
|
||||
|
||||
const nextConfig = {
|
||||
output: "standalone",
|
||||
async rewrites() {
|
||||
return [
|
||||
{ source: "/health", destination: `${backendUrl}/health` },
|
||||
{ source: "/settings/:path*", destination: `${backendUrl}/settings/:path*` },
|
||||
{ source: "/subjects/:path*", destination: `${backendUrl}/subjects/:path*` },
|
||||
{ source: "/sources/:path*", destination: `${backendUrl}/sources/:path*` },
|
||||
{ source: "/documents/:path*", destination: `${backendUrl}/documents/:path*` },
|
||||
{ source: "/jobs/:path*", destination: `${backendUrl}/jobs/:path*` },
|
||||
];
|
||||
},
|
||||
};
|
||||
|
||||
export default nextConfig;
|
||||
Generated
+3106
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,28 @@
|
||||
{
|
||||
"name": "knowledge-station-frontend",
|
||||
"version": "0.1.0",
|
||||
"private": true,
|
||||
"scripts": {
|
||||
"dev": "next dev",
|
||||
"build": "next build",
|
||||
"start": "next start",
|
||||
"lint": "next lint"
|
||||
},
|
||||
"dependencies": {
|
||||
"@tailwindcss/typography": "^0.5.15",
|
||||
"next": "^14.2.23",
|
||||
"react": "^18.3.1",
|
||||
"react-dom": "^18.3.1",
|
||||
"react-markdown": "^9.0.3",
|
||||
"remark-gfm": "^4.0.0"
|
||||
},
|
||||
"devDependencies": {
|
||||
"@types/node": "^22.10.10",
|
||||
"@types/react": "^18.3.18",
|
||||
"@types/react-dom": "^18.3.5",
|
||||
"autoprefixer": "^10.4.20",
|
||||
"postcss": "^8.5.1",
|
||||
"tailwindcss": "^3.4.17",
|
||||
"typescript": "^5.7.3"
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,6 @@
|
||||
module.exports = {
|
||||
plugins: {
|
||||
tailwindcss: {},
|
||||
autoprefixer: {},
|
||||
},
|
||||
};
|
||||
@@ -0,0 +1,18 @@
|
||||
import type { Config } from "tailwindcss";
|
||||
|
||||
const config: Config = {
|
||||
content: ["./app/**/*.{ts,tsx}", "./components/**/*.{ts,tsx}", "./lib/**/*.{ts,tsx}"],
|
||||
darkMode: "class",
|
||||
theme: {
|
||||
extend: {
|
||||
colors: {
|
||||
ink: "#0f172a",
|
||||
panel: "#111827",
|
||||
brand: "#2563eb",
|
||||
},
|
||||
},
|
||||
},
|
||||
plugins: [require("@tailwindcss/typography")],
|
||||
};
|
||||
|
||||
export default config;
|
||||
@@ -0,0 +1,22 @@
|
||||
{
|
||||
"compilerOptions": {
|
||||
"target": "ES2017",
|
||||
"lib": ["dom", "dom.iterable", "esnext"],
|
||||
"allowJs": false,
|
||||
"skipLibCheck": true,
|
||||
"strict": true,
|
||||
"noEmit": true,
|
||||
"esModuleInterop": true,
|
||||
"module": "esnext",
|
||||
"moduleResolution": "bundler",
|
||||
"baseUrl": ".",
|
||||
"resolveJsonModule": true,
|
||||
"isolatedModules": true,
|
||||
"jsx": "preserve",
|
||||
"incremental": true,
|
||||
"plugins": [{ "name": "next" }],
|
||||
"paths": { "@/*": ["./*"] }
|
||||
},
|
||||
"include": ["next-env.d.ts", "**/*.ts", "**/*.tsx", ".next/types/**/*.ts"],
|
||||
"exclude": ["node_modules"]
|
||||
}
|
||||
@@ -0,0 +1,19 @@
|
||||
[variables]
|
||||
NODE_ENV = "production"
|
||||
NEXT_PUBLIC_API_URL = ""
|
||||
PYTHONUNBUFFERED = "1"
|
||||
|
||||
[phases.setup]
|
||||
nixPkgs = ["python312", "nodejs_22", "ffmpeg"]
|
||||
|
||||
[phases.install]
|
||||
cmds = [
|
||||
"pip install --no-cache-dir -r requirements.txt",
|
||||
"cd frontend && npm ci"
|
||||
]
|
||||
|
||||
[phases.build]
|
||||
cmds = ["cd frontend && npm run build"]
|
||||
|
||||
[start]
|
||||
cmd = "bash scripts/start-coolify.sh"
|
||||
@@ -0,0 +1,8 @@
|
||||
fastapi==0.115.6
|
||||
uvicorn[standard]==0.34.0
|
||||
python-multipart==0.0.20
|
||||
pydantic-settings==2.7.1
|
||||
python-dotenv==1.0.1
|
||||
requests==2.32.3
|
||||
python-docx==1.1.2
|
||||
PyMuPDF==1.25.1
|
||||
@@ -0,0 +1,28 @@
|
||||
#!/usr/bin/env bash
|
||||
set -euo pipefail
|
||||
|
||||
export APP_DATA_DIR="${APP_DATA_DIR:-/app/data}"
|
||||
export HOSTNAME="0.0.0.0"
|
||||
|
||||
uvicorn backend.app.main:app --host 127.0.0.1 --port 8000 &
|
||||
backend_pid=$!
|
||||
|
||||
cleanup() {
|
||||
kill "$backend_pid" 2>/dev/null || true
|
||||
}
|
||||
trap cleanup EXIT INT TERM
|
||||
|
||||
cd frontend
|
||||
|
||||
if [ -f ".next/standalone/server.js" ]; then
|
||||
mkdir -p .next/standalone/.next
|
||||
rm -rf .next/standalone/.next/static .next/standalone/public
|
||||
cp -r .next/static .next/standalone/.next/static
|
||||
if [ -d "public" ]; then
|
||||
cp -r public .next/standalone/public
|
||||
fi
|
||||
cd .next/standalone
|
||||
PORT="${PORT:-3000}" node server.js
|
||||
else
|
||||
npm run start -- --hostname 0.0.0.0 --port "${PORT:-3000}"
|
||||
fi
|
||||
@@ -0,0 +1,167 @@
|
||||
param(
|
||||
[Parameter(Mandatory = $true)]
|
||||
[string]$Root
|
||||
)
|
||||
|
||||
$ErrorActionPreference = "Stop"
|
||||
$Root = $Root.Trim().Trim('"').TrimEnd("\")
|
||||
$Root = (Resolve-Path -LiteralPath $Root).Path
|
||||
$Venv = Join-Path $Root ".venv"
|
||||
$Python = Join-Path $Venv "Scripts\python.exe"
|
||||
$Pip = Join-Path $Venv "Scripts\pip.exe"
|
||||
$Frontend = Join-Path $Root "frontend"
|
||||
$Runtime = Join-Path $Root ".runtime"
|
||||
|
||||
function Write-Step($Message) {
|
||||
Write-Host ""
|
||||
Write-Host $Message -ForegroundColor Cyan
|
||||
}
|
||||
|
||||
function Test-Port([int]$Port) {
|
||||
$client = [Net.Sockets.TcpClient]::new()
|
||||
try {
|
||||
$client.Connect("127.0.0.1", $Port)
|
||||
return $true
|
||||
}
|
||||
catch {
|
||||
return $false
|
||||
}
|
||||
finally {
|
||||
$client.Close()
|
||||
}
|
||||
}
|
||||
|
||||
function Find-FreePort([int]$StartPort) {
|
||||
$port = $StartPort
|
||||
while (Test-Port $port) { $port++ }
|
||||
return $port
|
||||
}
|
||||
|
||||
function Get-LivePid([string]$PidFile) {
|
||||
if (!(Test-Path $PidFile)) { return $null }
|
||||
$raw = (Get-Content $PidFile -ErrorAction SilentlyContinue | Select-Object -First 1)
|
||||
if (!$raw) { return $null }
|
||||
$process = Get-Process -Id ([int]$raw) -ErrorAction SilentlyContinue
|
||||
if ($process) { return [int]$raw }
|
||||
Remove-Item $PidFile -Force -ErrorAction SilentlyContinue
|
||||
return $null
|
||||
}
|
||||
|
||||
function Stop-Tree([int]$Id) {
|
||||
Get-CimInstance Win32_Process | Where-Object { $_.ParentProcessId -eq $Id } | ForEach-Object { Stop-Tree $_.ProcessId }
|
||||
Stop-Process -Id $Id -Force -ErrorAction SilentlyContinue
|
||||
}
|
||||
|
||||
function Invoke-Checked([scriptblock]$Command, [string]$ErrorMessage) {
|
||||
& $Command
|
||||
if ($LASTEXITCODE -ne 0) { throw $ErrorMessage }
|
||||
}
|
||||
|
||||
Write-Host ""
|
||||
Write-Host "============================================"
|
||||
Write-Host " Knowledge Station - Plataforma completa"
|
||||
Write-Host "============================================"
|
||||
|
||||
New-Item -ItemType Directory -Force -Path $Runtime | Out-Null
|
||||
|
||||
$legacyPid = Join-Path $Runtime "server.pid"
|
||||
if (Test-Path $legacyPid) {
|
||||
$legacyRaw = Get-Content $legacyPid -ErrorAction SilentlyContinue | Select-Object -First 1
|
||||
if ($legacyRaw) { Stop-Tree ([int]$legacyRaw) }
|
||||
Remove-Item $legacyPid -Force -ErrorAction SilentlyContinue
|
||||
}
|
||||
|
||||
if (!(Test-Path $Python)) {
|
||||
Write-Step "Preparando Python por primera vez..."
|
||||
& py -m venv $Venv 2>$null
|
||||
if ($LASTEXITCODE -ne 0) {
|
||||
& python -m venv $Venv
|
||||
}
|
||||
if (!(Test-Path $Python)) {
|
||||
throw "No se pudo preparar Python. Instala Python 3.12 o superior."
|
||||
}
|
||||
}
|
||||
|
||||
$EnvFile = Join-Path $Root ".env"
|
||||
$EnvExample = Join-Path $Root ".env.example"
|
||||
if (!(Test-Path $EnvFile)) {
|
||||
Write-Step "Creando configuracion local .env..."
|
||||
Copy-Item $EnvExample $EnvFile
|
||||
Write-Host "Agrega tus API keys en .env para audio, video u OCR." -ForegroundColor Yellow
|
||||
}
|
||||
|
||||
Write-Step "Verificando dependencias de Python..."
|
||||
Invoke-Checked { & $Python -m pip install --upgrade pip --disable-pip-version-check | Out-Null } "No se pudo actualizar pip."
|
||||
Invoke-Checked { & $Pip install -r (Join-Path $Root "requirements.txt") } "No se pudieron instalar dependencias de Python."
|
||||
|
||||
Write-Step "Verificando Node.js..."
|
||||
$node = Get-Command node -ErrorAction SilentlyContinue
|
||||
$npm = Get-Command npm.cmd -ErrorAction SilentlyContinue
|
||||
if (!$npm) { $npm = Get-Command npm -ErrorAction SilentlyContinue }
|
||||
if (!$node -or !$npm) {
|
||||
throw "Node.js LTS no esta instalado o npm no esta disponible. Instala Node.js desde https://nodejs.org/."
|
||||
}
|
||||
|
||||
if (!(Test-Path (Join-Path $Frontend "node_modules"))) {
|
||||
Write-Step "Instalando dependencias del frontend. Esto puede tardar unos minutos..."
|
||||
Invoke-Checked { & $npm.Source install --prefix $Frontend } "No se pudieron instalar dependencias del frontend."
|
||||
}
|
||||
else {
|
||||
Write-Host "Dependencias del frontend encontradas."
|
||||
}
|
||||
|
||||
$BackendPidFile = Join-Path $Runtime "backend.pid"
|
||||
$FrontendPidFile = Join-Path $Runtime "frontend.pid"
|
||||
$BackendPortFile = Join-Path $Runtime "backend.port"
|
||||
$FrontendPortFile = Join-Path $Runtime "frontend.port"
|
||||
|
||||
Write-Step "Iniciando backend..."
|
||||
$backendPid = Get-LivePid $BackendPidFile
|
||||
if ($backendPid) {
|
||||
$backendPort = Get-Content $BackendPortFile -ErrorAction SilentlyContinue | Select-Object -First 1
|
||||
if (!$backendPort) { $backendPort = 8000 }
|
||||
Write-Host "Backend ya estaba activo en puerto $backendPort."
|
||||
}
|
||||
else {
|
||||
$backendPort = Find-FreePort 8000
|
||||
$backend = Start-Process -FilePath $Python -ArgumentList @("-m", "uvicorn", "backend.app.main:app", "--host", "127.0.0.1", "--port", [string]$backendPort, "--reload") -WorkingDirectory $Root -PassThru
|
||||
Set-Content $BackendPidFile $backend.Id
|
||||
Set-Content $BackendPortFile $backendPort
|
||||
Write-Host "Backend iniciado en puerto $backendPort."
|
||||
}
|
||||
|
||||
$ApiUrl = "http://127.0.0.1:$backendPort"
|
||||
Set-Content (Join-Path $Frontend ".env.local") "NEXT_PUBLIC_API_URL=$ApiUrl"
|
||||
|
||||
Write-Step "Iniciando frontend..."
|
||||
$frontendPid = Get-LivePid $FrontendPidFile
|
||||
if ($frontendPid) {
|
||||
$frontendPort = Get-Content $FrontendPortFile -ErrorAction SilentlyContinue | Select-Object -First 1
|
||||
if (!$frontendPort) { $frontendPort = 3000 }
|
||||
Write-Host "Frontend ya estaba activo en puerto $frontendPort."
|
||||
}
|
||||
else {
|
||||
$frontendPort = Find-FreePort 3000
|
||||
$nextDir = Join-Path $Frontend ".next"
|
||||
if (Test-Path $nextDir) {
|
||||
Get-ChildItem $nextDir -Recurse -Force -ErrorAction SilentlyContinue | ForEach-Object {
|
||||
try { $_.Attributes = "Normal" } catch {}
|
||||
}
|
||||
Remove-Item $nextDir -Recurse -Force -ErrorAction SilentlyContinue
|
||||
}
|
||||
$frontend = Start-Process -FilePath $npm.Source -ArgumentList @("run", "dev", "--", "-p", [string]$frontendPort) -WorkingDirectory $Frontend -PassThru
|
||||
Set-Content $FrontendPidFile $frontend.Id
|
||||
Set-Content $FrontendPortFile $frontendPort
|
||||
Write-Host "Frontend iniciado en puerto $frontendPort."
|
||||
}
|
||||
|
||||
$AppUrl = "http://127.0.0.1:$frontendPort"
|
||||
Set-Content (Join-Path $Runtime "last.url") $AppUrl
|
||||
|
||||
Start-Sleep -Seconds 4
|
||||
Write-Host ""
|
||||
Write-Host "Plataforma abierta:" -ForegroundColor Green
|
||||
Write-Host "Frontend: $AppUrl"
|
||||
Write-Host "API: $ApiUrl"
|
||||
Write-Host ""
|
||||
Start-Process $AppUrl
|
||||
@@ -0,0 +1,47 @@
|
||||
param(
|
||||
[Parameter(Mandatory = $true)]
|
||||
[string]$Root
|
||||
)
|
||||
|
||||
$ErrorActionPreference = "SilentlyContinue"
|
||||
$Root = $Root.Trim().Trim('"').TrimEnd("\")
|
||||
$Root = (Resolve-Path -LiteralPath $Root).Path.TrimEnd("\")
|
||||
$Runtime = Join-Path $Root ".runtime"
|
||||
|
||||
function Stop-Tree([int]$Id) {
|
||||
Get-CimInstance Win32_Process | Where-Object { $_.ParentProcessId -eq $Id } | ForEach-Object { Stop-Tree $_.ProcessId }
|
||||
Stop-Process -Id $Id -Force
|
||||
}
|
||||
|
||||
Write-Host ""
|
||||
Write-Host "============================================"
|
||||
Write-Host " Knowledge Station - Cerrar plataforma"
|
||||
Write-Host "============================================"
|
||||
Write-Host ""
|
||||
|
||||
if (!(Test-Path $Runtime)) {
|
||||
Write-Host "No hay registro de servicios abiertos."
|
||||
exit 0
|
||||
}
|
||||
|
||||
foreach ($name in @("backend.pid", "frontend.pid", "server.pid")) {
|
||||
$file = Join-Path $Runtime $name
|
||||
if (Test-Path $file) {
|
||||
$raw = Get-Content $file | Select-Object -First 1
|
||||
if ($raw) { Stop-Tree ([int]$raw) }
|
||||
}
|
||||
}
|
||||
|
||||
Get-CimInstance Win32_Process |
|
||||
Where-Object {
|
||||
$_.CommandLine -and
|
||||
$_.CommandLine -like "*$Root*" -and
|
||||
($_.CommandLine -like "*uvicorn*backend.app.main:app*" -or $_.CommandLine -like "*next*dev*")
|
||||
} |
|
||||
ForEach-Object { Stop-Tree $_.ProcessId }
|
||||
|
||||
foreach ($name in @("backend.pid", "frontend.pid", "server.pid", "backend.port", "frontend.port", "last.url")) {
|
||||
Remove-Item (Join-Path $Runtime $name) -Force
|
||||
}
|
||||
|
||||
Write-Host "Plataforma cerrada de forma segura." -ForegroundColor Green
|
||||
Reference in New Issue
Block a user