1
This commit is contained in:
@@ -0,0 +1,19 @@
|
|||||||
|
.git
|
||||||
|
.venv
|
||||||
|
__pycache__
|
||||||
|
*.pyc
|
||||||
|
.pytest_cache
|
||||||
|
.mypy_cache
|
||||||
|
.ruff_cache
|
||||||
|
.env
|
||||||
|
.env.*
|
||||||
|
data
|
||||||
|
frontend/.next
|
||||||
|
frontend/node_modules
|
||||||
|
frontend/.env.local
|
||||||
|
frontend/.env.*.local
|
||||||
|
node_modules
|
||||||
|
npm-debug.log*
|
||||||
|
Dockerfile*
|
||||||
|
docker-compose*.yml
|
||||||
|
docker-compose*.yaml
|
||||||
@@ -0,0 +1,5 @@
|
|||||||
|
APP_ENV=development
|
||||||
|
APP_DATA_DIR=./data
|
||||||
|
DATABASE_PATH=./data/app.db
|
||||||
|
MISTRAL_API_KEY=
|
||||||
|
DEEPGRAM_API_KEY=
|
||||||
+14
@@ -0,0 +1,14 @@
|
|||||||
|
.env
|
||||||
|
__pycache__/
|
||||||
|
*.py[cod]
|
||||||
|
.pytest_cache/
|
||||||
|
.venv/
|
||||||
|
venv/
|
||||||
|
node_modules/
|
||||||
|
.next/
|
||||||
|
.runtime/
|
||||||
|
data/
|
||||||
|
tmp/
|
||||||
|
*.db
|
||||||
|
*.sqlite
|
||||||
|
frontend/.env.local
|
||||||
@@ -0,0 +1,32 @@
|
|||||||
|
# AGENTS.md
|
||||||
|
|
||||||
|
## Repo Shape
|
||||||
|
- The primary app is now `backend/` (Python/FastAPI) plus `frontend/` (Next.js/TypeScript/Tailwind); `estacion documentos.html` is a browser-only prototype/reference for behavior.
|
||||||
|
- Runtime data is stored under `data/` and is gitignored; do not commit uploaded class files, generated Markdown, SQLite DBs, temporary audio, or API keys.
|
||||||
|
- The included `Griffin(2020).Comportamiento_organizacionalSOLOCAPITULO7.13Ed.pdf` is a local sample/source PDF for import behavior.
|
||||||
|
|
||||||
|
## Commands
|
||||||
|
- Local setup: `python -m venv .venv`, `.venv\Scripts\activate`, `pip install -r requirements.txt`, copy `.env.example` to `.env`.
|
||||||
|
- Run API: `uvicorn backend.app.main:app --reload` from the repo root.
|
||||||
|
- Run frontend: `cd frontend`, `npm install`, `npm run dev -- -p 3000` with `NEXT_PUBLIC_API_URL=http://127.0.0.1:8000`.
|
||||||
|
- Windows non-technical flow: double-click `abrir_estacion.bat`; it chooses free ports, starts backend/frontend, and opens the browser. Use `cerrar_estacion.bat` to stop only this project.
|
||||||
|
- Docker/Coolify check: `docker compose up --build`.
|
||||||
|
- Basic syntax check: `python -m compileall backend`.
|
||||||
|
|
||||||
|
## Platform Behavior
|
||||||
|
- Knowledge is organized as `Materia -> Semana -> fuentes raw -> Markdown procesado -> exports` under `data/subjects/<materia>/week-XX/`.
|
||||||
|
- `KnowledgeManager` owns filesystem layout and SQLite metadata; processors should not write final Markdown directly.
|
||||||
|
- `IngestionService` selects processors by `source_type` and wraps output with Markdown frontmatter for LLM ingestion.
|
||||||
|
- Text and Markdown are read directly; `.docx` uses `python-docx`; PDFs use PyMuPDF text extraction unless `use_ocr=true` is passed. PDF uploads can also pass `page_ranges` like `1-5,8,10-12`; ranges apply to both standard extraction and Mistral OCR.
|
||||||
|
- OCR uses Mistral `https://api.mistral.ai/v1/chat/completions` with model `pixtral-12b-2409`.
|
||||||
|
- Audio uses Deepgram `https://api.deepgram.com/v1/listen?model=nova-3&smart_format=true&language=es`.
|
||||||
|
- Video processing requires `ffmpeg`; audio is extracted to `data/tmp/`, transcribed with Deepgram, then deleted.
|
||||||
|
- The frontend calls the backend through `NEXT_PUBLIC_API_URL`; when ports change, update `frontend/.env.local` or use the adaptive `.bat`.
|
||||||
|
|
||||||
|
## Secrets And Deployment
|
||||||
|
- Use `.env` locally and Coolify environment variables in production; never hardcode or commit `MISTRAL_API_KEY` or `DEEPGRAM_API_KEY`.
|
||||||
|
- For Coolify, mount persistent storage for `/app/data` or all subjects, outputs, and SQLite state will be lost on redeploy.
|
||||||
|
|
||||||
|
## Prototype Notes
|
||||||
|
- Preserve the Spanish-language UX and Markdown-first output conventions from `estacion documentos.html` when building the new UI.
|
||||||
|
- The prototype sanitizes preview HTML with DOMPurify after `marked.parse`; keep equivalent sanitization in any future web UI.
|
||||||
+18
@@ -0,0 +1,18 @@
|
|||||||
|
FROM python:3.12-slim
|
||||||
|
|
||||||
|
ENV PYTHONDONTWRITEBYTECODE=1
|
||||||
|
ENV PYTHONUNBUFFERED=1
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
RUN apt-get update \
|
||||||
|
&& apt-get install -y --no-install-recommends ffmpeg \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
COPY requirements.txt ./
|
||||||
|
RUN pip install --no-cache-dir -r requirements.txt
|
||||||
|
|
||||||
|
COPY backend ./backend
|
||||||
|
|
||||||
|
EXPOSE 8000
|
||||||
|
CMD ["uvicorn", "backend.app.main:app", "--host", "0.0.0.0", "--port", "8000"]
|
||||||
Binary file not shown.
@@ -0,0 +1,89 @@
|
|||||||
|
# Knowledge Station
|
||||||
|
|
||||||
|
Plataforma para organizar conocimiento escolar por materia y semana, procesar documentos/audio/video y generar archivos Markdown optimizados para ingesta en LLM.
|
||||||
|
|
||||||
|
El archivo `estacion documentos.html` queda como prototipo funcional de referencia. La plataforma principal usa `backend/` (FastAPI) y `frontend/` (Next.js).
|
||||||
|
|
||||||
|
## Capacidades Iniciales
|
||||||
|
|
||||||
|
- Materias y semanas persistidas en SQLite.
|
||||||
|
- Estructura de archivos por materia/semana en `data/subjects/`.
|
||||||
|
- Ingesta de `.txt`, `.md`, `.docx`, `.pdf`, audio y video.
|
||||||
|
- PDF con texto mediante PyMuPDF.
|
||||||
|
- OCR opcional de PDF con Mistral.
|
||||||
|
- Audio con Deepgram en español.
|
||||||
|
- Video con extracción temporal de audio vía `ffmpeg` y transcripción con Deepgram.
|
||||||
|
- Exportación semanal consolidada a Markdown.
|
||||||
|
- UI dedicada con materias, semanas, subida de archivos, polling de trabajos y editor Markdown con preview.
|
||||||
|
|
||||||
|
## Desarrollo Local
|
||||||
|
|
||||||
|
```bash
|
||||||
|
python -m venv .venv
|
||||||
|
.venv\Scripts\activate
|
||||||
|
pip install -r requirements.txt
|
||||||
|
copy .env.example .env
|
||||||
|
uvicorn backend.app.main:app --reload
|
||||||
|
```
|
||||||
|
|
||||||
|
En otra terminal:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
cd frontend
|
||||||
|
npm install
|
||||||
|
set NEXT_PUBLIC_API_URL=http://127.0.0.1:8000
|
||||||
|
npm run dev -- -p 3000
|
||||||
|
```
|
||||||
|
|
||||||
|
La plataforma queda en `http://localhost:3000`, la API en `http://localhost:8000` y la documentación interactiva en `http://localhost:8000/docs`.
|
||||||
|
|
||||||
|
Para usuarios no tecnicos en Windows, usa `abrir_estacion.bat`; detecta puertos libres, prepara backend/frontend y abre el navegador. Usa `cerrar_estacion.bat` para detener todo.
|
||||||
|
|
||||||
|
## Docker / Coolify
|
||||||
|
|
||||||
|
```bash
|
||||||
|
docker compose up --build
|
||||||
|
```
|
||||||
|
|
||||||
|
Para Coolify, configura backend y frontend como servicios separados. Monta un volumen persistente para `/app/data` en el backend y define `NEXT_PUBLIC_API_URL` en el frontend apuntando a la URL publica del backend.
|
||||||
|
|
||||||
|
## Variables
|
||||||
|
|
||||||
|
- `APP_DATA_DIR`: carpeta de datos persistentes.
|
||||||
|
- `DATABASE_PATH`: ruta SQLite.
|
||||||
|
- `MISTRAL_API_KEY`: OCR para PDFs escaneados.
|
||||||
|
- `DEEPGRAM_API_KEY`: transcripción de audio/video.
|
||||||
|
|
||||||
|
No subas claves reales al repositorio.
|
||||||
|
|
||||||
|
## Flujo API Basico
|
||||||
|
|
||||||
|
Crear materia:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST http://localhost:8000/subjects -H "Content-Type: application/json" -d '{"name":"Comportamiento Organizacional"}'
|
||||||
|
```
|
||||||
|
|
||||||
|
Crear semana:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST http://localhost:8000/subjects/1/weeks -H "Content-Type: application/json" -d '{"number":1,"title":"Grupos y equipos"}'
|
||||||
|
```
|
||||||
|
|
||||||
|
Subir archivo:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -X POST -F "[email protected]" http://localhost:8000/subjects/1/weeks/1/files
|
||||||
|
```
|
||||||
|
|
||||||
|
Consultar trabajo:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl http://localhost:8000/jobs/1
|
||||||
|
```
|
||||||
|
|
||||||
|
Exportar semana:
|
||||||
|
|
||||||
|
```bash
|
||||||
|
curl -L http://localhost:8000/subjects/1/weeks/1/export -o semana-01.md
|
||||||
|
```
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
@echo off
|
||||||
|
setlocal
|
||||||
|
title Knowledge Station - Abrir
|
||||||
|
|
||||||
|
set "ROOT=%~dp0"
|
||||||
|
set "ROOT=%ROOT:~0,-1%"
|
||||||
|
powershell -NoProfile -ExecutionPolicy Bypass -File "%ROOT%\scripts\start-platform.ps1" -Root "%ROOT%"
|
||||||
|
if errorlevel 1 (
|
||||||
|
echo.
|
||||||
|
echo No se pudo abrir la plataforma. Revisa el mensaje anterior.
|
||||||
|
pause
|
||||||
|
exit /b 1
|
||||||
|
)
|
||||||
|
|
||||||
|
echo.
|
||||||
|
echo Puedes dejar esta ventana cerrada. Para apagar la plataforma usa cerrar_estacion.bat.
|
||||||
|
pause
|
||||||
|
exit /b 0
|
||||||
@@ -0,0 +1,16 @@
|
|||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from pydantic_settings import BaseSettings, SettingsConfigDict
|
||||||
|
|
||||||
|
|
||||||
|
class Settings(BaseSettings):
|
||||||
|
app_env: str = "development"
|
||||||
|
app_data_dir: Path = Path("./data")
|
||||||
|
database_path: Path = Path("./data/app.db")
|
||||||
|
mistral_api_key: str = ""
|
||||||
|
deepgram_api_key: str = ""
|
||||||
|
|
||||||
|
model_config = SettingsConfigDict(env_file=".env", env_file_encoding="utf-8", extra="ignore")
|
||||||
|
|
||||||
|
|
||||||
|
settings = Settings()
|
||||||
@@ -0,0 +1,281 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from fastapi import BackgroundTasks, FastAPI, File, HTTPException, Query, UploadFile
|
||||||
|
from fastapi.middleware.cors import CORSMiddleware
|
||||||
|
from fastapi.responses import FileResponse, PlainTextResponse
|
||||||
|
|
||||||
|
from backend.app.config import settings
|
||||||
|
from backend.app.managers.knowledge_manager import DuplicateSourceError, KnowledgeManager
|
||||||
|
from backend.app.models.schemas import MarkdownUpdate, SubjectCreate, WeekCreate, WeekUpdate
|
||||||
|
from backend.app.services.ingestion_service import IngestionService
|
||||||
|
|
||||||
|
app = FastAPI(title="Knowledge Station", version="0.1.0")
|
||||||
|
app.add_middleware(
|
||||||
|
CORSMiddleware,
|
||||||
|
allow_origins=["http://127.0.0.1:3000", "http://localhost:3000"],
|
||||||
|
allow_origin_regex=r"http://(127\.0\.0\.1|localhost):\d+",
|
||||||
|
allow_credentials=True,
|
||||||
|
allow_methods=["*"],
|
||||||
|
allow_headers=["*"],
|
||||||
|
)
|
||||||
|
manager = KnowledgeManager()
|
||||||
|
ingestion = IngestionService(manager)
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/health")
|
||||||
|
def health() -> dict[str, str]:
|
||||||
|
return {"status": "ok"}
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/settings/status")
|
||||||
|
def settings_status() -> dict:
|
||||||
|
return {
|
||||||
|
"mistral_configured": bool(settings.mistral_api_key),
|
||||||
|
"deepgram_configured": bool(settings.deepgram_api_key),
|
||||||
|
"data_dir": str(settings.app_data_dir),
|
||||||
|
"database_path": str(settings.database_path),
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/subjects")
|
||||||
|
def create_subject(payload: SubjectCreate) -> dict:
|
||||||
|
try:
|
||||||
|
return manager.create_subject(payload.name)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/subjects")
|
||||||
|
def list_subjects() -> list[dict]:
|
||||||
|
return manager.list_subjects()
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/subjects/{subject_id}")
|
||||||
|
def get_subject(subject_id: int) -> dict:
|
||||||
|
try:
|
||||||
|
return manager.get_subject(subject_id)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@app.delete("/subjects/{subject_id}")
|
||||||
|
def delete_subject(subject_id: int) -> dict:
|
||||||
|
try:
|
||||||
|
manager.delete_subject(subject_id)
|
||||||
|
return {"status": "deleted"}
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/subjects/{subject_id}/weeks")
|
||||||
|
def create_week(subject_id: int, payload: WeekCreate) -> dict:
|
||||||
|
try:
|
||||||
|
return manager.create_week(subject_id, payload.number, payload.title)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/subjects/{subject_id}/weeks")
|
||||||
|
def list_weeks(subject_id: int) -> list[dict]:
|
||||||
|
try:
|
||||||
|
manager.get_subject(subject_id)
|
||||||
|
return manager.list_weeks(subject_id)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/subjects/{subject_id}/weeks/{week_number}")
|
||||||
|
def get_week(subject_id: int, week_number: int) -> dict:
|
||||||
|
try:
|
||||||
|
return manager.get_week(subject_id, week_number)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@app.put("/subjects/{subject_id}/weeks/{week_number}")
|
||||||
|
def update_week(subject_id: int, week_number: int, payload: WeekUpdate) -> dict:
|
||||||
|
try:
|
||||||
|
return manager.update_week(subject_id, week_number, payload.number, payload.title)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@app.delete("/subjects/{subject_id}/weeks/{week_number}")
|
||||||
|
def delete_week(subject_id: int, week_number: int) -> dict:
|
||||||
|
try:
|
||||||
|
manager.delete_week(subject_id, week_number)
|
||||||
|
return {"status": "deleted"}
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/subjects/{subject_id}/weeks/{week_number}/files")
|
||||||
|
def upload_file(
|
||||||
|
subject_id: int,
|
||||||
|
week_number: int,
|
||||||
|
background_tasks: BackgroundTasks,
|
||||||
|
file: UploadFile = File(...),
|
||||||
|
use_ocr: bool = Query(False, description="Usar Mistral OCR para PDFs"),
|
||||||
|
page_ranges: str | None = Query(None, description="Rangos de paginas PDF, por ejemplo: 1-5,8,10-12"),
|
||||||
|
) -> dict:
|
||||||
|
try:
|
||||||
|
source = manager.save_upload(subject_id, week_number, file)
|
||||||
|
job = manager.create_job(source["id"], "Archivo recibido")
|
||||||
|
background_tasks.add_task(process_job, job["id"], source["id"], use_ocr, page_ranges)
|
||||||
|
return {"source": source, "job": job}
|
||||||
|
except DuplicateSourceError as exc:
|
||||||
|
raise HTTPException(status_code=409, detail=str(exc)) from exc
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/jobs/{job_id}")
|
||||||
|
def get_job(job_id: int) -> dict:
|
||||||
|
try:
|
||||||
|
return manager.get_job(job_id)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/jobs/{job_id}/retry")
|
||||||
|
def retry_job(
|
||||||
|
job_id: int,
|
||||||
|
background_tasks: BackgroundTasks,
|
||||||
|
use_ocr: bool = Query(False),
|
||||||
|
page_ranges: str | None = Query(None),
|
||||||
|
) -> dict:
|
||||||
|
try:
|
||||||
|
job = manager.get_job(job_id)
|
||||||
|
if not job.get("source_id"):
|
||||||
|
raise HTTPException(status_code=400, detail="El trabajo no tiene fuente asociada")
|
||||||
|
manager.update_job(job_id, "queued", "Reintento en cola")
|
||||||
|
background_tasks.add_task(process_job, job_id, job["source_id"], use_ocr, page_ranges)
|
||||||
|
return manager.get_job(job_id)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/subjects/{subject_id}/weeks/{week_number}/jobs")
|
||||||
|
def list_week_jobs(subject_id: int, week_number: int) -> list[dict]:
|
||||||
|
try:
|
||||||
|
return manager.list_jobs_for_week(subject_id, week_number)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/subjects/{subject_id}/weeks/{week_number}/sources")
|
||||||
|
def list_sources(subject_id: int, week_number: int) -> list[dict]:
|
||||||
|
try:
|
||||||
|
return manager.list_sources(subject_id, week_number)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@app.delete("/sources/{source_id}")
|
||||||
|
def delete_source(source_id: int) -> dict:
|
||||||
|
try:
|
||||||
|
manager.delete_source(source_id)
|
||||||
|
return {"status": "deleted"}
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@app.post("/sources/{source_id}/reprocess")
|
||||||
|
def reprocess_source(
|
||||||
|
source_id: int,
|
||||||
|
background_tasks: BackgroundTasks,
|
||||||
|
use_ocr: bool = Query(False),
|
||||||
|
page_ranges: str | None = Query(None),
|
||||||
|
) -> dict:
|
||||||
|
try:
|
||||||
|
source = manager.get_source(source_id)
|
||||||
|
if manager.source_has_active_job(source_id):
|
||||||
|
raise HTTPException(status_code=409, detail="La fuente ya tiene un trabajo en cola o en proceso")
|
||||||
|
job = manager.create_job(source["id"], "Reprocesamiento en cola")
|
||||||
|
background_tasks.add_task(process_job, job["id"], source["id"], use_ocr, page_ranges)
|
||||||
|
return {"source": source, "job": job}
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/subjects/{subject_id}/weeks/{week_number}/documents")
|
||||||
|
def list_documents(subject_id: int, week_number: int) -> list[dict]:
|
||||||
|
try:
|
||||||
|
return manager.list_documents(subject_id, week_number)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/documents/{document_id}")
|
||||||
|
def get_document(document_id: int) -> dict:
|
||||||
|
try:
|
||||||
|
return manager.get_document(document_id)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/documents/{document_id}/markdown", response_class=PlainTextResponse)
|
||||||
|
def get_document_markdown(document_id: int) -> str:
|
||||||
|
try:
|
||||||
|
document = manager.get_document(document_id)
|
||||||
|
return Path(document["markdown_path"]).read_text(encoding="utf-8")
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@app.put("/documents/{document_id}/markdown")
|
||||||
|
def update_document_markdown(document_id: int, payload: MarkdownUpdate) -> dict:
|
||||||
|
try:
|
||||||
|
return manager.update_document_markdown(document_id, payload.content)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/documents/{document_id}/download")
|
||||||
|
def download_document(document_id: int) -> FileResponse:
|
||||||
|
try:
|
||||||
|
document = manager.get_document(document_id)
|
||||||
|
path = Path(document["markdown_path"])
|
||||||
|
return FileResponse(path, filename=path.name, media_type="text/markdown")
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@app.delete("/documents/{document_id}")
|
||||||
|
def delete_document(document_id: int) -> dict:
|
||||||
|
try:
|
||||||
|
manager.delete_document(document_id)
|
||||||
|
return {"status": "deleted"}
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/subjects/{subject_id}/weeks/{week_number}/export")
|
||||||
|
def export_week(subject_id: int, week_number: int) -> FileResponse:
|
||||||
|
try:
|
||||||
|
path = manager.export_week_markdown(subject_id, week_number)
|
||||||
|
return FileResponse(path, filename=path.name, media_type="text/markdown")
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@app.get("/subjects/{subject_id}/weeks/{week_number}/export.zip")
|
||||||
|
def export_week_zip(subject_id: int, week_number: int) -> FileResponse:
|
||||||
|
try:
|
||||||
|
path = manager.export_week_markdown_zip(subject_id, week_number)
|
||||||
|
return FileResponse(path, filename=path.name, media_type="application/zip")
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
def process_job(job_id: int, source_id: int, use_ocr: bool, page_ranges: str | None = None) -> None:
|
||||||
|
manager.update_job(job_id, "processing", "Procesando archivo")
|
||||||
|
manager.update_source_status(source_id, "processing")
|
||||||
|
try:
|
||||||
|
document = ingestion.process_source(source_id, use_ocr=use_ocr, page_ranges=page_ranges)
|
||||||
|
manager.update_job(job_id, "completed", "Documento procesado", document_id=document["id"])
|
||||||
|
except Exception as exc:
|
||||||
|
manager.update_source_status(source_id, "failed")
|
||||||
|
manager.update_job(job_id, "failed", str(exc))
|
||||||
@@ -0,0 +1,548 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import shutil
|
||||||
|
import sqlite3
|
||||||
|
import re
|
||||||
|
import hashlib
|
||||||
|
import zipfile
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any, Iterable
|
||||||
|
|
||||||
|
from fastapi import UploadFile
|
||||||
|
|
||||||
|
from backend.app.config import settings
|
||||||
|
from backend.app.utils.slug import slugify
|
||||||
|
|
||||||
|
|
||||||
|
def utc_now() -> str:
|
||||||
|
return datetime.now(timezone.utc).isoformat()
|
||||||
|
|
||||||
|
|
||||||
|
class DuplicateSourceError(ValueError):
|
||||||
|
def __init__(self, existing_source: dict[str, Any]):
|
||||||
|
self.existing_source = existing_source
|
||||||
|
super().__init__(f"El archivo ya existe en esta semana: {existing_source['original_name']}")
|
||||||
|
|
||||||
|
|
||||||
|
class KnowledgeManager:
|
||||||
|
def __init__(self, data_dir: Path | None = None, database_path: Path | None = None):
|
||||||
|
self.data_dir = data_dir or settings.app_data_dir
|
||||||
|
self.database_path = database_path or settings.database_path
|
||||||
|
self.subjects_dir = self.data_dir / "subjects"
|
||||||
|
self.tmp_dir = self.data_dir / "tmp"
|
||||||
|
self.data_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
self.subjects_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
self.tmp_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
self.database_path.parent.mkdir(parents=True, exist_ok=True)
|
||||||
|
self._init_db()
|
||||||
|
|
||||||
|
def _connect(self) -> sqlite3.Connection:
|
||||||
|
conn = sqlite3.connect(self.database_path)
|
||||||
|
conn.row_factory = sqlite3.Row
|
||||||
|
return conn
|
||||||
|
|
||||||
|
def _init_db(self) -> None:
|
||||||
|
with self._connect() as conn:
|
||||||
|
conn.executescript(
|
||||||
|
"""
|
||||||
|
create table if not exists subjects (
|
||||||
|
id integer primary key autoincrement,
|
||||||
|
name text not null,
|
||||||
|
slug text not null unique,
|
||||||
|
created_at text not null
|
||||||
|
);
|
||||||
|
|
||||||
|
create table if not exists weeks (
|
||||||
|
id integer primary key autoincrement,
|
||||||
|
subject_id integer not null,
|
||||||
|
number integer not null,
|
||||||
|
title text,
|
||||||
|
created_at text not null,
|
||||||
|
unique(subject_id, number),
|
||||||
|
foreign key(subject_id) references subjects(id)
|
||||||
|
);
|
||||||
|
|
||||||
|
create table if not exists sources (
|
||||||
|
id integer primary key autoincrement,
|
||||||
|
week_id integer not null,
|
||||||
|
original_name text not null,
|
||||||
|
stored_path text not null,
|
||||||
|
source_type text not null,
|
||||||
|
status text not null,
|
||||||
|
file_size integer,
|
||||||
|
sha256 text,
|
||||||
|
mime_type text,
|
||||||
|
processor text,
|
||||||
|
page_ranges text,
|
||||||
|
processed_at text,
|
||||||
|
created_at text not null,
|
||||||
|
foreign key(week_id) references weeks(id)
|
||||||
|
);
|
||||||
|
|
||||||
|
create table if not exists documents (
|
||||||
|
id integer primary key autoincrement,
|
||||||
|
source_id integer not null,
|
||||||
|
title text not null,
|
||||||
|
markdown_path text not null,
|
||||||
|
created_at text not null,
|
||||||
|
foreign key(source_id) references sources(id)
|
||||||
|
);
|
||||||
|
|
||||||
|
create table if not exists jobs (
|
||||||
|
id integer primary key autoincrement,
|
||||||
|
status text not null,
|
||||||
|
message text not null,
|
||||||
|
source_id integer,
|
||||||
|
document_id integer,
|
||||||
|
created_at text not null,
|
||||||
|
updated_at text not null,
|
||||||
|
foreign key(source_id) references sources(id),
|
||||||
|
foreign key(document_id) references documents(id)
|
||||||
|
);
|
||||||
|
"""
|
||||||
|
)
|
||||||
|
self._ensure_columns(
|
||||||
|
conn,
|
||||||
|
"sources",
|
||||||
|
{
|
||||||
|
"file_size": "integer",
|
||||||
|
"sha256": "text",
|
||||||
|
"mime_type": "text",
|
||||||
|
"processor": "text",
|
||||||
|
"page_ranges": "text",
|
||||||
|
"processed_at": "text",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
self._backfill_source_file_metadata(conn)
|
||||||
|
|
||||||
|
def _ensure_columns(self, conn: sqlite3.Connection, table: str, columns: dict[str, str]) -> None:
|
||||||
|
existing = {row["name"] for row in conn.execute(f"pragma table_info({table})").fetchall()}
|
||||||
|
for name, definition in columns.items():
|
||||||
|
if name not in existing:
|
||||||
|
conn.execute(f"alter table {table} add column {name} {definition}")
|
||||||
|
|
||||||
|
def _backfill_source_file_metadata(self, conn: sqlite3.Connection) -> None:
|
||||||
|
rows = conn.execute("select id, stored_path from sources where sha256 is null or file_size is null").fetchall()
|
||||||
|
for row in rows:
|
||||||
|
path = Path(row["stored_path"])
|
||||||
|
if not path.exists():
|
||||||
|
continue
|
||||||
|
sha256, file_size = self._file_hash_and_size(path)
|
||||||
|
conn.execute("update sources set sha256 = ?, file_size = ? where id = ?", (sha256, file_size, row["id"]))
|
||||||
|
|
||||||
|
def create_subject(self, name: str) -> dict[str, Any]:
|
||||||
|
base_slug = slugify(name)
|
||||||
|
slug = base_slug
|
||||||
|
suffix = 2
|
||||||
|
with self._connect() as conn:
|
||||||
|
while conn.execute("select 1 from subjects where slug = ?", (slug,)).fetchone():
|
||||||
|
slug = f"{base_slug}-{suffix}"
|
||||||
|
suffix += 1
|
||||||
|
now = utc_now()
|
||||||
|
cursor = conn.execute(
|
||||||
|
"insert into subjects (name, slug, created_at) values (?, ?, ?)",
|
||||||
|
(name, slug, now),
|
||||||
|
)
|
||||||
|
(self.subjects_dir / slug).mkdir(parents=True, exist_ok=True)
|
||||||
|
return self.get_subject(cursor.lastrowid)
|
||||||
|
|
||||||
|
def get_subject(self, subject_id: int) -> dict[str, Any]:
|
||||||
|
with self._connect() as conn:
|
||||||
|
row = conn.execute("select * from subjects where id = ?", (subject_id,)).fetchone()
|
||||||
|
if not row:
|
||||||
|
raise ValueError("Materia no encontrada")
|
||||||
|
return dict(row)
|
||||||
|
|
||||||
|
def delete_subject(self, subject_id: int) -> None:
|
||||||
|
subject = self.get_subject(subject_id)
|
||||||
|
with self._connect() as conn:
|
||||||
|
week_rows = conn.execute("select id from weeks where subject_id = ?", (subject_id,)).fetchall()
|
||||||
|
week_ids = [row["id"] for row in week_rows]
|
||||||
|
for week_id in week_ids:
|
||||||
|
source_rows = conn.execute("select id from sources where week_id = ?", (week_id,)).fetchall()
|
||||||
|
source_ids = [row["id"] for row in source_rows]
|
||||||
|
for source_id in source_ids:
|
||||||
|
conn.execute("delete from documents where source_id = ?", (source_id,))
|
||||||
|
conn.execute("delete from jobs where source_id = ?", (source_id,))
|
||||||
|
conn.execute("delete from sources where week_id = ?", (week_id,))
|
||||||
|
conn.execute("delete from weeks where subject_id = ?", (subject_id,))
|
||||||
|
conn.execute("delete from subjects where id = ?", (subject_id,))
|
||||||
|
shutil.rmtree(self.subjects_dir / subject["slug"], ignore_errors=True)
|
||||||
|
|
||||||
|
def list_subjects(self) -> list[dict[str, Any]]:
|
||||||
|
with self._connect() as conn:
|
||||||
|
rows = conn.execute("select * from subjects order by created_at desc").fetchall()
|
||||||
|
return [dict(row) for row in rows]
|
||||||
|
|
||||||
|
def create_week(self, subject_id: int, number: int, title: str | None = None) -> dict[str, Any]:
|
||||||
|
subject = self.get_subject(subject_id)
|
||||||
|
now = utc_now()
|
||||||
|
with self._connect() as conn:
|
||||||
|
conn.execute(
|
||||||
|
"insert or ignore into weeks (subject_id, number, title, created_at) values (?, ?, ?, ?)",
|
||||||
|
(subject_id, number, title, now),
|
||||||
|
)
|
||||||
|
if title:
|
||||||
|
conn.execute("update weeks set title = ? where subject_id = ? and number = ?", (title, subject_id, number))
|
||||||
|
row = conn.execute(
|
||||||
|
"select * from weeks where subject_id = ? and number = ?",
|
||||||
|
(subject_id, number),
|
||||||
|
).fetchone()
|
||||||
|
self.week_dir(subject["slug"], number).mkdir(parents=True, exist_ok=True)
|
||||||
|
for child in ("raw", "processed", "exports"):
|
||||||
|
(self.week_dir(subject["slug"], number) / child).mkdir(parents=True, exist_ok=True)
|
||||||
|
return dict(row)
|
||||||
|
|
||||||
|
def list_weeks(self, subject_id: int) -> list[dict[str, Any]]:
|
||||||
|
with self._connect() as conn:
|
||||||
|
rows = conn.execute("select * from weeks where subject_id = ? order by number", (subject_id,)).fetchall()
|
||||||
|
return [dict(row) for row in rows]
|
||||||
|
|
||||||
|
def delete_week(self, subject_id: int, number: int) -> None:
|
||||||
|
subject = self.get_subject(subject_id)
|
||||||
|
week = self.get_week(subject_id, number)
|
||||||
|
with self._connect() as conn:
|
||||||
|
source_rows = conn.execute("select id from sources where week_id = ?", (week["id"],)).fetchall()
|
||||||
|
for row in source_rows:
|
||||||
|
conn.execute("delete from documents where source_id = ?", (row["id"],))
|
||||||
|
conn.execute("delete from jobs where source_id = ?", (row["id"],))
|
||||||
|
conn.execute("delete from sources where week_id = ?", (week["id"],))
|
||||||
|
conn.execute("delete from weeks where id = ?", (week["id"],))
|
||||||
|
shutil.rmtree(self.week_dir(subject["slug"], number), ignore_errors=True)
|
||||||
|
|
||||||
|
def get_week(self, subject_id: int, number: int) -> dict[str, Any]:
|
||||||
|
with self._connect() as conn:
|
||||||
|
row = conn.execute(
|
||||||
|
"select * from weeks where subject_id = ? and number = ?",
|
||||||
|
(subject_id, number),
|
||||||
|
).fetchone()
|
||||||
|
if not row:
|
||||||
|
raise ValueError("Semana no encontrada")
|
||||||
|
return dict(row)
|
||||||
|
|
||||||
|
def update_week(self, subject_id: int, current_number: int, new_number: int, title: str | None = None) -> dict[str, Any]:
|
||||||
|
subject = self.get_subject(subject_id)
|
||||||
|
week = self.get_week(subject_id, current_number)
|
||||||
|
if new_number != current_number:
|
||||||
|
with self._connect() as conn:
|
||||||
|
existing = conn.execute(
|
||||||
|
"select 1 from weeks where subject_id = ? and number = ?",
|
||||||
|
(subject_id, new_number),
|
||||||
|
).fetchone()
|
||||||
|
if existing:
|
||||||
|
raise ValueError("Ya existe una semana con ese numero")
|
||||||
|
|
||||||
|
old_dir = self.week_dir(subject["slug"], current_number)
|
||||||
|
new_dir = self.week_dir(subject["slug"], new_number)
|
||||||
|
if old_dir.exists() and new_dir.exists():
|
||||||
|
raise ValueError("Ya existe una carpeta para ese numero de semana")
|
||||||
|
if old_dir.exists():
|
||||||
|
old_dir.rename(new_dir)
|
||||||
|
else:
|
||||||
|
for child in ("raw", "processed", "exports"):
|
||||||
|
(new_dir / child).mkdir(parents=True, exist_ok=True)
|
||||||
|
|
||||||
|
with self._connect() as conn:
|
||||||
|
conn.execute(
|
||||||
|
"update weeks set number = ?, title = ? where id = ?",
|
||||||
|
(new_number, title, week["id"]),
|
||||||
|
)
|
||||||
|
return self.get_week(subject_id, new_number)
|
||||||
|
|
||||||
|
def week_dir(self, subject_slug: str, week_number: int) -> Path:
|
||||||
|
return self.subjects_dir / subject_slug / f"week-{week_number:02d}"
|
||||||
|
|
||||||
|
def save_upload(self, subject_id: int, week_number: int, upload: UploadFile) -> dict[str, Any]:
|
||||||
|
subject = self.get_subject(subject_id)
|
||||||
|
week = self.get_week(subject_id, week_number)
|
||||||
|
filename = Path(upload.filename or "archivo").name
|
||||||
|
raw_dir = self.week_dir(subject["slug"], week_number) / "raw"
|
||||||
|
raw_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
stored_path = self._unique_path(raw_dir / filename)
|
||||||
|
digest = hashlib.sha256()
|
||||||
|
file_size = 0
|
||||||
|
with stored_path.open("wb") as out_file:
|
||||||
|
while chunk := upload.file.read(1024 * 1024):
|
||||||
|
file_size += len(chunk)
|
||||||
|
digest.update(chunk)
|
||||||
|
out_file.write(chunk)
|
||||||
|
sha256 = digest.hexdigest()
|
||||||
|
source_type = self.detect_source_type(stored_path)
|
||||||
|
now = utc_now()
|
||||||
|
with self._connect() as conn:
|
||||||
|
duplicate = conn.execute(
|
||||||
|
"select * from sources where week_id = ? and sha256 = ? limit 1",
|
||||||
|
(week["id"], sha256),
|
||||||
|
).fetchone()
|
||||||
|
if duplicate:
|
||||||
|
stored_path.unlink(missing_ok=True)
|
||||||
|
raise DuplicateSourceError(dict(duplicate))
|
||||||
|
cursor = conn.execute(
|
||||||
|
"""
|
||||||
|
insert into sources (week_id, original_name, stored_path, source_type, status, file_size, sha256, mime_type, created_at)
|
||||||
|
values (?, ?, ?, ?, ?, ?, ?, ?, ?)
|
||||||
|
""",
|
||||||
|
(week["id"], filename, str(stored_path), source_type, "stored", file_size, sha256, upload.content_type, now),
|
||||||
|
)
|
||||||
|
return self.get_source(cursor.lastrowid)
|
||||||
|
|
||||||
|
def _file_hash_and_size(self, path: Path) -> tuple[str, int]:
|
||||||
|
digest = hashlib.sha256()
|
||||||
|
file_size = 0
|
||||||
|
with path.open("rb") as file:
|
||||||
|
while chunk := file.read(1024 * 1024):
|
||||||
|
file_size += len(chunk)
|
||||||
|
digest.update(chunk)
|
||||||
|
return digest.hexdigest(), file_size
|
||||||
|
|
||||||
|
def create_job(self, source_id: int, message: str = "En cola") -> dict[str, Any]:
|
||||||
|
now = utc_now()
|
||||||
|
with self._connect() as conn:
|
||||||
|
cursor = conn.execute(
|
||||||
|
"insert into jobs (status, message, source_id, created_at, updated_at) values (?, ?, ?, ?, ?)",
|
||||||
|
("queued", message, source_id, now, now),
|
||||||
|
)
|
||||||
|
return self.get_job(cursor.lastrowid)
|
||||||
|
|
||||||
|
def update_job(self, job_id: int, status: str, message: str, document_id: int | None = None) -> None:
|
||||||
|
with self._connect() as conn:
|
||||||
|
conn.execute(
|
||||||
|
"update jobs set status = ?, message = ?, document_id = coalesce(?, document_id), updated_at = ? where id = ?",
|
||||||
|
(status, message, document_id, utc_now(), job_id),
|
||||||
|
)
|
||||||
|
|
||||||
|
def update_source_status(self, source_id: int, status: str) -> None:
|
||||||
|
with self._connect() as conn:
|
||||||
|
conn.execute("update sources set status = ? where id = ?", (status, source_id))
|
||||||
|
|
||||||
|
def source_has_active_job(self, source_id: int) -> bool:
|
||||||
|
with self._connect() as conn:
|
||||||
|
row = conn.execute(
|
||||||
|
"select 1 from jobs where source_id = ? and status in ('queued', 'processing') limit 1",
|
||||||
|
(source_id,),
|
||||||
|
).fetchone()
|
||||||
|
return bool(row)
|
||||||
|
|
||||||
|
def get_job(self, job_id: int) -> dict[str, Any]:
|
||||||
|
with self._connect() as conn:
|
||||||
|
row = conn.execute("select * from jobs where id = ?", (job_id,)).fetchone()
|
||||||
|
if not row:
|
||||||
|
raise ValueError("Trabajo no encontrado")
|
||||||
|
return dict(row)
|
||||||
|
|
||||||
|
def list_jobs_for_week(self, subject_id: int, week_number: int) -> list[dict[str, Any]]:
|
||||||
|
week = self.get_week(subject_id, week_number)
|
||||||
|
with self._connect() as conn:
|
||||||
|
rows = conn.execute(
|
||||||
|
"""
|
||||||
|
select jobs.* from jobs
|
||||||
|
join sources on sources.id = jobs.source_id
|
||||||
|
where sources.week_id = ?
|
||||||
|
order by jobs.created_at desc
|
||||||
|
""",
|
||||||
|
(week["id"],),
|
||||||
|
).fetchall()
|
||||||
|
return [dict(row) for row in rows]
|
||||||
|
|
||||||
|
def get_source(self, source_id: int) -> dict[str, Any]:
|
||||||
|
with self._connect() as conn:
|
||||||
|
row = conn.execute("select * from sources where id = ?", (source_id,)).fetchone()
|
||||||
|
if not row:
|
||||||
|
raise ValueError("Fuente no encontrada")
|
||||||
|
return dict(row)
|
||||||
|
|
||||||
|
def list_sources(self, subject_id: int, week_number: int) -> list[dict[str, Any]]:
|
||||||
|
week = self.get_week(subject_id, week_number)
|
||||||
|
with self._connect() as conn:
|
||||||
|
rows = conn.execute(
|
||||||
|
"""
|
||||||
|
select sources.*, documents.id as document_id, documents.title as document_title,
|
||||||
|
documents.markdown_path as document_markdown_path, documents.created_at as document_created_at
|
||||||
|
from sources
|
||||||
|
left join documents on documents.source_id = sources.id
|
||||||
|
where sources.week_id = ?
|
||||||
|
order by sources.created_at desc
|
||||||
|
""",
|
||||||
|
(week["id"],),
|
||||||
|
).fetchall()
|
||||||
|
return [dict(row) for row in rows]
|
||||||
|
|
||||||
|
def delete_source(self, source_id: int) -> None:
|
||||||
|
source = self.get_source(source_id)
|
||||||
|
with self._connect() as conn:
|
||||||
|
doc_rows = conn.execute("select markdown_path from documents where source_id = ?", (source_id,)).fetchall()
|
||||||
|
conn.execute("delete from documents where source_id = ?", (source_id,))
|
||||||
|
conn.execute("delete from jobs where source_id = ?", (source_id,))
|
||||||
|
conn.execute("delete from sources where id = ?", (source_id,))
|
||||||
|
Path(source["stored_path"]).unlink(missing_ok=True)
|
||||||
|
for row in doc_rows:
|
||||||
|
Path(row["markdown_path"]).unlink(missing_ok=True)
|
||||||
|
|
||||||
|
def create_document(self, source_id: int, title: str, markdown: str, processor: str | None = None, page_ranges: str | None = None) -> dict[str, Any]:
|
||||||
|
source = self.get_source(source_id)
|
||||||
|
week_context = self.get_week_context(source["week_id"])
|
||||||
|
processed_dir = self.week_dir(week_context["subject_slug"], week_context["week_number"]) / "processed"
|
||||||
|
processed_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
markdown_path = self._unique_path(processed_dir / f"{slugify(title)}.md")
|
||||||
|
markdown_path.write_text(self._sanitize_markdown(markdown), encoding="utf-8")
|
||||||
|
now = utc_now()
|
||||||
|
with self._connect() as conn:
|
||||||
|
old_docs = conn.execute("select markdown_path from documents where source_id = ?", (source_id,)).fetchall()
|
||||||
|
conn.execute("delete from jobs where document_id in (select id from documents where source_id = ?)", (source_id,))
|
||||||
|
conn.execute("delete from documents where source_id = ?", (source_id,))
|
||||||
|
cursor = conn.execute(
|
||||||
|
"insert into documents (source_id, title, markdown_path, created_at) values (?, ?, ?, ?)",
|
||||||
|
(source_id, title, str(markdown_path), now),
|
||||||
|
)
|
||||||
|
conn.execute(
|
||||||
|
"update sources set status = ?, processor = ?, page_ranges = ?, processed_at = ? where id = ?",
|
||||||
|
("processed", processor, page_ranges, now, source_id),
|
||||||
|
)
|
||||||
|
for row in old_docs:
|
||||||
|
Path(row["markdown_path"]).unlink(missing_ok=True)
|
||||||
|
return self.get_document(cursor.lastrowid)
|
||||||
|
|
||||||
|
def get_document(self, document_id: int) -> dict[str, Any]:
|
||||||
|
with self._connect() as conn:
|
||||||
|
row = conn.execute("select * from documents where id = ?", (document_id,)).fetchone()
|
||||||
|
if not row:
|
||||||
|
raise ValueError("Documento no encontrado")
|
||||||
|
return dict(row)
|
||||||
|
|
||||||
|
def update_document_markdown(self, document_id: int, content: str) -> dict[str, Any]:
|
||||||
|
document = self.get_document(document_id)
|
||||||
|
Path(document["markdown_path"]).write_text(self._sanitize_markdown(content), encoding="utf-8")
|
||||||
|
return self.get_document(document_id)
|
||||||
|
|
||||||
|
def delete_document(self, document_id: int) -> None:
|
||||||
|
document = self.get_document(document_id)
|
||||||
|
Path(document["markdown_path"]).unlink(missing_ok=True)
|
||||||
|
with self._connect() as conn:
|
||||||
|
conn.execute("delete from jobs where document_id = ?", (document_id,))
|
||||||
|
conn.execute("delete from documents where id = ?", (document_id,))
|
||||||
|
|
||||||
|
def list_documents(self, subject_id: int, week_number: int) -> list[dict[str, Any]]:
|
||||||
|
week = self.get_week(subject_id, week_number)
|
||||||
|
with self._connect() as conn:
|
||||||
|
rows = conn.execute(
|
||||||
|
"""
|
||||||
|
select documents.*, sources.original_name as source_original_name,
|
||||||
|
sources.source_type as source_type, sources.status as source_status,
|
||||||
|
sources.processor as source_processor, sources.page_ranges as source_page_ranges
|
||||||
|
from documents
|
||||||
|
join sources on sources.id = documents.source_id
|
||||||
|
where sources.week_id = ?
|
||||||
|
order by documents.created_at desc
|
||||||
|
""",
|
||||||
|
(week["id"],),
|
||||||
|
).fetchall()
|
||||||
|
return [dict(row) for row in rows]
|
||||||
|
|
||||||
|
def get_week_context(self, week_id: int) -> dict[str, Any]:
|
||||||
|
with self._connect() as conn:
|
||||||
|
row = conn.execute(
|
||||||
|
"""
|
||||||
|
select weeks.id as week_id, weeks.number as week_number, weeks.title as week_title,
|
||||||
|
subjects.id as subject_id, subjects.name as subject_name, subjects.slug as subject_slug
|
||||||
|
from weeks join subjects on subjects.id = weeks.subject_id
|
||||||
|
where weeks.id = ?
|
||||||
|
""",
|
||||||
|
(week_id,),
|
||||||
|
).fetchone()
|
||||||
|
if not row:
|
||||||
|
raise ValueError("Contexto de semana no encontrado")
|
||||||
|
return dict(row)
|
||||||
|
|
||||||
|
def export_week_markdown(self, subject_id: int, week_number: int) -> Path:
|
||||||
|
subject = self.get_subject(subject_id)
|
||||||
|
docs = self.list_documents(subject_id, week_number)
|
||||||
|
export_dir = self.week_dir(subject["slug"], week_number) / "exports"
|
||||||
|
export_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
export_path = export_dir / f"{subject['slug']}-week-{week_number:02d}.md"
|
||||||
|
parts = [
|
||||||
|
"---\n"
|
||||||
|
f"subject: {subject['name']}\n"
|
||||||
|
f"subject_slug: {subject['slug']}\n"
|
||||||
|
f"week: {week_number}\n"
|
||||||
|
f"documents: {len(docs)}\n"
|
||||||
|
f"exported_at: {utc_now()}\n"
|
||||||
|
"---\n\n"
|
||||||
|
f"# {subject['name']} - Semana {week_number}\n"
|
||||||
|
]
|
||||||
|
for doc in docs:
|
||||||
|
path = Path(doc["markdown_path"])
|
||||||
|
content = self._strip_frontmatter(self._read_markdown(path)).strip()
|
||||||
|
parts.append(f"<!-- document_id: {doc['id']} -->\n\n{content}\n")
|
||||||
|
export_path.write_text(self._sanitize_markdown("\n\n---\n\n".join(parts)), encoding="utf-8")
|
||||||
|
return export_path
|
||||||
|
|
||||||
|
def export_week_markdown_zip(self, subject_id: int, week_number: int) -> Path:
|
||||||
|
subject = self.get_subject(subject_id)
|
||||||
|
docs = self.list_documents(subject_id, week_number)
|
||||||
|
export_dir = self.week_dir(subject["slug"], week_number) / "exports"
|
||||||
|
export_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
export_path = export_dir / f"{subject['slug']}-week-{week_number:02d}-documentos.zip"
|
||||||
|
used_names: set[str] = set()
|
||||||
|
with zipfile.ZipFile(export_path, "w", compression=zipfile.ZIP_DEFLATED) as archive:
|
||||||
|
for index, doc in enumerate(docs, start=1):
|
||||||
|
path = Path(doc["markdown_path"])
|
||||||
|
name = self._unique_archive_name(f"{index:02d}-{slugify(doc['title'])}.md", used_names)
|
||||||
|
archive.writestr(name, self._read_markdown(path))
|
||||||
|
return export_path
|
||||||
|
|
||||||
|
def _unique_archive_name(self, filename: str, used_names: set[str]) -> str:
|
||||||
|
path = Path(filename)
|
||||||
|
candidate = path.name
|
||||||
|
suffix = path.suffix
|
||||||
|
stem = path.stem
|
||||||
|
counter = 2
|
||||||
|
while candidate in used_names:
|
||||||
|
candidate = f"{stem}-{counter}{suffix}"
|
||||||
|
counter += 1
|
||||||
|
used_names.add(candidate)
|
||||||
|
return candidate
|
||||||
|
|
||||||
|
def _read_markdown(self, path: Path) -> str:
|
||||||
|
return self._sanitize_markdown(path.read_text(encoding="utf-8", errors="replace"))
|
||||||
|
|
||||||
|
def _sanitize_markdown(self, content: str) -> str:
|
||||||
|
return re.sub(r"[\x00-\x08\x0b\x0c\x0e-\x1f\x7f]", "", content)
|
||||||
|
|
||||||
|
def _strip_frontmatter(self, content: str) -> str:
|
||||||
|
lines = content.splitlines()
|
||||||
|
if not lines or lines[0].strip() != "---":
|
||||||
|
return content
|
||||||
|
for index, line in enumerate(lines[1:], start=1):
|
||||||
|
if line.strip() == "---":
|
||||||
|
return "\n".join(lines[index + 1 :]).lstrip()
|
||||||
|
return content
|
||||||
|
|
||||||
|
def detect_source_type(self, path: Path) -> str:
|
||||||
|
ext = path.suffix.lower().lstrip(".")
|
||||||
|
if ext in {"txt", "md"}:
|
||||||
|
return "text"
|
||||||
|
if ext == "docx":
|
||||||
|
return "docx"
|
||||||
|
if ext == "pdf":
|
||||||
|
return "pdf"
|
||||||
|
if ext in {"mp3", "wav", "m4a", "ogg", "flac", "webm"}:
|
||||||
|
return "audio"
|
||||||
|
if ext in {"mp4", "mov", "mkv", "avi"}:
|
||||||
|
return "video"
|
||||||
|
return "unknown"
|
||||||
|
|
||||||
|
def _unique_path(self, path: Path) -> Path:
|
||||||
|
if not path.exists():
|
||||||
|
return path
|
||||||
|
stem = path.stem
|
||||||
|
suffix = path.suffix
|
||||||
|
for index in range(2, 10_000):
|
||||||
|
candidate = path.with_name(f"{stem}-{index}{suffix}")
|
||||||
|
if not candidate.exists():
|
||||||
|
return candidate
|
||||||
|
raise RuntimeError("No se pudo generar un nombre de archivo unico")
|
||||||
|
|
||||||
|
|
||||||
|
def rows_to_models(rows: Iterable[dict[str, Any]]) -> list[dict[str, Any]]:
|
||||||
|
return [dict(row) for row in rows]
|
||||||
@@ -0,0 +1,55 @@
|
|||||||
|
from datetime import datetime
|
||||||
|
from typing import Optional
|
||||||
|
|
||||||
|
from pydantic import BaseModel, Field
|
||||||
|
|
||||||
|
|
||||||
|
class SubjectCreate(BaseModel):
|
||||||
|
name: str = Field(min_length=1)
|
||||||
|
|
||||||
|
|
||||||
|
class Subject(BaseModel):
|
||||||
|
id: int
|
||||||
|
name: str
|
||||||
|
slug: str
|
||||||
|
created_at: datetime
|
||||||
|
|
||||||
|
|
||||||
|
class WeekCreate(BaseModel):
|
||||||
|
number: int = Field(ge=1)
|
||||||
|
title: Optional[str] = None
|
||||||
|
|
||||||
|
|
||||||
|
class WeekUpdate(BaseModel):
|
||||||
|
number: int = Field(ge=1)
|
||||||
|
title: Optional[str] = None
|
||||||
|
|
||||||
|
|
||||||
|
class Week(BaseModel):
|
||||||
|
id: int
|
||||||
|
subject_id: int
|
||||||
|
number: int
|
||||||
|
title: Optional[str]
|
||||||
|
created_at: datetime
|
||||||
|
|
||||||
|
|
||||||
|
class MarkdownUpdate(BaseModel):
|
||||||
|
content: str
|
||||||
|
|
||||||
|
|
||||||
|
class Job(BaseModel):
|
||||||
|
id: int
|
||||||
|
status: str
|
||||||
|
message: str
|
||||||
|
source_id: Optional[int] = None
|
||||||
|
document_id: Optional[int] = None
|
||||||
|
created_at: datetime
|
||||||
|
updated_at: datetime
|
||||||
|
|
||||||
|
|
||||||
|
class Document(BaseModel):
|
||||||
|
id: int
|
||||||
|
source_id: int
|
||||||
|
title: str
|
||||||
|
markdown_path: str
|
||||||
|
created_at: datetime
|
||||||
@@ -0,0 +1,51 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from backend.app.managers.knowledge_manager import KnowledgeManager
|
||||||
|
from backend.app.services.markdown_builder import build_markdown
|
||||||
|
from backend.app.services.processors.audio_processor import AudioProcessor
|
||||||
|
from backend.app.services.processors.docx_processor import DocxProcessor
|
||||||
|
from backend.app.services.processors.pdf_processor import PdfProcessor
|
||||||
|
from backend.app.services.processors.text_processor import TextProcessor
|
||||||
|
from backend.app.services.processors.video_processor import VideoProcessor
|
||||||
|
|
||||||
|
|
||||||
|
class IngestionService:
|
||||||
|
def __init__(self, manager: KnowledgeManager):
|
||||||
|
self.manager = manager
|
||||||
|
|
||||||
|
def process_source(self, source_id: int, use_ocr: bool = False, page_ranges: str | None = None) -> dict:
|
||||||
|
source = self.manager.get_source(source_id)
|
||||||
|
context = self.manager.get_week_context(source["week_id"])
|
||||||
|
path = Path(source["stored_path"])
|
||||||
|
processor = self._processor_for(source["source_type"], use_ocr, page_ranges)
|
||||||
|
processed = processor.process(path)
|
||||||
|
markdown = build_markdown(
|
||||||
|
title=processed.title,
|
||||||
|
body=processed.body,
|
||||||
|
metadata={
|
||||||
|
"subject": context["subject_name"],
|
||||||
|
"subject_slug": context["subject_slug"],
|
||||||
|
"week": context["week_number"],
|
||||||
|
"source_file": source["original_name"],
|
||||||
|
"source_type": source["source_type"],
|
||||||
|
"processor": processed.processor,
|
||||||
|
"page_ranges": page_ranges if source["source_type"] == "pdf" and page_ranges else None,
|
||||||
|
"language": "es",
|
||||||
|
},
|
||||||
|
)
|
||||||
|
return self.manager.create_document(source_id, processed.title, markdown, processor=processed.processor, page_ranges=page_ranges)
|
||||||
|
|
||||||
|
def _processor_for(self, source_type: str, use_ocr: bool, page_ranges: str | None):
|
||||||
|
if source_type == "text":
|
||||||
|
return TextProcessor()
|
||||||
|
if source_type == "docx":
|
||||||
|
return DocxProcessor()
|
||||||
|
if source_type == "pdf":
|
||||||
|
return PdfProcessor(use_ocr=use_ocr, page_ranges=page_ranges)
|
||||||
|
if source_type == "audio":
|
||||||
|
return AudioProcessor()
|
||||||
|
if source_type == "video":
|
||||||
|
return VideoProcessor(self.manager.tmp_dir)
|
||||||
|
raise RuntimeError(f"Tipo de fuente no soportado: {source_type}")
|
||||||
@@ -0,0 +1,23 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from datetime import datetime, timezone
|
||||||
|
from pathlib import Path
|
||||||
|
from typing import Any
|
||||||
|
|
||||||
|
|
||||||
|
def build_markdown(title: str, body: str, metadata: dict[str, Any]) -> str:
|
||||||
|
frontmatter = {
|
||||||
|
**metadata,
|
||||||
|
"processed_at": datetime.now(timezone.utc).isoformat(),
|
||||||
|
}
|
||||||
|
lines = ["---"]
|
||||||
|
for key, value in frontmatter.items():
|
||||||
|
if value is not None:
|
||||||
|
safe_value = str(value).replace("\n", " ")
|
||||||
|
lines.append(f"{key}: {safe_value}")
|
||||||
|
lines.extend(["---", "", f"# {title}", "", body.strip(), ""])
|
||||||
|
return "\n".join(lines)
|
||||||
|
|
||||||
|
|
||||||
|
def title_from_path(path: Path) -> str:
|
||||||
|
return path.stem.replace("-", " ").replace("_", " ").strip().title() or "Documento"
|
||||||
@@ -0,0 +1,20 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from backend.app.services.markdown_builder import title_from_path
|
||||||
|
from backend.app.services.processors.base import ProcessedContent
|
||||||
|
from backend.app.services.providers.deepgram_client import DeepgramClient
|
||||||
|
|
||||||
|
|
||||||
|
class AudioProcessor:
|
||||||
|
def process(self, path: Path) -> ProcessedContent:
|
||||||
|
client = DeepgramClient()
|
||||||
|
data = client.transcribe_file(path)
|
||||||
|
transcript, confidence = client.transcript_from_response(data)
|
||||||
|
confidence_line = f"\n\n_Confianza estimada: {confidence * 100:.1f}%_" if isinstance(confidence, (int, float)) else ""
|
||||||
|
return ProcessedContent(
|
||||||
|
title=f"Transcripcion {title_from_path(path)}",
|
||||||
|
body=f"## Transcripcion\n\n{transcript}{confidence_line}",
|
||||||
|
processor="deepgram",
|
||||||
|
)
|
||||||
@@ -0,0 +1,10 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from dataclasses import dataclass
|
||||||
|
|
||||||
|
|
||||||
|
@dataclass
|
||||||
|
class ProcessedContent:
|
||||||
|
title: str
|
||||||
|
body: str
|
||||||
|
processor: str
|
||||||
@@ -0,0 +1,30 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from docx import Document
|
||||||
|
|
||||||
|
from backend.app.services.markdown_builder import title_from_path
|
||||||
|
from backend.app.services.processors.base import ProcessedContent
|
||||||
|
|
||||||
|
|
||||||
|
class DocxProcessor:
|
||||||
|
def process(self, path: Path) -> ProcessedContent:
|
||||||
|
doc = Document(path)
|
||||||
|
blocks: list[str] = []
|
||||||
|
for paragraph in doc.paragraphs:
|
||||||
|
text = paragraph.text.strip()
|
||||||
|
if text:
|
||||||
|
blocks.append(text)
|
||||||
|
for table in doc.tables:
|
||||||
|
rows = [[cell.text.strip().replace("\n", " ") for cell in row.cells] for row in table.rows]
|
||||||
|
if rows:
|
||||||
|
blocks.append(self._table_to_markdown(rows))
|
||||||
|
return ProcessedContent(title=title_from_path(path), body="\n\n".join(blocks), processor="docx")
|
||||||
|
|
||||||
|
def _table_to_markdown(self, rows: list[list[str]]) -> str:
|
||||||
|
header = rows[0]
|
||||||
|
lines = [f"| {' | '.join(header)} |", f"| {' | '.join(['---'] * len(header))} |"]
|
||||||
|
for row in rows[1:]:
|
||||||
|
lines.append(f"| {' | '.join(row)} |")
|
||||||
|
return "\n".join(lines)
|
||||||
@@ -0,0 +1,81 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import fitz
|
||||||
|
|
||||||
|
from backend.app.services.markdown_builder import title_from_path
|
||||||
|
from backend.app.services.processors.base import ProcessedContent
|
||||||
|
from backend.app.services.providers.mistral_client import MistralClient
|
||||||
|
|
||||||
|
|
||||||
|
class PdfProcessor:
|
||||||
|
def __init__(self, use_ocr: bool = False, page_ranges: str | None = None):
|
||||||
|
self.use_ocr = use_ocr
|
||||||
|
self.page_ranges = page_ranges
|
||||||
|
|
||||||
|
def process(self, path: Path) -> ProcessedContent:
|
||||||
|
if self.use_ocr:
|
||||||
|
return self._process_ocr(path)
|
||||||
|
doc = fitz.open(path)
|
||||||
|
pages: list[str] = []
|
||||||
|
for index in self._selected_pages(len(doc)):
|
||||||
|
page = doc[index - 1]
|
||||||
|
text = page.get_text("text").strip()
|
||||||
|
if text:
|
||||||
|
pages.append(f"## Pagina {index}\n\n{text}")
|
||||||
|
body = "\n\n".join(pages).strip()
|
||||||
|
if len(body) < 80:
|
||||||
|
raise RuntimeError("El PDF tiene poco texto extraible; procesalo con OCR")
|
||||||
|
return ProcessedContent(title=title_from_path(path), body=body, processor="pdf-text")
|
||||||
|
|
||||||
|
def _process_ocr(self, path: Path) -> ProcessedContent:
|
||||||
|
doc = fitz.open(path)
|
||||||
|
client = MistralClient()
|
||||||
|
parts: list[str] = []
|
||||||
|
temp_paths: list[Path] = []
|
||||||
|
try:
|
||||||
|
for index in self._selected_pages(len(doc)):
|
||||||
|
page = doc[index - 1]
|
||||||
|
pix = page.get_pixmap(matrix=fitz.Matrix(2, 2), alpha=False)
|
||||||
|
image_path = path.with_name(f"{path.stem}-page-{index}.jpg")
|
||||||
|
pix.save(image_path)
|
||||||
|
temp_paths.append(image_path)
|
||||||
|
parts.append(f"## Pagina {index}\n\n{client.ocr_image(image_path)}")
|
||||||
|
finally:
|
||||||
|
for temp_path in temp_paths:
|
||||||
|
temp_path.unlink(missing_ok=True)
|
||||||
|
return ProcessedContent(title=title_from_path(path), body="\n\n".join(parts), processor="mistral-ocr")
|
||||||
|
|
||||||
|
def _selected_pages(self, total_pages: int) -> list[int]:
|
||||||
|
if total_pages < 1:
|
||||||
|
raise RuntimeError("El PDF no tiene paginas")
|
||||||
|
if not self.page_ranges or not self.page_ranges.strip():
|
||||||
|
return list(range(1, total_pages + 1))
|
||||||
|
|
||||||
|
selected: list[int] = []
|
||||||
|
for raw_part in self.page_ranges.split(","):
|
||||||
|
part = raw_part.strip()
|
||||||
|
if not part:
|
||||||
|
continue
|
||||||
|
if "-" in part:
|
||||||
|
start_text, end_text = [piece.strip() for piece in part.split("-", 1)]
|
||||||
|
if not start_text.isdigit() or not end_text.isdigit():
|
||||||
|
raise RuntimeError(f"Rango de paginas invalido: {part}")
|
||||||
|
start = int(start_text)
|
||||||
|
end = int(end_text)
|
||||||
|
if start > end:
|
||||||
|
raise RuntimeError(f"Rango de paginas invertido: {part}")
|
||||||
|
selected.extend(range(start, end + 1))
|
||||||
|
else:
|
||||||
|
if not part.isdigit():
|
||||||
|
raise RuntimeError(f"Pagina invalida: {part}")
|
||||||
|
selected.append(int(part))
|
||||||
|
|
||||||
|
unique_pages = sorted(set(selected))
|
||||||
|
invalid = [page for page in unique_pages if page < 1 or page > total_pages]
|
||||||
|
if invalid:
|
||||||
|
raise RuntimeError(f"Paginas fuera de rango: {', '.join(map(str, invalid))}. El PDF tiene {total_pages} paginas")
|
||||||
|
if not unique_pages:
|
||||||
|
raise RuntimeError("No se seleccionaron paginas para procesar")
|
||||||
|
return unique_pages
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from backend.app.services.markdown_builder import title_from_path
|
||||||
|
from backend.app.services.processors.base import ProcessedContent
|
||||||
|
|
||||||
|
|
||||||
|
class TextProcessor:
|
||||||
|
def process(self, path: Path) -> ProcessedContent:
|
||||||
|
return ProcessedContent(
|
||||||
|
title=title_from_path(path),
|
||||||
|
body=path.read_text(encoding="utf-8", errors="replace"),
|
||||||
|
processor="text",
|
||||||
|
)
|
||||||
@@ -0,0 +1,46 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import subprocess
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
from backend.app.services.processors.audio_processor import AudioProcessor
|
||||||
|
from backend.app.services.processors.base import ProcessedContent
|
||||||
|
|
||||||
|
|
||||||
|
class VideoProcessor:
|
||||||
|
def __init__(self, tmp_dir: Path):
|
||||||
|
self.tmp_dir = tmp_dir
|
||||||
|
|
||||||
|
def process(self, path: Path) -> ProcessedContent:
|
||||||
|
self.tmp_dir.mkdir(parents=True, exist_ok=True)
|
||||||
|
audio_path = self.tmp_dir / f"{path.stem}.wav"
|
||||||
|
try:
|
||||||
|
subprocess.run(
|
||||||
|
[
|
||||||
|
"ffmpeg",
|
||||||
|
"-y",
|
||||||
|
"-i",
|
||||||
|
str(path),
|
||||||
|
"-vn",
|
||||||
|
"-acodec",
|
||||||
|
"pcm_s16le",
|
||||||
|
"-ar",
|
||||||
|
"16000",
|
||||||
|
"-ac",
|
||||||
|
"1",
|
||||||
|
str(audio_path),
|
||||||
|
],
|
||||||
|
check=True,
|
||||||
|
capture_output=True,
|
||||||
|
text=True,
|
||||||
|
)
|
||||||
|
processed = AudioProcessor().process(audio_path)
|
||||||
|
processed.title = f"Transcripcion {path.stem}"
|
||||||
|
processed.processor = "ffmpeg+deepgram"
|
||||||
|
return processed
|
||||||
|
except FileNotFoundError as exc:
|
||||||
|
raise RuntimeError("ffmpeg no esta instalado o no esta disponible en PATH") from exc
|
||||||
|
except subprocess.CalledProcessError as exc:
|
||||||
|
raise RuntimeError(f"No se pudo extraer audio del video: {exc.stderr}") from exc
|
||||||
|
finally:
|
||||||
|
audio_path.unlink(missing_ok=True)
|
||||||
@@ -0,0 +1,52 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import requests
|
||||||
|
|
||||||
|
from backend.app.config import settings
|
||||||
|
|
||||||
|
|
||||||
|
class DeepgramClient:
|
||||||
|
endpoint = "https://api.deepgram.com/v1/listen?model=nova-3&smart_format=true&language=es"
|
||||||
|
|
||||||
|
def transcribe_file(self, path: Path) -> dict:
|
||||||
|
if not settings.deepgram_api_key:
|
||||||
|
raise RuntimeError("DEEPGRAM_API_KEY no esta configurada")
|
||||||
|
|
||||||
|
headers = {
|
||||||
|
"Authorization": f"Token {settings.deepgram_api_key}",
|
||||||
|
"Content-Type": self._content_type(path),
|
||||||
|
}
|
||||||
|
with path.open("rb") as audio_file:
|
||||||
|
response = requests.post(self.endpoint, headers=headers, data=audio_file, timeout=600)
|
||||||
|
|
||||||
|
if response.status_code == 401:
|
||||||
|
raise RuntimeError("API Key de Deepgram invalida")
|
||||||
|
if not response.ok:
|
||||||
|
try:
|
||||||
|
error = response.json()
|
||||||
|
except ValueError:
|
||||||
|
error = response.text
|
||||||
|
raise RuntimeError(f"Deepgram error {response.status_code}: {error}")
|
||||||
|
return response.json()
|
||||||
|
|
||||||
|
def transcript_from_response(self, data: dict) -> tuple[str, float | None]:
|
||||||
|
alternative = data.get("results", {}).get("channels", [{}])[0].get("alternatives", [{}])[0]
|
||||||
|
transcript = alternative.get("paragraphs", {}).get("transcript") or alternative.get("transcript") or ""
|
||||||
|
confidence = alternative.get("confidence")
|
||||||
|
if not transcript.strip():
|
||||||
|
raise RuntimeError("Deepgram no devolvio transcripcion")
|
||||||
|
return transcript.strip(), confidence
|
||||||
|
|
||||||
|
def _content_type(self, path: Path) -> str:
|
||||||
|
ext = path.suffix.lower()
|
||||||
|
return {
|
||||||
|
".mp3": "audio/mpeg",
|
||||||
|
".wav": "audio/wav",
|
||||||
|
".m4a": "audio/mp4",
|
||||||
|
".mp4": "audio/mp4",
|
||||||
|
".webm": "audio/webm",
|
||||||
|
".ogg": "audio/ogg",
|
||||||
|
".flac": "audio/flac",
|
||||||
|
}.get(ext, "application/octet-stream")
|
||||||
@@ -0,0 +1,52 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import base64
|
||||||
|
from pathlib import Path
|
||||||
|
|
||||||
|
import requests
|
||||||
|
|
||||||
|
from backend.app.config import settings
|
||||||
|
|
||||||
|
|
||||||
|
class MistralClient:
|
||||||
|
endpoint = "https://api.mistral.ai/v1/chat/completions"
|
||||||
|
model = "pixtral-12b-2409"
|
||||||
|
|
||||||
|
def ocr_image(self, image_path: Path) -> str:
|
||||||
|
if not settings.mistral_api_key:
|
||||||
|
raise RuntimeError("MISTRAL_API_KEY no esta configurada")
|
||||||
|
data_url = self._image_data_url(image_path)
|
||||||
|
response = requests.post(
|
||||||
|
self.endpoint,
|
||||||
|
headers={
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
"Authorization": f"Bearer {settings.mistral_api_key}",
|
||||||
|
},
|
||||||
|
json={
|
||||||
|
"model": self.model,
|
||||||
|
"messages": [
|
||||||
|
{
|
||||||
|
"role": "user",
|
||||||
|
"content": [
|
||||||
|
{
|
||||||
|
"type": "text",
|
||||||
|
"text": "Extract all text from this image and format it as Markdown. If there are tables, represent them using Markdown table syntax. Do not include commentary.",
|
||||||
|
},
|
||||||
|
{"type": "image_url", "image_url": data_url},
|
||||||
|
],
|
||||||
|
}
|
||||||
|
],
|
||||||
|
"max_tokens": 3000,
|
||||||
|
},
|
||||||
|
timeout=180,
|
||||||
|
)
|
||||||
|
if response.status_code == 401:
|
||||||
|
raise RuntimeError("API Key de Mistral invalida")
|
||||||
|
if not response.ok:
|
||||||
|
raise RuntimeError(f"Mistral error {response.status_code}: {response.text}")
|
||||||
|
data = response.json()
|
||||||
|
return data["choices"][0]["message"]["content"]
|
||||||
|
|
||||||
|
def _image_data_url(self, image_path: Path) -> str:
|
||||||
|
encoded = base64.b64encode(image_path.read_bytes()).decode("ascii")
|
||||||
|
return f"data:image/jpeg;base64,{encoded}"
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
import re
|
||||||
|
import unicodedata
|
||||||
|
|
||||||
|
|
||||||
|
def slugify(value: str) -> str:
|
||||||
|
normalized = unicodedata.normalize("NFKD", value).encode("ascii", "ignore").decode("ascii")
|
||||||
|
slug = re.sub(r"[^a-zA-Z0-9]+", "-", normalized).strip("-").lower()
|
||||||
|
return slug or "sin-titulo"
|
||||||
@@ -0,0 +1,12 @@
|
|||||||
|
@echo off
|
||||||
|
setlocal
|
||||||
|
title Knowledge Station - Cerrar
|
||||||
|
|
||||||
|
set "ROOT=%~dp0"
|
||||||
|
set "ROOT=%ROOT:~0,-1%"
|
||||||
|
powershell -NoProfile -ExecutionPolicy Bypass -File "%ROOT%\scripts\stop-platform.ps1" -Root "%ROOT%"
|
||||||
|
|
||||||
|
echo.
|
||||||
|
echo Puedes cerrar esta ventana.
|
||||||
|
pause
|
||||||
|
exit /b 0
|
||||||
@@ -0,0 +1,43 @@
|
|||||||
|
services:
|
||||||
|
backend:
|
||||||
|
build:
|
||||||
|
context: .
|
||||||
|
dockerfile: Dockerfile
|
||||||
|
environment:
|
||||||
|
APP_ENV: production
|
||||||
|
APP_DATA_DIR: /app/data
|
||||||
|
DATABASE_PATH: /app/data/app.db
|
||||||
|
MISTRAL_API_KEY: ${MISTRAL_API_KEY:-}
|
||||||
|
DEEPGRAM_API_KEY: ${DEEPGRAM_API_KEY:-}
|
||||||
|
volumes:
|
||||||
|
- station-data:/app/data
|
||||||
|
expose:
|
||||||
|
- "8000"
|
||||||
|
restart: unless-stopped
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=5)"]
|
||||||
|
interval: 30s
|
||||||
|
timeout: 10s
|
||||||
|
retries: 3
|
||||||
|
start_period: 20s
|
||||||
|
|
||||||
|
frontend:
|
||||||
|
build:
|
||||||
|
context: ./frontend
|
||||||
|
dockerfile: Dockerfile
|
||||||
|
args:
|
||||||
|
NEXT_PUBLIC_API_URL: ""
|
||||||
|
BACKEND_INTERNAL_URL: http://backend:8000
|
||||||
|
environment:
|
||||||
|
NODE_ENV: production
|
||||||
|
PORT: 3000
|
||||||
|
HOSTNAME: 0.0.0.0
|
||||||
|
expose:
|
||||||
|
- "3000"
|
||||||
|
depends_on:
|
||||||
|
backend:
|
||||||
|
condition: service_healthy
|
||||||
|
restart: unless-stopped
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
station-data:
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
services:
|
||||||
|
backend:
|
||||||
|
build: .
|
||||||
|
ports:
|
||||||
|
- "8000:8000"
|
||||||
|
env_file:
|
||||||
|
- .env
|
||||||
|
volumes:
|
||||||
|
- ./data:/app/data
|
||||||
|
frontend:
|
||||||
|
build:
|
||||||
|
context: ./frontend
|
||||||
|
ports:
|
||||||
|
- "3000:3000"
|
||||||
|
environment:
|
||||||
|
NEXT_PUBLIC_API_URL: http://127.0.0.1:8000
|
||||||
|
depends_on:
|
||||||
|
- backend
|
||||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,6 @@
|
|||||||
|
.next
|
||||||
|
node_modules
|
||||||
|
.env.local
|
||||||
|
.env.*.local
|
||||||
|
npm-debug.log*
|
||||||
|
Dockerfile*
|
||||||
@@ -0,0 +1 @@
|
|||||||
|
NEXT_PUBLIC_API_URL=http://127.0.0.1:8000
|
||||||
@@ -0,0 +1,24 @@
|
|||||||
|
FROM node:20-alpine AS deps
|
||||||
|
WORKDIR /app
|
||||||
|
COPY package.json package-lock.json* ./
|
||||||
|
RUN npm ci
|
||||||
|
|
||||||
|
FROM node:20-alpine AS builder
|
||||||
|
WORKDIR /app
|
||||||
|
COPY --from=deps /app/node_modules ./node_modules
|
||||||
|
COPY . .
|
||||||
|
ARG NEXT_PUBLIC_API_URL=
|
||||||
|
ARG BACKEND_INTERNAL_URL=http://backend:8000
|
||||||
|
ENV NEXT_PUBLIC_API_URL=$NEXT_PUBLIC_API_URL
|
||||||
|
ENV BACKEND_INTERNAL_URL=$BACKEND_INTERNAL_URL
|
||||||
|
RUN npm run build
|
||||||
|
|
||||||
|
FROM node:20-alpine AS runner
|
||||||
|
WORKDIR /app
|
||||||
|
ENV NODE_ENV=production
|
||||||
|
ENV HOSTNAME=0.0.0.0
|
||||||
|
COPY --from=builder /app/.next/standalone ./
|
||||||
|
COPY --from=builder /app/.next/static ./.next/static
|
||||||
|
COPY --from=builder /app/public ./public
|
||||||
|
EXPOSE 3000
|
||||||
|
CMD ["node", "server.js"]
|
||||||
@@ -0,0 +1,65 @@
|
|||||||
|
"use client";
|
||||||
|
|
||||||
|
import { useEffect, useState } from "react";
|
||||||
|
import Link from "next/link";
|
||||||
|
import ReactMarkdown from "react-markdown";
|
||||||
|
import remarkGfm from "remark-gfm";
|
||||||
|
import { api, DocumentItem } from "@/lib/api";
|
||||||
|
import { Shell } from "@/components/Shell";
|
||||||
|
import { Button, Card } from "@/components/ui";
|
||||||
|
|
||||||
|
export default function DocumentPage({ params }: { params: { documentId: string } }) {
|
||||||
|
const documentId = Number(params.documentId);
|
||||||
|
const [document, setDocument] = useState<DocumentItem | null>(null);
|
||||||
|
const [content, setContent] = useState("");
|
||||||
|
const [savedContent, setSavedContent] = useState("");
|
||||||
|
const [status, setStatus] = useState("");
|
||||||
|
const dirty = content !== savedContent;
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
Promise.all([api.document(documentId), api.markdown(documentId)]).then(([doc, markdown]) => {
|
||||||
|
setDocument(doc); setContent(markdown); setSavedContent(markdown);
|
||||||
|
}).catch((err) => setStatus(err.message));
|
||||||
|
}, [documentId]);
|
||||||
|
|
||||||
|
async function save() {
|
||||||
|
setStatus("Guardando...");
|
||||||
|
await api.saveMarkdown(documentId, content);
|
||||||
|
setSavedContent(content);
|
||||||
|
setStatus("Guardado");
|
||||||
|
}
|
||||||
|
|
||||||
|
async function copy() {
|
||||||
|
await navigator.clipboard.writeText(content);
|
||||||
|
setStatus("Copiado al portapapeles");
|
||||||
|
}
|
||||||
|
|
||||||
|
return (
|
||||||
|
<Shell>
|
||||||
|
<Link href="/" className="text-sm text-blue-300">Volver al dashboard</Link>
|
||||||
|
<div className="mt-4 flex flex-wrap items-center justify-between gap-4">
|
||||||
|
<div>
|
||||||
|
<h1 className="text-3xl font-bold">{document?.title || "Documento"}</h1>
|
||||||
|
<p className="mt-1 text-sm text-slate-400">{dirty ? "Cambios sin guardar" : "Sin cambios pendientes"} {status && `- ${status}`}</p>
|
||||||
|
</div>
|
||||||
|
<div className="flex flex-wrap gap-2">
|
||||||
|
<Button onClick={save} disabled={!dirty}>Guardar</Button>
|
||||||
|
<Button className="bg-slate-700 hover:bg-slate-600" onClick={copy}>Copiar</Button>
|
||||||
|
<a className="rounded-xl border border-white/10 px-4 py-2 hover:bg-white/10" href={api.downloadDocumentUrl(documentId)}>Descargar .md</a>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div className="mt-6 grid gap-6 xl:grid-cols-2">
|
||||||
|
<Card className="p-0">
|
||||||
|
<div className="border-b border-white/10 px-4 py-3 font-medium">Editor Markdown</div>
|
||||||
|
<textarea value={content} onChange={(event) => setContent(event.target.value)} className="h-[72vh] w-full resize-none rounded-b-2xl bg-slate-950 p-4 font-mono text-sm leading-6 outline-none" />
|
||||||
|
</Card>
|
||||||
|
<Card className="overflow-auto">
|
||||||
|
<div className="mb-4 font-medium">Vista previa</div>
|
||||||
|
<article className="prose prose-invert prose-slate">
|
||||||
|
<ReactMarkdown remarkPlugins={[remarkGfm]}>{content}</ReactMarkdown>
|
||||||
|
</article>
|
||||||
|
</Card>
|
||||||
|
</div>
|
||||||
|
</Shell>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -0,0 +1,20 @@
|
|||||||
|
@tailwind base;
|
||||||
|
@tailwind components;
|
||||||
|
@tailwind utilities;
|
||||||
|
|
||||||
|
:root {
|
||||||
|
color-scheme: dark;
|
||||||
|
}
|
||||||
|
|
||||||
|
body {
|
||||||
|
background: #020617;
|
||||||
|
color: #e5e7eb;
|
||||||
|
}
|
||||||
|
|
||||||
|
textarea, input, button {
|
||||||
|
font: inherit;
|
||||||
|
}
|
||||||
|
|
||||||
|
.prose {
|
||||||
|
max-width: none;
|
||||||
|
}
|
||||||
@@ -0,0 +1,15 @@
|
|||||||
|
import type { Metadata } from "next";
|
||||||
|
import "./globals.css";
|
||||||
|
|
||||||
|
export const metadata: Metadata = {
|
||||||
|
title: "Knowledge Station",
|
||||||
|
description: "Procesa clases, documentos y transcripciones a Markdown.",
|
||||||
|
};
|
||||||
|
|
||||||
|
export default function RootLayout({ children }: { children: React.ReactNode }) {
|
||||||
|
return (
|
||||||
|
<html lang="es" className="dark">
|
||||||
|
<body>{children}</body>
|
||||||
|
</html>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -0,0 +1,87 @@
|
|||||||
|
"use client";
|
||||||
|
|
||||||
|
import Link from "next/link";
|
||||||
|
import { FormEvent, useEffect, useState } from "react";
|
||||||
|
import { api, SettingsStatus, Subject } from "@/lib/api";
|
||||||
|
import { Shell } from "@/components/Shell";
|
||||||
|
import { Button, Card, Input, Label } from "@/components/ui";
|
||||||
|
|
||||||
|
export default function DashboardPage() {
|
||||||
|
const [subjects, setSubjects] = useState<Subject[]>([]);
|
||||||
|
const [settings, setSettings] = useState<SettingsStatus | null>(null);
|
||||||
|
const [name, setName] = useState("");
|
||||||
|
const [loading, setLoading] = useState(true);
|
||||||
|
const [error, setError] = useState("");
|
||||||
|
|
||||||
|
async function load() {
|
||||||
|
setError("");
|
||||||
|
try {
|
||||||
|
const [subjectList, status] = await Promise.all([api.subjects(), api.settings()]);
|
||||||
|
setSubjects(subjectList);
|
||||||
|
setSettings(status);
|
||||||
|
} catch (err) {
|
||||||
|
setError(err instanceof Error ? err.message : "No se pudo conectar con la API");
|
||||||
|
} finally {
|
||||||
|
setLoading(false);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
useEffect(() => { load(); }, []);
|
||||||
|
|
||||||
|
async function createSubject(event: FormEvent) {
|
||||||
|
event.preventDefault();
|
||||||
|
if (!name.trim()) return;
|
||||||
|
await api.createSubject(name.trim());
|
||||||
|
setName("");
|
||||||
|
await load();
|
||||||
|
}
|
||||||
|
|
||||||
|
return (
|
||||||
|
<Shell>
|
||||||
|
<div className="grid gap-6 lg:grid-cols-[1fr_360px]">
|
||||||
|
<section>
|
||||||
|
<div className="mb-6">
|
||||||
|
<p className="text-sm uppercase tracking-[0.25em] text-blue-300">Plataforma personal</p>
|
||||||
|
<h1 className="mt-2 text-4xl font-bold tracking-tight">Materias</h1>
|
||||||
|
<p className="mt-3 max-w-2xl text-slate-400">Organiza clases, PDFs, audios y videos por materia y semana. Todo termina en Markdown listo para ingesta en LLM.</p>
|
||||||
|
</div>
|
||||||
|
|
||||||
|
{error && <Card className="mb-4 border-red-500/30 bg-red-500/10 text-red-200">{error}</Card>}
|
||||||
|
{loading ? <Card>Cargando...</Card> : (
|
||||||
|
<div className="grid gap-4 md:grid-cols-2">
|
||||||
|
{subjects.map((subject) => (
|
||||||
|
<Link key={subject.id} href={`/subjects/${subject.id}`}>
|
||||||
|
<Card className="transition hover:border-blue-400/40 hover:bg-blue-500/10">
|
||||||
|
<div className="text-xl font-semibold">{subject.name}</div>
|
||||||
|
<div className="mt-2 text-sm text-slate-400">{subject.slug}</div>
|
||||||
|
<div className="mt-5 text-sm text-blue-300">Abrir materia</div>
|
||||||
|
</Card>
|
||||||
|
</Link>
|
||||||
|
))}
|
||||||
|
{subjects.length === 0 && <Card className="md:col-span-2">No hay materias todavia. Crea la primera para empezar.</Card>}
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<aside className="space-y-4">
|
||||||
|
<Card>
|
||||||
|
<h2 className="text-lg font-semibold">Nueva materia</h2>
|
||||||
|
<form className="mt-4 space-y-3" onSubmit={createSubject}>
|
||||||
|
<Label>Nombre</Label>
|
||||||
|
<Input value={name} onChange={(event) => setName(event.target.value)} placeholder="Ej. Comportamiento Organizacional" />
|
||||||
|
<Button className="w-full" disabled={!name.trim()}>Crear materia</Button>
|
||||||
|
</form>
|
||||||
|
</Card>
|
||||||
|
|
||||||
|
<Card>
|
||||||
|
<h2 className="text-lg font-semibold">Servicios</h2>
|
||||||
|
<div className="mt-4 space-y-3 text-sm">
|
||||||
|
<div className="flex justify-between"><span>Deepgram</span><span className={settings?.deepgram_configured ? "text-emerald-300" : "text-amber-300"}>{settings?.deepgram_configured ? "Configurado" : "Falta key"}</span></div>
|
||||||
|
<div className="flex justify-between"><span>Mistral OCR</span><span className={settings?.mistral_configured ? "text-emerald-300" : "text-amber-300"}>{settings?.mistral_configured ? "Configurado" : "Falta key"}</span></div>
|
||||||
|
</div>
|
||||||
|
</Card>
|
||||||
|
</aside>
|
||||||
|
</div>
|
||||||
|
</Shell>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -0,0 +1,51 @@
|
|||||||
|
"use client";
|
||||||
|
|
||||||
|
import { useEffect, useState } from "react";
|
||||||
|
import { api, API_URL, SettingsStatus } from "@/lib/api";
|
||||||
|
import { Shell } from "@/components/Shell";
|
||||||
|
import { Card } from "@/components/ui";
|
||||||
|
|
||||||
|
export default function SettingsPage() {
|
||||||
|
const [settings, setSettings] = useState<SettingsStatus | null>(null);
|
||||||
|
const [apiOk, setApiOk] = useState(false);
|
||||||
|
const [error, setError] = useState("");
|
||||||
|
|
||||||
|
useEffect(() => {
|
||||||
|
Promise.all([api.health(), api.settings()]).then(([, status]) => {
|
||||||
|
setApiOk(true); setSettings(status);
|
||||||
|
}).catch((err) => setError(err.message));
|
||||||
|
}, []);
|
||||||
|
|
||||||
|
return (
|
||||||
|
<Shell>
|
||||||
|
<h1 className="text-4xl font-bold">Configuracion</h1>
|
||||||
|
<div className="mt-6 grid gap-4 lg:grid-cols-2">
|
||||||
|
<Card>
|
||||||
|
<h2 className="text-xl font-semibold">Estado</h2>
|
||||||
|
<div className="mt-4 space-y-3 text-sm">
|
||||||
|
<Row label="API" value={apiOk ? "Activa" : "No disponible"} ok={apiOk} />
|
||||||
|
<Row label="Deepgram" value={settings?.deepgram_configured ? "Configurado" : "Falta DEEPGRAM_API_KEY"} ok={!!settings?.deepgram_configured} />
|
||||||
|
<Row label="Mistral" value={settings?.mistral_configured ? "Configurado" : "Falta MISTRAL_API_KEY"} ok={!!settings?.mistral_configured} />
|
||||||
|
</div>
|
||||||
|
{error && <p className="mt-4 text-sm text-red-300">{error}</p>}
|
||||||
|
</Card>
|
||||||
|
<Card>
|
||||||
|
<h2 className="text-xl font-semibold">Rutas</h2>
|
||||||
|
<div className="mt-4 space-y-3 text-sm text-slate-300">
|
||||||
|
<p><span className="text-slate-500">API:</span> {API_URL}</p>
|
||||||
|
<p><span className="text-slate-500">Datos:</span> {settings?.data_dir || "-"}</p>
|
||||||
|
<p><span className="text-slate-500">SQLite:</span> {settings?.database_path || "-"}</p>
|
||||||
|
</div>
|
||||||
|
</Card>
|
||||||
|
</div>
|
||||||
|
<Card className="mt-4">
|
||||||
|
<h2 className="text-xl font-semibold">API Keys</h2>
|
||||||
|
<p className="mt-3 text-slate-400">Configura las claves en el archivo <code className="rounded bg-slate-900 px-2 py-1">.env</code>. No se muestran ni editan desde la UI para evitar pegarlas accidentalmente en capturas o commits.</p>
|
||||||
|
</Card>
|
||||||
|
</Shell>
|
||||||
|
);
|
||||||
|
}
|
||||||
|
|
||||||
|
function Row({ label, value, ok }: { label: string; value: string; ok: boolean }) {
|
||||||
|
return <div className="flex justify-between gap-4"><span>{label}</span><span className={ok ? "text-emerald-300" : "text-amber-300"}>{value}</span></div>;
|
||||||
|
}
|
||||||
@@ -0,0 +1,121 @@
|
|||||||
|
"use client";
|
||||||
|
|
||||||
|
import Link from "next/link";
|
||||||
|
import { FormEvent, useEffect, useState } from "react";
|
||||||
|
import { api, Subject, Week } from "@/lib/api";
|
||||||
|
import { Shell } from "@/components/Shell";
|
||||||
|
import { Button, Card, Input, Label } from "@/components/ui";
|
||||||
|
|
||||||
|
export default function SubjectPage({ params }: { params: { subjectId: string } }) {
|
||||||
|
const subjectId = Number(params.subjectId);
|
||||||
|
const [subject, setSubject] = useState<Subject | null>(null);
|
||||||
|
const [weeks, setWeeks] = useState<Week[]>([]);
|
||||||
|
const [number, setNumber] = useState(1);
|
||||||
|
const [title, setTitle] = useState("");
|
||||||
|
const [editingWeek, setEditingWeek] = useState<Week | null>(null);
|
||||||
|
const [editNumber, setEditNumber] = useState(1);
|
||||||
|
const [editTitle, setEditTitle] = useState("");
|
||||||
|
const [error, setError] = useState("");
|
||||||
|
|
||||||
|
async function load() {
|
||||||
|
try {
|
||||||
|
const [subjectData, weekList] = await Promise.all([api.subject(subjectId), api.weeks(subjectId)]);
|
||||||
|
setSubject(subjectData);
|
||||||
|
setWeeks(weekList);
|
||||||
|
setNumber((weekList.at(-1)?.number || 0) + 1);
|
||||||
|
} catch (err) {
|
||||||
|
setError(err instanceof Error ? err.message : "No se pudo cargar la materia");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
useEffect(() => { load(); }, [subjectId]);
|
||||||
|
|
||||||
|
async function createWeek(event: FormEvent) {
|
||||||
|
event.preventDefault();
|
||||||
|
try {
|
||||||
|
setError("");
|
||||||
|
await api.createWeek(subjectId, number, title);
|
||||||
|
setTitle("");
|
||||||
|
await load();
|
||||||
|
} catch (err) {
|
||||||
|
setError(err instanceof Error ? err.message : "No se pudo crear la semana");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function startEdit(week: Week) {
|
||||||
|
setEditingWeek(week);
|
||||||
|
setEditNumber(week.number);
|
||||||
|
setEditTitle(week.title || "");
|
||||||
|
setError("");
|
||||||
|
}
|
||||||
|
|
||||||
|
async function saveEdit(event: FormEvent) {
|
||||||
|
event.preventDefault();
|
||||||
|
if (!editingWeek) return;
|
||||||
|
try {
|
||||||
|
setError("");
|
||||||
|
await api.updateWeek(subjectId, editingWeek.number, editNumber, editTitle);
|
||||||
|
setEditingWeek(null);
|
||||||
|
await load();
|
||||||
|
} catch (err) {
|
||||||
|
setError(err instanceof Error ? err.message : "No se pudo editar la semana");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
return (
|
||||||
|
<Shell>
|
||||||
|
<Link href="/" className="text-sm text-blue-300">Volver a materias</Link>
|
||||||
|
<div className="mt-4 grid gap-6 lg:grid-cols-[1fr_360px]">
|
||||||
|
<section>
|
||||||
|
<h1 className="text-4xl font-bold tracking-tight">{subject?.name || "Materia"}</h1>
|
||||||
|
<p className="mt-2 text-slate-400">Divide el conocimiento por semanas y sube fuentes a cada una.</p>
|
||||||
|
{error && <Card className="mt-4 border-red-500/30 bg-red-500/10 text-red-200">{error}</Card>}
|
||||||
|
<div className="mt-6 grid gap-4 md:grid-cols-2">
|
||||||
|
{weeks.map((week) => (
|
||||||
|
<Card key={week.id} className="transition hover:border-blue-400/40 hover:bg-blue-500/10">
|
||||||
|
<div className="text-2xl font-semibold">Semana {week.number}</div>
|
||||||
|
<div className="mt-2 text-slate-400">{week.title || "Sin titulo"}</div>
|
||||||
|
<div className="mt-5 flex flex-wrap gap-2 text-sm">
|
||||||
|
<Link className="rounded-lg bg-blue-600 px-3 py-2 font-medium text-white hover:bg-blue-500" href={`/subjects/${subjectId}/weeks/${week.number}`}>Abrir semana</Link>
|
||||||
|
<button className="rounded-lg border border-white/10 px-3 py-2 text-slate-200 hover:bg-white/10" type="button" onClick={() => startEdit(week)}>Editar</button>
|
||||||
|
</div>
|
||||||
|
</Card>
|
||||||
|
))}
|
||||||
|
{weeks.length === 0 && <Card className="md:col-span-2">Todavia no hay semanas.</Card>}
|
||||||
|
</div>
|
||||||
|
</section>
|
||||||
|
<aside>
|
||||||
|
<Card>
|
||||||
|
{editingWeek ? (
|
||||||
|
<>
|
||||||
|
<h2 className="text-lg font-semibold">Editar semana</h2>
|
||||||
|
<p className="mt-1 text-sm text-slate-400">Cambia el numero o titulo. Si cambias el numero, se renombra la carpeta de la semana.</p>
|
||||||
|
<form className="mt-4 space-y-3" onSubmit={saveEdit}>
|
||||||
|
<Label>Numero</Label>
|
||||||
|
<Input type="number" min={1} value={editNumber} onChange={(event) => setEditNumber(Number(event.target.value))} />
|
||||||
|
<Label>Titulo</Label>
|
||||||
|
<Input value={editTitle} onChange={(event) => setEditTitle(event.target.value)} placeholder="Ej. Grupos y equipos" />
|
||||||
|
<div className="flex gap-2">
|
||||||
|
<Button className="flex-1">Guardar</Button>
|
||||||
|
<button className="flex-1 rounded-xl border border-white/10 px-4 py-2 hover:bg-white/10" type="button" onClick={() => setEditingWeek(null)}>Cancelar</button>
|
||||||
|
</div>
|
||||||
|
</form>
|
||||||
|
</>
|
||||||
|
) : (
|
||||||
|
<>
|
||||||
|
<h2 className="text-lg font-semibold">Nueva semana</h2>
|
||||||
|
<form className="mt-4 space-y-3" onSubmit={createWeek}>
|
||||||
|
<Label>Numero</Label>
|
||||||
|
<Input type="number" min={1} value={number} onChange={(event) => setNumber(Number(event.target.value))} />
|
||||||
|
<Label>Titulo opcional</Label>
|
||||||
|
<Input value={title} onChange={(event) => setTitle(event.target.value)} placeholder="Ej. Grupos y equipos" />
|
||||||
|
<Button className="w-full">Crear semana</Button>
|
||||||
|
</form>
|
||||||
|
</>
|
||||||
|
)}
|
||||||
|
</Card>
|
||||||
|
</aside>
|
||||||
|
</div>
|
||||||
|
</Shell>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -0,0 +1,343 @@
|
|||||||
|
"use client";
|
||||||
|
|
||||||
|
import Link from "next/link";
|
||||||
|
import { ChangeEvent, DragEvent, useEffect, useMemo, useState } from "react";
|
||||||
|
import { api, DocumentItem, Job, Source, Subject, Week } from "@/lib/api";
|
||||||
|
import { Shell } from "@/components/Shell";
|
||||||
|
import { Button, Card, ConfirmDialog, StatusBadge } from "@/components/ui";
|
||||||
|
|
||||||
|
type PendingDelete =
|
||||||
|
| { type: "document"; item: DocumentItem }
|
||||||
|
| { type: "source"; item: Source };
|
||||||
|
|
||||||
|
export default function WeekPage({ params }: { params: { subjectId: string; weekNumber: string } }) {
|
||||||
|
const subjectId = Number(params.subjectId);
|
||||||
|
const weekNumber = Number(params.weekNumber);
|
||||||
|
const [subject, setSubject] = useState<Subject | null>(null);
|
||||||
|
const [week, setWeek] = useState<Week | null>(null);
|
||||||
|
const [sources, setSources] = useState<Source[]>([]);
|
||||||
|
const [jobs, setJobs] = useState<Job[]>([]);
|
||||||
|
const [documents, setDocuments] = useState<DocumentItem[]>([]);
|
||||||
|
const [pdfMethod, setPdfMethod] = useState<"standard" | "mistral_ocr">("standard");
|
||||||
|
const [pageMode, setPageMode] = useState<"all" | "ranges">("all");
|
||||||
|
const [pageRanges, setPageRanges] = useState("");
|
||||||
|
const [pendingFiles, setPendingFiles] = useState<File[]>([]);
|
||||||
|
const [uploading, setUploading] = useState(false);
|
||||||
|
const [deletingId, setDeletingId] = useState<string | null>(null);
|
||||||
|
const [reprocessingId, setReprocessingId] = useState<number | null>(null);
|
||||||
|
const [pendingDelete, setPendingDelete] = useState<PendingDelete | null>(null);
|
||||||
|
const [dragActive, setDragActive] = useState(false);
|
||||||
|
const [error, setError] = useState("");
|
||||||
|
|
||||||
|
const activeJobs = useMemo(() => jobs.some((job) => ["queued", "processing"].includes(job.status)), [jobs]);
|
||||||
|
const pendingPdfCount = useMemo(() => pendingFiles.filter(isPdfFile).length, [pendingFiles]);
|
||||||
|
|
||||||
|
async function load() {
|
||||||
|
const [subjectData, weekData, sourceList, jobList, documentList] = await Promise.all([
|
||||||
|
api.subject(subjectId), api.week(subjectId, weekNumber), api.sources(subjectId, weekNumber), api.jobs(subjectId, weekNumber), api.documents(subjectId, weekNumber),
|
||||||
|
]);
|
||||||
|
setSubject(subjectData); setWeek(weekData); setSources(sourceList); setJobs(jobList); setDocuments(documentList);
|
||||||
|
}
|
||||||
|
|
||||||
|
useEffect(() => { load().catch((err) => setError(err.message)); }, [subjectId, weekNumber]);
|
||||||
|
useEffect(() => {
|
||||||
|
if (!activeJobs) return;
|
||||||
|
const timer = window.setInterval(() => load().catch((err) => setError(err.message)), 2500);
|
||||||
|
return () => window.clearInterval(timer);
|
||||||
|
}, [activeJobs]);
|
||||||
|
|
||||||
|
async function uploadFileList(files: File[]) {
|
||||||
|
if (!files.length) return;
|
||||||
|
setUploading(true); setError("");
|
||||||
|
try {
|
||||||
|
const ranges = pageMode === "ranges" ? pageRanges : undefined;
|
||||||
|
for (const file of files) await api.upload(subjectId, weekNumber, file, pdfMethod === "mistral_ocr", ranges);
|
||||||
|
await load();
|
||||||
|
} catch (err) {
|
||||||
|
setError(err instanceof Error ? err.message : "No se pudo subir archivo");
|
||||||
|
} finally {
|
||||||
|
setUploading(false);
|
||||||
|
setDragActive(false);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function isPdfFile(file: File) {
|
||||||
|
return file.type === "application/pdf" || file.name.toLowerCase().endsWith(".pdf");
|
||||||
|
}
|
||||||
|
|
||||||
|
async function prepareFiles(files: File[]) {
|
||||||
|
if (!files.length) return;
|
||||||
|
setError("");
|
||||||
|
setDragActive(false);
|
||||||
|
if (files.some(isPdfFile)) {
|
||||||
|
setPendingFiles(files);
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
await uploadFileList(files);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function confirmPendingUpload() {
|
||||||
|
const files = pendingFiles;
|
||||||
|
setPendingFiles([]);
|
||||||
|
await uploadFileList(files);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function uploadFiles(event: ChangeEvent<HTMLInputElement>) {
|
||||||
|
const files = Array.from(event.target.files || []);
|
||||||
|
try {
|
||||||
|
await prepareFiles(files);
|
||||||
|
} finally {
|
||||||
|
event.target.value = "";
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function handleDrag(event: DragEvent<HTMLDivElement>) {
|
||||||
|
event.preventDefault();
|
||||||
|
event.stopPropagation();
|
||||||
|
if (uploading) return;
|
||||||
|
if (event.type === "dragenter" || event.type === "dragover") setDragActive(true);
|
||||||
|
if (event.type === "dragleave") setDragActive(false);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function handleDrop(event: DragEvent<HTMLDivElement>) {
|
||||||
|
event.preventDefault();
|
||||||
|
event.stopPropagation();
|
||||||
|
if (uploading) return;
|
||||||
|
const files = Array.from(event.dataTransfer.files || []);
|
||||||
|
await prepareFiles(files);
|
||||||
|
}
|
||||||
|
|
||||||
|
async function deleteDocument(document: DocumentItem) {
|
||||||
|
setDeletingId(`document-${document.id}`); setError("");
|
||||||
|
try {
|
||||||
|
await api.deleteDocument(document.id);
|
||||||
|
await load();
|
||||||
|
setPendingDelete(null);
|
||||||
|
} catch (err) {
|
||||||
|
setError(err instanceof Error ? err.message : "No se pudo eliminar el documento");
|
||||||
|
} finally {
|
||||||
|
setDeletingId(null);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function deleteSource(source: Source) {
|
||||||
|
setDeletingId(`source-${source.id}`); setError("");
|
||||||
|
try {
|
||||||
|
await api.deleteSource(source.id);
|
||||||
|
await load();
|
||||||
|
setPendingDelete(null);
|
||||||
|
} catch (err) {
|
||||||
|
setError(err instanceof Error ? err.message : "No se pudo eliminar la fuente");
|
||||||
|
} finally {
|
||||||
|
setDeletingId(null);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function reprocessSource(source: Source) {
|
||||||
|
setReprocessingId(source.id); setError("");
|
||||||
|
try {
|
||||||
|
const ranges = source.source_type === "pdf" && pageMode === "ranges" ? pageRanges : undefined;
|
||||||
|
await api.reprocessSource(source.id, source.source_type === "pdf" && pdfMethod === "mistral_ocr", ranges);
|
||||||
|
await load();
|
||||||
|
} catch (err) {
|
||||||
|
setError(err instanceof Error ? err.message : "No se pudo reprocesar la fuente");
|
||||||
|
} finally {
|
||||||
|
setReprocessingId(null);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function formatFileSize(size?: number | null) {
|
||||||
|
if (!size) return "tamano pendiente";
|
||||||
|
if (size < 1024 * 1024) return `${Math.round(size / 1024)} KB`;
|
||||||
|
return `${(size / (1024 * 1024)).toFixed(1)} MB`;
|
||||||
|
}
|
||||||
|
|
||||||
|
function sourceProcessingLabel(source: Source) {
|
||||||
|
if (source.document_id) return `Genero: ${source.document_title}`;
|
||||||
|
if (["queued", "processing"].includes(source.status)) return "Procesamiento en curso";
|
||||||
|
if (source.status === "failed") return "Fallo el procesamiento";
|
||||||
|
return "Sin Markdown generado";
|
||||||
|
}
|
||||||
|
|
||||||
|
function confirmPendingDelete() {
|
||||||
|
if (!pendingDelete) return;
|
||||||
|
if (pendingDelete.type === "document") void deleteDocument(pendingDelete.item);
|
||||||
|
if (pendingDelete.type === "source") void deleteSource(pendingDelete.item);
|
||||||
|
}
|
||||||
|
|
||||||
|
const deleteDialog = pendingDelete ? {
|
||||||
|
title: pendingDelete.type === "document" ? "Eliminar Markdown procesado" : "Eliminar fuente",
|
||||||
|
description: pendingDelete.type === "document"
|
||||||
|
? `Se eliminara el archivo Markdown "${pendingDelete.item.title}". La fuente original quedara disponible.`
|
||||||
|
: `Se eliminara la fuente "${pendingDelete.item.original_name}" junto con sus trabajos y Markdown procesado asociado.`,
|
||||||
|
} : null;
|
||||||
|
|
||||||
|
return (
|
||||||
|
<Shell>
|
||||||
|
<Link href={`/subjects/${subjectId}`} className="text-sm text-blue-300">Volver a {subject?.name || "materia"}</Link>
|
||||||
|
<div className="mt-4 grid gap-6 lg:grid-cols-[1fr_380px]">
|
||||||
|
<section className="space-y-6">
|
||||||
|
<div>
|
||||||
|
<h1 className="text-4xl font-bold">Semana {weekNumber}</h1>
|
||||||
|
<p className="mt-2 text-slate-400">{week?.title || "Sube fuentes y genera Markdown procesado."}</p>
|
||||||
|
</div>
|
||||||
|
{error && <Card className="border-red-500/30 bg-red-500/10 text-red-200">{error}</Card>}
|
||||||
|
|
||||||
|
<Card>
|
||||||
|
<div className="flex flex-wrap items-center justify-between gap-4">
|
||||||
|
<div>
|
||||||
|
<h2 className="text-xl font-semibold">Subir archivos</h2>
|
||||||
|
<p className="mt-1 text-sm text-slate-400">PDF, DOCX, TXT, Markdown, audio y video.</p>
|
||||||
|
</div>
|
||||||
|
<label className="cursor-pointer rounded-xl bg-blue-600 px-4 py-2 font-medium hover:bg-blue-500">
|
||||||
|
{uploading ? "Subiendo..." : "Seleccionar archivos"}
|
||||||
|
<input className="hidden" type="file" multiple onChange={uploadFiles} disabled={uploading} />
|
||||||
|
</label>
|
||||||
|
</div>
|
||||||
|
<div
|
||||||
|
className={`mt-5 rounded-2xl border-2 border-dashed p-8 text-center transition ${dragActive ? "border-blue-400 bg-blue-500/15" : "border-white/15 bg-slate-950/50 hover:border-blue-400/50"} ${uploading ? "opacity-70" : ""}`}
|
||||||
|
onDragEnter={handleDrag}
|
||||||
|
onDragOver={handleDrag}
|
||||||
|
onDragLeave={handleDrag}
|
||||||
|
onDrop={handleDrop}
|
||||||
|
>
|
||||||
|
<div className="mx-auto grid h-14 w-14 place-items-center rounded-2xl bg-blue-500/15 text-2xl">↑</div>
|
||||||
|
<p className="mt-4 font-medium">Arrastra archivos aqui</p>
|
||||||
|
<p className="mt-2 text-sm text-slate-400">Puedes soltar varios archivos a la vez. Se procesaran en esta semana.</p>
|
||||||
|
<p className="mt-3 text-xs text-slate-500">Soporta PDF, DOCX, TXT, Markdown, audio y video.</p>
|
||||||
|
{uploading && <p className="mt-4 text-sm text-blue-300">Subiendo archivos...</p>}
|
||||||
|
</div>
|
||||||
|
{pendingFiles.length > 0 && (
|
||||||
|
<div className="mt-5 rounded-2xl border border-blue-400/40 bg-blue-500/10 p-4">
|
||||||
|
<div className="flex flex-wrap items-start justify-between gap-3">
|
||||||
|
<div>
|
||||||
|
<h3 className="font-medium">Configurar PDFs antes de subir</h3>
|
||||||
|
<p className="mt-1 text-sm text-slate-300">
|
||||||
|
Detecte {pendingPdfCount} PDF en {pendingFiles.length} archivo(s). Elige como procesarlos antes de continuar.
|
||||||
|
</p>
|
||||||
|
</div>
|
||||||
|
<button className="rounded-lg border border-white/10 px-3 py-1.5 text-sm hover:bg-white/10" type="button" onClick={() => setPendingFiles([])}>Cancelar</button>
|
||||||
|
</div>
|
||||||
|
<div className="mt-4 grid gap-4 md:grid-cols-2">
|
||||||
|
<div>
|
||||||
|
<label className="text-sm font-medium text-slate-300">Metodo de procesamiento</label>
|
||||||
|
<select
|
||||||
|
className="mt-2 w-full rounded-xl border border-white/10 bg-slate-900 px-3 py-2 text-sm outline-none ring-blue-500/30 focus:ring-4"
|
||||||
|
value={pdfMethod}
|
||||||
|
onChange={(event) => setPdfMethod(event.target.value as "standard" | "mistral_ocr")}
|
||||||
|
>
|
||||||
|
<option value="standard">Extraccion estandar de texto</option>
|
||||||
|
<option value="mistral_ocr">Mistral OCR (IA)</option>
|
||||||
|
</select>
|
||||||
|
</div>
|
||||||
|
<div>
|
||||||
|
<label className="text-sm font-medium text-slate-300">Paginas a procesar</label>
|
||||||
|
<select
|
||||||
|
className="mt-2 w-full rounded-xl border border-white/10 bg-slate-900 px-3 py-2 text-sm outline-none ring-blue-500/30 focus:ring-4"
|
||||||
|
value={pageMode}
|
||||||
|
onChange={(event) => setPageMode(event.target.value as "all" | "ranges")}
|
||||||
|
>
|
||||||
|
<option value="all">Todas las paginas</option>
|
||||||
|
<option value="ranges">Rangos especificos</option>
|
||||||
|
</select>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
{pageMode === "ranges" && (
|
||||||
|
<div className="mt-4">
|
||||||
|
<label className="text-sm font-medium text-slate-300">Rangos</label>
|
||||||
|
<input
|
||||||
|
className="mt-2 w-full rounded-xl border border-white/10 bg-slate-900 px-3 py-2 text-sm outline-none ring-blue-500/30 focus:ring-4"
|
||||||
|
value={pageRanges}
|
||||||
|
onChange={(event) => setPageRanges(event.target.value)}
|
||||||
|
placeholder="Ej: 1-5, 8, 10-12"
|
||||||
|
/>
|
||||||
|
<p className="mt-2 text-xs text-slate-500">Separa paginas o rangos con comas. Esta seleccion se aplicara a los PDFs pendientes.</p>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
<div className="mt-4 flex flex-wrap gap-2">
|
||||||
|
<Button onClick={confirmPendingUpload} disabled={uploading || (pageMode === "ranges" && !pageRanges.trim())}>
|
||||||
|
{uploading ? "Subiendo..." : "Procesar y subir archivos"}
|
||||||
|
</Button>
|
||||||
|
<p className="self-center text-xs text-slate-400">Los archivos que no sean PDF se subiran junto con esta tanda.</p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
)}
|
||||||
|
</Card>
|
||||||
|
|
||||||
|
<Card>
|
||||||
|
<div className="mb-4 flex flex-wrap items-center justify-between gap-3">
|
||||||
|
<h2 className="text-xl font-semibold">Documentos Markdown</h2>
|
||||||
|
<div className="flex flex-wrap gap-2">
|
||||||
|
<a className="rounded-xl border border-white/10 px-3 py-2 text-sm hover:bg-white/10" href={api.exportWeekUrl(subjectId, weekNumber)}>Exportar semana (.md unico)</a>
|
||||||
|
<a className="rounded-xl border border-white/10 px-3 py-2 text-sm hover:bg-white/10" href={api.exportWeekZipUrl(subjectId, weekNumber)}>Exportar documentos separados (.zip)</a>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div className="space-y-3">
|
||||||
|
{documents.map((doc) => (
|
||||||
|
<div key={doc.id} className="flex flex-wrap items-center justify-between gap-3 rounded-xl border border-white/10 p-4 hover:bg-white/5">
|
||||||
|
<Link href={`/documents/${doc.id}`} className="min-w-0 flex-1">
|
||||||
|
<div className="font-medium">{doc.title}</div>
|
||||||
|
<div className="mt-1 text-xs text-slate-400">Fuente: {doc.source_original_name || `#${doc.source_id}`}</div>
|
||||||
|
<div className="mt-1 truncate text-xs text-slate-500">{doc.markdown_path}</div>
|
||||||
|
</Link>
|
||||||
|
<Button
|
||||||
|
className="bg-red-600 px-3 py-1.5 text-sm hover:bg-red-500"
|
||||||
|
disabled={deletingId === `document-${doc.id}`}
|
||||||
|
onClick={() => setPendingDelete({ type: "document", item: doc })}
|
||||||
|
>
|
||||||
|
{deletingId === `document-${doc.id}` ? "Eliminando..." : "Eliminar"}
|
||||||
|
</Button>
|
||||||
|
</div>
|
||||||
|
))}
|
||||||
|
{documents.length === 0 && <p className="text-slate-400">Aun no hay documentos procesados.</p>}
|
||||||
|
</div>
|
||||||
|
</Card>
|
||||||
|
</section>
|
||||||
|
|
||||||
|
<aside className="space-y-4">
|
||||||
|
<Card>
|
||||||
|
<h2 className="text-lg font-semibold">Trabajos</h2>
|
||||||
|
<div className="mt-4 space-y-3">
|
||||||
|
{jobs.map((job) => <div key={job.id} className="rounded-xl bg-slate-900 p-3 text-sm"><div className="flex justify-between"><span>#{job.id}</span><StatusBadge status={job.status} /></div><p className="mt-2 text-slate-400">{job.message}</p></div>)}
|
||||||
|
{jobs.length === 0 && <p className="text-sm text-slate-400">Sin trabajos.</p>}
|
||||||
|
</div>
|
||||||
|
</Card>
|
||||||
|
<Card>
|
||||||
|
<h2 className="text-lg font-semibold">Fuentes</h2>
|
||||||
|
<div className="mt-4 space-y-3">
|
||||||
|
{sources.map((source) => {
|
||||||
|
const busy = ["queued", "processing"].includes(source.status);
|
||||||
|
return (
|
||||||
|
<div key={source.id} className="rounded-xl bg-slate-900 p-3 text-sm">
|
||||||
|
<div className="flex items-start justify-between gap-2"><span className="truncate">{source.original_name}</span><StatusBadge status={source.status} /></div>
|
||||||
|
<div className="mt-2 space-y-1 text-xs text-slate-500">
|
||||||
|
<p className={source.document_id ? "text-emerald-300" : source.status === "failed" ? "text-red-300" : "text-amber-300"}>{sourceProcessingLabel(source)}</p>
|
||||||
|
<p>{source.source_type} · {formatFileSize(source.file_size)}{source.processor ? ` · ${source.processor}` : ""}</p>
|
||||||
|
{source.sha256 && <p className="truncate">sha256: {source.sha256.slice(0, 12)}...</p>}
|
||||||
|
{source.page_ranges && <p>paginas: {source.page_ranges}</p>}
|
||||||
|
</div>
|
||||||
|
<div className="mt-3 flex flex-wrap justify-end gap-2">
|
||||||
|
{source.document_id && <Link className="rounded-lg border border-emerald-500/30 px-2 py-1 text-xs text-emerald-200 hover:bg-emerald-500/10" href={`/documents/${source.document_id}`}>Ver Markdown</Link>}
|
||||||
|
<button className="rounded-lg border border-white/10 px-2 py-1 text-xs text-slate-200 hover:bg-white/10 disabled:cursor-not-allowed disabled:opacity-50" disabled={busy || reprocessingId === source.id} onClick={() => reprocessSource(source)}>{reprocessingId === source.id ? "En cola..." : "Reprocesar"}</button>
|
||||||
|
<button className="rounded-lg border border-red-500/30 px-2 py-1 text-xs text-red-200 hover:bg-red-500/10 disabled:cursor-not-allowed disabled:opacity-50" disabled={deletingId === `source-${source.id}` || busy} onClick={() => setPendingDelete({ type: "source", item: source })}>{deletingId === `source-${source.id}` ? "Eliminando..." : "Eliminar"}</button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
})}
|
||||||
|
{sources.length === 0 && <p className="text-sm text-slate-400">Sin fuentes.</p>}
|
||||||
|
</div>
|
||||||
|
</Card>
|
||||||
|
</aside>
|
||||||
|
</div>
|
||||||
|
<ConfirmDialog
|
||||||
|
open={Boolean(deleteDialog)}
|
||||||
|
title={deleteDialog?.title || "Confirmar accion"}
|
||||||
|
description={deleteDialog?.description || ""}
|
||||||
|
confirmLabel="Eliminar"
|
||||||
|
loading={Boolean(deletingId)}
|
||||||
|
onConfirm={confirmPendingDelete}
|
||||||
|
onCancel={() => setPendingDelete(null)}
|
||||||
|
/>
|
||||||
|
</Shell>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -0,0 +1,25 @@
|
|||||||
|
import Link from "next/link";
|
||||||
|
import type { ReactNode } from "react";
|
||||||
|
|
||||||
|
export function Shell({ children }: { children: ReactNode }) {
|
||||||
|
return (
|
||||||
|
<div className="min-h-screen bg-slate-950 text-slate-100">
|
||||||
|
<header className="border-b border-white/10 bg-slate-950/90 backdrop-blur">
|
||||||
|
<div className="mx-auto flex max-w-7xl items-center justify-between px-5 py-4">
|
||||||
|
<Link href="/" className="flex items-center gap-3">
|
||||||
|
<div className="grid h-10 w-10 place-items-center rounded-2xl bg-blue-600 font-bold">KS</div>
|
||||||
|
<div>
|
||||||
|
<div className="font-semibold tracking-tight">Knowledge Station</div>
|
||||||
|
<div className="text-xs text-slate-400">Clases a Markdown para LLM</div>
|
||||||
|
</div>
|
||||||
|
</Link>
|
||||||
|
<nav className="flex items-center gap-2 text-sm text-slate-300">
|
||||||
|
<Link className="rounded-lg px-3 py-2 hover:bg-white/10" href="/">Materias</Link>
|
||||||
|
<Link className="rounded-lg px-3 py-2 hover:bg-white/10" href="/settings">Configuracion</Link>
|
||||||
|
</nav>
|
||||||
|
</div>
|
||||||
|
</header>
|
||||||
|
<main className="mx-auto max-w-7xl px-5 py-8">{children}</main>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -0,0 +1,67 @@
|
|||||||
|
export function Card({ children, className = "" }: { children: React.ReactNode; className?: string }) {
|
||||||
|
return <div className={`rounded-2xl border border-white/10 bg-white/[0.04] p-5 shadow-xl shadow-black/20 ${className}`}>{children}</div>;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function Button({ children, className = "", ...props }: React.ButtonHTMLAttributes<HTMLButtonElement>) {
|
||||||
|
return <button className={`rounded-xl bg-blue-600 px-4 py-2 font-medium text-white hover:bg-blue-500 disabled:cursor-not-allowed disabled:opacity-50 ${className}`} {...props}>{children}</button>;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function Input(props: React.InputHTMLAttributes<HTMLInputElement>) {
|
||||||
|
return <input className="w-full rounded-xl border border-white/10 bg-slate-900 px-4 py-2 text-slate-100 outline-none ring-blue-500/30 focus:ring-4" {...props} />;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function Label({ children }: { children: React.ReactNode }) {
|
||||||
|
return <label className="text-sm font-medium text-slate-300">{children}</label>;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function StatusBadge({ status }: { status: string }) {
|
||||||
|
const map: Record<string, string> = {
|
||||||
|
completed: "bg-emerald-500/15 text-emerald-300",
|
||||||
|
processing: "bg-blue-500/15 text-blue-300",
|
||||||
|
queued: "bg-amber-500/15 text-amber-300",
|
||||||
|
failed: "bg-red-500/15 text-red-300",
|
||||||
|
stored: "bg-slate-500/15 text-slate-300",
|
||||||
|
processed: "bg-emerald-500/15 text-emerald-300",
|
||||||
|
};
|
||||||
|
return <span className={`rounded-full px-2.5 py-1 text-xs font-medium ${map[status] || "bg-slate-500/15 text-slate-300"}`}>{status}</span>;
|
||||||
|
}
|
||||||
|
|
||||||
|
export function ConfirmDialog({
|
||||||
|
open,
|
||||||
|
title,
|
||||||
|
description,
|
||||||
|
confirmLabel = "Confirmar",
|
||||||
|
cancelLabel = "Cancelar",
|
||||||
|
loading = false,
|
||||||
|
onConfirm,
|
||||||
|
onCancel,
|
||||||
|
}: {
|
||||||
|
open: boolean;
|
||||||
|
title: string;
|
||||||
|
description: string;
|
||||||
|
confirmLabel?: string;
|
||||||
|
cancelLabel?: string;
|
||||||
|
loading?: boolean;
|
||||||
|
onConfirm: () => void;
|
||||||
|
onCancel: () => void;
|
||||||
|
}) {
|
||||||
|
if (!open) return null;
|
||||||
|
|
||||||
|
return (
|
||||||
|
<div className="fixed inset-0 z-50 grid place-items-center bg-slate-950/75 p-4 backdrop-blur-sm" role="dialog" aria-modal="true" aria-labelledby="confirm-dialog-title">
|
||||||
|
<div className="w-full max-w-md rounded-2xl border border-white/10 bg-slate-900 p-5 shadow-2xl shadow-black/40">
|
||||||
|
<div className="flex items-start gap-3">
|
||||||
|
<div className="grid h-10 w-10 shrink-0 place-items-center rounded-full bg-red-500/15 text-red-200">!</div>
|
||||||
|
<div>
|
||||||
|
<h2 id="confirm-dialog-title" className="text-lg font-semibold text-white">{title}</h2>
|
||||||
|
<p className="mt-2 text-sm leading-6 text-slate-300">{description}</p>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
<div className="mt-6 flex flex-col-reverse gap-2 sm:flex-row sm:justify-end">
|
||||||
|
<button className="rounded-xl border border-white/10 px-4 py-2 text-slate-200 hover:bg-white/10 disabled:cursor-not-allowed disabled:opacity-50" type="button" disabled={loading} onClick={onCancel}>{cancelLabel}</button>
|
||||||
|
<Button className="bg-red-600 hover:bg-red-500" disabled={loading} onClick={onConfirm}>{loading ? "Eliminando..." : confirmLabel}</Button>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
</div>
|
||||||
|
);
|
||||||
|
}
|
||||||
@@ -0,0 +1,141 @@
|
|||||||
|
export const API_URL = process.env.NEXT_PUBLIC_API_URL ?? "http://127.0.0.1:8000";
|
||||||
|
|
||||||
|
export type Subject = { id: number; name: string; slug: string; created_at: string };
|
||||||
|
export type Week = { id: number; subject_id: number; number: number; title?: string | null; created_at: string };
|
||||||
|
export type Source = {
|
||||||
|
id: number;
|
||||||
|
week_id: number;
|
||||||
|
original_name: string;
|
||||||
|
stored_path: string;
|
||||||
|
source_type: string;
|
||||||
|
status: string;
|
||||||
|
file_size?: number | null;
|
||||||
|
sha256?: string | null;
|
||||||
|
mime_type?: string | null;
|
||||||
|
processor?: string | null;
|
||||||
|
page_ranges?: string | null;
|
||||||
|
processed_at?: string | null;
|
||||||
|
document_id?: number | null;
|
||||||
|
document_title?: string | null;
|
||||||
|
document_markdown_path?: string | null;
|
||||||
|
document_created_at?: string | null;
|
||||||
|
created_at: string;
|
||||||
|
};
|
||||||
|
export type Job = { id: number; status: string; message: string; source_id?: number | null; document_id?: number | null; created_at: string; updated_at: string };
|
||||||
|
export type DocumentItem = {
|
||||||
|
id: number;
|
||||||
|
source_id: number;
|
||||||
|
title: string;
|
||||||
|
markdown_path: string;
|
||||||
|
source_original_name?: string | null;
|
||||||
|
source_type?: string | null;
|
||||||
|
source_status?: string | null;
|
||||||
|
source_processor?: string | null;
|
||||||
|
source_page_ranges?: string | null;
|
||||||
|
created_at: string;
|
||||||
|
};
|
||||||
|
export type SettingsStatus = { mistral_configured: boolean; deepgram_configured: boolean; data_dir: string; database_path: string };
|
||||||
|
|
||||||
|
async function parseResponse<T>(response: Response): Promise<T> {
|
||||||
|
if (!response.ok) {
|
||||||
|
const detail = await response.json().catch(() => null);
|
||||||
|
throw new Error(detail?.detail || `Error ${response.status}`);
|
||||||
|
}
|
||||||
|
return response.json() as Promise<T>;
|
||||||
|
}
|
||||||
|
|
||||||
|
export const api = {
|
||||||
|
async health() {
|
||||||
|
return parseResponse<{ status: string }>(await fetch(`${API_URL}/health`, { cache: "no-store" }));
|
||||||
|
},
|
||||||
|
async settings() {
|
||||||
|
return parseResponse<SettingsStatus>(await fetch(`${API_URL}/settings/status`, { cache: "no-store" }));
|
||||||
|
},
|
||||||
|
async subjects() {
|
||||||
|
return parseResponse<Subject[]>(await fetch(`${API_URL}/subjects`, { cache: "no-store" }));
|
||||||
|
},
|
||||||
|
async subject(id: number) {
|
||||||
|
return parseResponse<Subject>(await fetch(`${API_URL}/subjects/${id}`, { cache: "no-store" }));
|
||||||
|
},
|
||||||
|
async createSubject(name: string) {
|
||||||
|
return parseResponse<Subject>(await fetch(`${API_URL}/subjects`, {
|
||||||
|
method: "POST",
|
||||||
|
headers: { "Content-Type": "application/json" },
|
||||||
|
body: JSON.stringify({ name }),
|
||||||
|
}));
|
||||||
|
},
|
||||||
|
async weeks(subjectId: number) {
|
||||||
|
return parseResponse<Week[]>(await fetch(`${API_URL}/subjects/${subjectId}/weeks`, { cache: "no-store" }));
|
||||||
|
},
|
||||||
|
async week(subjectId: number, weekNumber: number) {
|
||||||
|
return parseResponse<Week>(await fetch(`${API_URL}/subjects/${subjectId}/weeks/${weekNumber}`, { cache: "no-store" }));
|
||||||
|
},
|
||||||
|
async createWeek(subjectId: number, number: number, title: string) {
|
||||||
|
return parseResponse<Week>(await fetch(`${API_URL}/subjects/${subjectId}/weeks`, {
|
||||||
|
method: "POST",
|
||||||
|
headers: { "Content-Type": "application/json" },
|
||||||
|
body: JSON.stringify({ number, title: title || null }),
|
||||||
|
}));
|
||||||
|
},
|
||||||
|
async updateWeek(subjectId: number, currentNumber: number, number: number, title: string) {
|
||||||
|
return parseResponse<Week>(await fetch(`${API_URL}/subjects/${subjectId}/weeks/${currentNumber}`, {
|
||||||
|
method: "PUT",
|
||||||
|
headers: { "Content-Type": "application/json" },
|
||||||
|
body: JSON.stringify({ number, title: title || null }),
|
||||||
|
}));
|
||||||
|
},
|
||||||
|
async sources(subjectId: number, weekNumber: number) {
|
||||||
|
return parseResponse<Source[]>(await fetch(`${API_URL}/subjects/${subjectId}/weeks/${weekNumber}/sources`, { cache: "no-store" }));
|
||||||
|
},
|
||||||
|
async jobs(subjectId: number, weekNumber: number) {
|
||||||
|
return parseResponse<Job[]>(await fetch(`${API_URL}/subjects/${subjectId}/weeks/${weekNumber}/jobs`, { cache: "no-store" }));
|
||||||
|
},
|
||||||
|
async documents(subjectId: number, weekNumber: number) {
|
||||||
|
return parseResponse<DocumentItem[]>(await fetch(`${API_URL}/subjects/${subjectId}/weeks/${weekNumber}/documents`, { cache: "no-store" }));
|
||||||
|
},
|
||||||
|
async deleteSource(sourceId: number) {
|
||||||
|
return parseResponse<{ status: string }>(await fetch(`${API_URL}/sources/${sourceId}`, { method: "DELETE" }));
|
||||||
|
},
|
||||||
|
async deleteDocument(documentId: number) {
|
||||||
|
return parseResponse<{ status: string }>(await fetch(`${API_URL}/documents/${documentId}`, { method: "DELETE" }));
|
||||||
|
},
|
||||||
|
async reprocessSource(sourceId: number, useOcr: boolean, pageRanges?: string) {
|
||||||
|
const params = new URLSearchParams({ use_ocr: String(useOcr) });
|
||||||
|
if (pageRanges?.trim()) params.set("page_ranges", pageRanges.trim());
|
||||||
|
return parseResponse<{ source: Source; job: Job }>(await fetch(`${API_URL}/sources/${sourceId}/reprocess?${params.toString()}`, { method: "POST" }));
|
||||||
|
},
|
||||||
|
async upload(subjectId: number, weekNumber: number, file: File, useOcr: boolean, pageRanges?: string) {
|
||||||
|
const form = new FormData();
|
||||||
|
form.append("file", file);
|
||||||
|
const params = new URLSearchParams({ use_ocr: String(useOcr) });
|
||||||
|
if (pageRanges?.trim()) params.set("page_ranges", pageRanges.trim());
|
||||||
|
return parseResponse<{ source: Source; job: Job }>(await fetch(`${API_URL}/subjects/${subjectId}/weeks/${weekNumber}/files?${params.toString()}`, {
|
||||||
|
method: "POST",
|
||||||
|
body: form,
|
||||||
|
}));
|
||||||
|
},
|
||||||
|
async markdown(documentId: number) {
|
||||||
|
const response = await fetch(`${API_URL}/documents/${documentId}/markdown`, { cache: "no-store" });
|
||||||
|
if (!response.ok) throw new Error(`Error ${response.status}`);
|
||||||
|
return response.text();
|
||||||
|
},
|
||||||
|
async document(documentId: number) {
|
||||||
|
return parseResponse<DocumentItem>(await fetch(`${API_URL}/documents/${documentId}`, { cache: "no-store" }));
|
||||||
|
},
|
||||||
|
async saveMarkdown(documentId: number, content: string) {
|
||||||
|
return parseResponse<DocumentItem>(await fetch(`${API_URL}/documents/${documentId}/markdown`, {
|
||||||
|
method: "PUT",
|
||||||
|
headers: { "Content-Type": "application/json" },
|
||||||
|
body: JSON.stringify({ content }),
|
||||||
|
}));
|
||||||
|
},
|
||||||
|
downloadDocumentUrl(documentId: number) {
|
||||||
|
return `${API_URL}/documents/${documentId}/download`;
|
||||||
|
},
|
||||||
|
exportWeekUrl(subjectId: number, weekNumber: number) {
|
||||||
|
return `${API_URL}/subjects/${subjectId}/weeks/${weekNumber}/export`;
|
||||||
|
},
|
||||||
|
exportWeekZipUrl(subjectId: number, weekNumber: number) {
|
||||||
|
return `${API_URL}/subjects/${subjectId}/weeks/${weekNumber}/export.zip`;
|
||||||
|
},
|
||||||
|
};
|
||||||
Vendored
+5
@@ -0,0 +1,5 @@
|
|||||||
|
/// <reference types="next" />
|
||||||
|
/// <reference types="next/image-types/global" />
|
||||||
|
|
||||||
|
// NOTE: This file should not be edited
|
||||||
|
// see https://nextjs.org/docs/app/building-your-application/configuring/typescript for more information.
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
/** @type {import('next').NextConfig} */
|
||||||
|
const backendUrl = process.env.BACKEND_INTERNAL_URL || "http://127.0.0.1:8000";
|
||||||
|
|
||||||
|
const nextConfig = {
|
||||||
|
output: "standalone",
|
||||||
|
async rewrites() {
|
||||||
|
return [
|
||||||
|
{ source: "/health", destination: `${backendUrl}/health` },
|
||||||
|
{ source: "/settings/:path*", destination: `${backendUrl}/settings/:path*` },
|
||||||
|
{ source: "/subjects/:path*", destination: `${backendUrl}/subjects/:path*` },
|
||||||
|
{ source: "/sources/:path*", destination: `${backendUrl}/sources/:path*` },
|
||||||
|
{ source: "/documents/:path*", destination: `${backendUrl}/documents/:path*` },
|
||||||
|
{ source: "/jobs/:path*", destination: `${backendUrl}/jobs/:path*` },
|
||||||
|
];
|
||||||
|
},
|
||||||
|
};
|
||||||
|
|
||||||
|
export default nextConfig;
|
||||||
Generated
+3106
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,28 @@
|
|||||||
|
{
|
||||||
|
"name": "knowledge-station-frontend",
|
||||||
|
"version": "0.1.0",
|
||||||
|
"private": true,
|
||||||
|
"scripts": {
|
||||||
|
"dev": "next dev",
|
||||||
|
"build": "next build",
|
||||||
|
"start": "next start",
|
||||||
|
"lint": "next lint"
|
||||||
|
},
|
||||||
|
"dependencies": {
|
||||||
|
"@tailwindcss/typography": "^0.5.15",
|
||||||
|
"next": "^14.2.23",
|
||||||
|
"react": "^18.3.1",
|
||||||
|
"react-dom": "^18.3.1",
|
||||||
|
"react-markdown": "^9.0.3",
|
||||||
|
"remark-gfm": "^4.0.0"
|
||||||
|
},
|
||||||
|
"devDependencies": {
|
||||||
|
"@types/node": "^22.10.10",
|
||||||
|
"@types/react": "^18.3.18",
|
||||||
|
"@types/react-dom": "^18.3.5",
|
||||||
|
"autoprefixer": "^10.4.20",
|
||||||
|
"postcss": "^8.5.1",
|
||||||
|
"tailwindcss": "^3.4.17",
|
||||||
|
"typescript": "^5.7.3"
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -0,0 +1,6 @@
|
|||||||
|
module.exports = {
|
||||||
|
plugins: {
|
||||||
|
tailwindcss: {},
|
||||||
|
autoprefixer: {},
|
||||||
|
},
|
||||||
|
};
|
||||||
@@ -0,0 +1,18 @@
|
|||||||
|
import type { Config } from "tailwindcss";
|
||||||
|
|
||||||
|
const config: Config = {
|
||||||
|
content: ["./app/**/*.{ts,tsx}", "./components/**/*.{ts,tsx}", "./lib/**/*.{ts,tsx}"],
|
||||||
|
darkMode: "class",
|
||||||
|
theme: {
|
||||||
|
extend: {
|
||||||
|
colors: {
|
||||||
|
ink: "#0f172a",
|
||||||
|
panel: "#111827",
|
||||||
|
brand: "#2563eb",
|
||||||
|
},
|
||||||
|
},
|
||||||
|
},
|
||||||
|
plugins: [require("@tailwindcss/typography")],
|
||||||
|
};
|
||||||
|
|
||||||
|
export default config;
|
||||||
@@ -0,0 +1,22 @@
|
|||||||
|
{
|
||||||
|
"compilerOptions": {
|
||||||
|
"target": "ES2017",
|
||||||
|
"lib": ["dom", "dom.iterable", "esnext"],
|
||||||
|
"allowJs": false,
|
||||||
|
"skipLibCheck": true,
|
||||||
|
"strict": true,
|
||||||
|
"noEmit": true,
|
||||||
|
"esModuleInterop": true,
|
||||||
|
"module": "esnext",
|
||||||
|
"moduleResolution": "bundler",
|
||||||
|
"baseUrl": ".",
|
||||||
|
"resolveJsonModule": true,
|
||||||
|
"isolatedModules": true,
|
||||||
|
"jsx": "preserve",
|
||||||
|
"incremental": true,
|
||||||
|
"plugins": [{ "name": "next" }],
|
||||||
|
"paths": { "@/*": ["./*"] }
|
||||||
|
},
|
||||||
|
"include": ["next-env.d.ts", "**/*.ts", "**/*.tsx", ".next/types/**/*.ts"],
|
||||||
|
"exclude": ["node_modules"]
|
||||||
|
}
|
||||||
@@ -0,0 +1,19 @@
|
|||||||
|
[variables]
|
||||||
|
NODE_ENV = "production"
|
||||||
|
NEXT_PUBLIC_API_URL = ""
|
||||||
|
PYTHONUNBUFFERED = "1"
|
||||||
|
|
||||||
|
[phases.setup]
|
||||||
|
nixPkgs = ["python312", "nodejs_22", "ffmpeg"]
|
||||||
|
|
||||||
|
[phases.install]
|
||||||
|
cmds = [
|
||||||
|
"pip install --no-cache-dir -r requirements.txt",
|
||||||
|
"cd frontend && npm ci"
|
||||||
|
]
|
||||||
|
|
||||||
|
[phases.build]
|
||||||
|
cmds = ["cd frontend && npm run build"]
|
||||||
|
|
||||||
|
[start]
|
||||||
|
cmd = "bash scripts/start-coolify.sh"
|
||||||
@@ -0,0 +1,8 @@
|
|||||||
|
fastapi==0.115.6
|
||||||
|
uvicorn[standard]==0.34.0
|
||||||
|
python-multipart==0.0.20
|
||||||
|
pydantic-settings==2.7.1
|
||||||
|
python-dotenv==1.0.1
|
||||||
|
requests==2.32.3
|
||||||
|
python-docx==1.1.2
|
||||||
|
PyMuPDF==1.25.1
|
||||||
@@ -0,0 +1,28 @@
|
|||||||
|
#!/usr/bin/env bash
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
export APP_DATA_DIR="${APP_DATA_DIR:-/app/data}"
|
||||||
|
export HOSTNAME="0.0.0.0"
|
||||||
|
|
||||||
|
uvicorn backend.app.main:app --host 127.0.0.1 --port 8000 &
|
||||||
|
backend_pid=$!
|
||||||
|
|
||||||
|
cleanup() {
|
||||||
|
kill "$backend_pid" 2>/dev/null || true
|
||||||
|
}
|
||||||
|
trap cleanup EXIT INT TERM
|
||||||
|
|
||||||
|
cd frontend
|
||||||
|
|
||||||
|
if [ -f ".next/standalone/server.js" ]; then
|
||||||
|
mkdir -p .next/standalone/.next
|
||||||
|
rm -rf .next/standalone/.next/static .next/standalone/public
|
||||||
|
cp -r .next/static .next/standalone/.next/static
|
||||||
|
if [ -d "public" ]; then
|
||||||
|
cp -r public .next/standalone/public
|
||||||
|
fi
|
||||||
|
cd .next/standalone
|
||||||
|
PORT="${PORT:-3000}" node server.js
|
||||||
|
else
|
||||||
|
npm run start -- --hostname 0.0.0.0 --port "${PORT:-3000}"
|
||||||
|
fi
|
||||||
@@ -0,0 +1,167 @@
|
|||||||
|
param(
|
||||||
|
[Parameter(Mandatory = $true)]
|
||||||
|
[string]$Root
|
||||||
|
)
|
||||||
|
|
||||||
|
$ErrorActionPreference = "Stop"
|
||||||
|
$Root = $Root.Trim().Trim('"').TrimEnd("\")
|
||||||
|
$Root = (Resolve-Path -LiteralPath $Root).Path
|
||||||
|
$Venv = Join-Path $Root ".venv"
|
||||||
|
$Python = Join-Path $Venv "Scripts\python.exe"
|
||||||
|
$Pip = Join-Path $Venv "Scripts\pip.exe"
|
||||||
|
$Frontend = Join-Path $Root "frontend"
|
||||||
|
$Runtime = Join-Path $Root ".runtime"
|
||||||
|
|
||||||
|
function Write-Step($Message) {
|
||||||
|
Write-Host ""
|
||||||
|
Write-Host $Message -ForegroundColor Cyan
|
||||||
|
}
|
||||||
|
|
||||||
|
function Test-Port([int]$Port) {
|
||||||
|
$client = [Net.Sockets.TcpClient]::new()
|
||||||
|
try {
|
||||||
|
$client.Connect("127.0.0.1", $Port)
|
||||||
|
return $true
|
||||||
|
}
|
||||||
|
catch {
|
||||||
|
return $false
|
||||||
|
}
|
||||||
|
finally {
|
||||||
|
$client.Close()
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function Find-FreePort([int]$StartPort) {
|
||||||
|
$port = $StartPort
|
||||||
|
while (Test-Port $port) { $port++ }
|
||||||
|
return $port
|
||||||
|
}
|
||||||
|
|
||||||
|
function Get-LivePid([string]$PidFile) {
|
||||||
|
if (!(Test-Path $PidFile)) { return $null }
|
||||||
|
$raw = (Get-Content $PidFile -ErrorAction SilentlyContinue | Select-Object -First 1)
|
||||||
|
if (!$raw) { return $null }
|
||||||
|
$process = Get-Process -Id ([int]$raw) -ErrorAction SilentlyContinue
|
||||||
|
if ($process) { return [int]$raw }
|
||||||
|
Remove-Item $PidFile -Force -ErrorAction SilentlyContinue
|
||||||
|
return $null
|
||||||
|
}
|
||||||
|
|
||||||
|
function Stop-Tree([int]$Id) {
|
||||||
|
Get-CimInstance Win32_Process | Where-Object { $_.ParentProcessId -eq $Id } | ForEach-Object { Stop-Tree $_.ProcessId }
|
||||||
|
Stop-Process -Id $Id -Force -ErrorAction SilentlyContinue
|
||||||
|
}
|
||||||
|
|
||||||
|
function Invoke-Checked([scriptblock]$Command, [string]$ErrorMessage) {
|
||||||
|
& $Command
|
||||||
|
if ($LASTEXITCODE -ne 0) { throw $ErrorMessage }
|
||||||
|
}
|
||||||
|
|
||||||
|
Write-Host ""
|
||||||
|
Write-Host "============================================"
|
||||||
|
Write-Host " Knowledge Station - Plataforma completa"
|
||||||
|
Write-Host "============================================"
|
||||||
|
|
||||||
|
New-Item -ItemType Directory -Force -Path $Runtime | Out-Null
|
||||||
|
|
||||||
|
$legacyPid = Join-Path $Runtime "server.pid"
|
||||||
|
if (Test-Path $legacyPid) {
|
||||||
|
$legacyRaw = Get-Content $legacyPid -ErrorAction SilentlyContinue | Select-Object -First 1
|
||||||
|
if ($legacyRaw) { Stop-Tree ([int]$legacyRaw) }
|
||||||
|
Remove-Item $legacyPid -Force -ErrorAction SilentlyContinue
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!(Test-Path $Python)) {
|
||||||
|
Write-Step "Preparando Python por primera vez..."
|
||||||
|
& py -m venv $Venv 2>$null
|
||||||
|
if ($LASTEXITCODE -ne 0) {
|
||||||
|
& python -m venv $Venv
|
||||||
|
}
|
||||||
|
if (!(Test-Path $Python)) {
|
||||||
|
throw "No se pudo preparar Python. Instala Python 3.12 o superior."
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
$EnvFile = Join-Path $Root ".env"
|
||||||
|
$EnvExample = Join-Path $Root ".env.example"
|
||||||
|
if (!(Test-Path $EnvFile)) {
|
||||||
|
Write-Step "Creando configuracion local .env..."
|
||||||
|
Copy-Item $EnvExample $EnvFile
|
||||||
|
Write-Host "Agrega tus API keys en .env para audio, video u OCR." -ForegroundColor Yellow
|
||||||
|
}
|
||||||
|
|
||||||
|
Write-Step "Verificando dependencias de Python..."
|
||||||
|
Invoke-Checked { & $Python -m pip install --upgrade pip --disable-pip-version-check | Out-Null } "No se pudo actualizar pip."
|
||||||
|
Invoke-Checked { & $Pip install -r (Join-Path $Root "requirements.txt") } "No se pudieron instalar dependencias de Python."
|
||||||
|
|
||||||
|
Write-Step "Verificando Node.js..."
|
||||||
|
$node = Get-Command node -ErrorAction SilentlyContinue
|
||||||
|
$npm = Get-Command npm.cmd -ErrorAction SilentlyContinue
|
||||||
|
if (!$npm) { $npm = Get-Command npm -ErrorAction SilentlyContinue }
|
||||||
|
if (!$node -or !$npm) {
|
||||||
|
throw "Node.js LTS no esta instalado o npm no esta disponible. Instala Node.js desde https://nodejs.org/."
|
||||||
|
}
|
||||||
|
|
||||||
|
if (!(Test-Path (Join-Path $Frontend "node_modules"))) {
|
||||||
|
Write-Step "Instalando dependencias del frontend. Esto puede tardar unos minutos..."
|
||||||
|
Invoke-Checked { & $npm.Source install --prefix $Frontend } "No se pudieron instalar dependencias del frontend."
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
Write-Host "Dependencias del frontend encontradas."
|
||||||
|
}
|
||||||
|
|
||||||
|
$BackendPidFile = Join-Path $Runtime "backend.pid"
|
||||||
|
$FrontendPidFile = Join-Path $Runtime "frontend.pid"
|
||||||
|
$BackendPortFile = Join-Path $Runtime "backend.port"
|
||||||
|
$FrontendPortFile = Join-Path $Runtime "frontend.port"
|
||||||
|
|
||||||
|
Write-Step "Iniciando backend..."
|
||||||
|
$backendPid = Get-LivePid $BackendPidFile
|
||||||
|
if ($backendPid) {
|
||||||
|
$backendPort = Get-Content $BackendPortFile -ErrorAction SilentlyContinue | Select-Object -First 1
|
||||||
|
if (!$backendPort) { $backendPort = 8000 }
|
||||||
|
Write-Host "Backend ya estaba activo en puerto $backendPort."
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
$backendPort = Find-FreePort 8000
|
||||||
|
$backend = Start-Process -FilePath $Python -ArgumentList @("-m", "uvicorn", "backend.app.main:app", "--host", "127.0.0.1", "--port", [string]$backendPort, "--reload") -WorkingDirectory $Root -PassThru
|
||||||
|
Set-Content $BackendPidFile $backend.Id
|
||||||
|
Set-Content $BackendPortFile $backendPort
|
||||||
|
Write-Host "Backend iniciado en puerto $backendPort."
|
||||||
|
}
|
||||||
|
|
||||||
|
$ApiUrl = "http://127.0.0.1:$backendPort"
|
||||||
|
Set-Content (Join-Path $Frontend ".env.local") "NEXT_PUBLIC_API_URL=$ApiUrl"
|
||||||
|
|
||||||
|
Write-Step "Iniciando frontend..."
|
||||||
|
$frontendPid = Get-LivePid $FrontendPidFile
|
||||||
|
if ($frontendPid) {
|
||||||
|
$frontendPort = Get-Content $FrontendPortFile -ErrorAction SilentlyContinue | Select-Object -First 1
|
||||||
|
if (!$frontendPort) { $frontendPort = 3000 }
|
||||||
|
Write-Host "Frontend ya estaba activo en puerto $frontendPort."
|
||||||
|
}
|
||||||
|
else {
|
||||||
|
$frontendPort = Find-FreePort 3000
|
||||||
|
$nextDir = Join-Path $Frontend ".next"
|
||||||
|
if (Test-Path $nextDir) {
|
||||||
|
Get-ChildItem $nextDir -Recurse -Force -ErrorAction SilentlyContinue | ForEach-Object {
|
||||||
|
try { $_.Attributes = "Normal" } catch {}
|
||||||
|
}
|
||||||
|
Remove-Item $nextDir -Recurse -Force -ErrorAction SilentlyContinue
|
||||||
|
}
|
||||||
|
$frontend = Start-Process -FilePath $npm.Source -ArgumentList @("run", "dev", "--", "-p", [string]$frontendPort) -WorkingDirectory $Frontend -PassThru
|
||||||
|
Set-Content $FrontendPidFile $frontend.Id
|
||||||
|
Set-Content $FrontendPortFile $frontendPort
|
||||||
|
Write-Host "Frontend iniciado en puerto $frontendPort."
|
||||||
|
}
|
||||||
|
|
||||||
|
$AppUrl = "http://127.0.0.1:$frontendPort"
|
||||||
|
Set-Content (Join-Path $Runtime "last.url") $AppUrl
|
||||||
|
|
||||||
|
Start-Sleep -Seconds 4
|
||||||
|
Write-Host ""
|
||||||
|
Write-Host "Plataforma abierta:" -ForegroundColor Green
|
||||||
|
Write-Host "Frontend: $AppUrl"
|
||||||
|
Write-Host "API: $ApiUrl"
|
||||||
|
Write-Host ""
|
||||||
|
Start-Process $AppUrl
|
||||||
@@ -0,0 +1,47 @@
|
|||||||
|
param(
|
||||||
|
[Parameter(Mandatory = $true)]
|
||||||
|
[string]$Root
|
||||||
|
)
|
||||||
|
|
||||||
|
$ErrorActionPreference = "SilentlyContinue"
|
||||||
|
$Root = $Root.Trim().Trim('"').TrimEnd("\")
|
||||||
|
$Root = (Resolve-Path -LiteralPath $Root).Path.TrimEnd("\")
|
||||||
|
$Runtime = Join-Path $Root ".runtime"
|
||||||
|
|
||||||
|
function Stop-Tree([int]$Id) {
|
||||||
|
Get-CimInstance Win32_Process | Where-Object { $_.ParentProcessId -eq $Id } | ForEach-Object { Stop-Tree $_.ProcessId }
|
||||||
|
Stop-Process -Id $Id -Force
|
||||||
|
}
|
||||||
|
|
||||||
|
Write-Host ""
|
||||||
|
Write-Host "============================================"
|
||||||
|
Write-Host " Knowledge Station - Cerrar plataforma"
|
||||||
|
Write-Host "============================================"
|
||||||
|
Write-Host ""
|
||||||
|
|
||||||
|
if (!(Test-Path $Runtime)) {
|
||||||
|
Write-Host "No hay registro de servicios abiertos."
|
||||||
|
exit 0
|
||||||
|
}
|
||||||
|
|
||||||
|
foreach ($name in @("backend.pid", "frontend.pid", "server.pid")) {
|
||||||
|
$file = Join-Path $Runtime $name
|
||||||
|
if (Test-Path $file) {
|
||||||
|
$raw = Get-Content $file | Select-Object -First 1
|
||||||
|
if ($raw) { Stop-Tree ([int]$raw) }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
Get-CimInstance Win32_Process |
|
||||||
|
Where-Object {
|
||||||
|
$_.CommandLine -and
|
||||||
|
$_.CommandLine -like "*$Root*" -and
|
||||||
|
($_.CommandLine -like "*uvicorn*backend.app.main:app*" -or $_.CommandLine -like "*next*dev*")
|
||||||
|
} |
|
||||||
|
ForEach-Object { Stop-Tree $_.ProcessId }
|
||||||
|
|
||||||
|
foreach ($name in @("backend.pid", "frontend.pid", "server.pid", "backend.port", "frontend.port", "last.url")) {
|
||||||
|
Remove-Item (Join-Path $Runtime $name) -Force
|
||||||
|
}
|
||||||
|
|
||||||
|
Write-Host "Plataforma cerrada de forma segura." -ForegroundColor Green
|
||||||
Reference in New Issue
Block a user