Compare commits
12
Commits
1ec4a37fde
...
main
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
239a7510c6 | ||
|
|
379f563d04 | ||
|
|
2160d4ff70 | ||
|
|
7361f7ced1 | ||
|
|
4f3bc01dcb | ||
|
|
0df056445e | ||
|
|
d18ab1f618 | ||
|
|
b04079eaaf | ||
|
|
7dc0f65045 | ||
|
|
8e2a7eadd2 | ||
|
|
052580b75b | ||
|
|
306d131136 |
+1
-1
@@ -14,6 +14,6 @@ frontend/.env.local
|
|||||||
frontend/.env.*.local
|
frontend/.env.*.local
|
||||||
node_modules
|
node_modules
|
||||||
npm-debug.log*
|
npm-debug.log*
|
||||||
Dockerfile*
|
Dockerfile
|
||||||
docker-compose*.yml
|
docker-compose*.yml
|
||||||
docker-compose*.yaml
|
docker-compose*.yaml
|
||||||
|
|||||||
+6
-1
@@ -2,5 +2,10 @@ APP_ENV=development
|
|||||||
APP_DATA_DIR=./data
|
APP_DATA_DIR=./data
|
||||||
DATABASE_PATH=./data/app.db
|
DATABASE_PATH=./data/app.db
|
||||||
MISTRAL_API_KEY=
|
MISTRAL_API_KEY=
|
||||||
MISTRAL_OCR_MODEL=mistral-ocr-latest
|
MISTRAL_OCR_MODEL=mistral-ocr-4-0
|
||||||
|
MISTRAL_OCR_BATCH_THRESHOLD=3
|
||||||
|
MISTRAL_OCR_BATCH_POLL_SECONDS=10
|
||||||
|
MISTRAL_OCR_BATCH_TIMEOUT_SECONDS=1800
|
||||||
|
MISTRAL_OCR_RETRY_ATTEMPTS=3
|
||||||
|
MISTRAL_OCR_RETRY_BACKOFF_BASE=2.0
|
||||||
DEEPGRAM_API_KEY=
|
DEEPGRAM_API_KEY=
|
||||||
|
|||||||
@@ -0,0 +1,46 @@
|
|||||||
|
# CLAUDE.md
|
||||||
|
|
||||||
|
This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.
|
||||||
|
|
||||||
|
> `AGENTS.md` contains the canonical command list, platform behavior notes, and deployment/secrets rules. Read it too — this file focuses on the cross-file architecture that AGENTS.md does not spell out.
|
||||||
|
|
||||||
|
## Commands
|
||||||
|
|
||||||
|
- **Backend (run from repo root, not `backend/`):** `uvicorn backend.app.main:app --reload`. Imports are absolute from the `backend.` package, so the working directory must be the repo root.
|
||||||
|
- **Frontend:** `cd frontend && npm run dev` (dev), `npm run build`, `npm run lint`.
|
||||||
|
- **Syntax check:** `python -m compileall backend` (there is no test suite in the repo).
|
||||||
|
- **Windows one-click:** `abrir_estacion.bat` → `scripts/start-platform.ps1` picks free ports, starts both servers, opens the browser; `cerrar_estacion.bat` stops only this project.
|
||||||
|
- **Container/prod:** `docker compose up --build`, or Nixpacks/`Dockerfile.coolify` → both run `scripts/start-coolify.sh`.
|
||||||
|
|
||||||
|
## Architecture
|
||||||
|
|
||||||
|
A two-process app for turning class materials (PDF/DOCX/text/audio/video) into LLM-ready Markdown, organized as **Materia → Semana → documentos**.
|
||||||
|
|
||||||
|
### Request → background-job flow
|
||||||
|
1. `backend/app/main.py` is the entire HTTP API (flat FastAPI app, no routers). Every endpoint is a thin try/except that delegates to `KnowledgeManager` and maps `ValueError` → 404/400, `DuplicateSourceError` → 409.
|
||||||
|
2. File upload saves the raw file, creates a **source** row + a **job** row, then schedules `process_job` via FastAPI `BackgroundTasks` (in-process — no external queue/worker). Job status transitions: `queued → processing → completed|failed`.
|
||||||
|
3. `process_job` calls `IngestionService.process_source`, which picks a processor by `source_type`, runs it, wraps the result in Markdown frontmatter, and persists a **document**.
|
||||||
|
|
||||||
|
### Key components
|
||||||
|
- **`KnowledgeManager`** (`managers/knowledge_manager.py`) — the single source of truth. Owns **both** the SQLite schema (`subjects`, `weeks`, `sources`, `documents`, `jobs`) and the on-disk layout under `data/subjects/<slug>/week-NN/{raw,processed,exports}/`. Filesystem and DB are kept in sync here; processors and endpoints must go through it, never write final Markdown or touch the DB directly. Schema migrations are done idempotently in `_init_db` via `_ensure_columns` (add-column-if-missing) — there is no migration framework.
|
||||||
|
- **`IngestionService`** (`services/ingestion_service.py`) — maps `source_type` → processor and assembles frontmatter metadata.
|
||||||
|
- **Processors** (`services/processors/`) — each returns a `ProcessedContent(title, body, processor)` dataclass (`base.py`). `text`, `docx` (python-docx), `pdf` (PyMuPDF text, or Mistral OCR when `use_ocr=true`), `audio` (Deepgram), `video` (ffmpeg → audio → Deepgram). PDFs accept `page_ranges` like `1-5,8,10-12`.
|
||||||
|
- **Providers** (`services/providers/`) — thin HTTP clients for Mistral OCR and Deepgram, keyed off `MISTRAL_API_KEY` / `DEEPGRAM_API_KEY`.
|
||||||
|
- **`markdown_builder.build_markdown`** — the only place that emits the YAML frontmatter block prepended to every processed document.
|
||||||
|
|
||||||
|
### Important invariants
|
||||||
|
- **One document per source.** `create_document` deletes any prior document/jobs for that source before inserting, then deletes the old Markdown file. Reprocessing replaces, never appends.
|
||||||
|
- **Dedup by hash.** `save_upload` rejects a file whose `sha256` already exists in the same week (`DuplicateSourceError` → 409).
|
||||||
|
- **Deletes cascade manually** (no FK cascade): deleting a subject/week/source/document removes child DB rows *and* unlinks files. Renaming a week (`update_week`) renames the directory and rewrites stored paths in the DB with SQL `replace()`.
|
||||||
|
- **Markdown is sanitized** (`_sanitize_markdown` strips control chars) on every read and write.
|
||||||
|
- **Week exports** (`export_week_markdown`) concatenate documents, stripping each doc's frontmatter and inserting `<!-- document_id: N -->` markers; the `.zip` variant ships individual files.
|
||||||
|
|
||||||
|
### Frontend
|
||||||
|
Next.js 14 App Router (`frontend/app/`), Tailwind, `react-markdown` + `remark-gfm`. All backend calls go through the single `api` object in `frontend/lib/api.ts`. `API_URL` resolves to `http://127.0.0.1:8000` in dev (via `NEXT_PUBLIC_API_URL`) or `/api` in prod, where `next.config.mjs` `rewrites()` proxy `/api/*` to `BACKEND_INTERNAL_URL`. Pages mirror the data model: `/`, `/subjects/[subjectId]`, `/subjects/[subjectId]/weeks/[weekNumber]`, `/documents/[documentId]`, `/settings`.
|
||||||
|
|
||||||
|
### Deployment shape
|
||||||
|
Single container runs **both** processes: `start-coolify.sh` launches uvicorn on `127.0.0.1:8000` (internal only) and Next.js standalone on `PORT` (default 3000, public). `/app/data` **must** be a mounted volume or all subjects/documents/SQLite state are lost on redeploy. `ffmpeg` is required at runtime for video.
|
||||||
|
|
||||||
|
## Conventions
|
||||||
|
- UX, error messages, and generated content are in **Spanish** — preserve this. Source comments/identifiers are mixed Spanish/English; match the surrounding file.
|
||||||
|
- The standalone `estacion documentos.html` is a browser-only prototype kept for reference behavior; the real app is `backend/` + `frontend/`.
|
||||||
@@ -0,0 +1,42 @@
|
|||||||
|
FROM node:22-bookworm-slim AS frontend-builder
|
||||||
|
|
||||||
|
WORKDIR /app/frontend
|
||||||
|
COPY frontend/package.json frontend/package-lock.json* ./
|
||||||
|
RUN npm ci
|
||||||
|
COPY frontend ./
|
||||||
|
ARG NEXT_PUBLIC_API_URL=
|
||||||
|
ARG BACKEND_INTERNAL_URL=http://127.0.0.1:8000
|
||||||
|
ENV NEXT_PUBLIC_API_URL=$NEXT_PUBLIC_API_URL
|
||||||
|
ENV BACKEND_INTERNAL_URL=$BACKEND_INTERNAL_URL
|
||||||
|
RUN npm run build
|
||||||
|
|
||||||
|
FROM python:3.12-slim-bookworm
|
||||||
|
|
||||||
|
ENV PYTHONDONTWRITEBYTECODE=1
|
||||||
|
ENV PYTHONUNBUFFERED=1
|
||||||
|
ENV NODE_ENV=production
|
||||||
|
ENV HOSTNAME=0.0.0.0
|
||||||
|
ENV DEBIAN_FRONTEND=noninteractive
|
||||||
|
|
||||||
|
WORKDIR /app
|
||||||
|
|
||||||
|
RUN apt-get update \
|
||||||
|
&& apt-get install -y --no-install-recommends ca-certificates ffmpeg \
|
||||||
|
&& rm -rf /var/lib/apt/lists/*
|
||||||
|
|
||||||
|
COPY --from=frontend-builder /usr/local/bin/node /usr/local/bin/node
|
||||||
|
COPY --from=frontend-builder /usr/local/bin/npm /usr/local/bin/npm
|
||||||
|
COPY --from=frontend-builder /usr/local/bin/npx /usr/local/bin/npx
|
||||||
|
COPY --from=frontend-builder /usr/local/lib/node_modules /usr/local/lib/node_modules
|
||||||
|
|
||||||
|
COPY requirements.txt ./
|
||||||
|
RUN pip install --no-cache-dir -r requirements.txt
|
||||||
|
|
||||||
|
COPY backend ./backend
|
||||||
|
COPY scripts ./scripts
|
||||||
|
COPY --from=frontend-builder /app/frontend/.next ./frontend/.next
|
||||||
|
COPY --from=frontend-builder /app/frontend/package.json ./frontend/package.json
|
||||||
|
COPY --from=frontend-builder /app/frontend/node_modules ./frontend/node_modules
|
||||||
|
|
||||||
|
EXPOSE 3000
|
||||||
|
CMD ["bash", "scripts/start-coolify.sh"]
|
||||||
@@ -8,7 +8,12 @@ class Settings(BaseSettings):
|
|||||||
app_data_dir: Path = Path("./data")
|
app_data_dir: Path = Path("./data")
|
||||||
database_path: Path = Path("./data/app.db")
|
database_path: Path = Path("./data/app.db")
|
||||||
mistral_api_key: str = ""
|
mistral_api_key: str = ""
|
||||||
mistral_ocr_model: str = "mistral-ocr-latest"
|
mistral_ocr_model: str = "mistral-ocr-4-0"
|
||||||
|
mistral_ocr_batch_threshold: int = 3
|
||||||
|
mistral_ocr_batch_poll_seconds: int = 10
|
||||||
|
mistral_ocr_batch_timeout_seconds: int = 1800
|
||||||
|
mistral_ocr_retry_attempts: int = 3
|
||||||
|
mistral_ocr_retry_backoff_base: float = 2.0
|
||||||
deepgram_api_key: str = ""
|
deepgram_api_key: str = ""
|
||||||
|
|
||||||
model_config = SettingsConfigDict(env_file=".env", env_file_encoding="utf-8", extra="ignore")
|
model_config = SettingsConfigDict(env_file=".env", env_file_encoding="utf-8", extra="ignore")
|
||||||
|
|||||||
+15
-2
@@ -8,7 +8,8 @@ from fastapi.responses import FileResponse, PlainTextResponse
|
|||||||
|
|
||||||
from backend.app.config import settings
|
from backend.app.config import settings
|
||||||
from backend.app.managers.knowledge_manager import DuplicateSourceError, KnowledgeManager
|
from backend.app.managers.knowledge_manager import DuplicateSourceError, KnowledgeManager
|
||||||
from backend.app.models.schemas import MarkdownUpdate, SubjectCreate, WeekCreate, WeekUpdate
|
from backend.app.models.schemas import MarkdownUpdate, SubjectCreate, SubjectUpdate, WeekCreate, WeekUpdate
|
||||||
|
from backend.app.services.errors import ProcessingError, classify_unknown_error
|
||||||
from backend.app.services.ingestion_service import IngestionService
|
from backend.app.services.ingestion_service import IngestionService
|
||||||
|
|
||||||
app = FastAPI(title="Knowledge Station", version="0.1.0")
|
app = FastAPI(title="Knowledge Station", version="0.1.0")
|
||||||
@@ -60,6 +61,14 @@ def get_subject(subject_id: int) -> dict:
|
|||||||
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
raise HTTPException(status_code=404, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
|
@app.put("/subjects/{subject_id}")
|
||||||
|
def update_subject(subject_id: int, payload: SubjectUpdate) -> dict:
|
||||||
|
try:
|
||||||
|
return manager.update_subject(subject_id, payload.name)
|
||||||
|
except ValueError as exc:
|
||||||
|
raise HTTPException(status_code=400, detail=str(exc)) from exc
|
||||||
|
|
||||||
|
|
||||||
@app.delete("/subjects/{subject_id}")
|
@app.delete("/subjects/{subject_id}")
|
||||||
def delete_subject(subject_id: int) -> dict:
|
def delete_subject(subject_id: int) -> dict:
|
||||||
try:
|
try:
|
||||||
@@ -276,6 +285,10 @@ def process_job(job_id: int, source_id: int, use_ocr: bool, page_ranges: str | N
|
|||||||
try:
|
try:
|
||||||
document = ingestion.process_source(source_id, use_ocr=use_ocr, page_ranges=page_ranges)
|
document = ingestion.process_source(source_id, use_ocr=use_ocr, page_ranges=page_ranges)
|
||||||
manager.update_job(job_id, "completed", "Documento procesado", document_id=document["id"])
|
manager.update_job(job_id, "completed", "Documento procesado", document_id=document["id"])
|
||||||
|
except ProcessingError as exc:
|
||||||
|
manager.update_source_status(source_id, "failed")
|
||||||
|
manager.update_job(job_id, "failed", exc.labeled)
|
||||||
except Exception as exc:
|
except Exception as exc:
|
||||||
manager.update_source_status(source_id, "failed")
|
manager.update_source_status(source_id, "failed")
|
||||||
manager.update_job(job_id, "failed", str(exc))
|
code = classify_unknown_error(exc)
|
||||||
|
manager.update_job(job_id, "failed", f"[{code}] {exc}")
|
||||||
|
|||||||
@@ -154,6 +154,14 @@ class KnowledgeManager:
|
|||||||
raise ValueError("Materia no encontrada")
|
raise ValueError("Materia no encontrada")
|
||||||
return dict(row)
|
return dict(row)
|
||||||
|
|
||||||
|
def update_subject(self, subject_id: int, name: str) -> dict[str, Any]:
|
||||||
|
if not name.strip():
|
||||||
|
raise ValueError("El nombre de la materia no puede estar vacio")
|
||||||
|
self.get_subject(subject_id)
|
||||||
|
with self._connect() as conn:
|
||||||
|
conn.execute("update subjects set name = ? where id = ?", (name.strip(), subject_id))
|
||||||
|
return self.get_subject(subject_id)
|
||||||
|
|
||||||
def delete_subject(self, subject_id: int) -> None:
|
def delete_subject(self, subject_id: int) -> None:
|
||||||
subject = self.get_subject(subject_id)
|
subject = self.get_subject(subject_id)
|
||||||
with self._connect() as conn:
|
with self._connect() as conn:
|
||||||
@@ -224,6 +232,8 @@ class KnowledgeManager:
|
|||||||
def update_week(self, subject_id: int, current_number: int, new_number: int, title: str | None = None) -> dict[str, Any]:
|
def update_week(self, subject_id: int, current_number: int, new_number: int, title: str | None = None) -> dict[str, Any]:
|
||||||
subject = self.get_subject(subject_id)
|
subject = self.get_subject(subject_id)
|
||||||
week = self.get_week(subject_id, current_number)
|
week = self.get_week(subject_id, current_number)
|
||||||
|
old_dir: Path | None = None
|
||||||
|
new_dir: Path | None = None
|
||||||
if new_number != current_number:
|
if new_number != current_number:
|
||||||
with self._connect() as conn:
|
with self._connect() as conn:
|
||||||
existing = conn.execute(
|
existing = conn.execute(
|
||||||
@@ -248,6 +258,22 @@ class KnowledgeManager:
|
|||||||
"update weeks set number = ?, title = ? where id = ?",
|
"update weeks set number = ?, title = ? where id = ?",
|
||||||
(new_number, title, week["id"]),
|
(new_number, title, week["id"]),
|
||||||
)
|
)
|
||||||
|
if old_dir and new_dir:
|
||||||
|
old_prefix = str(old_dir)
|
||||||
|
new_prefix = str(new_dir)
|
||||||
|
conn.execute(
|
||||||
|
"update sources set stored_path = replace(stored_path, ?, ?) where week_id = ? and stored_path like ?",
|
||||||
|
(old_prefix, new_prefix, week["id"], f"{old_prefix}%"),
|
||||||
|
)
|
||||||
|
conn.execute(
|
||||||
|
"""
|
||||||
|
update documents
|
||||||
|
set markdown_path = replace(markdown_path, ?, ?)
|
||||||
|
where source_id in (select id from sources where week_id = ?)
|
||||||
|
and markdown_path like ?
|
||||||
|
""",
|
||||||
|
(old_prefix, new_prefix, week["id"], f"{old_prefix}%"),
|
||||||
|
)
|
||||||
return self.get_week(subject_id, new_number)
|
return self.get_week(subject_id, new_number)
|
||||||
|
|
||||||
def week_dir(self, subject_slug: str, week_number: int) -> Path:
|
def week_dir(self, subject_slug: str, week_number: int) -> Path:
|
||||||
|
|||||||
@@ -8,6 +8,10 @@ class SubjectCreate(BaseModel):
|
|||||||
name: str = Field(min_length=1)
|
name: str = Field(min_length=1)
|
||||||
|
|
||||||
|
|
||||||
|
class SubjectUpdate(BaseModel):
|
||||||
|
name: str = Field(min_length=1)
|
||||||
|
|
||||||
|
|
||||||
class Subject(BaseModel):
|
class Subject(BaseModel):
|
||||||
id: int
|
id: int
|
||||||
name: str
|
name: str
|
||||||
|
|||||||
@@ -0,0 +1,85 @@
|
|||||||
|
from __future__ import annotations
|
||||||
|
|
||||||
|
import requests
|
||||||
|
|
||||||
|
|
||||||
|
class ProcessingError(Exception):
|
||||||
|
"""Error de procesamiento con un codigo corto y rastreable.
|
||||||
|
|
||||||
|
El atributo ``code`` es un identificador estable en MAYUSCULAS que se
|
||||||
|
antepone al mensaje del job (p.ej. ``[AUTH_MISTRAL] API Key invalida``)
|
||||||
|
para que el usuario pueda buscar y diagnosticar rapidamente el tipo de
|
||||||
|
fallo desde la UI.
|
||||||
|
"""
|
||||||
|
|
||||||
|
def __init__(self, code: str, detail: str) -> None:
|
||||||
|
super().__init__(f"[{code}] {detail}")
|
||||||
|
self.code = code
|
||||||
|
self.detail = detail
|
||||||
|
|
||||||
|
@property
|
||||||
|
def labeled(self) -> str:
|
||||||
|
return f"[{self.code}] {self.detail}"
|
||||||
|
|
||||||
|
|
||||||
|
class MistralAuthError(ProcessingError):
|
||||||
|
def __init__(self, detail: str = "API Key de Mistral invalida o no autorizada (401)") -> None:
|
||||||
|
super().__init__("AUTH_MISTRAL", detail)
|
||||||
|
|
||||||
|
|
||||||
|
class MistralRateLimitError(ProcessingError):
|
||||||
|
def __init__(self, detail: str = "Limite de uso de Mistral alcanzado (429)") -> None:
|
||||||
|
super().__init__("RATE_LIMIT", detail)
|
||||||
|
|
||||||
|
|
||||||
|
class MistralServerError(ProcessingError):
|
||||||
|
def __init__(self, status: int, body: str) -> None:
|
||||||
|
super().__init__("MISTRAL_SERVER", f"Mistral HTTP {status}: {body[:200]}")
|
||||||
|
|
||||||
|
|
||||||
|
class MistralBatchError(ProcessingError):
|
||||||
|
def __init__(self, detail: str) -> None:
|
||||||
|
super().__init__("BATCH", detail)
|
||||||
|
|
||||||
|
|
||||||
|
class MistralOCREmptyError(ProcessingError):
|
||||||
|
def __init__(self, detail: str = "OCR no devolvio contenido") -> None:
|
||||||
|
super().__init__("OCR_EMPTY", detail)
|
||||||
|
|
||||||
|
|
||||||
|
class NetworkError(ProcessingError):
|
||||||
|
def __init__(self, detail: str) -> None:
|
||||||
|
super().__init__("NETWORK", detail)
|
||||||
|
|
||||||
|
|
||||||
|
NON_RETRYABLE_CODES = {
|
||||||
|
"AUTH_MISTRAL",
|
||||||
|
"CONFIG",
|
||||||
|
"OCR_EMPTY",
|
||||||
|
"MISTRAL_CLIENT",
|
||||||
|
"BAD_PDF",
|
||||||
|
"FILE_IO",
|
||||||
|
}
|
||||||
|
|
||||||
|
|
||||||
|
def classify_requests_error(exc: requests.RequestException) -> ProcessingError:
|
||||||
|
if isinstance(exc, requests.Timeout):
|
||||||
|
return NetworkError(f"timeout de red con Mistral: {exc}")
|
||||||
|
if isinstance(exc, requests.ConnectionError):
|
||||||
|
return NetworkError(f"sin conexion a Mistral: {exc}")
|
||||||
|
return NetworkError(f"error de red con Mistral: {exc}")
|
||||||
|
|
||||||
|
|
||||||
|
def classify_unknown_error(exc: Exception) -> str:
|
||||||
|
"""Etiqueta breve para excepciones no envueltas en ProcessingError."""
|
||||||
|
name = type(exc).__name__.lower()
|
||||||
|
message = str(exc).lower()
|
||||||
|
if "fitz" in name or "pdf" in name or "pdf" in message or "muPDF" in message:
|
||||||
|
return "PDF_PARSE"
|
||||||
|
if isinstance(exc, FileNotFoundError):
|
||||||
|
return "FILE_IO"
|
||||||
|
if isinstance(exc, (PermissionError, OSError)):
|
||||||
|
return "FILE_IO"
|
||||||
|
if isinstance(exc, ValueError):
|
||||||
|
return "VALIDATION"
|
||||||
|
return "UNKNOWN"
|
||||||
@@ -4,8 +4,10 @@ from pathlib import Path
|
|||||||
|
|
||||||
import fitz
|
import fitz
|
||||||
|
|
||||||
|
from backend.app.config import settings
|
||||||
from backend.app.services.markdown_builder import title_from_path
|
from backend.app.services.markdown_builder import title_from_path
|
||||||
from backend.app.services.processors.base import ProcessedContent
|
from backend.app.services.processors.base import ProcessedContent
|
||||||
|
from backend.app.services.errors import ProcessingError
|
||||||
from backend.app.services.providers.mistral_client import MistralClient
|
from backend.app.services.providers.mistral_client import MistralClient
|
||||||
|
|
||||||
|
|
||||||
@@ -30,22 +32,77 @@ class PdfProcessor:
|
|||||||
return ProcessedContent(title=title_from_path(path), body=body, processor="pdf-text")
|
return ProcessedContent(title=title_from_path(path), body=body, processor="pdf-text")
|
||||||
|
|
||||||
def _process_ocr(self, path: Path) -> ProcessedContent:
|
def _process_ocr(self, path: Path) -> ProcessedContent:
|
||||||
doc = fitz.open(path)
|
|
||||||
client = MistralClient()
|
|
||||||
parts: list[str] = []
|
|
||||||
temp_paths: list[Path] = []
|
|
||||||
try:
|
try:
|
||||||
for index in self._selected_pages(len(doc)):
|
doc = fitz.open(path)
|
||||||
|
except Exception as exc:
|
||||||
|
raise ProcessingError("BAD_PDF", f"No se pudo abrir el PDF: {exc}") from exc
|
||||||
|
selected = self._selected_pages(len(doc))
|
||||||
|
client = MistralClient()
|
||||||
|
page_images: dict[int, Path] = {}
|
||||||
|
try:
|
||||||
|
for index in selected:
|
||||||
page = doc[index - 1]
|
page = doc[index - 1]
|
||||||
pix = page.get_pixmap(matrix=fitz.Matrix(2, 2), alpha=False)
|
pix = page.get_pixmap(matrix=fitz.Matrix(2, 2), alpha=False)
|
||||||
image_path = path.with_name(f"{path.stem}-page-{index}.jpg")
|
image_path = path.with_name(f"{path.stem}-page-{index}.jpg")
|
||||||
pix.save(image_path)
|
pix.save(image_path)
|
||||||
temp_paths.append(image_path)
|
page_images[index] = image_path
|
||||||
parts.append(f"## Pagina {index}\n\n{client.ocr_image(image_path)}")
|
|
||||||
|
results = self._ocr_pages(client, page_images)
|
||||||
finally:
|
finally:
|
||||||
for temp_path in temp_paths:
|
for image_path in page_images.values():
|
||||||
temp_path.unlink(missing_ok=True)
|
image_path.unlink(missing_ok=True)
|
||||||
return ProcessedContent(title=title_from_path(path), body="\n\n".join(parts), processor="mistral-ocr")
|
|
||||||
|
parts: list[str] = []
|
||||||
|
for index in selected:
|
||||||
|
markdown = results.get(index, "")
|
||||||
|
parts.append(f"## Pagina {index}\n\n{markdown}")
|
||||||
|
return ProcessedContent(
|
||||||
|
title=title_from_path(path),
|
||||||
|
body="\n\n".join(parts),
|
||||||
|
processor="mistral-ocr",
|
||||||
|
)
|
||||||
|
|
||||||
|
def _ocr_pages(self, client: MistralClient, page_images: dict[int, Path]) -> dict[int, str]:
|
||||||
|
"""OCR de las paginas con lote + fallback sincrono.
|
||||||
|
|
||||||
|
Estrategia:
|
||||||
|
1. Si hay >= ``batch_threshold`` paginas, intenta el job en lote.
|
||||||
|
2. Cualquier pagina no resuelta (o si el lote entero falla) cae a
|
||||||
|
OCR sincrono pagina a pagina con reintentos.
|
||||||
|
3. Las paginas que fallen de forma definitiva quedan como un
|
||||||
|
placeholder breve con el codigo de error rastreable.
|
||||||
|
"""
|
||||||
|
indices = list(page_images.keys())
|
||||||
|
results: dict[int, str] = {}
|
||||||
|
page_errors: dict[int, str] = {}
|
||||||
|
|
||||||
|
if len(indices) >= settings.mistral_ocr_batch_threshold:
|
||||||
|
ordered = sorted(indices)
|
||||||
|
paths = [page_images[i] for i in ordered]
|
||||||
|
try:
|
||||||
|
batch_results = client.ocr_images_batch(paths)
|
||||||
|
for pos, index in enumerate(ordered):
|
||||||
|
if pos in batch_results:
|
||||||
|
results[index] = batch_results[pos]
|
||||||
|
except ProcessingError:
|
||||||
|
# El lote fallo; continua al fallback sincrono para todas
|
||||||
|
# las paginas no resueltas.
|
||||||
|
pass
|
||||||
|
|
||||||
|
for index in indices:
|
||||||
|
if index in results:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
results[index] = client.ocr_image(page_images[index])
|
||||||
|
except ProcessingError as exc:
|
||||||
|
page_errors[index] = exc.labeled
|
||||||
|
except Exception as exc:
|
||||||
|
page_errors[index] = f"[OCR_ERROR] {exc}"
|
||||||
|
|
||||||
|
for index in indices:
|
||||||
|
if index not in results:
|
||||||
|
results[index] = f"> ⚠️ No se pudo procesar esta pagina: {page_errors.get(index, '[UNKNOWN]')}"
|
||||||
|
return results
|
||||||
|
|
||||||
def _selected_pages(self, total_pages: int) -> list[int]:
|
def _selected_pages(self, total_pages: int) -> list[int]:
|
||||||
if total_pages < 1:
|
if total_pages < 1:
|
||||||
|
|||||||
@@ -1,45 +1,287 @@
|
|||||||
from __future__ import annotations
|
from __future__ import annotations
|
||||||
|
|
||||||
import base64
|
import base64
|
||||||
|
import json
|
||||||
|
import time
|
||||||
from pathlib import Path
|
from pathlib import Path
|
||||||
|
|
||||||
import requests
|
import requests
|
||||||
|
|
||||||
from backend.app.config import settings
|
from backend.app.config import settings
|
||||||
|
from backend.app.services.errors import (
|
||||||
|
MistralAuthError,
|
||||||
|
MistralBatchError,
|
||||||
|
MistralOCREmptyError,
|
||||||
|
MistralRateLimitError,
|
||||||
|
MistralServerError,
|
||||||
|
NetworkError,
|
||||||
|
NON_RETRYABLE_CODES,
|
||||||
|
ProcessingError,
|
||||||
|
classify_requests_error,
|
||||||
|
)
|
||||||
|
|
||||||
|
|
||||||
class MistralClient:
|
class MistralClient:
|
||||||
endpoint = "https://api.mistral.ai/v1/ocr"
|
"""Cliente Mistral para OCR.
|
||||||
|
|
||||||
|
- ``ocr_image``: OCR sincrono de una pagina, con reintentos y backoff.
|
||||||
|
- ``ocr_images_batch``: OCR en lote via la API de Batch de Mistral
|
||||||
|
(endpoint ``/v1/ocr``), subiendo un fichero JSONL. Devuelve un dict
|
||||||
|
``{page_index: markdown}``. Si el job falla o caduca se lanza
|
||||||
|
``MistralBatchError`` para que el llamador caiga al modo sincrono.
|
||||||
|
"""
|
||||||
|
|
||||||
|
base_url = "https://api.mistral.ai/v1"
|
||||||
|
ocr_url = f"{base_url}/ocr"
|
||||||
|
files_url = f"{base_url}/files"
|
||||||
|
batch_url = f"{base_url}/batch/jobs"
|
||||||
|
|
||||||
|
def __init__(self) -> None:
|
||||||
|
self._api_key = settings.mistral_api_key
|
||||||
|
self._model = settings.mistral_ocr_model
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
# API publica
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
def ocr_image(self, image_path: Path) -> str:
|
def ocr_image(self, image_path: Path) -> str:
|
||||||
if not settings.mistral_api_key:
|
self._require_key()
|
||||||
raise RuntimeError("MISTRAL_API_KEY no esta configurada")
|
|
||||||
data_url = self._image_data_url(image_path)
|
data_url = self._image_data_url(image_path)
|
||||||
response = requests.post(
|
return self._retry(lambda: self._ocr_sync(data_url))
|
||||||
self.endpoint,
|
|
||||||
headers={
|
def ocr_images_batch(self, image_paths: list[Path]) -> dict[int, str]:
|
||||||
"Content-Type": "application/json",
|
"""OCR en lote. Devuelve ``{posicion_en_lista: markdown}``.
|
||||||
"Authorization": f"Bearer {settings.mistral_api_key}",
|
|
||||||
},
|
``posicion_en_lista`` es el indice (0-based) dentro de ``image_paths``,
|
||||||
json={
|
NO el numero de pagina del PDF; el llamador hace esa traduccion.
|
||||||
"model": settings.mistral_ocr_model,
|
"""
|
||||||
|
self._require_key()
|
||||||
|
if not image_paths:
|
||||||
|
return {}
|
||||||
|
|
||||||
|
payload = self._build_batch_payload(image_paths)
|
||||||
|
upload = self._retry(lambda: self._upload_batch_file(payload))
|
||||||
|
file_id = upload.get("id")
|
||||||
|
if not file_id:
|
||||||
|
raise MistralBatchError("la subida del fichero batch no devolvio id")
|
||||||
|
|
||||||
|
job = self._retry(lambda: self._create_batch_job(file_id))
|
||||||
|
job_id = job.get("id")
|
||||||
|
if not job_id:
|
||||||
|
raise MistralBatchError("la creacion del job batch no devolvio id")
|
||||||
|
|
||||||
|
final = self._poll_batch_job(job_id)
|
||||||
|
if final.get("status") != "SUCCESS":
|
||||||
|
reason = final.get("errors") or final.get("status") or "sin detalle"
|
||||||
|
raise MistralBatchError(f"job {job_id} finalizo como {final.get('status')}: {reason}")
|
||||||
|
|
||||||
|
output_file_id = final.get("output_file")
|
||||||
|
if not output_file_id:
|
||||||
|
raise MistralBatchError(f"job {job_id} sin output_file")
|
||||||
|
|
||||||
|
content = self._retry(lambda: self._download_file(output_file_id))
|
||||||
|
return self._parse_batch_output(content)
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
# OCR sincrono
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _ocr_sync(self, data_url: str) -> str:
|
||||||
|
try:
|
||||||
|
response = requests.post(
|
||||||
|
self.ocr_url,
|
||||||
|
headers=self._json_headers(),
|
||||||
|
json={
|
||||||
|
"model": self._model,
|
||||||
|
"document": {"type": "image_url", "image_url": data_url},
|
||||||
|
"include_image_base64": False,
|
||||||
|
},
|
||||||
|
timeout=180,
|
||||||
|
)
|
||||||
|
except requests.RequestException as exc:
|
||||||
|
raise classify_requests_error(exc) from exc
|
||||||
|
self._raise_for_status(response)
|
||||||
|
markdown = self._extract_markdown(response.json())
|
||||||
|
if not markdown:
|
||||||
|
raise MistralOCREmptyError()
|
||||||
|
return markdown
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
# Batch: construccion, subida, job, polling, descarga, parseo
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _build_batch_payload(self, image_paths: list[Path]) -> bytes:
|
||||||
|
lines: list[str] = []
|
||||||
|
for idx, path in enumerate(image_paths):
|
||||||
|
data_url = self._image_data_url(path)
|
||||||
|
body = {
|
||||||
"document": {"type": "image_url", "image_url": data_url},
|
"document": {"type": "image_url", "image_url": data_url},
|
||||||
"include_image_base64": False,
|
"include_image_base64": False,
|
||||||
},
|
}
|
||||||
timeout=180,
|
lines.append(
|
||||||
|
json.dumps(
|
||||||
|
{"custom_id": str(idx), "body": body},
|
||||||
|
separators=(",", ":"),
|
||||||
|
ensure_ascii=False,
|
||||||
|
)
|
||||||
|
)
|
||||||
|
return ("\n".join(lines)).encode("utf-8")
|
||||||
|
|
||||||
|
def _upload_batch_file(self, payload: bytes) -> dict:
|
||||||
|
try:
|
||||||
|
response = requests.post(
|
||||||
|
self.files_url,
|
||||||
|
headers={"Authorization": f"Bearer {self._api_key}"},
|
||||||
|
data={"purpose": "batch"},
|
||||||
|
files={"file": ("ocr-batch.jsonl", payload, "application/jsonl")},
|
||||||
|
timeout=300,
|
||||||
|
)
|
||||||
|
except requests.RequestException as exc:
|
||||||
|
raise classify_requests_error(exc) from exc
|
||||||
|
self._raise_for_status(response)
|
||||||
|
return response.json()
|
||||||
|
|
||||||
|
def _create_batch_job(self, file_id: str) -> dict:
|
||||||
|
try:
|
||||||
|
response = requests.post(
|
||||||
|
self.batch_url,
|
||||||
|
headers=self._json_headers(),
|
||||||
|
json={
|
||||||
|
"input_files": [file_id],
|
||||||
|
"model": self._model,
|
||||||
|
"endpoint": "/v1/ocr",
|
||||||
|
},
|
||||||
|
timeout=120,
|
||||||
|
)
|
||||||
|
except requests.RequestException as exc:
|
||||||
|
raise classify_requests_error(exc) from exc
|
||||||
|
self._raise_for_status(response)
|
||||||
|
return response.json()
|
||||||
|
|
||||||
|
def _poll_batch_job(self, job_id: str) -> dict:
|
||||||
|
terminal = {"SUCCESS", "FAILED", "TIMEOUT_EXCEEDED", "CANCELLED"}
|
||||||
|
deadline = time.time() + max(60, settings.mistral_ocr_batch_timeout_seconds)
|
||||||
|
delay = max(2, settings.mistral_ocr_batch_poll_seconds)
|
||||||
|
last: dict = {}
|
||||||
|
while time.time() < deadline:
|
||||||
|
try:
|
||||||
|
response = requests.get(
|
||||||
|
f"{self.batch_url}/{job_id}",
|
||||||
|
headers=self._json_headers(),
|
||||||
|
timeout=60,
|
||||||
|
)
|
||||||
|
except requests.RequestException:
|
||||||
|
time.sleep(delay)
|
||||||
|
continue
|
||||||
|
if response.ok:
|
||||||
|
last = response.json()
|
||||||
|
if last.get("status") in terminal:
|
||||||
|
return last
|
||||||
|
time.sleep(delay)
|
||||||
|
raise MistralBatchError(
|
||||||
|
f"job {job_id} excedio el timeout de polling "
|
||||||
|
f"({settings.mistral_ocr_batch_timeout_seconds}s); ultimo estado: "
|
||||||
|
f"{last.get('status') if last else 'desconocido'}"
|
||||||
)
|
)
|
||||||
if response.status_code == 401:
|
|
||||||
raise RuntimeError("API Key de Mistral invalida")
|
def _download_file(self, file_id: str) -> bytes:
|
||||||
if not response.ok:
|
try:
|
||||||
raise RuntimeError(f"Mistral error {response.status_code}: {response.text}")
|
response = requests.get(
|
||||||
data = response.json()
|
f"{self.files_url}/{file_id}/content",
|
||||||
|
headers=self._json_headers(),
|
||||||
|
timeout=300,
|
||||||
|
)
|
||||||
|
except requests.RequestException as exc:
|
||||||
|
raise classify_requests_error(exc) from exc
|
||||||
|
self._raise_for_status(response)
|
||||||
|
return response.content
|
||||||
|
|
||||||
|
def _parse_batch_output(self, content: bytes) -> dict[int, str]:
|
||||||
|
results: dict[int, str] = {}
|
||||||
|
for raw in content.decode("utf-8", errors="replace").splitlines():
|
||||||
|
line = raw.strip()
|
||||||
|
if not line:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
obj = json.loads(line)
|
||||||
|
except json.JSONDecodeError:
|
||||||
|
continue
|
||||||
|
custom_id = obj.get("custom_id")
|
||||||
|
if custom_id is None:
|
||||||
|
continue
|
||||||
|
try:
|
||||||
|
idx = int(custom_id)
|
||||||
|
except (TypeError, ValueError):
|
||||||
|
continue
|
||||||
|
response = obj.get("response") or {}
|
||||||
|
status_code = response.get("status_code", 200)
|
||||||
|
body = response.get("body") or {}
|
||||||
|
if status_code >= 400 or obj.get("error"):
|
||||||
|
# Pagina fallida en lote: se omite y el llamador cae a sync.
|
||||||
|
continue
|
||||||
|
markdown = self._extract_markdown(body)
|
||||||
|
if markdown:
|
||||||
|
results[idx] = markdown
|
||||||
|
return results
|
||||||
|
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
# Helpers compartidos
|
||||||
|
# ------------------------------------------------------------------
|
||||||
|
|
||||||
|
def _require_key(self) -> None:
|
||||||
|
if not self._api_key:
|
||||||
|
raise ProcessingError("CONFIG", "MISTRAL_API_KEY no esta configurada")
|
||||||
|
|
||||||
|
def _json_headers(self) -> dict[str, str]:
|
||||||
|
return {
|
||||||
|
"Authorization": f"Bearer {self._api_key}",
|
||||||
|
"Content-Type": "application/json",
|
||||||
|
"Accept": "application/json",
|
||||||
|
}
|
||||||
|
|
||||||
|
def _raise_for_status(self, response: requests.Response) -> None:
|
||||||
|
if response.ok:
|
||||||
|
return
|
||||||
|
try:
|
||||||
|
body = response.text
|
||||||
|
except Exception:
|
||||||
|
body = ""
|
||||||
|
status = response.status_code
|
||||||
|
if status == 401:
|
||||||
|
raise MistralAuthError()
|
||||||
|
if status == 429:
|
||||||
|
raise MistralRateLimitError()
|
||||||
|
if status >= 500:
|
||||||
|
raise MistralServerError(status, body)
|
||||||
|
raise ProcessingError("MISTRAL_CLIENT", f"HTTP {status}: {body[:200]}")
|
||||||
|
|
||||||
|
def _retry(self, fn):
|
||||||
|
attempts = max(1, settings.mistral_ocr_retry_attempts)
|
||||||
|
base_delay = max(0.5, settings.mistral_ocr_retry_backoff_base)
|
||||||
|
last_exc: Exception | None = None
|
||||||
|
for attempt in range(attempts):
|
||||||
|
try:
|
||||||
|
return fn()
|
||||||
|
except ProcessingError as exc:
|
||||||
|
if exc.code in NON_RETRYABLE_CODES:
|
||||||
|
raise
|
||||||
|
last_exc = exc
|
||||||
|
except Exception as exc:
|
||||||
|
last_exc = exc
|
||||||
|
if attempt < attempts - 1:
|
||||||
|
time.sleep(base_delay * (2 ** attempt))
|
||||||
|
if last_exc is not None:
|
||||||
|
raise last_exc
|
||||||
|
raise MistralBatchError("reintentos agotados sin excepcion conocida")
|
||||||
|
|
||||||
|
def _extract_markdown(self, data: dict) -> str:
|
||||||
pages = data.get("pages") or []
|
pages = data.get("pages") or []
|
||||||
markdown_pages = [page.get("markdown", "").strip() for page in pages if page.get("markdown")]
|
chunks = [page.get("markdown", "").strip() for page in pages if page.get("markdown")]
|
||||||
if markdown_pages:
|
if chunks:
|
||||||
return "\n\n".join(markdown_pages)
|
return "\n\n".join(chunks)
|
||||||
if data.get("markdown"):
|
if data.get("markdown"):
|
||||||
return data["markdown"].strip()
|
return str(data["markdown"]).strip()
|
||||||
raise RuntimeError("Mistral OCR no devolvio Markdown")
|
return ""
|
||||||
|
|
||||||
def _image_data_url(self, image_path: Path) -> str:
|
def _image_data_url(self, image_path: Path) -> str:
|
||||||
encoded = base64.b64encode(image_path.read_bytes()).decode("ascii")
|
encoded = base64.b64encode(image_path.read_bytes()).decode("ascii")
|
||||||
|
|||||||
+9
-28
@@ -1,8 +1,11 @@
|
|||||||
services:
|
services:
|
||||||
backend:
|
app:
|
||||||
build:
|
build:
|
||||||
context: .
|
context: .
|
||||||
dockerfile: Dockerfile
|
dockerfile: Dockerfile.coolify
|
||||||
|
args:
|
||||||
|
NEXT_PUBLIC_API_URL: ""
|
||||||
|
BACKEND_INTERNAL_URL: http://127.0.0.1:8000
|
||||||
environment:
|
environment:
|
||||||
APP_ENV: production
|
APP_ENV: production
|
||||||
APP_DATA_DIR: /app/data
|
APP_DATA_DIR: /app/data
|
||||||
@@ -10,37 +13,15 @@ services:
|
|||||||
MISTRAL_API_KEY: ${MISTRAL_API_KEY:-}
|
MISTRAL_API_KEY: ${MISTRAL_API_KEY:-}
|
||||||
MISTRAL_OCR_MODEL: ${MISTRAL_OCR_MODEL:-mistral-ocr-latest}
|
MISTRAL_OCR_MODEL: ${MISTRAL_OCR_MODEL:-mistral-ocr-latest}
|
||||||
DEEPGRAM_API_KEY: ${DEEPGRAM_API_KEY:-}
|
DEEPGRAM_API_KEY: ${DEEPGRAM_API_KEY:-}
|
||||||
volumes:
|
|
||||||
- station-data:/app/data
|
|
||||||
expose:
|
|
||||||
- "8000"
|
|
||||||
ports:
|
|
||||||
- "8000"
|
|
||||||
restart: unless-stopped
|
|
||||||
healthcheck:
|
|
||||||
test: ["CMD", "python", "-c", "import urllib.request; urllib.request.urlopen('http://127.0.0.1:8000/health', timeout=5)"]
|
|
||||||
interval: 30s
|
|
||||||
timeout: 10s
|
|
||||||
retries: 3
|
|
||||||
start_period: 20s
|
|
||||||
|
|
||||||
frontend:
|
|
||||||
build:
|
|
||||||
context: ./frontend
|
|
||||||
dockerfile: Dockerfile
|
|
||||||
args:
|
|
||||||
NEXT_PUBLIC_API_URL: ""
|
|
||||||
BACKEND_INTERNAL_URL: http://backend:8000
|
|
||||||
environment:
|
|
||||||
NODE_ENV: production
|
NODE_ENV: production
|
||||||
PORT: 3000
|
PORT: 3000
|
||||||
HOSTNAME: 0.0.0.0
|
HOSTNAME: 0.0.0.0
|
||||||
|
SERVICE_FQDN_APP: ${SERVICE_FQDN_APP:-}
|
||||||
|
SERVICE_URL_APP: ${SERVICE_URL_APP:-}
|
||||||
|
volumes:
|
||||||
|
- station-data:/app/data
|
||||||
expose:
|
expose:
|
||||||
- "3000"
|
- "3000"
|
||||||
ports:
|
|
||||||
- "3000"
|
|
||||||
depends_on:
|
|
||||||
- backend
|
|
||||||
restart: unless-stopped
|
restart: unless-stopped
|
||||||
healthcheck:
|
healthcheck:
|
||||||
test: ["CMD", "node", "-e", "fetch('http://127.0.0.1:3000').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))"]
|
test: ["CMD", "node", "-e", "fetch('http://127.0.0.1:3000').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))"]
|
||||||
|
|||||||
+36
-15
@@ -1,18 +1,39 @@
|
|||||||
services:
|
services:
|
||||||
backend:
|
app:
|
||||||
build: .
|
|
||||||
ports:
|
|
||||||
- "8000:8000"
|
|
||||||
env_file:
|
|
||||||
- .env
|
|
||||||
volumes:
|
|
||||||
- ./data:/app/data
|
|
||||||
frontend:
|
|
||||||
build:
|
build:
|
||||||
context: ./frontend
|
context: .
|
||||||
ports:
|
dockerfile: Dockerfile.coolify
|
||||||
- "3000:3000"
|
args:
|
||||||
|
NEXT_PUBLIC_API_URL: ""
|
||||||
|
BACKEND_INTERNAL_URL: http://127.0.0.1:8000
|
||||||
environment:
|
environment:
|
||||||
NEXT_PUBLIC_API_URL: http://127.0.0.1:8000
|
APP_ENV: production
|
||||||
depends_on:
|
APP_DATA_DIR: /app/data
|
||||||
- backend
|
DATABASE_PATH: /app/data/app.db
|
||||||
|
MISTRAL_API_KEY: ${MISTRAL_API_KEY:-}
|
||||||
|
MISTRAL_OCR_MODEL: ${MISTRAL_OCR_MODEL:-mistral-ocr-4-0}
|
||||||
|
MISTRAL_OCR_BATCH_THRESHOLD: ${MISTRAL_OCR_BATCH_THRESHOLD:-3}
|
||||||
|
MISTRAL_OCR_BATCH_POLL_SECONDS: ${MISTRAL_OCR_BATCH_POLL_SECONDS:-10}
|
||||||
|
MISTRAL_OCR_BATCH_TIMEOUT_SECONDS: ${MISTRAL_OCR_BATCH_TIMEOUT_SECONDS:-1800}
|
||||||
|
MISTRAL_OCR_RETRY_ATTEMPTS: ${MISTRAL_OCR_RETRY_ATTEMPTS:-3}
|
||||||
|
MISTRAL_OCR_RETRY_BACKOFF_BASE: ${MISTRAL_OCR_RETRY_BACKOFF_BASE:-2.0}
|
||||||
|
DEEPGRAM_API_KEY: ${DEEPGRAM_API_KEY:-}
|
||||||
|
NODE_ENV: production
|
||||||
|
PORT: 3000
|
||||||
|
HOSTNAME: 0.0.0.0
|
||||||
|
SERVICE_FQDN_APP: ${SERVICE_FQDN_APP:-}
|
||||||
|
SERVICE_URL_APP: ${SERVICE_URL_APP:-}
|
||||||
|
volumes:
|
||||||
|
- station-data:/app/data
|
||||||
|
expose:
|
||||||
|
- "3000"
|
||||||
|
restart: unless-stopped
|
||||||
|
healthcheck:
|
||||||
|
test: ["CMD", "node", "-e", "fetch('http://127.0.0.1:3000').then(r=>process.exit(r.ok?0:1)).catch(()=>process.exit(1))"]
|
||||||
|
interval: 30s
|
||||||
|
timeout: 10s
|
||||||
|
retries: 3
|
||||||
|
start_period: 30s
|
||||||
|
|
||||||
|
volumes:
|
||||||
|
station-data:
|
||||||
|
|||||||
+67
-10
@@ -10,6 +10,8 @@ export default function DashboardPage() {
|
|||||||
const [subjects, setSubjects] = useState<Subject[]>([]);
|
const [subjects, setSubjects] = useState<Subject[]>([]);
|
||||||
const [settings, setSettings] = useState<SettingsStatus | null>(null);
|
const [settings, setSettings] = useState<SettingsStatus | null>(null);
|
||||||
const [name, setName] = useState("");
|
const [name, setName] = useState("");
|
||||||
|
const [editingSubject, setEditingSubject] = useState<Subject | null>(null);
|
||||||
|
const [editName, setEditName] = useState("");
|
||||||
const [loading, setLoading] = useState(true);
|
const [loading, setLoading] = useState(true);
|
||||||
const [error, setError] = useState("");
|
const [error, setError] = useState("");
|
||||||
|
|
||||||
@@ -31,9 +33,47 @@ export default function DashboardPage() {
|
|||||||
async function createSubject(event: FormEvent) {
|
async function createSubject(event: FormEvent) {
|
||||||
event.preventDefault();
|
event.preventDefault();
|
||||||
if (!name.trim()) return;
|
if (!name.trim()) return;
|
||||||
await api.createSubject(name.trim());
|
try {
|
||||||
setName("");
|
setError("");
|
||||||
await load();
|
await api.createSubject(name.trim());
|
||||||
|
setName("");
|
||||||
|
await load();
|
||||||
|
} catch (err) {
|
||||||
|
setError(err instanceof Error ? err.message : "No se pudo crear la materia");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
function startEdit(subject: Subject) {
|
||||||
|
setEditingSubject(subject);
|
||||||
|
setEditName(subject.name);
|
||||||
|
setError("");
|
||||||
|
}
|
||||||
|
|
||||||
|
async function saveEdit(event: FormEvent) {
|
||||||
|
event.preventDefault();
|
||||||
|
if (!editingSubject || !editName.trim()) return;
|
||||||
|
try {
|
||||||
|
setError("");
|
||||||
|
await api.updateSubject(editingSubject.id, editName.trim());
|
||||||
|
setEditingSubject(null);
|
||||||
|
setEditName("");
|
||||||
|
await load();
|
||||||
|
} catch (err) {
|
||||||
|
setError(err instanceof Error ? err.message : "No se pudo editar la materia");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
async function deleteSubject(subject: Subject) {
|
||||||
|
const confirmed = window.confirm(`Eliminar la materia "${subject.name}" tambien borrara todas sus semanas, archivos subidos, Markdown procesado y exports. Esta accion no se puede deshacer.`);
|
||||||
|
if (!confirmed) return;
|
||||||
|
try {
|
||||||
|
setError("");
|
||||||
|
await api.deleteSubject(subject.id);
|
||||||
|
if (editingSubject?.id === subject.id) setEditingSubject(null);
|
||||||
|
await load();
|
||||||
|
} catch (err) {
|
||||||
|
setError(err instanceof Error ? err.message : "No se pudo eliminar la materia");
|
||||||
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
return (
|
return (
|
||||||
@@ -50,13 +90,15 @@ export default function DashboardPage() {
|
|||||||
{loading ? <Card>Cargando...</Card> : (
|
{loading ? <Card>Cargando...</Card> : (
|
||||||
<div className="grid gap-4 md:grid-cols-2">
|
<div className="grid gap-4 md:grid-cols-2">
|
||||||
{subjects.map((subject) => (
|
{subjects.map((subject) => (
|
||||||
<Link key={subject.id} href={`/subjects/${subject.id}`}>
|
<Card key={subject.id} className="transition hover:border-blue-400/40 hover:bg-blue-500/10">
|
||||||
<Card className="transition hover:border-blue-400/40 hover:bg-blue-500/10">
|
<div className="text-xl font-semibold">{subject.name}</div>
|
||||||
<div className="text-xl font-semibold">{subject.name}</div>
|
<div className="mt-2 text-sm text-slate-400">{subject.slug}</div>
|
||||||
<div className="mt-2 text-sm text-slate-400">{subject.slug}</div>
|
<div className="mt-5 flex flex-wrap gap-2 text-sm">
|
||||||
<div className="mt-5 text-sm text-blue-300">Abrir materia</div>
|
<Link className="rounded-lg bg-blue-600 px-3 py-2 font-medium text-white hover:bg-blue-500" href={`/subjects/${subject.id}`}>Abrir materia</Link>
|
||||||
</Card>
|
<button className="rounded-lg border border-white/10 px-3 py-2 text-slate-200 hover:bg-white/10" type="button" onClick={() => startEdit(subject)}>Editar</button>
|
||||||
</Link>
|
<button className="rounded-lg border border-red-400/30 px-3 py-2 text-red-200 hover:bg-red-500/10" type="button" onClick={() => deleteSubject(subject)}>Eliminar</button>
|
||||||
|
</div>
|
||||||
|
</Card>
|
||||||
))}
|
))}
|
||||||
{subjects.length === 0 && <Card className="md:col-span-2">No hay materias todavia. Crea la primera para empezar.</Card>}
|
{subjects.length === 0 && <Card className="md:col-span-2">No hay materias todavia. Crea la primera para empezar.</Card>}
|
||||||
</div>
|
</div>
|
||||||
@@ -64,6 +106,21 @@ export default function DashboardPage() {
|
|||||||
</section>
|
</section>
|
||||||
|
|
||||||
<aside className="space-y-4">
|
<aside className="space-y-4">
|
||||||
|
{editingSubject && (
|
||||||
|
<Card>
|
||||||
|
<h2 className="text-lg font-semibold">Editar materia</h2>
|
||||||
|
<p className="mt-1 text-sm text-slate-400">Solo cambia el nombre visible. La carpeta interna queda igual para mantener rutas estables.</p>
|
||||||
|
<form className="mt-4 space-y-3" onSubmit={saveEdit}>
|
||||||
|
<Label>Nombre</Label>
|
||||||
|
<Input value={editName} onChange={(event) => setEditName(event.target.value)} />
|
||||||
|
<div className="flex gap-2">
|
||||||
|
<Button className="flex-1" disabled={!editName.trim()}>Guardar</Button>
|
||||||
|
<button className="flex-1 rounded-xl border border-white/10 px-4 py-2 hover:bg-white/10" type="button" onClick={() => setEditingSubject(null)}>Cancelar</button>
|
||||||
|
</div>
|
||||||
|
</form>
|
||||||
|
</Card>
|
||||||
|
)}
|
||||||
|
|
||||||
<Card>
|
<Card>
|
||||||
<h2 className="text-lg font-semibold">Nueva materia</h2>
|
<h2 className="text-lg font-semibold">Nueva materia</h2>
|
||||||
<form className="mt-4 space-y-3" onSubmit={createSubject}>
|
<form className="mt-4 space-y-3" onSubmit={createSubject}>
|
||||||
|
|||||||
@@ -62,6 +62,19 @@ export default function SubjectPage({ params }: { params: { subjectId: string }
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
async function deleteWeek(week: Week) {
|
||||||
|
const confirmed = window.confirm(`Eliminar la semana ${week.number} tambien borrara sus archivos subidos, Markdown procesado y exports. Esta accion no se puede deshacer.`);
|
||||||
|
if (!confirmed) return;
|
||||||
|
try {
|
||||||
|
setError("");
|
||||||
|
await api.deleteWeek(subjectId, week.number);
|
||||||
|
if (editingWeek?.id === week.id) setEditingWeek(null);
|
||||||
|
await load();
|
||||||
|
} catch (err) {
|
||||||
|
setError(err instanceof Error ? err.message : "No se pudo eliminar la semana");
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
return (
|
return (
|
||||||
<Shell>
|
<Shell>
|
||||||
<Link href="/" className="text-sm text-blue-300">Volver a materias</Link>
|
<Link href="/" className="text-sm text-blue-300">Volver a materias</Link>
|
||||||
@@ -78,6 +91,7 @@ export default function SubjectPage({ params }: { params: { subjectId: string }
|
|||||||
<div className="mt-5 flex flex-wrap gap-2 text-sm">
|
<div className="mt-5 flex flex-wrap gap-2 text-sm">
|
||||||
<Link className="rounded-lg bg-blue-600 px-3 py-2 font-medium text-white hover:bg-blue-500" href={`/subjects/${subjectId}/weeks/${week.number}`}>Abrir semana</Link>
|
<Link className="rounded-lg bg-blue-600 px-3 py-2 font-medium text-white hover:bg-blue-500" href={`/subjects/${subjectId}/weeks/${week.number}`}>Abrir semana</Link>
|
||||||
<button className="rounded-lg border border-white/10 px-3 py-2 text-slate-200 hover:bg-white/10" type="button" onClick={() => startEdit(week)}>Editar</button>
|
<button className="rounded-lg border border-white/10 px-3 py-2 text-slate-200 hover:bg-white/10" type="button" onClick={() => startEdit(week)}>Editar</button>
|
||||||
|
<button className="rounded-lg border border-red-400/30 px-3 py-2 text-red-200 hover:bg-red-500/10" type="button" onClick={() => deleteWeek(week)}>Eliminar</button>
|
||||||
</div>
|
</div>
|
||||||
</Card>
|
</Card>
|
||||||
))}
|
))}
|
||||||
|
|||||||
@@ -65,6 +65,16 @@ export const api = {
|
|||||||
body: JSON.stringify({ name }),
|
body: JSON.stringify({ name }),
|
||||||
}));
|
}));
|
||||||
},
|
},
|
||||||
|
async updateSubject(id: number, name: string) {
|
||||||
|
return parseResponse<Subject>(await fetch(`${API_URL}/subjects/${id}`, {
|
||||||
|
method: "PUT",
|
||||||
|
headers: { "Content-Type": "application/json" },
|
||||||
|
body: JSON.stringify({ name }),
|
||||||
|
}));
|
||||||
|
},
|
||||||
|
async deleteSubject(id: number) {
|
||||||
|
return parseResponse<{ status: string }>(await fetch(`${API_URL}/subjects/${id}`, { method: "DELETE" }));
|
||||||
|
},
|
||||||
async weeks(subjectId: number) {
|
async weeks(subjectId: number) {
|
||||||
return parseResponse<Week[]>(await fetch(`${API_URL}/subjects/${subjectId}/weeks`, { cache: "no-store" }));
|
return parseResponse<Week[]>(await fetch(`${API_URL}/subjects/${subjectId}/weeks`, { cache: "no-store" }));
|
||||||
},
|
},
|
||||||
@@ -85,6 +95,9 @@ export const api = {
|
|||||||
body: JSON.stringify({ number, title: title || null }),
|
body: JSON.stringify({ number, title: title || null }),
|
||||||
}));
|
}));
|
||||||
},
|
},
|
||||||
|
async deleteWeek(subjectId: number, weekNumber: number) {
|
||||||
|
return parseResponse<{ status: string }>(await fetch(`${API_URL}/subjects/${subjectId}/weeks/${weekNumber}`, { method: "DELETE" }));
|
||||||
|
},
|
||||||
async sources(subjectId: number, weekNumber: number) {
|
async sources(subjectId: number, weekNumber: number) {
|
||||||
return parseResponse<Source[]>(await fetch(`${API_URL}/subjects/${subjectId}/weeks/${weekNumber}/sources`, { cache: "no-store" }));
|
return parseResponse<Source[]>(await fetch(`${API_URL}/subjects/${subjectId}/weeks/${weekNumber}/sources`, { cache: "no-store" }));
|
||||||
},
|
},
|
||||||
|
|||||||
Reference in New Issue
Block a user