# STATE — LocalAI runtime

Ostatnia aktualizacja: po wdrożeniu BGE-Reranker-v2-m3 (2026-07-01).

## Kontener

| Element | Wartość |
|---------|---------|
| Nazwa | `localai` |
| Obraz | `localai/localai:v4.4.3-gpu-nvidia-cuda-13` |
| Status | running (healthy) |
| Port hosta | **8070** → 8080 w kontenerze (`0.0.0.0` — LAN) |
| GPU | `CUDA_VISIBLE_DEVICES=0` (RTX 3090 Ti) |
| API auth | `LOCALAI_API_KEY` w compose — 401 bez Bearer |

## Publiczny endpoint (NPMPlus)

| Element | Wartość |
|---------|---------|
| Domena | `https://llm.rtx1.mobile.agency-ai.dev/v1` |
| Upstream | `http://127.0.0.1:8070` (lub LAN `192.168.100.5:8070`) |

## Backendy

| Backend | Zainstalowany |
|---------|---------------|
| `cuda13-llama-cpp` (alias `llama-cpp`) | tak |
| `turboquant` | **nie** (odłożone) |

## Modele (`/data/apps/localai/models/`)

| Model | GGUF | YAML | Status API |
|-------|------|------|------------|
| `gemma-4-12b-it-qat-q4_0` | tak (~6.5 GB) | KV `q8_0`, `flash_attention`, `context_size: 8192` | chat **OK** |
| `bge-m3-FP16.gguf` | tak (1.1 GB) | `embeddings: true`, `known_usecases: [embedding]` | embed **OK**, 1024 dims |
| `bge-reranker-v2-m3-FP16.gguf` | tak (1.1 GB) | `reranking: true`, `known_usecases: [rerank]`, `backend: llama-cpp` | rerank **OK** |

Szablony w repo:

- Gemma KV: [`../profiles/gemma-4-12b-q4-kv-q8.yaml.example`](../profiles/gemma-4-12b-q4-kv-q8.yaml.example)
- BGE embed: [`../profiles/bge-m3-FP16-embedding.yaml.example`](../profiles/bge-m3-FP16-embedding.yaml.example)
- BGE rerank: [`../profiles/bge-reranker-v2-m3-FP16-rerank.yaml.example`](../profiles/bge-reranker-v2-m3-FP16-rerank.yaml.example)

## Weryfikacja (2026-06-30)

| Test | Wynik |
|------|-------|
| `GET /readyz` | 200 |
| `GET /v1/models` (auth) | 200 — 2 modele |
| `POST /v1/chat/completions` (gemma) | 200 |
| `POST /v1/embeddings` (bge-m3) | **200**, wektor **1024** |
| `POST /v1/rerank` (bge-reranker) | **200**, indeks 2 (panda) na top |
| Chat → embed (SINGLE_ACTIVE_BACKEND) | 200 / 200 |
| Embeddings z RTX1 przez publiczną domenę | timeout (hairpin NAT) — dev powinien testować z zewnątrz |

## Ścieżki

| Co | Gdzie |
|----|-------|
| Repo stack | `/home/tomasz-syn-grzegorza/cursor/ubuntu-bare-metal/stacks/localai` |
| Modele runtime | `/data/apps/localai/models` |
| Raport dla dev | [`EMBEDDING-STATUS-REPORT.md`](EMBEDDING-STATUS-REPORT.md) |