Initial import: bare-metal stacks, Server UI, GPU fan, tutorials.

Infrastructure configs for GMKtec K11 (Docker, vLLM, LocalAI, ComfyUI, control-plane, gpu-fan agent, Server UI with CLI/file explorer/GPU fan curve). Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-05 12:02:04 +00:00
commit 359afb3a59
153 changed files with 18169 additions and 0 deletions
@@ -0,0 +1,62 @@
+# STATE — LocalAI runtime
+
+Ostatnia aktualizacja: po wdrożeniu BGE-Reranker-v2-m3 (2026-07-01).
+
+## Kontener
+
+| Element | Wartość |
+|---------|---------|
+| Nazwa | `localai` |
+| Obraz | `localai/localai:v4.4.3-gpu-nvidia-cuda-13` |
+| Status | running (healthy) |
+| Port hosta | **8070** → 8080 w kontenerze (`0.0.0.0` — LAN) |
+| GPU | `CUDA_VISIBLE_DEVICES=0` (RTX 3090 Ti) |
+| API auth | `LOCALAI_API_KEY` w compose — 401 bez Bearer |
+
+## Publiczny endpoint (NPMPlus)
+
+| Element | Wartość |
+|---------|---------|
+| Domena | `https://llm.rtx1.mobile.agency-ai.dev/v1` |
+| Upstream | `http://127.0.0.1:8070` (lub LAN `192.168.100.5:8070`) |
+
+## Backendy
+
+| Backend | Zainstalowany |
+|---------|---------------|
+| `cuda13-llama-cpp` (alias `llama-cpp`) | tak |
+| `turboquant` | **nie** (odłożone) |
+
+## Modele (`/data/apps/localai/models/`)
+
+| Model | GGUF | YAML | Status API |
+|-------|------|------|------------|
+| `gemma-4-12b-it-qat-q4_0` | tak (~6.5 GB) | KV `q8_0`, `flash_attention`, `context_size: 8192` | chat **OK** |
+| `bge-m3-FP16.gguf` | tak (1.1 GB) | `embeddings: true`, `known_usecases: [embedding]` | embed **OK**, 1024 dims |
+| `bge-reranker-v2-m3-FP16.gguf` | tak (1.1 GB) | `reranking: true`, `known_usecases: [rerank]`, `backend: llama-cpp` | rerank **OK** |
+
+Szablony w repo:
+
+- Gemma KV: [`../profiles/gemma-4-12b-q4-kv-q8.yaml.example`](../profiles/gemma-4-12b-q4-kv-q8.yaml.example)
+- BGE embed: [`../profiles/bge-m3-FP16-embedding.yaml.example`](../profiles/bge-m3-FP16-embedding.yaml.example)
+- BGE rerank: [`../profiles/bge-reranker-v2-m3-FP16-rerank.yaml.example`](../profiles/bge-reranker-v2-m3-FP16-rerank.yaml.example)
+
+## Weryfikacja (2026-06-30)
+
+| Test | Wynik |
+|------|-------|
+| `GET /readyz` | 200 |
+| `GET /v1/models` (auth) | 200 — 2 modele |
+| `POST /v1/chat/completions` (gemma) | 200 |
+| `POST /v1/embeddings` (bge-m3) | **200**, wektor **1024** |
+| `POST /v1/rerank` (bge-reranker) | **200**, indeks 2 (panda) na top |
+| Chat → embed (SINGLE_ACTIVE_BACKEND) | 200 / 200 |
+| Embeddings z RTX1 przez publiczną domenę | timeout (hairpin NAT) — dev powinien testować z zewnątrz |
+
+## Ścieżki
+
+| Co | Gdzie |
+|----|-------|
+| Repo stack | `/home/tomasz-syn-grzegorza/cursor/ubuntu-bare-metal/stacks/localai` |
+| Modele runtime | `/data/apps/localai/models` |
+| Raport dla dev | [`EMBEDDING-STATUS-REPORT.md`](EMBEDDING-STATUS-REPORT.md) |