Initial import: bare-metal stacks, Server UI, GPU fan, tutorials.

Infrastructure configs for GMKtec K11 (Docker, vLLM, LocalAI, ComfyUI, control-plane, gpu-fan agent, Server UI with CLI/file explorer/GPU fan curve). Co-authored-by: Cursor <cursoragent@cursor.com>
2026-07-05 12:02:04 +00:00
commit 359afb3a59
153 changed files with 18169 additions and 0 deletions
@@ -0,0 +1,23 @@
+# Data disk mount point
+DATA_ROOT=/data
+
+# --- REQUIRED before start (no default model) ---
+VLLM_MODEL=
+SERVED_MODEL_NAME=qwen3.6-27b
+
+# --- Context and VRAM ---
+MAX_MODEL_LEN=131072
+MAX_NUM_SEQS=1
+GPU_MEMORY_UTILIZATION=0.95
+KV_CACHE_DTYPE=fp8
+
+# --- Model quantization (leave empty for full-precision models) ---
+QUANTIZATION=awq
+
+# Model-specific vLLM flags (space-separated)
+VLLM_EXTRA_ARGS=--language-model-only --enforce-eager --reasoning-parser qwen3
+
+# Hugging Face token — only for gated models
+HF_TOKEN=
+
+VLLM_PORT=8000
@@ -0,0 +1,182 @@
+# vLLM stack
+
+Serwer inference vLLM z API kompatybilnym z OpenAI. **Brak panelu UI** — konfiguracja przez plik `.env`, profile i katalog modeli.
+
+## Jak to działa
+
+```mermaid
+flowchart LR
+    client["Klient curl / OpenAI SDK"]
+    api["vLLM :8000 /v1/*"]
+    gpu["RTX 3090 Ti"]
+    data["/data/apps/vllm/huggingface"]
+
+    client --> api
+    api --> gpu
+    api --> data
+```
+
+| Element | Opis |
+|---------|------|
+| Obraz | `vllm/vllm-openai` |
+| Port | `8000` (OpenAI-compatible) |
+| Konfiguracja | `.env` + profile + `models.catalog.yaml` |
+| Modele vLLM | Hugging Face AWQ → `/data/apps/vllm/huggingface` |
+| Modele GGUF | Katalog + `/data/apps/gguf/` → przyszły [`stacks/llamacpp/`](../llamacpp/) |
+| UI | **Brak** — opcjonalnie Open WebUI w przyszłości |
+
+## GGUF vs AWQ (ważne)
+
+| Źródło | Format | Runtime |
+|--------|--------|---------|
+| [lmstudio-community GGUF](https://huggingface.co/lmstudio-community) | `.gguf` Q4 | **llama.cpp** (planowany) |
+| Hugging Face AWQ | safetensors INT4 | **vLLM** (teraz) |
+
+Standardowy `vllm/vllm-openai` **nie ładuje plików `.gguf`**. Linki GGUF z katalogu są pod przyszły host llama.cpp. Na vLLM używamy `Qwen/Qwen3.6-27B-Instruct-AWQ` jako odpowiednika Q4.
+
+vLLM = **jeden model w VRAM** na kontener. Kilka modeli może leżeć na dysku — przełączanie = zmiana profilu + restart.
+
+## Mapowanie z LM Studio / Ollama
+
+| LM Studio / Ollama | vLLM |
+|--------------------|------|
+| Model GGUF Q4 (lmstudio) | AWQ z HF + `QUANTIZATION=awq` (interim) |
+| K Cache Q4_0 | `KV_CACHE_DTYPE=fp8` |
+| V Cache Q4_0 | j.w. — vLLM nie ma flagi `Q4_0` |
+| Context 128K | `MAX_MODEL_LEN=131072` |
+| 1 wątek / 1 request | `MAX_NUM_SEQS=1` |
+| GPU layers max | `GPU_MEMORY_UTILIZATION=0.95` |
+
+Docelowo GGUF + natywne K/V `q4_0`: [`stacks/llamacpp/README.md`](../llamacpp/README.md).
+
+## Struktura katalogów
+
+```
+stacks/vllm/
+├── README.md
+├── models.catalog.yaml       # lista modeli (bez auto-pobierania)
+├── docker-compose.yml
+├── .env.example
+├── profiles/
+│   ├── _template.env
+│   └── qwen3.6-27b-awq-128k.env
+└── scripts/
+    ├── catalog-lib.sh
+    ├── list-models.sh
+    ├── download-model.sh
+    ├── switch-model.sh
+    ├── start.sh
+    └── vllm-entrypoint.sh
+```
+
+Na dysku `/data`:
+
+```
+/data/apps/
+├── vllm/huggingface/     # cache HF (AWQ)
+└── gguf/                 # przyszłe GGUF (puste katalogi tworzone przez skrypty)
+    ├── qwen3.6-27b/
+    └── gemma-4-12b/
+```
+
+## Model catalog
+
+Plik `models.catalog.yaml` zawiera modele docelowe (GGUF) i interim (vLLM AWQ). **Nic nie pobiera się przy instalacji.**
+
+```bash
+./scripts/list-models.sh
+```
+
+| ID | Runtime | Opis |
+|----|---------|------|
+| `qwen3.6-27b-q4-gguf` | llamacpp | Qwen3.6-27B Q4_K_M z lmstudio-community |
+| `gemma-4-12b-q4-gguf` | llamacpp | Gemma 4 12B Q4_0 (+ mmproj) |
+| `qwen3.6-27b-awq-vllm` | vllm | AWQ interim — użyj teraz |
+
+## Workflow
+
+### 1. Przygotuj stack (bez modelu)
+
+```bash
+cd /home/tomasz-syn-grzegorza/cursor/ubuntu-bare-metal/stacks/vllm
+cp .env.example .env
+```
+
+### 2. Zobacz katalog
+
+```bash
+./scripts/list-models.sh
+```
+
+### 3. Pobierz model na żądanie
+
+```bash
+# vLLM interim (AWQ → cache HF)
+./scripts/download-model.sh qwen3.6-27b-awq-vllm
+
+# później — GGUF do /data/apps/gguf (dla llama.cpp)
+# ./scripts/download-model.sh qwen3.6-27b-q4-gguf
+```
+
+### 4. Przełącz profil i uruchom
+
+```bash
+./scripts/switch-model.sh qwen3.6-27b-awq-128k
+# lub pierwszy start:
+./scripts/start.sh qwen3.6-27b-awq-128k
+```
+
+`start.sh` odrzuca `.gguf` w `VLLM_MODEL` i wskazuje katalog.
+
+### 5. Logi i test
+
+```bash
+docker compose --profile vllm logs -f vllm
+curl -s http://localhost:8000/v1/models | jq .
+```
+
+## Switching models (A / B na dysku)
+
+1. Model B może już być pobrany (`download-model.sh`) — leży na dysku, nie w VRAM.
+2. Przełącz profil: `./scripts/switch-model.sh <profile>` — kopiuje profil → `.env`, restartuje kontener.
+3. Tylko **jeden** model aktywny w VRAM naraz.
+
+Nowy profil vLLM: skopiuj `profiles/_template.env`, dostosuj `VLLM_EXTRA_ARGS` i `QUANTIZATION`.
+
+## Zmienne `.env`
+
+| Zmienna | Opis | Domyślnie |
+|---------|------|-----------|
+| `VLLM_MODEL` | **Wymagane** — ID modelu Hugging Face | *(pusty)* |
+| `SERVED_MODEL_NAME` | Nazwa w API | `qwen3.6-27b` |
+| `MAX_MODEL_LEN` | Okno kontekstu (tokeny) | `131072` (128K) |
+| `MAX_NUM_SEQS` | Równoległe sekwencje | `1` |
+| `GPU_MEMORY_UTILIZATION` | % VRAM dla vLLM | `0.95` |
+| `KV_CACHE_DTYPE` | Kwantyzacja KV cache | `fp8` |
+| `QUANTIZATION` | Typ kwantyzacji wag (`awq` lub pusty) | `awq` |
+| `VLLM_EXTRA_ARGS` | Dodatkowe flagi vLLM (spacje) | `--language-model-only --enforce-eager --reasoning-parser qwen3` |
+| `DATA_ROOT` | Mount dysku danych | `/data` |
+| `HF_TOKEN` | Token Hugging Face (gated) | *(pusty)* |
+
+`QUANTIZATION` puste = model pełnej precyzji (bez `--quantization`). Flagi buduje `scripts/vllm-entrypoint.sh`.
+
+## Tuning po OOM
+
+1. `MAX_MODEL_LEN=98304` lub `65536`
+2. `GPU_MEMORY_UTILIZATION=0.90`
+3. `KV_CACHE_DTYPE=turboquant_k8v4`
+
+## Zarządzanie
+
+```bash
+docker compose --profile vllm ps
+docker compose --profile vllm logs -f vllm
+docker compose --profile vllm restart vllm
+docker compose --profile vllm down
+```
+
+## Dokumentacja
+
+Pełny tutorial: [manual-tutorial/04-vllm-stack.md](../../manual-tutorial/04-vllm-stack.md) (część B).
+
+GGUF (planowany): [stacks/llamacpp/README.md](../llamacpp/README.md).
@@ -0,0 +1 @@
+docker-compose.yml
@@ -0,0 +1,26 @@
+services:
+  vllm:
+    image: vllm/vllm-openai:latest
+    container_name: vllm
+    profiles:
+      - vllm
+    restart: unless-stopped
+    ipc: host
+    ports:
+      - "${VLLM_PORT:-8000}:8000"
+    environment:
+      - HF_TOKEN=${HF_TOKEN:-}
+      - CUDA_VISIBLE_DEVICES=0
+      - VLLM_MODEL=${VLLM_MODEL}
+      - SERVED_MODEL_NAME=${SERVED_MODEL_NAME:-qwen3.6-27b}
+      - MAX_MODEL_LEN=${MAX_MODEL_LEN:-131072}
+      - MAX_NUM_SEQS=${MAX_NUM_SEQS:-1}
+      - GPU_MEMORY_UTILIZATION=${GPU_MEMORY_UTILIZATION:-0.95}
+      - KV_CACHE_DTYPE=${KV_CACHE_DTYPE:-fp8}
+      - QUANTIZATION=${QUANTIZATION:-}
+      - VLLM_EXTRA_ARGS=${VLLM_EXTRA_ARGS:---language-model-only --enforce-eager --reasoning-parser qwen3}
+    volumes:
+      - ${DATA_ROOT:-/data}/apps/vllm/huggingface:/root/.cache/huggingface
+      - ./scripts/vllm-entrypoint.sh:/usr/local/bin/vllm-entrypoint.sh:ro
+    gpus: all
+    entrypoint: ["/bin/bash", "/usr/local/bin/vllm-entrypoint.sh"]
@@ -0,0 +1,32 @@
+# Model catalog — no automatic download on stack install.
+# Use: ./scripts/list-models.sh
+#      ./scripts/download-model.sh <id>
+
+models:
+  - id: qwen3.6-27b-q4-gguf
+    name: Qwen3.6-27B Q4_K_M
+    runtime: llamacpp
+    hf_repo: lmstudio-community/Qwen3.6-27B-GGUF
+    gguf_file: Qwen3.6-27B-Q4_K_M.gguf
+    download_url: https://huggingface.co/lmstudio-community/Qwen3.6-27B-GGUF/resolve/main/Qwen3.6-27B-Q4_K_M.gguf
+    size_gb: 17
+    local_path: /data/apps/gguf/qwen3.6-27b/Qwen3.6-27B-Q4_K_M.gguf
+
+  - id: gemma-4-12b-q4-gguf
+    name: Gemma 4 12B Q4_0
+    runtime: llamacpp
+    hf_repo: lmstudio-community/gemma-4-12B-it-QAT-GGUF
+    gguf_file: gemma-4-12B-it-QAT-Q4_0.gguf
+    mmproj_file: mmproj-gemma-4-12B-it-QAT-BF16.gguf
+    download_url: https://huggingface.co/lmstudio-community/gemma-4-12B-it-QAT-GGUF/resolve/main/gemma-4-12B-it-QAT-Q4_0.gguf
+    mmproj_url: https://huggingface.co/lmstudio-community/gemma-4-12B-it-QAT-GGUF/resolve/main/mmproj-gemma-4-12B-it-QAT-BF16.gguf
+    size_gb: 7
+    local_dir: /data/apps/gguf/gemma-4-12b
+
+  - id: qwen3.6-27b-awq-vllm
+    name: Qwen3.6-27B AWQ (vLLM interim)
+    runtime: vllm
+    hf_model: Qwen/Qwen3.6-27B-Instruct-AWQ
+    profile: qwen3.6-27b-awq-128k
+    size_gb: 15
+    note: AWQ on vLLM — Q4 equivalent until llama.cpp stack is ready
@@ -0,0 +1,18 @@
+# Template for a new vLLM profile
+# Copy: cp profiles/_template.env profiles/my-model.env
+# GGUF files from lmstudio-community require stacks/llamacpp (future), not vLLM.
+
+DATA_ROOT=/data
+
+VLLM_MODEL=
+SERVED_MODEL_NAME=my-model
+QUANTIZATION=awq
+VLLM_EXTRA_ARGS=--language-model-only --enforce-eager
+
+MAX_MODEL_LEN=131072
+MAX_NUM_SEQS=1
+GPU_MEMORY_UTILIZATION=0.95
+KV_CACHE_DTYPE=fp8
+
+HF_TOKEN=
+VLLM_PORT=8000
@@ -0,0 +1,19 @@
+# Preset: Qwen3.6-27B AWQ, 128K context, RTX 3090 Ti 24GB
+# vLLM interim — Q4 equivalent (not GGUF; use llamacpp for lmstudio GGUF files)
+# Usage: ./scripts/switch-model.sh qwen3.6-27b-awq-128k
+
+DATA_ROOT=/data
+
+VLLM_MODEL=Qwen/Qwen3.6-27B-Instruct-AWQ
+
+SERVED_MODEL_NAME=qwen3.6-27b-awq
+QUANTIZATION=awq
+VLLM_EXTRA_ARGS=--language-model-only --enforce-eager --reasoning-parser qwen3
+
+MAX_MODEL_LEN=131072
+MAX_NUM_SEQS=1
+GPU_MEMORY_UTILIZATION=0.95
+KV_CACHE_DTYPE=fp8
+
+HF_TOKEN=
+VLLM_PORT=8000
@@ -0,0 +1,63 @@
+#!/usr/bin/env bash
+# Shared helpers for models.catalog.yaml (simple parser, no PyYAML required).
+
+CATALOG_FILE="${CATALOG_FILE:-$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)/models.catalog.yaml}"
+
+catalog_list_ids() {
+  grep '^  - id:' "${CATALOG_FILE}" | awk '{print $3}'
+}
+
+catalog_get_field() {
+  local model_id="$1"
+  local field="$2"
+  awk -v id="${model_id}" -v key="${field}:" '
+    $0 ~ "^  - id: " id "$" { found=1; next }
+    found && $0 ~ "^  - id:" { exit }
+    found && index($0, key) == 5 { sub(/^    [^:]+: /, ""); print; exit }
+  ' "${CATALOG_FILE}"
+}
+
+catalog_ensure_dirs() {
+  local data_root="${1:-/data}"
+  mkdir -p \
+    "${data_root}/apps/vllm/huggingface" \
+    "${data_root}/apps/gguf/qwen3.6-27b" \
+    "${data_root}/apps/gguf/gemma-4-12b"
+}
+
+catalog_model_downloaded() {
+  local model_id="$1"
+  local runtime
+  runtime="$(catalog_get_field "${model_id}" runtime)"
+
+  case "${runtime}" in
+    vllm)
+      local hf_model
+      hf_model="$(catalog_get_field "${model_id}" hf_model)"
+      local cache="${DATA_ROOT:-/data}/apps/vllm/huggingface"
+      # Heuristic: HF hub cache contains repo name
+      local repo_path
+      repo_path=$(echo "${hf_model}" | tr '/' '--')
+      if find "${cache}/hub" -maxdepth 3 -type d -name "models--${repo_path}" 2>/dev/null | grep -q .; then
+        return 0
+      fi
+      return 1
+      ;;
+    llamacpp)
+      local local_path local_dir gguf_file
+      local_path="$(catalog_get_field "${model_id}" local_path)"
+      local_dir="$(catalog_get_field "${model_id}" local_dir)"
+      gguf_file="$(catalog_get_field "${model_id}" gguf_file)"
+      if [[ -n "${local_path}" && -f "${local_path}" ]]; then
+        return 0
+      fi
+      if [[ -n "${local_dir}" && -n "${gguf_file}" && -f "${local_dir}/${gguf_file}" ]]; then
+        return 0
+      fi
+      return 1
+      ;;
+    *)
+      return 1
+      ;;
+  esac
+}
@@ -0,0 +1,114 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+STACK_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
+
+# shellcheck disable=SC1091
+source "${SCRIPT_DIR}/catalog-lib.sh"
+
+cd "${STACK_DIR}"
+
+if [[ $# -lt 1 ]]; then
+  echo "Usage: $0 <catalog-model-id>"
+  echo ""
+  echo "Available models:"
+  catalog_list_ids | sed 's/^/  /'
+  exit 1
+fi
+
+MODEL_ID="$1"
+
+if ! catalog_list_ids | grep -qx "${MODEL_ID}"; then
+  echo "ERROR: Unknown model id: ${MODEL_ID}"
+  echo "Run: ./scripts/list-models.sh"
+  exit 1
+fi
+
+if [[ -f .env ]]; then
+  set -a
+  # shellcheck disable=SC1091
+  source .env
+  set +a
+fi
+
+DATA_ROOT="${DATA_ROOT:-/data}"
+catalog_ensure_dirs "${DATA_ROOT}"
+
+RUNTIME="$(catalog_get_field "${MODEL_ID}" runtime)"
+NAME="$(catalog_get_field "${MODEL_ID}" name)"
+
+echo "=== Download: ${NAME} (${MODEL_ID}) ==="
+echo "Runtime: ${RUNTIME}"
+echo ""
+
+case "${RUNTIME}" in
+  vllm)
+    HF_MODEL="$(catalog_get_field "${MODEL_ID}" hf_model)"
+    echo "Target: Hugging Face cache at ${DATA_ROOT}/apps/vllm/huggingface"
+    echo "Model:  ${HF_MODEL}"
+    echo ""
+
+    HF_ENV=()
+    if [[ -n "${HF_TOKEN:-}" ]]; then
+      HF_ENV=(-e "HF_TOKEN=${HF_TOKEN}")
+    fi
+
+    docker run --rm \
+      "${HF_ENV[@]}" \
+      -v "${DATA_ROOT}/apps/vllm/huggingface:/root/.cache/huggingface" \
+      ghcr.io/huggingface/huggingface-cli:latest \
+      download "${HF_MODEL}"
+  ;;
+  llamacpp)
+    DOWNLOAD_URL="$(catalog_get_field "${MODEL_ID}" download_url)"
+    LOCAL_PATH="$(catalog_get_field "${MODEL_ID}" local_path)"
+    LOCAL_DIR="$(catalog_get_field "${MODEL_ID}" local_dir)"
+    GGUF_FILE="$(catalog_get_field "${MODEL_ID}" gguf_file)"
+    MMPROJ_URL="$(catalog_get_field "${MODEL_ID}" mmproj_url)"
+    MMPROJ_FILE="$(catalog_get_field "${MODEL_ID}" mmproj_file)"
+
+    if [[ -n "${LOCAL_PATH}" ]]; then
+      DEST="${LOCAL_PATH}"
+      mkdir -p "$(dirname "${DEST}")"
+      if [[ -f "${DEST}" ]]; then
+        echo "Already exists: ${DEST}"
+      else
+        echo "Downloading ${DEST} ..."
+        wget -c -O "${DEST}" "${DOWNLOAD_URL}"
+      fi
+    elif [[ -n "${LOCAL_DIR}" && -n "${GGUF_FILE}" ]]; then
+      mkdir -p "${LOCAL_DIR}"
+      DEST="${LOCAL_DIR}/${GGUF_FILE}"
+      if [[ -f "${DEST}" ]]; then
+        echo "Already exists: ${DEST}"
+      else
+        echo "Downloading ${DEST} ..."
+        wget -c -O "${DEST}" "${DOWNLOAD_URL}"
+      fi
+      if [[ -n "${MMPROJ_URL}" && -n "${MMPROJ_FILE}" ]]; then
+        MMPROJ_DEST="${LOCAL_DIR}/${MMPROJ_FILE}"
+        if [[ -f "${MMPROJ_DEST}" ]]; then
+          echo "Already exists: ${MMPROJ_DEST}"
+        else
+          echo "Downloading ${MMPROJ_DEST} ..."
+          wget -c -O "${MMPROJ_DEST}" "${MMPROJ_URL}"
+        fi
+      fi
+    else
+      echo "ERROR: Catalog entry missing local_path or local_dir/gguf_file"
+      exit 1
+    fi
+
+    echo ""
+    echo "GGUF saved for future llama.cpp stack (stacks/llamacpp/README.md)."
+    echo "Standard vLLM cannot load .gguf files."
+  ;;
+  *)
+    echo "ERROR: Unknown runtime: ${RUNTIME}"
+    exit 1
+    ;;
+esac
+
+echo ""
+echo "Done."
@@ -0,0 +1,46 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+STACK_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
+
+# shellcheck disable=SC1091
+source "${SCRIPT_DIR}/catalog-lib.sh"
+
+cd "${STACK_DIR}"
+
+if [[ -f .env ]]; then
+  set -a
+  # shellcheck disable=SC1091
+  source .env
+  set +a
+fi
+
+DATA_ROOT="${DATA_ROOT:-/data}"
+catalog_ensure_dirs "${DATA_ROOT}"
+
+echo "=== Model catalog ==="
+echo "Catalog: ${CATALOG_FILE}"
+echo "Data:    ${DATA_ROOT}"
+echo ""
+
+printf "%-24s %-10s %-8s %s\n" "ID" "RUNTIME" "ON DISK" "NAME"
+printf "%-24s %-10s %-8s %s\n" "----" "-------" "-------" "----"
+
+while IFS= read -r model_id; do
+  name="$(catalog_get_field "${model_id}" name)"
+  runtime="$(catalog_get_field "${model_id}" runtime)"
+  if catalog_model_downloaded "${model_id}"; then
+    on_disk="yes"
+  else
+    on_disk="no"
+  fi
+  printf "%-24s %-10s %-8s %s\n" "${model_id}" "${runtime}" "${on_disk}" "${name}"
+done < <(catalog_list_ids)
+
+echo ""
+echo "Download on demand:  ./scripts/download-model.sh <id>"
+echo "Switch vLLM profile: ./scripts/switch-model.sh <profile-name>"
+echo ""
+echo "Profiles in profiles/:"
+ls -1 "${STACK_DIR}/profiles/"*.env 2>/dev/null | xargs -n1 basename | sed 's/^/  /' || true
@@ -0,0 +1,88 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+STACK_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
+
+# shellcheck disable=SC1091
+source "${SCRIPT_DIR}/catalog-lib.sh"
+
+cd "${STACK_DIR}"
+
+PROFILE_ARG="${1:-}"
+
+if [[ -n "${PROFILE_ARG}" ]]; then
+  PROFILE_FILE="${STACK_DIR}/profiles/${PROFILE_ARG}.env"
+  if [[ ! -f "${PROFILE_FILE}" ]]; then
+    echo "ERROR: Profile not found: ${PROFILE_ARG}"
+    echo "       Available:"
+    ls -1 "${STACK_DIR}/profiles/"*.env 2>/dev/null | xargs -n1 basename | sed 's/\.env$//' | sed 's/^/         /'
+    exit 1
+  fi
+  cp "${PROFILE_FILE}" .env
+  echo "Applied profile: ${PROFILE_ARG}"
+fi
+
+if [[ ! -f .env ]]; then
+  echo "ERROR: .env not found. Run: cp .env.example .env"
+  echo "       Or use a profile: ./scripts/start.sh qwen3.6-27b-awq-128k"
+  exit 1
+fi
+
+set -a
+# shellcheck disable=SC1091
+source .env
+set +a
+
+DATA_ROOT="${DATA_ROOT:-/data}"
+
+if [[ -z "${VLLM_MODEL:-}" ]]; then
+  echo "ERROR: VLLM_MODEL is empty in .env"
+  echo "       Set a model, use a profile, or run:"
+  echo "       ./scripts/download-model.sh qwen3.6-27b-awq-vllm"
+  echo "       ./scripts/switch-model.sh qwen3.6-27b-awq-128k"
+  exit 1
+fi
+
+if [[ "${VLLM_MODEL}" == *".gguf"* ]]; then
+  echo "ERROR: VLLM_MODEL points to a .gguf file — standard vLLM cannot load GGUF."
+  echo "       GGUF models are in models.catalog.yaml for future llama.cpp (stacks/llamacpp/)."
+  echo "       For vLLM use AWQ from Hugging Face, e.g.:"
+  echo "       ./scripts/download-model.sh qwen3.6-27b-awq-vllm"
+  echo "       ./scripts/switch-model.sh qwen3.6-27b-awq-128k"
+  exit 1
+fi
+
+if ! mountpoint -q "${DATA_ROOT}" 2>/dev/null; then
+  echo "ERROR: ${DATA_ROOT} is not mounted"
+  echo "       Complete disk setup (tutorial 04 part A) first"
+  exit 1
+fi
+
+catalog_ensure_dirs "${DATA_ROOT}"
+
+if ! docker info &>/dev/null; then
+  echo "ERROR: Docker is not running"
+  exit 1
+fi
+
+chmod +x "${SCRIPT_DIR}/vllm-entrypoint.sh" 2>/dev/null || true
+
+echo "=== vLLM stack ==="
+echo "Model:       ${VLLM_MODEL}"
+echo "Served as:   ${SERVED_MODEL_NAME:-qwen3.6-27b}"
+echo "Context:     ${MAX_MODEL_LEN:-131072} tokens"
+echo "KV cache:    ${KV_CACHE_DTYPE:-fp8}"
+echo "Quantize:    ${QUANTIZATION:-(none)}"
+echo "Data root:   ${DATA_ROOT}"
+echo ""
+
+docker compose --profile vllm pull
+docker compose --profile vllm up -d
+
+echo ""
+echo "Started. Follow logs:"
+echo "  docker compose --profile vllm logs -f vllm"
+echo ""
+echo "Test when ready:"
+echo "  curl -s http://localhost:${VLLM_PORT:-8000}/v1/models | jq ."
@@ -0,0 +1,73 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+STACK_DIR="$(cd "${SCRIPT_DIR}/.." && pwd)"
+
+# shellcheck disable=SC1091
+source "${SCRIPT_DIR}/catalog-lib.sh"
+
+cd "${STACK_DIR}"
+
+PROFILE="${1:-}"
+
+if [[ -z "${PROFILE}" ]]; then
+  echo "Usage: $0 <profile-name>"
+  echo ""
+  echo "Profiles:"
+  ls -1 "${STACK_DIR}/profiles/"*.env 2>/dev/null | xargs -n1 basename | sed 's/\.env$//' | sed 's/^/  /'
+  exit 1
+fi
+
+PROFILE_FILE="${STACK_DIR}/profiles/${PROFILE}.env"
+if [[ ! -f "${PROFILE_FILE}" ]]; then
+  echo "ERROR: Profile not found: ${PROFILE_FILE}"
+  echo ""
+  echo "Available profiles:"
+  ls -1 "${STACK_DIR}/profiles/"*.env 2>/dev/null | xargs -n1 basename | sed 's/\.env$//' | sed 's/^/  /'
+  exit 1
+fi
+
+if [[ -f .env ]]; then
+  set -a
+  # shellcheck disable=SC1091
+  source .env
+  set +a
+fi
+
+DATA_ROOT="${DATA_ROOT:-/data}"
+catalog_ensure_dirs "${DATA_ROOT}"
+
+echo "=== Switch vLLM profile: ${PROFILE} ==="
+cp "${PROFILE_FILE}" .env
+
+set -a
+# shellcheck disable=SC1091
+source .env
+set +a
+
+if [[ "${VLLM_MODEL:-}" == *".gguf"* ]]; then
+  echo "ERROR: VLLM_MODEL points to a .gguf file — vLLM does not support GGUF."
+  echo "       Use stacks/llamacpp (future) or an AWQ Hugging Face model."
+  echo "       See: ./scripts/list-models.sh"
+  exit 1
+fi
+
+if [[ -z "${VLLM_MODEL:-}" ]]; then
+  echo "ERROR: Profile has empty VLLM_MODEL"
+  exit 1
+fi
+
+echo "Model:     ${VLLM_MODEL}"
+echo "Served as: ${SERVED_MODEL_NAME:-qwen3.6-27b}"
+echo "Context:   ${MAX_MODEL_LEN:-131072}"
+echo ""
+
+if docker compose --profile vllm ps -q vllm 2>/dev/null | grep -q .; then
+  echo "Restarting vLLM container ..."
+  docker compose --profile vllm down
+  "${SCRIPT_DIR}/start.sh"
+else
+  echo "vLLM not running. Start with:"
+  echo "  ./scripts/start.sh"
+fi
@@ -0,0 +1,26 @@
+#!/usr/bin/env bash
+set -euo pipefail
+
+ARGS=(
+  --model "${VLLM_MODEL}"
+  --served-model-name "${SERVED_MODEL_NAME:-qwen3.6-27b}"
+  --host 0.0.0.0
+  --port 8000
+  --max-model-len "${MAX_MODEL_LEN:-131072}"
+  --max-num-seqs "${MAX_NUM_SEQS:-1}"
+  --gpu-memory-utilization "${GPU_MEMORY_UTILIZATION:-0.95}"
+  --kv-cache-dtype "${KV_CACHE_DTYPE:-fp8}"
+  --trust-remote-code
+)
+
+if [[ -n "${QUANTIZATION:-}" ]]; then
+  ARGS+=(--quantization "${QUANTIZATION}")
+fi
+
+if [[ -n "${VLLM_EXTRA_ARGS:-}" ]]; then
+  # shellcheck disable=SC2206
+  EXTRA=( ${VLLM_EXTRA_ARGS} )
+  ARGS+=("${EXTRA[@]}")
+fi
+
+exec vllm serve "${ARGS[@]}"