Initial import: bare-metal stacks, Server UI, GPU fan, tutorials.
Infrastructure configs for GMKtec K11 (Docker, vLLM, LocalAI, ComfyUI, control-plane, gpu-fan agent, Server UI with CLI/file explorer/GPU fan curve). Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -0,0 +1,19 @@
|
||||
# Preset: Qwen3.6-27B AWQ, 128K context, RTX 3090 Ti 24GB
|
||||
# vLLM interim — Q4 equivalent (not GGUF; use llamacpp for lmstudio GGUF files)
|
||||
# Usage: ./scripts/switch-model.sh qwen3.6-27b-awq-128k
|
||||
|
||||
DATA_ROOT=/data
|
||||
|
||||
VLLM_MODEL=Qwen/Qwen3.6-27B-Instruct-AWQ
|
||||
|
||||
SERVED_MODEL_NAME=qwen3.6-27b-awq
|
||||
QUANTIZATION=awq
|
||||
VLLM_EXTRA_ARGS=--language-model-only --enforce-eager --reasoning-parser qwen3
|
||||
|
||||
MAX_MODEL_LEN=131072
|
||||
MAX_NUM_SEQS=1
|
||||
GPU_MEMORY_UTILIZATION=0.95
|
||||
KV_CACHE_DTYPE=fp8
|
||||
|
||||
HF_TOKEN=
|
||||
VLLM_PORT=8000
|
||||
Reference in New Issue
Block a user