tomasz-syn-grzegorza
359afb3a59
Infrastructure configs for GMKtec K11 (Docker, vLLM, LocalAI, ComfyUI, control-plane, gpu-fan agent, Server UI with CLI/file explorer/GPU fan curve). Co-authored-by: Cursor <cursoragent@cursor.com>
28 lines
847 B
Plaintext
28 lines
847 B
Plaintext
# BGE-Reranker-v2-m3 for LocalAI (llama-cpp / cuda13-llama-cpp backend).
|
|
# Copy to /data/apps/localai/models/bge-reranker-v2-m3-FP16.gguf.yaml after GGUF download.
|
|
#
|
|
# GGUF: bge-reranker-v2-m3-FP16.gguf (~1.1 GB) in same directory as this YAML.
|
|
# API id: bge-reranker-v2-m3-FP16.gguf
|
|
# Endpoint: POST /v1/rerank
|
|
#
|
|
# Do NOT use backend: rerankers for GGUF — that backend is for HuggingFace transformers.
|
|
# cuda13-llama-cpp is selected automatically on the cuda-13 image when backend: llama-cpp.
|
|
|
|
name: bge-reranker-v2-m3-FP16.gguf
|
|
backend: llama-cpp
|
|
reranking: true
|
|
embeddings: false
|
|
description: BGE-Reranker-v2-m3 cross-encoder (FP16 GGUF)
|
|
known_usecases:
|
|
- rerank
|
|
parameters:
|
|
model: bge-reranker-v2-m3-FP16.gguf
|
|
context_size: 8192
|
|
template:
|
|
use_tokenizer_template: true
|
|
function:
|
|
grammar:
|
|
disable: true
|
|
options:
|
|
- use_jinja:true
|