I have runned a llm by llama.cpp by command line:
./llama-server.exe -m F:\ws4llamacpp\models\unsloth\Qwen3.5-9B-MTP-GGUF\Qwen3.5-9B-UD-IQ3_XXS.gguf --jinja -np 1 --no-warmup -tb 6 -ub 1408 --poll 100 --prio-batch 3 --fit on --fit-target 1536 --no-mmap --cache-type-k q4_0 --cache-type-v q4_0 --flash-attn on --reasoning on -ngl 99 --ctx-size 65536 --spec-type draft-mtp --spec-draft-n-max 6 --host 0.0.0.0 --port 1234
how to add connection point to the custom model ?
I hava searched some outdated information like "python -m lcb_runner.runner.main --model custom --config my_model_config.yml" https://blog.csdn.net/gitblog_00080/article/details/154597459. It is useless now.
I have runned a llm by llama.cpp by command line:
./llama-server.exe -m F:\ws4llamacpp\models\unsloth\Qwen3.5-9B-MTP-GGUF\Qwen3.5-9B-UD-IQ3_XXS.gguf --jinja -np 1 --no-warmup -tb 6 -ub 1408 --poll 100 --prio-batch 3 --fit on --fit-target 1536 --no-mmap --cache-type-k q4_0 --cache-type-v q4_0 --flash-attn on --reasoning on -ngl 99 --ctx-size 65536 --spec-type draft-mtp --spec-draft-n-max 6 --host 0.0.0.0 --port 1234
how to add connection point to the custom model ?
I hava searched some outdated information like "python -m lcb_runner.runner.main --model custom --config my_model_config.yml" https://blog.csdn.net/gitblog_00080/article/details/154597459. It is useless now.