Add a model master
Create a model master by sending a model's serving parameters to the CosmicAC API.
Set the default serving parameters that CosmicAC applies to every new job for a model. To fill those parameters from a published vLLM recipe instead, see Add a model master from a vLLM recipe.
Prerequisites
You need the following before you start:
- A running CosmicAC deployment. See Installation.
- Your CosmicAC base URL, the address where your CosmicAC UI is reachable. CosmicAC serves the API under /api, so each request uses <base-url>/api.
- A terminal with curl.
Add the model master
To add a model master, send a POST request with the model's serving parameters. Use the same format as Set up model masters. This example adds MiniMax M2.5:
curl -X POST <base-url>/api/v1/model-masters \
-H "Content-Type: application/json" \
-d '{
"job_type": "INFERENCE_VLLM",
"base_os_image": "Ubuntu 22.04 + CUDA 13.0",
"disk_gb": 500,
"cuda_driver_version": "CUDA 13.0",
"model_name": "MiniMaxAI/MiniMax-M2.5",
"runtime_image": "vllm/vllm-openai:v0.15.1",
"data_type": "Auto",
"quantisation": null,
"tensor_parallel": 4,
"per_replica_gpu_count": 4,
"gpu_memory_utilisation": 0.85,
"max_model_length": 27000,
"max_concurrent_sequences": 256,
"reasoning_parser": "deepseek_r1",
"multimodal": true,
"replica": 1,
"require_auth_header": true,
"min": ["base_os_image"],
"max": ["gpu_memory_utilisation"],
"inference_param_overrides": {
"root_disk_size_gb": 500,
"env": [
{ "name": "TRUST_REMOTE_CODE", "value": "true" },
{ "name": "SWAP_SPACE", "value": "0" },
{ "name": "ENABLE_EXPERT_PARALLEL", "value": "true" },
{ "name": "ENFORCE_EAGER", "value": "false" }
]
}
}'For the recommended values, see Recommended model parameters.
Runtime image format
Set runtime_image to a Docker image reference, such as vllm/vllm-openai:v0.15.1. CosmicAC serves the model on the image you name here.
The earlier label format, such as vLLM 0.15.0 + CUDA 12.9, no longer works. If a model master still uses that format, CosmicAC serves the model on a fallback image from your deployment rather than the version you set.