> ## Documentation Index
> Fetch the complete documentation index at: https://anaconda.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# anaconda ai launch

## Usage

```bash theme={null}
anaconda ai launch <MODEL> [OPTIONS] 
```

Comments

Replace \<MODEL> with the model name and quantization.\\

Additional server options can be passed as `--key=value` or `--key` boolean flags.

## Description

Launches an OpenAI-compatible inference server for a quantized model. If the model has not been downloaded, it is downloaded automatically before the server starts.

By default, the server runs in the foreground and is stopped and removed when you press `Ctrl+C`. Use `--detach` to leave the server running in the background after the command exits.

## Arguments

Argument

Default

Description

`MODEL`

Model name with quantization (positional, required)

## Options

Option

Default

Description

`--rm` / `--detach`

`--rm`

`--rm` stops and removes the server on `Ctrl+C`. `--detach` leaves the server running.

`--show`

`false`

Open the server URL in a browser when the server starts

`--backend <BACKEND>`

config value

Select the inference backend

`--at <SITE>`

Target a specific site defined in your configuration

`--json`

`false`

Print output as JSON

`-h, --help`

Show help for this command

### Server options

Additional server options can be appended as `--key=value` pairs or `--key` boolean flags. These options are passed directly to the backend server.

> [!NOTE]
>
> Server options are backend-specific. The options below apply to the `anaconda-desktop` backend and map to [llama-server parameters](https://github.com/ggml-org/llama.cpp/tree/master/tools/server#usage). The `ai-catalyst` backend might support different options.

Common server options for anaconda-desktop include:

Option

Description

`--ctx_size=<N>`

Context window size

`--n_gpu_layers=<N>`

Number of layers to offload to GPU

`--jinja`

Enable Jinja template processing

## Examples

**Launch a server in the foreground**

```bash theme={null}
anaconda ai launch OpenHermes-2.5-Mistral-7B/Q4_K_M
```

**Launch in the background (detached)**

```bash theme={null}
anaconda ai launch OpenHermes-2.5-Mistral-7B/Q4_K_M --detach
```

The command returns to the terminal immediately while the server runs in the background.

**Launch with custom server options**

```bash theme={null}
anaconda ai launch OpenHermes-2.5-Mistral-7B/Q4_K_M --ctx_size=512 --jinja
```

**Launch and open the server in a browser**

```bash theme={null}
anaconda ai launch OpenHermes-2.5-Mistral-7B/Q4_K_M --show
```