> ## Documentation Index
> Fetch the complete documentation index at: https://anaconda.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Servers

The `.servers` accessor provides methods for creating, listing, starting, and stopping servers.

> [!NOTE]
>
> Some methods like `.match()` and the parameter classes below are backend-specific and might not be available on all backends.

Method

Return

Description

`.list()`

`List[Server]`

List running servers

`.get('<server-id>')`

`Server`

Look up a running server by name or UUID

`.match()`

`Server`

Find a running server matching configuration

`.create(model, extra_options=None)`

`Server`

Create new server configuration

`.start('<server-id>')`

`None`

Start the API server

`.status('<server-id>')`

`str`

Get server status

`.stop('<server-id>')`

`None`

Stop a running server

`.delete('<server-id>')`

`None`

Remove server configuration

## Creating servers

The `.create()` method creates a new server configuration. By default, it downloads the model file (if not already downloaded) and selects a random, unused port for the server.

> [!NOTE]
>
> If a server with the specified configuration is already running, the existing configuration is returned, and no new server is created.

Create servers using one of the following approaches:

> [tab] Using model string reference
>
> ```py theme={null}
> from anaconda_ai import AnacondaAIClient
>
> client = AnacondaAIClient()
> server = client.servers.create('OpenHermes-2.5-Mistral-7B/Q4_K_M')
> ```

> [tab] Using QuantizedFile object
>
> ```py theme={null}
> from anaconda_ai import AnacondaAIClient
>
> client = AnacondaAIClient()
> quant = client.models.get('OpenHermes-2.5-Mistral-7B').get_quantization('Q4_K_M')
> server = client.servers.create(quant)
> ```

> [!TIP]
>
> To look up an existing running server rather than create one, use `.get()`. Pass the server's name or UUID as a string:
>
> ```py theme={null}
> server = client.servers.get('my-server-name')
> ```

## Server configuration parameters

The `.create()` method accepts an `extra_options` parameter to customize server behavior:

```py theme={null}
def create(
    model: str,
    download_if_needed: bool = True,
    extra_options: Optional[Dict[str, Any]] = None,
    show_progress: bool = True,
    console: Optional[Console] = None
) -> Server:
```

> [!NOTE]
>
> Server configuration options are backend-specific. The parameters below apply to the `anaconda-desktop` backend.

Common server options include:

Parameter

Type

Description

`host`

`str`

Server host address (default: "127.0.0.1")

`port`

`int`

Server port (0 for random unused port)

`ctx_size`

`int`

Context window size

`n_gpu_layers`

`int`

Number of layers to offload to GPU

`main_gpu`

`int`

Main GPU device ID

`temp`

`float`

Sampling temperature

`top_k`

`int`

Top-k sampling parameter

`top_p`

`float`

Top-p (nucleus) sampling parameter

`batch_size`

`int`

Batch size for processing

`embedding`

`bool`

Enable embedding mode for embedding models

`jinja`

`bool`

Enable Jinja template processing

For example, to create a server with custom configuration:

```py theme={null}
server = client.servers.create(
    'OpenHermes-2.5-Mistral-7B/Q4_K_M',
    extra_options={
        'host': '127.0.0.1',
        'port': 9999,
        'ctx_size': 4096,
        'n_gpu_layers': 20,
        'temp': 0.7,
        'top_p': 0.9
    },
    download_if_needed=False
)
```

## Managing servers

New servers are not automatically started when their configuration is created. You can start or stop a server using the following methods:

> [tab] Using the server object
>
> ```python theme={null}
>     server.start()
>     server.stop()
> ```

> [tab] Using the .servers accessor
>
> ```python theme={null}
>     # Start or stop the server object returned from .create
>     client.servers.start(server)
>     client.servers.stop(server)
> ```

> [tab] Using server as a context manager
>
> This approach automatically starts and stops the server:
>
> ```python theme={null}
>     with client.servers.create('OpenHermes-2.5-Mistral-7B/Q4_K_M') as server:
>         openai_client = server.openai_client()
>         # Make requests to the server
> ```

## Server attributes

Attribute

Description

`.url`

Full server URL (example: `http://127.0.0.1:8000`)

`.openai_url`

URL with `/v1` for OpenAI compatibility

`.openai_client()`

Pre-configured OpenAI client

`async_openai_client()`

Pre-configured async OpenAI client