> ## Documentation Index
> Fetch the complete documentation index at: https://anaconda.com/docs/llms.txt
> Use this file to discover all available pages before exploring further.

# Getting started

`anaconda-ai` is a CLI and Python SDK for managing AI models curated by Anaconda. With it, you can list and download quantized LLMs, launch and manage inference servers, and work with a vector database, all from the command line.

## Installation

```bash theme={null}
conda install anaconda-ai
```

## Backends

`anaconda-ai` communicates with a backend to manage models and servers. The following backends are supported:

> [!NOTE]
>
> Anaconda Desktop is currently in beta.
>
> ***
>
> The `ai-catalyst` backend supports multi-site configuration, allowing you to work with multiple Anaconda Platform sites. For setup and usage, see the [anaconda-auth site configuration](https://anaconda.com/docs/cli-reference/anaconda-auth/getting-started.md#site-configuration) documentation and use the [`anaconda sites`](https://anaconda.com/docs/cli-reference/anaconda-auth/commands/sites) CLI commands.

Backend

Description

Supported Features

`anaconda-desktop`

Backend for Anaconda Desktop's [AI features](https://anaconda.com/docs/anaconda-desktop/getting-started-desktop.md#downloading-and-interacting-with-models) (default)

Models, Servers, Server Parameters, Vector Database

`ai-catalyst`

Backend for Anaconda Platform's AI features

Models, Servers, Multi-Site

To change the default backend, run:

```bash theme={null}
anaconda ai config --backend <BACKEND> --yes
```

Comments

Replace \<BACKEND> with the backend you want to use.

You can also override the backend per command with the `--backend` option.

## Configuration

Configuration settings are defined in `~/.anaconda/config.toml` under `[plugin.ai]`. Each parameter can also be set through an environment variable.

Parameter

Environment variable

Description

Default

`backend`

`ANACONDA_AI_BACKEND`

Inference backend to use

`anaconda-desktop`

`stop_server_on_exit`

`ANACONDA_AI_STOP_SERVER_ON_EXIT`

Automatically stop servers when the Python interpreter exits

`true`

`server_operations_timeout`

`ANACONDA_AI_SERVER_OPERATIONS_TIMEOUT`

Timeout in seconds for server start and stop operations

`60`

`show_blocked_models`

`ANACONDA_AI_SHOW_BLOCKED_MODELS`

Display models that are unavailable on the current system

`false`

```toml Example configuration theme={null}
# ~/.anaconda/config.toml
[plugin.ai]
backend = "anaconda-desktop"
stop_server_on_exit = true
server_operations_timeout = 60
show_blocked_models = false
```

## Model reference format

Quantized models follow this reference format:

```text theme={null}
<AUTHOR>/<MODEL>/<QUANT>.<EXT>
```

* `<AUTHOR>`: Publisher name (optional).\

* `<MODEL>`: Model name.\

* `<QUANT>`: Quantization method (`Q4_K_M`, `Q5_K_M`, `Q6_K`, `Q8_0`).\

* `<EXT>`: File extension, usually `.gguf` (optional).

> [!NOTE]
>
> The model name and quantization method must be separated by either `/` or `_`:
>
> ```text theme={null}
> OpenHermes-2.5-Mistral-7B/Q4_K_M
> meta-llama/llama-2-7b-chat-hf/Q4_K_M
> TinyLlama/TinyLlama-1.1B-Chat-v1.0_Q4_K_M.gguf
> ```

Framework integrations (LangChain, LiteLLM, and so on) also support server references in the format `server/<server-id>`. See the [SDK servers documentation](https://anaconda.com/docs/cli-reference/anaconda-ai/sdk/servers.md) for details.

## Basic usage

### List available models

```bash theme={null}
anaconda ai models
```

### View details for a specific model

```bash theme={null}
anaconda ai models OpenHermes-2.5-Mistral-7B
```

### Download a model

```bash theme={null}
anaconda ai download OpenHermes-2.5-Mistral-7B/Q4_K_M
```

### Launch an inference server

```bash theme={null}
anaconda ai launch OpenHermes-2.5-Mistral-7B/Q4_K_M
```

The server starts and remains attached to your terminal. Press `Ctrl+C` to stop and remove the server.

> [!NOTE]
>
> To leave the server running in the background, use the `--detach` flag:
>
> ```bash theme={null}
> anaconda ai launch OpenHermes-2.5-Mistral-7B/Q4_K_M --detach
> ```

### List running servers

```bash theme={null}
anaconda ai servers
```

### Stop a running server

```bash theme={null}
anaconda ai stop <SERVER_ID>
```

Comments

Replace \<SERVER\_ID> with the ID of the server to stop. Use `anaconda ai servers` to find the server ID.

### Vector database

Launch a pgvector database for storing and querying embeddings:

```bash theme={null}
anaconda ai launch-vectordb
```

List, create, and manage database tables:

```bash theme={null}
anaconda ai list-tables
anaconda ai create-table my_table '{"columns": [{"name": "id", "type": "SERIAL PRIMARY KEY"}, {"name": "embedding", "type": "vector(384)"}]}'
anaconda ai drop-table my_table
```

Stop the vector database when finished:

```bash theme={null}
anaconda ai stop-vectordb
```

> [!NOTE]
>
> Vector database functionality requires a compatible backend. Not all backends support vector database operations.

For the full command reference, see the [command index](https://anaconda.com/docs/cli-reference/anaconda-ai/commands).

For the Python SDK and framework integrations, see the [Anaconda AI SDK documentation](https://anaconda.com/docs/cli-reference/anaconda-ai/sdk/overview).