Instructor

The Instructor integration provides structured output capabilities by dynamically extending the instructor.from_provider() method. This integration is needed until the provider system in Instructor supports entrypoints.

You must import the Anaconda AI Instructor integration module before using instructor.from_provider() with Anaconda models.

Here is a minimal setup example for using Instructor with Anaconda AI:

import instructor
import anaconda_ai.integrations.instructor
from pydantic import BaseModel

class UserInfo(BaseModel):
    name: str
    age: int

client = instructor.from_provider("anaconda/openhermes-2.5-mistral-7b/q4_k_m")

user_info = await client.create(
    model="any-model",  # ignored
    response_model=UserInfo,
    messages=[{"role": "user", "content": "My name is John and I'm 30 years old."}]
)

To use an already running server, use anaconda/server/<server-name>:

client = instructor.from_provider("anaconda/server/my-server")

For more information on using Instructor, see the official documentation.