DSPy

The SDK integrates with DSPy using litellm, allowing you to use quantized local models with any DSPy module that relies on the LM interface.

Install the dspy package:

conda install dspy

Only pip install packages in your conda environment once all other packages and their dependencies have been installed. For more information on installing pip packages in your conda environment, see Installing pip packages.

Here is an example of how to use DSPy with Anaconda’s models:

import dspy
import anaconda_ai.integrations.litellm

lm = dspy.LM('anaconda/openhermes-2.5-mistral-7b/q4_k_m')
dspy.configure(lm=lm)

chain = dspy.ChainOfThought("question -> answer")
chain(question="Who are you?")

To use an already running server, use anaconda/server/<server-name>:

lm = dspy.LM('anaconda/server/my-server')

For more information on using the dspy package, see the official documentation.