Configuring persistent environments and sample projects

When you create a new project in Data Science & AI Workbench, you must select a template environment for the project. Anaconda provides several template Anaconda environments, along with Python, R, or Hadoop-Spark environments for you to choose from.

If these environments do not suit your needs, you can create new environments and provide them for your users in one of two locations. These “persistent” environments can be placed in the list of template environments that are available upon project creation or in the sample projects gallery for users to clone.

Creating and managing your environments using the process outlined here ensures they are pre-solved and fixed, which not only guarantees consistent operation across different projects and deployments, but also shields the environments from the variability introduced by updates to software dependencies and changes to the conda resolver itself.

This results in stable, reliable bases for your team. For more information and best practices for managing environments, see Managing environments for reproducibility.

Creating your environment

1. Log in to Workbench as a user with administrator permissions.

   The anaconda-enterprise user has the proper permissions.

2. Click Create and select New Project.

3. Select any Environment and Resource Profile, then click Create.

4. Open a session for the project.

5. Open a terminal in the project session.

6. Configure conda to create new environments in the environments directory by running the following command:

   export CONDA_ENVS_DIRS=/opt/continuum/envs
   export CONDA_PKGS_DIRS=/opt/continuum/envs/.pkgs
   

7. Use conda to create or clone your environment.

   Create an environment

   Here is an example command for creating a conda environment:

   # Replace <ENV_NAME> with a name for your new environment
   conda create --yes --name <ENV_NAME> python=3.8 ipykernel
   

   python and ipykernel must be included in every environment you create for Workbench.

Providing your environment to users

There are two methods for providing your environment to your users:

• Place the environment in the sample projects gallery and allow other users to clone the environment as a project.
• Include the environment as an available option in the Environment dropdown when creating a new project.

Both of these methods are accomplished by creating a project template archive (.tar.bz2) file that uses the environment you just created, and then placing the file in the correct directory.

The project template archive file must contain, at a minimum, the anaconda-project.yml file. For more information and help building projects, see the official Anaconda Project documentation.

1. In your terminal, navigate to the project directory by running the following command:

    cd /opt/continuum/project
   

2. Create a directory to store an anaconda-project.yml file for your custom environment by running the following command:

    # Replace <PROJECT> with the name of your project
    mkdir <PROJECT>
   

3. Create a copy of your current project’s anaconda-project.yml file in the directory you just created by running the following command:

    # Replace <PROJECT> with the name of your project
    cp anaconda-project.yml <PROJECT>
   

4. Enter the project directory you just created by running the following command:

    # Replace <PROJECT> with the name of your project
    cd /opt/continuum/project/<PROJECT>
   

5. Using your preferred file editor, edit the anaconda-project.yml file in your project directory to represent your custom environment. This involves updating the name:, description:, packages:, and env_specs: sections of the file.

   Delete the commented section below env_specs: showing available packages. It is likely that these do not align with your custom environment.

   Your project’s name: will display on the Sample Projects grid or the Environments dropdown list after the project is created.

   Example anaconda-project.yml

    # Replace <PROJECT> with your projects name
    # Replace <A_BRIEF_DESCRIPTION> with a description of your environment
    # Replace <ENV_NAME> with the name of the environment you created or cloned
   
    name: <PROJECT>
   
    description: <A_BRIEF_DESCRIPTION>
   
    packages:
      - python=3.8
      - ipykernel
   
    platforms:
      - linux-64
   
    env_specs:
      <ENV_NAME>: {}
   

6. Create an archive file for the project, then set permissions for it by running the following commands:

    # Replace <PROJECT> with your project name
    cd /opt/continuum/project
    tar cfj <PROJECT>.tar.bz2 <PROJECT>
    chmod 644 <PROJECT>.tar.bz2
   

   This archive file is the template that other projects will use as a starting point for their own projects.

7. Move your project archive file to the sample project gallery by running the following command:

    # Replace <PROJECT> with your project name
    mv /opt/continuum/project/<PROJECT>.tar.bz2 /gallery
   

8. Enter the sample gallery directory by running the following command:

    cd /gallery
   

9. (Optional) If you want to include the environment as an available option in the Environment dropdown when creating a new project, add the project archive file name (<PROJECT>.tar.bz2) to the TEMPLATES file. Save and close the file when complete. Otherwise, skip this step.

10. Update the sample projects gallery to include your project by running the following command:

        python reindex.py
    

11. Verify that you can either find and select your sample project on the Sample Projects page, or select your environment from the Environment dropdown when creating a new project.