Persistent Python Environments

Following the Shell Automation, we avoid installing packages into the ephemeral base environment. Instead, we create Mamba environments directly in your persistent storage.

Overview

Create the environment in your persistent storage.
Register the environment as a Jupyter Kernel.
Automate the registration so it survives restarts.

Step 1: Create a Persistent Environment

Instead of letting Mamba use its default location, we explicitly tell it to create the environment folder inside your mounted personal storage.

# Create an env named 'my-analysis' inside your persistent folder
mamba create -p ~/workspace/Temporary/$SCISERVER_USER_NAME/scratch/envs/my-analysis python=3.12 ipykernel astropy numpy pandas

Note: Using the -p (prefix) flag forces Mamba to write the environment files to a path that survives container restarts.

Step 2: Make Jupyter Aware of Your Environment

JupyterLab needs a kernel spec to know your environment exists.

Activate your new environment:

mamba activate ~/workspace/Temporary/$SCISERVER_USER_NAME/scratch/envs/my-analysis

Register the kernel:

python -m ipykernel install --user --name=my-analysis --display-name "My Analysis"

Crucial Detail: The kernel spec is written to ~/.local/share/jupyter/kernels. This directory is ephemeral. The spec is just a pointer; you will need to re-run this registration whenever you start a new container.

Step 3: Automate Registration

To avoid running the registration command every time you launch a container, add it to your Automation Engine (your persistent .bashrc).

Add this block to your .bashrc file:

# Auto-register your persistent kernel on login
if [ -f ~/workspace/Temporary/$SCISERVER_USER_NAME/scratch/envs/my-analysis/bin/python ]; then
    python -m ipykernel install --user --name=my-analysis --display-name "Python (My Analysis)" 2>/dev/null
    echo "Persistent kernel 'my-analysis' ready."
fi

You can follow this same pattern for more than one environment. Such that you can have all required environments available on restart.

For more details on how to create and edit your persistent .bashrc, see Shell Automation.

Summary of Paths

Concept	Path	Persistence?
Environment Files	`~/workspace/Storage/$SCISERVER_USER_NAME/persistent/envs/...`	✅ Yes
Mamba Cache (pkgs)	`~/workspace/Storage/$SCISERVER_USER_NAME/persistent/mamba_cache`	✅ Yes
Kernel Spec	`~/.local/share/jupyter/kernels/`	❌ No (Automate via .bashrc)

Optional/Advanced: Move the Mamba Cache

To speed up future installs, point the package download cache to persistent storage:

export MAMBA_ROOT_PREFIX=~/workspace/Storage/$SCISERVER_USER_NAME/persistent/mamba_cache