Taming Python Installation with Pyenv for AI/ML and MLOps Projects

4 min readOct 8, 2024

Python is the beloved language of data scientists, machine learning engineers, and anyone dabbling in the fascinating world of AI. But with its ever-evolving landscape of versions and dependencies, managing your Python environment can quickly turn into a chaotic nightmare. Fear not, fellow data wizards! This blog post introduces you to pyenv, your secret weapon for taming the Python beast and ensuring a smooth, reproducible, and headache-free development experience.

The Python Version Problem

AI/ML practitioners are juggling between multiple projects. One requires Python 3.9 for compatibility with a crucial library, while another demands the shiny new features of Python 3.13. Switching between these versions manually can be a recipe for disaster, leading to broken dependencies and frustrating debugging sessions. System-wide Python installations become a minefield, and virtual environments, while helpful, can still be cumbersome to manage across multiple projects.

Enter Pyenv: The Python Version Manager

pyenv is a lightweight command-line tool that allows you to effortlessly install and switch between multiple Python versions on your system. It acts as a gatekeeper, intercepting Python commands and directing them to the appropriate version based on your project’s needs. This means you can have Python 3.11, 3.12, 3.13, and other versions coexisting peacefully, each neatly tucked away in its own isolated environment.

Why Choose Pyenv?

Simplicity: pyenv’s command-line interface is clean and intuitive, making it easy to learn and use.
Flexibility: Install and manage any Python version, from legacy releases to the latest cutting-edge builds.
Project-Specific Versions: Specify the exact Python version for each project, ensuring consistent and reproducible results.
No Root Privileges Required: Install Python versions locally without needing administrator access, perfect for shared systems or environments where you don’t have root privileges.
Lightweight and Fast: pyenv’s minimal footprint ensures it doesn’t bog down your system, providing quick and efficient version switching.

Choosing the Right Python Version

For new AI/ML and MLOps projects, embracing the latest stable Python release is generally recommended. Newer versions often bring performance enhancements, bug fixes, and exciting new features that can boost your productivity. However, compatibility with existing libraries and frameworks is paramount. Always check the requirements of your production environment before upgrading to ensure a smooth transition. The official Python website provides a list of supported versions, and it’s wise to steer clear of unsupported releases to avoid security vulnerabilities and compatibility issues.

Installing Pyenv

The installation process is straightforward, varying slightly depending on your operating system. The comprehensive instructions on the official pyenv GitHub repository (https://github.com/pyenv/pyenv#installation) provide detailed guidance for different platforms.

Getting Pyenv

For macOS, UNIX and Windows+WSL systems, use the automatic installer:

curl https://pyenv.run | bash

Setting Up Your Shell Environment

For bash and zsh shells, add these lines to both ~/.bashrc ~/.zshrc or ~/.profile:

export PYENV_ROOT="$HOME/.pyenv"
command -v pyenv >/dev/null || export PATH="$PYENV_ROOT/bin:$PATH"
eval "$(pyenv init -)"

Restart Your Shell

Finally, restart your shell to update your shell PATH:

exec "$SHELL"

Important: Be sure to install Python build dependencies on your system after the installation.

Using Pyenv

Once installed, using pyenv is a breeze:

Install a Python Version:

pyenv install 3.12.0

Set a Global Version (Optional):

pyenv global 3.12.0

Set a Local Version (Project-Specific): Create a .python-version file in your project’s root directory containing the desired version (e.g., 3.12.0). pyenv automatically switches to this version when you enter the project directory.

# in .python-version
3.12

Check Active Version:

pyenv version

Example Workflow

Let’s say you’re starting a new project that requires Python 3.12:

mkdir my-mlops-project
cd my-mlops-project
pyenv install 3.12
echo "3.12" > .python-version
python --version  # Verify the correct version is active

Now, whenever you work within the my-mlops-project directory, pyenv ensures you’re using the correct Python version.

Conclusion

pyenv is an indispensable tool for any data scientist or MLOps engineer working with Python. It simplifies version management, promotes project isolation, and ensures reproducibility, allowing you to focus on what you do best: crafting innovative AI/ML solutions. So, ditch the Python version headaches and embrace the power of pyenv! Your future self will thank you.

To learn more about best practices for MLOps Coding, check this course: https://mlops-coding-course.fmind.dev/