17 Python

RStudio Connect offers a flexible way of deploying Jupyter Notebooks and reticulated R content against a variety of Python versions.

A compatible version of Python is identified when content is deployed. That Python installation is used any time Python is needed for that content. Package installation and rendering content that utilized Python will use the version of Python discovered at deploy-time.

RStudio Connect allows different content to rely on different versions of Python. For example, Alice’s Jupyter Notebook may require version 2.7.15 of Python while Bob’s Shiny application with reticulate needs Python version 3.7.1. Those two deployments using different Python versions can coexist in RStudio Connect without conflict.

This chapter discusses how to install Python, how RStudio Connect can be configured to support one or more versions of Python, how Python version compatibility is determined, and other configuration settings relevant to Python support in RStudio Connect.

Available Python installations are analyzed at startup. RStudio Connect logs the provided Python versions and whether it can use each installation.

Note that each Python installation is required to have the pip and virtualenv Python packages installed. virtualenv is used to create content-specific environments and pip is used to install Python packages.

Changing any of the configuration items discussed in this chapter requires a restart of RStudio Connect.

17.1 Enabling Python Support

Python support can enabled by setting the Python.Enabled property to true. Note that it is false by default.

; /etc/rstudio-connect/rstudio-connect.gcfg
[Python]
Enabled = true

RStudio Connect also needs to be configured to use a specific Python installation in order to fully support Python content.

The Python.Executable property can be used to specify locations for Python installations. Specify this property once for each Python installation directory.

; /etc/rstudio-connect/rstudio-connect.gcfg
[Python]
Enabled = true
Executable = /shared/Python/2.7.9/bin/python
Executable = /shared/Python/3.5.6/bin/python3
Executable = /shared/Python/3.7.1/bin/python3

The Python.Executable property is permitted to be a symbolic link to a Python installation.

We recommend installing Python from source rather than with a package manager like apt, yum, or zypper. See section 17.2 to learn how to install Python from source.

17.2 Installing Python

The minimum Python 2 version supported is 2.7.9 and the minimum Python 3 supported version is 3.4.0.

Each Python installation must additionally have the pip, setuptools, and virtualenv packages installed.

17.2.1 Build Python from Source

To build Python from source, first, acquire the build dependencies for Python.

In Ubuntu, you can install build dependencies with

apt-get build-dep python
apt-get install libffi-dev libgdbm-dev libsqlite3-dev libssl-dev zlib1g-dev

In RedHat Enterprise Linux and CentOS, you can install build dependencies with

yum-builddep python python-libs
yum install libffi-devel openssl-devel zlib zlib-devel

In SUSE, build dependencies should be installed directly with zypper install <dependency>. A partial list of the libraries that you might need is below.

zypper install \
    automake \
    fdupes \
    gcc \
    gcc-c++ \
    gcc-fortran \
    gdbm-devel \
    gettext-tools \
    gmp-devel \
    intltool \
    libbz2-devel \
    libexpat-devel \
    libffi-devel \
    libnsl-devel \
    lzma-devel \
    make \
    ncurses-devel \
    netcfg \
    openssl-devel \
    pkgconfig \
    readline-devel \
    sqlite-devel \
    xz \
    zlib-devel

Second, you should download and unpack the source tarball for the version of Python that you want to install. To install Python 3.7.1, this might look like the following:

# Download and extract source code
export VERSION=3.7.1 PYTHON_MAJOR=3
wget https://www.python.org/ftp/python/${VERSION}/Python-${VERSION}.tgz
tar -xzvf Python-${VERSION}.tgz
cd Python-${VERSION}

The third step is to configure, make, and install Python. The recommended install location in this case is at /opt/Python/${VERSION}. If you would like to choose a different location, you can change --prefix=.

# Build Python from source
./configure \
    --prefix=/opt/Python/${VERSION} \
    --enable-shared \
    --enable-ipv6 \
    LDFLAGS=-Wl,-rpath=/opt/Python/${VERSION}/lib,--disable-new-dtags
make

# Install Python
sudo make install

It is very important that the Python installation folder is not moved once it is installed from source.

To test that the installation went smoothly, execute:

/opt/Python/${VERSION}/bin/python${PYTHON_MAJOR} --version

The final step is to install pip, setuptools, and virtualenv into the Python installation.

# Install pip
wget https://bootstrap.pypa.io/get-pip.py
sudo /opt/Python/${VERSION}/bin/python${PYTHON_MAJOR} get-pip.py

# Install virtualenv
sudo /opt/Python/${VERSION}/bin/pip install virtualenv

Ensure setuptools is installed in the Python installation by running sudo /opt/Python/${VERSION}/bin/pip list | grep setuptools. If setuptools is missing you can install it via sudo /opt/Python/${VERSION}/bin/pip install setuptools.

Once you have Python installed on the server, it is important to understand how Connect discovers Python and chooses the Python version that an asset will use. 17.4 explains more about this process.

17.3 Upgrading Python

RStudio Connect supports running multiple versions of Python. In most cases, upgrading Python should consist of building the new version of Python and retaining the previous version. We strongly recommend supporting multiple versions of Python instead of upgrading and maintaining a single version of Python. Supporting multiple versions of Python is the best way to ensure applications or reports published with specific package dependencies will continue to run.

After installing another Python version:

  1. Stop RStudio Connect; see 5.1
  2. Add an additional Python.Executable property to the configuration file that points to the newly built Python installation.
  3. Start RStudio Connect; see 5.1

Following an upgrade, content dependent on Python will be rebuilt on-demand. For example, during the next execution of a scheduled Jupyter Notebook, RStudio Connect will automatically reinstall and rebuild all of the necessary packages before rendering the report. During the package updates, RStudio Connect will present a message and spinner indicating to the end user that the rendered Jupyter Notebook will be available after the packages are successfully installed and built for the new version of Python.

Not all packages can be reinstalled and rebuilt on newer versions of Python. Rebuilding and restoring packages can take a significant amount of time and could delay or prevent the rendering of a Jupyter Notebook or reticulated R content.

17.4 Python Version Matching

RStudio Connect attempts to find a Python installation that is appropriate for your content. By default, it applies a “major-minor” matching approach. This algorithm attempts to find a version of Python to use with your content. More deployments will succeed but not always with the same version of Python that is used by the author.

If you would prefer a strict association between authored and deployed Python versions, you can choose to use an “exact” matching approach.

The “major-minor” algorithm requires exact MAJOR.MINOR matching but is flexible about the patch level. This is a useful option when your desktop and server may occasionally have different update cycles when installing bug fix releases.

e.g. If content was deployed with Python 3.4.0 and RStudio Connect is configured with Python 3.4.3, then Python 3.4.3 will be used as the MAJOR.MINOR numbers match (3.4 and 3.4) but if RStudio Connect is configured with only Python 3.5.6 then the content will fail to deploy as the MAJOR.MINOR versions are different (3.4 v. 3.5).

An inconsistent version of Python occasionally causes problems when installing package dependencies. For the best results, make sure that RStudio Connect has access to the same versions of Python used to author content.

The Python version matching approach is controlled with the Python.VersionMatching configuration setting.

major-minor

Find a Python installation that is close to the version of Python used when authoring content, requiring an exact MAJOR.MINOR version match. If a compatible version cannot be found, content will fail to deploy.

  1. Use exact version match.

    e.g. If content was deployed with Python 2.7.10 and RStudio Connect is configured with Python 2.7.9, Python 2.7.10, and Python 2.7.15, then Python 2.7.10 will be used.

  2. If there are matching MAJOR.MINOR releases, use least-greater version.

    e.g. If content was deployed with Python 2.7.9 and RStudio Connect is configured with Python 2.7.10 and Python 2.7.15, then Python 2.7.10 will be used as it is the lowest matching version greater than 2.7.9.

  3. If there are matching MAJOR.MINOR releases, use latest of these.

    e.g. If content was deployed with Python 2.7.15 and RStudio Connect is configured with Python 2.7.9 and Python 2.7.10, then Python 2.7.10 will be used as it is the highest matching version less than 2.7.15.

exact

Finds a Python installation that exactly matches the version of Python used when authoring the deployed content. If a matching version cannot be found, content will fail to deploy.