14 Connect Server API Cookbook

This chapter contains recipes for interacting with the Connect Server API from code. Most recipes are written in R, but the recipes are intended to be straightforward enough to be implemented in any programming language you wish to use.

14.1 Getting Started

You will need to know the URL for your RStudio Connect server and an API Key for your account on the server.

Your RStudio Connect server URL is the same URL you use to access the RStudio Connect dashboard, minus the connect path. If you access the dashboard at https://rsc.company.com/connect/, the server URL is https://rsc.company.com/.

The API Keys chapter (5) explains how to provision an RStudio Connect API Key. We recommend that you create an API Key for each different application that needs API access to RStudio Connect.

Use environment variables to obtain the RStudio Connect server URL and API Key. Environment variables keep the literal URL and Key values from your source code, meaning you can share that code without worrying about accidentally sharing your API Key.

Here is a sample .Renviron file that you can use in your development environment. It defines the server URL and API Key you use while developing. The .Renviron file is loaded by every R session spawned under your user account.

# ~/.Renviron
# The CONNECT_SERVER URL must have a trailing slash.
CONNECT_SERVER="https://rsc.company.com/"
CONNECT_API_KEY="mysupersecretapikey"

Your code will obtain the RStudio Connect Server URL and API Key from the environment. The Sys.getenv function lets us load environment variables in R.

connectServer <- Sys.getenv("CONNECT_SERVER")
apiKey <- Sys.getenv("CONNECT_API_KEY")

14.1.1 Environments and Environment Variables

Configuring your API Key and RStudio Connect server URL with environment variables gives you quite a bit of flexibility when deploying your code into different RStudio Connect servers running in different environments.

Code that uses Sys.getenv to load the server URL and API Key can be used in development, staging, or production environments. The same code can run in all three environments.

Use .Renviron files to configure environment variables in development.

Give these environment variables values when code is deployed to RStudio Connect. The “Vars” tab in the RStudio Connect dashboard lets you configure environment variables for each piece of content. Section 4.6 discusses how to use the “Vars” tab to configure environment variables.

14.1.2 Sticky Sessions

RStudio Connect can be deployed with multiple instances in a highly available, load-balanced configuration. The load balancer routing traffic to these RStudio Connect servers must be configured with sticky sessions using session cookies.

The High Availability and Load Balancing chapter of the RStudio Connect Admin Guide provides details about running a cluster of RStudio Connect instances.

Sticky Session cookies are returned by the load balancer to a client with the first HTTP response. The client adds that cookie to all subsequent requests. The load balancer uses session cookies to determine what server should receive the incoming request.

RStudio Connect needs sticky sessions so requests from the same client can be routed to the same server. This is how your browser maintains connectivity to the server running a Shiny application on your behalf.

14.1.2.1 `curl`

The curl command-line utility can use an on-disk cookie jar to receive and send HTTP cookies, including those used for sticky sessions. The -c and --cookie-jar options tell curl to write cookies to the named file. The -b and --cookie options tell curl to read cookies from that file.

#
# Write cookies from our first request.
curl -c cookie-jar.txt \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    "http://rsc.company.com/content/24/mean?samples=5"
# Use those session cookies later.
curl -b cookie-jar.txt \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    "http://rsc.company.com/content/24/mean?samples=5"

The Cookies chapter of the Everything curl book has more information about using cookies with curl.

14.1.2.2 R with `httr`

The httr R package automatically maintains cookies across requests within an R session; no additional code is needed.

library(httr)

connectServer <- Sys.getenv("CONNECT_SERVER")
connectAPIKey <- Sys.getenv("CONNECT_API_KEY")

# The initial request in an R session will have no HTTP session cookies.
resp <- httr::GET(connectServer, 
    path = "/content/24/mean", 
    query = list(samples = 5),
    add_headers(Authorization = paste0("Key ", connectAPIKey)))
# ...

# Later requests retain cookies set by the previous request.
resp <- httr::GET(connectServer, 
    path = "/content/24/mean", 
    query = list(samples = 10),
    add_headers(Authorization = paste0("Key ", connectAPIKey)))
# ...

14.1.2.3 Python2 with `urllib2`

The cookielib module is part of the Python2 standard library. This is a basic example that retains cookies in-memory within a Python process.

import cookielib
import os
import urllib
import urllib2
import urlparse

connect_server = os.getenv("CONNECT_SERVER")
connect_api_key = os.getenv("CONNECT_API_KEY")

def build_url(base, path, **kwargs):
    query = urllib.urlencode(kwargs)
    parts = urlparse.urlparse(base)
    parts = parts._replace(path = path, query = query)
    return parts.geturl()

jar = cookielib.CookieJar()
processor = urllib2.HTTPCookieProcessor(jar)
opener = urllib2.build_opener(processor)

headers = { "Authorization": "Key %s" % connect_api_key }

# The initial request using the cookie jar will have no HTTP session cookies.
request_url = build_url(connect_server, "/content/24/mean", samples = 5)
request = urllib2.Request(request_url, headers = headers)
response = opener.open(request)
# ...

# Later requests retain cookies set by the previous request.
request_url = build_url(connect_server, "/content/24/mean", samples = 10)
request = urllib2.Request(request_url, headers = headers)
response = opener.open(request)
# ...

14.1.2.4 Python3 with `urllib`

The http.cookiejar package is part of the Python3 standard library. This is a basic example that retains cookies in-memory within a Python process.

import http.cookiejar
import json
import os
import urllib.parse
import urllib.request

connect_server = os.getenv("CONNECT_SERVER")
connect_api_key = os.getenv("CONNECT_API_KEY")

def build_url(base, path, **kwargs):
    query = urllib.parse.urlencode(kwargs)
    parts = urllib.parse.urlparse(base)
    parts = parts._replace(path = path, query = query)
    return parts.geturl()

jar = http.cookiejar.CookieJar()
processor = urllib.request.HTTPCookieProcessor(jar)
opener = urllib.request.build_opener(processor)

headers = { "Authorization": "Key %s" % connect_api_key }

# The initial request using the cookie jar will have no HTTP session cookies.
request_url = build_url(connect_server, "/content/24/mean", samples = 5)
request = urllib.request.Request(request_url, headers = headers)
response = opener.open(request)
# ...

# Later requests retain cookies set by the previous request.
request_url = build_url(connect_server, "/content/24/mean", samples = 10)
request = urllib.request.Request(request_url, headers = headers)
response = opener.open(request)
# ...

14.2 Recipes

14.2.1 R Versions Available to RStudio Connect

This recipe compares your local R version against the R installations available on your RStudio Connect server. It uses the GET /server_settings/r endpoint to obtain the R installations available to RStudio Connect.

Obtain the server URL and API Key from environment variables
Obtain your local R version using R.version
Call the “Get R Installation Info” endpoint. See the API Documentation for more information.
Parse the response using httr::content.
Check the response for the local R version. If it is not listed, the RStudio Connect server does not contain the local R version.

# set up environment
# Note that the connectServer string must have a trailing slash
connectServer <- Sys.getenv("CONNECT_SERVER")
apiKey <- Sys.getenv("CONNECT_API_KEY")

library(httr)
myRVersion <- paste(R.version$major, R.version$minor, sep = ".")
resp <- GET(
    paste0(connectServer, "__api__/v1/server_settings/r"),
    add_headers(Authorization = paste("Key", apiKey))
)
resp <- content(resp, as="parsed")
if (myRVersion %in% unlist(resp)) {
    print("The local R version was found on the RStudio Connect server")
} else {
    print(paste("Cannot find R version", myRVersion,"on the RStudio Connect server"))
}

14.2.2 Keyset (Cursor) Pagination

The following snippet pages through the audit logs, which uses keyset pagination, starting from the most recent entries, 25 entries at a time.

Obtain the server URL and API Key from environment variables.
Call the “Get audit logs” endpoint to get the first page. See the API Documentation for more information.
Parse the response using httr::content.
Print the current page.
Repeat steps 1 through 3 until there are no more results. Note also that the paging.next property in the response is the URL of the next page.

# set up environment
library(httr)
# Note that the connectServer string must have a trailing slash
connectServer <- Sys.getenv("CONNECT_SERVER")
apiKey <- Sys.getenv("CONNECT_API_KEY")

# get audit logs
authHeader <- add_headers(Authorization = paste("Key", apiKey))
resp <- GET(
  paste0(connectServer, "__api__/v1/audit_logs?ascOrder=false&limit=25"),
  authHeader
)
payload <- content(resp)
# print first 25!
print(payload$results)
# now step through the remaining audit logs
while(!is.null(payload$paging[["next"]])) {
  resp <- GET(payload$paging[["next"]], authHeader)
  payload <- content(resp)
  # print the next 25
  print(payload$results)
}

14.2.3 Offset Pagination

The following snippet pages through the user’s list, which uses offset pagination, 25 entries at a time.

Obtain the server URL and API Key from environment variables.
Call the “Get all users” endpoint to get the first page. See the API Documentation for more information.
Parse the response using httr::content.
Print the current page.
Repeat steps 1 through 3 until there is no more “next” page. Note also that the query parameter page_number determines the page to return.

# set up environment
library(httr)
# Note that the connectServer string must have a trailing slash
connectServer <- Sys.getenv("CONNECT_SERVER")
apiKey <- Sys.getenv("CONNECT_API_KEY")

# get user's list
authHeader <- add_headers(Authorization = paste("Key", apiKey))
apiPrefix <- "__api__/v1/users?page_size=25"
resp <- GET(
  paste0(connectServer, apiPrefix),
  authHeader
)
# get the first page
payload <- content(resp)
# and step through the pages, printing out the results (if any)
while(length(payload$result) > 0) {
  # print the result
  print(payload$results)
  # get the next page
  nextPage <- payload$current_page + 1
  resp <- GET(
    paste0(connectServer, apiPrefix, "&page_number=", nextPage),
    authHeader
  )
  payload <- content(resp)
}

14.2.4 Create an RStudio Connect User from LDAP or OAuth2

The following snippets search for a user in LDAP or OAuth2 and then create an RStudio Connect account for that user.

There are two steps:

Search for the user via the GET /users/remote endpoint. A user with no account on Connect will lack a guid. Note the temp_ticket for the desired user account.
Use the PUT /users endpoint with the temp_ticket to create a corresponding account on RStudio Connect.

14.2.4.1 Search for a User

# set up environment
library(httr)
# Note that the connectServer string must have a trailing slash
connectServer <- Sys.getenv("CONNECT_SERVER")
apiKey <- Sys.getenv("CONNECT_API_KEY")

# set the search parameter
prefix <- "julie"

# make the query request
authHeader <- paste("Key", apiKey)
response <- GET(
    paste0(connectServer, "__api__/v1/users/remote"),
    add_headers(Authorization = authHeader),
    query = list(prefix = prefix)
)

results <- content(response)$results

# print the results of the API call
formatGuid <- function(guid) {
    if (is.null(guid))
        "NULL"
    else
        guid
}
cat(sprintf("FIRST\tLAST\tUSERNAME\tGUID\n"))
for (user in results) {
    cat(
        sprintf(
            "%s\t%s\t%s\t\t%s\n",
            user$first_name,
            user$last_name,
            user$username,
            formatGuid(user$guid)
        )
    )
}

The output looks like the following:

FIRST   LAST    USERNAME    GUID
Julie   Goolly  julie1      15f5f51d-08ff-4e5b-beba-4ccf24e248dd
Julie   Jolly   julie2      NULL

In this particular case, there are two users matching the search for the prefix julie. The user julie1 has a GUID value, which means that this user already has an account in RStudio Connect. The user julie2 does not yet have an account in RStudio Connect.

Included in the API response for each user is a temp_ticket value that can be used to give the user an account in RStudio Connect. In the example above, the second user, julie2, needs an account, so you will need that user’s temp_ticket:

tempTicket <- results[[2]]$temp_ticket

You can use this tempTicket value in the next section to create the account.

14.2.4.2 Create an RStudio Connect User Account

Using the tempTicket value from the previous section, you can give the user an RStudio Connect account with an HTTP PUT request:

# use a tempTicket value from searching /users/remote

response <- PUT(
    paste0(connectServer, "__api__/v1/users"),
    add_headers(Authorization = authHeader),
    body = list(temp_ticket = tempTicket),
    encode = "json"
)

print(content(response))

When the call succeeds, the response will contain a non-NULL guid value, which is a unique identifier for the user account.

If the user already exists in Connect, the response will contain an error:

$error
[1] "The requested username is already in use."

14.2.5 Create an RStudio Connect Group from LDAP

The following snippets search for a group in LDAP and then create an RStudio Connect group for that LDAP group.

There are two steps:

Search for the group via the GET /groups/remote endpoint. A group that does not exist on Connect will lack a guid. Note the temp_ticket for the desired group.
Use the PUT /groups endpoint with the temp_ticket to create a corresponding group on RStudio Connect.

14.2.5.1 Search for a Group

# set up environment
library(httr)
# Note that the connectServer string must have a trailing slash
connectServer <- Sys.getenv("CONNECT_SERVER")
apiKey <- Sys.getenv("CONNECT_API_KEY")

# set the search parameter
prefix <- "accounting"

# make the query request
authHeader <- paste("Key", apiKey)
response <- GET(
    paste0(connectServer, "__api__/v1/groups/remote"),
    add_headers(Authorization = authHeader),
    query = list(prefix = prefix)
)

results <- content(response)$results

# print the results of the API call
formatGuid <- function(guid) {
    if (is.null(guid))
        "NULL"
    else
        guid
}
cat(sprintf("NAME\tGUID\n"))
for (group in results) {
    cat(
        sprintf(
            "%s\t%s\n",
            group$name,
            formatGuid(group$guid)
        )
    )
}

The output looks like the following:

NAME          GUID
accounting1   15f5f51d-08ff-4e5b-beba-4ccf24e248dd
accounting2   NULL

In this particular case, there are two groups matching the search for the prefix accounting. The group accounting1 has a GUID value, which means that this group already exists in RStudio Connect. The group accounting2 does not yet have a corresponding group in RStudio Connect.

Included in the API response for each group is a temp_ticket value that can be used to create the group in RStudio Connect. In the example above, the second group, accounting2, does not exist in RStudio Connect, so you will need the temp_ticket for this group:

tempTicket <- results[[2]]$temp_ticket

You can use this tempTicket value in the next section to create the group.

14.2.5.2 Create an RStudio Connect Group

Using the tempTicket value from the previous section, you can create an RStudio Connect group with an HTTP PUT request:

# use a tempTicket value from searching /groups/remote

response <- PUT(
    paste0(connectServer, "__api__/v1/groups"),
    add_headers(Authorization = authHeader),
    body = list(temp_ticket = tempTicket),
    encode = "json"
)

print(content(response))

When the call succeeds, the response will contain a non-NULL guid value, which is a unique identifier for the group.

If the group already exists in Connect, the response will contain an error:

$error
[1] "The requested group name is already in use."

14.2.6 User Activity

RStudio Connect records different types of user activity for different types of content.

Shiny applications - records information about each visit and the length of that visit.
Static and rendered content - records information about each visit.

Static content includes plots and other HTML content not rendered by the server. Rendered content includes R Markdown documents, parameterized R Markdown, and Jupyter notebooks.

The GET /instrumentation/shiny/usage Shiny usage API provides details about Shiny application sessions. Sample code for this API is in Section 14.2.6.1.

The GET /instrumentation/content/visits content visit API gives details about visits to static and rendered content. Section 14.2.6.2 contains sample code for this API.

An R Markdown dashboard using instrumentation APIs can be found in the sol-eng/usage GitHub repository.

14.2.6.1 User Activity: Shiny Applications

This recipe uses the GET /instrumentation/shiny/usage Shiny usage API to page through Shiny application usage. Information is returned in pages following the keyset pagination model.

The keyset pagination recipe explains how to perform multiple, paged requests.

Obtain the server URL and API Key from environment variables.
Call the “Get shiny app usage” endpoint to get the first page. See the API Documentation for more information.
Parse the response using httr::content.
Print the current page.
Repeat steps 1 through 3 until there are no more results. Note also that the paging.next property in the response is the URL of the next page.

# set up environment
library(httr)
# Note that the connectServer string must have a trailing slash
connectServer <- Sys.getenv("CONNECT_SERVER")
apiKey <- Sys.getenv("CONNECT_API_KEY")

# get usage information
authHeader <- add_headers(Authorization = paste("Key", apiKey))
usageURL <- paste0(connectServer, "__api__/v1/instrumentation/shiny/usage?limit=25")
resp <- GET(usageURL, authHeader)
payload <- content(resp)
# print first 25!
print(payload$results)
# now step through the remaining audit logs
while(!is.null(payload$paging[["next"]])) {
  resp <- GET(payload$paging[["next"]], authHeader)
  payload <- content(resp)
  # print the next 25
  print(payload$results)
}

14.2.6.2 User Activity: Rendered and Static Content

This recipe uses the GET /instrumentation/content/visits content visit API to page through information about static and rendered content visits. Information is returned in pages following the keyset pagination model.

The keyset pagination recipe explains how to perform multiple, paged requests.

Obtain the server URL and API Key from environment variables.
Call the “Get rendered/static content visits” endpoint to get the first page. See the API Documentation for more information.
Parse the response using httr::content.
Print the current page.
Repeat steps 1 through 3 until there are no more results. Note also that the paging.next property in the response is the URL of the next page.

# set up environment
library(httr)
# Note that the connectServer string must have a trailing slash
connectServer <- Sys.getenv("CONNECT_SERVER")
apiKey <- Sys.getenv("CONNECT_API_KEY")

# get visit information
authHeader <- add_headers(Authorization = paste("Key", apiKey))
hitsURL <- paste0(connectServer, "__api__/v1/instrumentation/content/visits?limit=25")
resp <- GET(hitsURL, authHeader)
payload <- content(resp)
# print first 25!
print(payload$results)
# now step through the remaining audit logs
while(!is.null(payload$paging[["next"]])) {
  resp <- GET(payload$paging[["next"]], authHeader)
  payload <- content(resp)
  # print the next 25
  print(payload$results)
}

14.2.7 Deploying Content

This section explains how to use Connect Server APIs to create content in RStudio Connect and deploy code associated with that content. These APIs can be used for any type of content supported by RStudio Connect, including Shiny applications, R Markdown notebooks, Plumber APIs, and Jupyter notebooks.

The deployment APIs are experimental and will continue to evolve in upcoming RStudio Connect releases. Please try using these APIs to build your own deployment tools. Let your Customer Success representative know about your experience!

The rstudio/connect-api-deploy-shiny GitHub repository contains a sample Shiny application and uses the recipes in this section in deployment scripts that you can use as examples when building your own workflows.

The Connect Server API Reference contains documentation for each of the endpoints used in these recipes.

These recipes use bash snippets and rely on curl to perform HTTP requests. We use the CONNECT_SERVER and CONNECT_API_KEY environment variables introduced in the Getting Started section of this cookbook.

These recipes do not prescribe a single workflow. Some example workflows include:

Automate the creation of a new, uniquely named Shiny application every quarter to analyze the sales leads. The latest quarter contains new dashboard functionality but you cannot alter prior quarters; those applications need to capture that point-in-time.
A Plumber API that receives updates after the R code supporting that API is fully tested by your continuous integration environment. These tests confirm that all updates remain backwards-compatible.
A team collaborates on an application over the course of a two week sprint cycle. The code is shared with Git and progress tracked in GitHub issues. The team performs production updates at the end of each sprint with a Docker-based deployment environment.
Your organization does not permit data scientists to publish directly to the production server. Production updates are scheduled events and gated by successful user-acceptance testing. A deployment engineer, who is not an R user, uses scripts to create and publish content in production by interacting with the Connect Server APIs.

14.2.7.1 Workflow

The content deployment workflow includes several steps:

Create a new content item; content can receive multiple deployments (Section 14.2.7.2).
Create a bundle capturing your code and its dependencies (Section 14.2.7.3).
Upload the bundle archive to RStudio Connect (Section 14.2.7.4).
Deploy (activating) that bundle and monitor its progress (Section 14.2.7.5).

You can choose to create a new content item with each deployment or repeatedly target the same content item. It is good practice to re-use an existing content item as you continue to develop that application or report. Create new content items for new artifacts.

You must create a new content item when changing the type of content. You cannot deploy a bundle containing an R Markdown document to a content item already running a Shiny application.

14.2.7.2 Creating Content

The POST /experimental/content content creation API is used to create a new content item in RStudio Connect. It takes a JSON document as input.

The Content definition in the Connect Server API Reference describes the full set of fields that may be supplied to this endpoint. Our example is only going to provide two: name and title.

The name field is required and must be unique across all content within your account. It is a descriptive, URL-friendly identifier.

The title field is where you define a user-friendly identifier. The title field is optional; when set, title is shown in the RStudio Connect dashboard instead of name.

export DATA='{"name": "shakespeare", "title": "Shakespeare Word Clouds"}'
curl --silent --show-error -L --max-redirs 0 --fail -X POST \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    --data "${DATA}" \
    "${CONNECT_SERVER}__api__/v1/experimental/content"
# => {
# =>   "guid": "ccbd1a41-90a0-4b7b-89c7-16dd9ad47eb5",
# =>   "name": "shakespeare",
# =>   "title": "Shakespeare Word Clouds",
# =>   ...
# =>   "owner_guid": "0b609163-aad5-4bfd-a723-444e446344e3",
# =>   "url": "http://localhost:3939/content/271/",
# => }

The JSON response is the full set of content fields. Those you did not supply when creating the content will receive default values. The Connect Server API Reference describes all the request and response fields for the POST /experimental/content content creation endpoint. Fields marked read-only should not be specified when creating content. If you happen to include read-only properties, they will be ignored. Read-only fields are computed internally by RStudio Connect as other operations occur.

Let’s define a CONTENT_GUID environment variable containing the guid of the content we just created. We will use this variable in the remaining deployment examples.

export CONTENT_GUID="ccbd1a41-90a0-4b7b-89c7-16dd9ad47eb5"

14.2.7.3 Creating a Bundle

The RStudio Connect content “bundle” represents a point-in-time representation of your code. You can associate a number of bundles with your content, though only one bundle is active. The active bundle is used to render your report, run your application, and supplies what you see when you visit that content in your browser.

Create bundles to match your workflow:

As you improve / enhance your content
Corresponding to a Git commit or merge
Upon completion of tests run in your continuous integration environment
After review and approval by your stakeholders

The bundle is uploaded to RStudio Connect as a .tar.gz archive. You will use the tar utility to create this file. Before we create the archive, let’s consider what should go inside.

All source files used by your content. This is usually a collection of .R, .Rmd, .py and .ipynb files. Include any required HTML, CSS, and Javascript resources, as well.
Data files, images, or other resources that are loaded when executing or viewing your content. This might be .png, .jpg, .gif, .csv files. If your report uses an Excel spreadsheet as input, include it!
A manifest.json. This JSON file describes the requirements of your content. For R content, this includes a full snapshot of all of your package requirements. The manifest.json is created with the rsconnect::writeManifest function.

From the command-line:
```
# This directory should be your current working directory.
Rscript -e 'rsconnect::writeManifest()'
```
From an R console:
```
# This directory should be your current working directory.
rsconnect::writeManifest()
```
We recommend committing the manifest.json into your source control system and regenerating it whenever you push new versions of your code – especially when updating packages or otherwise changing its dependencies!

Create your bundle .tar.gz file once you have collected the set of files to include. Here is an example that archives a simple Shiny application; the app.R contains the R source and data is a directory with data files loaded by the application.

tar czf bundle.tar.gz manifest.json app.R data

You MUST bundle the manifest.json and primary content files at the top-level; do NOT include the containing directory in the archive.

14.2.7.4 Uploading Bundles

The CONTENT_GUID environment variable is the content that will own the bundle that is uploaded. Bundles are associated with exactly one piece of content.

We use the POST /experimental/content/{guid}/upload upload content bundle endpoint with the bundle.tar.gz file as its payload:

curl --silent --show-error -L --max-redirs 0 --fail -X POST \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    --data-binary @"bundle.tar.gz" \
    "${CONNECT_SERVER}__api__/v1/experimental/content/${CONTENT_GUID}/upload"
# => {"bundle_id":"485","bundle_size":162987}

The response from the upload endpoint contains an identifier for the created bundle and the number of bytes received.

You MUST use the --data-binary argument to curl, which sends the data file without additional processing. Do NOT use the --data argument: it submits data in the same way as a browser when you “submit” a form and is not appropriate.

Extract the bundle ID from the upload response and assign it to a BUNDLE_ID environment variable:

export BUNDLE_ID="485"

14.2.7.5 Deploying a Bundle

This recipe explains how to deploy, or activate, an uploaded bundle. It assumes that CONTENT_GUID references the target content item and BUNDLE_ID indicates the bundle to deploy.

Bundle deployment triggers an asynchronous task that makes the uploaded data available for serving. The workflow applied to the bundled files varies depending on the type of content.

This uses the POST /experimental/content/{guid}/deploy deploy content bundle endpoint.

# Build the JSON input naming the bundle to deploy.
export DATA='{"bundle_id":"'"${BUNDLE_ID}"'"}'
# Trigger a deployment.
curl --silent --show-error -L --max-redirs 0 --fail -X POST \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    --data "${DATA}" \
    "${CONNECT_SERVER}__api__/v1/experimental/content/${CONTENT_GUID}/deploy"
# => {"task_id":"BkkakQAXicqIGxC1"}

The result from a deployment request includes a task identifier that we use to poll about the progress of that deployment task.

export TASK="BkkakQAXicqIGxC1"

14.2.7.6 Task Polling

The recipe explains how to poll for updates to a task. It assumes that the task identifier is present in the TASK environment variable.

The GET /experimental/tasks/{id} get task endpoint let you obtain the latest information about a dispatched operation.

There are two ways to poll for task information; you can request complete or incremental task output. The first URL query argument controls how much data is returned.

Here is a typical initial task progress request. It does not specify the first URL query argument, meaning all available output is returned. When first is not given, the value first=0 is assumed.

curl --silent --show-error -L --max-redirs 0 --fail \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    "${CONNECT_SERVER}__api__/v1/experimental/tasks/${TASK}?wait=1"
# => {
# =>   "id": "BkkakQAXicqIGxC1",
# =>   "output": [
# =>     "Building Shiny application...",
# =>     "Bundle requested R version 3.5.1; using ...",
# =>   ],
# =>   "finished": false,
# =>   "code": 0,
# =>   "error": "",
# =>   "last": 2
# => }

The wait=1 argument tells the server to collect output for up to one second. This long-polling approach is an alternative to explicitly sleeping within your polling loop.

The last field in the response lets us incrementally fetch task output. Our initial request returned two output lines; we want our next request to continue from that point. Here is a request for task progress that does not include the first two lines of output.

export FIRST=2
curl --silent --show-error -L --max-redirs 0 --fail \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    "${CONNECT_SERVER}__api__/v1/experimental/tasks/${TASK}?wait=1&first=${FIRST}"
# => {
# =>   "id": "BkkakQAXicqIGxC1",
# =>   "output": [
# =>    "Removing prior manifest.json to packrat transformation.",
# =>    "Performing manifest.json to packrat transformation.",
# =>   ],
# =>   "finished": false,
# =>   "code": 0,
# =>   "error": "",
# =>   "last": 4
# => }

Continue incrementally fetching task progress until the response is marked as finished. The final lines of output are included in this response.

export FIRST=86
curl --silent --show-error -L --max-redirs 0 --fail \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    "${CONNECT_SERVER}__api__/v1/experimental/tasks/${TASK}?wait=1&first=${FIRST}"
# => {
# =>   "id": "BkkakQAXicqIGxC1",
# =>   "output": [
# =>     "Completed packrat build against R version: '3.4.4'",
# =>     "Launching Shiny application..."
# =>   ],
# =>   "finished": true,
# =>   "code": 0,
# =>   "error": "",
# =>   "last": 88
# => }

Errors are indicated in the response by a non-zero code and an error message. It is likely that the output stream also includes information that will help you understand the cause of the error. Problems installing R packages, for example, will appear in the lines of output.

14.2.8 Managing Content

This section explains how to use the Connect Server APIs to obtain details about your existing content and perform updates. See Deploying Content (Section 14.2.7) for help creating content and deploying your code.

The content APIs are experimental and will continue to evolve in upcoming RStudio Connect releases. Please try using these APIs to build your own deployment tools. Let your Customer Success representative know about your experience!

Not all operations are supported by the Connect Server APIs. Please use the RStudio Connect dashboard for changes not yet available in the API.

14.2.8.1 Read a Content Item

One of your most common operations will be to read the current state for a content item. The GET /experimental/content/{guid} get content endpoint can tell you its name, the runtime fields, and lots of other information.

curl --silent --show-error -L --max-redirs 0 --fail \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    "${CONNECT_SERVER}__api__/v1/experimental/content/ccbd1a41-90a0-4b7b-89c7-16dd9ad47eb5"
# => {
# =>   "guid": "ccbd1a41-90a0-4b7b-89c7-16dd9ad47eb5",
# =>   "name": "shakespeare",
# =>   "title": "Shakespeare Word Clouds",
# =>   ...
# =>   "init_timeout": null,
# =>   "min_processes": null,
# =>   ...
# => }

The Connect Server API Reference describes all the response fields for the GET /experimental/content/{guid} get content endpoint.

14.2.8.2 Update a Single Content Item Field

The first thing we want to change is the content title; our team has decided to rename this content from “Shakespeare Word Clouds” to “Word Clouds from Shakespeare”. It’s a small change that we can make with a call to the POST /experimental/content/{guid} update content endpoint:

export DATA='{"title": "Word Clouds from Shakespeare"}'
curl --silent --show-error -L --max-redirs 0 --fail -X POST \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    --data "${DATA}" \
    "${CONNECT_SERVER}__api__/v1/experimental/content/ccbd1a41-90a0-4b7b-89c7-16dd9ad47eb5"
# => {
# =>   "guid": "ccbd1a41-90a0-4b7b-89c7-16dd9ad47eb5",
# =>   "name": "shakespeare",
# =>   "title": "Word Clouds from Shakespeare",
# =>   ...
# =>   "init_timeout": null,
# =>   "min_processes": null,
# => }

The JSON object returned by the update is the result of our changes and has the same shape as the read call.

In this example, we are including only title - the single field we want to change. You can make an update call using any portion of the content object as the POST payload. Fields in the JSON payload receive updates and other fields remain unchanged. The Connect Server API Reference describes all the request and response fields for the POST /experimental/content/{guid} update content endpoint.

Fields marked “read-only” are ignored by the update operation. Read-only fields are computed internally by RStudio Connect as other operations occur.

14.2.8.3 Update Multiple Content Item Fields

We can update other fields with the same approach we used when updating title in Section 14.2.8.2. The title update changed one field; our next example updates two fields that we want to manage together.

Let’s assume that our application has a fairly expensive startup. The developer is working to shorten its initialization, but that effort is not complete.

We want to allow additional time for that initialization and leave an instance of that process running at all times. We will adjust the init_timeout and min_processes settings.

export DATA='{"init_timeout": 300, "min_processes": 2}'
curl --silent --show-error -L --max-redirs 0 --fail -X POST \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    --data "${DATA}" \
    "${CONNECT_SERVER}__api__/v1/experimental/content/ccbd1a41-90a0-4b7b-89c7-16dd9ad47eb5"
# => {
# =>   "guid": "ccbd1a41-90a0-4b7b-89c7-16dd9ad47eb5",
# =>   "name": "shakespeare",
# =>   "title": "Word Clouds from Shakespeare",
# =>   ...
# =>   "init_timeout": 300,
# =>   "min_processes": 2,
# => }

That application is now configured to keep two processes running on each server and allows up to five minutes for successful startup.

We can undo the changes to init_timeout and min_processes after performance improvements to the Shiny application have been deployed. The init_timeout and min_processes had a null value before we applied our first set of changes.

The content item fields related to process execution can take a null value, which indicates to use the associated system default. A non-null value overrides the default for that specific property. The Content definition in the Connect Server API Reference describes which fields have server configuration defaults.

Here is an update request that gives a null value to both init_timeout and min_processes.

export DATA='{"init_timeout": null, "min_processes": null}'
curl --silent --show-error -L --max-redirs 0 --fail -X POST \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    --data "${DATA}" \
    "${CONNECT_SERVER}__api__/v1/experimental/content/ccbd1a41-90a0-4b7b-89c7-16dd9ad47eb5"
# => {
# =>   "guid": "ccbd1a41-90a0-4b7b-89c7-16dd9ad47eb5",
# =>   "name": "shakespeare",
# =>   "title": "Word Clouds from Shakespeare",
# =>   ...
# =>   "init_timeout": null,
# =>   "min_processes": null,
# => }

14.2.9 Content Promotion

This section explains how to use the Connect Server APIs to download code from content in one location and deploy to another. This workflow can help take application changes from a staging environment and deploy into production.

The bundle and content APIs are experimental and will continue to evolve in upcoming RStudio Connect releases. Please try using these APIs to build your own deployment tools. Let your Customer Success representative know about your experience!

The Connect Server API Reference contains documentation for each of the endpoints used in these recipes.

The content promotion recipe uses bash snippets and relies on curl to perform HTTP requests.

14.2.9.1 Scenario

Here is one scenario that can take advantage of this content promotion recipe; your organization may have similar separation of permissions and environments.

Your data scientists write Shiny applications and R Markdown reports. They rapidly iterate, experiment, and share updates. Content updates are deployed to an RStudio Connect staging environment. This environment lets the team share in-progress work without altering your business critical production content.

The RStudio Connect production environment hosts content that is visible to your customers and stakeholders. Deployment into production is done by a deployment engineer - not the data scientist.

The data science team develops updates to a Shiny application, which are then peer reviewed, tested, and approved for production use. The application developers do not have permissions to update content in the production environment. They hand-off to the deployment engineer, who has production privileges. The deployment engineer downloads the exact bundle archive from the staging environment and deploys into production.

14.2.9.2 Before Starting

You will need the following pieces of information about your staging environment before attempting to download a bundle archive:

STAGING_SERVER - The base URL for your RStudio Connect staging environment, such as https://connect-staging.company.com/.
STAGING_CONTENT_GUID - The source content GUID within your staging environment.
STAGING_API_KEY - An RStudio Connect API Key within your staging environment. The user associated with this API Key must be a collaborator for your source staging content.

You will need the following pieces of information about your production environment before attempting to deploy a bundle archive:

PROD_SERVER - The base URL for your RStudio Connect production environment, such as https://connect.company.com/.
PROD_CONTENT_GUID - The target content GUID within your production environment. This workflow assumes the target content already exists. Use the Creating Content recipe in Section 14.2.7.2 to create a new content item.
PROD_API_KEY - An RStudio Connect API Key within your production environment. The user associated with this API Key must be a collaborator (or owner) of your target production content.

The STAGING_SERVER and PROD_SERVER values appear elsewhere as CONNECT_SERVER. The STAGING_API_KEY and PROD_API_KEY are CONNECT_API_KEY elsewhere. The STAGING_CONTENT_GUID and PROD_CONTENT_GUID values are CONTENT_GUID in other recipes.

These recipes use the CONNECT_SERVER and CONNECT_API_KEY environment variables introduced in the Getting Started section of this cookbook.

Do not accidentally mix-and-match your staging/production configuration. API Keys for staging will not be recognized in your production environment.

14.2.9.3 Workflow

The content promotion workflow includes three steps:

Download the archive file for a source bundle from the staging environment (Section 14.2.9.4).
Upload the archive file into the production environment (Section 14.2.9.5).
Deploy the new production bundle (Section 14.2.9.6).

14.2.9.4 Bundle Download (staging)

The GET /experimental/content/{guid} endpoint returns information about a single content item and indicates the active bundle with its bundle_id field.

curl --silent --show-error -L --max-redirs 0 --fail \
    -H "Authorization: Key ${STAGING_API_KEY}" \
    "${STAGING_SERVER}__api__/v1/experimental/content/${STAGING_CONTENT_GUID}"
# => {
# =>   "guid": "b99b9b77-a8ae-4ecd-93aa-3c23baf9cefe",
# =>   "title": "staging content",
# =>   ...
# =>   "bundle_id": "584",
# =>   ...
# => }

Extract the bundle ID from the JSON response and assign it to a STAGING_BUNDLE_ID environment variable:

export STAGING_BUNDLE_ID="584"

We will use this bundle ID to download its archive file using the GET /experimental/bundles/{id}/download bundle download endpoint.

curl --silent --show-error -L --max-redirs 0 --fail -J -O \
    -H "Authorization: Key ${STAGING_API_KEY}" \
    "${STAGING_SERVER}__api__/v1/experimental/bundles/${STAGING_BUNDLE_ID}/download"

RStudio Connect suggests the filename bundle-${STAGING_BUNDLE_ID}.tar.gz in the Content-Disposition HTTP response header. The -J -O options tell curl to save the downloaded archive file using that filename.

Let’s define an environment variable named STAGING_BUNDLE_FILE containing name of the downloaded archive file.

# bundle-584.tar.gz
export STAGING_BUNDLE_FILE="bundle-${STAGING_BUNDLE_ID}.tar.gz"

14.2.9.5 Bundle Upload (production)

Given our staging bundle archive identified by the environment variable STAGING_BUNDLE_FILE, we can follow the Uploading Bundles recipe found in Section 14.2.7.4.

This is the one command where we are mixing “staging” and “production” variables. The archive file we obtained from the staging environment is being uploaded to production using the POST /experimental/content/{guid}/upload upload content bundle endpoint.

curl --silent --show-error -L --max-redirs 0 --fail -X POST \
    -H "Authorization: Key ${PROD_API_KEY}" \
    --data-binary @"${STAGING_BUNDLE_FILE}" \
    "${PROD_SERVER}__api__/v1/experimental/content/${PROD_CONTENT_GUID}/upload"
# => {"bundle_id":"242","bundle_size":162991}

Extract the bundle ID from the upload response and assign it to a PROD_BUNDLE_ID environment variable:

export PROD_BUNDLE_ID="242"

14.2.9.6 Bundle Deploy (production)

Given our target production bundle identified by the environment variable PROD_BUNDLE_ID, we can follow the Deploying a Bundle recipe found in Section 14.2.7.5. This uses the POST /experimental/content/{guid}/deploy deploy content bundle endpoint.

# Build the JSON input naming the bundle to deploy.
export DATA='{"bundle_id":"'"${PROD_BUNDLE_ID}"'"}'
# Trigger a deployment.
curl --silent --show-error -L --max-redirs 0 --fail -X POST \
    -H "Authorization: Key ${PROD_API_KEY}" \
    --data "${DATA}" \
    "${PROD_SERVER}__api__/v1/experimental/content/${PROD_CONTENT_GUID}/deploy"
# => {"task_id":"t0yiLB6bd6RKlesX"}

You can monitor the progress of this deployment by polling against the GET /experimental/tasks/{id} get task endpoint, as explained in Section 14.2.7.6. Remember to poll against the PROD_SERVER URL and not your staging environment.