14 Connect Server API Cookbook

This section contains recipes for writing scripts that use the Connect Server API. It is assumed that you know the R Programming Language, but the recipes are intended to be straightforward enough to be implemented in any programming language you wish to use.

14.1 Configuring Your Scripts

It may be useful to store your RStudio Connect server URL and your API key in an environment variable, so that you can share any tools you create without exposing your personal information to other users. For example:

# ~/.Renviron
# Note that the connectServer string must have a trailing slash
CONNECT_API_KEY="mysupersecretapikey"
CONNECT_SERVER="https://connect.example.com/"

The .Renviron file will be loaded in every R session spawned under your user account.

14.2 Recipes

14.2.1 Detect Whether RStudio Connect Has Your Local R Version

  1. Obtain the server path and API keys from environment variables
  2. Obtain your local R version using R.version
  3. Call the “Get R Installation Info” endpoint. See the API Documentation for more information.
  4. Parse the response using httr::content.
  5. Check the response for the local R version. If it is not listed, the RStudio Connect server does not contain the local R version.
# set up environment
# Note that the connectServer string must have a trailing slash
connectServer <- Sys.getenv("CONNECT_SERVER")
apiKey <- Sys.getenv("CONNECT_API_KEY")

library(httr)
myRVersion <- paste(R.version$major, R.version$minor, sep = ".")
resp <- GET(
    paste0(connectServer, "__api__/v1/server_settings/r"),
    add_headers(Authorization = paste("Key", apiKey))
)
resp <- content(resp, as="parsed")
if (!(myRVersion %in% unlist(resp))) {
    print(paste("Cannot find R version", myRVersion,"on the RStudio Connect server"))
} else {
    print("The local R version was found on the RStudio Connect server")
}

14.2.2 Use Keyset (Cursor) Pagination

The following snippet pages through the audit logs, which uses keyset pagination, starting from the most recent entries, 25 entries at a time.

  1. Obtain the server path and API keys from environment variables.
  2. Call the “Get audit logs” endpoint to get the first page. See the API Documentation for more information.
  3. Parse the response using httr::content.
  4. Print the current page.
  5. Repeat steps 1 through 3 until there are no more results. Note also that the paging.next property in the response is the URL of the next page.
# set up environment
library(httr)
# Note that the connectServer string must have a trailing slash
connectServer <- Sys.getenv("CONNECT_SERVER")
apiKey <- Sys.getenv("CONNECT_API_KEY")

# get audit logs
authHeader <- add_headers(Authorization = paste("Key", apiKey))
resp <- GET(
  paste0(connectServer, "__api__/v1/audit_logs?ascOrder=false&limit=25"),
  authHeader
)
payload <- content(resp)
# print first 25!
print(payload$results)
# now step through the remaining audit logs
while(!is.null(payload$paging[["next"]])) {
  resp <- GET(payload$paging[["next"]], authHeader)
  payload <- content(resp)
  # print the next 25
  print(payload$results)
}

14.2.3 Use Offset Pagination

The following snippet pages through the user’s list, which uses offset pagination, 25 entries at a time.

  1. Obtain the server path and API keys from environment variables.
  2. Call the “Get all users” endpoint to get the first page. See the API Documentation for more information.
  3. Parse the response using httr::content.
  4. Print the current page.
  5. Repeat steps 1 through 3 until there is no more “next” page. Note also that the query parameter page_number determines the page to return.
# set up environment
library(httr)
# Note that the connectServer string must have a trailing slash
connectServer <- Sys.getenv("CONNECT_SERVER")
apiKey <- Sys.getenv("CONNECT_API_KEY")

# get user's list
authHeader <- add_headers(Authorization = paste("Key", apiKey))
apiPrefix <- "__api__/v1/users?page_size=25"
resp <- GET(
  paste0(connectServer, apiPrefix),
  authHeader
)
# get the first page
payload <- content(resp)
# and step through the pages, printing out the results (if any)
while(length(payload$result) > 0) {
  # print the result
  print(payload$results)
  # get the next page
  nextPage <- payload$current_page + 1
  resp <- GET(
    paste0(connectServer, apiPrefix, "&page_number=", nextPage),
    authHeader
  )
  payload <- content(resp)
}

14.2.4 Create an RStudio Connect User from LDAP or OAuth2

The following snippets search for a user in LDAP or OAuth2 and then create an RStudio Connect account for that user.

There are two steps:

  1. Search for the user via the GET /users/remote endpoint. A user with no account on Connect will lack a guid. Note the temp_ticket for the desired user account.
  2. Use the PUT /users endpoint with the temp_ticket to create a corresponding account on RStudio Connect.

14.2.4.1 Search for a User

# set up environment
library(httr)
# Note that the connectServer string must have a trailing slash
connectServer <- Sys.getenv("CONNECT_SERVER")
apiKey <- Sys.getenv("CONNECT_API_KEY")

# set the search parameter
prefix <- "julie"

# make the query request
authHeader <- paste("Key", apiKey)
response <- GET(
    paste0(connectServer, "__api__/v1/users/remote"),
    add_headers(Authorization = authHeader),
    query = list(prefix = prefix)
)

results <- content(response)$results

# print the results of the API call
formatGuid <- function(guid) {
    if (is.null(guid))
        "NULL"
    else
        guid
}
cat(sprintf("FIRST\tLAST\tUSERNAME\tGUID\n"))
for (user in results) {
    cat(
        sprintf(
            "%s\t%s\t%s\t\t%s\n",
            user$first_name,
            user$last_name,
            user$username,
            formatGuid(user$guid)
        )
    )
}

The output looks like the following:

FIRST   LAST    USERNAME    GUID
Julie   Goolly  julie1      15f5f51d-08ff-4e5b-beba-4ccf24e248dd
Julie   Jolly   julie2      NULL

In this particular case, there are two users matching the search for the prefix julie. The user julie1 has a GUID value, which means that this user already has an account in RStudio Connect. The user julie2 does not yet have an account in RStudio Connect.

Included in the API response for each user is a temp_ticket value that can be used to give the user an account in RStudio Connect. In the example above, the second user, julie2, needs an account, so you will need that user’s temp_ticket:

tempTicket <- results[[2]]$temp_ticket

You can use this tempTicket value in the next section to create the account.

14.2.4.2 Create an RStudio Connect User Account

Using the tempTicket value from the previous section, you can give the user an RStudio Connect account with an HTTP PUT request:

# use a tempTicket value from searching /users/remote

response <- PUT(
    paste0(connectServer, "__api__/v1/users"),
    add_headers(Authorization = authHeader),
    body = list(temp_ticket = tempTicket),
    encode = "json"
)

print(content(response))

When the call succeeds, the response will contain a non-NULL guid value, which is a unique identifier for the user account.

If the user already exists in Connect, the response will contain an error:

$error
[1] "The requested username is already in use."

14.2.5 Create an RStudio Connect Group from LDAP

The following snippets search for a group in LDAP and then create an RStudio Connect group for that LDAP group.

There are two steps:

  1. Search for the group via the GET /groups/remote endpoint. A group that does not exist on Connect will lack a guid. Note the temp_ticket for the desired group.
  2. Use the PUT /groups endpoint with the temp_ticket to create a corresponding group on RStudio Connect.

14.2.5.1 Search for a Group

# set up environment
library(httr)
# Note that the connectServer string must have a trailing slash
connectServer <- Sys.getenv("CONNECT_SERVER")
apiKey <- Sys.getenv("CONNECT_API_KEY")

# set the search parameter
prefix <- "accounting"

# make the query request
authHeader <- paste("Key", apiKey)
response <- GET(
    paste0(connectServer, "__api__/v1/groups/remote"),
    add_headers(Authorization = authHeader),
    query = list(prefix = prefix)
)

results <- content(response)$results

# print the results of the API call
formatGuid <- function(guid) {
    if (is.null(guid))
        "NULL"
    else
        guid
}
cat(sprintf("NAME\tGUID\n"))
for (group in results) {
    cat(
        sprintf(
            "%s\t%s\n",
            group$name,
            formatGuid(group$guid)
        )
    )
}

The output looks like the following:

NAME          GUID
accounting1   15f5f51d-08ff-4e5b-beba-4ccf24e248dd
accounting2   NULL

In this particular case, there are two groups matching the search for the prefix accounting. The group accounting1 has a GUID value, which means that this group already exists in RStudio Connect. The group accounting2 does not yet have a corresponding group in RStudio Connect.

Included in the API response for each group is a temp_ticket value that can be used to create the group in RStudio Connect. In the example above, the second group, accounting2, does not exist in RStudio Connect, so you will need the temp_ticket for this group:

tempTicket <- results[[2]]$temp_ticket

You can use this tempTicket value in the next section to create the group.

14.2.5.2 Create an RStudio Connect Group

Using the tempTicket value from the previous section, you can create an RStudio Connect group with an HTTP PUT request:

# use a tempTicket value from searching /groups/remote

response <- PUT(
    paste0(connectServer, "__api__/v1/groups"),
    add_headers(Authorization = authHeader),
    body = list(temp_ticket = tempTicket),
    encode = "json"
)

print(content(response))

When the call succeeds, the response will contain a non-NULL guid value, which is a unique identifier for the group.

If the group already exists in Connect, the response will contain an error:

$error
[1] "The requested group name is already in use."

14.2.6 User Activity

RStudio Connect server records user activity differently depending on what type of content is being accessed by the user. The recorded activity can be grouped into two categories, each having separate API endpoints.

The first category contains user activity for Shiny Applications.

The second category combines user activity for static and rendered content. Static content includes plots, spreadsheets and sites. Rendered content includes R Markdown documents (both with and without parameters) and Jupyter Notebooks.

The following sections provide recipes for using the APIs associated with these two usage categories.

14.2.6.1 User Activity: Shiny Applications

The following snippet pages through Shiny application usage. It returns information in pages and follows the keyset pagination model. The keyset pagination recipe explains how to perform multiple, paged requests.

  1. Obtain the server path and API keys from environment variables.
  2. Call the “Get shiny app usage” endpoint to get the first page. See the API Documentation for more information.
  3. Parse the response using httr::content.
  4. Print the current page.
  5. Repeat steps 1 through 3 until there are no more results. Note also that the paging.next property in the response is the URL of the next page.
# set up environment
library(httr)
# Note that the connectServer string must have a trailing slash
connectServer <- Sys.getenv("CONNECT_SERVER")
apiKey <- Sys.getenv("CONNECT_API_KEY")

# get usage information
authHeader <- add_headers(Authorization = paste("Key", apiKey))
usageURL <- paste0(connectServer, "__api__/v1/instrumentation/shiny/usage?limit=25")
resp <- GET(usageURL, authHeader)
payload <- content(resp)
# print first 25!
print(payload$results)
# now step through the remaining audit logs
while(!is.null(payload$paging[["next"]])) {
  resp <- GET(payload$paging[["next"]], authHeader)
  payload <- content(resp)
  # print the next 25
  print(payload$results)
}

14.2.6.2 User Activity: Rendered and Static Content

The following snippet pages through rendered/static content visits. It returns information in pages and follows the keyset pagination model.

The keyset pagination recipe explains how to perform multiple, paged requests.

  1. Obtain the server path and API keys from environment variables.
  2. Call the “Get rendered/static content visits” endpoint to get the first page. See the API Documentation for more information.
  3. Parse the response using httr::content.
  4. Print the current page.
  5. Repeat steps 1 through 3 until there are no more results. Note also that the paging.next property in the response is the URL of the next page.
# set up environment
library(httr)
# Note that the connectServer string must have a trailing slash
connectServer <- Sys.getenv("CONNECT_SERVER")
apiKey <- Sys.getenv("CONNECT_API_KEY")

# get visit information
authHeader <- add_headers(Authorization = paste("Key", apiKey))
hitsURL <- paste0(connectServer, "__api__/v1/instrumentation/content/visits?limit=25")
resp <- GET(hitsURL, authHeader)
payload <- content(resp)
# print first 25!
print(payload$results)
# now step through the remaining audit logs
while(!is.null(payload$paging[["next"]])) {
  resp <- GET(payload$paging[["next"]], authHeader)
  payload <- content(resp)
  # print the next 25
  print(payload$results)
}

14.2.7 Deploying Content

This section explains how to use the deployment APIs to create content in RStudio Connect and deploy code associated with that content. The deployment APIs can be used for any type of content supported by RStudio Connect, including Shiny applications, R Markdown notebooks, Plumber APIs, and Jupyter notebooks.

The deployment APIs are experimental and will continue to evolve in upcoming RStudio Connect releases. Please try using these APIs to build your own deployment tools. Let us (sol-eng@rstudio.com) know about your experience!

The rstudio/connect-api-deploy-shiny GitHub repository contains a sample Shiny application and uses the recipes in this section in deployment scripts that you can use as examples when building your own workflows.

The RStudio Connect API Reference contains documentation for each of the endpoints used in these recipes.

The deployment examples use bash snippets and rely on curl to perform HTTP requests. We re-use the CONNECT_SERVER and CONNECT_API_KEY environment variables introduced earlier in this cookbook.

These recipes do not prescribe a single workflow. Some example workflows include:

  • Automate the creation of a new, uniquely named Shiny application every quarter to analyze the sales leads. The latest quarter contains new dashboard functionality but you cannot alter prior quarters; those applications need to capture that point-in-time.

  • A Plumber API that receives updates after the R code supporting that API is fully tested by your continuous integration environment. These tests confirm that all updates remain backwards-compatible.

  • A team collaborates on an application over the course of a two week sprint cycle. The code is shared with Git and progress tracked in GitHub issues. The team performs production updates at the end of each sprint with a Docker-based deployment environment.

  • Your organization does not permit data scientists to publish directly to the production server. Production updates are scheduled events and gated by successful user-acceptance testing. A deployment engineer, who is not an R user, uses scripts to create and publish content in production by interacting with the RStudio Connect APIs.

14.2.7.1 Creating Content

The POST /experimental/content content creation API is used to create a new content item in RStudio Connect. It takes a JSON document as input.

The RStudio Connect Server API describes the full set of fields that may be supplied to this endpoint. Our example is only going to provide two: name and title.

The name field is required and must be unique across all content within your account. It is a descriptive, URL-friendly identifier.

The title field is where you define a user-friendly identifier. The title field is optional; when set, title is shown in the RStudio Connect dashboard instead of name.

export DATA='{"name": "shakespeare", "title": "Shakespeare Word Clouds"}'
curl --silent --show-error -L --max-redirs 0 --fail -X POST \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    --data "${DATA}" \
    "${CONNECT_SERVER}__api__/v1/experimental/content"
# => {
# =>   "guid": "ccbd1a41-90a0-4b7b-89c7-16dd9ad47eb5",
# =>   "name": "shakespeare",
# =>   "title": "Shakespeare Word Clouds",
# =>   ...
# =>   "owner_guid": "0b609163-aad5-4bfd-a723-444e446344e3",
# =>   "url": "http://localhost:3939/content/271/",
# => }

The JSON response is the full set of content fields. Those you did not supply when creating the content will receive default values. The read-only fields that are not permitted by creation are computed internally by RStudio Connect as other operations occur.

Let’s define a CONTENT_GUID environment variable containing the identifier of this content we just created. We will use this variable in the remaining deployment examples.

export CONTENT_GUID="ccbd1a41-90a0-4b7b-89c7-16dd9ad47eb5"

14.2.7.2 Creating a Bundle

The RStudio Connect content “bundle” represents a point-in-time representation of your code. You can associate a number of bundles with your content, though only one bundle is active. The active bundle is used to render your report, run your application, and supplies what you see when you visit that content in your browser.

Create bundles to match your workflow:

  • As you improve / enhance your content
  • Corresponding to a Git commit or merge
  • Upon completion of tests run in your continuous integration environment
  • After review and approval by your stakeholders

The bundle is uploaded to RStudio Connect as a .tar.gz archive. You will use the tar utility to create this file. Before we create the archive, let’s consider what should go inside.

  • All source files used by your content. This is usually a collection of .R, .Rmd, .py and .ipynb files. Include any required HTML, CSS, and Javascript resources, as well.
  • Data files, images, or other resources that are loaded when executing or viewing your content. This might be .png, .jpg, .gif, .csv files. If your report uses an Excel spreadsheet as input, include it!

  • A manifest.json. This JSON file describes the requirements of your content. For R content, this includes a full snapshot of all of your package requirements. The manifest.json is created with the rsconnect::writeManifest function.

    From the command-line:

    # This directory should be your current working directory.
Rscript -e 'rsconnect::writeManifest()'

    From an R console:

    # This directory should be your current working directory.
rsconnect::writeManifest()

    We recommend committing the manifest.json into your source control system and regenerating it whenever you push new versions of your code – especially when updating packages or otherwise changing its dependencies!

Create your bundle .tar.gz file once you have collected the set of files to include. Here is an example that archives a simple Shiny application; the app.R contains the R source and data is a directory with data files loaded by the application.

tar czf bundle.tar.gz manifest.json app.R data

You MUST bundle the manifest.json and primary content files at the top-level; do NOT include the containing directory in the archive.

14.2.7.3 Uploading Bundles

The CONTENT_GUID environment variable is the content that will own the bundle that is uploaded. Bundles are associated with exactly one piece of content.

We upload the bundle.tar.gz file by sending an HTTP POST request with that file as its payload:

curl --silent --show-error -L --max-redirs 0 --fail -X POST \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    --data-binary @"bundle.tar.gz" \
    "${CONNECT_SERVER}__api__/v1/experimental/content/${CONTENT_GUID}/upload"
# => {"bundle_id":"485","bundle_size":162987}

The response from the upload endpoint contains an identifier for the created bundle and the number of bytes received.

You MUST use the --data-binary argument to curl, which sends the data file without additional processing. Do NOT use the --data argument: it submits data in the same way as a browser when you “submit” a form and is not appropriate.

Extract the bundle ID from the upload response and assign it to a BUNDLE_ID environment variable:

export BUNDLE_ID="485"

14.2.7.4 Deploying a Bundle

Bundle deployment triggers an asynchronous task that makes the uploaded data available for serving. The workflow applied to the bundled files varies depending on the type of content.

# Build the JSON input naming the bundle to deploy.
export DATA='{"bundle_id":"'"${BUNDLE_ID}"'"}'
# Trigger a deployment.
curl --silent --show-error -L --max-redirs 0 --fail -X POST \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    --data "${DATA}" \
    "${CONNECT_SERVER}__api__/v1/experimental/content/${CONTENT_GUID}/deploy"
# => {"task_id":"BkkakQAXicqIGxC1"}

The result from a deployment request includes a task identifier that we use to poll about the progress of that deployment task.

export TASK="BkkakQAXicqIGxC1"

There are two ways to poll for task information; you can request complete or incremental task output. The first URL query argument controls how much data is returned.

Here is a typical initial task progress request. It does not specify the first URL query argument, meaning all available output is returned. When first is not given, the value first=0 is assumed.

curl --silent --show-error -L --max-redirs 0 --fail \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    "${CONNECT_SERVER}__api__/v1/experimental/tasks/${TASK}?wait=1"
# => {
# =>   "id": "BkkakQAXicqIGxC1",
# =>   "output": [
# =>     "Building Shiny application...",
# =>     "Bundle requested R version 3.5.1; using ...",
# =>   ],
# =>   "finished": false,
# =>   "code": 0,
# =>   "error": "",
# =>   "last": 2
# => }

The wait=1 argument tells the server to collect output for up to one second. This long-polling approach is an alternative to explicitly sleeping within your polling loop.

The last field in the response lets us incrementally fetch task output. Our initial request returned two output lines; we want our next request to continue from that point. Here is a request for task progress that does not include the first two lines of output.

export FIRST=2
curl --silent --show-error -L --max-redirs 0 --fail \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    "${CONNECT_SERVER}__api__/v1/experimental/tasks/${TASK}?wait=1&first=${FIRST}"
# => {
# =>   "id": "BkkakQAXicqIGxC1",
# =>   "output": [
# =>    "Removing prior manifest.json to packrat transformation.",
# =>    "Performing manifest.json to packrat transformation.",
# =>   ],
# =>   "finished": false,
# =>   "code": 0,
# =>   "error": "",
# =>   "last": 4
# => }

Continue incrementally fetching task progress until the response is marked as finished. The final lines of output are included in this response.

export FIRST=86
curl --silent --show-error -L --max-redirs 0 --fail \
    -H "Authorization: Key ${CONNECT_API_KEY}" \
    "${CONNECT_SERVER}__api__/v1/experimental/tasks/${TASK}?wait=1&first=${FIRST}"
# => {
# =>   "id": "BkkakQAXicqIGxC1",
# =>   "output": [
# =>     "Completed packrat build against R version: '3.4.4'",
# =>     "Launching Shiny application..."
# =>   ],
# =>   "finished": true,
# =>   "code": 0,
# =>   "error": "",
# =>   "last": 88
# => }

Errors are indicated in the response by a non-zero code and an error message. It is likely that the output stream also includes information that will help you understand the cause of the error. Problems installing R packages, for example, will appear in the lines of output.