Deploy RShiny on Kubernetes with a Helm Chart

If you want a very robust solution to deploy your RShiny applications a managed Kubernetes service on AWS, or EKS, is a great choice! Kubernetes has, for now, fully dominated the DevOps and deployment space in terms of deploying containerized services.

If you'd like to learn more about deploying RShiny please consider checking out my FREE Deploy RShiny on AWS Guide!

Should I Use Kubernetes to Deploy my RShiny App?

As with all things, there are pros and cons to using Kubernetes.

Cons

• Kubernetes is a beast to get started with.
• You need to have your RShiny App packaged with docker.

Pros

• Kubernetes is incredibly well supported across a range of providers, including AWS, GCP, Azure, Digital Ocean, etc.
• Kubernetes is complicated, but with complicated comes armies of engineers who want to abstract away the complicated into configuration files. So you if you stick with the growing pains it's likely you will get a lot of benefit and things will get less painful over time.
• Kubernetes has been embraced by the "people who deploy stuff" community, which means that there are a lot of resources out for it.
• If you need to scale Kubernetes is an excellent way to do that.
• Kubernetes can do a lots of neat features, including a JobsApi and a CronJobsApi. This means you can do things like download datasets everyday, which can be very handy for RShiny applications that use datasets.

AWS Specific Pros

A lot of these are probably applicable to other platforms, but I work on AWS and so here we are.

AWS has a managed solutions

• For Code (think GitHub) - CodeCommit
• For Private Docker containers (think dockerhub) - ECR
• For CI/CD (CircleCi, Travis, etc) - CodeDeploy

You can also mix and match as you like. For instance, you can host your private images on DockerHub, and still deploy to AWS Kubernetes (EKS), or you can have you CI/CD pipeline setup with Circle and still use AWS.

Next up is the fact that AWS infrastructure can be completely hands off and automated. You don't have to, and probably you really shouldn't, go through the console to create complex infrastructure like clusters. Instead you can use a tool like Terraform or Cloudformation. These tools can automate everything from your Kubernetes cluster creation to deploying your app.

I'm sold! What now?

Well, first of all you'll need an RShiny image to test with. I really recommend picking something VERY simple to test with so you can see if the issue is your image or your setup. For this tutorial we'll be using the Rocker Shiny image.

You'll also need to have a Kubernetes cluster available. Deploying one is beyond the scope of this tutorial, but I do have a template available.

Create your helm chart

I'm going to go through step by step what modifications you need to make. If you'd like to skip this part you can grab the helm chart HERE.

The first thing you'll want to (after installing helm v3) do is to create a helm chart. A bit about helm charts!

Helm uses a packaging format called charts. A chart is a collection of files that describe a related set of Kubernetes resources. A single chart might be used to deploy something simple, like a memcached pod, or something complex, like a full web app stack with HTTP servers, databases, caches, and so on. Charts are created as files laid out in a particular directory tree, then they can be packaged into versioned archives to be deployed.

From helm.sh

From here we will learn by doing, so bring up your terminal and run:

helm create rshiny-eks

You'll see a directory called rshiny-eks. As a quick side note many editors and IDE can recognize helm charts if you install a plugin. I have found it to be moderately helpful.

Let's open up the rshiny-eks/values.yaml and take a look!

Specify an image

You specify an image by changing the image.repository key. The chart is created by default with nginx. If you are on an entirely new cluster I definitely recommend deploying the chart as is. I always like to deploy NGINX on any new platform, because if I can't deploy NGINX I have bigger and scarier problems. ;-)

# rshiny/values.yaml
image:
  repository: nginx
  pullPolicy: IfNotPresent

Let's change this rocker/shiny.

# rshiny/values.yaml
image:
  repository: rocker/shiny 
  pullPolicy: IfNotPresent

Caution - USE TAGS

The tag will default to the appVersion in the Chart.yaml. No one actually seems to follow this and adds in a tag to the values.yaml.

# rshiny/values.yaml
image:
  repository: rocker/shiny 
  # add in a TAG key
  # don't use latest!
  tag: 3.6.1
  pullPolicy: IfNotPresent

Then open up on templates/deployment.yaml and make an update to the image key.

Change this -

# templates/deployment.yaml
image: "{{ .Values.image.repository }}:{{ .Chart.AppVersion }}"

To:

# templates/deployment.yaml
image: "{{ .Values.image.repository }}:{{ .Values.image.tag }}"

SideNote

Normally, what helm does is to look at the Chart.yaml for the appVersion, which then corresponds to the tag. This is problematic with automatic deployments because there is no way to change this from the command line. I find it easier to just modify the helm chart.

Add in the SessionAffinity

RShiny likes having SessionAffinity. This just makes your sessions sticky, which means that users aren't getting shuffled around to different pods mid experience.

Open up your rshiny-eks/templates/deployment.yaml and add in a sessionAffinity key under the spec.

# rshiny-eks/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: {{ include "rshiny-eks.fullname" . }}
  labels:
    {{- include "rshiny-eks.labels" . | nindent 4 }}
spec:
  replicas: {{ .Values.replicaCount }}
  # ADD sessionAffinity HERE 
  sessionAffinity: ClientIP

A note on ports

This setup assumes that your RShiny app is running on port 80 in the container. If it isn't you'll need to change the containerPort value in rshiny-eks/templates/deployment.yaml.

Optional - Dependencies to nicely expose a URL

If you're just deploying as a test, you don't need these because AWS will give you a public IP address, but if you need to map your Kubernetes instance to an actual URL you'll need to modify some things.

# Chart.yaml
dependencies:
  - name: nginx-ingress
    version: 1.27.0
    repository: https://kubernetes-charts.storage.googleapis.com
  - name: cert-manager
    version: v0.12.0
    repository: https://charts.jetstack.io

In the values.yaml delete the ingress block and replace it with this.

# values.yaml
ingress:
  enabled: true
  annotations:
    kubernetes.io/ingress.class: nginx
#     if you need to have https uncomment
#    cert-manager.io/cluster-issuer: letsencrypt-production
  hosts:
# your real domain
# You will need to get the service url and add it to your DNS panel as an A name record.
# this may take some time to propogate the first time around due to DNS settings.
# Don't freak out!
    - host: www.mydomain.com
      paths:
        - "/"
# Get the service url
# with kubectl get svc -o wide | grep MY_RELEASE_NAME
# Add this to your DNS panel as an A name to get the above to work
    - host: lots-o-numbas.aws-region.elb.amazonaws.com
      paths:
        - "/"
  tls: []
  # If you're using https you'll need to update this
  #  - secretName: chart-example-tls
  #    hosts:
  #      - chart-example.local

Deploy your helm chart

Install any dependencies to your helmchart

You'll need to be in the parent directory of rshiny-eks.

helm dep up rshiny-eks 

This will install any dependencies. If you didn't do the optional step you won't have any, but keep it in mind for the next time around!

Deploy the Helm Chart

Now, one really handy feature of helm is that you can modify the values.yaml on the commandline by using the --set flag. What we're going to do here is to set the image tag with our actual tag. (I'll just keep on keeping on with being a broken record. Use a tag! You will regret it if you don't.)

# helm upgrade --install RELEASE_NAME ./path-on-filesystem
# If you didn't install the optional dependencies use the
# --set service.type=LoadBalancer
helm upgrade --install rshiny-eks ./rshiny-eks  \
    --set image.tag=$SOME_REAL_TAG_HERE  \
    --set service.type=LoadBalancer

# If you installed the optional dependencies you don't need to specify the service.type 
helm upgrade --install rshiny-eks ./rshiny-eks  \
    --set image.tag=$SOME_REAL_TAG_HERE 

Get the service address

Once you've done that you can get the IP address of the service to put in your browser and see the SHINY!

kubectl get svc -o wide | grep rshiny-eks | grep LoadBalancer

This will get you something that looks like this -

rshiny-eks-nginx-ingress-controller        LoadBalancer   172.20.108.106   lots-o-string.aws-region.elb.amazonaws.com   80:32168/TCP   1d   app=nginx-ingress,component=controller,release=rshiny-eks

Open up your browser and type in lots-o-string.aws-region.elb.amazonaws.com! It may take a few minutes to bring up the service.

Troubleshooting

Get a shell

My favorite thing about DevOps is that we have all of these SUPER FANCY ways of deploying absolutely everything. Then we have backdoors that are essentially "ok shut up and give me a shell". Kubernetes is no exception here. We can get a shell, copy files, forward ports. HA!

#If you used a different release name grep for that
kubectl get pods |grep rshiny

Get the name of the pod and once you have that you can get a shell directly on the container.

kubectl exec -it name-of-pod bash

View Logs

You can also view the logs, which is everything that's thrown to stdout/stderr.

kubectl logs name-of-pod

Describe the Pod

If you're container is failing and you're trying to figure out why try describing the pod.

kubectl describe pod name-of-pod

WrapUp

So that's it! Once you have your Kubernetes cluster up and running you can deploy your RShiny app using helm and distribute the awesomeness!

If you'd like to learn more about how I deploy RShiny Apps on AWS I have a Free Guide available that includes deploying with Docker, Lightsail, EC2, and Kubernetes when you sign up for my DevOps for Data Scientists Weekly Tutorial List!

Get in Touch

If you have any questions, comments, or would like to request a tutorial please get in touch with me at [email protected] .

Looking for More?

If you're looking for more in-depth guides, ebooks, and project templates please consider checking out the Dabble of DevOps store and the RShiny Project Template.