Deploy RShiny on Kubernetes with a Helm Chart

helm kubernetes r rshiny Apr 01, 2020

If you want a very robust solution to deploy your RShiny applications a managed Kubernetes service on AWS, or EKS, is a great choice! Kubernetes has, for now, fully dominated the DevOps and deployment space in terms of deploying containerized services.

Should I Use Kubernetes to Deploy my RShiny App?

As with all things, there are pros and cons to using Kubernetes.

Cons

  • Kubernetes is a beast to get started with.
  • You need to have your RShiny App packaged with docker.

Pros

  • Kubernetes is incredibly well supported across a range of providers, including AWS, GCP, Azure, Digital Ocean, etc.
  • Kubernetes is complicated, but with complicated comes armies of engineers who want to abstract away the complicated into configuration files. So you if you stick with the growing pains it's likely you will get a lot of benefit and things will get less painful over time.
  • Kubernetes has been embraced by the "people who deploy stuff" community, which means that there are a lot of resources out for it.
  • ...
Continue Reading...

Deploy RShiny on AWS EC2

aws ec2 r rshiny Mar 22, 2020

If you're deploying applications on AWS one of the easier ways to get started is to simply deploy everything on EC2. If you're not familiar with AWS EC2, EC2 instances are compute instances. They are like your (linux) desktop, except that yous spin them up and kill them on demand with AWS console (or CLI or some other infrastructure automation tool).

Should I use EC2 to Deploy my RShiny App?

If you're looking for a solution that has some built in scale and ability to throw a Load Balancer at it EC2 is fine. It's also nice because it's just a computer. You don't have to find a work around to get you a computer the way you do with Docker (docker run) or Kubernetes (kube exec). You deal with an EC2 instance the way you would any remote server, but its on AWS so you get some additional niceness such as the backups, Elastic IPs, and Load Balancers.

This is slightly more difficult than using LightSail. If you are completely new to AWS and on a tight deadline you may want to start there....

Continue Reading...

Deploy your RShiny App Locally with Docker

docker r rshiny Dec 10, 2019

My favorite way to deploy RShiny locally is to simply package it into a docker image and run it. Running any application in docker makes it easily transportable, and is a generally acceptable way of distributing applications in 2019.

This solution does not require rshiny-server or shinyapps.io, not because I have anything against either of those solutions. I just tend to stick to a few favorite deployment methods to keep my head from spinning straight off my body. ;-)

If you're not familiar with docker then I have a FREE course available. The first module is plenty to get you up and running and can be completed in an hour or two. For the most part, if you can use a command line you can use docker.

Package your R Shiny App in Docker

Now that we've covered some housekeeping let's get started building your docker image. Like any project, you want to have all your relevant code in a directory. This way it is accessible to the docker build process.

Project Directory Structure 

...
Continue Reading...

Learn Apache Airflow By Example – 4 Part Series

apacheairflow docker python Dec 02, 2019

Introduction

I've been having a blast the last few months learning ApacheAirflow. It's become an indispensable tool in my getting stuff done toolbox.

Follow me on Twitter @jillianerowe, or send me a message with questions, topic suggestions, ice cream recommendations, or general shenanigans. 

 

Get the Source Code

...

Continue Reading...

Deploy your RShiny App on AWS Series - Lightsail

aws r rshiny Nov 24, 2019
 

If your head is spinning over which deployment scenario to choose for your RShiny app look no further! I have a whole series planned out for you on various deployment options and why you should choose each one.

Why should you use AWS Lightsail for RShiny Deployment?

Deployment scenarios are like snowflakes. No two are exactly alike! ;-) You need different level of power and control for different deployment scenarios. Here we are going to talk about RShiny deployments, but it applies to just about everything.

Lightsail is a relatively recent addition to the whole AWS deployment ecosystem.  It makes it much simpler and more streamlined to deploy a web application than some of their other, more powerful solutions.

Lightsail would be a good choice for deployment for you if:

  • You don't feel comfortable deploying web applications or configuring web servers (Apache, NGINX).
  • You are just fine with configuring webservers but you need something quick and easy.
  • You have a smallish...
Continue Reading...

Drive Traffic to Your Freelance Tech Business by Starting a Blog

career Nov 21, 2019

If you're looking to start a freelance business in tech, or any business really, then you'll need to find a way to have the clients knocking down your doors. To make this a very short story you mostly do this by putting yourself out there, being helpful, and talking to people.

More specifically, you can start a blog to demonstrate your awesome knowledge (get the word out there) with awesome content (helping people!). 

I've been a software engineer for nearly 10 years (how time flies!) and about 6 months ago I started to seriously consider the idea that I wanted to venture off on my own. Be my own boss, be mistress of my own destiny, all that kind of thing. My biggest worry was well how do I find my own clients?

One of the first pieces of advice I got was to start a blog, and it's worked out fairly well for me, so here we are! 

I'm not affiliated with any of these products or services. These are simply my opinions.

Buy a Domain Name

Well, first of all, think of a name...

Continue Reading...

How To Become a Freelance Software or DevOps Engineer

career freelance Nov 18, 2019

Your first steps

Getting started freelancing is one of those things you can google for the rest of your life and it will just leave your head spinning. I know this because I've been working towards being my very own Jillian Inc for the last 6 months or so. Here are a few straightforward tasks you can set for yourself in order to get started.

Register a Business

I got a lot of mostly wishy washy advice on this. There are people out there that say if you're not making much just don't bother. That may be a fair enough point, but I would say if you truly want to pursue freelancing, even on a moonlighting basis, to just register a business.

First of all, you can expense stuff to your business (legit business expenses only!) and it will probably save you the cost of registering the business. It will also make your taxes a lot more straightforward. Being a software engineer who likes open source this felt a bit too beaurocratic for me, but seriously, just do it.

...
Continue Reading...

Setup a Bioinformatics Demultiplex Server from Scratch

Install Demultiplex Software

Installing demultiplexing such as bcl2fastq, CellRanger, LongRanger, demuxlet, and whatever else pops up, holds a special place in those that do Bioinformatics and Genomics hearts and potential support groups. It has been enough of an issue in my professional life that I thought I would dedicate a series to setting up servers for different analysis types.

Don't install system packages

This is my big chance to go on a total rant about bioinformatics servers!

Don't install all kinds of software as system packages. Ok? Just don't do it. It may not backfire on you today, or tomorrow, but someday it will!

I'm going to make a few caveats to that. Things like zlib, openssl, and ssh are fine. I'll even cheat sometimes and yum install some development tools. Mostly, what I am talking about here is bioinformatics software. Don't bother installing bcl2fastq, blast, augustus, R, python, dask, or pretty much anything else as system dependencies.

There are better...

Continue Reading...

AWS Elastic Compute Clusters for Genomics

aws bioinformatics hpc Oct 30, 2019

Amazon Web Services (AWS) gives you the ability to build out all kinds of cool infrastructure. Databases, web servers, Kubernetes backed applications, Spark clusters, machine learning models, and even High-Performance Computing Clusters with AWS ParallelCluster.

 

Not just clusters, but Elastic Clusters!

One of the cooler aspects of using a cloud provider like AWS is the ability to scale up and down based on requests or need. This is generally called Elastic, and applies to a whole lot of services. Storage, Kubernetes, load balancers, and compute clusters. This is first of all just awesome, because writing up something to scale up or down based on demand would be a major pain, and can give the best of all worlds.

Example Genomic Analysis Computional Needs

Let's say you're running a genomics analysis. First, you run your alignment, which takes (for the sake of argument) 20 nodes. Then you do variant calling, which takes 5 compute nodes, haplotype...

Continue Reading...

Deploy HPC Modules From Bioconda Packages

The Struggle is Real

I have been working in Bioinformatics for nearly 10 years, mostly on the computational side of things. I have spent a lot of that time building and installing software. Some of those wounds will never heal! Luckily, along came Anaconda, the scientific distribution of Python, along with the awesome BioConda who took on the task of installing bioinformatics software with relative ease! I don't know if Anaconda necessarily wanted to make life easier for those installing software on HPC systems, but in any case they did. 

(Disclaimer, I am technically a core team member of BioConda, but I'm really kind of a slacker core member and the real credit goes to the rest of the team!)

Deploy Modules with EasyBuild

One of my main goals in life is to deploy conda packages as HPC Modules. Deploying HPC Modules can be a bit of a pain. There are a lot of naming conventions, environmental variables, file permissions, recursive file...

Continue Reading...
1 2 3 4
Close

50% Complete

Two Step

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.