Dockerize all the Things!

If you've read anything on this blog you will know I am all about docker and I am all about world domination with distributed computing, particularly with Python Applications.

This week marks something new for me. I've written plenty of documentation, oodles of Power Point presentations with accompanying in-person training. This is the first for me to start recording myself for video, and it was an exciting start! It feels a bit silly, because there are probably millions of people on YouTube or other video formats by now, but it was a first for me. 

Check it out over here on the NEW Dabble of DevOps Youtube channel!

More Python + Docker Resources

If you want to see a deeper dive on building python applications in Docker containers check out my Develop a Python Application in Docker and Deploy to AWS series.

Please watch, comment, share, and let me know what you think!

Happy teching!

AWS is shiny

AWS is so, so shiny and awesome, but when I first got started I felt overwhelmed by the sheer amount of stuff. There is so much flexibility, and cli tools, and ways to do anything. Initially, all I wanted to be able to do the things I'd always been able to do, like spin up a server and deploy a docker web application. 

Today I am going to share 3 relatively easy methods to deploy your Docker projects on AWS. None of these solutions requires you to have much more AWS specific knowledge than knowing how to login. Having a bit of knowledge of security groups and virtual private clouds (VPC) will be helpful, but never fear! You can deploy your apps!

AWS EC2 Instance

Using the AWS Elastic Compute Cloud (get it? 2 Cs!) is probably the most straightforward and comfortable way for someone, including someone who knows nothing about the AWS ecosystem, to get started, but this is most certainly not a shiny solution.

Deploying your web application with NGINX

I was so stoked the first time I'd managed to build a web application. It was solving an actual problem real people were facing. It was love at first site between me and infinite scroll data tables.  

Then I had to deploy my application, and several other applications on the same server, and I was less than pleased!

Writing all the code was such a big hurtle for me, and then I had to figure out the deployment feature too. Luckily, I worked with a great IT department and they were able to point me towards some great resources. These days I do all my development in docker and keep as a library of templates for deploying my projects.

Flask + NGINX

These days I mostly use Traefik to do my load balancing and proxy passes, but its important to understand the fundamentals before you go throwing around all kinds of mostly optional prettiness. 

In this example I am deploying a Python Flask application...

Query a Redis DataStore from a Python Application

Redis is one of my favorite technologies to use when I'm building web apps. It's just super. If you haven't used it its sort of like a database, without the structure, and its blazing fast because its in memory. Have you ever set up a web application that took longer than the millisecond of patience people have when using the internet? There's a good bet you could cache quite a few requests and speed the whole thing up!

I used to do all kinds of nonsense where I would save things to temporary files and pick them back up later, or had cron jobs to stuff computations into pickle files. No more! Now I stick all that in a Redis in memory cache, which is way better and super fast!

By the end of this blog post you will have all the tools you need to query a Redis database, with some dummy data, from a python application, all without installing a single thing besides docker onto your computer! 

Learning Curve

If...

MongoDB + Python

I only just recently jumped on the MongoDB bandwagon, but I really, really like it. To me, working with MongoDB feels so much more natural. It feels like working with code. MongoDB collections are arrays of objects, as opposed to a SQL DB where we kind of make it look like an object. Compared to SQL databases it is a much newer technology, but it is quickly gaining traction and becoming more and more robust.

With that said, it's nice to have a docker-compose template on hand to quickly spin up a MongoDB + Python Stack! That is exactly what we have here today. If you read my Setup a MySQL + Python Docker Dev Stack post, this will feel very similar.

version: '3' services: mongo: image:...

Query a Postgres DB from a Python Application

By the end of this blog post you will have all the tools you need to query a Postgres database, with some dummy data, from a python application, all without installing a single thing besides docker onto your computer! 

Learning Curve

If you read my Setup a MySQL + Python Docker Dev Stack, you'll know all about my how much trouble I had figuring out that pesky connection string. It was such a nitpicky little detail, but so crucial!

That single lesson, while very frustrating, has held through with every database I have ever connected to. It doesn't matter what kind of database you are connecting to. It could be mysql, postgres, mongodb, a redis cache. You just need to know the user, password, host, and default port! 

Onwards with an Example!

If you've ever worked with scientific software you will know that installing them is not necessarily straightforward. I think this is changing quite a bit with tools like conda and docker, but sometimes we need to just sit down and debug an installation. There is hope here, because if you can get it working just once, and put it in a docker container, you don't ever have to worry about getting it working on another server! 

CellProfiler

CellProfiler is a free open-source software designed to enable biologists without training in computer vision or programming to quantitatively measure phenotypes from thousands of images automatically. More information can be found in the CellProfiler Wiki. 

CellProfiler GitHub

Cool stuff. CellProfiler is an industry standard, and beyond useful for many scientists. It's totally worth packaging up into a docker container, and there is in fact one available from the wonderful people at CellProfiler. I needed to...

Overview

If you are anything like me, you heavily rely on a debugger. Likewise, you install all your development environments into a docker container because installing things on your own computer is FOR THE BIRDS. I cannot tell you what a relief it is to have all my development environments safely in docker containers. I break operating systems frequently. It's a serious problem for me and I needed an intervention. For me, this intervention was docker. I do not care how often I break a docker container, because 1. it's versioned, and 2. I can always just rebuilt it. MAGICAL.

The Problem

Many IDEs and editors assume you are working from a local environment. I think this is changing with so many people moving towards cloud computing and technologies like docker. I don't have an exact list of IDEs that do and do not have use an interpreter from a docker or docker-compose environment. 

The Solution (Well, a solutions anyways! )

The JetBrains IDEs support...