Develop and Deploy Python Applications with Docker

docker python Jun 28, 2019

Dockerize all the Things!

If you've read anything on this blog you will know I am all about docker and I am all about world domination with distributed computing, particularly with Python Applications.

This week marks something new for me. I've written plenty of documentation, oodles of Power Point presentations with accompanying in-person training. This is the first for me to start recording myself for video, and it was an exciting start! It feels a bit silly, because there are probably millions of people on YouTube or other video formats by now, but it was a first for me. 

Check it out over here on the NEW Dabble of DevOps Youtube channel!

 

More Python + Docker Resources

If you want to see a deeper dive on building python applications in Docker containers check out my Develop a Python Application in Docker and Deploy to AWS series.

Please watch, comment, share, and let me know what you think!

Happy teching!

Continue Reading...

3 Easy Ways to Deploy your Docker Web App on AWS

Uncategorized Jun 21, 2019

AWS is shiny

AWS is so, so shiny and awesome, but when I first got started I felt overwhelmed by the sheer amount of stuff. There is so much flexibility, and cli tools, and ways to do anything. Initially, all I wanted to be able to do the things I'd always been able to do, like spin up a server and deploy a docker web application. 

Today I am going to share 3 relatively easy methods to deploy your Docker projects on AWS. None of these solutions requires you to have much more AWS specific knowledge than knowing how to login. Having a bit of knowledge of security groups and virtual private clouds (VPC) will be helpful, but never fear! You can deploy your apps!

AWS EC2 Instance

Using the AWS Elastic Compute Cloud (get it? 2 Cs!) is probably the most straightforward and comfortable way for someone, including someone who knows nothing about the AWS ecosystem, to get started, but this is most certainly not a shiny solution.

Elastic Compute Cloud

The AWS...

Continue Reading...

Configure Nginx to Serve a Flask App

Uncategorized Jun 15, 2019

Deploying your web application with NGINX

I was so stoked the first time I'd managed to build a web application. It was solving an actual problem real people were facing. It was love at first site between me and infinite scroll data tables.  

Then I had to deploy my application, and several other applications on the same server, and I was less than pleased!

Writing all the code was such a big hurtle for me, and then I had to figure out the deployment feature too. Luckily, I worked with a great IT department and they were able to point me towards some great resources. These days I do all my development in docker and keep as a library of templates for deploying my projects.

Flask + NGINX

These days I mostly use Traefik to do my load balancing and proxy passes, but its important to understand the fundamentals before you go throwing around all kinds of mostly optional prettiness. 

In this example I am deploying a Python Flask application...

Continue Reading...

Docker Stacks with Databases

Uncategorized May 31, 2019

Connecting your Python App to a Database

All without installing a thing! Besides docker, of course! When I was first starting getting with docker I had SO MUCH trouble figuring out the connection strings. Once I got it sorted I wrote myself up a series of templates I could refer to. 

There's no real purpose to this post. I just wanted to have all the related templates in one place.

Python + MySQL

Spin up a MySQL and connect to it with python! Read the blog post.

Python + Postgres

If you want performance go for PostgresSQL and python. Read the blog post.

Python + MongoDB

I love MongoDB! It feels like CODE. Read the blog post. 

Python + Redis

If you’re looking for a fantastically fast datastore than simply fire up a Redis datastore ! You can check out the entire blog post here.

 
Continue Reading...

Setup a Redis + Python Docker Dev Stack

Uncategorized May 24, 2019

Query a Redis DataStore from a Python Application

Redis is one of my favorite technologies to use when I'm building web apps. It's just super. If you haven't used it its sort of like a database, without the structure, and its blazing fast because its in memory. Have you ever set up a web application that took longer than the millisecond of patience people have when using the internet? There's a good bet you could cache quite a few requests and speed the whole thing up!

I used to do all kinds of nonsense where I would save things to temporary files and pick them back up later, or had cron jobs to stuff computations into pickle files. No more! Now I stick all that in a Redis in memory cache, which is way better and super fast!

By the end of this blog post you will have all the tools you need to query a Redis database, with some dummy data, from a python application, all without installing a single thing besides docker onto your computer! 

Learning Curve

If...

Continue Reading...

Setup a MongoDB + Python Docker Dev Stack

Uncategorized May 13, 2019

MongoDB + Python

I only just recently jumped on the MongoDB bandwagon, but I really, really like it. To me, working with MongoDB feels so much more natural. It feels like working with code. MongoDB collections are arrays of objects, as opposed to a SQL DB where we kind of make it look like an object. Compared to SQL databases it is a much newer technology, but it is quickly gaining traction and becoming more and more robust.

With that said, it's nice to have a docker-compose template on hand to quickly spin up a MongoDB + Python Stack! That is exactly what we have here today. If you read my Setup a MySQL + Python Docker Dev Stack post, this will feel very similar.

MongoDB in Docker Compose

First of all, let's examine what just MongoDB looks like in docker compose. I'm going to go over only a very small portion of the potential configuration here. If you need more check out the official MongoDB docker image.

version: '3' services: mongo: image:...
Continue Reading...

Setup a Postgres + Python Docker Dev Stack

Uncategorized May 06, 2019

Query a Postgres DB from a Python Application

By the end of this blog post you will have all the tools you need to query a Postgres database, with some dummy data, from a python application, all without installing a single thing besides docker onto your computer! 

Learning Curve

If you read my Setup a MySQL + Python Docker Dev Stack, you'll know all about my how much trouble I had figuring out that pesky connection string. It was such a nitpicky little detail, but so crucial!

That single lesson, while very frustrating, has held through with every database I have ever connected to. It doesn't matter what kind of database you are connecting to. It could be mysql, postgres, mongodb, a redis cache. You just need to know the user, password, host, and default port! 

Onwards with an Example!

Docker-Compose networking MAGIC

When you spin up a docker-compose stack it kind of acts as if there are IT goblins living in your computer and creating hostnames...

Continue Reading...

Setup a MySQL + Python Docker Dev Stack

Uncategorized Apr 05, 2019
 

Learning Curve

The Pain

The first time I setup an app to connect to a mysql database I spent at least a full hour fiddling with that connection string, before throwing my hands up in frustration and taking a walk. I find walking to be very therapeutic, so I calmed down and figured out that I was mixing up the ports. I had a MySQL container and a node.js container. I already had something running on port 3306 on that computer, so I exposed the port on MySQL as 3307, but tried to connect to it in the node.js container as localhost:3307. 

Figuring it Out

Now I can say, well dummy, all the containers in a docker-compose stack talk to one another because docker does magic with networking, and the hostname is the same as the service name, and the port is the default internal port of the application. Hindsight and all that. 

Onwards with an Example!

Docker-Compose networking MAGIC

If you read my learning curve shenanigans above you will generally...

Continue Reading...

Creating a Custom CellProfiler Docker Image

Uncategorized Mar 25, 2019

Overview

If you've ever worked with scientific software you will know that installing them is not necessarily straightforward. I think this is changing quite a bit with tools like conda and docker, but sometimes we need to just sit down and debug an installation. There is hope here, because if you can get it working just once, and put it in a docker container, you don't ever have to worry about getting it working on another server! 

CellProfiler

CellProfiler is a free open-source software designed to enable biologists without training in computer vision or programming to quantitatively measure phenotypes from thousands of images automatically. More information can be found in the CellProfiler Wiki
CellProfiler GitHub

Cool stuff. CellProfiler is an industry standard, and beyond useful for many scientists. It's totally worth packaging up into a docker container, and there is in fact one available from the wonderful people at CellProfiler. I needed to...

Continue Reading...

Setup PyCharm to use a Project Interpreter in a Docker Compose Service

Uncategorized Mar 22, 2019

Overview

If you are anything like me, you heavily rely on a debugger. Likewise, you install all your development environments into a docker container because installing things on your own computer is FOR THE BIRDS. I cannot tell you what a relief it is to have all my development environments safely in docker containers. I break operating systems frequently. It's a serious problem for me and I needed an intervention. For me, this intervention was docker. I do not care how often I break a docker container, because 1. it's versioned, and 2. I can always just rebuilt it. MAGICAL.

The Problem

Many IDEs and editors assume you are working from a local environment. I think this is changing with so many people moving towards cloud computing and technologies like docker. I don't have an exact list of IDEs that do and do not have use an interpreter from a docker or docker-compose environment. 

The Solution (Well, a solutions anyways! )

The JetBrains IDEs support...

Continue Reading...
Close

50% Complete

Two Step

Lorem ipsum dolor sit amet, consectetur adipiscing elit, sed do eiusmod tempor incididunt ut labore et dolore magna aliqua.