Sign in Subscribe

apache airflow

Get a Fully Configured Apache Airflow Docker Dev Stack with Bitnami

Get a Fully Configured Apache Airflow Docker Dev Stack with Bitnami

I've been using it for around 2 years now to build out custom workflow interfaces, like those used for Laboratory Information Management Systems (LIMs), Computer Vision pre and postprocessing pipelines, and to set and forget other genomics pipelines. My favorite feature of Airflow is how completely agnostic it

Manage High Content Screening CellProfiler Pipelines with Apache Airflow

Manage High Content Screening CellProfiler Pipelines with Apache Airflow

If you are running a High Content Screening Pipeline you probably have a lot of moving pieces. As a non exhaustive list you need to: * Trigger CellProfiler Analyses, either from a LIMS system, by watching a filesystem, or some other process. * Keep track of dependencies of CellProfiler Analyses - first

Dask Tips and Tricks – HighLevelGraphs

Dask Tips and Tricks – HighLevelGraphs

For those of you who have written MPI, this is kind of like that, except you don't have to write MPI!If you would like to know more about basic Dask syntax, check out my blog post on Parallelizing For Loops with Dask. Dask Syntax Normally when using

Dask Tips and Tricks – Parallelize a For Loop

Dask Tips and Tricks – Parallelize a For Loop

If you're in the scientific computing space there's a good chance you've used Python. A relatively recent addition to the family of awesome libraries in Python for Scientific Computing isDask. It is a super cool library that allows you to parallelize your code with

Deploy Bioinformatics Modules on HPC

Deploy Bioinformatics Modules on HPC

Deploying scientific software in an HPC environment can be challenging. Deploying bioinformatics software anyhow anywhere, or say a High Performance Computing Cluster (HPC), can be especially challenging! Life, at least in this regard, has become so much better in the last few years. Anaconda, the scientific python distro along with

3 Easy Ways to Deploy your Docker Web App on AWS

3 Easy Ways to Deploy your Docker Web App on AWS

AWS is shiny AWS is so, so shiny and awesome, but when I first got started I felt overwhelmed by the sheer amount of stuff. There is so much flexibility, and cli tools, and ways to do anything. Initially, all I wanted to be able to do the things I&

Configure Nginx to Serve a Flask App

Configure Nginx to Serve a Flask App

Deploying your web application with NGINX I was so stoked the first time I'd managed to build a web application. It was solving an actual problem real people were facing. It was love at first site between me and infinite scroll data tables. Then I had to deploy