BioAnalyze (Page 1)

Sign in Subscribe

Latest

Speeding Up UMI-tools with Cython: A 4.6x Performance Win

Speeding Up UMI-tools with Cython: A 4.6x Performance Win

Overview You know that feeling when you're running a bioinformatics pipeline and you watch the progress bar crawl along at a glacial pace? You grab a coffee, check your email, maybe start a second analysis on another dataset... and 90 seconds later your first job is still processing

Bioinformatics Pipeline Engineering: Do This Not That: Split Tasks

Bioinformatics Pipeline Engineering: Do This Not That: Split Tasks

Overview In the last installment of this series, we talked about how to handle large files without bringing down the wrath of your local sys admin. Today we're tackling how to think about breaking up tasks. You've probably written it. I've definitely written it.

Bioinformatics Pipeline Engineering: Do This Not That: Large Files

Bioinformatics Pipeline Engineering: Do This Not That: Large Files

Overview If you've worked with bioinformatics data, you've likely encountered the dreaded experience: your Python script chugging along, consuming memory at an alarming rate, and then—crash. Out of memory. Your computer becomes unresponsive, and you're left frustrated wondering why your beautiful analysis script

ML/AIOps for Drug Discovery with Boltz2 and MLflow

ML/AIOps for Drug Discovery with Boltz2 and MLflow

Overview If you've ever worked with scientific software you know that it doesn't play well with the other children on the playground. Some of my favorite work related activities are getting rid of code and finding existing code, that I didn't write, to use.

Building a Natural Language Interface for OpenTargets Data with AI on AWS

Building a Natural Language Interface for OpenTargets Data with AI on AWS

With the advent of AI, knowledge bases and MCP servers I'm less inclined to write code than I ever have been. This blog post details how I built a complete natural language interface for querying OpenTargets data using Amazon Q Developer, AWS infrastructure and the Model Context Protocol

Using LLMs to Query PubMed Knowledgebases for BioMedical Research

Part 1: Creating the Knowledgebase In this article, we'll explore how to leverage large language models (LLMs) to search and query scientific papers from PubMed, a free resource for accessing biomedical and life sciences literature. We'll be using AWS Bedrock as our AI backend, PostgreSQL as

Bioinformatics Solutions on AWS Book - Preview

Bioinformatics Solutions on AWS Book - Preview

This is the introductory chapter to my new book, Bioinformatics Solutions on AWS. (I am aware that the links surrounded by brackets don't work. This is meant to be a book, not a website, but I will use the power of find and sed in order to bend

python

Bioinformatics Solutions on AWS For Exploratory Analysis

Bioinformatics Solutions on AWS For Exploratory Analysis

This is part 1 of a series I have in the works about Bioinformatics Solutions on AWS. Each part of the series will stand on its own, so you won't be missing anything by reading one part and not the others. So let's dive right in!

Deploy Dash with Helm on Kubernetes with AWS EKS

Deploy Dash with Helm on Kubernetes with AWS EKS

Today we are going to be talking about Deploying a Dash App on Kubernetes with a Helm Chart using the AWS Managed Kubernetes Service EKS. For this post, I'm going to assume that you have an EKS cluster up and running because I want to focus more on

Deploy and Scale your Dask Cluster with Kubernetes

Deploy and Scale your Dask Cluster with Kubernetes

Dask is a parallel computing library for Python. I think of it as being like MPI without actually having to write MPI code, which I greatly appreciate! Dask natively scales Python Dask provides advanced parallelism for analytics, enabling performance at scale for the tools you love https://dask.org One

Learn Apache Airflow By Example – 4 Part Series

Learn Apache Airflow By Example – 4 Part Series

Introduction I've been having a blast the last few months learning ApacheAirflow. It's become an indispensable tool in mygetting stuff done toolbox. Follow me on Twitter @jillianerowe, or send me a message with questions, topic suggestions, ice cream recommendations, or general shenanigans. * Learn Apache Airflow By

Dask Tips and Tricks – HighLevelGraphs

Dask Tips and Tricks – HighLevelGraphs

For those of you who have written MPI, this is kind of like that, except you don't have to write MPI!If you would like to know more about basic Dask syntax, check out my blog post on Parallelizing For Loops with Dask. Dask Syntax Normally when using

Dask Tips and Tricks – Parallelize a For Loop

Dask Tips and Tricks – Parallelize a For Loop

If you're in the scientific computing space there's a good chance you've used Python. A relatively recent addition to the family of awesome libraries in Python for Scientific Computing isDask. It is a super cool library that allows you to parallelize your code with

Dask on HPC

Dask on HPC

Recently I saw that Dask, a distributed Python library, created some really handy wrappers for running Dask projects on a High-Performance Computing Cluster, HPC. Most people who use HPC are pretty well versed in technologies like MPI, and just generally abusing multiple compute nodes all at once, but I think

AWS

Building a Natural Language Interface for OpenTargets Data with AI on AWS

Building a Natural Language Interface for OpenTargets Data with AI on AWS

With the advent of AI, knowledge bases and MCP servers I'm less inclined to write code than I ever have been. This blog post details how I built a complete natural language interface for querying OpenTargets data using Amazon Q Developer, AWS infrastructure and the Model Context Protocol

Bioinformatics Solutions on AWS Book - Preview

Bioinformatics Solutions on AWS Book - Preview

This is the introductory chapter to my new book, Bioinformatics Solutions on AWS. (I am aware that the links surrounded by brackets don't work. This is meant to be a book, not a website, but I will use the power of find and sed in order to bend

Bioinformatics Solutions on AWS For Exploratory Analysis

Bioinformatics Solutions on AWS For Exploratory Analysis

This is part 1 of a series I have in the works about Bioinformatics Solutions on AWS. Each part of the series will stand on its own, so you won't be missing anything by reading one part and not the others. So let's dive right in!

Deploy Dash with Helm on Kubernetes with AWS EKS

Deploy Dash with Helm on Kubernetes with AWS EKS

Today we are going to be talking about Deploying a Dash App on Kubernetes with a Helm Chart using the AWS Managed Kubernetes Service EKS. For this post, I'm going to assume that you have an EKS cluster up and running because I want to focus more on

RShiny Authentication with Polished on AWS Kubernetes

RShiny Authentication with Polished on AWS Kubernetes

If you're looking for a hassle free way to add authentication to your RShiny Apps you should check out polished.tech. In their own words: Polished is an R package that adds authentication, user management, and other goodies to your Shiny apps. Polished.tech is the quickest and

Deploy and Scale your Dask Cluster with Kubernetes

Deploy and Scale your Dask Cluster with Kubernetes

Dask is a parallel computing library for Python. I think of it as being like MPI without actually having to write MPI code, which I greatly appreciate! Dask natively scales Python Dask provides advanced parallelism for analytics, enabling performance at scale for the tools you love https://dask.org One

Deploy RShiny with Kubernetes using AWS EKS and Terraform

Deploy RShiny with Kubernetes using AWS EKS and Terraform

Deploy RShiny on AWS EKS with Terraform Today we will be deploying the rocker/shiny image on Kubernetes with AWS, or EKS. If you're following along with the deploy RShiny on AWS Series, you'll know that I covered deploying RShiny with a helm chart . Today, I

docker

Deploy RShiny with the Rocker/Shiny Docker Image

Deploy RShiny with the Rocker/Shiny Docker Image

If you are deploying RShiny with docker you can roll your own image , or you can use the rocker Dockerhub image . Which solution you go for will depend upon your own needs. Normally, I am using a lot of bioinformatics and/or data science packages and nothing really beats the

Deploy your RShiny App Locally with Docker

Deploy your RShiny App Locally with Docker

My favorite way to deploy RShiny locally is to simply package it into a docker image and run it. Running any application in docker makes it easily transportable, and is a generally acceptable way of distributing applications in 2019. This solution does not require rshiny-server or shinyapps.io, not because

Learn Apache Airflow By Example – 4 Part Series

Learn Apache Airflow By Example – 4 Part Series

Introduction I've been having a blast the last few months learning ApacheAirflow. It's become an indispensable tool in mygetting stuff done toolbox. Follow me on Twitter @jillianerowe, or send me a message with questions, topic suggestions, ice cream recommendations, or general shenanigans. * Learn Apache Airflow By

Dask Tips and Tricks – HighLevelGraphs

Dask Tips and Tricks – HighLevelGraphs

For those of you who have written MPI, this is kind of like that, except you don't have to write MPI!If you would like to know more about basic Dask syntax, check out my blog post on Parallelizing For Loops with Dask. Dask Syntax Normally when using

Dask Tips and Tricks – Parallelize a For Loop

Dask Tips and Tricks – Parallelize a For Loop

If you're in the scientific computing space there's a good chance you've used Python. A relatively recent addition to the family of awesome libraries in Python for Scientific Computing isDask. It is a super cool library that allows you to parallelize your code with

Deploy Bioinformatics Modules on HPC

Deploy Bioinformatics Modules on HPC

Deploying scientific software in an HPC environment can be challenging. Deploying bioinformatics software anyhow anywhere, or say a High Performance Computing Cluster (HPC), can be especially challenging! Life, at least in this regard, has become so much better in the last few years. Anaconda, the scientific python distro along with

Kubernetes on AWS – Getting Started with EKS

Kubernetes on AWS – Getting Started with EKS

AWS Elastic Kubernetes Service (EKS) is a fully managed service AWS launched recently. Elastic services in AWS it means that the number of instances actually in use scales up or down based on the demand. This is first of all seriously cool, and second of all can cut down on

Bioinformatics

More

Speeding Up UMI-tools with Cython: A 4.6x Performance Win

Bioinformatics Pipeline Engineering: Do This Not That: Split Tasks

Bioinformatics Pipeline Engineering: Do This Not That: Large Files

Bioinformatics Solutions on AWS Book - Preview

Bioinformatics Solutions on AWS For Exploratory Analysis

distributed computing

More

Bioinformatics Solutions on AWS For Exploratory Analysis

Dask Tips and Tricks – HighLevelGraphs

Dask Tips and Tricks – Parallelize a For Loop

Dask on HPC

Deploy Bioinformatics Modules on HPC

rshiny

More

RShiny Authentication with Polished on AWS Kubernetes

Deploy RShiny with Kubernetes using AWS EKS and Terraform

Deploy RShiny with the Rocker/Shiny Docker Image

Deploy RShiny on Kubernetes with a Helm Chart

Deploy RShiny on AWS EC2