Build out your data science infrastructure with Apache Airflow
This project template is production ready and documented with example Dockerfiles to include all necessary CLI tools and a fully configurable Apache Airflow development stack.
The development stack is based off of the Bitnami Docker Compose stack.
Included are Dockerfiles to customize each of the images, full examples on how to create DAGs, plugins, and custom REST APIs with either Flask or Flask-RESTful.
What you see is what you get
No surprises. Just take a look at the images below. They are screenshots directly from the project template.
Seriously though. This is exactly what you get.
Table of Contents
- Airflow Architecture
- Airflow Versions
- Best Practices
- Bitnami (or why are you so obsessed with Bitnami?)
- Development Stack
- Build Custom Images
- Supply DAGs
- Bring up the Stack
- Watch for Changes
- Build out your Data Science Infrastructure
- Trigger DAGs
- Configure Apache Airflow
- Add Connections
- Turn DAGs on or off at runtime
- Remote Operators
- Custom Operators
- REST APIs
- Continuous Integration and Deployment
- Deploy to Production with Kubernetes