What you'll get
- Airflow Architecture - the different components of Apache Airflow demystified
- Airflow Versions - for the love of all that is good in the world please pin your versions!
- Best Practices - well, what I think are best practices anyways.
- Bitnami - We use the Bitnami Airflow stack as our base for world domination!
- Development Stack
- Build Custom Images
- Supply DAGs
- Bring up the Stack
- Watch for Changes
- Build out your Data Science Infrastructure
- Trigger DAGs
- Configure Apache Airflow
- CORs
- Add Connections
- Turn DAGs on or off at runtime
- Remote Operators
- Plugins
- Custom Operators
- REST APIs
- Continuous Integration and Deployment
- Deploy to Production with Kubernetes
- Cheatsheet
- References
You'll get a docker compose that that you can bring up with a single command, instructions on how to build and modify custom docker images, build and upload your images to your favorite docker registry, build out custom REST APIs with Flask or Flask RESTful, deploy to Kubernetes, and more!
If you have any questions, comments, or suggestions, please don't hesitate to reach out to me at [email protected].