Airflow tutorial 2: Set up airflow environment with docker

1. Airflow problem

• open source는 압도적인 속도로 자라난다.
• 모든 디펜던씨의 변화들을 다루고 유지하기는 힘들다.
• Airflow env를 config하는데 시간이 오래걸린다.
• 모든 개발자에게 developmet와 production environments를 어떻게 공유해야할까?

2. Docker

• Docker는 application들을 developing, shipping, running하기위한 플랫폼이다.
• Docker provides the ability to package and run an application in a loosely isolated environment called a container.
• The isolation and security allow you to run many containers simultaneously on a given host, regardless of its operating system: Mac, Windows, PC, cloud, data center, …
• 

2-1. Benefits of using Docker

• Docker is freeing us from the task of managing, maintaining all of the Airflow dependencies, and deployment.
• Easy to share and deploy different versions and environments.
• Keep track through Github tags and releases.
• Ease of deployment from testing to production environment.

Reference

https://www.applydatascience.com/airflow/set-up-airflow-env-with-docker/