[DE]window에 apache airflow 설치하기(feat.WLS)

AIRFLOW는 데이터 흐름을 알기 쉽게 보여주는 좋은 프로그램이지만(실제로 현장에서도 많이 사용하는 편) 아쉽게도 윈도우에 설치가 되지 않는다..

그래서 윈도우에 WLS를 설치하고 가상의 linux환경을 만들어 우회로로 설치해야한다. (MAC살껄....)

이리저리 시도했지만 실패하고..
비슷한 상황의 블로그를 찾아 시도한 내용을 공유한다.

참고 블로그 : https://dschloe.github.io/settings/apache_airflow_using_wsl2/

문제발생_삐용삐용

Collecting psycopg2>=2.8.0
  Using cached psycopg2-2.9.5.tar.gz (384 kB)
  Preparing metadata (setup.py) ... error
  error: subprocess-exited-with-error

  × python setup.py egg_info did not run successfully.
  │ exit code: 1
  ╰─> [25 lines of output]
      /mnt/c/users/Chaaaaabbo/airflow/venv/lib/python3.10/site-packages/setuptools/config/setupcfg.py:508: SetuptoolsDeprecationWarning: The license_file parameter is deprecated, use license_files instead.
        warnings.warn(msg, warning_class)
      running egg_info
      creating /tmp/pip-pip-egg-info-t0kgp7x6/psycopg2.egg-info
      writing /tmp/pip-pip-egg-info-t0kgp7x6/psycopg2.egg-info/PKG-INFO
      writing dependency_links to /tmp/pip-pip-egg-info-t0kgp7x6/psycopg2.egg-info/dependency_links.txt
      writing top-level names to /tmp/pip-pip-egg-info-t0kgp7x6/psycopg2.egg-info/top_level.txt
      writing manifest file '/tmp/pip-pip-egg-info-t0kgp7x6/psycopg2.egg-info/SOURCES.txt'

      Error: pg_config executable not found.

      pg_config is required to build psycopg2 from source.  Please add the directory
      containing pg_config to the $PATH or specify the full executable path with the
      option:

          python setup.py build_ext --pg-config /path/to/pg_config build ...

      or with the pg_config option in 'setup.cfg'.

      If you prefer to avoid building psycopg2 from source, please install the PyPI
      'psycopg2-binary' package instead.

      For further information please check the 'doc/src/install.rst' file (also at
      <https://www.psycopg.org/docs/install.html>).

      [end of output]

  note: This error originates from a subprocess, and is likely not a problem with pip.
error: metadata-generation-failed

× Encountered error while generating package metadata.
╰─> See above for output.

note: This is an issue with the package mentioned above, not pip.
hint: See above for details.

psycopg2 관련 모듈이 업데이트 되면서 postgresql 바이너리를 찾지 못하는 것 같다.
그래서 중간에 미리 소스에서 다운 받으라고도 하고...!

sudo apt-get install libpq-dev를 통해 설치해주면 문제없이 지나가게 된다! 홍홍홍

그리고나서
pip3 install 'apache-airflow[postgres, slack, celery]'를 진행하면 airflow 설치 완료!

눈물의 아파치 에어플로우....! 이제 설정으로 넘어가볼까

참고문서

https://stackoverflow.com/questions/11618898/pg-config-executable-not-found