BigData

Prefect (vs airflow)

Kyle79 2021. 10. 29. 16:02

 

 

https://www.prefect.io/

 

Prefect - The New Standard in Dataflow Automation

Prefect is a new data workflow management system, designed for modern infrastructure. Prefect provides a flexible open-source Python framework to easily combine tasks into workflows, then deploy, schedule, and monitor their execution through the Prefect UI

www.prefect.io

 

 

 

https://dailyheumsi.tistory.com/264

 

Prefect - Quick Review

Workflow 툴인 Prefect에 대해서 Quick하게 리뷰해보려고 한다. 이게 대체 뭔가 함은... 일단은 Apache Airflow 같이 워크 플로우를 정의하고 스케쥴링 돌리며, 이 모든 것을 체계적으로 매니징하는 오케스

dailyheumsi.tistory.com

 

 

https://hub.docker.com/r/prefecthq/prefect

 

Docker Hub

 

hub.docker.com

 

 

https://github.com/dojeda/prefect-helm

 

GitHub - dojeda/prefect-helm: A helm chart for Prefect

A helm chart for Prefect. Contribute to dojeda/prefect-helm development by creating an account on GitHub.

github.com

 

 

https://towardsdatascience.com/distributed-data-pipelines-made-easy-with-aws-eks-and-prefect-106984923b30

 

Distributed data pipelines made easy with AWS EKS and Prefect

How to set up a distributed cloud workflow orchestration system within minutes and focus on providing value rather than on managing…

towardsdatascience.com

 

 

https://teamdable.github.io/techblog/Prefect-Part-2?fbclid=IwAR2jYrqE30cb2VzUv0ubSY8fakylRVPY11QCQMfIcCNFTGGmQboEbjPkkv8 

 

Prefect Part 2 - AWS

안녕하세요. 오태호입니다.

teamdable.github.io

 

 

 

 

 

 

'BigData' 카테고리의 다른 글

Redis HyperLogLog - Unique Data Check  (0) 2021.11.11
Kafka Uber 사례 - Kafka Lag 분산 프로세스  (0) 2021.11.11
cp-helm-charts & kafka-rest  (0) 2021.09.14
TablePlus  (0) 2021.09.03
Redis Desktop Manager & Medis  (0) 2021.09.02