MWAA + Amazon EMR on EKS
aws.amazon.com/ko/blogs/korea/introducing-amazon-managed-workflows-for-apache-airflow-mwaa/
Amazon Managed Workflows for Apache Airflow(MWAA) 정식 출시 | Amazon Web Services
데이터 처리 파이프라인의 양과 복잡성이 증가함에 따라 전체 프로세스를 일련의 작은 태스크로 분해하여 간소화하고 워크플로의 일부로 이러한 태스크의 실행을 조정할 수 있습니다. 이를 위
aws.amazon.com
aws.amazon.com/ko/blogs/korea/new-amazon-emr-on-amazon-elastic-kubernetes-service-eks/
새로운 기능 — Amazon EMR on Amazon Elastic Kubernetes Service(EKS) | Amazon Web Services
수만 명의 고객이 Amazon EMR을 사용하여 Apache Spark, Hive, HBase, Flink,Hudi 및 Presto와 같은 프레임워크에서 빅 데이터 분석 애플리케이션을 대규모로 실행합니다. EMR은 이러한 프레임워크의 프로비저닝
aws.amazon.com
https://aws.amazon.com/ko/blogs/opensource/deploying-spark-jobs-on-amazon-eks/
Deploying Spark jobs on Amazon EKS | Amazon Web Services
UPDATE, March 2021: This blog post describes how to deploy self-managed Apache Spark jobs on Amazon EKS. AWS now provides a fully managed service with Amazon EMR on Amazon EKS. This new deployment option allows customers to automate the provisioning and ma
aws.amazon.com
https://aws.amazon.com/ko/emr/features/eks/?sc_detail=blog_cta1
Amazon EMR on Amazon EKS – Amazon Web Services
With a few clicks in the Amazon EMR console, you can choose the Apache Spark version and deploy an EMR workload to Amazon EKS. EMR automatically packages the workload into a container, and provides pre-built connectors for integrating with other AWS servic
aws.amazon.com
https://aws.amazon.com/ko/blogs/containers/running-airflow-workflow-jobs-on-amazon-eks-spot-nodes/
Running Airflow Workflow Jobs on Amazon EKS with EC2 Spot Instances | Amazon Web Services
Apache Airflow is an open-source distributed workflow management platform for authoring, scheduling, and monitoring multi-stage workflows. It is designed to be extensible, and it’s compatible with several services like Amazon Elastic Kubernetes Service (
aws.amazon.com
engineering.linecorp.com/ko/blog/data-engineering-with-airflow-k8s-2/
Kubernetes를 이용한 효율적인 데이터 엔지니어링(Airflow on Kubernetes VS Airflow Kubernetes Executor) - 2 - LINE
안녕하세요. LINE Financial Data Platform을 운영하고 개발하고 있는 이웅규입니다. 이 글은 지난 NAVER DEVIEW 2020에서 발표했던 Kubernetes를 이용한 효율적인 데이터 엔지니어링 (Airflow on Kubernetes VS Airflow Ku
engineering.linecorp.com
https://github.com/aws/aws-mwaa-local-runner
aws/aws-mwaa-local-runner
This repository provides a command line interface (CLI) utility that replicates an Amazon Managed Workflows for Apache Airflow (MWAA) environment locally. - aws/aws-mwaa-local-runner
github.com
https://docs.aws.amazon.com/mwaa/latest/userguide/airflow-versions.html
Apache Airflow versions on Amazon Managed Workflows for Apache Airflow (MWAA) - Amazon Managed Workflows for Apache Airflow
Thanks for letting us know this page needs work. We're sorry we let you down. If you've got a moment, please tell us how we can make the documentation better.
docs.aws.amazon.com