BigData

Airflow/EMR on EKS Using CDK (spark job, aws sso)

Kyle79 2021. 5. 10. 16:48

 

 

 

laptrinhx.com/using-aws-cdk-to-deploy-your-amazon-managed-workflows-for-apache-airflow-environment-2530058942/

 

Using AWS CDK to deploy your Amazon Managed Workflows for Apache Airflow environment

Using AWS CDK to deploy your Amazon Managed Workflows for Apache Airflow environment What better way to celebrate CDK Day tomorrow, than to return to a previous blog where I...

laptrinhx.com

 

 

 

github.com/094459/blogpost-cdk-mwaa

 

094459/blogpost-cdk-mwaa

Contribute to 094459/blogpost-cdk-mwaa development by creating an account on GitHub.

github.com

 

 

## CDK 를 통해서 EKS on Fargate 자동 생성의 최신 블로그포스트와 Github 내용

 

https://aws.amazon.com/ko/blogs/containers/building-and-deploying-fargate-with-eks-in-an-enterprise-context-using-the-aws-cloud-development-kit-and-cdk8s/

 

Building and deploying Fargate with EKS in an enterprise context using the AWS Cloud Development Kit and cdk8s+ | Amazon Web Ser

Amazon Elastic Kubernetes Service (Amazon EKS) is a fully managed service that helps customers run their Kubernetes (K8s) clusters at scale by minimizing the effort required to operate the Kubernetes control plane. When you combine Amazon EKS to manage the

aws.amazon.com

 

 

github.com/aws-samples/cdk-eks-fargate

 

aws-samples/cdk-eks-fargate

Contribute to aws-samples/cdk-eks-fargate development by creating an account on GitHub.

github.com

 

 

 

github.com/aws-samples/aws-cdk-for-emr-on-eks/blob/main/emr_eks_cdk/emr_eks_cdk_stack.py

 

aws-samples/aws-cdk-for-emr-on-eks

Contribute to aws-samples/aws-cdk-for-emr-on-eks development by creating an account on GitHub.

github.com

 

 

 

https://www.eksworkshop.com/advanced/430_emr_on_eks/

 

EKSworkshop.com

In this workshop, we will explore multiple ways to configure VPC, ALB, and EC2 Kubernetes workers, and Amazon Elastic Kubernetes Service.

www.eksworkshop.com

 

 

 

--------------------------------------------------------------------------

emr on eks 's spark job

 

https://github.com/tatwan/airflow-spark-aws-emr

 

tatwan/airflow-spark-aws-emr

Contribute to tatwan/airflow-spark-aws-emr development by creating an account on GitHub.

github.com

 

 

https://itcareersholland.nl/orchestrate-an-amazon-emr-on-amazon-eks-spark-job-with-aws-step-functions/

 

Orchestrate an Amazon EMR on Amazon EKS Spark job with AWS Step Functions – ITcareersholland

 

itcareersholland.nl

 

 

https://itnext.io/running-spark-jobs-on-amazon-emr-with-apache-airflow-2e16647fea0c

 

Running Spark Jobs on Amazon EMR with Apache Airflow

Using the new Amazon Managed Workflows for Apache Airflow (MWAA) on AWS

itnext.io

 

 

 

 

--------------------------------------------------------------------------

aws sso

 

https://aws.amazon.com/ko/blogs/korea/single-sign-on-between-okta-universal-directory-and-aws/

 

AWS Single Sign-On(SSO)와 Okta Universal Directory 통합하기 | Amazon Web Services

AWS 클라우드를 채택하는 기업에서 사용자의 클라우드 자원 접근 자격 증명을 효율적으로 관리해야 합니다. 자격 증명을 관리할 단일의 중앙 위치가 있으면 정책 적용 및 액세스 권한 관리 작업

aws.amazon.com

 

'BigData' 카테고리의 다른 글

Apache Airflow & BigQuery  (0) 2021.05.10
Airflow Slack notifications  (0) 2021.05.10
Monitor Amazon EKS on AWS Fargate with Datadog  (0) 2021.04.30
MWAA on EKS fargate and DAG  (0) 2021.04.30
puppeteer  (0) 2021.04.22