NIFI RDB to HDFS https://gist.github.com/cheerupdi/87eacaa87b74feee4de0bb5eba0216d2 6. NiFi RDB to HDFS 6. NiFi RDB to HDFS. GitHub Gist: instantly share code, notes, and snippets. gist.github.com BigData 2021.05.31
NIFI with Kafka https://towardsdatascience.com/big-data-managing-the-flow-of-data-with-apache-nifi-and-apache-kafka-af674cd8f926 Apache NiFi And Kafka Docker Example An example of how to publish data to kafka docker container using a nifi processor. towardsdatascience.com BigData 2021.05.31
NIFI with OIDC https://eyeballs.tistory.com/410 [NiFi] standalone NiFi 에 security 적용하는 방법(OIDC) google OAuth2 를 이용하여 NiFi 를 암호화하는 방법에 대해 설명한다. NiFi 에서는 OIDC(OpenID Connect) 라고 부르는가 봄. Apache NiFi 가 기본 설정값으로 설치되었다면, NiFi WebUI 에는 누구나 접근이 가능.. eyeballs.tistory.com BigData 2021.05.31
NiFi with Prometheus and Grafana https://github.com/m-hogue/nifi-k8s m-hogue/nifi-k8s Simple Kubernetes YAMLs for deploying Apache NiFi and supporting components in k8s - m-hogue/nifi-k8s github.com https://grafana.com/grafana/dashboards/11387 NiFi Monitoring dashboard for Grafana General view of Nifi grafana.com ## 1.13.2 에는 spark-receiver 가 있다. docker run --name nifi \ -p 9090:9090 \ -d \ -e NIFI_WEB_HTTP_PORT='9090' \ apache.. BigData 2021.05.31
AWS2 ECR Login aws sts get-caller-identity를 사용하여, Account ID 확인 $ aws ecr get-login-password --region ap-northeast-2 | docker login --username AWS --password-stdin "$(aws sts get-caller-identity --query Account --output text).dkr.ecr.ap-northeast-2.amazonaws.com" 혹은 $ aws ecr get-login-password --region ap-northeast-2 | docker login --username AWS --password-stdin {ECR URI} AWS 2021.05.28
Dagster https://github.com/dagster-io/dagster dagster-io/dagster A data orchestrator for machine learning, analytics, and ETL. - dagster-io/dagster github.com https://docs.dagster.io/deployment/guides/docker Dagster Docs A data orchestrator for machine learning, analytics, and ETL docs.dagster.io BigData 2021.05.27
Git 권한 되돌리기 git config --global --add alias.permission-reset '!git diff -p -R --no-ext-diff --no-color | grep -E "^(diff|(old|new) mode)" --color=never | git apply' git permission-reset git - git 내에서 파일과 디렉토리의 권한을 수정 한 경우 어떻게 복원합니까? - IT 툴 넷 (pythonq.com) git - git 내에서 파일과 디렉토리의 권한을 수정 한 경우 어떻게 복원합니까? - IT 툴 넷 git - git 내에서 파일과 디렉토리의 권한을 수정 한 경우 어떻게 복원합니까? 출처 git file file-permissions 자식 체크 아웃이 있습니다. 모든 파일 .. ETC. 2021.05.27
Using the Amazon SageMaker Studio Image Build https://aws.amazon.com/ko/blogs/machine-learning/using-the-amazon-sagemaker-studio-image-build-cli-to-build-container-images-from-your-studio-notebooks/ Using the Amazon SageMaker Studio Image Build CLI to build container images from your Studio notebooks | Amazon Web Services The new Amazon SageMaker Studio Image Build convenience package allows data scientists and developers to easily build cu.. AWS 2021.05.25
AWS SSO https://activate.workshop.aws/020_landingzone/sso/steps/10-activate-sso.html Activate Academy: early stage path Activate AWS SSO Activate AWS SSO To allow users to sign in to your AWS environment with Single Sign-On (SSO), you need to activate AWS SSO. The basic procedure is outlined here, but you can check the official documentation for more details. Go to the AWS activate.workshop.aws https://.. AWS 2021.05.25
Comparing Machine Learning Algorithms' Performance using Machine Learning Operations (MLOps) and the AWS Cloud Development Kit (CDK) https://aws.amazon.com/ko/blogs/architecture/field-notes-comparing-algorithm-performance-using-mlops-and-the-aws-cloud-development-kit/ Field Notes: Comparing Algorithm Performance Using MLOps and the AWS Cloud Development Kit | Amazon Web Services Comparing machine learning algorithm performance is fundamental for machine learning practitioners, and data scientists. The goal is to evaluate the .. AWS 2021.05.25