쉼터

https://lascrea.tistory.com/204 SaltStack SaltStack 대규모 인프라를 관리하기 위한 자동화 관리 시스템 (Puppet & Chef와 같은 System Management Tool) Server (Master) - Agent (Minion) 간 ZeroMQ를 통해 통신하며 Server → Agent의 요청은 비동기 병.. lascrea.tistory.com https://bluese05.tistory.com/41 Saltstack 너란 녀석. - 장점과 구조(1) Saltstack 이 뭔가요. 인프라의 규모가 커질 수록 서버 설치와 설정에 대한 부담이 늘어나기 마련이다. 특히 트래픽이 급격히 늘어나는 경우, 이에 빠르게 대처하여 서버를 구축하기 위해서는 자� blue..

https://medium.com/swlh/bigquery-python-for-production-data-science-aa020aa5b7e2 BigQuery + Python for Production Data Science For the past year and a half, we at Brandfolder have been moving toward using BigQuery as a datalake. It began by storing a copy of all… medium.com

https://pypi.org/project/s3fs/#description s3fs Convenient Filesystem interface over S3 pypi.org import s3fs bytes_to_write = df.to_csv(None).encode() fs = s3fs.S3FileSystem(key=key, secret=secret) with fs.open('s3://bucket/path/to/file.csv', 'wb') as f: f.write(bytes_to_write) s3fs는 파일을 여는 rb 및 wb 모드 만 지원하므로이 bytes_to_write 작업을 수행 한 것입니다. import s3fs s3 = s3fs.S3FileSystem(anon=False) # Use 'w'..

https://www.ibm.com/cloud/architecture/architectures/cloud-data-lake Cloud data lake - IBM Cloud Architecture Center Use object storage as a highly elastic, available, durable, and affordable storage repository and connect and flow data from various data sources in the cloud and on premises. www.ibm.com

https://medium.com/google-cloud/visualising-bigquery-41bf6833b98 Visualizing BigQuery query plans Tldr: visualise BigQuery execution on https://bqvisualiser.appspot.com medium.com https://github.com/GoogleCloudPlatform/professional-services/tree/master/tools/bq-visualizer GoogleCloudPlatform/professional-services Common solutions and tools developed by Google Cloud's Professional Services team -..

https://meetup.toast.com/posts/122 GitHub의 Merge, Squash and Merge, Rebase and Merge 정확히 이해하기 : TOAST Meetup GitHub의 Merge, Squash and Merge, Rebase and Merge 정확히 이해하기 meetup.toast.com

http://aws-lakeformation-workshop.s3-website.ap-northeast-2.amazonaws.com/ AWS Data Lake Workshop :: AWS Data Lake Workshop AWS Data Lake Workshop 본 워크샵은 AWS Lake Formation을 이용하여 데이터 분석 파이프라인을 구축 및 학습하는 것이 목적입니다. AWS Lake Formation은 내부적으로 AWS Glue 서비스 기반으로 다양한 AWS 분석 서비 aws-lakeformation-workshop.s3-website.ap-northeast-2.amazonaws.com

Airflow 1.10.10 version: "3.7" x-airflow-environment: &airflow-environment AIRFLOW__CORE__EXECUTOR: CeleryExecutor AIRFLOW__WEBSERVER__RBAC: "True" AIRFLOW__CORE__LOAD_EXAMPLES: "False" AIRFLOW__CELERY__BROKER_URL: "redis://:@redis:6379/0" AIRFLOW__CORE__SQL_ALCHEMY_CONN: postgresql+psycopg2://airflow:airflow@postgres:5432/airflow services: postgres: image: postgres:11.5 environment: POSTGRES_US..

https://cloud.hosting.kr/aws-waf/ https://cloud.hosting.kr/aws-waf/ cloud.hosting.kr https://cloud.hosting.kr/aws-waf-sql-ingection/ https://cloud.hosting.kr/aws-waf-sql-ingection/ cloud.hosting.kr

https://stackoverflow.com/questions/33887194/how-to-set-multiple-commands-in-one-yaml-file-with-kubernetes How to set multiple commands in one yaml file with Kubernetes? In this official document, it can run command in a yaml config file: https://kubernetes.io/docs/tasks/configure-pod-container/ apiVersion: v1 kind: Pod metadata: name: hello-world spec: # stackoverflow.com