# practical-data-pipeline

## practical-data-pipeline

- [Practical Data Pipeline](https://1ambda.gitbook.io/practical-data-pipeline/master.md)
- [시작 전에 드리는 당부의 말](https://1ambda.gitbook.io/practical-data-pipeline/disclaimer.md): Disclaimer
- [1.1 데이터 파이프라인](https://1ambda.gitbook.io/practical-data-pipeline/01-data-infra/1.1.md): Data Infrastructure
- [1.2 데이터 입수 (Ingestion)](https://1ambda.gitbook.io/practical-data-pipeline/01-data-infra/1.2.md)
- [1.2 데이터 가공 (Processing)](https://1ambda.gitbook.io/practical-data-pipeline/01-data-infra/1.2-processing.md)
- [1.3 데이터 저장 (Storage)](https://1ambda.gitbook.io/practical-data-pipeline/01-data-infra/1.3-storage.md)
- [1.4 데이터 분석 (Analysis)](https://1ambda.gitbook.io/practical-data-pipeline/01-data-infra/1.3.md)
- [2.1 데이터 처리](https://1ambda.gitbook.io/practical-data-pipeline/02-processing/2.1.md)
- [2.2 배치 (Batch) 처리](https://1ambda.gitbook.io/practical-data-pipeline/02-processing/2.2-batch.md)
- [2.1.1 Spark Intro](https://1ambda.gitbook.io/practical-data-pipeline/02-processing/2.2-batch/2.1.1-spark-intro.md)
- [2.1.2 Spark Tutorial](https://1ambda.gitbook.io/practical-data-pipeline/02-processing/2.2-batch/2.1.2-spark-tutorial.md)
- [2.1.3 Spark Concept](https://1ambda.gitbook.io/practical-data-pipeline/02-processing/2.2-batch/2.1.3-spark-concept.md)
- [2.1.4 Spark Architecture](https://1ambda.gitbook.io/practical-data-pipeline/02-processing/2.2-batch/2.1.4-spark-architecture.md)
- [2.1.5 Spark DataFrame](https://1ambda.gitbook.io/practical-data-pipeline/02-processing/2.2-batch/2.1.5-spark-dataframe.md)
- [2.1.6 Spark Persistence](https://1ambda.gitbook.io/practical-data-pipeline/02-processing/2.2-batch/2.1.6-spark-persistence.md)
- [2.1.7 Spark Cache](https://1ambda.gitbook.io/practical-data-pipeline/02-processing/2.2-batch/2.1.7-spark-cache.md)
- [2.1.8 Spark SQL & Table](https://1ambda.gitbook.io/practical-data-pipeline/02-processing/2.2-batch/2.1.8-spark-sql-and-table.md)
- [2.1.9 Spark Join](https://1ambda.gitbook.io/practical-data-pipeline/02-processing/2.2-batch/2.1.9-spark-join.md)
- [2.2.1 Spark Memory](https://1ambda.gitbook.io/practical-data-pipeline/02-processing/2.2-batch/2.1.5-spark-memory-management.md)
- [2.2.2 Spark Versions](https://1ambda.gitbook.io/practical-data-pipeline/02-processing/2.2-batch/2.2.2-spark-versions.md)
- [2.3 워크플로우 (Workflow) 관리](https://1ambda.gitbook.io/practical-data-pipeline/02-processing/2.3-workflow.md)
- [2.4 스트림 (Stream) 처리](https://1ambda.gitbook.io/practical-data-pipeline/02-processing/2.4-stream.md)
- [2.4.1 Kafka Intro](https://1ambda.gitbook.io/practical-data-pipeline/02-processing/2.4-stream/2.4.1-kafka-intro.md)
- [2.4.2 Kafka Advanced](https://1ambda.gitbook.io/practical-data-pipeline/02-processing/2.4-stream/2.4.2-kafka-advanced.md)
- [2.4.3 Spark Streaming](https://1ambda.gitbook.io/practical-data-pipeline/02-processing/2.4-stream/2.4.3-spark-streaming.md)
- [2.4.4 Streaming Window](https://1ambda.gitbook.io/practical-data-pipeline/02-processing/2.4-stream/2.4.4-streaming-window.md)
- [2.4.5 Streaming State](https://1ambda.gitbook.io/practical-data-pipeline/02-processing/2.4-stream/2.4.5-streaming-state.md)
- [2.4.6 Streaming Sink](https://1ambda.gitbook.io/practical-data-pipeline/02-processing/2.4-stream/2.4.6-streaming-sink.md)
- [4.1 Kafka](https://1ambda.gitbook.io/practical-data-pipeline/04-data-storage/untitled.md)
- [4.1 Kafka Concept](https://1ambda.gitbook.io/practical-data-pipeline/04-data-storage/untitled/4.1-kafka-concept.md)
- [4.2 Kafka Advanced](https://1ambda.gitbook.io/practical-data-pipeline/04-data-storage/untitled/4.2-kafka-advanced.md)
- [4.3 Kafka Versions](https://1ambda.gitbook.io/practical-data-pipeline/04-data-storage/untitled/4.3-kafka-versions.md)
- [4.2 Redis](https://1ambda.gitbook.io/practical-data-pipeline/04-data-storage/4.2.md)
- [4.3 RDB (MySQL)](https://1ambda.gitbook.io/practical-data-pipeline/04-data-storage/4.3-rdb-mysql.md)
- [4.4 ElasticSearch](https://1ambda.gitbook.io/practical-data-pipeline/04-data-storage/4.4.md)
- [4.5 KV Storage (DynamoDB)](https://1ambda.gitbook.io/practical-data-pipeline/04-data-storage/4.5-kv-storage-dynamodb.md)
- [4.6 Druid](https://1ambda.gitbook.io/practical-data-pipeline/04-data-storage/4.6-druid.md)
- [5.1 데이터 서비스](https://1ambda.gitbook.io/practical-data-pipeline/05-data-application/untitled.md)
- [5.2 통계 서비스](https://1ambda.gitbook.io/practical-data-pipeline/05-data-application/5.2.md)
- [5.3 추천 서비스](https://1ambda.gitbook.io/practical-data-pipeline/05-data-application/5.3.md)
- [5.4 A/B 테스팅](https://1ambda.gitbook.io/practical-data-pipeline/05-data-application/5.4.md)
- [Week 1 - Data Pipeline](https://1ambda.gitbook.io/practical-data-pipeline/08-case-study/week-1-data-pipeline.md)
- [Week 2 - EMR & Kubernetes](https://1ambda.gitbook.io/practical-data-pipeline/08-case-study/week-2-emr-and-kubernetes.md)
- [Week 3 - Metastore](https://1ambda.gitbook.io/practical-data-pipeline/08-case-study/week-3-metastore.md)
- [Week 4 - KV & Delta Storage](https://1ambda.gitbook.io/practical-data-pipeline/08-case-study/week-4-kv-and-delta-storage.md)
- [Week 5 - Kafka Rebalancing](https://1ambda.gitbook.io/practical-data-pipeline/08-case-study/week-5-kafka-rebalancing.md)
- [Week 6 - ML Pipeline](https://1ambda.gitbook.io/practical-data-pipeline/08-case-study/week-6-ml-pipeline.md)
- [Spark 설치 및 환경 구성](https://1ambda.gitbook.io/practical-data-pipeline/09-installation/spark.md)
- [Spark - Local Shell 환경](https://1ambda.gitbook.io/practical-data-pipeline/09-installation/spark/spark-local-shell.md)
- [Spark - Local Jupyter 환경](https://1ambda.gitbook.io/practical-data-pipeline/09-installation/spark/spark-local-jupyter.md)
- [Spark - Kubernetes 환경](https://1ambda.gitbook.io/practical-data-pipeline/09-installation/spark/spark-kubernetes.md)
- [Spark - EMR 환경](https://1ambda.gitbook.io/practical-data-pipeline/09-installation/spark/spark-emr.md)
- [Spark - Databricks 환경 (SaaS)](https://1ambda.gitbook.io/practical-data-pipeline/09-installation/spark/spark-databricks-saas.md)
- [Flink 설치 및 환경 구성](https://1ambda.gitbook.io/practical-data-pipeline/09-installation/flink.md)
- [Kafka 설치 및 환경 구성](https://1ambda.gitbook.io/practical-data-pipeline/09-installation/kafka.md)
- [MySQL 설치 및 환경 구성](https://1ambda.gitbook.io/practical-data-pipeline/09-installation/mysql.md)
- [DynamoDB 사용을 위한 환경 구성](https://1ambda.gitbook.io/practical-data-pipeline/09-installation/dynamodb.md)
- [ElasticSearch 설치 및 환경 구성](https://1ambda.gitbook.io/practical-data-pipeline/09-installation/elasticsearch.md)
- [Presto 설치 및 환경 구성](https://1ambda.gitbook.io/practical-data-pipeline/09-installation/presto.md)
- [Druid 설치 및 환경 구성](https://1ambda.gitbook.io/practical-data-pipeline/09-installation/druid.md)