Skip to main content
Back to top
Ctrl
+
K
DE4E: Data Engineering for Everybody
π Check List
1. Introduction to Data Engineering
μ°λ¦¬λ λΉ λ°μ΄ν° μλμ μ΄κ³ μμ΅λλ€
1.1. λ°μ΄ν° μμ§λμ΄λ§μ΄λ 무μμΌκΉ?
1.2. μ¬λ‘λ₯Ό ν΅ν΄ μμ보λ λ°μ΄ν° μμ§λμ΄λ§
1.3. λ°μ΄ν° μμ§λμ΄λ μ΄λ»κ² μ§ννκ³ μμκΉ?
1.4. λ°μ΄ν° μμ§λμ΄λ μ΄λ€ κ³ λ―Όμ ν κΉ?
Recap
ποΈΒ μ°Έκ³ μλ£(Reference)
2. Data Source and Data Collection
2.1 λ°μ΄ν° μμ§λμ΄κ° λ€λ£¨λ λ°μ΄ν°
2.2 λ°μ΄ν°λ₯Ό μμ§νλ λ°©λ²
2.3 λ°μ΄ν° μμ§μμμ κ³ λ €ν μ
2.4 λ°μ΄ν° μ²λ¦¬λ°©μ μμ보기
2.5 π» Index μ€μ΅
ποΈΒ μ°Έκ³ μλ£(Reference)
3. Data Transformation and Cleaning
(μμ λͺ©)
4. Data Storage
4.1 data warehouse / data lake / data lakehouse
4.2 Data Warehouse
4.3 Data Mart
4.4 Data Lake
4.5 Data Lakehouse
4.6 π» Data Pipeline μ€μ΅
4.7 Databases
ποΈΒ μ°Έκ³ μλ£(Reference)
5. Data Processing Frameworks I
5.1 Data Processingμ μν κΈ°μ΄ μ§μ
5.2 λκ·λͺ¨ λ°μ΄ν° μ²λ¦¬ νμμ±κ³Ό νλ μμν¬
5.3 λΆμ°μ²λ¦¬μ λ³λ ¬μ²λ¦¬
5.4 μΊμ±(Cache + ing)
5.5 μΈλ±μ±
5.6 λ©λͺ¨λ¦¬ μ΅μ ν
5.7 μΈλ©λͺ¨λ¦¬ λλΉλ?
5.8 λΆμ° μμ€ν
5.9 Hadoop
5.10 Hadoop Ecosystem
5.11 HDFS
5.12 MapReduce
π» MapReduce μ€μ΅
ποΈΒ μ°Έκ³ μλ£(Reference)
6. Data Processing Frameworks II
6.1 Stream Processing Framework
6.2 μνμΉ μ€νν¬(Apache Spark)
π» μ€μ΅: μνμΉ μ€νν¬
6.3 μνμΉ μΉ΄νμΉ΄ (Apache Kafka)
6.4 μνμΉ μ€ν° (Apache Storm)
6.5 μνμΉ νλ§ν¬ (Apache Flink)
6.6 λ°μ΄ν° μ²λ¦¬λ°©λ² λΉκ΅
ποΈΒ μ°Έκ³ μλ£(Reference)
7. Introduction to Apache Airflow
7.1 Introduction to Apache Airflow
7.2 Airflow Executor
7.3 Airflow Use Cases
7.4 Airflow Hands-On
π» μ€μ΅: Airflow
ποΈΒ μ°Έκ³ μλ£(Reference)
8. Cloud Computing and Data Enginnering
8.1 Cloud Computing
8.2 Data Engineering
8.3 ν΄λΌμ°λ μ»΄ν¨ν μμμ λ°μ΄ν° μμ§λμ΄λ§κ³Ό νλ‘μΈμ€λ€
8.4.
μ¬λ‘ νμ΅ - ν΄λΌμ°λ μ»΄ν¨ν μμμ λ°μ΄ν° μμ§λμ΄λ§
8.5.
λ©ν° ν΄λΌμ°λ νκ²½μμμ λ°μ΄ν° μμ§λμ΄λ§
ποΈΒ μ°Έκ³ μλ£(Reference)
π Project - MovieFlix
1. μν€ν μ² μ€λͺ
2. Infra
3. λ°μ΄ν° μμ±
4. Data Flow
5. Monitoring
ποΈΒ μ°Έκ³ μλ£(Reference)
Repository
Open issue
.md
.pdf
4.1 data warehouse / data lake / data lakehouse
4.1 data warehouse / data lake / data lakehouse
#
