Visão Geral

O Curso Data Engineering with Snowflake oferece uma formação completa para profissionais que desejam projetar, desenvolver e otimizar pipelines de dados na plataforma Snowflake Data Cloud.
Durante o treinamento, o participante aprenderá desde os conceitos fundamentais de arquitetura e modelagem de dados até a implementação prática de pipelines de ingestão, transformação, integração com ferramentas externas e otimização de performance.

O curso inclui laboratórios práticos, abordando casos reais de engenharia de dados corporativa, garantindo uma experiência imersiva e aplicável ao ambiente de produção.

Objetivo

Após realizar o Curso Data Engineering with Snowflake, você será capaz de:

• Compreender a arquitetura do Snowflake Data Cloud
• Criar, gerenciar e otimizar data pipelines e data warehouses
• Implementar estratégias de data ingestion, staging e transformation
• Aplicar técnicas de performance tuning e cost optimization
• Integrar o Snowflake com ferramentas externas de BI, ETL e Data Lakes

Publico Alvo

• Engenheiros de Dados e Analistas de Dados
• Arquitetos de Soluções e Profissionais de BI
• Desenvolvedores e Administradores que atuam com Data Warehousing
• Profissionais que desejam migrar workloads de dados para Snowflake

Pre-Requisitos

• Conhecimento básico de bancos de dados relacionais (SQL)
• Familiaridade com conceitos de Data Warehouse e ETL
• Noções de linguagem Python e conceitos de nuvem (Azure, AWS ou GCP)

Materiais

Conteúdo Programatico

Module 1: Introduction to Snowflake and Data Cloud Concepts

1. Overview of Snowflake architecture and key components
2. Virtual Warehouses, Databases, Schemas, and Stages
3. Cloud-agnostic design: AWS, Azure, GCP
4. Snowflake services layer, compute layer, and storage layer
5. Separation of compute and storage explained

Module 2: Snowflake Setup and Environment Configuration

1. Creating and managing Snowflake accounts
2. Understanding roles, users, and access controls
3. Working with the Snowflake Web UI, Snowsight, and SnowSQL
4. Configuring warehouses and resource monitors
5. Integration with cloud storage (S3, Azure Blob, GCS)

Module 3: Data Loading and Unloading

1. File staging (internal and external stages)
2. COPY INTO and PUT commands
3. Working with structured and semi-structured data (JSON, Parquet, Avro)
4. Data unloading and exporting best practices
5. Monitoring data load operations

Module 4: Data Modeling and Schema Design

1. Star and Snowflake schema design patterns
2. Fact and dimension tables
3. Time-travel and zero-copy cloning for version control
4. Data retention, cloning, and fail-safe mechanisms
5. Handling schema evolution

Module 5: Data Transformation and Pipeline Design

1. Building transformation pipelines with SQL and Snowflake Streams
2. Tasks and scheduling automation
3. Using Snowpark for Python and Java
4. Data transformation using dbt (data build tool)
5. Real-time data processing with Snowpipe

Module 6: Performance Tuning and Query Optimization

1. Query profiling using Query History and Query Profile
2. Warehouse scaling and caching strategies
3. Clustering keys and micro-partitioning
4. Query result reuse and materialized views
5. Identifying and resolving performance bottlenecks

Module 7: Security and Governance

1. Role-Based Access Control (RBAC) in Snowflake
2. Network policies and encryption at rest/in transit
3. Data masking and row access policies
4. Monitoring user activities and resource usage
5. Integration with identity providers (Azure AD, Okta, etc.)

Module 8: Cost Management and Optimization

1. Understanding Snowflake credit usage and billing
2. Query cost analysis and warehouse right-sizing
3. Auto-suspend and auto-resume features
4. Using Resource Monitors for cost governance
5. Best practices for cost-efficient pipelines

Module 9: Integrations and Ecosystem

1. Connecting Snowflake with Power BI, Tableau, and Looker
2. Integration with Apache Airflow for orchestration
3. ETL/ELT integration with Azure Data Factory, Matillion, and Informatica
4. Working with Kafka and Snowpipe for streaming ingestion
5. Exporting data to data lakes or external systems

Module 10: Hands-on Labs

1. End-to-end pipeline development in Snowflake
2. Automating data ingestion from cloud storage
3. Query performance tuning exercises
4. Implementing row-level security and masking policies
5. Cost optimization simulation in a production-like scenario

Module 11: Best Practices and Real-World Scenarios

1. Enterprise data architecture using Snowflake
2. Multi-cluster warehouses for concurrency scaling
3. Data sharing between organizations (Data Exchange)
4. Migration strategies from on-premise databases to Snowflake
5. Snowflake deployment lifecycle management

Module 12: Capstone Project

1. Design, build, and optimize a complete Snowflake data engineering solution
2. Integration with BI dashboard and external storage
3. Performance benchmarking and documentation
4. Instructor-led review and feedback session

Module 13: Advanced Topics and Future Trends

1. Working with Snowflake Native Apps and Data Marketplace
2. Streamlit and Snowpark for Data Applications
3. Using AI/ML integrations within Snowflake
4. Preview features: Dynamic tables, Unistore, and Cortex AI
5. Future of cloud-based data engineering