Visão Geral

Curso DataOps for IT Professionals. Este curso de formação DataOps para Profissionais de TI ensina aos participantes como elevar a qualidade de seus dados, aumentando a eficácia do trabalho analítico baseado nesses dados que suportam as decisões organizacionais. Os participantes aprendem como incorporar planos práticos e assistência técnica ao longo de todo o ciclo de vida dos dados, incluindo aquisição, armazenamento, processamento e consumo de dados.

Objetivo

Após realizar este Curso DataOps for IT Professionals você será capaz de:

• Entenda os desafios do processamento de dados corporativos e dos sistemas de TI
• Corrigir dados de entrada "ruins"
• Execute a limpeza de dados
• Lidar com dados ausentes e duplicados
• Aplicar consistência de dados
• Implementar governança de dados

Pre-Requisitos

• Experiência prática de trabalho em processamento de dados.

Conteúdo Programatico

DataOps Introduction

1. DataOps Enterprise Data Technologies
2. Enterprise Data Processing Challenges and IT Systems' Woes:
   1. Data Quality
   2. What Makes Information Systems Cluttered and Myopic
   3. Fragmented Data Sources
   4. Different Data Formats
   5. System Interoperability
   6. Maintenance Issues
3. Data-Related Roles
4. Data Engineering
5. What is DataOps?
6. The DataOps Technology and Methodology Stack
7. The DataOps Manifesto
8. Agile Development
9. DevOps
10. The Lean Manufacturing Methodology
11. Key Components of a DataOps Platform
12. Overview of DataOps Tools and Services
13. Overview of DataOps Platforms

Data Quality

1. Data Quality Definitions
2. Dimensions of Data Quality
3. Defining "Bad" Data
   1. Missing Data
   2. Wrong/Incorrect Data or Data Format
   3. Inconsistent Data
   4. Outdated (Stale) Information
   5. Unverifiable Data
   6. Withheld Data
4. Common Causes for “Bad" Data
   1. Human Factor
   2. Infrastructure- and Network-Related Issues
   3. Software Defects
   4. Using the Wrong Tool for the Job
   5. Using Untrusted Data
   6. Aggregation of Data from Disparate Data Sources that have Impedance Mismatch
   7. Wrong QoS Settings of Queueing Systems
   8. Wrong Caching System Settings, e.g. TTL
   9. Not Using the "Ground Truth" Data
   10. Differently Configured Development/UAT/Production Systems
   11. Confusing Big-Endian and Little-Endian Byte Order
5. Ensuring Data Quality
   1. Ensuring Integrity of Datasets 
6. Dealing with "Bad" Input Data
   1. DDL-enforced Schema & Schema-on-Demand (-on-Read)
   2. SQL Constraints as Rules for Column-Level and Table-Wide Data
   3. XML Schema Definition (XSD) for XML Documents
   4. Validating JSON Documents
   5. Regular Expressions
   6. Data Cleansing of Data at Rest
   7. Controlling Integrity of Data-in-Transit
   8. Database Normalization
   9. Using Assertions in Applications
   10. Operationalizing Input Data Validation
7. Data Consistency and Availability
8. Dealing with Duplicate Data
9. Dealing with Missing (NaN) Data
10. Master (Authoritative) Data Management
11. Enforcing Data Consistency with the scikit-learn LabelEncoder Class
12. Data Provenance
13. The Event Sourcing Pattern
14. Adopting the Culture of Automation
15. On-going Auditing
16. Monitoring and Alerting
17. UiPath
18.  Workflow (Pipeline) Orchestration Systems

How to Lead with Data

1. Enterprise Architecture Components
   1. Business Architecture
   2. Information Architecture
   3. Application Architecture
   4. Technology Architecture
2. DataOps Functional Architecture
3. The Snowflake Data Cloud
4. Cloud Design for System Resiliency
5. New Data Architecture:
   1. Data Ownership
   2. Shared Environment Security Controls

Data Governance (Optional)

1. The Need for Data Governance
2. Controlling the Decision-Making Process
3. Controlling "Agile IT"
4. Types of Requirements
   1. Product
   2. Process
5. Scoping Requirements
6. Governance Gotchas
7. Governance Best Practices