Visão Geral

Este Curso Distributed Task Automation with Python Celery, apresenta uma abordagem completa para automação de tarefas distribuídas utilizando Python e o framework Celery. O participante aprenderá desde os conceitos fundamentais de sistemas distribuídos, mensageria e brokers, até a implementação prática de ambientes containerizados com Docker, integração com bancos de dados PostgreSQL, uso do RabbitMQ como message broker e orquestração de tarefas assíncronas em escala. O curso cobre tanto fundamentos quanto tópicos avançados de Celery, incluindo monitoramento, escalabilidade, alta disponibilidade e boas práticas para ambientes de produção.

Objetivo

Após realizar este curso Distributed Task Automation with Python Celery, você será capaz de:

• Projetar arquiteturas de automação de tarefas distribuídas
• Compreender o funcionamento de message brokers e sistemas de mensageria
• Configurar e utilizar RabbitMQ e PostgreSQL em ambientes containerizados
• Desenvolver aplicações Python assíncronas utilizando Celery
• Gerenciar filas, workers e resultados de tarefas
• Implementar tarefas escaláveis, tolerantes a falhas e resilientes
• Orquestrar ambientes complexos utilizando Docker e Docker Compose
• Monitorar, ajustar desempenho e preparar aplicações Celery para produção

Publico Alvo

• Desenvolvedores Python
• Engenheiros de Software
• Engenheiros DevOps
• Arquitetos de Soluções
• Profissionais de TI que trabalham com sistemas distribuídos
• Analistas e engenheiros de dados
• Estudantes de tecnologia com foco em backend e cloud

Pre-Requisitos

• Conhecimentos básicos de Python
• Noções de sistemas operacionais Linux
• Conceitos básicos de redes e arquitetura cliente-servidor
• Noções básicas de bancos de dados relacionais
• Familiaridade básica com linha de comando

Materiais

Conteúdo Programatico

Module 1: Overview of Distributed Task Automation

1. What is Distributed Task Automation
2. Use cases and real-world scenarios
3. Synchronous vs asynchronous task execution

Module 2: Celery Fundamentals

1. Overview of Celery and its Features
2. Celery architecture and components
3. Celery compared to Faust
4. When to use Celery

Module 3: Message Broker Concepts

1. What is Message Broker
2. Messaging patterns
3. Reliability, durability, and acknowledgments

Module 4: RabbitMQ Deep Dive

1. What is RabbitMQ
2. Exchanges, queues, and bindings
3. RabbitMQ + PostgreSQL compared Kafka + Zookeeper
4. Choosing the right broker for distributed tasks

Module 5: Development Environment Setup

1. Configure Visual Studio Code for Python Script Programming
2. Python Code Linting & Reformatting with Ruff & MyPy
3. Debugging Python Scripts with Visual Studio Code

Module 6: Containerization Fundamentals

1. What is a Container
2. What is Docker
3. What is Docker Hub
4. Images and Containers

Module 7: Docker in Practice

1. Create an Image with Dockerfile
2. Run Containers
3. Configure Containers with Environment Variables

Module 8: Docker Compose Essentials

1. What is Docker Compose
2. Docker Compose Networking
3. Docker Compose Volume

Module 9: Remote Data Storage Concepts

1. What is Remote Data Storage
2. Data persistence strategies

Module 10: PostgreSQL with Docker

1. Running a Data Storage in a Container
2. Running PostgreSQL in a Container
3. Running PostgreSQL Client Tool in a Container
4. Configure with Docker Compose

Module 11: Python and PostgreSQL Integration

1. Read/Write Data to PostgreSQL with Python SQLAlchemy
2. Managing sessions and transactions
3. Error handling and migrations

Module 12: Remote Message Broker Implementation

1. Running a Message Broker in a Container
2. Messaging service design considerations

Module 13: RabbitMQ with Docker

1. Running RabbitMQ in a Container
2. Running RabbitMQ Client Tool in a Container
3. Configure with Docker Compose

Module 14: Python and RabbitMQ Integration

1. Read/Write Data to RabbitMQ with Python and Pika
2. Message acknowledgment and durability
3. Error handling in message publishing

Module 15: Celery Basics

1. Installing Celery and its dependencies
2. Setting up a simple Celery project
3. Defining and running tasks
4. Task decorators and options
5. Passing arguments and results between tasks
6. Task retries and error handling
7. Monitoring and managing Celery workers

Module 16: Advanced Celery Concepts

1. Task serialization and message brokers
2. Task routing and prioritization
3. Task result backends
4. Grouping and chaining tasks
5. Scheduling periodic tasks with Celery beat

Module 17: Scaling and Deployment

1. Load balancing tasks with multiple workers
2. Deploying Celery in a production environment
3. Configuring Celery for high availability
4. Best practices for handling long-running tasks

Module 18: Monitoring and Performance Tuning

1. Monitoring Celery workers
2. Performance bottlenecks
3. Logging and observability
4. Tuning brokers and workers

Module 19: Course Wrap-up

1. Summary of Key Concepts
2. Common pitfalls and best practices
3. Q&A
4. Further Resources and Next Steps