Confluent Apache Kafka: Cluster Operations and Configuration Training Course

Module 1: Introduction to Confluent Apache Kafka Architecture and Cluster Configuration

• Overview of Apache Kafka and the Confluent Platform.
• Core concepts: Kafka brokers, producers, consumers, and topics.
• The function of Kafka in data streaming and messaging pipelines.
• Confluent Platform extensions: REST Proxy, Schema Registry, and Control Center.
• Kafka cluster architecture: understanding brokers, partitions, and replication.

Module 2: Zookeeper Quorum Configuration

• Introduction to Zookeeper.
• The specific role Zookeeper plays within a Kafka cluster.
• Determining appropriate Zookeeper quorum sizes.
• Configuring Zookeeper effectively.
• Implementing Secure Shell (SSH) on servers.
• Hands-on: Configuring Zookeeper (as a team and as a service).
• Utilizing the Zookeeper Command Line Interface (CLI).
• Hands-on: Setting up the Zookeeper quorum.
• Understanding the Zookeeper internal file system.
• Key performance factors impacting Zookeeper.
• Demo: Exploring Zookeeper management tools and Zoonavigator.

Module 3: Kafka Cluster Configuration

• Foundations of Kafka operation.
• Essential Kafka configuration settings.
• Hands-on: Configuring Kafka on a single broker.
• Hands-on: Executing core Kafka commands.
• Hands-on: Establishing a multi-broker Kafka cluster.
• Hands-on: Testing the Kafka cluster.
• Verifying connectivity to the Kafka cluster.
• Configuring Advertised.listeners: key settings explained.
• Topic configuration best practices.
• Setting up message ingestion and delivery across topics.
• Hands-on: Demonstrating Kafka’s resilience capabilities.
• Kafka performance considerations: I/O, network, RAM, CPU, and OS.
• Additional factors influencing Kafka performance.
• Hands-on: Modifying broker configurations as needed.

Module 4: Advanced Kafka Configuration

• Configuring Landoop Kafka Topics UI, Confluent REST Proxy, and Confluent Schema Registry.
• Sending and receiving messages via CLI, Java, and Spring Framework.
• Utilizing metrics and monitoring tools (e.g., Confluent Control Center, Elasticsearch).
• Managing log files and offsets.
• Strategies for high availability and disaster recovery.
• Ensuring high availability through replication.
• Performance tuning for producers and consumers.
• Developing disaster recovery strategies.
• Managing failover control and data recovery.
• Setting up connectors.
• Implementing Kafka Connect.
• Exploring Kafka security features.

Summary and Next Steps