Apache Airflow for Data Science: Automating Machine Learning Pipelines Training Course
Apache Airflow serves as an open-source platform designed for orchestrating workflows and automating complex data pipelines.
This instructor-led, live training (available online or onsite) targets intermediate-level professionals eager to automate and manage machine learning workflows. The curriculum covers model training, validation, and deployment using Apache Airflow.
Upon completion of this training, participants will be equipped to:
- Configure Apache Airflow to orchestrate machine learning workflows.
- Automate tasks such as data preprocessing, model training, and validation.
- Integrate Airflow with various machine learning frameworks and tools.
- Deploy machine learning models through automated pipelines.
- Monitor and optimize machine learning workflows within production environments.
Course Format
- Interactive lectures and group discussions.
- Numerous exercises and practical practice sessions.
- Hands-on implementation within a live-lab environment.
Course Customization Options
- For those seeking tailored training for this course, please reach out to us to arrange your specific requirements.
Course Outline
Introduction to Apache Airflow for Machine Learning
- Overview of Apache Airflow and its relevance to data science
- Key features for automating machine learning workflows
- Setting up Airflow for data science projects
Building Machine Learning Pipelines with Airflow
- Designing DAGs for end-to-end ML workflows
- Using operators for data ingestion, preprocessing, and feature engineering
- Scheduling and managing pipeline dependencies
Model Training and Validation
- Automating model training tasks with Airflow
- Integrating Airflow with ML frameworks (e.g., TensorFlow, PyTorch)
- Validating models and storing evaluation metrics
Model Deployment and Monitoring
- Deploying machine learning models using automated pipelines
- Monitoring deployed models with Airflow tasks
- Handling retraining and model updates
Advanced Customization and Integration
- Developing custom operators for ML-specific tasks
- Integrating Airflow with cloud platforms and ML services
- Extending Airflow workflows with plugins and sensors
Optimizing and Scaling ML Pipelines
- Improving workflow performance for large-scale data
- Scaling Airflow deployments with Celery and Kubernetes
- Best practices for production-grade ML workflows
Case Studies and Practical Applications
- Real-world examples of ML automation using Airflow
- Hands-on exercise: Building an end-to-end ML pipeline
- Discussion of challenges and solutions in ML workflow management
Summary and Next Steps
Requirements
- Familiarity with machine learning workflows and core concepts
- Basic understanding of Apache Airflow, including DAGs and operators
- Proficiency in Python programming
Target Audience
- Data scientists
- Machine learning engineers
- AI developers
Need help picking the right course?
southafrica@nobleprog.co.za or +27 (0)10 005 5793
Apache Airflow for Data Science: Automating Machine Learning Pipelines Training Course - Enquiry
Related Courses
AdaBoost Python for Machine Learning
14 HoursThis instructor-led live training in Kenya (online or onsite) is intended for data scientists and software engineers who wish to employ AdaBoost to develop boosting algorithms for machine learning using Python.
By the end of this training, participants will be able to:
- Set up the necessary development environment to start building machine learning models with AdaBoost.
- Understand the ensemble learning approach and how to implement adaptive boosting.
- Learn how to build AdaBoost models to boost machine learning algorithms in Python.
- Use hyperparameter tuning to increase the accuracy and performance of AdaBoost models.
AlphaFold: AI-Driven Protein Structure Prediction and Interpretation
7 HoursThis instructor-led live training in Kenya (online or onsite) is aimed at biologists who wish to understand how AlphaFold works and use AlphaFold models as guides in their experimental studies.
By the end of this training, participants will be able to:
- Understand the basic principles of AlphaFold.
- Learn how AlphaFold works.
- Learn how to interpret AlphaFold predictions and results.
Anaconda Ecosystem for Data Scientists
14 HoursThis instructor-led live training in Kenya (online or onsite) is designed for data scientists who wish to utilize the Anaconda ecosystem to capture, manage, and deploy packages and data analysis workflows on a single platform.
By the end of this training, participants will be able to:
- Install and configure Anaconda components and libraries.
- Understand the core concepts, features, and benefits of Anaconda.
- Manage packages, environments, and channels using Anaconda Navigator.
- Use Conda, R, and Python packages for data science and machine learning.
- Get to know some practical use cases and techniques for managing multiple data environments.
Creating Custom Chatbots with Google AutoML
14 HoursThis instructor-led live training in Kenya (online or on-site) is designed for participants with varying levels of expertise who aim to leverage Google's AutoML platform to construct customized chatbots for diverse applications.
By the conclusion of this training, participants will be able to:
- Understand the fundamentals of chatbot development.
- Navigate the Google Cloud Platform and access AutoML.
- Prepare data for training chatbot models.
- Train and evaluate custom chatbot models using AutoML.
- Deploy and integrate chatbots into various platforms and channels.
- Monitor and optimize chatbot performance over time.
Pattern Recognition
21 HoursThis instructor-led, live training in Kenya (online or onsite) offers an introduction to the fields of pattern recognition and machine learning. It covers practical applications in statistics, computer science, signal processing, computer vision, data mining, and bioinformatics.
By the end of this training, participants will be able to:
- Apply core statistical methods to pattern recognition.
- Use key models like neural networks and kernel methods for data analysis.
- Implement advanced techniques for complex problem-solving.
- Improve prediction accuracy by combining different models.
DataRobot
7 HoursThis instructor-led live training in Kenya (online or onsite) targets data scientists and analysts who want to automate, evaluate, and manage predictive models using DataRobot's machine learning capabilities.
By the end of this training, participants will be able to:
- Load datasets in DataRobot to analyze, assess, and quality check data.
- Build and train models to identify important variables and meet prediction targets.
- Interpret models to create valuable insights that are useful in making business decisions.
- Monitor and manage models to maintain an optimized prediction performance.
Edge AI with TensorFlow Lite
14 HoursThis instructor-led live training in Kenya (online or onsite) is designed for intermediate developers, data scientists, and AI practitioners aiming to harness TensorFlow Lite for Edge AI solutions.
By the conclusion of this training, participants will be able to:
- Comprehend the basics of TensorFlow Lite and its significance in Edge AI.
- Construct and optimize AI models using TensorFlow Lite.
- Deploy TensorFlow Lite models on diverse edge devices.
- Make use of tools and techniques for converting and optimizing models.
- Execute practical Edge AI applications with TensorFlow Lite.
Google Cloud AutoML
7 HoursThis instructor-led live training in Kenya (online or onsite) is designed for data scientists, data analysts, and developers who want to explore AutoML products and features to create and deploy custom ML training models with minimal effort.
By the end of this training, participants will be able to:
- Navigate the AutoML product suite to implement diverse services for various data types.
- Prepare and annotate datasets to generate custom ML models.
- Train and oversee models to ensure they yield accurate and fair machine learning outcomes.
- Leverage trained models for making predictions that address business goals and needs.
Kaggle
14 HoursThis instructor-led live training in Kenya (online or onsite) is aimed at data scientists and developers who wish to learn and build their careers in Data Science using Kaggle.
By the end of this training, participants will be able to:
- Learn about data science and machine learning.
- Explore data analytics.
- Learn about Kaggle and how it works.
Kubeflow Essentials: Build, Train & Serve with Kubernetes
14 HoursKubeflow is an open-source platform engineered to simplify the construction, training, and deployment of machine learning workloads on Kubernetes.
This instructor-led, live training (available online or on-site) is tailored for beginner to intermediate-level professionals aiming to establish reliable ML workflows using Kubeflow.
Upon completing this training, participants will acquire the following skills:
- Navigating the Kubeflow ecosystem and its core components.
- Creating reproducible workflows via Kubeflow Pipelines.
- Executing scalable training jobs on Kubernetes.
- Efficiently serving machine learning models using Kubeflow Serving.
Course Format
- Guided presentations and collaborative discussions.
- Hands-on labs involving real Kubeflow components.
- Practical exercises to construct end-to-end ML workflows.
Course Customization Options
- Customized versions of this training can be organized to align with your team’s technology stack and project requirements.
Kubeflow Fundamentals
28 HoursThis instructor-led live training in Kenya (online or onsite) targets developers and data scientists who wish to build, deploy, and manage machine learning workflows on Kubernetes.
By the end of this training, participants will be able to:
- Install and configure Kubeflow on-premises and in the cloud.
- Build, deploy, and manage ML workflows based on Docker containers and Kubernetes.
- Run complete machine learning pipelines across diverse architectures and cloud environments.
- Use Kubeflow to create and manage Jupyter notebooks.
- Develop ML training, hyperparameter tuning, and serving workloads across multiple platforms.
Machine Learning for Mobile Apps using Google’s ML Kit
14 HoursThis instructor-led live training (online or on-site) is designed for developers who intend to utilize Google’s ML Kit to build machine learning models optimized for mobile device processing.
By the conclusion of this training, participants will be able to:
- Establish the necessary development environment to initiate the creation of machine learning features for mobile apps.
- Incorporate new machine learning technologies into Android and iOS applications using ML Kit APIs.
- Enhance and optimize existing apps by leveraging the ML Kit SDK for on-device processing and deployment.
Machine Learning with Random Forest
14 HoursThis instructor-led live training in Kenya (available online or onsite) is tailored for data scientists and software engineers who aim to utilize Random Forest to build machine learning algorithms for extensive datasets.
By the conclusion of this training, participants will be able to:
- Establish the necessary development environment to commence building machine learning models with Random Forest.
- Comprehend the advantages of Random Forest and learn how to implement it to address classification and regression issues.
- Master the techniques for handling large datasets and interpreting multiple decision trees within Random Forest.
- Evaluate and optimize machine learning model performance through hyperparameter tuning.
Advanced Analytics with RapidMiner
14 HoursThis instructor-led live training in Kenya (online or onsite) targets intermediate-level data analysts who wish to learn how to use RapidMiner for estimating and projecting values and utilizing analytical tools for time series forecasting.
By the end of this training, participants will be able to:
- Apply the CRISP-DM methodology, select appropriate machine learning algorithms, and enhance model construction and performance.
- Use RapidMiner to estimate and project values, and utilize analytical tools for time series forecasting.
GPU Data Science with NVIDIA RAPIDS
14 HoursThis instructor-led, live training in Kenya (online or onsite) targets data scientists and developers who want to use RAPIDS to create GPU-accelerated data pipelines, workflows, and visualizations, while applying machine learning algorithms like XGBoost and cuML.
By the end of this training, participants will be able to:
- Set up the necessary development environment to build data models with NVIDIA RAPIDS.
- Understand the features, components, and advantages of RAPIDS.
- Leverage GPUs to accelerate end-to-end data and analytics pipelines.
- Implement GPU-accelerated data preparation and ETL with cuDF and Apache Arrow.
- Learn how to perform machine learning tasks with XGBoost and cuML algorithms.
- Build data visualizations and execute graph analysis with cuXfilter and cuGraph.
