AI Workloads on Kubernetes: Deploying Machine Learning Models at Scale Training Course
Kubernetes serves as a robust, scalable platform for deploying, serving, and managing machine learning models within production settings.
This instructor-led live training, available both online and onsite, targets intermediate-level professionals looking to reliably operationalize their ML models on Kubernetes.
Upon completion of this training, participants will be equipped with the skills to:
- Containerize and ready ML models for efficient deployment.
- Serve models utilizing contemporary inference frameworks.
- Optimize workloads through autoscaling, GPU configuration, and resource tuning.
- Execute model rollout strategies, including A/B testing and canary deployments.
Course Format
- A mix of lectures, architectural analysis, and guided discussions.
- Extensive practical exercises featuring real-world deployment scenarios.
- Hands-on implementation within a live Kubernetes environment.
Customization Options
- Should you require this course tailored to your specific environment or toolchain, please reach out to us to explore customization possibilities.
Course Outline
Preparing Machine Learning Models for Deployment
- Packaging models using Docker.
- Exporting models from TensorFlow and PyTorch.
- Considerations for versioning and storage.
Model Serving on Kubernetes
- Overview of inference servers.
- Deploying TensorFlow Serving and TorchServe.
- Setting up model endpoints.
Inference Optimization Techniques
- Batching strategies.
- Handling concurrent requests.
- Tuning latency and throughput.
Autoscaling ML Workloads
- Horizontal Pod Autoscaler (HPA).
- Vertical Pod Autoscaler (VPA).
- Kubernetes Event-Driven Autoscaling (KEDA).
GPU Provisioning and Resource Management
- Configuring GPU nodes.
- Overview of the NVIDIA device plugin.
- Setting resource requests and limits for ML workloads.
Model Rollout and Release Strategies
- Blue/green deployments.
- Canary rollout patterns.
- A/B testing for model evaluation.
Monitoring and Observability for ML in Production
- Metrics for inference workloads.
- Logging and tracing practices.
- Dashboards and alerting mechanisms.
Security and Reliability Considerations
- Securing model endpoints.
- Network policies and access control.
- Ensuring high availability.
Summary and Next Steps
Requirements
- Familiarity with containerized application workflows.
- Experience with Python-based machine learning models.
- Understanding of Kubernetes fundamentals.
Target Audience
- ML Engineers.
- DevOps Engineers.
- Platform Engineering Teams.
Need help picking the right course?
southafrica@nobleprog.co.za or +27 (0)10 005 5793
AI Workloads on Kubernetes: Deploying Machine Learning Models at Scale Training Course - Enquiry
Testimonials (2)
About the microservices and how to maintenance kubernetes
Yufri Isnaini Rochmat Maulana - Bank Indonesia
Course - Advanced Platform Engineering: Scaling with Microservices and Kubernetes
The training met expectations with its clear explanations, real-world examples, and hands-on labs that made complex topics easy to understand. It provided valuable insights into container orchestration, security, scaling and many other advanced topics.
Anna Wyszomirska-Szmyd - Akamai
Course - Docker and Kubernetes advanced
Related Courses
Advanced Platform Engineering: Scaling with Microservices and Kubernetes
35 HoursThis instructor-led, live training in Kenya (online or onsite) is aimed at advanced-level platform engineers and DevOps professionals who wish to master scaling applications using microservices and Kubernetes.
By the end of this training, participants will be able to:
- Design and implement scalable microservices architectures.
- Deploy and manage applications on Kubernetes clusters.
- Utilize Helm charts for efficient service deployment.
- Monitor and maintain the health of microservices in production.
- Apply best practices for security and compliance in a Kubernetes environment.
AI-Driven Kubernetes Operations: Autoscaling, Scheduling & Resource Optimization
21 HoursKubernetes is a container orchestration platform widely used for managing distributed applications at scale.
This instructor-led, live training (online or onsite) is aimed at advanced-level practitioners who wish to apply AI and machine learning techniques to optimize Kubernetes resource usage, scheduling decisions, and autoscaling strategies.
Upon completion of this programme, participants will be able to:
- Apply AI/ML models to improve workload scheduling decisions in Kubernetes.
- Use predictive analytics to optimize CPU, GPU, and memory allocation.
- Implement intelligent autoscaling using reinforcement learning and metric forecasting.
- Reduce infrastructure cost and latency through automated resource optimization.
Format of the Course
- Instructor-guided technical presentations and deep-dive discussions.
- Hands-on lab work using real Kubernetes clusters.
- Practical exercises applying AI models to real operational scenarios.
Course Customization Options
- To tailor this course to your platform setup or operational requirements, please contact us for customization.
Certified Kubernetes Administrator (CKA) - exam preparation
21 HoursThe Certified Kubernetes Administrator (CKA) certification was established by The Linux Foundation in collaboration with the Cloud Native Computing Foundation (CNCF).
Today, Kubernetes stands as a premier platform for container orchestration.
NobleProg has been delivering Docker and Kubernetes training since 2015. With over 360 successfully completed training projects, we have become one of the most recognized training providers globally in the field of containerization.
Since 2019, we have also been assisting our customers in validating their performance in Kubernetes environments by preparing them and encouraging them to take the CKA and CKAD exams.
This instructor-led live training (available online or onsite) is designed for System Administrators and Kubernetes users who wish to validate their knowledge by passing the CKA exam.
Additionally, the training emphasizes gaining practical experience in Kubernetes Administration; therefore, we recommend participating even if you do not plan to take the CKA exam.
Course Format
- Interactive lectures and discussions.
- Ample exercises and practice sessions.
- Hands-on implementation in a live-lab environment.
Course Customization Options
- To request customized training for this course, please contact us to arrange it.
- For more information about CKA certification, please visit: https://training.linuxfoundation.org/certification/certified-kubernetes-administrator-cka
Certified Kubernetes Application Developer (CKAD) - exam preparation
21 HoursThe Certified Kubernetes Application Developer (CKAD) programme was developed by The Linux Foundation and the Cloud Native Computing Foundation (CNCF), the host organisation of Kubernetes.
This instructor-led, live training (available online or onsite) is designed for Developers who wish to validate their skills in designing, building, configuring, and exposing cloud native applications on Kubernetes.
Furthermore, the training emphasises gaining practical experience in Kubernetes application development. Therefore, we recommend participating even if you do not intend to sit for the CKAD exam.
NobleProg has been delivering Docker & Kubernetes training since 2015. With over 360 successfully completed training projects, we have become one of the most recognised training companies globally in the field of containerisation. Since 2019, we have also assisted our customers in validating their performance in k8s environments by preparing them and encouraging them to pass the CKA and CKAD exams.
Format of the Course
- Interactive lectures and discussions.
- Extensive exercises and practice.
- Hands-on implementation in a live-lab environment.
Course Customization Options
- To request customized training for this course, please contact us to arrange.
- To learn more about CKAD, please visit: https://training.linuxfoundation.org/certification/certified-kubernetes-application-developer-ckad/
Certified Kubernetes Security Specialist (CKS)
21 HoursThis instructor-led, live training in Kenya (online or on-site) is designed for Kubernetes practitioners who wish to prepare for the CKS exam.
By the end of this training, participants will understand how to secure Kubernetes environments and container-based applications throughout the various stages of an application's lifecycle: build, deployment, and runtime.
Cloud‑Native DevOps Mastery: Designing, Deploying & Operating Scalable Kubernetes Microservices
49 HoursThis intensive 7-day programme offers a practical, immersive experience in designing, deploying, and running cloud-native applications through contemporary DevOps methodologies.
Participants will delve into the creation of scalable microservices architectures, the optimization of containerized environments, and the effective management of production workloads via Kubernetes. The curriculum encompasses advanced deployment tactics, GitOps-driven automation, and observability standards designed to guarantee system reliability and peak performance.
Significant emphasis is placed on navigating real-world operational hurdles, such as incident response, failure simulation, and root cause analysis. The programme culminates in the application of AI-enhanced tools to streamline troubleshooting and expedite operational decision-making.
Upon completion of this training, participants will possess a robust understanding of how to construct, deploy, monitor, and sustain resilient distributed systems within a Kubernetes ecosystem.
Introduction to Containers, Kubernetes & OpenShift
35 HoursLearn the core principles of containers, Kubernetes, and OpenShift through a practical, hands-on training program designed specifically for developers, DevOps engineers, and IT professionals. Participants will acquire skills to build containerized applications, deploy workloads, manage Kubernetes resources, and utilize OpenShift to streamline modern application delivery in cloud and hybrid environments.
Docker, Kubernetes and OpenShift 3 for Administrators
35 HoursIn this instructor-led, live training in Kenya, participants will learn how to manage Red Hat OpenShift Container Platform.
By the end of this training, participants will be able to:
- Create, configure, manage, and troubleshoot OpenShift clusters.
- Deploy containerized applications on-premise, in public cloud or on a hosted cloud.
- Secure OpenShift Container Platform
- Monitor and gather metrics.
- Manage storage.
Docker and Kubernetes: Building and Scaling a Containerized Application
21 HoursIn this instructor-led live training in Kenya (onsite or remote), participants will learn how to create and manage Docker containers before deploying a sample application inside one. Attendees will also discover how to automate, scale, and manage their containerized applications within a Kubernetes cluster. The training progresses to more advanced topics, guiding participants through the processes of securing, scaling, and monitoring a Kubernetes cluster.
By the end of this training, participants will be able to:
- Set up and run a Docker container.
- Deploy a containerized server and web application.
- Build and manage Docker images.
- Set up a Docker and Kubernetes cluster.
- Use Kubernetes to deploy and manage a clustered web application.
- Secure, scale and monitor a Kubernetes cluster.
Docker and Kubernetes
21 HoursCourse Objectives: Gain theoretical and practical skills in Docker and Kubernetes.
Docker and Kubernetes advanced
35 HoursThe 'Advanced Docker and Kubernetes' course is a comprehensive training covering the Kubernetes platform and its ecosystem. It is addressed to participants with varied backgrounds, ensuring a complete scope of both basic and advanced Kubernetes features. It includes key concepts such as Pods, Labels, Controllers, Services, Secrets, Persistent Data Volumes, Claims, Namespaces, Quotas, Container Networking Model, Service Discovery, Scaling, Load Balancing, Cluster Management, Kubernetes Installation, Cluster Security, Access Control, Controller High Availability, Monitoring and Logging, Autoscaling, Advanced Scheduling, Microservices-based Applications, Application Design Patterns, and Deploying Applications on a Kubernetes Cluster.
Edge AI on Kubernetes: Deploying Intelligent Applications at the Edge
21 HoursEdge AI represents a paradigm centered on executing machine learning inference close to data sources to deliver low-latency, efficient, and scalable processing.
This instructor-led live training, available both online and onsite, is designed for intermediate to advanced practitioners looking to deploy, orchestrate, and optimize AI workloads within Kubernetes-based edge environments.
Upon completion of this course, participants will be able to:
- Configure lightweight Kubernetes distributions suitable for edge deployments.
- Effectively deploy AI inference workloads across constrained edge nodes.
- Address connectivity challenges and implement synchronization patterns.
- Optimize performance, storage, and networking for real-world edge scenarios.
Format of the Course
- Guided presentations enriched with real-world examples.
- Scenario-based labs and practical edge deployment exercises.
- Hands-on experience with Kubernetes edge frameworks.
Course Customization Options
- To request customized training tailored to your specific edge platform requirements, please contact us to arrange.
Deploying Kubernetes Applications with Helm
7 HoursThis instructor-led, live training in Kenya (online or onsite) is aimed at engineers who wish to use Helm to streamline the process of installing and managing Kubernetes applications.
By the end of this training, participants will be able to:
- Install and configure Helm.
- Create reproducible builds of Kubernetes applications.
- Share applications as Helm charts.
- Run third-party applications saved as Helm charts.
- Manage releases of Helm packages.
Introduction to Minikube and Kubernetes
21 HoursThis guided, live training in Kenya (online or in-person) is designed for beginner to intermediate-level software developers and DevOps professionals who wish to learn how to set up and manage a local Kubernetes environment using Minikube.
By the end of this training, participants will be able to:
- Install and configure Minikube on their local machine.
- Understand the basic concepts and architecture of Kubernetes.
- Deploy and manage containers using kubectl and the Minikube dashboard.
- Set up persistent storage and networking solutions for Kubernetes.
- Utilize Minikube for developing, testing, and debugging applications.
Minikube for Developers
14 HoursThis instructor-led, live training in Kenya (online or onsite) is aimed at intermediate-level developers and DevOps engineers who wish to use Minikube as a part of their development workflow.
By the end of this training, participants will be able to:
- Set up and manage a local Kubernetes environment using Minikube.
- Understand how to deploy, manage, and debug applications on Minikube.
- Integrate Minikube into their continuous integration and deployment pipelines.
- Optimize their development process using Minikube's advanced features.
- Apply best practices for local Kubernetes development.