Course Outline
Readying Machine Learning Models for Production Deployment
- Encapsulating models using Docker
- Exporting models from TensorFlow and PyTorch ecosystems
- Best practices for version control and model storage
Serving Models via Kubernetes
- Fundamentals of inference server architectures
- Implementation of TensorFlow Serving and TorchServe
- Establishing and managing model endpoints
Strategies for Inference Optimization
- Implementing efficient batching techniques
- Managing high concurrency and request loads
- Refining latency and throughput metrics
Automated Scaling for ML Workloads
- Leveraging the Horizontal Pod Autoscaler (HPA)
- Utilizing the Vertical Pod Autoscaler (VPA)
- Integrating Kubernetes Event-Driven Autoscaling (KEDA)
GPU Allocation and Resource Stewardship
- Configuration of GPU-enabled nodes
- Insights into the NVIDIA device plugin
- Defining resource requests and limits for ML tasks
Model Release and Rollout Methodologies
- Executing blue/green deployment patterns
- Designing canary rollout mechanisms
- Conducting A/B tests for model performance validation
Production Monitoring and Observability for ML
- Tracking key metrics for inference services
- Implementing robust logging and distributed tracing
- Building dashboards and configuring alerting systems
Addressing Security and System Reliability
- Hardening model endpoints against threats
- Applying network policies and granular access controls
- Safeguarding high availability standards
Recap and Future Directions
Requirements
- Proficiency in managing containerized application lifecycles
- Practical experience with Python-based machine learning pipelines
- Core knowledge of Kubernetes principles
Target Participants
- Machine Learning Engineers
- DevOps Engineers
- Platform Engineering Teams
Testimonials (3)
basic understanding of container/kubernetes and how they interact features of the openshift plattform
Eric Scholze - NOW IT GmbH
Course - Introduction to Containers, Kubernetes & OpenShift
About the microservices and how to maintenance kubernetes
Yufri Isnaini Rochmat Maulana - Bank Indonesia
Course - Advanced Platform Engineering: Scaling with Microservices and Kubernetes
The training met expectations with its clear explanations, real-world examples, and hands-on labs that made complex topics easy to understand. It provided valuable insights into container orchestration, security, scaling and many other advanced topics.
