Thank you for sending your enquiry! One of our team members will contact you shortly.
Thank you for sending your booking! One of our team members will contact you shortly.
Course Outline
NVIDIA AI and GPU Infrastructure Foundations
Outcome: Establish a shared technical foundation and correlate AI workload characteristics with GPU infrastructure decisions.
AI Workloads and Accelerated Computing
- Concepts of AI, machine learning, deep learning, generative AI, and modern AI factories.
- Training, fine-tuning, and inference: distinct resource and performance profiles.
- Differences in behavior between CPUs and GPUs for parallel AI workloads.
NVIDIA GPU Architecture
- GPU generations, streaming multiprocessors, CUDA cores, and Tensor Cores.
- Device memory, bandwidth, caches, and data movement considerations.
- PCIe, SXM, NVLink, and NVSwitch in single- and multi-GPU systems.
- Interpreting device identity, topology, and capabilities using NVIDIA utilities.
NVIDIA Hardware and Software Ecosystem
- The roles of CPUs, GPUs, and BlueField DPUs in accelerated infrastructure.
- DGX, HGX, and NVIDIA-Certified server concepts and deployment models.
- Drivers, CUDA, container runtime components, NGC, and NVIDIA AI Enterprise.
- Deployment options including bare metal, virtualized, Kubernetes, and cloud environments.
AI Infrastructure Building Blocks
- Responsibilities of the compute, network, storage, and management planes.
- Reference architectures and support matrices as tools for design and validation.
- Core concepts in monitoring, orchestration, MLOps, and job scheduling.
HANDS-ON LAB | Discover and Validate a GPU Node
- Examine GPU model, driver, firmware-visible state, and CUDA compatibility.
- Analyze GPU-to-CPU and GPU-to-GPU topology to identify potential bottlenecks.
- Create a baseline inventory for use in subsequent virtualization and monitoring labs.
AI Server Architecture and Infrastructure Deployment
Outcome: Translate workload requirements into a sustainable AI server, network, and storage design.
AI Server Platforms and Topology
- PCIe and SXM platform characteristics; design patterns for single- and multi-GPU configurations.
- CPU architecture, memory, NUMA locality, and PCIe lane considerations.
- NVLink/NVSwitch topology, GPU peer access, and scale-up communication.
- BMC, firmware, BIOS, and operating system prerequisites for GPU servers.
Networking for AI Workloads
- The roles of Ethernet and InfiniBand in training and inference environments.
- RDMA, RoCE, GPUDirect RDMA, and latency/bandwidth considerations.
- Separation of north-south, east-west, and management networks.
- Fabric redundancy, observability, and common operational failure points.
Storage and the AI Data Path
- Access patterns for datasets, checkpoints, models, and inference serving.
- Considerations for local, shared file, object, and parallel storage.
- Throughput, IOPS, metadata performance, and GPUDirect Storage concepts.
Deployment and Operational Readiness
- Sizing compute, memory, network, and storage based on workload demand.
- Validation of power, cooling, rack, cabling, and environmental factors.
- Alignment of support matrices, firmware/drivers, and acceptance testing.
- Planning for capacity headroom, resilience, and lifecycle management.
HANDS-ON LAB | Design and Validate an AI Infrastructure Blueprint
- Map workload profiles to server, GPU, network, and storage requirements.
- Review topology to identify risks related to oversubscription, NUMA, or data paths.
- Develop a concise deployment-readiness and acceptance checklist.
GPU Virtualization, NVIDIA vGPU and VFIO Passthrough
Outcome: Select and implement the optimal GPU assignment model for virtualized AI workloads.
GPU Resource Assignment Models
- Comparison of full physical GPU assignment, time-sliced vGPU, MIG-backed vGPU, and MIG.
- Trade-offs regarding isolation, predictability, density, live operations, and licensing.
- Workload placement patterns for development, inference, fine-tuning, and shared services.
NVIDIA vGPU Architecture and Administration
- vGPU Manager, guest drivers, profile selection, and frame-buffer allocation.
- Supported hypervisors, guest operating systems, and compatibility validation.
- NVIDIA license services, entitlements, and operational dependencies.
- Provisioning, validation, capacity control, and common failure modes.
Linux VFIO GPU Passthrough
- IOMMU concepts, PCIe device groups, and platform prerequisites.
- Identifying GPU and companion devices; binding devices to vfio-pci.
- Assigning a GPU to a virtual machine and installing the guest driver.
- Validating device isolation and resolving driver-binding conflicts.
Performance and Troubleshooting
- Considerations for NUMA placement, CPU pinning, huge pages, and memory locking.
- Host/guest driver compatibility and device visibility diagnostics.
- Criteria for choosing between passthrough and vGPU in enterprise operations.
HANDS-ON LAB | Assign and Validate GPU Resources in a VM
- Verify IOMMU and device grouping; prepare the GPU for passthrough or vGPU assignment.
- Attach the GPU resource, install or validate guest components, and confirm workload access.
- Diagnose a prepared device-visibility or compatibility fault.
Multi-Instance GPU and NVIDIA AI Enterprise
Outcome: Partition supported GPUs using MIG and deploy governed AI software components with NVIDIA AI Enterprise.
MIG Architecture and Use Cases
- Spatial GPU partitioning, hardware isolation, and predictable resource allocation.
- GPU instances, compute instances, profiles, and supported configuration patterns.
- Comparison of MIG with time slicing, vGPU, and full-GPU assignment.
- Suitable use cases for inference, development, smaller training, and multi-tenancy.
MIG Configuration and Lifecycle
- Enabling MIG mode and creating, listing, and removing instances.
- Mapping profiles to workloads and exposing devices to containers or schedulers.
- Concepts of MIG-backed vGPU and platform-dependent limitations.
- Considerations for reconfiguration, maintenance, persistence, and troubleshooting.
NVIDIA AI Enterprise
- Platform architecture, infrastructure layer, and application software layer.
- Licensing, NGC access, release branches, and lifecycle/support considerations.
- Drivers, NVIDIA Container Toolkit, GPU Operator, and supported deployment patterns.
- Deployment paths for bare metal, virtualized, and Kubernetes environments.
Containerized AI Workloads
- Accessing approved containers and artifacts from NGC.
- Running GPU-enabled containers and validating resource visibility.
- Security, image provenance, secrets management, and controlled operational access.
HANDS-ON LAB | Partition a GPU and Deploy a Workload
- Create a supported MIG configuration and verify instance allocation.
- Expose a selected GPU or MIG device to a containerized workload.
- Validate isolation, utilization, and basic workload health.
Monitoring, Operations, Troubleshooting and Capstone
Outcome: Manage NVIDIA AI infrastructure through repeatable monitoring, incident, change, and capacity management practices.
GPU Monitoring and Observability
- Using nvidia-smi and NVIDIA DCGM to inspect health and utilization.
- Core metrics: compute, memory, power, temperature, clocks, PCIe/NVLink, and ECC.
- Integration patterns for DCGM Exporter, Prometheus, and Grafana.
- Establishing baselines, alert thresholds, dashboards, and capacity trends.
Troubleshooting and Performance
- Addressing GPU discovery failures, driver/library mismatches, and container-runtime issues.
- Handling Xid events, ECC errors, thermal or power throttling, and resource contention.
- Distinguishing between infrastructure, virtualization, scheduler, and workload issues.
- Evidence collection, safe remediation, escalation, and post-incident review.
Operational Best Practices
- Lifecycle planning for drivers, firmware, and NVIDIA AI Enterprise.
- Change windows, rollback readiness, and compatibility validation.
- Capacity management, workload placement, and service-level considerations.
- Access control, auditability, image governance, and operational documentation.
- Runbooks for health checks, incident response, and routine maintenance.
HANDS-ON LAB | Final Operational Capstone
- Review a scenario and select an appropriate GPU resource-assignment model.
- Validate the environment, deploy or inspect a workload, and establish monitoring.
- Diagnose injected symptoms and document an evidence-based remediation path.
- Present a concise deployment and operations runbook for peer review.
Assessment and Completion
- Brief knowledge checks at the end of each day.
- Completion of guided practical exercises and validation tasks.
- Participation in the final troubleshooting and operations capstone.
- NobleProg course completion certificate for participants meeting the agreed criteria.
Recommended Lab Environment
The final configuration depends on the client's target platform. A suitable lab typically includes:
- An NVIDIA data-center GPU platform supporting the selected vGPU and/or MIG exercises.
- A supported hypervisor or Linux/KVM host with IOMMU/VFIO enabled where applicable.
- Supported Linux guest or bare-metal operating systems with administrative access.
- Required NVIDIA drivers, vGPU components, and license service access.
- NVIDIA AI Enterprise entitlement or evaluation access and an NGC account/API key.
- Docker or another supported container runtime; Kubernetes if agreed for the delivery.
- Monitoring components such as DCGM, DCGM Exporter, Prometheus, and Grafana.
- Internet or registry access, or pre-staged packages and container images for restricted sites.
LAB ASSUMPTION | Hands-on vGPU, VFIO, and MIG activities require compatible hardware, software, and entitlements. NobleProg will validate the lab bill of materials and adapt exercises before delivery. Where a live configuration is not feasible, the affected exercise may use an instructor demonstration or prepared simulation.
Participant Deliverables
- NobleProg course materials and lab instructions.
- GPU infrastructure validation checklist.
- Sample monitoring and operational runbook structure.
- NobleProg electronic certificate of attendance or completion.
Requirements
Participants should possess a practical infrastructure background and ensure access to the agreed-upon laboratory environment.
- Proficiency in Linux and command-line operations
- Familiarity with server hardware, networking, and storage systems
- Basic understanding of virtual machines and hypervisors
- Foundational knowledge of AI, machine learning, and GPU workloads
- Awareness of containers and Kubernetes is beneficial but not mandatory
- Administrative access to the lab for practical exercises
70 Hours
