GPU Programming with CUDA Training Course

Introduction

• What is CUDA?
• CUDA vs OpenCL vs SYCL
• Overview of CUDA features and architecture
• Setting up the Development Environment

Getting Started

• Creating a new CUDA project using Visual Studio Code
• Exploring the project structure and files
• Compiling and running the program
• Displaying output using printf and fprintf

CUDA API

• Understanding the role of the CUDA API in host programs
• Querying device information and capabilities via the CUDA API
• Allocating and deallocating device memory using the CUDA API
• Copying data between host and device with the CUDA API
• Launching kernels and synchronizing threads via the CUDA API
• Handling errors and exceptions with the CUDA API

CUDA C/C++

• Understanding the role of CUDA C/C++ in device programs
• Writing kernels with CUDA C/C++ to execute on the GPU and manipulate data
• Using CUDA C/C++ data types, qualifiers, operators, and expressions
• Leveraging CUDA C/C++ built-in functions for math, atomics, warps, etc.
• Utilizing CUDA C/C++ built-in variables such as threadIdx, blockIdx, and blockDim
• Using CUDA C/C++ libraries like cuBLAS, cuFFT, and cuRAND

CUDA Memory Model

• Understanding the differences between host and device memory models
• Using CUDA memory spaces: global, shared, constant, and local
• Working with CUDA memory objects: pointers, arrays, textures, and surfaces
• Applying CUDA memory access modes: read-only, write-only, read-write, etc.
• Implementing the CUDA memory consistency model and synchronization mechanisms

CUDA Execution Model

• Understanding the differences between host and device execution models
• Defining parallelism using CUDA threads, blocks, and grids
• Using CUDA thread functions such as threadIdx, blockIdx, and blockDim
• Applying CUDA block functions like __syncthreads and __threadfence_block
• Utilizing CUDA grid functions including gridDim, gridSync, and cooperative groups

Debugging

• Understanding common errors and bugs in CUDA programs
• Using the Visual Studio Code debugger to inspect variables, breakpoints, and call stacks
• Debugging CUDA programs on Linux using CUDA-GDB
• Detecting memory errors and leaks with CUDA-MEMCHECK
• Debugging and analyzing CUDA programs on Windows using NVIDIA Nsight

Optimization

• Understanding factors that affect CUDA program performance
• Using CUDA coalescing techniques to improve memory throughput
• Applying CUDA caching and prefetching techniques to reduce memory latency
• Optimizing memory access and bandwidth with CUDA shared and local memory techniques
• Measuring and improving execution time and resource utilization using CUDA profiling and profiling tools