Fundamentals of Accelerated Computing with CUDA C/C++ Training Course

GPU Computing and CUDA Architecture

• Architectural differences between CPUs and GPUs.
• NVIDIA GPU streaming multiprocessor model.
• Overview of the CUDA programming model.
• Heterogeneous computing and the host-device paradigm.

Setting Up the CUDA Development Environment

• Installing CUDA Toolkit 13.x.
• NVCC compiler and build workflow.
• Verifying the environment through device queries.
• IDE integration and development tools.

Writing and Launching CUDA Kernels

• Syntax and qualifiers for kernel functions.
• Launch configuration and execution mechanics.
• Vector addition and basic data-parallel patterns.
• CUDA error checking macros.

CUDA Thread Hierarchy and Execution Model

• Organization of grids, blocks, and threads.
• Thread indexing and global ID calculation.
• Warp execution and the SIMT model.
• Occupancy and resource utilization strategies.

GPU Memory Architecture and Management

• Memory types: global, shared, constant, and registers.
• Allocating and freeing device memory.
• Host-to-device and device-to-host transfers.
• Utilizing shared memory for intra-block collaboration.

Unified Memory and Data Migration

• The unified memory model and managed allocations.
• Page migration and on-demand paging mechanisms.
• Asynchronous prefetching using cudaMemPrefetchAsync.
• Memory advice hints for optimizing access patterns.

System-Wide Profiling with Nsight Systems

• Analyzing timelines in Nsight Systems.
• Identifying CPU-GPU synchronization points.
• Visualizing kernel execution and memory transfers.
• Interpreting system-level performance data.

Kernel Optimization with Nsight Compute

• Interactive kernel profiling using Nsight Compute.
• Analyzing memory throughput and bandwidth.
• Evaluating compute utilization and warp state statistics.
• Guided analysis and optimization rules.

Concurrent Streams and Asynchronous Operations

• CUDA streams and the default stream behavior.
• Overlapping kernel execution with data transfers.
• Stream synchronization and CUDA events.
• Design patterns for multi-stream pipelines.

Error Handling and Debugging Tools

• CUDA API error codes and recovery strategies.
• Using compute-sanitizer for memory access checks.
• Debugging kernels with cuda-gdb.
• Assertions and synchronous error detection techniques.

Profile-Driven Optimization Workflow

• Iterative profiling methodology.
• Identifying and prioritizing bottlenecks.
• Performance regression testing.
• Documenting optimization decisions.

End-to-End Accelerated Application Project

• Designing a complete GPU-accelerated solution.
• Integrating profiling throughout the development lifecycle.
• Performance benchmarking and reporting.
• Deployment considerations for production environments.