68 lines
2.1 KiB
Markdown
68 lines
2.1 KiB
Markdown
GPU Programming Performance Workshop
|
|
|
|
Problem
|
|
- Compute for vectors A, B, C of size N (float32):
|
|
1) D = A + B
|
|
2) E = D * C + B
|
|
3) result = sum(E)
|
|
- Repeat for --iters iterations. Report time and estimated GB/s.
|
|
|
|
Directory layout
|
|
- cpp_single/main.cpp
|
|
- cpp_omp/main.cpp
|
|
- cuda/main.cu
|
|
- pytorch/baseline.py
|
|
- pytorch/optimized.py
|
|
|
|
Prereqs
|
|
- GCC C++17 compiler (g++)
|
|
- OpenMP (optional for cpp_omp)
|
|
- NVIDIA CUDA toolkit for building cuda/main.cu
|
|
- Python 3.9+ and PyTorch (with CUDA for GPU runs)
|
|
|
|
Build
|
|
- Single-threaded C++:
|
|
g++ -O3 -march=native -std=c++17 -DNDEBUG cpp_single/main.cpp -o bin_cpp_single
|
|
|
|
- OpenMP C++:
|
|
g++ -O3 -march=native -std=c++17 -fopenmp -DNDEBUG cpp_omp/main.cpp -o bin_cpp_omp
|
|
|
|
- CUDA:
|
|
nvcc -O3 -arch=native cuda/main.cu -o bin_cuda
|
|
If -arch=native not supported, use e.g.:
|
|
nvcc -O3 -arch=sm_80 cuda/main.cu -o bin_cuda
|
|
|
|
Run
|
|
- CPU single-thread:
|
|
./bin_cpp_single 100000000 10
|
|
|
|
- CPU OpenMP (set threads):
|
|
export OMP_NUM_THREADS=8
|
|
./bin_cpp_omp 100000000 10
|
|
|
|
- CUDA:
|
|
./bin_cuda 100000000 10
|
|
|
|
- PyTorch baseline (CPU or GPU auto-detect):
|
|
python pytorch/baseline.py --N 100000000 --iters 10 --device cuda
|
|
python pytorch/baseline.py --N 100000000 --iters 10 --device cpu
|
|
|
|
- PyTorch optimized:
|
|
python pytorch/optimized.py --N 100000000 --iters 10
|
|
|
|
Notes
|
|
- Memory: N=100M uses ~400 MB for A,B,C and ~400 MB for D,E. Ensure enough RAM/GPU memory.
|
|
- If you hit OOM on GPU, reduce N (e.g., 50_000_000).
|
|
- Throughput model assumes 7 floats per element per iter moved; actual may vary.
|
|
- For fair GPU timing, we synchronize after each iter.
|
|
- To compare kernel launch overhead, try small N (e.g., 1_000_000) and more iters.
|
|
- To compare bandwidth limits, try large N (e.g., 200_000_000) and fewer iters.
|
|
- PyTorch optimized uses pinned memory, in-place ops, preallocation, and CUDA Graphs.
|
|
- You can profile with:
|
|
- nsight systems: nsys profile ./bin_cuda 50000000 50
|
|
- nvprof (legacy): nvprof ./bin_cuda 50000000 50
|
|
- torch profiler: see torch.profiler in docs.
|
|
|
|
Validation
|
|
- All variants print "result" which should be numerically close across methods
|
|
(tiny differences expected due to different reduction orders and precision).
|