GPU Programming

• Compute
  • CUDA (NVIDIA)
  • HIP/ROCm (AMD)
  • AI/ML specific
    • MLX (Apple Silicon)
  • Parallel programming specific
    • OpenCL
  • GPU Acceleration specific
    • MPS (Apple Metal)
• Compute + Graphics
  • OpenGL
  • Vulkan
  • Direct3D (Microsoft DirectX)
  • Metal (Apple)
  • WebGPU

CUDA

• Compute Unified Device Architecture
• Developed by NVIDIA
• written in C++
• Compilation Steps
  • CUDA
  • PTX
    • Parallel Thread Execution
  • SASS
    • Native assembly instructions for NVIDIA GPUs
    • Architecture-specific and subject to change
• SASS documentation is less accessible and less stable than PTX documentation
• CUDA: https://docs.nvidia.com/cuda/cuda-c-programming-guide/
• PTX: https://docs.nvidia.com/cuda/parallel-thread-execution/

HIP / ROCm

• HIP: Heterogeneous-Compute Interface for Portability
• ROCm: Radeon Open Compute (software platform)
• Developed by AMD
• HIP uses C++
• Compilation steps
  • HIP
  • AMDGPU IR
    • AMD GPU Intermediate Representation
    • Intermediate representation based on LLVM
  • GFX ISA
    • Graphics Core Next Instruction Set Architecture
    • Native assembly instructions for AMD GPUs
• HIP / ROCm: https://rocm.docs.amd.com/
• AMDGPU IR: https://rocm.docs.amd.com/projects/HIP/en/latest/understand/compilers.html
• GFX ISA: https://github.com/GPUOpen-Tools/isa_spec_manager/blob/main/documentation/spec_documentation.md

MLX

• Array framework for machine learning
• Developed by Apple’s ML research
• Designed for Apple silicon
• Provides Python and C++ APIs
  • Similar to NumPy APIs
• Supports CPU and GPU execution
• Uses a unified memory model on Apple silicon, allowing supported devices to access shared arrays without explicit copies
• Documentation: https://ml-explore.github.io/mlx/build/html/index.html

OpenCL

• Open Computing Language
• Developed by Khronos Group
• Open standard for parallel programming across CPUs, GPUs, and other processors
• OpenCL uses C
• Compilation steps
  • OpenCL
  • SPIR-V
    • Standard Portable Intermediate Representation – V
    • Portable intermediate representation for parallel compute and graphics
  • GPU-specific machine code
• Documentation: https://registry.khronos.org/OpenCL/

MPS

• Metal Performance Shaders
• Apple framework of optimized graphics and compute operations built on Metal
• Provides reusable GPU-accelerated primitives, including image processing and machine-learning operations
• MPSGraph provides a higher-level way to describe and execute compute and machine-learning graphs
• Used by frameworks such as PyTorch’s MPS backend to access Apple GPUs
• This is a framework layered on Metal, rather than a standalone cross-platform GPU API
• Documentation: https://developer.apple.com/documentation/metalperformanceshaders

OpenGL

• Developed by the Khronos Group
• Cross-platform graphics API
• Primarily designed for rendering graphics
• OpenGL 4.3 and later also support compute shaders for general-purpose GPU work
• GLSL (OpenGL Shading Language) is used to write graphics and compute shaders
• Compilation steps
  • GLSL
  • GPU-specific machine code
• Documentation: https://www.khronos.org/opengl/

Vulkan

• Developed by Khronos Group
• Cross-platform graphics and compute API
• Shaders can be written in GLSL (OpenGL Shading Language) or HLSL (High-Level Shading Language)
• Compilation steps
  • GLSL / HLSL
  • SPIR-V
  • GPU-specific machine code
• Documentation: https://docs.vulkan.org/

Direct3D

• Microsoft’s graphics and compute API family within DirectX
• Direct3D 12 is a low-level API for Windows and Xbox with explicit control over GPU resources and command submission
• Compute shaders can run general-purpose workloads as well as graphics workloads
• Shaders are commonly written in HLSL (High-Level Shading Language), developed by Microsoft
• Compilation steps
  • HLSL
  • DXIL (DirectX Intermediate Language)
  • Driver- and GPU-specific machine code
• Documentation: https://learn.microsoft.com/en-us/windows/win32/direct3d12/

Metal

• Apple’s graphics and compute API for Apple platforms
• Designed for low-overhead access to Apple GPUs
• MSL (Metal Shading Language) is based on C++ and is used to write GPU shaders and compute kernels
• Compilation steps
  • Metal Shading Language
  • AIR (Apple Intermediate Representation)
  • GPU-specific machine code
• Metal Performance Shaders provides optimized operations on top of Metal
• Documentation: https://developer.apple.com/metal/

WebGPU

• Cross-platform web API for graphics and general-purpose GPU compute
• Standardized by the W3C; exposed to web applications through JavaScript APIs
• WGSL (WebGPU Shading Language) is the standard shader language
• Implementations map WebGPU operations to native graphics APIs such as Vulkan, Metal, or Direct3D 12
• Compilation steps
  • WGSL
  • Implementation-specific shader representation
  • Backend- and GPU-specific machine code
• Documentation: https://www.w3.org/TR/webgpu/
• WGSL specification: https://www.w3.org/TR/WGSL/