Resource Accounting and Allocation for GPUs

• Applicant: Professor Dr.-Ing. Frank Bellosa
• Subject Area: Security and Dependability, Operating-, Communication- and Distributed Systems
• Term: from 2015 to 2021
• Project identifier: Deutsche Forschungsgemeinschaft (DFG) - Project number 270449430

• Final Report Year: 2021

Final Report Abstract

The RAAG project addresses these challenges of efficient virtualization of graphics processing units (GPUs) by a sampling based approach for resource accounting and usage characterization to support virtualization mechanisms and policies, both on the level of GPU computation as well as GPU memory allocation. We present a novel accounting mechanism for GPU computation time that monitors the GPU’s state to detect GPU-internal context switches, and infers the amount of GPU computation time for each process from the time between these context switches. This approach allows side effect free GPU time accounting without interference to the command submission and without modification of the application. Resource scheduling and fair access to resources requires mechanisms to preempt and relocate resources. We present a mechanism to swap memory between GPU memory and system memory at runtime to allow oversubscription of GPU memory and to dynamically allow an adaptive level of sharing for fast GPU memory. This approach allows a transparent re-allocation of GPU memory without permanent overhead. Basend on the swappping mechanism, we present a memory-aware swapping policy for GPUs. Therefore, the memory-acces patterns of GPU applications have to be analyzed offline with the information derived from GPU performance monitoring counters. Based on insights about these patterns we derive a swapping policy for small chunks of GPU memory that respects the GPU memory-access characteristics for the fair and efficient allocation of GPU- and system-memory. Our event-driven approach to GPU memory management is the first published policy that does not require a permanent software scheduling of GPU commands.

Publications

• Gpuswap: Enabling oversubscription of gpu memory through transparent swapping. In Proceedings of the 11th ACM SIGPLAN/SIGOPS International Conference on Virtual Execution Environments, VEE ’15, pages 65–77, New York, NY, USA, 2015. ACM
  Jens Kehne, Jonathan Metter, and Frank Bellosa
  (See online at https://doi.org/10.1145/2817817.2731192)
• Gprioswap: Towards a swapping policy for gpus. In Proceedings of the 10th ACM International Systems and Storage Conference, SYSTOR ’17, pages 10:1–10:10, New York, NY, USA, 2017. ACM
  Jens Kehne, Marius Hillenbrand, Jonathan Metter, Mathias Gottschlag, Martin Merkel, and Frank Bellosa
  (See online at https://doi.org/10.1145/3078468.3078474)
• Loga: Low-overhead gpu accounting using events. In Proceedings of the 10th ACM International Systems and Storage Conference, SYSTOR ’17, pages 6:1–6:11, New York, NY, USA, 2017. ACM
  Jens Kehne, Stanislav Spassov, Marius Hillenbrand, Marc Rittinghaus, and Frank Bellosa
  (See online at https://doi.org/10.1145/3078468.3078473)
• Transparent Memory Extension for Shared GPUs. PhD thesis, Karlsruhe Institute for Technology, 2019
  Jens Kehne