Architecture Overview
The overall architecture of HAMi is shown as below:
HAMi consists of the following components:
- HAMi MutatingWebhook
- HAMi scheduler-extender
- Device-plugin (HAMi-device-plugin)
- In-container resource control (HAMi-Core)
HAMi MutatingWebhook
HAMi MutatingWebhook checks if this task can be handled by HAMi. It scans the resource field of each pod submitted. If each resource the pod requires is either 'CPU', 'Memory' or a HAMi-resource, then it will set the schedulerName field of this pod to 'hami-scheduler'.
HAMi Scheduler
The HAMi scheduler is responsible for assigning tasks to the appropriate nodes and devices. At the same time, the scheduler needs to maintain a global view of heterogeneous computing devices for monitoring.
Device Plugin
The device-plugin layer obtains the scheduling result from the annotations field of the task and maps the corresponding device to the container.
HAMi-Core
The in-container resource control is responsible for monitoring the resource usage within the container and providing hard isolation capabilities.
🔬Related Hands-on Labs
Build a GPU Kubernetes cluster from scratch on a cloud VM and install HAMi.
Deploy a KServe Standard vLLM service and run two Predictor replicas on one NVIDIA GPU through native HAMi DRA claims.
Learn the HAMi control plane on a laptop, no GPU required.
Simulate 8 A100 GPUs with HAMi scheduling features, no real GPU needed.
