pod优先级与抢占测试

mhc-fly 2021-09-07 原文

pod优先级与抢占测试

# kubectl describe node k8s-n2
Name:               k8s-n2
Roles:              <none>
Labels:             beta.kubernetes.io/arch=amd64
                    beta.kubernetes.io/os=linux
                    gpu_type=k20m
                    kubernetes.io/hostname=k8s-n2
                    local_storage=ssd
                    role=gpu-node
                    storage=ceph
                    type=physics
Annotations:        node.alpha.kubernetes.io/ttl=0
                    volumes.kubernetes.io/controller-managed-attach-detach=true
CreationTimestamp: Thu, 03 May 2018 06:38:34 +0000
Taints:             <none>
Unschedulable:      false
Addresses:
InternalIP: 109.105.1.208
Hostname:    k8s-n2
Capacity:
cpu:                  64
ephemeral-storage:    229351Mi
hugepages-1Gi:        0
hugepages-2Mi:        0
memory:               165091016Ki
nvidia.com/gpu-k20m: 3
pods:                 110
Allocatable:
cpu:                  64
ephemeral-storage:    216442758401
hugepages-1Gi:        0
hugepages-2Mi:        0
memory:               164988616Ki
nvidia.com/gpu-k20m: 0
pods:                 110

============================================================

1. 每个deploy一个gpu，起了三个low都running，此时资源耗尽

第四个low起不来，第五个middle杀掉了第三个，第五个running

起第六个high，需要2个gpu，杀掉了第一个和第二个，这俩low的，第六个running

删掉第六个，第三，第四 running

删掉第五个，第一个起

在另一个ns里起第七个high，2个gpu，杀掉的一个第2或3，第七个没起来，始终杀不掉两个

最终，全都挂掉

0/10 nodes are available: 10 Insufficient nvidia.com/gpu-k20m, 3 Insufficient memory, 9 node(s) didn\’t match node selector.

Update plugin resources failed due to requested number of devices unavailable for nvidia.com/gpu-k20m. Requested: 1, Available: 0, which is unexpected.