error: suite "openshift/conformance/parallel" does not contain any tests"[sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic Snapshot] snapshottable should create snapshot with defaults [Feature:VolumeSnapshotDataSource] [Suite:openshift/conformance/parallel] [Suite:k8s]" started: (0/1/1) "[sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic Snapshot] snapshottable should create snapshot with defaults [Feature:VolumeSnapshotDataSource] [Suite:openshift/conformance/parallel] [Suite:k8s]" I0615 15:59:24.431129 660282 test_context.go:423] Tolerating taints "node-role.kubernetes.io/master" when considering if nodes are ready Jun 15 15:59:24.454: INFO: Waiting up to 30m0s for all (but 100) nodes to be schedulable Jun 15 15:59:25.477: INFO: Waiting up to 10m0s for all pods (need at least 0) in namespace 'kube-system' to be running and ready Jun 15 15:59:26.023: INFO: 0 / 0 pods in namespace 'kube-system' are running and ready (0 seconds elapsed) Jun 15 15:59:26.023: INFO: expected 0 pod replicas in namespace 'kube-system', 0 are Running and Ready. Jun 15 15:59:26.023: INFO: Waiting up to 5m0s for all daemonsets in namespace 'kube-system' to start Jun 15 15:59:26.204: INFO: e2e test version: v1.18.3 Jun 15 15:59:26.373: INFO: kube-apiserver version: v1.18.3+a637491 Jun 15 15:59:26.550: INFO: Cluster IP family: ipv4 [BeforeEach] [Top Level] /home/fidencio/src/upstream/origin/_output/local/go/src/github.com/openshift/origin/test/extended/util/framework.go:1413 [BeforeEach] [Top Level] /home/fidencio/src/upstream/origin/_output/local/go/src/github.com/openshift/origin/test/extended/util/framework.go:1413 [BeforeEach] [Top Level] /home/fidencio/src/upstream/origin/_output/local/go/src/github.com/openshift/origin/test/extended/util/test.go:58 [BeforeEach] [Testpattern: Dynamic Snapshot] snapshottable /home/fidencio/src/upstream/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/base.go:95 [BeforeEach] [Testpattern: Dynamic Snapshot] snapshottable /home/fidencio/src/upstream/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/snapshottable.go:87 [BeforeEach] [Testpattern: Dynamic Snapshot] snapshottable /home/fidencio/src/upstream/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:178 STEP: Creating a kubernetes client STEP: Building a namespace api object, basename snapshotting Jun 15 15:59:27.220: INFO: About to run a Kube e2e test, ensuring namespace is privileged Jun 15 15:59:29.768: INFO: No PodSecurityPolicies found; assuming PodSecurityPolicy is disabled. STEP: Waiting for a default service account to be provisioned in namespace [It] should create snapshot with defaults [Feature:VolumeSnapshotDataSource] [Suite:openshift/conformance/parallel] [Suite:k8s] /home/fidencio/src/upstream/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/storage/testsuites/snapshottable.go:108 STEP: deploying csi-hostpath driver Jun 15 15:59:30.828: INFO: creating *v1.ServiceAccount: e2e-snapshotting-7614/csi-attacher Jun 15 15:59:30.999: INFO: creating *v1.ClusterRole: external-attacher-runner-e2e-snapshotting-7614 Jun 15 15:59:30.999: INFO: Define cluster role external-attacher-runner-e2e-snapshotting-7614 Jun 15 15:59:31.209: INFO: creating *v1.ClusterRoleBinding: csi-attacher-role-e2e-snapshotting-7614 Jun 15 15:59:31.381: INFO: creating *v1.Role: e2e-snapshotting-7614/external-attacher-cfg-e2e-snapshotting-7614 Jun 15 15:59:31.553: INFO: creating *v1.RoleBinding: e2e-snapshotting-7614/csi-attacher-role-cfg Jun 15 15:59:31.824: INFO: creating *v1.ServiceAccount: e2e-snapshotting-7614/csi-provisioner Jun 15 15:59:32.029: INFO: creating *v1.ClusterRole: external-provisioner-runner-e2e-snapshotting-7614 Jun 15 15:59:32.029: INFO: Define cluster role external-provisioner-runner-e2e-snapshotting-7614 Jun 15 15:59:32.234: INFO: creating *v1.ClusterRoleBinding: csi-provisioner-role-e2e-snapshotting-7614 Jun 15 15:59:32.405: INFO: creating *v1.Role: e2e-snapshotting-7614/external-provisioner-cfg-e2e-snapshotting-7614 Jun 15 15:59:32.577: INFO: creating *v1.RoleBinding: e2e-snapshotting-7614/csi-provisioner-role-cfg Jun 15 15:59:32.750: INFO: creating *v1.ServiceAccount: e2e-snapshotting-7614/csi-snapshotter Jun 15 15:59:32.922: INFO: creating *v1.ClusterRole: external-snapshotter-runner-e2e-snapshotting-7614 Jun 15 15:59:32.922: INFO: Define cluster role external-snapshotter-runner-e2e-snapshotting-7614 Jun 15 15:59:33.094: INFO: creating *v1.ClusterRoleBinding: csi-snapshotter-role-e2e-snapshotting-7614 Jun 15 15:59:33.266: INFO: creating *v1.Role: e2e-snapshotting-7614/external-snapshotter-leaderelection-e2e-snapshotting-7614 Jun 15 15:59:33.438: INFO: creating *v1.RoleBinding: e2e-snapshotting-7614/external-snapshotter-leaderelection Jun 15 15:59:33.669: INFO: creating *v1.ServiceAccount: e2e-snapshotting-7614/csi-resizer Jun 15 15:59:33.841: INFO: creating *v1.ClusterRole: external-resizer-runner-e2e-snapshotting-7614 Jun 15 15:59:33.841: INFO: Define cluster role external-resizer-runner-e2e-snapshotting-7614 Jun 15 15:59:34.020: INFO: creating *v1.ClusterRoleBinding: csi-resizer-role-e2e-snapshotting-7614 Jun 15 15:59:34.281: INFO: creating *v1.Role: e2e-snapshotting-7614/external-resizer-cfg-e2e-snapshotting-7614 Jun 15 15:59:34.454: INFO: creating *v1.RoleBinding: e2e-snapshotting-7614/csi-resizer-role-cfg Jun 15 15:59:34.692: INFO: creating *v1.Service: e2e-snapshotting-7614/csi-hostpath-attacher Jun 15 15:59:34.896: INFO: creating *v1.StatefulSet: e2e-snapshotting-7614/csi-hostpath-attacher Jun 15 15:59:35.083: INFO: creating *v1.CSIDriver: csi-hostpath-e2e-snapshotting-7614 pod event: ADDED: e2e-snapshotting-7614/csi-hostpath-attacher-0 Pending: [] pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpath-attacher-0 Pending: [{PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST }] pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpath-attacher-0 Pending: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST } {Ready False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST ContainersNotReady containers with unready status: [csi-attacher]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST ContainersNotReady containers with unready status: [csi-attacher]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST }] csi-attacher: WAITING: ContainerCreating - pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpath-attacher-0 Pending: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST } {Ready False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST ContainersNotReady containers with unready status: [csi-attacher]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST ContainersNotReady containers with unready status: [csi-attacher]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST }] csi-attacher: WAITING: ContainerCreating - Jun 15 15:59:35.259: INFO: creating *v1.Service: e2e-snapshotting-7614/csi-hostpathplugin Jun 15 15:59:35.434: INFO: creating *v1.StatefulSet: e2e-snapshotting-7614/csi-hostpathplugin Jun 15 15:59:35.718: INFO: creating *v1.Service: e2e-snapshotting-7614/csi-hostpath-provisioner pod event: ADDED: e2e-snapshotting-7614/csi-hostpathplugin-0 Pending: [] pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpathplugin-0 Pending: [{PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST }] pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpathplugin-0 Pending: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST } {Ready False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST ContainersNotReady containers with unready status: [node-driver-registrar hostpath liveness-probe]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST ContainersNotReady containers with unready status: [node-driver-registrar hostpath liveness-probe]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST }] hostpath: WAITING: ContainerCreating - liveness-probe: WAITING: ContainerCreating - node-driver-registrar: WAITING: ContainerCreating - pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpathplugin-0 Pending: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST } {Ready False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST ContainersNotReady containers with unready status: [node-driver-registrar hostpath liveness-probe]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST ContainersNotReady containers with unready status: [node-driver-registrar hostpath liveness-probe]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST }] hostpath: WAITING: ContainerCreating - liveness-probe: WAITING: ContainerCreating - node-driver-registrar: WAITING: ContainerCreating - Jun 15 15:59:35.922: INFO: creating *v1.StatefulSet: e2e-snapshotting-7614/csi-hostpath-provisioner Jun 15 15:59:36.125: INFO: creating *v1.Service: e2e-snapshotting-7614/csi-hostpath-resizer pod event: ADDED: e2e-snapshotting-7614/csi-hostpath-provisioner-0 Pending: [] pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpath-provisioner-0 Pending: [{PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST }] pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpath-provisioner-0 Pending: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST } {Ready False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST ContainersNotReady containers with unready status: [csi-provisioner]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST ContainersNotReady containers with unready status: [csi-provisioner]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST }] csi-provisioner: WAITING: ContainerCreating - pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpath-provisioner-0 Pending: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST } {Ready False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST ContainersNotReady containers with unready status: [csi-provisioner]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST ContainersNotReady containers with unready status: [csi-provisioner]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST }] csi-provisioner: WAITING: ContainerCreating - Jun 15 15:59:36.300: INFO: creating *v1.StatefulSet: e2e-snapshotting-7614/csi-hostpath-resizer Jun 15 15:59:36.472: INFO: creating *v1.Service: e2e-snapshotting-7614/csi-hostpath-snapshotter pod event: ADDED: e2e-snapshotting-7614/csi-hostpath-resizer-0 Pending: [] pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpath-resizer-0 Pending: [{PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST }] pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpath-resizer-0 Pending: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST } {Ready False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST ContainersNotReady containers with unready status: [csi-resizer]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST ContainersNotReady containers with unready status: [csi-resizer]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST }] csi-resizer: WAITING: ContainerCreating - pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpath-resizer-0 Pending: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST } {Ready False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST ContainersNotReady containers with unready status: [csi-resizer]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST ContainersNotReady containers with unready status: [csi-resizer]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST }] csi-resizer: WAITING: ContainerCreating - Jun 15 15:59:36.648: INFO: creating *v1.StatefulSet: e2e-snapshotting-7614/csi-hostpath-snapshotter pod event: ADDED: e2e-snapshotting-7614/csi-hostpath-snapshotter-0 Pending: [] Jun 15 15:59:36.842: INFO: creating *v1.ClusterRoleBinding: psp-csi-hostpath-role-e2e-snapshotting-7614 pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpath-snapshotter-0 Pending: [{PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST }] pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpath-snapshotter-0 Pending: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST } {Ready False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST ContainersNotReady containers with unready status: [csi-snapshotter]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST ContainersNotReady containers with unready status: [csi-snapshotter]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST }] csi-snapshotter: WAITING: ContainerCreating - pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpath-snapshotter-0 Pending: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST } {Ready False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST ContainersNotReady containers with unready status: [csi-snapshotter]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST ContainersNotReady containers with unready status: [csi-snapshotter]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST }] csi-snapshotter: WAITING: ContainerCreating - Jun 15 15:59:37.016: INFO: In creating storage class object and pvc object for driver - sc: &StorageClass{ObjectMeta:{e2e-snapshotting-7614-csi-hostpath-e2e-snapshotting-7614-st46sw 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Provisioner:csi-hostpath-e2e-snapshotting-7614,Parameters:map[string]string{},ReclaimPolicy:nil,MountOptions:[],AllowVolumeExpansion:nil,VolumeBindingMode:*Immediate,AllowedTopologies:[]TopologySelectorTerm{},}, pvc: &PersistentVolumeClaim{ObjectMeta:{ pvc- e2e-snapshotting-7614 0 0001-01-01 00:00:00 +0000 UTC map[] map[] [] [] []},Spec:PersistentVolumeClaimSpec{AccessModes:[ReadWriteOnce],Resources:ResourceRequirements{Limits:ResourceList{},Requests:ResourceList{storage: {{1048576 0} {} 1Mi BinarySI},},},VolumeName:,Selector:nil,StorageClassName:*e2e-snapshotting-7614-csi-hostpath-e2e-snapshotting-7614-st46sw,VolumeMode:nil,DataSource:nil,},Status:PersistentVolumeClaimStatus{Phase:,AccessModes:[],Capacity:ResourceList{},Conditions:[]PersistentVolumeClaimCondition{},},} STEP: creating a StorageClass e2e-snapshotting-7614-csi-hostpath-e2e-snapshotting-7614-st46sw STEP: creating a claim STEP: starting a pod to use the claim Jun 15 15:59:37.543: INFO: Waiting up to 5m0s for PersistentVolumeClaims [pvc-glxfp] to have phase Bound pod event: ADDED: e2e-snapshotting-7614/pvc-snapshottable-tester-9782d Pending: [] pod event: MODIFIED: e2e-snapshotting-7614/pvc-snapshottable-tester-9782d Pending: [{PodScheduled False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:37 +0200 CEST Unschedulable running "VolumeBinding" filter plugin for pod "pvc-snapshottable-tester-9782d": pod has unbound immediate PersistentVolumeClaims}] pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpath-attacher-0 Pending: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST } {Ready False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST ContainersNotReady containers with unready status: [csi-attacher]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST ContainersNotReady containers with unready status: [csi-attacher]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST }] csi-attacher: WAITING: ContainerCreating - Jun 15 15:59:37.712: INFO: PersistentVolumeClaim pvc-glxfp found but phase is Pending instead of Bound. pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpathplugin-0 Pending: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST } {Ready False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST ContainersNotReady containers with unready status: [node-driver-registrar hostpath liveness-probe]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST ContainersNotReady containers with unready status: [node-driver-registrar hostpath liveness-probe]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST }] hostpath: WAITING: ContainerCreating - liveness-probe: WAITING: ContainerCreating - node-driver-registrar: WAITING: ContainerCreating - pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpath-attacher-0 Running: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST } {Ready True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:38 +0200 CEST } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:38 +0200 CEST } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST }] csi-attacher: RUNNING pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpath-provisioner-0 Pending: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST } {Ready False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST ContainersNotReady containers with unready status: [csi-provisioner]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST ContainersNotReady containers with unready status: [csi-provisioner]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST }] csi-provisioner: WAITING: ContainerCreating - csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:38.049431 1 main.go:91] Version: v2.2.0-0-g97411fa7 csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:38.052014 1 connection.go:153] Connecting to unix:///csi/csi.sock pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpath-resizer-0 Pending: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST } {Ready False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST ContainersNotReady containers with unready status: [csi-resizer]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST ContainersNotReady containers with unready status: [csi-resizer]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST }] csi-resizer: WAITING: ContainerCreating - pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpath-snapshotter-0 Pending: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST } {Ready False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST ContainersNotReady containers with unready status: [csi-snapshotter]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST ContainersNotReady containers with unready status: [csi-snapshotter]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST }] csi-snapshotter: WAITING: ContainerCreating - csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:39.053094 1 common.go:111] Probing CSI driver for readiness csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:39.053120 1 connection.go:182] GRPC call: /csi.v1.Identity/Probe csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:39.053125 1 connection.go:183] GRPC request: {} csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:39.058739 1 connection.go:185] GRPC response: {} csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:39.059327 1 connection.go:186] GRPC error: csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:39.059342 1 connection.go:182] GRPC call: /csi.v1.Identity/GetPluginInfo csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:39.059347 1 connection.go:183] GRPC request: {} csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:39.060894 1 connection.go:185] GRPC response: {"name":"csi-hostpath-e2e-snapshotting-7614","vendor_version":"v1.4.0-rc2-0-gee6beeaf"} csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:39.061261 1 connection.go:186] GRPC error: csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:39.061269 1 main.go:136] CSI driver name: "csi-hostpath-e2e-snapshotting-7614" csi-hostpath-attacher-0/csi-attacher@worker-0-1: W0615 13:59:39.061285 1 metrics.go:142] metrics endpoint will not be started because `metrics-address` was not specified. csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:39.061290 1 connection.go:182] GRPC call: /csi.v1.Identity/GetPluginCapabilities csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:39.061293 1 connection.go:183] GRPC request: {} csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:39.067709 1 connection.go:185] GRPC response: {"capabilities":[{"Type":{"Service":{"type":1}}},{"Type":{"Service":{"type":2}}}]} csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:39.068771 1 connection.go:186] GRPC error: csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:39.068780 1 connection.go:182] GRPC call: /csi.v1.Controller/ControllerGetCapabilities csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:39.068784 1 connection.go:183] GRPC request: {} csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:39.073085 1 connection.go:185] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":9}}}]} csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:39.076356 1 connection.go:186] GRPC error: csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:39.076370 1 main.go:165] CSI driver does not support ControllerPublishUnpublish, using trivial handler csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:39.076376 1 connection.go:182] GRPC call: /csi.v1.Controller/ControllerGetCapabilities csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:39.076380 1 connection.go:183] GRPC request: {} csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:39.087153 1 connection.go:185] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":9}}}]} csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:39.089182 1 connection.go:186] GRPC error: csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:39.089339 1 controller.go:121] Starting CSI attacher csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:39.089588 1 reflector.go:153] Starting reflector *v1beta1.VolumeAttachment (10m0s) from k8s.io/client-go/informers/factory.go:135 csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:39.089600 1 reflector.go:153] Starting reflector *v1.PersistentVolume (10m0s) from k8s.io/client-go/informers/factory.go:135 csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:39.089616 1 reflector.go:188] Listing and watching *v1.PersistentVolume from k8s.io/client-go/informers/factory.go:135 csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:39.089609 1 reflector.go:188] Listing and watching *v1beta1.VolumeAttachment from k8s.io/client-go/informers/factory.go:135 csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:39.189481 1 shared_informer.go:227] caches populated pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpath-provisioner-0 Running: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST } {Ready True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:39 +0200 CEST } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:39 +0200 CEST } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST }] csi-provisioner: RUNNING pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpathplugin-0 Running: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST } {Ready True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:39 +0200 CEST } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:39 +0200 CEST } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST }] hostpath: RUNNING liveness-probe: RUNNING node-driver-registrar: RUNNING csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: W0615 13:59:39.221199 1 deprecatedflags.go:53] Warning: option provisioner="csi-hostpath-e2e-snapshotting-7614" is deprecated and has no effect csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.221267 1 feature_gate.go:243] feature gates: &{map[Topology:true]} csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.221287 1 csi-provisioner.go:107] Version: v1.6.0-0-g321fa5c1c csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.221302 1 csi-provisioner.go:121] Building kube configs for running in cluster... csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.232418 1 connection.go:153] Connecting to unix:///csi/csi.sock csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.233936 1 common.go:111] Probing CSI driver for readiness csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.233958 1 connection.go:182] GRPC call: /csi.v1.Identity/Probe csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.233963 1 connection.go:183] GRPC request: {} csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.236410 1 connection.go:185] GRPC response: {} csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.236834 1 connection.go:186] GRPC error: csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.236844 1 connection.go:182] GRPC call: /csi.v1.Identity/GetPluginInfo csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.236848 1 connection.go:183] GRPC request: {} csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.238188 1 connection.go:185] GRPC response: {"name":"csi-hostpath-e2e-snapshotting-7614","vendor_version":"v1.4.0-rc2-0-gee6beeaf"} csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.238577 1 connection.go:186] GRPC error: csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.238588 1 csi-provisioner.go:163] Detected CSI driver csi-hostpath-e2e-snapshotting-7614 csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: W0615 13:59:39.238593 1 metrics.go:142] metrics endpoint will not be started because `metrics-address` was not specified. csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.238600 1 connection.go:182] GRPC call: /csi.v1.Identity/GetPluginCapabilities csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.238603 1 connection.go:183] GRPC request: {} csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.240622 1 connection.go:185] GRPC response: {"capabilities":[{"Type":{"Service":{"type":1}}},{"Type":{"Service":{"type":2}}}]} csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.241629 1 connection.go:186] GRPC error: csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.241638 1 connection.go:182] GRPC call: /csi.v1.Controller/ControllerGetCapabilities csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.241643 1 connection.go:183] GRPC request: {} csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.246094 1 connection.go:185] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":9}}}]} csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.250116 1 connection.go:186] GRPC error: csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.250889 1 controller.go:709] Using saving PVs to API server in background csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.251073 1 reflector.go:153] Starting reflector *v1.StorageClass (1h0m0s) from k8s.io/client-go/informers/factory.go:135 csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.251093 1 reflector.go:188] Listing and watching *v1.StorageClass from k8s.io/client-go/informers/factory.go:135 csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.251346 1 reflector.go:153] Starting reflector *v1beta1.CSINode (1h0m0s) from k8s.io/client-go/informers/factory.go:135 csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.251366 1 reflector.go:188] Listing and watching *v1beta1.CSINode from k8s.io/client-go/informers/factory.go:135 csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.251414 1 reflector.go:153] Starting reflector *v1.Node (1h0m0s) from k8s.io/client-go/informers/factory.go:135 csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.251428 1 reflector.go:188] Listing and watching *v1.Node from k8s.io/client-go/informers/factory.go:135 csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.251879 1 reflector.go:153] Starting reflector *v1.PersistentVolumeClaim (15m0s) from k8s.io/client-go/informers/factory.go:135 csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.251891 1 reflector.go:188] Listing and watching *v1.PersistentVolumeClaim from k8s.io/client-go/informers/factory.go:135 csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.351795 1 shared_informer.go:227] caches populated csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.351829 1 shared_informer.go:227] caches populated csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.351835 1 shared_informer.go:227] caches populated csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.351839 1 shared_informer.go:227] caches populated csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.351851 1 controller.go:799] Starting provisioner controller csi-hostpath-e2e-snapshotting-7614_csi-hostpath-provisioner-0_ae959b1a-a035-492a-9663-6fe80d301e93! csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.351943 1 clone_controller.go:58] Starting CloningProtection controller csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.351990 1 clone_controller.go:74] Started CloningProtection controller csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.352113 1 reflector.go:153] Starting reflector *v1.PersistentVolumeClaim (15m0s) from github.com/kubernetes-csi/external-provisioner/pkg/controller/clone_controller.go:72 csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.352123 1 reflector.go:188] Listing and watching *v1.PersistentVolumeClaim from github.com/kubernetes-csi/external-provisioner/pkg/controller/clone_controller.go:72 csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.352217 1 volume_store.go:97] Starting save volume queue csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.352341 1 reflector.go:153] Starting reflector *v1.PersistentVolume (15m0s) from sigs.k8s.io/sig-storage-lib-external-provisioner/v5/controller/controller.go:833 csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.352349 1 reflector.go:188] Listing and watching *v1.PersistentVolume from sigs.k8s.io/sig-storage-lib-external-provisioner/v5/controller/controller.go:833 csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.352558 1 reflector.go:153] Starting reflector *v1.StorageClass (15m0s) from sigs.k8s.io/sig-storage-lib-external-provisioner/v5/controller/controller.go:836 csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.352572 1 reflector.go:188] Listing and watching *v1.StorageClass from sigs.k8s.io/sig-storage-lib-external-provisioner/v5/controller/controller.go:836 csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.452797 1 shared_informer.go:227] caches populated csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.452946 1 controller.go:1284] provision "e2e-snapshotting-7614/pvc-glxfp" class "e2e-snapshotting-7614-csi-hostpath-e2e-snapshotting-7614-st46sw": started csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.452954 1 controller.go:848] Started provisioner controller csi-hostpath-e2e-snapshotting-7614_csi-hostpath-provisioner-0_ae959b1a-a035-492a-9663-6fe80d301e93! csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: W0615 13:59:39.456715 1 topology.go:343] No topology keys found on any node csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.456786 1 event.go:281] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"e2e-snapshotting-7614", Name:"pvc-glxfp", UID:"a5748dab-0e0d-4d15-94d7-6cef2d40befb", APIVersion:"v1", ResourceVersion:"1086607", FieldPath:""}): type: 'Normal' reason: 'Provisioning' External provisioner is provisioning volume for claim "e2e-snapshotting-7614/pvc-glxfp" csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.456737 1 controller.go:562] CreateVolumeRequest {Name:pvc-a5748dab-0e0d-4d15-94d7-6cef2d40befb CapacityRange:required_bytes:1048576 VolumeCapabilities:[mount: access_mode: ] Parameters:map[] Secrets:map[] VolumeContentSource: AccessibilityRequirements: XXX_NoUnkeyedLiteral:{} XXX_unrecognized:[] XXX_sizecache:0} csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.457038 1 connection.go:182] GRPC call: /csi.v1.Controller/CreateVolume csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.457045 1 connection.go:183] GRPC request: {"capacity_range":{"required_bytes":1048576},"name":"pvc-a5748dab-0e0d-4d15-94d7-6cef2d40befb","volume_capabilities":[{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":1}}]} csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.467950 1 connection.go:185] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.hostpath.csi/node":"worker-0-1"}}],"capacity_bytes":1048576,"volume_id":"745ac6fc-af10-11ea-bb75-0ab98b83000d"}} csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.469113 1 connection.go:186] GRPC error: csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.469133 1 controller.go:644] create volume rep: {CapacityBytes:1048576 VolumeId:745ac6fc-af10-11ea-bb75-0ab98b83000d VolumeContext:map[] ContentSource: AccessibleTopology:[segments: ] XXX_NoUnkeyedLiteral:{} XXX_unrecognized:[] XXX_sizecache:0} csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.469195 1 controller.go:726] successfully created PV pvc-a5748dab-0e0d-4d15-94d7-6cef2d40befb for PVC pvc-glxfp and csi volume name 745ac6fc-af10-11ea-bb75-0ab98b83000d csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.469206 1 controller.go:742] successfully created PV {GCEPersistentDisk:nil AWSElasticBlockStore:nil HostPath:nil Glusterfs:nil NFS:nil RBD:nil ISCSI:nil Cinder:nil CephFS:nil FC:nil Flocker:nil FlexVolume:nil AzureFile:nil VsphereVolume:nil Quobyte:nil AzureDisk:nil PhotonPersistentDisk:nil PortworxVolume:nil ScaleIO:nil Local:nil StorageOS:nil CSI:&CSIPersistentVolumeSource{Driver:csi-hostpath-e2e-snapshotting-7614,VolumeHandle:745ac6fc-af10-11ea-bb75-0ab98b83000d,ReadOnly:false,FSType:ext4,VolumeAttributes:map[string]string{storage.kubernetes.io/csiProvisionerIdentity: 1592229579250-8081-csi-hostpath-e2e-snapshotting-7614,},ControllerPublishSecretRef:nil,NodeStageSecretRef:nil,NodePublishSecretRef:nil,ControllerExpandSecretRef:nil,}} csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.469434 1 controller.go:1392] provision "e2e-snapshotting-7614/pvc-glxfp" class "e2e-snapshotting-7614-csi-hostpath-e2e-snapshotting-7614-st46sw": volume "pvc-a5748dab-0e0d-4d15-94d7-6cef2d40befb" provisioned csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.469457 1 controller.go:1409] provision "e2e-snapshotting-7614/pvc-glxfp" class "e2e-snapshotting-7614-csi-hostpath-e2e-snapshotting-7614-st46sw": succeeded csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.469463 1 volume_store.go:154] Saving volume pvc-a5748dab-0e0d-4d15-94d7-6cef2d40befb csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.478190 1 volume_store.go:157] Volume pvc-a5748dab-0e0d-4d15-94d7-6cef2d40befb saved csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.478265 1 controller.go:1045] Claim processing succeeded, removing PVC a5748dab-0e0d-4d15-94d7-6cef2d40befb from claims in progress csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 13:59:39.478368 1 event.go:281] Event(v1.ObjectReference{Kind:"PersistentVolumeClaim", Namespace:"e2e-snapshotting-7614", Name:"pvc-glxfp", UID:"a5748dab-0e0d-4d15-94d7-6cef2d40befb", APIVersion:"v1", ResourceVersion:"1086607", FieldPath:""}): type: 'Normal' reason: 'ProvisioningSucceeded' Successfully provisioned volume pvc-a5748dab-0e0d-4d15-94d7-6cef2d40befb Jun 15 15:59:40.006: INFO: PersistentVolumeClaim pvc-glxfp found and phase=Bound (2.462549795s) STEP: checking the claim csi-hostpathplugin-0/node-driver-registrar@worker-0-1: I0615 13:59:38.434142 1 main.go:110] Version: v1.3.0-0-g6e9fff3e csi-hostpathplugin-0/node-driver-registrar@worker-0-1: I0615 13:59:38.434234 1 main.go:120] Attempting to open a gRPC connection with: "/csi/csi.sock" csi-hostpathplugin-0/node-driver-registrar@worker-0-1: I0615 13:59:38.434258 1 connection.go:151] Connecting to unix:///csi/csi.sock csi-hostpathplugin-0/node-driver-registrar@worker-0-1: I0615 13:59:39.434882 1 main.go:127] Calling CSI driver to discover driver name csi-hostpathplugin-0/node-driver-registrar@worker-0-1: I0615 13:59:39.434972 1 connection.go:180] GRPC call: /csi.v1.Identity/GetPluginInfo csi-hostpathplugin-0/node-driver-registrar@worker-0-1: I0615 13:59:39.434980 1 connection.go:181] GRPC request: {} csi-hostpathplugin-0/node-driver-registrar@worker-0-1: I0615 13:59:39.438178 1 connection.go:183] GRPC response: {"name":"csi-hostpath-e2e-snapshotting-7614","vendor_version":"v1.4.0-rc2-0-gee6beeaf"} csi-hostpathplugin-0/node-driver-registrar@worker-0-1: I0615 13:59:39.439002 1 connection.go:184] GRPC error: csi-hostpathplugin-0/node-driver-registrar@worker-0-1: I0615 13:59:39.439011 1 main.go:137] CSI driver name: "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/node-driver-registrar@worker-0-1: I0615 13:59:39.439064 1 node_register.go:51] Starting Registration Server at: /registration/csi-hostpath-e2e-snapshotting-7614-reg.sock csi-hostpathplugin-0/node-driver-registrar@worker-0-1: I0615 13:59:39.439306 1 node_register.go:60] Registration Server started at: /registration/csi-hostpath-e2e-snapshotting-7614-reg.sock csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:38.840530 1 hostpath.go:117] Driver: csi-hostpath-e2e-snapshotting-7614 csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:38.840613 1 hostpath.go:118] Version: v1.4.0-rc2-0-gee6beeaf csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:38.840631 1 controllerserver.go:565] Enabling controller service capability: CREATE_DELETE_VOLUME csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:38.840637 1 controllerserver.go:565] Enabling controller service capability: CREATE_DELETE_SNAPSHOT csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:38.840640 1 controllerserver.go:565] Enabling controller service capability: LIST_SNAPSHOTS csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:38.840643 1 controllerserver.go:565] Enabling controller service capability: CLONE_VOLUME csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:38.840647 1 controllerserver.go:565] Enabling controller service capability: EXPAND_VOLUME csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:38.840983 1 server.go:100] Listening for connections on address: &net.UnixAddr{Name:"//csi/csi.sock", Net:"unix"} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.055933 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.055961 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.057544 1 server.go:123] GRPC response: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.059991 1 server.go:117] GRPC call: /csi.v1.Identity/GetPluginInfo csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.060005 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.060336 1 identityserver.go:40] Using default GetPluginInfo csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.060341 1 server.go:123] GRPC response: {"name":"csi-hostpath-e2e-snapshotting-7614","vendor_version":"v1.4.0-rc2-0-gee6beeaf"} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.064962 1 server.go:117] GRPC call: /csi.v1.Identity/GetPluginCapabilities csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.064982 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.065563 1 identityserver.go:61] Using default capabilities csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.065573 1 server.go:123] GRPC response: {"capabilities":[{"Type":{"Service":{"type":1}}},{"Type":{"Service":{"type":2}}}]} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.069288 1 server.go:117] GRPC call: /csi.v1.Controller/ControllerGetCapabilities csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.069310 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.069695 1 server.go:123] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":9}}}]} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.082872 1 server.go:117] GRPC call: /csi.v1.Controller/ControllerGetCapabilities csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.082896 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.084290 1 server.go:123] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":9}}}]} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.096604 1 server.go:117] GRPC call: /csi.v1.Identity/GetPluginInfo csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.096623 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.097014 1 identityserver.go:40] Using default GetPluginInfo csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.097020 1 server.go:123] GRPC response: {"name":"csi-hostpath-e2e-snapshotting-7614","vendor_version":"v1.4.0-rc2-0-gee6beeaf"} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.234997 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.235016 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.235474 1 server.go:123] GRPC response: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.237321 1 server.go:117] GRPC call: /csi.v1.Identity/GetPluginInfo csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.237334 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.237710 1 identityserver.go:40] Using default GetPluginInfo csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.237715 1 server.go:123] GRPC response: {"name":"csi-hostpath-e2e-snapshotting-7614","vendor_version":"v1.4.0-rc2-0-gee6beeaf"} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.239214 1 server.go:117] GRPC call: /csi.v1.Identity/GetPluginCapabilities csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.239227 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.239569 1 identityserver.go:61] Using default capabilities csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.239575 1 server.go:123] GRPC response: {"capabilities":[{"Type":{"Service":{"type":1}}},{"Type":{"Service":{"type":2}}}]} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.242118 1 server.go:117] GRPC call: /csi.v1.Controller/ControllerGetCapabilities csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.242130 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.243161 1 server.go:123] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":9}}}]} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.426694 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.426713 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.427083 1 server.go:123] GRPC response: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.428808 1 server.go:117] GRPC call: /csi.v1.Identity/GetPluginInfo csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.428821 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.429172 1 identityserver.go:40] Using default GetPluginInfo csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.429178 1 server.go:123] GRPC response: {"name":"csi-hostpath-e2e-snapshotting-7614","vendor_version":"v1.4.0-rc2-0-gee6beeaf"} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.431744 1 server.go:117] GRPC call: /csi.v1.Identity/GetPluginCapabilities csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.431756 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.432099 1 identityserver.go:61] Using default capabilities csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.432105 1 server.go:123] GRPC response: {"capabilities":[{"Type":{"Service":{"type":1}}},{"Type":{"Service":{"type":2}}}]} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.435472 1 server.go:117] GRPC call: /csi.v1.Controller/ControllerGetCapabilities csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.435484 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.436068 1 server.go:117] GRPC call: /csi.v1.Identity/GetPluginInfo csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.436612 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.437650 1 identityserver.go:40] Using default GetPluginInfo csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.437666 1 server.go:123] GRPC response: {"name":"csi-hostpath-e2e-snapshotting-7614","vendor_version":"v1.4.0-rc2-0-gee6beeaf"} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.435893 1 server.go:123] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":9}}}]} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.464873 1 server.go:117] GRPC call: /csi.v1.Controller/CreateVolume csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.464893 1 server.go:118] GRPC request: {"capacity_range":{"required_bytes":1048576},"name":"pvc-a5748dab-0e0d-4d15-94d7-6cef2d40befb","volume_capabilities":[{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":1}}]} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.466630 1 controllerserver.go:165] created volume 745ac6fc-af10-11ea-bb75-0ab98b83000d at path /csi-data-dir/745ac6fc-af10-11ea-bb75-0ab98b83000d csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.466645 1 server.go:123] GRPC response: {"volume":{"accessible_topology":[{"segments":{"topology.hostpath.csi/node":"worker-0-1"}}],"capacity_bytes":1048576,"volume_id":"745ac6fc-af10-11ea-bb75-0ab98b83000d"}} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.498449 1 server.go:117] GRPC call: /csi.v1.Identity/GetPluginInfo csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.498474 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.499396 1 identityserver.go:40] Using default GetPluginInfo csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.499404 1 server.go:123] GRPC response: {"name":"csi-hostpath-e2e-snapshotting-7614","vendor_version":"v1.4.0-rc2-0-gee6beeaf"} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.502799 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.502812 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.503186 1 server.go:123] GRPC response: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.504650 1 server.go:117] GRPC call: /csi.v1.Controller/ControllerGetCapabilities csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.504838 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:39.505420 1 server.go:123] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":9}}}]} csi-hostpathplugin-0/node-driver-registrar@worker-0-1: I0615 13:59:40.114858 1 main.go:77] Received GetInfo call: &InfoRequest{} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:40.116173 1 server.go:117] GRPC call: /csi.v1.Node/NodeGetInfo csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:40.116199 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:40.116987 1 server.go:123] GRPC response: {"accessible_topology":{"segments":{"topology.hostpath.csi/node":"worker-0-1"}},"max_volumes_per_node":10,"node_id":"worker-0-1"} csi-hostpathplugin-0/node-driver-registrar@worker-0-1: I0615 13:59:40.138971 1 main.go:87] Received NotifyRegistrationStatus call: &RegistrationStatus{PluginRegistered:true,Error:,} STEP: creating a SnapshotClass csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 13:59:39.094878 1 connection.go:151] Connecting to unix:///csi/csi.sock csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 13:59:39.096035 1 main.go:86] calling CSI driver to discover driver name csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 13:59:39.097680 1 main.go:91] CSI driver name: "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 13:59:39.097697 1 main.go:100] Serving requests to /healthz on: 0.0.0.0:9898 pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpath-resizer-0 Running: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST } {Ready True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:40 +0200 CEST } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:40 +0200 CEST } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST }] csi-resizer: RUNNING pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpath-snapshotter-0 Running: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST } {Ready True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:40 +0200 CEST } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:40 +0200 CEST } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST }] csi-snapshotter: RUNNING STEP: creating a snapshot Jun 15 15:59:40.733: INFO: Waiting up to 5m0s for VolumeSnapshot snapshot-f5ghg to become ready csi-hostpath-resizer-0/csi-resizer@worker-0-1: I0615 13:59:39.421138 1 main.go:68] Version : v0.5.0-0-g6a95232 csi-hostpath-resizer-0/csi-resizer@worker-0-1: I0615 13:59:39.423023 1 connection.go:153] Connecting to unix:///csi/csi.sock csi-hostpath-resizer-0/csi-resizer@worker-0-1: I0615 13:59:39.423979 1 common.go:111] Probing CSI driver for readiness csi-hostpath-resizer-0/csi-resizer@worker-0-1: I0615 13:59:39.423998 1 connection.go:182] GRPC call: /csi.v1.Identity/Probe csi-hostpath-resizer-0/csi-resizer@worker-0-1: I0615 13:59:39.424003 1 connection.go:183] GRPC request: {} csi-hostpath-resizer-0/csi-resizer@worker-0-1: I0615 13:59:39.427778 1 connection.go:185] GRPC response: {} csi-hostpath-resizer-0/csi-resizer@worker-0-1: I0615 13:59:39.428244 1 connection.go:186] GRPC error: csi-hostpath-resizer-0/csi-resizer@worker-0-1: I0615 13:59:39.428259 1 connection.go:182] GRPC call: /csi.v1.Identity/GetPluginInfo csi-hostpath-resizer-0/csi-resizer@worker-0-1: I0615 13:59:39.428263 1 connection.go:183] GRPC request: {} csi-hostpath-resizer-0/csi-resizer@worker-0-1: I0615 13:59:39.430847 1 connection.go:185] GRPC response: {"name":"csi-hostpath-e2e-snapshotting-7614","vendor_version":"v1.4.0-rc2-0-gee6beeaf"} csi-hostpath-resizer-0/csi-resizer@worker-0-1: I0615 13:59:39.431212 1 connection.go:186] GRPC error: csi-hostpath-resizer-0/csi-resizer@worker-0-1: I0615 13:59:39.431224 1 csi_resizer.go:79] CSI driver name: "csi-hostpath-e2e-snapshotting-7614" csi-hostpath-resizer-0/csi-resizer@worker-0-1: W0615 13:59:39.431229 1 metrics.go:142] metrics endpoint will not be started because `metrics-address` was not specified. csi-hostpath-resizer-0/csi-resizer@worker-0-1: I0615 13:59:39.431236 1 connection.go:182] GRPC call: /csi.v1.Identity/GetPluginCapabilities csi-hostpath-resizer-0/csi-resizer@worker-0-1: I0615 13:59:39.431240 1 connection.go:183] GRPC request: {} csi-hostpath-resizer-0/csi-resizer@worker-0-1: I0615 13:59:39.433305 1 connection.go:185] GRPC response: {"capabilities":[{"Type":{"Service":{"type":1}}},{"Type":{"Service":{"type":2}}}]} csi-hostpath-resizer-0/csi-resizer@worker-0-1: I0615 13:59:39.434479 1 connection.go:186] GRPC error: Jun 15 15:59:40.938: INFO: VolumeSnapshot snapshot-f5ghg found but is not ready. csi-hostpath-resizer-0/csi-resizer@worker-0-1: I0615 13:59:39.434491 1 connection.go:182] GRPC call: /csi.v1.Controller/ControllerGetCapabilities csi-hostpath-resizer-0/csi-resizer@worker-0-1: I0615 13:59:39.434495 1 connection.go:183] GRPC request: {} csi-hostpath-resizer-0/csi-resizer@worker-0-1: I0615 13:59:39.439025 1 connection.go:185] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":9}}}]} csi-hostpath-resizer-0/csi-resizer@worker-0-1: I0615 13:59:39.443133 1 connection.go:186] GRPC error: csi-hostpath-resizer-0/csi-resizer@worker-0-1: I0615 13:59:39.443328 1 controller.go:190] Starting external resizer csi-hostpath-e2e-snapshotting-7614 csi-hostpath-resizer-0/csi-resizer@worker-0-1: I0615 13:59:39.443562 1 reflector.go:153] Starting reflector *v1.PersistentVolumeClaim (10m0s) from k8s.io/client-go/informers/factory.go:135 csi-hostpath-resizer-0/csi-resizer@worker-0-1: I0615 13:59:39.443728 1 reflector.go:188] Listing and watching *v1.PersistentVolumeClaim from k8s.io/client-go/informers/factory.go:135 csi-hostpath-resizer-0/csi-resizer@worker-0-1: I0615 13:59:39.443819 1 reflector.go:153] Starting reflector *v1.PersistentVolume (10m0s) from k8s.io/client-go/informers/factory.go:135 csi-hostpath-resizer-0/csi-resizer@worker-0-1: I0615 13:59:39.443841 1 reflector.go:188] Listing and watching *v1.PersistentVolume from k8s.io/client-go/informers/factory.go:135 csi-hostpath-resizer-0/csi-resizer@worker-0-1: I0615 13:59:39.543501 1 shared_informer.go:227] caches populated csi-hostpath-resizer-0/csi-resizer@worker-0-1: I0615 13:59:39.543598 1 controller.go:225] Started PVC processing "e2e-snapshotting-7614/pvc-glxfp" csi-hostpath-resizer-0/csi-resizer@worker-0-1: I0615 13:59:39.543635 1 controller.go:244] No need to resize PVC "e2e-snapshotting-7614/pvc-glxfp" csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:39.487259 1 main.go:87] Version: v2.1.0-0-ged6cd4a7 csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:39.491182 1 connection.go:153] Connecting to unix:///csi/csi.sock csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:39.491906 1 connection.go:182] GRPC call: /csi.v1.Identity/GetPluginInfo csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:39.491924 1 connection.go:183] GRPC request: {} csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:39.501378 1 connection.go:185] GRPC response: {"name":"csi-hostpath-e2e-snapshotting-7614","vendor_version":"v1.4.0-rc2-0-gee6beeaf"} csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:39.502023 1 connection.go:186] GRPC error: csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:39.502036 1 main.go:136] CSI driver name: "csi-hostpath-e2e-snapshotting-7614" csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: W0615 13:59:39.502046 1 metrics.go:142] metrics endpoint will not be started because `metrics-address` was not specified. csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:39.502053 1 common.go:111] Probing CSI driver for readiness csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:39.502068 1 connection.go:182] GRPC call: /csi.v1.Identity/Probe csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:39.502074 1 connection.go:183] GRPC request: {} csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:39.503717 1 connection.go:185] GRPC response: {} csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:39.504133 1 connection.go:186] GRPC error: csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:39.504141 1 connection.go:182] GRPC call: /csi.v1.Controller/ControllerGetCapabilities csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:39.504145 1 connection.go:183] GRPC request: {} csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:39.509038 1 connection.go:185] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":5}}},{"Type":{"Rpc":{"type":6}}},{"Type":{"Rpc":{"type":7}}},{"Type":{"Rpc":{"type":9}}}]} csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:39.515533 1 connection.go:186] GRPC error: csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:39.515551 1 main.go:162] Start NewCSISnapshotSideCarController with snapshotter [csi-hostpath-e2e-snapshotting-7614] kubeconfig [] csiTimeout [1m0s] csiAddress [/csi/csi.sock] resyncPeriod [1m0s] snapshotNamePrefix [snapshot] snapshotNameUUIDLength [824636400536] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:39.515942 1 snapshot_controller_base.go:115] Starting CSI snapshotter csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:39.516782 1 reflector.go:153] Starting reflector *v1beta1.VolumeSnapshotContent (1m0s) from github.com/kubernetes-csi/external-snapshotter/pkg/client/informers/externalversions/factory.go:117 csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:39.516818 1 reflector.go:188] Listing and watching *v1beta1.VolumeSnapshotContent from github.com/kubernetes-csi/external-snapshotter/pkg/client/informers/externalversions/factory.go:117 csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:39.516818 1 reflector.go:153] Starting reflector *v1beta1.VolumeSnapshotClass (1m0s) from github.com/kubernetes-csi/external-snapshotter/pkg/client/informers/externalversions/factory.go:117 csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:39.516844 1 reflector.go:188] Listing and watching *v1beta1.VolumeSnapshotClass from github.com/kubernetes-csi/external-snapshotter/pkg/client/informers/externalversions/factory.go:117 csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:39.616302 1 shared_informer.go:227] caches populated csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:39.616341 1 snapshot_controller_base.go:279] controller initialized csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:41.936958 1 snapshot_controller_base.go:144] enqueued "snapcontent-5716b93d-7a10-4484-b45d-72361ae56279" for sync csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:41.936994 1 snapshot_controller_base.go:159] contentWorker[snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:41.937011 1 util.go:141] storeObjectUpdate: adding content "snapcontent-5716b93d-7a10-4484-b45d-72361ae56279", version 1086689 csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:41.937021 1 snapshot_controller.go:58] synchronizing VolumeSnapshotContent[snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:41.937039 1 snapshot_controller.go:569] Check if VolumeSnapshotContent[snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] should be deleted. csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:41.937044 1 snapshot_controller.go:78] syncContent: Call CreateSnapshot for content snapcontent-5716b93d-7a10-4484-b45d-72361ae56279 csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:41.937051 1 snapshot_controller.go:133] createSnapshot for content [snapcontent-5716b93d-7a10-4484-b45d-72361ae56279]: started csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:41.937056 1 snapshot_controller.go:112] scheduleOperation[create-snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:41.937073 1 snapshot_controller.go:305] createSnapshotWrapper: Creating snapshot for content snapcontent-5716b93d-7a10-4484-b45d-72361ae56279 through the plugin ... csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:41.937080 1 snapshot_controller.go:218] getCSISnapshotInput for content [snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:41.937084 1 snapshot_controller.go:477] getSnapshotClass: VolumeSnapshotClassName [e2e-snapshotting-7614-csi-hostpath-e2e-snapshotting-7614-vsc] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:41.937088 1 snapshot_controller.go:606] setAnnVolumeSnapshotBeingCreated: set annotation [snapshot.storage.kubernetes.io/volumesnapshot-being-created:yes] on content [snapcontent-5716b93d-7a10-4484-b45d-72361ae56279]. csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:41.937873 1 snapshot_controller_base.go:144] enqueued "snapcontent-5716b93d-7a10-4484-b45d-72361ae56279" for sync csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:41.937902 1 snapshot_controller_base.go:159] contentWorker[snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:41.937913 1 util.go:169] storeObjectUpdate updating content "snapcontent-5716b93d-7a10-4484-b45d-72361ae56279" with version 1086690 csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:41.937919 1 snapshot_controller.go:58] synchronizing VolumeSnapshotContent[snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:41.937935 1 snapshot_controller.go:569] Check if VolumeSnapshotContent[snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] should be deleted. csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:41.937939 1 snapshot_controller.go:78] syncContent: Call CreateSnapshot for content snapcontent-5716b93d-7a10-4484-b45d-72361ae56279 csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:41.937945 1 snapshot_controller.go:133] createSnapshot for content [snapcontent-5716b93d-7a10-4484-b45d-72361ae56279]: started csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:41.937949 1 snapshot_controller.go:112] scheduleOperation[create-snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:41.937953 1 snapshot_controller.go:118] operation "create-snapcontent-5716b93d-7a10-4484-b45d-72361ae56279" is already running, skipping csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:41.942641 1 snapshot_controller.go:179] updateContentStatusWithEvent[snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 13:59:41.946697 1 snapshot_controller.go:200] updating VolumeSnapshotContent[snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] error status failed Operation cannot be fulfilled on volumesnapshotcontents.snapshot.storage.k8s.io "snapcontent-5716b93d-7a10-4484-b45d-72361ae56279": the object has been modified; please apply your changes to the latest version and try again csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: E0615 13:59:41.946717 1 snapshot_controller.go:139] createSnapshot [create-snapcontent-5716b93d-7a10-4484-b45d-72361ae56279]: error occurred in createSnapshotWrapper: failed to add VolumeSnapshotBeingCreated annotation on the content snapcontent-5716b93d-7a10-4484-b45d-72361ae56279: "snapshot controller failed to update snapcontent-5716b93d-7a10-4484-b45d-72361ae56279 on API server: Operation cannot be fulfilled on volumesnapshotcontents.snapshot.storage.k8s.io \"snapcontent-5716b93d-7a10-4484-b45d-72361ae56279\": the object has been modified; please apply your changes to the latest version and try again" csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: E0615 13:59:41.946775 1 goroutinemap.go:150] Operation for "create-snapcontent-5716b93d-7a10-4484-b45d-72361ae56279" failed. No retries permitted until 2020-06-15 13:59:42.446732641 +0000 UTC m=+2.974424247 (durationBeforeRetry 500ms). Error: "failed to add VolumeSnapshotBeingCreated annotation on the content snapcontent-5716b93d-7a10-4484-b45d-72361ae56279: \"snapshot controller failed to update snapcontent-5716b93d-7a10-4484-b45d-72361ae56279 on API server: Operation cannot be fulfilled on volumesnapshotcontents.snapshot.storage.k8s.io \\\"snapcontent-5716b93d-7a10-4484-b45d-72361ae56279\\\": the object has been modified; please apply your changes to the latest version and try again\"" Jun 15 15:59:43.192: INFO: VolumeSnapshot snapshot-f5ghg found but is not ready. pod event: MODIFIED: e2e-snapshotting-7614/pvc-snapshottable-tester-9782d Pending: [{PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:44 +0200 CEST }] pod event: MODIFIED: e2e-snapshotting-7614/pvc-snapshottable-tester-9782d Pending: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:44 +0200 CEST } {Ready False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:44 +0200 CEST ContainersNotReady containers with unready status: [volume-tester]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:44 +0200 CEST ContainersNotReady containers with unready status: [volume-tester]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:44 +0200 CEST }] volume-tester: WAITING: ContainerCreating - csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:44.249058 1 controller.go:198] Started VA processing "csi-6bd11b9b560a5e4dfb383fff654d114f70567e04a59b3caf5c3d59ebbc19637e" csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:44.249095 1 trivial_handler.go:53] Trivial sync[csi-6bd11b9b560a5e4dfb383fff654d114f70567e04a59b3caf5c3d59ebbc19637e] started csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:44.249100 1 util.go:35] Marking as attached "csi-6bd11b9b560a5e4dfb383fff654d114f70567e04a59b3caf5c3d59ebbc19637e" csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:44.254341 1 util.go:48] Marked as attached "csi-6bd11b9b560a5e4dfb383fff654d114f70567e04a59b3caf5c3d59ebbc19637e" csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:44.254374 1 trivial_handler.go:61] Marked VolumeAttachment csi-6bd11b9b560a5e4dfb383fff654d114f70567e04a59b3caf5c3d59ebbc19637e as attached csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:44.254405 1 controller.go:198] Started VA processing "csi-6bd11b9b560a5e4dfb383fff654d114f70567e04a59b3caf5c3d59ebbc19637e" csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:44.254413 1 trivial_handler.go:53] Trivial sync[csi-6bd11b9b560a5e4dfb383fff654d114f70567e04a59b3caf5c3d59ebbc19637e] started pod event: MODIFIED: e2e-snapshotting-7614/pvc-snapshottable-tester-9782d Pending: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:44 +0200 CEST } {Ready False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:44 +0200 CEST ContainersNotReady containers with unready status: [volume-tester]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:44 +0200 CEST ContainersNotReady containers with unready status: [volume-tester]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:44 +0200 CEST }] volume-tester: WAITING: ContainerCreating - Jun 15 15:59:45.377: INFO: VolumeSnapshot snapshot-f5ghg found but is not ready. Jun 15 15:59:47.549: INFO: VolumeSnapshot snapshot-f5ghg found but is not ready. csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:49.496292 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:49.496318 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:49.497449 1 server.go:123] GRPC response: {} csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 13:59:49.495968 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 13:59:49.497945 1 main.go:71] Health check succeeded Jun 15 15:59:49.721: INFO: VolumeSnapshot snapshot-f5ghg found but is not ready. csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 13:59:51.495568 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:51.495934 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:51.495962 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 13:59:51.496932 1 main.go:71] Health check succeeded csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:51.496426 1 server.go:123] GRPC response: {} Jun 15 15:59:51.997: INFO: VolumeSnapshot snapshot-f5ghg found but is not ready. csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:51.970916 1 server.go:117] GRPC call: /csi.v1.Node/NodeGetCapabilities csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:51.970945 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:51.971389 1 server.go:123] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":3}}}]} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:51.975936 1 server.go:117] GRPC call: /csi.v1.Node/NodeStageVolume csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:51.975962 1 server.go:118] GRPC request: {"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a5748dab-0e0d-4d15-94d7-6cef2d40befb/globalmount","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":1}},"volume_context":{"storage.kubernetes.io/csiProvisionerIdentity":"1592229579250-8081-csi-hostpath-e2e-snapshotting-7614"},"volume_id":"745ac6fc-af10-11ea-bb75-0ab98b83000d"} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:51.977223 1 server.go:123] GRPC response: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:51.980706 1 server.go:117] GRPC call: /csi.v1.Node/NodeGetCapabilities csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:51.980720 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:51.981158 1 server.go:123] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":3}}}]} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:51.988239 1 server.go:117] GRPC call: /csi.v1.Node/NodePublishVolume csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:51.988254 1 server.go:118] GRPC request: {"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a5748dab-0e0d-4d15-94d7-6cef2d40befb/globalmount","target_path":"/var/lib/kubelet/pods/033eb98b-31e5-49f0-8ddf-dd2d0d43bc7d/volumes/kubernetes.io~csi/pvc-a5748dab-0e0d-4d15-94d7-6cef2d40befb/mount","volume_capability":{"AccessType":{"Mount":{"fs_type":"ext4"}},"access_mode":{"mode":1}},"volume_context":{"csi.storage.k8s.io/ephemeral":"false","csi.storage.k8s.io/pod.name":"pvc-snapshottable-tester-9782d","csi.storage.k8s.io/pod.namespace":"e2e-snapshotting-7614","csi.storage.k8s.io/pod.uid":"033eb98b-31e5-49f0-8ddf-dd2d0d43bc7d","csi.storage.k8s.io/serviceAccount.name":"default","storage.kubernetes.io/csiProvisionerIdentity":"1592229579250-8081-csi-hostpath-e2e-snapshotting-7614"},"volume_id":"745ac6fc-af10-11ea-bb75-0ab98b83000d"} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:51.989524 1 mount_linux.go:164] Detected OS without systemd csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:51.990258 1 nodeserver.go:166] target /var/lib/kubelet/pods/033eb98b-31e5-49f0-8ddf-dd2d0d43bc7d/volumes/kubernetes.io~csi/pvc-a5748dab-0e0d-4d15-94d7-6cef2d40befb/mount csi-hostpathplugin-0/hostpath@worker-0-1: fstype ext4 csi-hostpathplugin-0/hostpath@worker-0-1: device csi-hostpathplugin-0/hostpath@worker-0-1: readonly false csi-hostpathplugin-0/hostpath@worker-0-1: volumeId 745ac6fc-af10-11ea-bb75-0ab98b83000d csi-hostpathplugin-0/hostpath@worker-0-1: attributes map[csi.storage.k8s.io/ephemeral:false csi.storage.k8s.io/pod.name:pvc-snapshottable-tester-9782d csi.storage.k8s.io/pod.namespace:e2e-snapshotting-7614 csi.storage.k8s.io/pod.uid:033eb98b-31e5-49f0-8ddf-dd2d0d43bc7d csi.storage.k8s.io/serviceAccount.name:default storage.kubernetes.io/csiProvisionerIdentity:1592229579250-8081-csi-hostpath-e2e-snapshotting-7614] csi-hostpathplugin-0/hostpath@worker-0-1: mountflags [] csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:51.990310 1 mount_linux.go:164] Detected OS without systemd csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:51.990323 1 mount_linux.go:146] Mounting cmd (mount) with arguments ([-o bind /csi-data-dir/745ac6fc-af10-11ea-bb75-0ab98b83000d /var/lib/kubelet/pods/033eb98b-31e5-49f0-8ddf-dd2d0d43bc7d/volumes/kubernetes.io~csi/pvc-a5748dab-0e0d-4d15-94d7-6cef2d40befb/mount]) csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:51.992268 1 mount_linux.go:146] Mounting cmd (mount) with arguments ([-o bind,remount /csi-data-dir/745ac6fc-af10-11ea-bb75-0ab98b83000d /var/lib/kubelet/pods/033eb98b-31e5-49f0-8ddf-dd2d0d43bc7d/volumes/kubernetes.io~csi/pvc-a5748dab-0e0d-4d15-94d7-6cef2d40befb/mount]) csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:51.993427 1 server.go:123] GRPC response: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:53.494343 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 13:59:53.494054 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 13:59:53.495207 1 main.go:71] Health check succeeded csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:53.494368 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:53.494789 1 server.go:123] GRPC response: {} Jun 15 15:59:54.256: INFO: VolumeSnapshot snapshot-f5ghg found but is not ready. pod event: MODIFIED: e2e-snapshotting-7614/pvc-snapshottable-tester-9782d Pending: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:44 +0200 CEST } {Ready False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:44 +0200 CEST ContainersNotReady containers with unready status: [volume-tester]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:44 +0200 CEST ContainersNotReady containers with unready status: [volume-tester]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:44 +0200 CEST }] volume-tester: WAITING: ContainerCreating - pod event: MODIFIED: e2e-snapshotting-7614/pvc-snapshottable-tester-9782d Succeeded: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:44 +0200 CEST PodCompleted } {Ready False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:44 +0200 CEST PodCompleted } {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:44 +0200 CEST PodCompleted } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:44 +0200 CEST }] volume-tester: TERMINATED: Completed - csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:55.496603 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 13:59:55.496326 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 13:59:55.498381 1 main.go:71] Health check succeeded csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:55.496796 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:55.497260 1 server.go:123] GRPC response: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:55.584600 1 server.go:117] GRPC call: /csi.v1.Node/NodeUnpublishVolume csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:55.584627 1 server.go:118] GRPC request: {"target_path":"/var/lib/kubelet/pods/033eb98b-31e5-49f0-8ddf-dd2d0d43bc7d/volumes/kubernetes.io~csi/pvc-a5748dab-0e0d-4d15-94d7-6cef2d40befb/mount","volume_id":"745ac6fc-af10-11ea-bb75-0ab98b83000d"} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:55.587584 1 mount_linux.go:164] Detected OS without systemd csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:55.588845 1 mount_linux.go:164] Detected OS without systemd csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:55.588863 1 mount_linux.go:211] Unmounting /var/lib/kubelet/pods/033eb98b-31e5-49f0-8ddf-dd2d0d43bc7d/volumes/kubernetes.io~csi/pvc-a5748dab-0e0d-4d15-94d7-6cef2d40befb/mount csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:55.597880 1 nodeserver.go:225] hostpath: volume /var/lib/kubelet/pods/033eb98b-31e5-49f0-8ddf-dd2d0d43bc7d/volumes/kubernetes.io~csi/pvc-a5748dab-0e0d-4d15-94d7-6cef2d40befb/mount has been unpublished. csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:55.597904 1 server.go:123] GRPC response: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:55.690973 1 server.go:117] GRPC call: /csi.v1.Node/NodeGetCapabilities csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:55.690999 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:55.691690 1 server.go:123] GRPC response: {"capabilities":[{"Type":{"Rpc":{"type":1}}},{"Type":{"Rpc":{"type":3}}}]} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:55.700106 1 server.go:117] GRPC call: /csi.v1.Node/NodeUnstageVolume csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:55.700116 1 server.go:118] GRPC request: {"staging_target_path":"/var/lib/kubelet/plugins/kubernetes.io/csi/pv/pvc-a5748dab-0e0d-4d15-94d7-6cef2d40befb/globalmount","volume_id":"745ac6fc-af10-11ea-bb75-0ab98b83000d"} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:55.700450 1 server.go:123] GRPC response: {} Jun 15 15:59:56.504: INFO: VolumeSnapshot snapshot-f5ghg found but is not ready. csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 13:59:57.494213 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:57.494507 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:57.494537 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:57.495154 1 server.go:123] GRPC response: {} csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 13:59:57.496086 1 main.go:71] Health check succeeded Jun 15 15:59:58.674: INFO: VolumeSnapshot snapshot-f5ghg found but is not ready. csi-hostpath-attacher-0/csi-attacher@worker-0-1: I0615 13:59:59.095561 1 controller.go:232] Started PV processing "pvc-a5748dab-0e0d-4d15-94d7-6cef2d40befb" csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 13:59:59.494271 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:59.494543 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 13:59:59.495480 1 main.go:71] Health check succeeded csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:59.494568 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 13:59:59.494982 1 server.go:123] GRPC response: {} Jun 15 16:00:00.906: INFO: VolumeSnapshot snapshot-f5ghg found but is not ready. csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:01.494729 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:01.494358 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:01.496450 1 main.go:71] Health check succeeded csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:01.494939 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:01.495638 1 server.go:123] GRPC response: {} Jun 15 16:00:03.078: INFO: VolumeSnapshot snapshot-f5ghg found but is not ready. csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:03.498036 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:03.498056 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:03.497714 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:03.498410 1 server.go:123] GRPC response: {} csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:03.498885 1 main.go:71] Health check succeeded Jun 15 16:00:05.286: INFO: VolumeSnapshot snapshot-f5ghg found but is not ready. csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:05.494664 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:05.494854 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:05.495274 1 server.go:123] GRPC response: {} csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:05.494407 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:05.495736 1 main.go:71] Health check succeeded Jun 15 16:00:07.469: INFO: VolumeSnapshot snapshot-f5ghg found but is not ready. csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:07.494461 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:07.494168 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:07.494485 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:07.495564 1 main.go:71] Health check succeeded csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:07.494889 1 server.go:123] GRPC response: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:09.498137 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:09.497888 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:09.498161 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:09.499459 1 server.go:123] GRPC response: {} csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:09.499966 1 main.go:71] Health check succeeded Jun 15 16:00:09.639: INFO: VolumeSnapshot snapshot-f5ghg found but is not ready. csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:11.494519 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:11.494538 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:11.494281 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:11.495328 1 main.go:71] Health check succeeded csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:11.494919 1 server.go:123] GRPC response: {} Jun 15 16:00:11.864: INFO: VolumeSnapshot snapshot-f5ghg found but is not ready. csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:13.494146 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:13.494449 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:13.494483 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:13.495128 1 server.go:123] GRPC response: {} csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:13.495784 1 main.go:71] Health check succeeded Jun 15 16:00:14.118: INFO: VolumeSnapshot snapshot-f5ghg found but is not ready. csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:15.494538 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:15.494562 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:15.494290 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:15.495057 1 server.go:123] GRPC response: {} csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:15.496212 1 main.go:71] Health check succeeded Jun 15 16:00:16.312: INFO: VolumeSnapshot snapshot-f5ghg found but is not ready. csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:17.494149 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:17.494416 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:17.494445 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:17.495339 1 main.go:71] Health check succeeded csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:17.494888 1 server.go:123] GRPC response: {} Jun 15 16:00:18.496: INFO: VolumeSnapshot snapshot-f5ghg found but is not ready. csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:19.494627 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:19.494390 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:19.494654 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:19.495608 1 main.go:71] Health check succeeded csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:19.495161 1 server.go:123] GRPC response: {} Jun 15 16:00:20.771: INFO: VolumeSnapshot snapshot-f5ghg found but is not ready. csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:21.494470 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:21.494488 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:21.494915 1 server.go:123] GRPC response: {} csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:21.494224 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:21.495352 1 main.go:71] Health check succeeded Jun 15 16:00:23.002: INFO: VolumeSnapshot snapshot-f5ghg found but is not ready. csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:23.496153 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:23.496514 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:23.496542 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:23.497216 1 server.go:123] GRPC response: {} csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:23.500096 1 main.go:71] Health check succeeded Jun 15 16:00:25.180: INFO: VolumeSnapshot snapshot-f5ghg found but is not ready. csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:25.494545 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:25.494254 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:25.494569 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:25.495024 1 server.go:123] GRPC response: {} csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:25.495459 1 main.go:71] Health check succeeded Jun 15 16:00:27.349: INFO: VolumeSnapshot snapshot-f5ghg found but is not ready. csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:27.494613 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:27.494641 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:27.494291 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:27.495197 1 server.go:123] GRPC response: {} csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:27.495669 1 main.go:71] Health check succeeded csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:29.494271 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:29.494620 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:29.494643 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:29.495076 1 server.go:123] GRPC response: {} Jun 15 16:00:29.580: INFO: VolumeSnapshot snapshot-f5ghg found but is not ready. csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:29.495515 1 main.go:71] Health check succeeded csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:31.494540 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:31.494274 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:31.496325 1 main.go:71] Health check succeeded csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:31.494664 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:31.495744 1 server.go:123] GRPC response: {} Jun 15 16:00:31.750: INFO: VolumeSnapshot snapshot-f5ghg found but is not ready. csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:33.494228 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:33.494526 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:33.494592 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:33.495066 1 server.go:123] GRPC response: {} csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:33.495549 1 main.go:71] Health check succeeded Jun 15 16:00:33.924: INFO: VolumeSnapshot snapshot-f5ghg found but is not ready. csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:35.494596 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:35.494283 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:35.494622 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:35.495349 1 server.go:123] GRPC response: {} csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:35.495895 1 main.go:71] Health check succeeded Jun 15 16:00:36.131: INFO: VolumeSnapshot snapshot-f5ghg found but is not ready. csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:37.494711 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:37.494828 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:37.495325 1 server.go:123] GRPC response: {} csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:37.494432 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:37.495835 1 main.go:71] Health check succeeded Jun 15 16:00:38.313: INFO: VolumeSnapshot snapshot-f5ghg found but is not ready. csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:39.494614 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:39.494238 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:39.495477 1 main.go:71] Health check succeeded csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:39.494642 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:39.495058 1 server.go:123] GRPC response: {} csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.526837 1 reflector.go:278] github.com/kubernetes-csi/external-snapshotter/pkg/client/informers/externalversions/factory.go:117: forcing resync csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.526907 1 snapshot_controller_base.go:144] enqueued "snapcontent-5716b93d-7a10-4484-b45d-72361ae56279" for sync csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.526921 1 snapshot_controller_base.go:159] contentWorker[snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.526933 1 util.go:169] storeObjectUpdate updating content "snapcontent-5716b93d-7a10-4484-b45d-72361ae56279" with version 1086690 csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.526940 1 snapshot_controller.go:58] synchronizing VolumeSnapshotContent[snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.526957 1 snapshot_controller.go:569] Check if VolumeSnapshotContent[snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] should be deleted. csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.526962 1 snapshot_controller.go:78] syncContent: Call CreateSnapshot for content snapcontent-5716b93d-7a10-4484-b45d-72361ae56279 csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.526966 1 snapshot_controller.go:133] createSnapshot for content [snapcontent-5716b93d-7a10-4484-b45d-72361ae56279]: started csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.526970 1 snapshot_controller.go:112] scheduleOperation[create-snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.526982 1 snapshot_controller.go:305] createSnapshotWrapper: Creating snapshot for content snapcontent-5716b93d-7a10-4484-b45d-72361ae56279 through the plugin ... csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.526986 1 snapshot_controller.go:218] getCSISnapshotInput for content [snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.526989 1 snapshot_controller.go:477] getSnapshotClass: VolumeSnapshotClassName [e2e-snapshotting-7614-csi-hostpath-e2e-snapshotting-7614-vsc] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.526994 1 snapshot_controller.go:606] setAnnVolumeSnapshotBeingCreated: set annotation [snapshot.storage.kubernetes.io/volumesnapshot-being-created:yes] on content [snapcontent-5716b93d-7a10-4484-b45d-72361ae56279]. csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.540050 1 util.go:169] storeObjectUpdate updating content "snapcontent-5716b93d-7a10-4484-b45d-72361ae56279" with version 1087045 csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.540082 1 snapshot_controller.go:621] setAnnVolumeSnapshotBeingCreated: volume snapshot content &{TypeMeta:{Kind: APIVersion:} ObjectMeta:{Name:snapcontent-5716b93d-7a10-4484-b45d-72361ae56279 GenerateName: Namespace: SelfLink:/apis/snapshot.storage.k8s.io/v1beta1/volumesnapshotcontents/snapcontent-5716b93d-7a10-4484-b45d-72361ae56279 UID:22c36c7c-c767-4675-bf54-375b70f7efa9 ResourceVersion:1087045 Generation:1 CreationTimestamp:2020-06-15 13:59:40 +0000 UTC DeletionTimestamp: DeletionGracePeriodSeconds: Labels:map[] Annotations:map[snapshot.storage.kubernetes.io/volumesnapshot-being-created:yes] OwnerReferences:[] Finalizers:[snapshot.storage.kubernetes.io/volumesnapshotcontent-bound-protection] ClusterName: ManagedFields:[{Manager:snapshot-controller Operation:Update APIVersion:snapshot.storage.k8s.io/v1beta1 Time:2020-06-15 13:59:40 +0000 UTC FieldsType:FieldsV1 FieldsV1:&FieldsV1{Raw:*[123 34 102 58 109 101 116 97 100 97 116 97 34 58 123 34 102 58 102 105 110 97 108 105 122 101 114 115 34 58 123 34 46 34 58 123 125 44 34 118 58 92 34 115 110 97 112 115 104 111 116 46 115 116 111 114 97 103 101 46 107 117 98 101 114 110 101 116 101 115 46 105 111 47 118 111 108 117 109 101 115 110 97 112 115 104 111 116 99 111 110 116 101 110 116 45 98 111 117 110 100 45 112 114 111 116 101 99 116 105 111 110 92 34 34 58 123 125 125 125 44 34 102 58 115 112 101 99 34 58 123 34 46 34 58 123 125 44 34 102 58 100 101 108 101 116 105 111 110 80 111 108 105 99 121 34 58 123 125 44 34 102 58 100 114 105 118 101 114 34 58 123 125 44 34 102 58 115 111 117 114 99 101 34 58 123 34 46 34 58 123 125 44 34 102 58 118 111 108 117 109 101 72 97 110 100 108 101 34 58 123 125 125 44 34 102 58 118 111 108 117 109 101 83 110 97 112 115 104 111 116 67 108 97 115 115 78 97 109 101 34 58 123 125 44 34 102 58 118 111 108 117 109 101 83 110 97 112 115 104 111 116 82 101 102 34 58 123 34 46 34 58 123 125 44 34 102 58 97 112 105 86 101 114 115 105 111 110 34 58 123 125 44 34 102 58 107 105 110 100 34 58 123 125 44 34 102 58 110 97 109 101 34 58 123 125 44 34 102 58 110 97 109 101 115 112 97 99 101 34 58 123 125 44 34 102 58 114 101 115 111 117 114 99 101 86 101 114 115 105 111 110 34 58 123 125 44 34 102 58 117 105 100 34 58 123 125 125 125 125],}} {Manager:csi-snapshotter Operation:Update APIVersion:snapshot.storage.k8s.io/v1beta1 Time:2020-06-15 14:00:39 +0000 UTC FieldsType:FieldsV1 FieldsV1:&FieldsV1{Raw:*[123 34 102 58 109 101 116 97 100 97 116 97 34 58 123 34 102 58 97 110 110 111 116 97 116 105 111 110 115 34 58 123 34 46 34 58 123 125 44 34 102 58 115 110 97 112 115 104 111 116 46 115 116 111 114 97 103 101 46 107 117 98 101 114 110 101 116 101 115 46 105 111 47 118 111 108 117 109 101 115 110 97 112 115 104 111 116 45 98 101 105 110 103 45 99 114 101 97 116 101 100 34 58 123 125 125 125 125],}}]} Spec:{VolumeSnapshotRef:{Kind:VolumeSnapshot Namespace:e2e-snapshotting-7614 Name:snapshot-f5ghg UID:5716b93d-7a10-4484-b45d-72361ae56279 APIVersion:snapshot.storage.k8s.io/v1beta1 ResourceVersion:1086686 FieldPath:} DeletionPolicy:Delete Driver:csi-hostpath-e2e-snapshotting-7614 VolumeSnapshotClassName:0xc000612ce0 Source:{VolumeHandle:0xc000612cd0 SnapshotHandle:}} Status:} csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.540564 1 snapshotter.go:56] CSI CreateSnapshot: snapshot-5716b93d-7a10-4484-b45d-72361ae56279 csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.540580 1 connection.go:182] GRPC call: /csi.v1.Identity/GetPluginInfo csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.540586 1 connection.go:183] GRPC request: {} csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.543012 1 snapshot_controller_base.go:144] enqueued "snapcontent-5716b93d-7a10-4484-b45d-72361ae56279" for sync csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.543047 1 snapshot_controller_base.go:159] contentWorker[snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.543058 1 util.go:169] storeObjectUpdate updating content "snapcontent-5716b93d-7a10-4484-b45d-72361ae56279" with version 1087045 csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:39.541706 1 server.go:117] GRPC call: /csi.v1.Identity/GetPluginInfo csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:39.541731 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:39.542114 1 identityserver.go:40] Using default GetPluginInfo csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:39.542120 1 server.go:123] GRPC response: {"name":"csi-hostpath-e2e-snapshotting-7614","vendor_version":"v1.4.0-rc2-0-gee6beeaf"} csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.543065 1 snapshot_controller.go:58] synchronizing VolumeSnapshotContent[snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.543083 1 snapshot_controller.go:569] Check if VolumeSnapshotContent[snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] should be deleted. csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.543088 1 snapshot_controller.go:78] syncContent: Call CreateSnapshot for content snapcontent-5716b93d-7a10-4484-b45d-72361ae56279 csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.543092 1 snapshot_controller.go:133] createSnapshot for content [snapcontent-5716b93d-7a10-4484-b45d-72361ae56279]: started csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.543097 1 snapshot_controller.go:112] scheduleOperation[create-snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.543101 1 snapshot_controller.go:118] operation "create-snapcontent-5716b93d-7a10-4484-b45d-72361ae56279" is already running, skipping csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.542737 1 connection.go:185] GRPC response: {"name":"csi-hostpath-e2e-snapshotting-7614","vendor_version":"v1.4.0-rc2-0-gee6beeaf"} csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.543315 1 connection.go:186] GRPC error: csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.543323 1 connection.go:182] GRPC call: /csi.v1.Controller/CreateSnapshot csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.543333 1 connection.go:183] GRPC request: {"name":"snapshot-5716b93d-7a10-4484-b45d-72361ae56279","source_volume_id":"745ac6fc-af10-11ea-bb75-0ab98b83000d"} csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.548984 1 connection.go:185] GRPC response: {"snapshot":{"creation_time":{"nanos":544763860,"seconds":1592229639},"ready_to_use":true,"size_bytes":1048576,"snapshot_id":"9829fd2f-af10-11ea-bb75-0ab98b83000d","source_volume_id":"745ac6fc-af10-11ea-bb75-0ab98b83000d"}} csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.549761 1 connection.go:186] GRPC error: csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:39.544220 1 server.go:117] GRPC call: /csi.v1.Controller/CreateSnapshot csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:39.544235 1 server.go:118] GRPC request: {"name":"snapshot-5716b93d-7a10-4484-b45d-72361ae56279","source_volume_id":"745ac6fc-af10-11ea-bb75-0ab98b83000d"} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:39.544780 1 controllerserver.go:340] Creating snapshot of Filsystem Mode Volume csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:39.546547 1 controllerserver.go:349] create volume snapshot /csi-data-dir/9829fd2f-af10-11ea-bb75-0ab98b83000d.snap csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:39.546584 1 server.go:123] GRPC response: {"snapshot":{"creation_time":{"nanos":544763860,"seconds":1592229639},"ready_to_use":true,"size_bytes":1048576,"snapshot_id":"9829fd2f-af10-11ea-bb75-0ab98b83000d","source_volume_id":"745ac6fc-af10-11ea-bb75-0ab98b83000d"}} csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.549769 1 snapshotter.go:76] CSI CreateSnapshot: snapshot-5716b93d-7a10-4484-b45d-72361ae56279 driver name [csi-hostpath-e2e-snapshotting-7614] snapshot ID [9829fd2f-af10-11ea-bb75-0ab98b83000d] time stamp [&{1592229639 544763860 {} [] 0}] size [1048576] readyToUse [true] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.549785 1 snapshot_controller.go:340] Created snapshot: driver csi-hostpath-e2e-snapshotting-7614, snapshotId 9829fd2f-af10-11ea-bb75-0ab98b83000d, creationTime 2020-06-15 14:00:39.54476386 +0000 UTC, size 1048576, readyToUse true csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.549797 1 snapshot_controller.go:422] updateSnapshotContentStatus: updating VolumeSnapshotContent [snapcontent-5716b93d-7a10-4484-b45d-72361ae56279], snapshotHandle 9829fd2f-af10-11ea-bb75-0ab98b83000d, readyToUse true, createdAt 1592229639544763860, size 1048576 csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.561143 1 snapshot_controller_base.go:144] enqueued "snapcontent-5716b93d-7a10-4484-b45d-72361ae56279" for sync csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.561181 1 snapshot_controller_base.go:159] contentWorker[snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.561191 1 util.go:169] storeObjectUpdate updating content "snapcontent-5716b93d-7a10-4484-b45d-72361ae56279" with version 1087046 csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.561198 1 snapshot_controller.go:58] synchronizing VolumeSnapshotContent[snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.561219 1 snapshot_controller.go:569] Check if VolumeSnapshotContent[snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] should be deleted. csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.568111 1 snapshot_controller.go:643] Removed VolumeSnapshotBeingCreated annotation from volume snapshot content snapcontent-5716b93d-7a10-4484-b45d-72361ae56279 csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.568142 1 util.go:169] storeObjectUpdate updating content "snapcontent-5716b93d-7a10-4484-b45d-72361ae56279" with version 1087047 csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.568153 1 util.go:165] storeObjectUpdate: ignoring content "snapcontent-5716b93d-7a10-4484-b45d-72361ae56279" version 1087046 csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.568875 1 snapshot_controller_base.go:144] enqueued "snapcontent-5716b93d-7a10-4484-b45d-72361ae56279" for sync csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: E0615 14:00:39.572027 1 snapshot_controller_base.go:250] could not sync content "snapcontent-5716b93d-7a10-4484-b45d-72361ae56279": failed to remove VolumeSnapshotBeingCreated annotation from the content snapcontent-5716b93d-7a10-4484-b45d-72361ae56279: "snapshot controller failed to update snapcontent-5716b93d-7a10-4484-b45d-72361ae56279 on API server: Operation cannot be fulfilled on volumesnapshotcontents.snapshot.storage.k8s.io \"snapcontent-5716b93d-7a10-4484-b45d-72361ae56279\": the object has been modified; please apply your changes to the latest version and try again" csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.572059 1 snapshot_controller_base.go:159] contentWorker[snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.572071 1 util.go:169] storeObjectUpdate updating content "snapcontent-5716b93d-7a10-4484-b45d-72361ae56279" with version 1087047 csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.572078 1 snapshot_controller.go:58] synchronizing VolumeSnapshotContent[snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:39.572082 1 snapshot_controller.go:569] Check if VolumeSnapshotContent[snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] should be deleted. Jun 15 16:00:40.536: INFO: VolumeSnapshot snapshot-f5ghg found but is not ready. csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:41.495049 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:41.494822 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:41.496077 1 main.go:71] Health check succeeded csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:41.495074 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:41.495580 1 server.go:123] GRPC response: {} Jun 15 16:00:42.708: INFO: VolumeSnapshot snapshot-f5ghg found but is not ready. csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:43.494227 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:43.494497 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:43.494526 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:43.495844 1 main.go:71] Health check succeeded csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:43.495187 1 server.go:123] GRPC response: {} Jun 15 16:00:44.938: INFO: VolumeSnapshot snapshot-f5ghg found and is ready after 1m4.20538771s STEP: checking the snapshot STEP: checking the SnapshotContent Jun 15 16:00:45.322: INFO: deleting snapshot "e2e-snapshotting-7614"/"snapshot-f5ghg" Jun 15 16:00:45.495: INFO: deleting SnapshotClass e2e-snapshotting-7614-csi-hostpath-e2e-snapshotting-7614-vsc csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:45.494575 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:45.494600 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:45.494332 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:45.495144 1 server.go:123] GRPC response: {} csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:45.495552 1 main.go:71] Health check succeeded Jun 15 16:00:45.860: INFO: Pod pvc-snapshottable-tester-9782d has the following logs: STEP: Deleting pod pvc-snapshottable-tester-9782d in namespace e2e-snapshotting-7614 pod event: MODIFIED: e2e-snapshotting-7614/pvc-snapshottable-tester-9782d Succeeded: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:44 +0200 CEST PodCompleted } {Ready False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:44 +0200 CEST PodCompleted } {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:44 +0200 CEST PodCompleted } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:44 +0200 CEST }] volume-tester: TERMINATED: Completed - pod event: DELETED: e2e-snapshotting-7614/pvc-snapshottable-tester-9782d Succeeded: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:44 +0200 CEST PodCompleted } {Ready False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:44 +0200 CEST PodCompleted } {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:44 +0200 CEST PodCompleted } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:44 +0200 CEST }] volume-tester: TERMINATED: Completed - Jun 15 16:00:46.271: INFO: deleting claim "e2e-snapshotting-7614"/"pvc-glxfp" csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:46.380607 1 server.go:117] GRPC call: /csi.v1.Controller/DeleteVolume csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:46.380627 1 server.go:118] GRPC request: {"volume_id":"745ac6fc-af10-11ea-bb75-0ab98b83000d"} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:46.381116 1 hostpath.go:231] deleting hostpath volume: 745ac6fc-af10-11ea-bb75-0ab98b83000d csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:46.381288 1 controllerserver.go:224] volume 745ac6fc-af10-11ea-bb75-0ab98b83000d successfully deleted csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:46.381299 1 server.go:123] GRPC response: {} csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 14:00:46.376411 1 controller.go:1422] delete "pvc-a5748dab-0e0d-4d15-94d7-6cef2d40befb": started csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 14:00:46.379661 1 connection.go:182] GRPC call: /csi.v1.Controller/DeleteVolume csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 14:00:46.379706 1 connection.go:183] GRPC request: {"volume_id":"745ac6fc-af10-11ea-bb75-0ab98b83000d"} csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 14:00:46.381897 1 connection.go:185] GRPC response: {} csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 14:00:46.382296 1 connection.go:186] GRPC error: csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 14:00:46.382308 1 controller.go:1450] delete "pvc-a5748dab-0e0d-4d15-94d7-6cef2d40befb": volume deleted csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 14:00:46.387885 1 controller.go:1496] delete "pvc-a5748dab-0e0d-4d15-94d7-6cef2d40befb": persistentvolume deleted csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: I0615 14:00:46.387909 1 controller.go:1498] delete "pvc-a5748dab-0e0d-4d15-94d7-6cef2d40befb": succeeded Jun 15 16:00:46.474: INFO: deleting storage class e2e-snapshotting-7614-csi-hostpath-e2e-snapshotting-7614-st46sw STEP: uninstalling csi-hostpath driver Jun 15 16:00:46.678: INFO: deleting *v1.ServiceAccount: e2e-snapshotting-7614/csi-attacher Jun 15 16:00:46.884: INFO: deleting *v1.ClusterRole: external-attacher-runner-e2e-snapshotting-7614 Jun 15 16:00:47.089: INFO: deleting *v1.ClusterRoleBinding: csi-attacher-role-e2e-snapshotting-7614 Jun 15 16:00:47.294: INFO: deleting *v1.Role: e2e-snapshotting-7614/external-attacher-cfg-e2e-snapshotting-7614 Jun 15 16:00:47.499: INFO: deleting *v1.RoleBinding: e2e-snapshotting-7614/csi-attacher-role-cfg csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:47.494485 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:47.494602 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:47.495129 1 server.go:123] GRPC response: {} csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:47.494247 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:47.495578 1 main.go:71] Health check succeeded Jun 15 16:00:47.673: INFO: deleting *v1.ServiceAccount: e2e-snapshotting-7614/csi-provisioner Jun 15 16:00:47.909: INFO: deleting *v1.ClusterRole: external-provisioner-runner-e2e-snapshotting-7614 Jun 15 16:00:48.081: INFO: deleting *v1.ClusterRoleBinding: csi-provisioner-role-e2e-snapshotting-7614 Jun 15 16:00:48.255: INFO: deleting *v1.Role: e2e-snapshotting-7614/external-provisioner-cfg-e2e-snapshotting-7614 Jun 15 16:00:48.428: INFO: deleting *v1.RoleBinding: e2e-snapshotting-7614/csi-provisioner-role-cfg Jun 15 16:00:48.625: INFO: deleting *v1.ServiceAccount: e2e-snapshotting-7614/csi-snapshotter Jun 15 16:00:48.797: INFO: deleting *v1.ClusterRole: external-snapshotter-runner-e2e-snapshotting-7614 Jun 15 16:00:49.040: INFO: deleting *v1.ClusterRoleBinding: csi-snapshotter-role-e2e-snapshotting-7614 Jun 15 16:00:49.240: INFO: deleting *v1.Role: e2e-snapshotting-7614/external-snapshotter-leaderelection-e2e-snapshotting-7614 csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:49.239498 1 snapshot_controller_base.go:144] enqueued "snapcontent-5716b93d-7a10-4484-b45d-72361ae56279" for sync csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:49.239540 1 snapshot_controller_base.go:159] contentWorker[snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:49.239554 1 util.go:169] storeObjectUpdate updating content "snapcontent-5716b93d-7a10-4484-b45d-72361ae56279" with version 1087129 csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:49.239561 1 snapshot_controller.go:58] synchronizing VolumeSnapshotContent[snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:49.239584 1 snapshot_controller.go:569] Check if VolumeSnapshotContent[snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] should be deleted. csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:49.239790 1 snapshot_controller_base.go:144] enqueued "snapcontent-5716b93d-7a10-4484-b45d-72361ae56279" for sync csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:49.239805 1 snapshot_controller_base.go:159] contentWorker[snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:49.239814 1 util.go:169] storeObjectUpdate updating content "snapcontent-5716b93d-7a10-4484-b45d-72361ae56279" with version 1087130 csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:49.239819 1 snapshot_controller.go:58] synchronizing VolumeSnapshotContent[snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:49.239832 1 snapshot_controller.go:569] Check if VolumeSnapshotContent[snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] should be deleted. csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:49.239837 1 snapshot_controller.go:61] VolumeSnapshotContent[snapcontent-5716b93d-7a10-4484-b45d-72361ae56279]: the policy is Delete csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:49.239842 1 snapshot_controller.go:102] schedule to delete snapshot, operation named delete-snapcontent-5716b93d-7a10-4484-b45d-72361ae56279 csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:49.239846 1 snapshot_controller.go:112] scheduleOperation[delete-snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:49.239862 1 snapshot_controller.go:367] deleteCSISnapshotOperation [snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] started csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:49.239867 1 snapshot_controller.go:218] getCSISnapshotInput for content [snapcontent-5716b93d-7a10-4484-b45d-72361ae56279] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:49.239870 1 snapshot_controller.go:477] getSnapshotClass: VolumeSnapshotClassName [e2e-snapshotting-7614-csi-hostpath-e2e-snapshotting-7614-vsc] csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: E0615 14:00:49.239876 1 snapshot_controller.go:481] failed to retrieve snapshot class e2e-snapshotting-7614-csi-hostpath-e2e-snapshotting-7614-vsc from the informer: "volumesnapshotclass.snapshot.storage.k8s.io \"e2e-snapshotting-7614-csi-hostpath-e2e-snapshotting-7614-vsc\" not found" csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: E0615 14:00:49.239891 1 snapshot_controller.go:224] getCSISnapshotInput failed to getClassFromVolumeSnapshot failed to retrieve snapshot class e2e-snapshotting-7614-csi-hostpath-e2e-snapshotting-7614-vsc from the informer: "volumesnapshotclass.snapshot.storage.k8s.io \"e2e-snapshotting-7614-csi-hostpath-e2e-snapshotting-7614-vsc\" not found" csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: E0615 14:00:49.239935 1 goroutinemap.go:150] Operation for "delete-snapcontent-5716b93d-7a10-4484-b45d-72361ae56279" failed. No retries permitted until 2020-06-15 14:00:49.739912566 +0000 UTC m=+70.267604163 (durationBeforeRetry 500ms). Error: "failed to get input parameters to delete snapshot for content snapcontent-5716b93d-7a10-4484-b45d-72361ae56279: \"failed to retrieve snapshot class e2e-snapshotting-7614-csi-hostpath-e2e-snapshotting-7614-vsc from the informer: \\\"volumesnapshotclass.snapshot.storage.k8s.io \\\\\\\"e2e-snapshotting-7614-csi-hostpath-e2e-snapshotting-7614-vsc\\\\\\\" not found\\\"\"" csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: I0615 14:00:49.240465 1 event.go:281] Event(v1.ObjectReference{Kind:"VolumeSnapshotContent", Namespace:"", Name:"snapcontent-5716b93d-7a10-4484-b45d-72361ae56279", UID:"22c36c7c-c767-4675-bf54-375b70f7efa9", APIVersion:"snapshot.storage.k8s.io/v1beta1", ResourceVersion:"1087130", FieldPath:""}): type: 'Warning' reason: 'SnapshotDeleteError' Failed to get snapshot class or credentials csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: E0615 14:00:49.241720 1 event.go:263] Server rejected event '&v1.Event{TypeMeta:v1.TypeMeta{Kind:"", APIVersion:""}, ObjectMeta:v1.ObjectMeta{Name:"snapcontent-5716b93d-7a10-4484-b45d-72361ae56279.1618bc6e13ca0d98", GenerateName:"", Namespace:"default", SelfLink:"", UID:"", ResourceVersion:"", Generation:0, CreationTimestamp:v1.Time{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, DeletionTimestamp:(*v1.Time)(nil), DeletionGracePeriodSeconds:(*int64)(nil), Labels:map[string]string(nil), Annotations:map[string]string(nil), OwnerReferences:[]v1.OwnerReference(nil), Finalizers:[]string(nil), ClusterName:"", ManagedFields:[]v1.ManagedFieldsEntry(nil)}, InvolvedObject:v1.ObjectReference{Kind:"VolumeSnapshotContent", Namespace:"", Name:"snapcontent-5716b93d-7a10-4484-b45d-72361ae56279", UID:"22c36c7c-c767-4675-bf54-375b70f7efa9", APIVersion:"snapshot.storage.k8s.io/v1beta1", ResourceVersion:"1087130", FieldPath:""}, Reason:"SnapshotDeleteError", Message:"Failed to get snapshot class or credentials", Source:v1.EventSource{Component:"csi-snapshotter csi-hostpath-e2e-snapshotting-7614", Host:""}, FirstTimestamp:v1.Time{Time:time.Time{wall:0xbfb1fda44e4ca398, ext:69767595237, loc:(*time.Location)(0x2352980)}}, LastTimestamp:v1.Time{Time:time.Time{wall:0xbfb1fda44e4ca398, ext:69767595237, loc:(*time.Location)(0x2352980)}}, Count:1, Type:"Warning", EventTime:v1.MicroTime{Time:time.Time{wall:0x0, ext:0, loc:(*time.Location)(nil)}}, Series:(*v1.EventSeries)(nil), Action:"", Related:(*v1.ObjectReference)(nil), ReportingController:"", ReportingInstance:""}': 'events is forbidden: User "system:serviceaccount:e2e-snapshotting-7614:csi-snapshotter" cannot create resource "events" in API group "" in the namespace "default": RBAC: clusterrole.rbac.authorization.k8s.io "e2e-test-privileged-psp" not found' (will not retry!) Jun 15 16:00:49.411: INFO: deleting *v1.RoleBinding: e2e-snapshotting-7614/external-snapshotter-leaderelection csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:49.494884 1 server.go:117] GRPC call: /csi.v1.Identity/Probe csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:49.494535 1 main.go:53] Sending probe request to CSI driver "csi-hostpath-e2e-snapshotting-7614" csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:49.494922 1 server.go:118] GRPC request: {} csi-hostpathplugin-0/hostpath@worker-0-1: I0615 14:00:49.495553 1 server.go:123] GRPC response: {} csi-hostpathplugin-0/liveness-probe@worker-0-1: I0615 14:00:49.496222 1 main.go:71] Health check succeeded Jun 15 16:00:49.584: INFO: deleting *v1.ServiceAccount: e2e-snapshotting-7614/csi-resizer Jun 15 16:00:49.756: INFO: deleting *v1.ClusterRole: external-resizer-runner-e2e-snapshotting-7614 Jun 15 16:00:49.930: INFO: deleting *v1.ClusterRoleBinding: csi-resizer-role-e2e-snapshotting-7614 Jun 15 16:00:50.103: INFO: deleting *v1.Role: e2e-snapshotting-7614/external-resizer-cfg-e2e-snapshotting-7614 Jun 15 16:00:50.288: INFO: deleting *v1.RoleBinding: e2e-snapshotting-7614/csi-resizer-role-cfg Jun 15 16:00:50.570: INFO: deleting *v1.Service: e2e-snapshotting-7614/csi-hostpath-attacher Jun 15 16:00:50.767: INFO: deleting *v1.StatefulSet: e2e-snapshotting-7614/csi-hostpath-attacher Jun 15 16:00:50.941: INFO: deleting *v1.CSIDriver: csi-hostpath-e2e-snapshotting-7614 pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpath-attacher-0 Running: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST } {Ready True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:38 +0200 CEST } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:38 +0200 CEST } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST }] csi-attacher: RUNNING Jun 15 16:00:51.185: INFO: deleting *v1.Service: e2e-snapshotting-7614/csi-hostpathplugin Jun 15 16:00:51.367: INFO: deleting *v1.StatefulSet: e2e-snapshotting-7614/csi-hostpathplugin Jun 15 16:00:51.541: INFO: deleting *v1.Service: e2e-snapshotting-7614/csi-hostpath-provisioner pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpathplugin-0 Running: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST } {Ready True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:39 +0200 CEST } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:39 +0200 CEST } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST }] hostpath: RUNNING liveness-probe: RUNNING node-driver-registrar: RUNNING csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: E0615 14:00:51.503497 1 connection.go:131] Lost connection to unix:///csi/csi.sock. csi-hostpath-provisioner-0/csi-provisioner@worker-0-1: F0615 14:00:51.503573 1 connection.go:87] Lost connection to CSI driver, exiting csi-hostpath-resizer-0/csi-resizer@worker-0-1: E0615 14:00:51.503928 1 connection.go:131] Lost connection to unix:///csi/csi.sock. csi-hostpath-resizer-0/csi-resizer@worker-0-1: F0615 14:00:51.503991 1 connection.go:87] Lost connection to CSI driver, exiting csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: E0615 14:00:51.503462 1 connection.go:131] Lost connection to unix:///csi/csi.sock. csi-hostpath-snapshotter-0/csi-snapshotter@worker-0-1: F0615 14:00:51.503545 1 connection.go:87] Lost connection to CSI driver, exiting pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpath-attacher-0 Running: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST } {Ready False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 16:00:51 +0200 CEST ContainersNotReady containers with unready status: [csi-attacher]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 16:00:51 +0200 CEST ContainersNotReady containers with unready status: [csi-attacher]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST }] csi-attacher: TERMINATED: Error - Jun 15 16:00:51.762: INFO: deleting *v1.StatefulSet: e2e-snapshotting-7614/csi-hostpath-provisioner Jun 15 16:00:51.937: INFO: deleting *v1.Service: e2e-snapshotting-7614/csi-hostpath-resizer csi-hostpath-attacher-0/csi-attacher@worker-0-1: unable to retrieve container logs for cri-o://88fc1ec21877148b5d7b4b43f43dbcbb597604a0c4e8727d9f5ec0c4fd3e70ae pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpath-provisioner-0 Running: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST } {Ready True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:39 +0200 CEST } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:39 +0200 CEST } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST }] csi-provisioner: RUNNING Jun 15 16:00:52.215: INFO: deleting *v1.StatefulSet: e2e-snapshotting-7614/csi-hostpath-resizer Jun 15 16:00:52.391: INFO: deleting *v1.Service: e2e-snapshotting-7614/csi-hostpath-snapshotter pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpath-resizer-0 Running: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST } {Ready True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:40 +0200 CEST } {ContainersReady True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:40 +0200 CEST } {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST }] csi-resizer: RUNNING Jun 15 16:00:52.573: INFO: deleting *v1.StatefulSet: e2e-snapshotting-7614/csi-hostpath-snapshotter pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpath-snapshotter-0 Running: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST } {Ready False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 16:00:52 +0200 CEST ContainersNotReady containers with unready status: [csi-snapshotter]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 16:00:52 +0200 CEST ContainersNotReady containers with unready status: [csi-snapshotter]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST }] csi-snapshotter: TERMINATED: Error - Lost connection to CSI driver, exiting pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpath-provisioner-0 Running: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST } {Ready False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 16:00:52 +0200 CEST ContainersNotReady containers with unready status: [csi-provisioner]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 16:00:52 +0200 CEST ContainersNotReady containers with unready status: [csi-provisioner]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST }] csi-provisioner: TERMINATED: Error - Lost connection to CSI driver, exiting pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpathplugin-0 Running: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST } {Ready False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 16:00:52 +0200 CEST ContainersNotReady containers with unready status: [node-driver-registrar hostpath liveness-probe]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 16:00:52 +0200 CEST ContainersNotReady containers with unready status: [node-driver-registrar hostpath liveness-probe]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST }] hostpath: TERMINATED: Error - liveness-probe: TERMINATED: Error - node-driver-registrar: TERMINATED: Error - Jun 15 16:00:52.755: INFO: deleting *v1.ClusterRoleBinding: psp-csi-hostpath-role-e2e-snapshotting-7614 pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpath-attacher-0 Running: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST } {Ready False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 16:00:51 +0200 CEST ContainersNotReady containers with unready status: [csi-attacher]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 16:00:51 +0200 CEST ContainersNotReady containers with unready status: [csi-attacher]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST }] csi-attacher: TERMINATED: Error - pod event: DELETED: e2e-snapshotting-7614/csi-hostpath-attacher-0 Running: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST } {Ready False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 16:00:51 +0200 CEST ContainersNotReady containers with unready status: [csi-attacher]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 16:00:51 +0200 CEST ContainersNotReady containers with unready status: [csi-attacher]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:35 +0200 CEST }] csi-attacher: TERMINATED: Error - WARNING: pod log: csi-hostpath-attacher-0/csi-attacher: pods "csi-hostpath-attacher-0" not found pod event: MODIFIED: e2e-snapshotting-7614/csi-hostpath-snapshotter-0 Running: [{Initialized True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST } {Ready False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 16:00:52 +0200 CEST ContainersNotReady containers with unready status: [csi-snapshotter]} {ContainersReady False 0001-01-01 00:00:00 +0000 UTC 2020-06-15 16:00:52 +0200 CEST ContainersNotReady containers with unready status: [csi-snapshotter]} {PodScheduled True 0001-01-01 00:00:00 +0000 UTC 2020-06-15 15:59:36 +0200 CEST }] csi-snapshotter: TERMINATED: Error - Lost connection to CSI driver, exiting [AfterEach] [Testpattern: Dynamic Snapshot] snapshottable /home/fidencio/src/upstream/origin/_output/local/go/src/github.com/openshift/origin/vendor/k8s.io/kubernetes/test/e2e/framework/framework.go:179 Jun 15 16:00:52.927: INFO: Waiting up to 7m0s for all (but 100) nodes to be ready WARNING: pod log: csi-hostpathplugin-0/node-driver-registrar: Get "https://api.kata-fidencio-0.qe.lab.redhat.com:6443/api/v1/namespaces/e2e-snapshotting-7614/pods/csi-hostpathplugin-0/log?container=node-driver-registrar&follow=true": context canceled WARNING: pod log: csi-hostpathplugin-0/node-driver-registrar: context canceled STEP: Destroying namespace "e2e-snapshotting-7614" for this suite. Jun 15 16:00:53.464: INFO: Running AfterSuite actions on all nodes Jun 15 16:00:53.464: INFO: Running AfterSuite actions on node 1 passed: (1m29s) 2020-06-15T14:00:53 "[sig-storage] CSI Volumes [Driver: csi-hostpath] [Testpattern: Dynamic Snapshot] snapshottable should create snapshot with defaults [Feature:VolumeSnapshotDataSource] [Suite:openshift/conformance/parallel] [Suite:k8s]" Timeline: Jun 15 13:59:35.118 I ns/e2e-snapshotting-7614 statefulset/csi-hostpath-attacher reason/SuccessfulCreate create Pod csi-hostpath-attacher-0 in StatefulSet csi-hostpath-attacher successful Jun 15 13:59:35.128 I ns/e2e-snapshotting-7614 pod/csi-hostpath-attacher-0 node/ reason/Created Jun 15 13:59:35.154 I ns/e2e-snapshotting-7614 pod/csi-hostpath-attacher-0 node/worker-0-1 reason/Scheduled Jun 15 13:59:35.722 I ns/e2e-snapshotting-7614 statefulset/csi-hostpathplugin reason/SuccessfulCreate create Pod csi-hostpathplugin-0 in StatefulSet csi-hostpathplugin successful Jun 15 13:59:35.724 I ns/e2e-snapshotting-7614 pod/csi-hostpathplugin-0 node/worker-0-1 reason/Scheduled Jun 15 13:59:35.724 I ns/e2e-snapshotting-7614 pod/csi-hostpathplugin-0 node/ reason/Created Jun 15 13:59:36.130 I ns/e2e-snapshotting-7614 pod/csi-hostpath-provisioner-0 node/ reason/Created Jun 15 13:59:36.131 I ns/e2e-snapshotting-7614 statefulset/csi-hostpath-provisioner reason/SuccessfulCreate create Pod csi-hostpath-provisioner-0 in StatefulSet csi-hostpath-provisioner successful Jun 15 13:59:36.293 I ns/e2e-snapshotting-7614 pod/csi-hostpath-provisioner-0 node/worker-0-1 reason/Scheduled Jun 15 13:59:36.492 I ns/e2e-snapshotting-7614 pod/csi-hostpath-resizer-0 node/ reason/Created Jun 15 13:59:36.494 I ns/e2e-snapshotting-7614 statefulset/csi-hostpath-resizer reason/SuccessfulCreate create Pod csi-hostpath-resizer-0 in StatefulSet csi-hostpath-resizer successful Jun 15 13:59:36.499 I ns/e2e-snapshotting-7614 pod/csi-hostpath-resizer-0 node/worker-0-1 reason/Scheduled Jun 15 13:59:36.844 I ns/e2e-snapshotting-7614 pod/csi-hostpath-snapshotter-0 node/ reason/Created Jun 15 13:59:36.845 I ns/e2e-snapshotting-7614 statefulset/csi-hostpath-snapshotter reason/SuccessfulCreate create Pod csi-hostpath-snapshotter-0 in StatefulSet csi-hostpath-snapshotter successful Jun 15 13:59:36.853 I ns/e2e-snapshotting-7614 pod/csi-hostpath-snapshotter-0 node/worker-0-1 reason/Scheduled Jun 15 13:59:37.372 I ns/e2e-snapshotting-7614 persistentvolumeclaim/pvc-glxfp reason/ExternalProvisioning waiting for a volume to be created, either by external provisioner "csi-hostpath-e2e-snapshotting-7614" or manually created by system administrator Jun 15 13:59:37.545 I ns/e2e-snapshotting-7614 pod/pvc-snapshottable-tester-9782d node/ reason/Created Jun 15 13:59:37.550 W ns/e2e-snapshotting-7614 pod/pvc-snapshottable-tester-9782d reason/FailedScheduling running "VolumeBinding" filter plugin for pod "pvc-snapshottable-tester-9782d": pod has unbound immediate PersistentVolumeClaims Jun 15 13:59:37.648 W ns/e2e-snapshotting-7614 pod/pvc-snapshottable-tester-9782d reason/FailedScheduling running "VolumeBinding" filter plugin for pod "pvc-snapshottable-tester-9782d": pod has unbound immediate PersistentVolumeClaims Jun 15 13:59:37.660 I ns/e2e-snapshotting-7614 pod/csi-hostpath-attacher-0 reason/AddedInterface Add eth0 [10.131.0.5/23] Jun 15 13:59:37.968 I ns/e2e-snapshotting-7614 pod/csi-hostpath-attacher-0 node/worker-0-1 container/csi-attacher reason/Pulled image/gcr.io/k8s-staging-csi/csi-attacher:v2.2.0 Jun 15 13:59:38.028 I ns/e2e-snapshotting-7614 pod/csi-hostpathplugin-0 reason/AddedInterface Add eth0 [10.131.0.12/23] Jun 15 13:59:38.116 I ns/e2e-snapshotting-7614 pod/csi-hostpath-attacher-0 node/worker-0-1 container/csi-attacher reason/Created Jun 15 13:59:38.158 I ns/e2e-snapshotting-7614 pod/csi-hostpath-attacher-0 node/worker-0-1 container/csi-attacher reason/Started Jun 15 13:59:38.322 I ns/e2e-snapshotting-7614 pod/csi-hostpathplugin-0 node/worker-0-1 container/node-driver-registrar reason/Pulled image/gcr.io/k8s-staging-csi/csi-node-driver-registrar:v1.3.0 Jun 15 13:59:38.584 I ns/e2e-snapshotting-7614 pod/csi-hostpathplugin-0 node/worker-0-1 container/node-driver-registrar reason/Created Jun 15 13:59:38.584 I ns/e2e-snapshotting-7614 pod/csi-hostpathplugin-0 node/worker-0-1 container/node-driver-registrar reason/Started Jun 15 13:59:38.585 I ns/e2e-snapshotting-7614 pod/csi-hostpath-attacher-0 node/worker-0-1 container/csi-attacher reason/Ready Jun 15 13:59:38.585 I ns/e2e-snapshotting-7614 pod/csi-hostpath-provisioner-0 reason/AddedInterface Add eth0 [10.131.0.13/23] Jun 15 13:59:38.585 I ns/e2e-snapshotting-7614 pod/csi-hostpathplugin-0 node/worker-0-1 container/hostpath reason/Pulled image/gcr.io/k8s-staging-csi/hostpathplugin:v1.4.0-rc2 Jun 15 13:59:38.900 I ns/e2e-snapshotting-7614 pod/csi-hostpathplugin-0 node/worker-0-1 container/hostpath reason/Created Jun 15 13:59:38.958 I ns/e2e-snapshotting-7614 pod/csi-hostpathplugin-0 node/worker-0-1 container/hostpath reason/Started Jun 15 13:59:38.963 I ns/e2e-snapshotting-7614 pod/csi-hostpathplugin-0 node/worker-0-1 container/liveness-probe reason/Pulled image/gcr.io/k8s-staging-csi/livenessprobe:v1.1.0 Jun 15 13:59:38.992 I ns/e2e-snapshotting-7614 pod/csi-hostpath-resizer-0 reason/AddedInterface Add eth0 [10.131.0.14/23] Jun 15 13:59:39.016 I ns/e2e-snapshotting-7614 pod/csi-hostpath-snapshotter-0 reason/AddedInterface Add eth0 [10.131.0.15/23] Jun 15 13:59:39.119 I ns/e2e-snapshotting-7614 pod/csi-hostpath-provisioner-0 node/worker-0-1 container/csi-provisioner reason/Pulled image/gcr.io/k8s-staging-csi/csi-provisioner:v1.6.0 Jun 15 13:59:39.162 I ns/e2e-snapshotting-7614 pod/csi-hostpathplugin-0 node/worker-0-1 container/liveness-probe reason/Created Jun 15 13:59:39.198 I ns/e2e-snapshotting-7614 pod/csi-hostpathplugin-0 node/worker-0-1 container/liveness-probe reason/Started Jun 15 13:59:39.274 I ns/e2e-snapshotting-7614 pod/csi-hostpath-provisioner-0 node/worker-0-1 container/csi-provisioner reason/Created Jun 15 13:59:39.306 I ns/e2e-snapshotting-7614 pod/csi-hostpath-provisioner-0 node/worker-0-1 container/csi-provisioner reason/Started Jun 15 13:59:39.308 - 14s W ns/default pod/recycler-for-nfs-pv10 node/master-0-1 pod has been pending longer than a minute Jun 15 13:59:39.338 I ns/e2e-snapshotting-7614 pod/csi-hostpath-resizer-0 node/worker-0-1 container/csi-resizer reason/Pulled image/gcr.io/k8s-staging-csi/csi-resizer:v0.5.0 Jun 15 13:59:39.390 I ns/e2e-snapshotting-7614 pod/csi-hostpath-snapshotter-0 node/worker-0-1 container/csi-snapshotter reason/Pulled image/gcr.io/k8s-staging-csi/csi-snapshotter:v2.1.0 Jun 15 13:59:39.487 I ns/e2e-snapshotting-7614 pod/csi-hostpath-provisioner-0 node/worker-0-1 container/csi-provisioner reason/Ready Jun 15 13:59:39.501 I ns/e2e-snapshotting-7614 pod/csi-hostpathplugin-0 node/worker-0-1 container/hostpath reason/Ready Jun 15 13:59:39.501 I ns/e2e-snapshotting-7614 pod/csi-hostpathplugin-0 node/worker-0-1 container/node-driver-registrar reason/Ready Jun 15 13:59:39.501 I ns/e2e-snapshotting-7614 pod/csi-hostpathplugin-0 node/worker-0-1 container/liveness-probe reason/Ready Jun 15 13:59:39.544 I ns/e2e-snapshotting-7614 persistentvolumeclaim/pvc-glxfp reason/Provisioning External provisioner is provisioning volume for claim "e2e-snapshotting-7614/pvc-glxfp" Jun 15 13:59:39.563 I ns/e2e-snapshotting-7614 pod/csi-hostpath-resizer-0 node/worker-0-1 container/csi-resizer reason/Created Jun 15 13:59:39.566 I ns/e2e-snapshotting-7614 persistentvolumeclaim/pvc-glxfp reason/ProvisioningSucceeded Successfully provisioned volume pvc-a5748dab-0e0d-4d15-94d7-6cef2d40befb Jun 15 13:59:39.571 W ns/e2e-snapshotting-7614 pod/pvc-snapshottable-tester-9782d reason/FailedScheduling running "VolumeBinding" filter plugin for pod "pvc-snapshottable-tester-9782d": pod has unbound immediate PersistentVolumeClaims Jun 15 13:59:39.761 I ns/e2e-snapshotting-7614 pod/csi-hostpath-resizer-0 node/worker-0-1 container/csi-resizer reason/Started Jun 15 13:59:39.961 I ns/e2e-snapshotting-7614 pod/csi-hostpath-snapshotter-0 node/worker-0-1 container/csi-snapshotter reason/Created Jun 15 13:59:40.187 I ns/e2e-snapshotting-7614 pod/csi-hostpath-snapshotter-0 node/worker-0-1 container/csi-snapshotter reason/Started Jun 15 13:59:40.494 I ns/e2e-snapshotting-7614 pod/csi-hostpath-resizer-0 node/worker-0-1 container/csi-resizer reason/Ready Jun 15 13:59:40.505 I ns/e2e-snapshotting-7614 pod/csi-hostpath-snapshotter-0 node/worker-0-1 container/csi-snapshotter reason/Ready Jun 15 13:59:44.235 I ns/e2e-snapshotting-7614 pod/pvc-snapshottable-tester-9782d node/worker-0-1 reason/Scheduled Jun 15 13:59:44.402 I ns/e2e-snapshotting-7614 pod/pvc-snapshottable-tester-9782d reason/SuccessfulAttachVolume AttachVolume.Attach succeeded for volume "pvc-a5748dab-0e0d-4d15-94d7-6cef2d40befb" Jun 15 13:59:54.364 I ns/e2e-snapshotting-7614 pod/pvc-snapshottable-tester-9782d reason/AddedInterface Add eth0 [10.131.0.16/23] Jun 15 13:59:54.530 I ns/e2e-snapshotting-7614 pod/pvc-snapshottable-tester-9782d node/worker-0-1 container/volume-tester reason/Pulled image/docker.io/library/busybox:1.29 Jun 15 13:59:54.644 I ns/e2e-snapshotting-7614 pod/pvc-snapshottable-tester-9782d node/worker-0-1 container/volume-tester reason/Created Jun 15 13:59:54.676 I ns/e2e-snapshotting-7614 pod/pvc-snapshottable-tester-9782d node/worker-0-1 container/volume-tester reason/Started Jun 15 13:59:59.679 I ns/openshift-config-operator deployment/openshift-config-operator reason/KubeCloudConfigController openshift-config-managed/kube-cloud-config ConfigMap was deleted as no longer required (3756 times) Jun 15 14:00:01.260 W ns/default pod/recycler-for-nfs-pv10 node/master-0-1 reason/FailedMount Unable to attach or mount volumes: unmounted volumes=[vol], unattached volumes=[vol default-token-w29vn]: timed out waiting for the condition (4 times) Jun 15 14:00:01.602 I ns/default pod/recycler-for-nfs-pv10 node/master-0-1 reason/DeadlineExceeded Pod was active on the node longer than the specified deadline Jun 15 14:00:01.609 I persistentvolume/nfs-pv10 reason/RecyclerPod Recycler pod: Pod was active on the node longer than the specified deadline (2906 times) Jun 15 14:00:01.609 E ns/default pod/recycler-for-nfs-pv10 node/master-0-1 reason/Failed (DeadlineExceeded): Pod was active on the node longer than the specified deadline Jun 15 14:00:01.687 W ns/default pod/recycler-for-nfs-pv10 node/master-0-1 reason/GracefulDelete in 0s Jun 15 14:00:01.777 W ns/default pod/recycler-for-nfs-pv10 node/master-0-1 reason/Deleted Jun 15 14:00:02.326 W ns/default pod/recycler-for-nfs-pv10 node/master-0-1 reason/FailedMount (combined from similar events): MountVolume.SetUp failed for volume "vol" : mount failed: exit status 32\nMounting command: systemd-run\nMounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/b84f7ca3-7cda-42c3-b721-16ed9446aa6e/volumes/kubernetes.io~nfs/vol --scope -- mount -t nfs registry.kata-fidencio-0.qe.lab.redhat.com:/mnt/pv10 /var/lib/kubelet/pods/b84f7ca3-7cda-42c3-b721-16ed9446aa6e/volumes/kubernetes.io~nfs/vol\nOutput: Running scope as unit: run-r8064333991d34597875fbd6720999d31.scope\nmount.nfs: mounting registry.kata-fidencio-0.qe.lab.redhat.com:/mnt/pv10 failed, reason given by server: No such file or directory\n (6 times) Jun 15 14:00:08.587 I ns/openshift-cluster-storage-operator pod/csi-snapshot-controller-654675b99d-kds5h node/worker-0-0 container/snapshot-controller reason/Pulled image/quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:4ae0585c03dc997ea5deb5291ad75528fdc24d59ccc48966bac39bb369235f64 Jun 15 14:00:08.793 I ns/openshift-cluster-storage-operator deployment/csi-snapshot-controller-operator reason/OperatorStatusChanged Status for clusteroperator/csi-snapshot-controller changed: Progressing changed from False to True ("Progressing: Waiting for Deployment to deploy csi-snapshot-controller pods"),Available changed from True to False ("Available: Waiting for Deployment to deploy csi-snapshot-controller pods") (361 times) Jun 15 14:00:08.793 I ns/openshift-cluster-storage-operator pod/csi-snapshot-controller-654675b99d-kds5h node/worker-0-0 container/snapshot-controller reason/Created Jun 15 14:00:08.794 E ns/openshift-cluster-storage-operator pod/csi-snapshot-controller-654675b99d-kds5h node/worker-0-0 container/snapshot-controller container exited with code 255 (Error): Jun 15 14:00:08.794 I ns/openshift-cluster-storage-operator pod/csi-snapshot-controller-654675b99d-kds5h node/worker-0-0 container/snapshot-controller reason/Started Jun 15 14:00:08.794 I ns/openshift-cluster-storage-operator lease/snapshot-controller-leader reason/LeaderElection csi-snapshot-controller-654675b99d-kds5h became leader Jun 15 14:00:08.795 W clusteroperator/csi-snapshot-controller changed Available to False: _AsExpected: Available: Waiting for Deployment to deploy csi-snapshot-controller pods Jun 15 14:00:08.795 W clusteroperator/csi-snapshot-controller changed Progressing to True: _AsExpected: Progressing: Waiting for Deployment to deploy csi-snapshot-controller pods Jun 15 14:00:09.564 I ns/openshift-cluster-storage-operator pod/csi-snapshot-controller-654675b99d-kds5h node/worker-0-0 container/snapshot-controller reason/Ready Jun 15 14:00:09.564 W ns/openshift-cluster-storage-operator pod/csi-snapshot-controller-654675b99d-kds5h node/worker-0-0 container/snapshot-controller reason/Restarted Jun 15 14:00:09.604 W clusteroperator/csi-snapshot-controller changed Available to True Jun 15 14:00:09.604 W clusteroperator/csi-snapshot-controller changed Progressing to False Jun 15 14:00:09.608 I ns/openshift-cluster-storage-operator deployment/csi-snapshot-controller-operator reason/OperatorStatusChanged Status for clusteroperator/csi-snapshot-controller changed: Progressing changed from True to False (""),Available changed from False to True ("") (361 times) Jun 15 14:00:14.732 W persistentvolume/nfs-pv10 reason/RecyclerPod (combined from similar events): Recycler pod: MountVolume.SetUp failed for volume "vol" : mount failed: exit status 32\nMounting command: systemd-run\nMounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/51afac3f-ad3b-429c-a928-d8c4e19106cd/volumes/kubernetes.io~nfs/vol --scope -- mount -t nfs registry.kata-fidencio-0.qe.lab.redhat.com:/mnt/pv10 /var/lib/kubelet/pods/51afac3f-ad3b-429c-a928-d8c4e19106cd/volumes/kubernetes.io~nfs/vol\nOutput: Running scope as unit: run-rca5d5d35aff742a987d2567ab399b09f.scope\nmount.nfs: mounting registry.kata-fidencio-0.qe.lab.redhat.com:/mnt/pv10 failed, reason given by server: No such file or directory\n (27783 times) Jun 15 14:00:14.732 I ns/default pod/recycler-for-nfs-pv10 node/ reason/Created Jun 15 14:00:14.733 I ns/default pod/recycler-for-nfs-pv10 node/worker-0-0 reason/Scheduled Jun 15 14:00:15.012 W ns/default pod/recycler-for-nfs-pv10 node/worker-0-0 reason/FailedMount MountVolume.SetUp failed for volume "vol" : mount failed: exit status 32\nMounting command: systemd-run\nMounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/8d03cfe3-1128-49cf-b9bb-c8a55fd2ce26/volumes/kubernetes.io~nfs/vol --scope -- mount -t nfs registry.kata-fidencio-0.qe.lab.redhat.com:/mnt/pv10 /var/lib/kubelet/pods/8d03cfe3-1128-49cf-b9bb-c8a55fd2ce26/volumes/kubernetes.io~nfs/vol\nOutput: Running scope as unit: run-r78afc55c32084c74a084192871790487.scope\nmount.nfs: mounting registry.kata-fidencio-0.qe.lab.redhat.com:/mnt/pv10 failed, reason given by server: No such file or directory\n Jun 15 14:00:15.712 W ns/default pod/recycler-for-nfs-pv10 node/worker-0-0 reason/FailedMount MountVolume.SetUp failed for volume "vol" : mount failed: exit status 32\nMounting command: systemd-run\nMounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/8d03cfe3-1128-49cf-b9bb-c8a55fd2ce26/volumes/kubernetes.io~nfs/vol --scope -- mount -t nfs registry.kata-fidencio-0.qe.lab.redhat.com:/mnt/pv10 /var/lib/kubelet/pods/8d03cfe3-1128-49cf-b9bb-c8a55fd2ce26/volumes/kubernetes.io~nfs/vol\nOutput: Running scope as unit: run-rcd1aff9e6af448399259123decac86d9.scope\nmount.nfs: mounting registry.kata-fidencio-0.qe.lab.redhat.com:/mnt/pv10 failed, reason given by server: No such file or directory\n Jun 15 14:00:16.929 W ns/default pod/recycler-for-nfs-pv10 node/worker-0-0 reason/FailedMount MountVolume.SetUp failed for volume "vol" : mount failed: exit status 32\nMounting command: systemd-run\nMounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/8d03cfe3-1128-49cf-b9bb-c8a55fd2ce26/volumes/kubernetes.io~nfs/vol --scope -- mount -t nfs registry.kata-fidencio-0.qe.lab.redhat.com:/mnt/pv10 /var/lib/kubelet/pods/8d03cfe3-1128-49cf-b9bb-c8a55fd2ce26/volumes/kubernetes.io~nfs/vol\nOutput: Running scope as unit: run-r73ec919dfffb4338a08ea582f6bba531.scope\nmount.nfs: mounting registry.kata-fidencio-0.qe.lab.redhat.com:/mnt/pv10 failed, reason given by server: No such file or directory\n Jun 15 14:00:19.066 W ns/default pod/recycler-for-nfs-pv10 node/worker-0-0 reason/FailedMount MountVolume.SetUp failed for volume "vol" : mount failed: exit status 32\nMounting command: systemd-run\nMounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/8d03cfe3-1128-49cf-b9bb-c8a55fd2ce26/volumes/kubernetes.io~nfs/vol --scope -- mount -t nfs registry.kata-fidencio-0.qe.lab.redhat.com:/mnt/pv10 /var/lib/kubelet/pods/8d03cfe3-1128-49cf-b9bb-c8a55fd2ce26/volumes/kubernetes.io~nfs/vol\nOutput: Running scope as unit: run-r057e9b5971a64cb7a8d826fdf32ad1ef.scope\nmount.nfs: mounting registry.kata-fidencio-0.qe.lab.redhat.com:/mnt/pv10 failed, reason given by server: No such file or directory\n Jun 15 14:00:23.313 W ns/default pod/recycler-for-nfs-pv10 node/worker-0-0 reason/FailedMount MountVolume.SetUp failed for volume "vol" : mount failed: exit status 32\nMounting command: systemd-run\nMounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/8d03cfe3-1128-49cf-b9bb-c8a55fd2ce26/volumes/kubernetes.io~nfs/vol --scope -- mount -t nfs registry.kata-fidencio-0.qe.lab.redhat.com:/mnt/pv10 /var/lib/kubelet/pods/8d03cfe3-1128-49cf-b9bb-c8a55fd2ce26/volumes/kubernetes.io~nfs/vol\nOutput: Running scope as unit: run-rd5903c9fb0974c5a86cd776c569bf8dd.scope\nmount.nfs: mounting registry.kata-fidencio-0.qe.lab.redhat.com:/mnt/pv10 failed, reason given by server: No such file or directory\n Jun 15 14:00:31.458 W ns/default pod/recycler-for-nfs-pv10 node/worker-0-0 reason/FailedMount MountVolume.SetUp failed for volume "vol" : mount failed: exit status 32\nMounting command: systemd-run\nMounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/8d03cfe3-1128-49cf-b9bb-c8a55fd2ce26/volumes/kubernetes.io~nfs/vol --scope -- mount -t nfs registry.kata-fidencio-0.qe.lab.redhat.com:/mnt/pv10 /var/lib/kubelet/pods/8d03cfe3-1128-49cf-b9bb-c8a55fd2ce26/volumes/kubernetes.io~nfs/vol\nOutput: Running scope as unit: run-rd6e9d34442dd4cdbad38850b22b48364.scope\nmount.nfs: mounting registry.kata-fidencio-0.qe.lab.redhat.com:/mnt/pv10 failed, reason given by server: No such file or directory\n Jun 15 14:00:46.035 W ns/e2e-snapshotting-7614 pod/pvc-snapshottable-tester-9782d node/worker-0-1 reason/GracefulDelete in 0s Jun 15 14:00:46.271 W ns/e2e-snapshotting-7614 pod/pvc-snapshottable-tester-9782d node/worker-0-1 reason/Deleted Jun 15 14:00:47.684 W ns/default pod/recycler-for-nfs-pv10 node/worker-0-0 reason/FailedMount MountVolume.SetUp failed for volume "vol" : mount failed: exit status 32\nMounting command: systemd-run\nMounting arguments: --description=Kubernetes transient mount for /var/lib/kubelet/pods/8d03cfe3-1128-49cf-b9bb-c8a55fd2ce26/volumes/kubernetes.io~nfs/vol --scope -- mount -t nfs registry.kata-fidencio-0.qe.lab.redhat.com:/mnt/pv10 /var/lib/kubelet/pods/8d03cfe3-1128-49cf-b9bb-c8a55fd2ce26/volumes/kubernetes.io~nfs/vol\nOutput: Running scope as unit: run-r11408f1d81604a3884a384182ae10c56.scope\nmount.nfs: mounting registry.kata-fidencio-0.qe.lab.redhat.com:/mnt/pv10 failed, reason given by server: No such file or directory\n Jun 15 14:00:48.713 W ns/e2e-snapshotting-7614 volumesnapshot/snapshot-f5ghg reason/GetSnapshotClassFailed Failed to get snapshot class with error volumesnapshotclass.snapshot.storage.k8s.io "e2e-snapshotting-7614-csi-hostpath-e2e-snapshotting-7614-vsc" not found Jun 15 14:00:50.953 W ns/e2e-snapshotting-7614 pod/csi-hostpath-attacher-0 node/worker-0-1 reason/GracefulDelete in 30s Jun 15 14:00:50.958 I ns/e2e-snapshotting-7614 pod/csi-hostpath-attacher-0 node/worker-0-1 container/csi-attacher reason/Killing Jun 15 14:00:51.647 I ns/e2e-snapshotting-7614 pod/csi-hostpathplugin-0 node/worker-0-1 container/node-driver-registrar reason/Killing Jun 15 14:00:51.648 W ns/e2e-snapshotting-7614 pod/csi-hostpathplugin-0 node/worker-0-1 reason/GracefulDelete in 30s Jun 15 14:00:51.720 I ns/e2e-snapshotting-7614 pod/csi-hostpathplugin-0 node/worker-0-1 container/liveness-probe reason/Killing Jun 15 14:00:51.720 I ns/e2e-snapshotting-7614 pod/csi-hostpathplugin-0 node/worker-0-1 container/hostpath reason/Killing Jun 15 14:00:51.720 W ns/e2e-snapshotting-7614 pod/csi-hostpathplugin-0 node/worker-0-1 reason/Unhealthy Liveness probe failed: Get http://10.131.0.12:9898/healthz: read tcp 10.131.0.2:45164->10.131.0.12:9898: read: connection reset by peer Jun 15 14:00:51.721 E ns/e2e-snapshotting-7614 pod/csi-hostpath-attacher-0 node/worker-0-1 container/csi-attacher container exited with code 2 (Error): Jun 15 14:00:51.949 W ns/e2e-snapshotting-7614 pod/csi-hostpath-provisioner-0 node/worker-0-1 reason/GracefulDelete in 30s Jun 15 14:00:51.954 I ns/e2e-snapshotting-7614 pod/csi-hostpath-provisioner-0 node/worker-0-1 container/csi-provisioner reason/Killing Jun 15 14:00:52.407 W ns/e2e-snapshotting-7614 pod/csi-hostpath-resizer-0 node/worker-0-1 reason/GracefulDelete in 30s Jun 15 14:00:52.413 I ns/e2e-snapshotting-7614 pod/csi-hostpath-resizer-0 node/worker-0-1 container/csi-resizer reason/Killing Jun 15 14:00:52.662 I ns/e2e-snapshotting-7614 pod/csi-hostpath-snapshotter-0 node/worker-0-1 container/csi-snapshotter reason/Pulled image/gcr.io/k8s-staging-csi/csi-snapshotter:v2.1.0 Jun 15 14:00:52.664 E ns/e2e-snapshotting-7614 pod/csi-hostpath-snapshotter-0 node/worker-0-1 container/csi-snapshotter container exited with code 255 (Error): Lost connection to CSI driver, exiting Jun 15 14:00:52.674 E ns/e2e-snapshotting-7614 pod/csi-hostpath-provisioner-0 node/worker-0-1 container/csi-provisioner container exited with code 255 (Error): Lost connection to CSI driver, exiting Jun 15 14:00:52.832 E ns/e2e-snapshotting-7614 pod/csi-hostpathplugin-0 node/worker-0-1 container/hostpath container exited with code 2 (Error): Jun 15 14:00:52.832 E ns/e2e-snapshotting-7614 pod/csi-hostpathplugin-0 node/worker-0-1 container/liveness-probe container exited with code 2 (Error): Jun 15 14:00:52.832 E ns/e2e-snapshotting-7614 pod/csi-hostpathplugin-0 node/worker-0-1 container/node-driver-registrar container exited with code 2 (Error): Jun 15 14:00:52.839 I ns/e2e-snapshotting-7614 pod/csi-hostpath-snapshotter-0 node/worker-0-1 container/csi-snapshotter reason/Created Jun 15 14:00:52.839 W ns/e2e-snapshotting-7614 pod/csi-hostpath-attacher-0 node/worker-0-1 reason/Deleted Jun 15 14:00:52.841 W ns/e2e-snapshotting-7614 pod/csi-hostpath-snapshotter-0 node/worker-0-1 reason/GracefulDelete in 30s Jun 15 14:00:52.849 W ns/e2e-snapshotting-7614 pod/csi-hostpath-snapshotter-0 node/worker-0-1 reason/FailedMount MountVolume.SetUp failed for volume "csi-snapshotter-token-g6sr5" : secret "csi-snapshotter-token-g6sr5" not found Jun 15 14:00:52.882 I ns/e2e-snapshotting-7614 pod/csi-hostpath-snapshotter-0 node/worker-0-1 container/csi-snapshotter reason/Started Jun 15 14:00:53.352 W ns/e2e-snapshotting-7614 pod/csi-hostpath-snapshotter-0 node/worker-0-1 reason/FailedMount MountVolume.SetUp failed for volume "csi-snapshotter-token-g6sr5" : secret "csi-snapshotter-token-g6sr5" not found (2 times) 1 pass, 0 skip (1m29s)