Hi guys,
we have a strange behavior here in two different VMware Cluster environments. Here are the details ...
NetApp FAS2520 with Data ONTAP Version 9.0 (iSCSI is used for the LUNs) and an additional DS2246 shelf
2 aggregates with 5.48TB (24x 900GB SAS HDDs only)
3 thick volumes per controller for a total of 6 thick volumes (without svm_iscsi_root)
HPE DL360 G9 ESXi-hosts with vSphere 6.7.0 Build 15160138 (2 of them in a VMware cluster)
A total of 6 datastores (VMFS 5) based on the 6 NetApp iSCSI LUNs mentioned above + 2 local dastores (VMFS 5)
One of the volume has a total grow size up to 1.01TB (Autogrow Mode: grow) / Storage Efficiency activated / Snapshot Reserves: 0% / Thick provisioned.
Inside of this volume, is a 1TB LUN with Space reservation disabled.
The VMware datastore which is based on this NetApp LUN (and volume) has a size of 1TB (according to the size of the LUN) and there are 3 VMware VMDK data disks from one VM in this datastore, the OS disk of this VM is located in another datastore (different lun and volume) on the same controller.
Inside of this VMware datastore is an .ssd.sf directory and one directory with the name of the sole VM. Inside of the VM directory there are 9 files (3 for each VM data disk), a vmdk descriptor file, a vmdk flat file and a vmdk CBT file.
The 3 VMDK data disks are all THIN but could grow to a total size of up to 750GB, so that 250GB would still be available. Currently approximately 577GB of 1TB are used, so there is enough free space available and the datastore or the NetApp LUN shouldn't go offline.
This particular LUN goes offline every 2 to 3 weeks and makes this productive VM unavailable. According to the System Manager the LUN is full and the volume behind the LUN is nearly full, because it could grow 0.01TB more than the full LUN size.
Switching the LUN online, makes the datastore within the VMware cluster available again. The datastore reports it has still around 450GB available. The LUN reports it is full.
Because this happened a few times, I created a new volume and a new lun inside of this volume and presented this iSCSI LUN as a new datastore and used VMware Storage vMotion to move the VMDK data disks to this newly created datastore, but without any luck. The LUN went offline again, telling me, it's full and after bringing the LUN online again, the datastore has enough free space again.
The NetApp volume behind that NetApp LUN or VMware datastore does not create any NetApp snapshots (checked within System Manager => SVMs => Volumes => Snapshot Copies) and because Snapshot Reserves = 0%.
We have this behavior in 2 different VMware-Clusters, each in its own specific location and it only happens on one of this six datastores (lun or volume) presented to the ESXi-hosts.
I never had those kind of behavior before on this particular FAS2520 (nor on a FAS2040 or FAS2240-2 or FAS2556 or FAS8020).
This behavior first occured, when VMware was upgraded from ESXi 6.0 U2 to ESXi 6.7 U2 a couple of months ago.
Has anyone had a similar behavior or might the Data ONTAP v9.0 be not fully compatible with ESXi 6.7 U2 version?
Any useful comment is much appreciated.
Best regards,
Didi7