I've been playing around with Ontap Select 9.3 for our remote branch solution.
I've been able to get it installed and running and have started my preformance testing.
Each node has an internal Intel NVME PCI-E Flash card. I've deployed the 2 node cluster using the "small" config.
I'm seeing terrible write latency....steady 200ms+ when preforming a storage vmotions. Volumes are presented as NFS like our physcal netapps are.
I can actually storage vmotion faster between two old test SATA Netapp's than I can to/from the Ontap Select w/ NVME.
Looking at the preformance graphs shows the underlying NVME datastore has sub 1ms latnecy between the Node VM and the datastore.
So it looks like the latency is within the ontap stack.
I certianly woud not expect baremetal speed. But considering we are using very fast NVME PCI-E cards I'd expect much better preformance than we are seeing.
This same gear was also used for a vSAN test and its preformance was way way higher.
So is there somthing I'm missing? I do see the virtual e0a adapters are set to 1000GB full but changing it to 10GB Full did not seem to help. These host are all connected via 40GB Mellanox switches.
I've also tried disabling dedup/compression on the volume but that did not see to any impact either.