Last night I was working on my vSphere test cluster, and an old memory popped up into mind. I just want to share on my blog since this extremely important peice of information could affect you or one of your own deployments.
There are numerous guides on the internet out there telling you how to set up your own SAN or NAS using iSCSI for your own ESXi boxes. One thing these guides don’t factor in is environments that have multiple ESXi hosts.
VMware ESXi (and ESX) when using iSCSi and a shared VMFS volume, require an iSCSI target (iSCSI server) that supports SCSI reservations. Most opensource iSCSI targets like IETD (iSCSI Enterprise Target) DO NOT SUPPORT this standard as of the time of this article being written.
With this being said, operating an iSCSI VMFS volume shared to multiple hosts could ultimately end up in corruption and data loss.
One opensource iSCSI target that I know of that DOES support SCSI reservations is Lio-Target (available here). I personally haven’t experimented with it, but it does support it. One of these days I’m going to do some of my own testing with it.
Keep in mind that instead of using iSCSI you could alternatively use NFS. I use NFS at my company and it works great as a ESXi data store for 3 ESXi boxes. For those of you curious, NFS data stores DO support High Availability (FT), and Vmotion!
UPDATE: Tried out FreeNAS running on a SuperMicro Server with a HP Smart Array 6400 Array controller with an MSA20 storage unit. FreeNAS started spewing a very large amount of errors when accessing VMFS over iSCSI. I think this could be due to some of the disks I have in the array that aren’t being flagged as bad, but are in fact bad.