Cannot remove passthrough setup on ESXi

Question

I have an ESXi host, but it does not have local hard disk. It use disks which are shared via fibre channel. Afterthat, I setup passthrough for the fibre channel. After rebooting ESXi host, we cannot find any datastores. And I also cannot remove passthrough setup (after removing and reboot, it's still there). Does anyone know how to fix it?

Answer 1

The following best practice prevents this problem from occurring:

Always unmark devices for passthrough before removing them from the server. 

The following workaround describes how to force the change from passthrough mode to non-passthrough mode:

1. Edit the /etc/vmware/esx.conf file by changing the owner of the PCI device from passthru to vmkernel. The following is an example of the necessary edit: Change from: /device/000:02.0/owner = "passthru" Change to: /device/000:02.0/owner = "vmkernel"

2. Reboot the ESX/ESXi host.

Answer 2

I ran into this issue this evening as I had 2 identical SAS controllers. PhungHV's information was correct, however when you are in this mode you also cannot access the disks to change the file to get out of the mode. An annoying catch 22 at best. What happens is that VMware boots and reads the settings, then marks the device as a passthrough which causes VMWARE to loose access to it.

To resolve this issue I booted to an Ubuntu live disk. I mapped partition #5

• Mount Boot Drive Partition #5

  mount /dev/sdX5 /mnt
  

• Make temp folder to work in

  mkdir /tmp/vmware
  

• Copy State.tgz

  cp /mnt/state.tgz /tmp/vmware
  

• Extract local.tgz from state.tgz

  tar -zxf /tmp/vmware/state.tgz
  

• Extract etc from local.tgz

  tar -zxf /tmp/vmware/local.tgz
  

• Edit the Config File

  nano -w /tmp/vmware/etc/vmware/esx.conf
  

• Find appropriate reference (or all if your not sure) of passthrough and change to vmkernel

• Create new local.tgz

  cd /tmp/vmware
  tar czf local.tgz etc/
  

• create new State.tgz

  tar czf state.tgz local.tgz
  

• Copy to partition

  cp state.tgz /mnt/state.tgz
  

• Reboot

  reboot
  

Answer 3

Using the Vsphere Client select your host, then the Configuration tab, select Advanced Settings in the Hardware section, then Configure Passthrough and de-select the PCI device you're concerned about and reboot the host.

Answer 4

In vmware esxi 6.5, editing the esx.conf contained in the state.tgz didn't change the esx.conf file used during boot. The file extracted from state.tgz and local.tgz did not contain a reference to the device assigned to passthru (in my case /device/00000:003:00.0/vmkname = "vmhba0" ). Adding the device entry to the file and copying back to the vmware partition did not change the settings - the esx.conf file was edited or overwritten from another location.

I had to re-install 6.5 from the vmware installer and then recreate the network and register the vms. All the datastores can be preserved with the (Install ESXi, Preserve VMFS Datastore) option.

Answer 5

I recently faced a similar issue with a RAID card passthrough problem in ESXi 7 on a Dell R730XD server. After accidentally enabling passthrough, I couldn't revert the setting, and all datastores disappeared. The root cause was /bootbank pointing to a /tmp directory because it can't find a proper disk device to persist bootbank info.

Solution:

1. Use this command to disable passthrough immediately:

esxcli hardware pci pcipassthru -a -d=0000:03:00.0 -e=false

2. Persist the settings:

a. Generate a new state.tgz file:

/sbin/backup.sh 1

b. Copy state.tgz to the correct location:

cp /tmp/_yourdir/state.tgz /vmfs/volumes/BOOTBANK1/

3. Reboot the server.

This approach restored the RAID card functionality immediately and persisted the correct settings without needing multiple restart cycles.

Answer 6

I realize that this is an old post but for all those that may end up here anyway, this may be helpful...

I believe the answer from John Doe (edited by Thomas) is the correct one but if your ESXi had a chance to create a backup of the config before you enabled passthrough, there may be an even simpler solution.

Reboot ESXi and at the boot screen press Shift-R to enter ESXi Recovery Mode. If you have a backup of the config, you can select it on that screen and it should fix the issue.

Some more details from http://pleasework.robbievance.net/howto-the-wrong-way-to-use-vmware-directpath/:

When you make any changes to ESXi, those changes are committed only to the in-memory configuration and thus will not persist after a reboot. To combat this, VMware has a shell script called /sbin/auto-backup.sh that runs automatically. What this script does is take all of the collective configuration files (including esx.conf) and stores them in a compressed file called local.tgz. That file is then compressed again and saved as state.tgz. Two copies of this file exist on two different partitions on the local file system, each from different points in time. Therefore, to correct the issue above, it appears all I needed to do was reboot the server and when ESXi was booting, press Shift-R to enter recovery mode and select the alternate bootbank.