Generating Full,kernel dump from VMware Virtual Machines

Good day All,

Welcome back!!!

Lately we started to see lot of VM's going down and when we reach out to VMware they analyze and as usual asking us to go to Microsoft other than VMware issue.

Lot of the Virtual machines when they hung manual hard reboot is required.During the hung if you try to use keyboard to generate dump so far i was never successful and end up hard reboot the Server with no dump .

So for all these problems now we have tool call VMSS2CORE, this tool is internally used by VMware, now its available for download, click here

 Well wondering what this tool will do for you, if you go over link it explains in brief for starters .. lets assume  VM is hung so during the outage take a snapshot of the VM  and then reboot the VM.
After you took the snapshot go and browse datastore for the snapshot file and you will see a file with .vmsn as below..


download the .vmsn file and place it in the same directory where you downloaded and kept vms2core-win.exe


Now all you have to do the vmss2core command as below

Full memory dump for Windows 2003\2008 VM:
vmss2core-win -W filename.vmsn

Kernel dump for Windows 2003\2008 VM:
vmss2core-win -WK filename.vmsn


Full memory dump for Windows 2012 VM:

vmss2core-win -W8 filename.vmsn

Note:
I tested the tool on windows 2003\2008\2012.. WK which is to create Kernel dump works for Windows 2003/2008 but didn't work for Windows 2012.

If you go over the tools description it will tell you that you can create dump by suspending the VM and downloading the .VMSS file.

this is my take on taking the dump using the suspend state

Advantage:
Suspend state File size will be smaller when compared to Snapshot File size will be very helpful for larger Virtual Machines.

Dis-advantage:
You will be in outage when the Server hung, so putting in suspend mode and then wait till you download to local disk will consume lot of time which will add to your overall downtime window.
So if downtime is not a issue then try suspend state.


this helped me and hopefully this will someone, till next one hope you all have a good day!!!

Esxi host - Pause Flood Protection - VM's locked.

Good day All,
Welcome back!!!

Today will go over a issue we had couple of days ago on a Esxi host. Just to give a brief on our environment we have a HP C7000 Frame and 6 blades are running Esxi covering about 100 odd Virtual machines. So out of the 6 blade we had about 15 VM's running on a Esxi host and the host went to isolation mode.

During the outage we logged in to ILO and try to reboot the ESXi host to see if this kicks start's the HA feature but no luck as any traffic from the blade was completely blocked or stopped.

As we started to run out of time we went for work ground to fix the issue as during the outage we couldn't identify the root cause.
So following are the steps we performed

1. Disconnected the failed ESXi
2.Took a list of all VM's
3.Power down all the VM's
4. Removed the whole ESXi from cluster and from inventory
5.Browsed the datastore for each VM, add them to inventory and powered on
6, During the boot we choose the option "i moved it"

Note sure when Esxi host was hard rebooted why Datastore heart-beating didn't kick start HA and move the VM's that still we are investigating.

Hope this helps someone and until next one you all have a good day!!!