HP BLADE WILL NOT POWER UP AFTER MIDPLANE REPLACEMENT

All,

So other day we started with a planned activity of replacing a midplane on a C3000 enclosure..
what was the issue, let me elaborate more..

Issue: Fan speeds on the enclosure was always high at 90-95% and even 100% and 1 or 2 fans always keep failing..

Troubleshooting done before we wanted to replace midplane:

1. We update ILO firmware that did't help
2. replace fans couple of times and that failed too
3.change HP power settings that did't help

So last option the next step was to replace mid-plane..

We had about 150 VM's but fortunately we had a other ESX host on the cluster on different chassis and we moved all the VM's to the host and we had no down time...

Well we started with replacing the mid-plane that was pretty quick but after that none of the Blades started to power up..
So steps done to troubleshoot

1. We did a reset of all the VC Flex modules and that did't help
2. We tried power up the blades and still not luck.


so we decided to move the blade to a different enclosure then we tried powering up and i was able to power it on.. then we where confident that its VC issue..


Seeing that we said oh man looks like VC database is gone...


After some research we say link on HP website and said to our self what a miss... it says that we need to put back the old serial number of midplane so that VC can see the enclosure settings.. well the article made sense..

So we removed all the blades out, removed all the VC Flex modules and opened a Putty session to Active on-board administrator.. and run the following command
SET ENCLOSURE SERIAL_NUMBER

and then we put back the active VC Flex module and went to Management Console and said

1. Create a new VC domain
2.  In the Next screen  say  to repair the VC database with  changed OA module
click yes..

that's all it started to repair the VC database and all the settings started to come all..

The good part of all these is i took a VC database backup before we proceed with change.. so even though it was a mess we were confident that anything mess up we can recovery it using the restore....


So the big lesson learned is always check all possible scenarios .. it does't matter how small the change is we should always think about the worst possible scenario and always take good backups...

Till i encounter the next issue you all have a good day..

Vmtool Uninstall/Reinstall on Registry windows 2003

Vmtool Uninstall/Install on Windows 2003

Summary

i) If you are unable to upgrade or Uninstall Vmtools in Windows 2003.

ii)Unable to remove Vmtools from Add/Remove programs.

You can perform the following step to install Vmtools.

Note:

Before uninstall please take backup of Registry for safer side.

1. Log in to Virtual Machine which having VMtools issues.

Windows 2003 virtual machines

1. Start the virtual machine and log on as the Administrator.
2. Open the Windows Registry editor. Click Start > Run, type regedit, and press Enter.
3. Delete these registry keys if they exist:
• HKEY_CLASSES_ROOT\Installer\Features\005014B32081E884E91FB41199E24004
• HKEY_CLASSES_ROOT\Installer\Products\005014B32081E884E91FB41199E24004
• HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Installer\Features\005014B32081E884E91FB41199E24004
• HKEY_LOCAL_MACHINE\SOFTWARE\Classes\Installer\Products\005014B32081E884E91FB41199E24004
• HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Installer\UserData\S-1-5-18\Components\0B150AC107B12D11A9DD0006794C4E25
• HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\{3B410500-1802-488E-9EF1-4B11992E0440}
• HKEY_LOCAL_MACHINE\SOFTWARE\VMware, Inc.
4. Some services might need to be removed manually from the registry. Delete these as well if they exist:
• HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\VMTools
• HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\VMUpgradeHelper
• HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\VMware Physical Disk Helper Service
• HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\vmvss

6. Reboot the Server

7. Go to Vmware Console and Start installing Vmware tools.

8.Once installation is completed reboot the servers

9. To check the Status of Vmtools whether installed properly or not go to Vm summary you can find it..

  

REMOTE DESKTOP CONNECTION ERROR, AFTER 2 ACTIVE SESSIONS

Good day All,
The other day when i was working a colleague of mine pulled me and showed me a Server when tried to RDP was showing the below error...

So he started to ping  and there was ping response and he even tried with file share and it was working fine too.. So he determined that Server is online.. so he logged on to Server via ILO and he saw server is health and online..So he used the console and logged in..

He poked around could't find any problem still he was not able to RDP to server.. so he disconnected all the RDP sessions which were active and disconnected state and boom RDP started to work...
now that puzzled me too why after disconnecting sessions we were able to RDP ??????

Well now i had to replicate the issue .. So i took 2 RDP Session to same server and when i tried the 3 RDP session boom i got the same error, usually if you try a 3 rd session it should still open a connection and pop us with message saying if you want to disconnect any active user session, but this is strange as it was not going that far just failing with error "THIS COMPUTER CAN'T CONNECT TO THE REMOTE COMPUTER"

So first thing i opened was Remote desktop Host configuration mmc..  and checked the connections properties..

well i did't notice anything strange so next thing was checking the local group policy and local security policy and that was looking good too..
So i power on my lab VM and started to check the connection settings.. it was all the same.. so i said let me delete the RDP listener and re-create it again.. so I deleted the RDP-Tcp connection on my lab VM and started the new connection wizard..

So when i came to this option i kind of stopped and said wait a mint let me check something on the registry...

So browsed to below path in registry

HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Control\Terminal Server\WinStations\RDP-Tcp

MaxInstanceCount=dword::00000002

strange.. so looks like some one when creating the listener instead of choosing concurrent "Unlimited connections"  selected the connections to 2 which was restricting any connections more than 2 sessions..


So changed the registry settings as below and that fixed the issue...

MaxInstanceCount=dword:ffffffff   - unlimited current connections



Hope this helps someone... thanks for reading through.. and have a good day...

WHERE DID MY SPACE GO?????

The other day when i was working some one called and said that Drive D which is 64 GB shows only 300 MB free space..When they run tree size report it shows only 10 GB of all the contents, so where did 54 GB space go? and they told me that only way to reclaim the disk space is to run chkdsk.. that kind of puzzled me..

So i started to dig more..  first thing i checked was if any backup software is installed, yup  i saw IBM TSM backup software installed so went ahead and checked any backup was still running.. well in this case it was not...
Next thing i went to check was if Volume shadow copy Service was running and yup it was running too.. so i kind of smiled to myself knowing that i know where is the space now?

So i opened a CMD prompt and typed

VSSADMIN LIST SHADOWSTORAGE

Note: the below pic doesn't show 54 GB consumed, when i was writing this article it was still up to 5 GB

So it shows that  100% of Volume D will be used for storing shadow copies when TSM initiates Backup.
Well now that i know what is consuming the space, i just restarted the volume shadow copy services and yup that returned back all the space..

Most storage software after completion of backup usually deletes the  shadow copies on the drive, so before you do something check to see if any backup is still running.. if still a issue then you do the following things as below..

1. Request backup to exclude Drive from using shadow copies
2.You can use VSSADMIN tool and configure such that it does't use the Drive


Hope it helps someone.. thanks for reading through.. and have a good day