Tuesday, March 10, 2015

NETWORK UNPLUGGED ON BLADE 460G7

Good day All,

Last couple of weeks has been extremely busy couldn't share much.. promised to myself saying to do at-least 2-3 post, let see how it goes..

Well the issue i am going to share has went for like 3 weeks before we buried to bed.
Ok this all started on a Blade in C7000 with 3 NIC's showing unplugged..
Initial Troubleshooting steps we performed still the issue persisited are
1. Reset the blade in OA
2. Re seating the blade Physically
3.Firmware Upgrade for NIC's
4.Motherboard replacement

After all the initial troubleshooting failing we decided to move to next level that is replace Virtual Connect Module.. As the NIC which was failing was going through one VC module 2 we thought we will replace that and give it a try.. Before we went to Business to get downtime, one more alert popped up and this time one more Server has same issue but this time the traffic is going through the other VC Module which is 1 in our case.

So like weeks of discussing with Vendor and internally we came up with the below plan

1. VC Domain and OA Backup
2. Reset the VC Modules
3. Replacing the VC Modules.. and for every one benefit replacing VC Module is not that easy..
Please find the link which will be useful
http://h20564.www2.hp.com/hpsc/doc/public/display?docId=mmr_kc-0115081
As we had to replace 2 VC Modules we had to get 2 Temp IP's and 2 Free slots in the c7000 enclosure to Flash the VC Modules to 4.20 and as our Frames was linked while using the vscu utility to update the Firmware you have to use -f option to force upgrade.
4. All the 3 steps fails last try was to replace Midplane.. and i already have post which i did, the challenges we faced to replace Midplane, click the Link and also more detail information on the vendor website

As it was our critical Server infrastructure we had to do lot of Pre-works to make sure we are all ready , like
1. Screen shots for all the Server profiles and VLAN Information
2. Shared up links
3.IP's etc..

So the big day came we anxiously started with our step which was to Reset the VC Modules..
Way to do it:

1. Please check which is the active VC Module( Incase you not aware , login to Virtual Connect Manager and click on Help and then click "About Virtual Connect Manager' it will list all the VC Modules listing which is Primary
2. Failover the VCM from Primary to Standby and reseat the VC Module
3.Failed over now Primary to Standby and reseat the VC Module
4. Guess what all the NIC's got connected and issue got resolved..

We are glad that we never had to do rest of the steps :)

Hope this helps someone and if anyone as question free to post it and glad to help!!!!





3 comments: