Blade NIC showing unplugged

Good day All,

Welcome back!!!  Couple of weeks ago we had a interesting incident so thought of sharing to all so that it helps someone.
We got a alert ticket and one of the Production Server lost both Production Team NIC's on a blade and was unreachable , strange is we had NIC's for Backup and Management both are up and online.

So troubleshooting steps we performed but issue didn't get resolved:

1.We updated to latest drivers didn't work
2.Replaced NICs didn't work
3.Replaced mother board didn't work
4.Removed blade and put in next slot didn't work
5.Removed blade and put in second half of slot(bay 14) that didn't work too
6. We tried created new profile nope that didn't help


Solution:


 We created a internal only network and then assigned 1 NIC and then on  other Blade Server we assigned a NIC to same Internal network , assigned a 192.168.1.1 and 1.2 IP and pinged both NIC's it started to work.
Now we know that its not NIC issue as when we configured internally network its pinging between both blades.

So finally what we decided was to drop the existing VLAN and then just recreated a same VLAN,...Vola.. issue got resolved.
We reached out to Level 2 HPTS guys but they had no idea why something would like fix the issue.
Hopefully this will help someone, until next one you all have good day!!!!!!!!!!!!