Wednesday morning 5am, I get a call that nothing it working at work. On my drive in I call our desktop support person to see what he can access….Nothing. The server room is dark. Not good.
We had storms Tuesday night, but we have a huge UPS and a generator…..yah…
I get into work, into the server room…The UPS is off!!! it has power its just off…..Once I turn it on.. all the servers and network equipment power up. I spend the rest of the morning cleaning up errors from the equipment going down hard.
Now I need to findout why the UPS was off. We call the vendor in and all they can do is gather logs and get back to me.
Thursday the vendor returns to replace batteries and power modules. They switch to pass through mode so they can switch out the parts. I start getting calls from our engineering that they cannot get to anything on the network. After a little checking their switch is out. This switch should be powerd by the UPS. So why did the servers stay up and the switch is down??? Oh yeah, the switch is plugged into another older UPS and the that UPS is plugged into our main UPS ( I know, not good) The old UPS has bad batteries. The vendor then explains that the bad ups could have sent a jolt when the power when out to our main ups that caused it to overload (we normally run at 85% utilized)
Side note: Because of our high utilization of the current UPS, I can’t move my Dell Blades onto the UPS. I either had to replace our main UPS or by a small UPS for the blades only. Plus if we end up going to POE on most of switches for VOIP that will likely overload the current UPS….
Then Thursday night, I log into the China server to look over a few things….(I’ll keep this short) The reinstalled 3 machines and have them running in a workgroup so they can have admin rights and share files off those machines instead of the server. AARRRRGGGG!!!!!!
Atleast late Friday we got the ISA server installed and OWA 2003 95% working. Nice to end the week on a good note.