GNAX Down

[rlhanson]

I just had a client switch to me last week and today we're down all day. It's frustrating but at least GH deals with it nicely, just saw a post on another forum where a host just buried themselves with what seems to be a flipant response to the problem. lol
http://www.webhostingtalk.com/showth...=932654&page=5

[salmon]

Has GNAX and NetDepot always been one of the same company?

[Matt]

Given that this is a somewhat recently recurring problem, are things becoming slack there at the DC?

Lightning strikes?

The servers that are not online all appear to have hardware issues related to the strike. We are still confirming this and making a plan of attack for the best way to get these units back online, there will be additional downtime for these devices are replacement parts are installed.

[Matt]

Has GNAX and NetDepot always been one of the same company?

No, netdepot is newer. They are 2 different companies with 2 different services, but with a common owner of the facilty.

[rlhanson]

Matt -
I've forwarded the info to my clients, thanks so much for the update! I know the next question will be what the anticipated repair time will be...
Do you have any idea how many servers are requiring repair at the facility or are they giving an ETA at all?

[Matt]

It depends on what the issue is for the affected device, I don't have a list here yet so I can't anticipate things yet, but I will be posting here as soon as I do know so that you can notify your customers as soon as possible.

[Matt]

Missy and Penny have bad power supplies, they are being replaced now. I expect 30-60 minutes on those machines and then fsck may be required after they have power running again.

[charlesh]

I didn't mean to be critical of Gnax. Of course it's understandable if the hardware is fried, that's way out of their control.
You guys, Glowhost, do a great job of response. I am always impressed by your service. I just have no reference or idea about them (GNAX). All I see is that for 5 years I haven't ever had this major of an interruption, so hence the question, that's all.

I hope that all our data is in-tact. That's my biggest worry and keeps me up at night, even though I have a physical backup drive. That doesn't do too much good if the server goes up in a puff of smoke...

By the way, Lynne has over 400 posts and I have around 150. We're both Master Glow Jedis. I think Lynne needs a new title when all this is over.

[rlhanson]

Thanks for the update, it helps to give my clients something about what happened.

[Matt]

Well in all fairness I think there could be more done to mitigate a strike like this. How it was able to get past all the fancy new electrical components they have put into this new datacenter I am surprised that the surge was powerful enough to make is so far in that it was able to fry multiple power supplies.

I am not too happy about it right now because in addition to the hardware that smoked, the UPS that are supposed to prevent any power loss and also add another layer of clean power to the machines before they hit the servers also failed (well, one of them did) and that too should have never happened. That caused nearly everything we had in there to have unclean shutdown, which in turn causes Lunx to run filesystem checks which can take hours to do.

In general I happy happy with GNAX over the years. We've had servers in most all of the neutral datacenters and they all have issues like this, some more than others. The reason we have been in GNAX so long is because they seem to have less of these issues than most of the others.

But they have some explaining to do about this electrical problem if you ask me.

I am with you about data and backups. Hopefully you guys all have your data in at least 3 places. If not, now would be a good time to consider it. Typically you wont loose your primary drive and backup at the same time even if they are on the same machine unless the machine itself physically catches fire (or similar physical meltdown) but anything is possible when we are talking about these types of issues.

Lynne needs 100 more posts to get the next title I believe

Er, scratch that, only 9 more to go for her