jje Posted August 8, 2011 Share Posted August 8, 2011 At 5:00AM (PST) time, all servers experienced a Power Failure at our datacenter. The outage lasted for 18 hours, and ended at 10:00PM (PST) time. The power outage was out of our control since our servers are located in the Hurricane Electric datacentre, which was experiencing some power problems. At the moment, the Stevie and Cody are now back online and functioning normally, but Johnny is still recovering. Thank you for your cooperation in this situation. Link to comment Share on other sites More sharing options...
PenTester Posted August 8, 2011 Share Posted August 8, 2011 At 5:00AM (PST) time, all servers experienced a Power Failure at our datacentre. The outage lasted for 18 hours, and ended at 10:00PM (PST) time. The power outage was out of our control since our servers are located in the Hurricane Electric datacentre, which was experiencing some power problems. At the moment, the Stevie and Cody are now back online and functioning normally, but Johnny is still recovering. Thank you for your cooperation in this situation. thank you my site is back. Link to comment Share on other sites More sharing options...
kiddo Posted August 8, 2011 Share Posted August 8, 2011 At 5:00AM (PST) time, all servers experienced a Power Failure at our datacentre. The outage lasted for 18 hours, and ended at 10:00PM (PST) time. The power outage was out of our control since our servers are located in the Hurricane Electric datacentre, which was experiencing some power problems. At the moment, the Stevie and Cody are now back online and functioning normally, but Johnny is still recovering. Thank you for your cooperation in this situation. Thanks for info jje Now my site has back online And also tq to heliohost for this great hosting service Link to comment Share on other sites More sharing options...
Tjoene Posted August 8, 2011 Share Posted August 8, 2011 Thanks for the info!! Site is back again! Link to comment Share on other sites More sharing options...
i7Grendel Posted August 8, 2011 Share Posted August 8, 2011 You'd expect a proper datacenter to have a backup power generator. That could have prevented all of this. Link to comment Share on other sites More sharing options...
Xoviat Posted August 8, 2011 Share Posted August 8, 2011 Thanks for the info! Any idea when Johnny will be back up and running again? A rough estimate would be great. Are we talking a few of hours or sometime tomorrow? Link to comment Share on other sites More sharing options...
Guest Geoff Posted August 8, 2011 Share Posted August 8, 2011 You'd expect a proper datacenter to have a backup power generator. That could have prevented all of this. Well, on our datacenter's website, it does state: Your equipment will be hosted in our state-of-the-art colocation data centers: 24x7x365 on-site tech staff; direct, highly scalable and super fast connections to most of the top Internet backbones; power generator; UPS; cooling system; pre-action fire suppression system and facility-wide security. An ideal solution for server colocation, redundancy, VoIP, data center and IT facility outsourcing. I'm not sure what went wrong, apparently both the generator and UPS failed. Link to comment Share on other sites More sharing options...
Byron Posted August 8, 2011 Share Posted August 8, 2011 Well, on our datacenter's website, it does state: Your equipment will be hosted in our state-of-the-art colocation data centers: 24x7x365 on-site tech staff; direct, highly scalable and super fast connections to most of the top Internet backbones; power generator; UPS; cooling system; pre-action fire suppression system and facility-wide security. An ideal solution for server colocation, redundancy, VoIP, data center and IT facility outsourcing. I'm not sure what went wrong, apparently both the generator and UPS failed. Maybe that only applies to paid accounts? They host our servers for free. Link to comment Share on other sites More sharing options...
Piotr GRD Posted August 8, 2011 Share Posted August 8, 2011 They (Hurricane Electric) have a Twitter account where they posted: http://twitter.com/#!/henet/status/100203056502816769 Approx 6:15am PDT FMT1 UPS breaker tripped, reset and power restored. From a quick search, ie. on webhostingtalk.com, I can see that they have problems with power outages every now and then. The other company that I know with the same frequent problems with power is BurstNET Technologies. After all such power failures there are often problems with not all servers booting up properly. Even if the power outage is short (by looking at my monitoring results I think it was something around 22 minutes or so, but I may be wrong) the server may boot up properly and instantly OR may need intervention of the technician in the datacenter. And the time depends on how many technicians are there, how many servers they have to handle with and how much other things to do they have, does they check all the servers by themselves or do anything only on a support request of the server owner. The last thing is the queue, what priority the specific server has. If what byron wrote is true, then obviously HelioHost servers will be far far at the end of the queue of things to do / servers to manually restart in such situations. Link to comment Share on other sites More sharing options...
jje Posted August 8, 2011 Author Share Posted August 8, 2011 Retweeted. Link to comment Share on other sites More sharing options...
Ashoat Posted August 8, 2011 Share Posted August 8, 2011 HE's report: At approximately 6:00am PDT, at the Fremont 1 facility, power was interrupted to the datacenter floor when the main UPS input breaker tripped. The breaker was reset, the maintenance bypass closed, and power was restored at approximately 6:20am PDT. Support technicians on site visually inspected customer equipment located in the affected area, restarting any that needed assistance. UPS technicians will be onsite today to assist us in determining the cause of the failure. Moving forward, we are now going to replace this UPS system with another brand as soon as possible and we are going to go from an N+1 UPS configuration to a 1+1 electrical system, with each PDU having a static transfer switch to a separate electrical distribution system so that if something happens all the PDUs can be switched over. The time line for this is in the next few months due to the lead time on getting this type of equipment. The power failure actually only lasted about twenty minutes, but since most of our servers are running ext2 to avoid wasting too much CPU time on kjournald we had to run a long fsck on them. Not sure what's going on with Johnny right now. I can't SSH in, so I'll have to log in via VMWare vSphere when I get home. Link to comment Share on other sites More sharing options...
elmyra Posted August 8, 2011 Share Posted August 8, 2011 Thanks very much for the information. I was wondering what happened! I'm glad to see my site back up. But I have to ask, and I'm sorry if this is in the wrong place - but the topic seems related to me. My site is down for anywhere between 30 minutes to an hour every day. Is that because of power failures, or what? That's really my only complaint about your service. Otherwise, I think you guys do a fantastic job. Link to comment Share on other sites More sharing options...
Guest Geoff Posted August 8, 2011 Share Posted August 8, 2011 Is that because of power failures, or what? If you're on johnny, it's probably because of instability. Link to comment Share on other sites More sharing options...
elmyra Posted August 8, 2011 Share Posted August 8, 2011 Is that because of power failures, or what? If you're on johnny, it's probably because of instability. Thank you for the answer, but no - I'm on stevie. The link to my control panel starts with "http://stevie.heliohost.org:" If I should be asking in another topic, please let me know. I'm still new at this! *blush* Link to comment Share on other sites More sharing options...
Derpy Hooves Posted August 8, 2011 Share Posted August 8, 2011 I had the same problem as Elmyra actually. The site would just be down randomly throughout the day. Sometimes for a few seconds, sometimes for up to an hour. I was on Stevie. I basically moved everything to a server I've been paying for because the members of my forum were having so much trouble. I guess you get what you pay for Link to comment Share on other sites More sharing options...
Recommended Posts