Nathan Johnson Posted September 21, 2011 Posted September 21, 2011 Hi, I'm wondering why Stevie's server load has been so high recently? It used to be at 1-5, but now its the opposite. Johnny is low and Stevie is high. Saving in the cPanel editor and refreshing pages has been a pain. Is there a reason for it to be so high? Hopefully it gets better soon
ecc456 Posted September 21, 2011 Posted September 21, 2011 Dont get me wrong, I love Heliohost. But what the hell has been going on lately?!! All accounts keep on getting queued and if its not that then the server load is 40+ !!! and the server keeps crashing! Is someone abusing resources? They need to be banned immediately
Guest xaav Posted September 21, 2011 Posted September 21, 2011 I really have no clue. When looking on htop, there aren't a few processes causing high load, there are many processes causing 10% cpu each, adding up to 100% cpu. Also, right now it seems we have 7528 accounts on stevie, which is only about 1/4 of the total accounts. I'm going to escalate this, because the high load is baffling me.
Guest xaav Posted September 21, 2011 Posted September 21, 2011 This support request is being escalated to our root admin.
Nathan Johnson Posted September 21, 2011 Author Posted September 21, 2011 This support request is being escalated to our root admin. Okay. Thanks for your help. And, sorry about the 2 posts. I meant for the second post to be the main one and the first wasn't supposed to be posted at all but it still posted. It was being slow and I didn't know it had already posted.
Krydos Posted September 21, 2011 Posted September 21, 2011 (Deleted the duplicate post.) I've also noticed that stevie has been rather loady lately, and I looked at the processes too and I couldn't see any particular user abusing the system either. (Merged similar topic.)
Nathan Johnson Posted September 21, 2011 Author Posted September 21, 2011 (Deleted the duplicate post.) I've also noticed that stevie has been rather loady lately, and I looked at the processes too and I couldn't see any particular user abusing the system either. (Merged similar topic.) (Thanks for deleting the duplicate post.) The server load is starting to come down a bit. Hopefully it says like that
anush Posted September 21, 2011 Posted September 21, 2011 (Deleted the duplicate post.) I've also noticed that stevie has been rather loady lately, and I looked at the processes too and I couldn't see any particular user abusing the system either. (Merged similar topic.) (Thanks for deleting the duplicate post.) The server load is starting to come down a bit. Hopefully it says like that but still my site http://techstream.org/ takes around 10s to load and some times it gets timed out also. I request the administrators to check after the problem with Steve and take necessary decisions at the earliest. Thank you.
Guest xaav Posted September 21, 2011 Posted September 21, 2011 The load is down because I limited MaxClients to help reduce load. This seems to indicate that someone is spamming requests, but I don't know how to find out if this is the case, and I definitely don't know how to stop it.
Krydos Posted September 21, 2011 Posted September 21, 2011 I request the administrators to check after the problem with Steve and take necessary decisions at the earliest. That's why this thread is in the escalated requests forum because we are checking the problem and taking necessary actions to correct the problem.
Guest xaav Posted September 22, 2011 Posted September 22, 2011 At this point, I've installed mod_evasive and gotten the load down to reasonable levels, however it appears apache is still being bogged down servicing those requests, causing legitimate requests to take much longer. cPanel and FTP seem unaffected.
rvt Posted September 23, 2011 Posted September 23, 2011 The load is down because I limited MaxClients to help reduce load. This seems to indicate that someone is spamming requests, but I don't know how to find out if this is the case, and I definitely don't know how to stop it. Check the access logs. Perhaps this is along the lines of a DOS attack? Given that adjusting the max clients helped this issue, that is where my gut tells me to start looking.
Guest xaav Posted September 23, 2011 Posted September 23, 2011 Thanks for that advice. Seems 94.249.185.37 is DOSing stevie. According to a traceroute, that IP originates in germany. I wonder what someone would have against us... What I'm wondering now is why mod_evasive isn't picking it up. Okay, I've added that IP to iptables, and it looks like the problem has been fixed. However, I still don't understand why mod_evasive wasn't picking the DOS attack up...
rvt Posted September 24, 2011 Posted September 24, 2011 What I'm wondering now is why mod_evasive isn't picking it up. Okay, I've added that IP to iptables, and it looks like the problem has been fixed. However, I still don't understand why mod_evasive wasn't picking the DOS attack up... In short, mod_evasive isn't everything it is advertised to be. If I recall correctly when looking at it before, it doesn't work well in a shared hosting environment because of how the Apache logging system works. In reality, it is a way for hosting companies to say, "We have custom anti-DOS protection which stops 90+% of DOS attacks against us." Glad you found something though. For future reference, if you (or anyone) is getting DOS'ed, it will ALWAYS show up in the access logs. If you start getting DDOS'ed (where it is distributed across hundreds or even thousands of IP's), then looking at the logs can be very challenging because the attack is spread out across a bunch of IP's making it harder to track.
Guest xaav Posted September 24, 2011 Posted September 24, 2011 The thing is, we don't have logging on for performance reasons. I had to enable it to find out where the attack was coming from. I'm going to uninstall mod_evasive since it wasn't that helpful, and try dos deflate instead. Installed. Hopefully these attacks will be prevented in the future.
Recommended Posts