[Solved] Failed WAR deploy

[shuffly2]

Hi team,

 

without any action my Java WAR application went down about 2 hours ago.

I am trying to deploy it again from the Java Tomcat section, but the deploy is failing with errors.

Is it possible to know the reason? Everything was working fine until 2 hours ago and I did not change anything within it. 

 

User: shuffly

Domain: shuffly-backend.heliohost.org

 

Thanks for the support

[wolstech]

If I had to guess, it's the same issue that's been going on a lot on Tommy the past few weeks. Tomcat keeps crashing.

 

Last time Krydos posted an error for a war issue on Tommy, the error was java.lang.OutOfMemoryError due to Tomcat having ran out (probably someone running bad code). I'd bet something similar this time around as well.

 

Lets have Krydos take a look.

 

 

[Krydos]

For the past month or so your .war has been crashing Tomcat on Tommy for everyone. You've caused a very considerable amount of downtime for everyone who uses java on Tommy.

 

The first few times I just assumed was a fluke, and then I upgraded Tomcat to version 9.0 hoping it would solve the issue. After it continued crashing for something like the 10th time I actually put in the effort to troubleshoot the problem. What would happen is after a day or two of running without a restart Tomcat would just randomly go completely unresponsive. It wouldn't even shut down cleanly using bin/shutdown.sh. So I'd have to send it a signal 9 and restart it. The log files complained about running out of memory, but after some research I found that it was more likely it was running out of available threads. I doubled the number of available threads to Tomcat, and it seemed to help. It would take roughly twice as long for Tomcat to crash, but it was still crashing. Then I went through the very tedious task of analyzing the stack trace for thousands of active threads on Tomcat that had been running for over a day, but hadn't crashed entirely yet. What I found was that your .war is creating thousands of threads named pool-512-thread2, etc. and when the system reaches its maximum it shuts down. Just to be absolutely sure it was you causing the issue I undeployed your .war and Tomcat has been running for quite a while now, and the thread count is about 60 rather than several thousand.

 

Please test your .war thoroughly on your home computer to make sure you have fixed the thread bug, and when you're confident you have it fixed let us know and I'll remove the block that prevents your .war from being deployed.

[shuffly2]

Hi Krydos,

Thanks for the response and sorry for the problem. I did test the application on my machine and it was not creating the problem, probably because i did not have the same traffic as the application on the server. Sorry again for everything.

 

I will change the code and fix this problem. Just to be ready: once that it will be changed, what should I do to deploy it again? Reply to this case to let you know about it?

Thanks for the support.

 

Best regards

[Krydos]

Yes, reply to this post when you're certain the issue is fixed and I will remove the block preventing your account from deploying a .war. Once the block is removed you'll be able to use the java page in cpanel to deploy as usual.

[shuffly2]

Hi Krydos,

 

I changed the code of my WAR file and executed some tests on my machine.

I did the analysis and now the problem should be solved: the number of thread is always under control during all the period of the load tests of around 10 hours that I executed on my machine.

 

Is it possible to have the java panel enabled again?

If the problem persists, feel free to remove my WAR again without any problem. I will try other ways to solve it.

 

Thanks a lot for the assistance and sorry for the inconvenient, if I knew that I would have changed my code immeditially.

 

Best regards

[Krydos]

Deployed.