[Answered] Running Background Cgi Process

[MasonIII]

One of the scripts I'm running on my site is intended to run in the background and check a site periodically for change. I managed this by writing the entire script and functions in python and uploaded it to the cgi-bin. Then at the very end, I put:

 

while True:

p.running()

time.sleep(600)

 

While testing this, it seemed to work as in it would stay running and if a change occurred, the rest of the script would run accordingly. However today, it has been often stopping for no apparent reason. In the error logs, I get

 

Timeout waiting for output from CGI script

 

Then for a few minutes afterwards I get a 500 server error when I try to restart the service. Looking around on the forums, this seems to indicate that I've reached the max amount of processes. I'm wondering if there is a correct way to implement running this script on the server continuously. It takes about 10-15 second to run (depending on how slow the site its checking is and how many times it needs to run as it takes in user input). As far as I know, since it works occasionally is that there is a problem when the way I implemented the continuous loop.

 

Thanks for the assistance.

[Krydos]

I think a better question would be "Why do you need to run a script continuously?"

 

Based on your own pseudocode it doesn't even do anything at all for 10 minutes. Wouldn't it work better to use a free cron service like https://www.setcronjob.com/ to hit your page once every 10 minutes, and then you wouldn't have to waste one of your limited processes just sleeping most of the time?

[MasonIII]

I didn't know a resource like that existed. Sorry I'm a bit new here. Thanks for showing that to me.

 

However, the reason I wanted to run it continuously is because it takes input from a form on my site and then checks the site according to said information. This information is stored in a text file that the script reads and edits as results are found or users request removal of their information. By using a free cron service, the information wouldn't be able to be dynamically altered by user input (I think, correct me if I'm wrong please).

 

Would using a cronjob to hit the site link to run the script every minute as an alternative to figuring out how to keep the script running for more than an hour be bad for some reason?

 

Thanks again for the help.

[Shinryuu]

Would using a cronjob to hit the site link to run the script every minute as an alternative to figuring out how to keep the script running for more than an hour be bad for some reason?

 

cronjobs put a high load on servers, we limit them to 2/user/day. Because of that same reason setcronjob.com only allows 50 jobs/user/day before you need to pay up. To call the script every minute would be 60*24=1440 uses, which is covered by their $10/yr plan.

[MasonIII]

Good point. I guess an alternative is to work with the random stopping of my script after about an hour of running and use setcronjob to hit the link every hour to start it up again.

 

If you have any idea why it stops though, that would be appreciated so I can better improve my code.

 

Thanks for the help.

[Shinryuu]

Don't know about Python but PHP has a configuration for max execution time, maybe Python has something similar.