Python + Flask - 500 Internal Server Error

[talesaz]

Hi guys,

I’m getting a 500 Internal Server Error when running an application written in Python with Flask. Is there any tool on Heliohost that allows viewing logs or troubleshooting to identify the cause of the error?

Thanks! 

[wolstech]

There's a Logs option in Plesk that show Python errors...they're a bit hard to read because of how Plesk formats them,. but the whole thing including the trace is there. The last line kind of explains it though...your error is this:

ModuleNotFoundError: No module named 'pymysql'

The modules available on Tommy are shown here: https://krydos1.heliohost.org/pyinfo/info3.12.py You can also request that module be added if you need it.

[talesaz]

I am encountering an intermittent issue with my Flask application using Flask-SQLAlchemy. The application works for a while, but after refreshing the page a few times, it throws the following error:

RuntimeError: Either 'SQLALCHEMY_DATABASE_URI' or 'SQLALCHEMY_BINDS' must be set.

Error Context: The error appears in the Apache logs as follows:

mod_wsgi (pid=3543435): Exception occurred processing WSGI script '/home/talesaz.heliohost.us/httpdocs/bma/flask.wsgi'.
RuntimeError: Either 'SQLALCHEMY_DATABASE_URI' or 'SQLALCHEMY_BINDS' must be set.

What I Have Tried:
1) Direct Configuration in config.py: I am not using .env files anymore. All configurations are directly set in config.py.
2) Database Connection Test: I added a test in run.py to check the database connection.
3) Error Handling: I added a generic error handler to capture unexpected exceptions.

Behavior Observed:
- The application works initially and connects to the database successfully.
- After refreshing the page a few times, the error appears.
- Refreshing the page again sometimes resolves the issue temporarily.
- When running the application locally (on my machine) and connecting to the same MySQL database hosted on HelioHost, everything works perfectly without any issues.
- The problem only occurs when the application is hosted directly on HelioHost.

[wolstech]

Krydos might have more of an idea about why this would happen. I'm not terribly familiar with Python...escalating.

[Krydos]

Did you request WSGI restart access?

Quote

Multiple Apache processes are running on the server, and each time you refresh your site you are randomly assigned to one of these processes. If that particular process has already displayed your site, it shows the cached version of your code; otherwise, it shows the new code changes. This means that during the first 2 hours after a site change, you may intermittently see old or new content, depending on which process you get assigned to. This situation will resolve when Apache is restarted, which happens every 2 hours.

Source: https://wiki.helionet.org/Flask#WSGI_Uses_Caching

[talesaz]

33 minutes ago, Krydos said:

Did you request WSGI restart access?

Source: https://wiki.helionet.org/Flask#WSGI_Uses_Caching

Sorry, I didn’t read this article earlier, so I didn’t restart. Could you grant me WSGI Control Access? My username is talesaz, and my URL is talesaz.heliohost.us:https://talesaz.heliohost.us/bma.

[Krydos]

WSGI control access has been enabled on the domain talesaz.heliohost.us. To restart your Flask app and load new code changes in simply edit /home/talesaz.heliohost.us/httpdocs/bma/flask.wsgi. Adding a blank line, removing a blank line, adding a space, or removing a space are examples of editing the file. As long as the last modified timestamp changes it will clear the server cache. Let us know if you run into any issues.

If you get a 500 error check the error logs, and if there isn't much useful information in there let us know and we may be able to get some more logs for you. Plesk isn't very good at filtering Flask logs into the right user accessible logs. At least with WSGI restart access you won't get 500 errors from bugs that you've already fixed, which is very confusing and frustrating.

[talesaz]

Now it works! Thank you so much! Sometimes the website still shows a 500 Internal Server Error, but much less frequently than yesterday, and refreshing the page usually resolves it. The logs don’t provide much information about how to address the issue:

2025-04-15 23:28:23 Error 186.205.7.186 500 GET /bma/flask.wsgi/usuarios/cadastrar HTTP/1.0https://talesaz.heliohost.us/bma/flask.wsgi/conteudo Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36 1000 Apache SSL/TLS access 2025-04-15 23:28:25 Access 186.205.7.186 200 GET /bma/flask.wsgi/usuarios/cadastrar HTTP/1.0https://talesaz.heliohost.us/bma/flask.wsgi/conteudo Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36 10.8 K Apache SSL/TLS access

If you could provide more detailed logs for this event at this timestamp, I would greatly appreciate it.

[Krydos]

[Tue Apr 15 23:37:05.906471 2025] [wsgi:error] [pid 291720:tid 291730] [remote 186.205.7.186:53756]     ret = self._execute_context(
[Tue Apr 15 23:37:05.906475 2025] [wsgi:error] [pid 291720:tid 291730] [remote 186.205.7.186:53756]           ^^^^^^^^^^^^^^^^^^^^^^
[Tue Apr 15 23:37:05.906480 2025] [wsgi:error] [pid 291720:tid 291730] [remote 186.205.7.186:53756]   File "/usr/local/python3.12/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1818, in _execute_context
[Tue Apr 15 23:37:05.906484 2025] [wsgi:error] [pid 291720:tid 291730] [remote 186.205.7.186:53756]     self._handle_dbapi_exception(
[Tue Apr 15 23:37:05.906489 2025] [wsgi:error] [pid 291720:tid 291730] [remote 186.205.7.186:53756]   File "/usr/local/python3.12/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 2352, in _handle_dbapi_exception
[Tue Apr 15 23:37:05.906495 2025] [wsgi:error] [pid 291720:tid 291730] [remote 186.205.7.186:53756]     raise sqlalchemy_exception.with_traceback(exc_info[2]) from e
[Tue Apr 15 23:37:05.906499 2025] [wsgi:error] [pid 291720:tid 291730] [remote 186.205.7.186:53756]   File "/usr/local/python3.12/lib/python3.12/site-packages/sqlalchemy/engine/base.py", line 1812, in _execute_context
[Tue Apr 15 23:37:05.906504 2025] [wsgi:error] [pid 291720:tid 291730] [remote 186.205.7.186:53756]     context = constructor(
[Tue Apr 15 23:37:05.906509 2025] [wsgi:error] [pid 291720:tid 291730] [remote 186.205.7.186:53756]               ^^^^^^^^^^^^
[Tue Apr 15 23:37:05.906513 2025] [wsgi:error] [pid 291720:tid 291730] [remote 186.205.7.186:53756]   File "/usr/local/python3.12/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 1429, in _init_compiled
[Tue Apr 15 23:37:05.906518 2025] [wsgi:error] [pid 291720:tid 291730] [remote 186.205.7.186:53756]     self.cursor = self.create_cursor()
[Tue Apr 15 23:37:05.906523 2025] [wsgi:error] [pid 291720:tid 291730] [remote 186.205.7.186:53756]                   ^^^^^^^^^^^^^^^^^^^^
[Tue Apr 15 23:37:05.906527 2025] [wsgi:error] [pid 291720:tid 291730] [remote 186.205.7.186:53756]   File "/usr/local/python3.12/lib/python3.12/site-packages/sqlalchemy/engine/default.py", line 1768, in create_cursor
[Tue Apr 15 23:37:05.906532 2025] [wsgi:error] [pid 291720:tid 291730] [remote 186.205.7.186:53756]     return self.create_default_cursor()
[Tue Apr 15 23:37:05.906536 2025] [wsgi:error] [pid 291720:tid 291730] [remote 186.205.7.186:53756]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[Tue Apr 15 23:37:05.906541 2025] [wsgi:error] [pid 291720:tid 291730] [remote 186.205.7.186:53756]   File "/usr/local/python3.12/lib/python3.12/site-packages/sqlalchemy/dialects/mysql/mysqlconnector.py", line 59, in create_default_cursor
[Tue Apr 15 23:37:05.906546 2025] [wsgi:error] [pid 291720:tid 291730] [remote 186.205.7.186:53756]     return self._dbapi_connection.cursor(buffered=True)
[Tue Apr 15 23:37:05.906550 2025] [wsgi:error] [pid 291720:tid 291730] [remote 186.205.7.186:53756]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[Tue Apr 15 23:37:05.906555 2025] [wsgi:error] [pid 291720:tid 291730] [remote 186.205.7.186:53756]   File "/usr/local/python3.12/lib/python3.12/site-packages/sqlalchemy/pool/base.py", line 1486, in cursor
[Tue Apr 15 23:37:05.906560 2025] [wsgi:error] [pid 291720:tid 291730] [remote 186.205.7.186:53756]     return self.dbapi_connection.cursor(*args, **kwargs)
[Tue Apr 15 23:37:05.906564 2025] [wsgi:error] [pid 291720:tid 291730] [remote 186.205.7.186:53756]            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
[Tue Apr 15 23:37:05.906569 2025] [wsgi:error] [pid 291720:tid 291730] [remote 186.205.7.186:53756]   File "/usr/local/python3.12/lib/python3.12/site-packages/mysql/connector/connection_cext.py", line 824, in cursor
[Tue Apr 15 23:37:05.906573 2025] [wsgi:error] [pid 291720:tid 291730] [remote 186.205.7.186:53756]     raise OperationalError("MySQL Connection not available.")
[Tue Apr 15 23:37:05.906581 2025] [wsgi:error] [pid 291720:tid 291730] [remote 186.205.7.186:53756] sqlalchemy.exc.OperationalError: (mysql.connector.errors.OperationalError) MySQL Connection not available.

MariaDB hasn't restarted for 8 hours, and has had 100% uptime during this error. How many database connections are you using? We limit people to 10 connections so if you try to open more than 10 connections it would explain why you can't open more connections. We used to offer unlimited database connections, but some jerk decided to take advantage of our generosity by opening 173 connections and never closing any of  them. One jerk always ruins the fun for everyone else.

[talesaz]

I understand that the database connection limit on HelioHost is restricted to 10 simultaneous connections. To ensure that my application does not exceed this limit, I tried two different approaches, but both resulted in errors.

First Approach: Using SQLAlchemy Advanced Configuration
I configured the SQLALCHEMY_ENGINE_OPTIONS as follows:

SQLALCHEMY_ENGINE_OPTIONS = {
    'pool_size': 5,
    'max_overflow': 2,
    'pool_timeout': 20,
    'pool_recycle': 280,
    'connect_args': {
        'connect_timeout': 10  # 10-second timeout for database connections
    }
}

However, this resulted in the following errors:

2025-04-16 03:19:26    Error    186.205.7.186    504    GET /bma/ HTTP/1.0        Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36    967    Apache SSL/TLS access
2025-04-16 03:19:47    Error    186.205.7.186    504    GET /bma/flask.wsgi/usuarios HTTP/1.0    https://talesaz.heliohost.us/bma/flask.wsgi/usuarios/11    Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36    967    Apache SSL/TLS access
2025-04-16 03:20:26    Error    186.205.7.186    Timeout when reading response headers from daemon process 'talesaz_wsgi': /home/talesaz.heliohost.us/httpdocs/bma/flask.wsgi    Apache error

Second Approach: Managing Connections Manually
I then tried to manually open and close connections for each request using the following code:

@app.before_request
def before_request():
    g.db = db.session()

@app.teardown_request
def teardown_request(exception):
    db.session.remove()

    
Unfortunately, this also resulted in errors:

2025-04-16 03:55:58    Error    186.205.7.186    500    GET /bma/ HTTP/1.0        Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36    986    Apache SSL/TLS access
2025-04-16 03:59:18    Error    186.205.7.186    500    GET /bma/ HTTP/1.0        Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/135.0.0.0 Safari/537.36    986    Apache SSL/TLS access

Could you please provide any guidance or tips on how to ensure that my application does not exceed the connection limit imposed by HelioHost? I would greatly appreciate any advice on how to configure SQLAlchemy or manage connections more effectively in this environment.

Thank you in advance for your help!