I am getting random "500 Internal Server Errors" only for very short periods of time, help is appreciated

Hello, and Happy Holidays to all

Sorry to bother these days, but I am getting random “500 Internal Server Errors”

Internal Server Error
The server encountered an internal error or misconfiguration and was unable to complete your request.

Please contact the server administrator at root@localhost to inform them of the time this error occurred, and the actions you performed just before this error.

More information about this error may be available in the server error log.

ERROR LOG:

[Thu Dec 30 00:12:29.137142 2021] [fcgid:warn] [pid 30732] [client IP:54898] mod_fcgid: read data timeout in 31 seconds, referer: mywebsite.com is available for purchase - Sedo.com
[Thu Dec 30 00:12:29.137187 2021] [core:error] [pid 30732] [client IP:54898] End of script output before headers: index.php, referer: mywebsite.com is available for purchase - Sedo.com

Those 2 lines were repeated in the server error log ten times and then it worked again, the Error 500 was about 4 or 5 minutes.
I checked the documentation here: 500 Internal Server Error

  1. PHP is Workin as per <?php phpinfo(); and the website and admin modules are working almost all the time except on those random occasions of Error 500
  2. Permissions are fine otherwise would not able to run most of the time
  3. In my php7.3/php.ini configuration I do not have register_global as recommended in Grav Documentation
    register_global = On
  4. The VPS have other database websites with Options -Indexes that do not have error 500 so this should not be an issue
  5. I uncommented RewriteBase / in the .htaccess but then index.php was not found so I left commented and it worked again.
  6. I did # chmod 755 cache/ -R because it was 6775
    But I got this error after clearing the cache:
    InvalidArgumentException
    The directory “/home/mywebsite/public_html/new/cache/doctrine/3fc30ab2” does not exist and could not be created.
    So I set permission back to 6775 and it worked again.

The website is on the test, running provisionally in a subdirectory: /new/
Sorry, but I do not know how to solve a problem that only occurs sometimes for a short period of time.
Any hint is appreciated.

Regards and Happy New Year :champagne: :fireworks:

Environment:
Centos: 7.9.2009
PHP: 7.3.29
Apache: 2.4.6
Grav was upgraded last night to latest versions including the plugins

It seems from my reading that you have a script that is running too long and times out.

It’s a fairly commonly mentioned error message (even the 31 seconds), eg here: php - mod_fcgid: read data timeout errors - Server Fault

These errors mean, that the scripts were running longer than 31 seconds and thus they were terminated, as your fcgid.conf says so. The standard timeout is 40 seconds btw. You can easily check this behaviour by writing a test.php:

<?php sleep(32); ?>

This should give you an error 500 and put this error in your logs.

One fix is set the timeouts higher (if you can). But I guess you would ideally want to track down the root cause.

Thank you @ozfiddler . Yes that instruction <?php sleep(32); ?> causes timeout and 500 error. And probably if I increase the limit it will also give 500 error too. I did not know 40 is the standard, I will change it to 40, but I never touched that parameter, thanks for the hint.

Is there a way to monitor all Grav’s programs to find which one is causing the issue or is only index.php? I am using Grav Quark theme with very little changes.

Thanks, regards and happy and successful New Year.