(Problems not related to KeyHelp belong in the Offtopic forum)
I am sure, it's something in the KeyHelp configs.
Server operating system + version
Debian 12.2 Bookworm
Server virtualization technology used
None, physical server.
KeyHelp version + build number
23.2.1 (Build 3125)
Problem description / error messages
Ok, so this nightmare started about 2 weeks ago, I rebooted the machine for a kernel upgrade and mysql/mariadb wouldn't restart, it would just give out a very weird error about options. In any case, I had to manually remove all mariadb packages and configs, reinstalled the mysql server, copied back the databases and inno files and mysql started, but some websites started to experience problems, mainly the ones hosting nextcloud installations.
Revelant portion of apache2 config for the vhost in question :
Code: Select all
# PHP support
<FilesMatch "\.php$">
<If "-f %{REQUEST_FILENAME}">
SetHandler "proxy:unix:/run/php/keyhelp_sysadmins_php82.socket|fcgi://keyhelp_sysadmins_php82.socket"
</If>
</FilesMatch>
# Set proxy properties (the string "unique-domain-name-string" should match
# the one set in the FilesMatch directive.
<Proxy fcgi://keyhelp_sysadmins_php82.socket>
ProxySet connectiontimeout=5 timeout=600
</Proxy>
Code: Select all
[Fri Dec 08 14:53:49.715651 2023] [proxy:error] [pid 1385417:tid 140482944431808] (2)No such file or directory: AH02454: FCGI: attempt to connect to Unix domain socket /run/php/keyhelp_sysadmins_php82.socket (keyhelp_sysadmins_php82.socket:8000) failed
[Fri Dec 08 14:53:49.715781 2023] [proxy_fcgi:error] [pid 1385417:tid 140482944431808] [client 86.x.x.x:50719] AH01079: failed to make connection to backend: httpd-UDS
Code: Select all
srw-rw-rw- 1 sysadmins sysadmins 0 Nov 27 17:18 /run/php/keyhelp_sysadmins_php74.socket
srw-rw-rw- 1 sysadmins sysadmins 0 Nov 27 17:17 /run/php/keyhelp_sysadmins_php81.socket
srw-rw-rw- 1 sysadmins sysadmins 0 Dec 8 14:53 /run/php/keyhelp_sysadmins_php82.socket
lrwxrwxrwx 1 root root 39 Nov 30 16:19 /run/php/keyhelp_sysadmins.socket -> /run/php/keyhelp_sysadmins_php82.socket
Code: Select all
;
[sysadmins_php82]
; Unix user/group of processes
user = sysadmins
group = sysadmins
; Define how to accept requests
listen = /run/php/keyhelp_sysadmins_php82.socket
listen.owner = sysadmins
listen.group = sysadmins
listen.mode = 0666
Code: Select all
[08-Dec-2023 15:22:45 Europe/Bucharest] PHP Warning: Zend OPcache can't be temporary enabled (it may be only disabled till the end of request) in Unknown on line 0
Code: Select all
root@key:/var/log/php-fpm# /opt/keyhelp/php/8.2/bin/php -m|grep Zend
Zend OPcache
[Zend Modules]
Zend OPcache
Code: Select all
root@key:/var/log/php-fpm# dpkg -l|grep php8.2
ii libapache2-mod-php8.2 8.2.7-1~deb12u1 amd64 server-side, HTML-embedded scripting language (Apache 2 module)
ii php8.2 8.2.7-1~deb12u1 all server-side, HTML-embedded scripting language (metapackage)
ii php8.2-apcu 5.1.22+4.0.11-2 amd64 APC User Cache for PHP
ii php8.2-bcmath 8.2.7-1~deb12u1 amd64 Bcmath module for PHP
ii php8.2-bz2 8.2.7-1~deb12u1 amd64 bzip2 module for PHP
ii php8.2-cli 8.2.7-1~deb12u1 amd64 command-line interpreter for the PHP scripting language
ii php8.2-common 8.2.7-1~deb12u1 amd64 documentation, examples and common module for PHP
ii php8.2-curl 8.2.7-1~deb12u1 amd64 CURL module for PHP
ii php8.2-fpm 8.2.7-1~deb12u1 amd64 server-side, HTML-embedded scripting language (FPM-CGI binary)
ii php8.2-gd 8.2.7-1~deb12u1 amd64 GD module for PHP
ii php8.2-gmp 8.2.7-1~deb12u1 amd64 GMP module for PHP
ii php8.2-igbinary 3.2.13-1 amd64 igbinary PHP serializer
ii php8.2-imagick 3.7.0-4 amd64 Provides a wrapper to the ImageMagick library
ii php8.2-intl 8.2.7-1~deb12u1 amd64 Internationalisation module for PHP
ii php8.2-ldap 8.2.7-1~deb12u1 amd64 LDAP module for PHP
ii php8.2-mbstring 8.2.7-1~deb12u1 amd64 MBSTRING module for PHP
ii php8.2-mysql 8.2.7-1~deb12u1 amd64 MySQL module for PHP
ii php8.2-opcache 8.2.7-1~deb12u1 amd64 Zend OpCache module for PHP
ii php8.2-phpdbg 8.2.7-1~deb12u1 amd64 server-side, HTML-embedded scripting language (PHPDBG binary)
ii php8.2-pspell 8.2.7-1~deb12u1 amd64 pspell module for PHP
ii php8.2-readline 8.2.7-1~deb12u1 amd64 readline module for PHP
ii php8.2-redis 5.3.7+4.3.0-3 amd64 PHP extension for interfacing with Redis
ii php8.2-soap 8.2.7-1~deb12u1 amd64 SOAP module for PHP
ii php8.2-xml 8.2.7-1~deb12u1 amd64 DOM, SimpleXML, XML, and XSL module for PHP
ii php8.2-zip 8.2.7-1~deb12u1 amd64 Zip module for PHP
root@key:/var/log/php-fpm#
An error from the nextcloud log :
Code: Select all
{"reqId":"fSNbZNsMoL0A0UMh6GdI","level":3,"time":"2023-11-27T13:22:50+00:00","remoteAddr":"86.x.x.x","user":"Victor","app":"PHP","method":"PROPFIND","url":"/remote.php/dav/files/Victor/Kituri","message":"Zend OPcache can't be temporary enabled (it may be only disabled till the end of request) at Unknown#0","userAgent":"Mozilla/5.0 (Windows) mirall/3.10.1stable-Win64 (build 20231025) (Nextcloud, windows-10.0.22631 ClientArchitecture: x86_64 OsArchitecture: x86_64)","version":"27.1.3.2","data":{"app":"PHP"}}
To work properly just like before.
Actual result
Doesn't work.
Steps to reproduce
No idea, I just rebooted the server and all hell broke loose.
Additional information
(e.g. recent changes to the server, excerpts from log files (/var/log/*, /var/log/keyhelp/php-error.log, etc.))
I've put an phpinfo file trying to debug issues and it's available on https://cloud.sysadmins.ro/blah.php if anyone wants to see it.
Other problems I'm experiencing are with dovecot.
I have a user with about 80gb of mails and I'm unable to raise the vsz_limit from the Dovecot configs (I raise it but Dovecot ignores it)
Code: Select all
2023-12-08T14:40:23.534501+02:00 key dovecot: imap(lorena@xxx.com)<1384136><uGzy4v4LL8RWfgeL>: Disconnected: Logged out in=212 out=1163
2023-12-08T14:40:23.592587+02:00 key dovecot: imap(lorena@xxx.com)<1384133><mwAi4/4LMsRWfgeL>: Fatal: block_alloc(4194304): Out of memory
2023-12-08T14:40:23.598097+02:00 key dovecot: imap(lorena@xxx.vom)<1384133><mwAi4/4LMsRWfgeL>: Error: Raw backtrace: #0 test_subprocess_fork[0x7f1749b95a20] -> #1 backtrace_append[0x7f1749b95c90] -> #2 backtrace_get[0x7f1749b95e20] -> #3 execvp_const[0x7f1749ba2f90] -> #4 i_syslog_fatal_handler[0x7f1749ba3900] -> #5 i_fatal_status[0x7f1749af9758] -> #6 i_fatal_status[0x7f1749afda58] -> #7 pool_alloconly_create_clean[0x7f1749bc4f30] -> #8 hash_table_create_direct[0x7f1749ba8a10] -> #9 maildir_uidlist_sync_next_uid[0x7f1749d091e0] -> #10 maildir_uidlist_sync_next[0x7f1749d096d0] -> #11 maildir_list_index_update_sync[0x7f1749d02750] -> #12 maildir_list_index_update_sync[0x7f1749d031f0] -> #13 maildir_list_index_update_sync[0x7f1749d036a0] -> #14 maildir_storage_sync_force[0x7f1749d037a0] -> #15 maildir_file_do[0x7f1749d09a70] -> #16 maildir_sync_is_synced[0x7f1749cfe280] -> #17 mail_get_stream_because[0x7f1749cccdd0] -> #18 index_mail_prefetch[0x7f1749d43010] -> #19 mail_prefetch[0x7f1749cc82a0] -> #20 index_search_get_mail[0x7f1749d4b2a0] -> #21 index_storage_search_next_nonblock[0x7f1749d4b800] -> #22 mailbox_search_next_nonblock[0x7f1749cd97d0] -> #23 mailbox_search_next[0x7f1749cd9820] -> #24 imap_fetch_begin[0x55f6e1100170] -> #25 imap_fetch_more[0x55f6e1100550] -> #26 cmd_fetch[0x55f6e10f4c00] -> #27 command_exec[0x55f6e10fd880] -> #28 cmd_x_cancel[0x55f6e1103510] -> #29 cmd_x_cancel[0x55f6e1103510] -> #30 cmd_x_cancel[0x55f6e1103510] -> #31 client_handle_input[0x55f6e1103880] -> #32 client_input[0x55f6e1103d80] -> #33 io_loop_call_io[0x7f1749bbac70] -> #34 io_loop_handler_run_internal[0x7f1749bbc970] -> #35 io_loop_handler_run[0x7f1749bbcb00] -> #36 io_loop_run[0x7f1749bbccd0] -> #37 master_service_run[0x7f1749b2d180] -> #38 main[0x55f6e10efa00] -> #39 __libc_init_first[0x7f17498e2150] -> #40 __libc_start_main[0x7f17498e2200] -> #41 _start[0x55f6e10f0000]
2023-12-08T14:40:23.600070+02:00 key dovecot: imap(lorena@xxx.com)<1384133><mwAi4/4LMsRWfgeL>: Fatal: master: service(imap): child 1384133 returned error 83 (Out of memory (service imap { vsz_limit=256 MB }, you may need to increase it) - set CORE_OUTOFMEM=1 environment to get core dump)
But this is is a side issue, my main concern is nextcloud not running.
Forgot to mention, I used keyhelp-toolbox to re-write all the config files of apache/php etc multiple times, for panel configs and user configs, nothing changed.
Thank you!
Victor