Automatic Update Fails, Fills Disk Until Server Crash (v26.0)  [SOLVED]

Have you discovered a bug? Tell us about it.
Post Reply
lwhost
Posts: 4
Joined: Mon 9. Sep 2024, 04:09

Automatic Update Fails, Fills Disk Until Server Crash (v26.0)

Post by lwhost »

I'm sure the cause of the problem is within KeyHelp
Yes

Server operating system + version
Debian 12

Server virtualization technology used
KVM

KeyHelp version + build number
(e.g. 22.0 - Build 2366)
25.3?

Problem description / error messages
Disk rapidly filled up '/home/keyhelp/keyhelp.backup/panel_updates' with timestamped files, ie 'keyhelp_25.3_2026-04-09_04-11-05.sql
' and 'keyhelp_25.3_2026-04-09_04-11-05.tar.gz' until server crashed.

Server is currently displaying 'The server control panel is currently being updated.' via the web interface.
I've tried manually rebooting the server and clearing the old panel backup files.

Previously this server was very stable and without issue.
Could I please have some pointers.

When trying to manually update it:

Code: Select all

root@server:/home/keyhelp/keyhelp.backup/panel_updates# keyhelp run panel-update
[PID-2869] [2026-04-09 04:12:47] INFO | maintenance connection okay
[PID-2869] [2026-04-09 04:12:47] INFO | forced to run "panel_update.php"
[PID-2869] [2026-04-09 04:12:47] ERROR | panel update should be running (database flag), but corresponding PID is not running anymore, rescheduli                       ng panel update...
[PID-2869] [2026-04-09 04:12:47] INFO | jobs to run: panel_update.php
[PID-2869] [2026-04-09 04:12:47] INFO | >>> trying to run "panel-update"
[PID-2869] [2026-04-09 04:12:47] WARN | last run died abruptly
[PID-2869] [2026-04-09 04:12:47] INFO | lock "panel-update" acquired
[PID-2869] [2026-04-09 04:12:47] INFO | processing the job ...
[2026-04-09 04:12:47] INFO | maintenance connection okay
[2026-04-09 04:12:47] INFO | trying to connect to updateserver
[2026-04-09 04:12:47] INFO | enable full access to temporary directories
[2026-04-09 04:12:48] INFO | new version detected - starting update process from "25.3" to "26.0"
[2026-04-09 04:12:48] INFO | first update step: "26.0" | total steps to reach latest version:
[2026-04-09 04:12:48] INFO | create folder for panel update "/home/keyhelp/www/keyhelp_tmp/"
[2026-04-09 04:12:48] INFO | start download of "https://install.keyhelp.de/files/releases/26.0/26.0-php82.tar.gz"
/home/keyhelp/www/keyhelp_tmp/26.0.t 100%[===================================================================>]  18.75M  40.0MB/s    in 0.5s
[2026-04-09 04:12:48] INFO | file saved to "/home/keyhelp/www/keyhelp_tmp/26.0.tar.gz"
[2026-04-09 04:12:48] INFO | checksum check successful
[2026-04-09 04:12:48] INFO | lock panel
[2026-04-09 04:12:48] INFO | log out all users
[2026-04-09 04:12:48] INFO | run --force="update" one last time before new files and db structure may change things
[2026-04-09 04:12:51] INFO | checking running maintenance intervals ...
[2026-04-09 04:12:51] INFO | all maintenance intervals finished, so we can go on
[2026-04-09 04:12:51] INFO | archive extraction finished
[2026-04-09 04:12:51] INFO | changed owner of "/home/keyhelp/www/keyhelp_tmp/" to "keyhelp"
[2026-04-09 04:12:51] INFO | Setup backup directory "/home/keyhelp/keyhelp.backup/panel_updates"
[2026-04-09 04:12:51] INFO | Database backup saved to "/home/keyhelp/keyhelp.backup/panel_updates/keyhelp_25.3_2026-04-09_04-12-51.sql"
[2026-04-09 04:12:53] INFO | File backup saved to "/home/keyhelp/keyhelp.backup/panel_updates/keyhelp_25.3_2026-04-09_04-12-51.tar.gz"
[2026-04-09 04:12:54] INFO | remove old keyhelp folder
[2026-04-09 04:12:56] INFO | copy from "/home/keyhelp/www/keyhelp_tmp/" to "/home/keyhelp/www/keyhelp/"
[2026-04-09 04:12:56] INFO | remove "/home/keyhelp/www/keyhelp_tmp/"
[2026-04-09 04:12:57] INFO | run update file "/home/keyhelp/www/keyhelp/install/updates/update_25.3.php"
[2026-04-09 04:12:57] INFO | [KeyHelp Database] Add keys to columns

Notice: DB-Connection timed out, now reconnecting... in /home/keyhelp/www/keyhelp/core/Database/DBConnection.php on line 184

Warning: MySQL - Query Error: Too many keys specified; max 64 keys allowed (1069) in /home/keyhelp/www/keyhelp/core/Database/DBConnection.php on                        line 330

Notice: DB-Connection timed out, now reconnecting... in /home/keyhelp/www/keyhelp/core/Database/DBConnection.php on line 184

Warning: MySQL - Query Error: Too many keys specified; max 64 keys allowed (1069) in /home/keyhelp/www/keyhelp/core/Database/DBConnection.php on                        line 330
[2026-04-09 04:13:03] INFO | [KeyHelp Database] Update scheduled_tasks -> php_interpreter column
[2026-04-09 04:13:03] INFO | [KeyHelp Database] Add columns for PHP environment variables

Notice: DB-Connection timed out, now reconnecting... in /home/keyhelp/www/keyhelp/core/Database/DBConnection.php on line 184

Warning: MySQL - Query Error: Duplicate column name 'env_variables' (1060) in /home/keyhelp/www/keyhelp/core/Database/DBConnection.php on line 33                       0

Notice: DB-Connection timed out, now reconnecting... in /home/keyhelp/www/keyhelp/core/Database/DBConnection.php on line 184

Warning: MySQL - Query Error: Duplicate column name 'php_env_variables' (1060) in /home/keyhelp/www/keyhelp/core/Database/DBConnection.php on lin                       e 330
[2026-04-09 04:13:09] INFO | [KeyHelp Database] Add Fail2Ban whitelist setting
[2026-04-09 04:13:09] INFO | [KeyHelp Database] Update backup_repository -> max_snapshot column

Notice: DB-Connection timed out, now reconnecting... in /home/keyhelp/www/keyhelp/core/Database/DBConnection.php on line 184

Warning: MySQL - Query Error: Unknown column 'max_snapshots' in 'backup_repositories' (1054) in /home/keyhelp/www/keyhelp/core/Database/DBConnect                       ion.php on line 330

Notice: DB-Connection timed out, now reconnecting... in /home/keyhelp/www/keyhelp/core/Database/DBConnection.php on line 184

Warning: MySQL - Query Error: Unknown column 'max_snapshots' in 'WHERE' (1054) in /home/keyhelp/www/keyhelp/core/Database/DBConnection.php on lin                       e 330

Notice: DB-Connection timed out, now reconnecting... in /home/keyhelp/www/keyhelp/core/Database/DBConnection.php on line 184

Warning: MySQL - Query Error: Unknown column 'max_snapshots' in 'backup_repositories' (1054) in /home/keyhelp/www/keyhelp/core/Database/DBConnect                       ion.php on line 330
[2026-04-09 04:13:18] INFO | [Backup] Add new rotation policies

Notice: DB-Connection timed out, now reconnecting... in /home/keyhelp/www/keyhelp/core/Database/DBConnection.php on line 184

Warning: MySQL - Query Error: Duplicate column name 'rotation_mode' (1060) in /home/keyhelp/www/keyhelp/core/Database/DBConnection.php on line 33                       0

Notice: DB-Connection timed out, now reconnecting... in /home/keyhelp/www/keyhelp/core/Database/DBConnection.php on line 184

Warning: MySQL - Query Error: Unknown column 'max_snapshots' in 'WHERE' (1054) in /home/keyhelp/www/keyhelp/core/Database/DBConnection.php on lin                       e 330

Notice: DB-Connection timed out, now reconnecting... in /home/keyhelp/www/keyhelp/core/Database/DBConnection.php on line 184

Warning: MySQL - Query Error: Can't DROP COLUMN `max_snapshots`; check that it exists (1091) in /home/keyhelp/www/keyhelp/core/Database/DBConnect                       ion.php on line 330
[2026-04-09 04:13:27] INFO | [Maintenance Intervals] Update job name
[2026-04-09 04:13:27] INFO | [KeyHelp] Delete outdated client statistics
[2026-04-09 04:13:27] INFO | [KeyHelp] Delete outdated panel backups
[2026-04-09 04:13:27] INFO | [Postfix] Update configuration
sh: 1: postconf: not found

Fatal error: Uncaught Error: Class "Cli" not found in /home/keyhelp/www/keyhelp/install/updates/update_25.3.php:173
Stack trace:
#0 /home/keyhelp/www/keyhelp/cronjob/jobs/panel_update.php(460): unknown()
#1 /home/keyhelp/www/keyhelp/cronjob/mastercronjob.php(454): unknown()
#2 {main}
  thrown in /home/keyhelp/www/keyhelp/install/updates/update_25.3.php on line 173
lwhost
Posts: 4
Joined: Mon 9. Sep 2024, 04:09

Re: Automatic Update Fails, Fills Disk Until Server Crash (v26.0)

Post by lwhost »

If others are running into this issue.

You can temporarily disable the panel_update.php script stored here: /home/keyhelp/www/keyhelp/cronjob/jobs/

With a command such as:

Code: Select all

mv /home/keyhelp/www/keyhelp/cronjob/jobs/panel_update.php /home/keyhelp/www/keyhelp/cronjob/jobs/panel_update.php.disabled
This is not a fix, but may prevent the disk rapidly filling up and server crashing.
User avatar
Alexander
Keyweb AG
Posts: 4818
Joined: Wed 20. Jan 2016, 02:23

Re: Automatic Update Fails, Fills Disk Until Server Crash (v26.0)

Post by Alexander »

Hello,

you have issues in your database. May I have a look what is off on your server?

In this case,i would need SSH access for the root account on your server (via private message).
Mit freundlichen Grüßen / Best regards
Alexander Mahr

**************************************************************
Keyweb AG - Die Hosting Marke
Neuwerkstr. 45/46, 99084 Erfurt / Germany
http://www.keyweb.de - http://www.keyhelp.de
**************************************************************
User avatar
Alexander
Keyweb AG
Posts: 4818
Joined: Wed 20. Jan 2016, 02:23

Re: Automatic Update Fails, Fills Disk Until Server Crash (v26.0)  [SOLVED]

Post by Alexander »

The issue was fixed on the server. lwhost sever should now run again as expected.

If was caused by a combination of a bug and some special configuration on the server itself (uninstalled Postfix in particular).
Mit freundlichen Grüßen / Best regards
Alexander Mahr

**************************************************************
Keyweb AG - Die Hosting Marke
Neuwerkstr. 45/46, 99084 Erfurt / Germany
http://www.keyweb.de - http://www.keyhelp.de
**************************************************************
lwhost
Posts: 4
Joined: Mon 9. Sep 2024, 04:09

Re: Automatic Update Fails, Fills Disk Until Server Crash (v26.0)

Post by lwhost »

Thanks Alexander,
All has been resolved.
Post Reply