Antivirus enabled and CPU-load goes nuts.  [SOLVED]

Have you discovered a bug? Tell us about it.
Post Reply
User avatar
BasHeijermans
KeyHelp Translator
Posts: 158
Joined: Mon 20. Jun 2022, 12:01
Location: Heppen Belgium
Contact:

Antivirus enabled and CPU-load goes nuts.

Post by BasHeijermans »

Not sure if this is a bug or a feature :lol:

I noticed that if I enable Antivirus scanning my tiny-VPS (1 CPU E-2136 @ 3.30Ghz + 2GB ram + 50GB HD) goes nuts.

The normal load without virus scanning is about 27% but with scanner it goes >250%

Also email will become terribly slow, upto 30 seconds to send a simple email.

When I turn the scanner off, the system is quick and resposive.

Swapping is not used and of the 2GB memory there is always 1.4GB free, so no memory problems.

Could this be an issue of the scanner? Or something else?

No matter what I do, as soon as I turn it on, it goes nuts on CPU-power.

Thanks,

PS. I do not really need the Antivirus scanner, never used it before, but wondering why it eats CPU-power for breakfast all the time.

Bas.
Greetings Bas.

Ik heb KeyHelp naar het Nederlands vertaald, contacteer me als er translatie fouten zijn.
(I have translated KeyHelp into Dutch, contact me if there are translation errors.)
Next version 24 of KeyHelp 100% translated.
tab-kh
Posts: 450
Joined: Thu 22. Apr 2021, 23:06

Re: Antivirus enabled and CPU-load goes nuts.

Post by tab-kh »

I'm not sure if it isnt really the 2GB RAM. If you install Keyhelp on a server with less than 4GB, the E-Mail Antivirus scanmner is disabled by default. And I can imagine, that this is so for a reason.

Edit: My still quite small server with 2 Intel Xeon Gold vCPU and 8 GB RAM scans without complaining. A server with 4 GB RAM has been reported to fail with scanning enabled.
User avatar
BasHeijermans
KeyHelp Translator
Posts: 158
Joined: Mon 20. Jun 2022, 12:01
Location: Heppen Belgium
Contact:

Re: Antivirus enabled and CPU-load goes nuts.

Post by BasHeijermans »

Thanks for putting me on the right track.

After a bit of searching arround, it turns out it loads the virus-database in memory to make scanning faster.
For me this is not needed.

After I added: ConcurrentDatabaseReload no in /etc/clamav/clamd.conf

It didn't overload the server anymore, well so far, have to watch it a few hours.

But it looks promissing.

Edit: That didn't work either, it ate all memory to a stopping point when scanning directories.
Have to look further.

Edit2, changed the default scanning databases to just limited for virus and malware-scanning

Code: Select all

https://ftp.swin.edu.au/sanesecurity/phish.ndb
https://ftp.swin.edu.au/sanesecurity/rogue.hdb
https://ftp.swin.edu.au/sanesecurity/sanesecurity.ftm
https://ftp.swin.edu.au/sanesecurity/sigwhitelist.ign2
https://ftp.swin.edu.au/sanesecurity/spamattach.hdb
https://ftp.swin.edu.au/sanesecurity/foxhole_generic.cdb
https://ftp.swin.edu.au/sanesecurity/foxhole_filename.cdb
https://ftp.swin.edu.au/sanesecurity/malwarehash.hsb
https://ftp.swin.edu.au/sanesecurity/hackingteam.hsb
https://ftp.swin.edu.au/sanesecurity/winnow_malware.hdb
https://ftp.swin.edu.au/sanesecurity/winnow_malware_links.ndb
https://ftp.swin.edu.au/sanesecurity/winnow_extended_malware.hdb
https://ftp.swin.edu.au/sanesecurity/winnow.attachments.hdb
https://ftp.swin.edu.au/sanesecurity/winnow_bad_cw.hdb
https://ftp.swin.edu.au/sanesecurity/bofhland_cracked_URL.ndb
https://ftp.swin.edu.au/sanesecurity/bofhland_malware_URL.ndb
https://ftp.swin.edu.au/sanesecurity/bofhland_phishing_URL.ndb
https://ftp.swin.edu.au/sanesecurity/bofhland_malware_attach.hdb
I removed all others and used the first settings as well.

Then started scanning again, it didn't not use 1.3GB memory as before but ~500MB, it didn't slowdown the server so far and the panel as well as email where responsive.
Looks to me one of the default-config-databases is simply too huge to be run in a 2GB system.

Going to let it run again for a bit, and see what happens.
Greetings Bas.

Ik heb KeyHelp naar het Nederlands vertaald, contacteer me als er translatie fouten zijn.
(I have translated KeyHelp into Dutch, contact me if there are translation errors.)
Next version 24 of KeyHelp 100% translated.
User avatar
Alexander
Keyweb AG
Posts: 3810
Joined: Wed 20. Jan 2016, 02:23

Re: Antivirus enabled and CPU-load goes nuts.

Post by Alexander »

Hello,

when you've done your testing, let me know the results so I can also investigate it a bit and maybe make some improvements for an upcoming release.

Thank you!
Mit freundlichen Grüßen / Best regards
Alexander Mahr

**************************************************************
Keyweb AG - Die Hosting Marke
Neuwerkstr. 45/46, 99084 Erfurt / Germany
http://www.keyweb.de - http://www.keyhelp.de
**************************************************************
User avatar
BasHeijermans
KeyHelp Translator
Posts: 158
Joined: Mon 20. Jun 2022, 12:01
Location: Heppen Belgium
Contact:

Re: Antivirus enabled and CPU-load goes nuts.

Post by BasHeijermans »

Hi Alex,

It is running for about 24 hours with the above settings.
BasHeijermans wrote: Thu 7. Jul 2022, 13:23 After I added: ConcurrentDatabaseReload no in /etc/clamav/clamd.conf
and these databases:

Code: Select all

https://ftp.swin.edu.au/sanesecurity/phish.ndb
https://ftp.swin.edu.au/sanesecurity/rogue.hdb
https://ftp.swin.edu.au/sanesecurity/sanesecurity.ftm
https://ftp.swin.edu.au/sanesecurity/sigwhitelist.ign2
https://ftp.swin.edu.au/sanesecurity/spamattach.hdb
https://ftp.swin.edu.au/sanesecurity/foxhole_generic.cdb
https://ftp.swin.edu.au/sanesecurity/foxhole_filename.cdb
https://ftp.swin.edu.au/sanesecurity/malwarehash.hsb
https://ftp.swin.edu.au/sanesecurity/hackingteam.hsb
https://ftp.swin.edu.au/sanesecurity/winnow_malware.hdb
https://ftp.swin.edu.au/sanesecurity/winnow_malware_links.ndb
https://ftp.swin.edu.au/sanesecurity/winnow_extended_malware.hdb
https://ftp.swin.edu.au/sanesecurity/winnow.attachments.hdb
https://ftp.swin.edu.au/sanesecurity/winnow_bad_cw.hdb
https://ftp.swin.edu.au/sanesecurity/bofhland_cracked_URL.ndb
https://ftp.swin.edu.au/sanesecurity/bofhland_malware_URL.ndb
https://ftp.swin.edu.au/sanesecurity/bofhland_phishing_URL.ndb
https://ftp.swin.edu.au/sanesecurity/bofhland_malware_attach.hdb
Load doesn't go nuts. It's handling fine under 2GB ram and no swapping.
Schermafdruk op 2022-07-08 13-18-26.png
Still 500MB of ram free. Tiny server doesn't do more then 2 accounts, not even a webpage.
Just handling mail and DNS.

I took all databases that check for Virus and give LOW chance of false positives, not interested in Spam databases as SpamAssasin does that job very well.
Greetings Bas.

Ik heb KeyHelp naar het Nederlands vertaald, contacteer me als er translatie fouten zijn.
(I have translated KeyHelp into Dutch, contact me if there are translation errors.)
Next version 24 of KeyHelp 100% translated.
User avatar
BasHeijermans
KeyHelp Translator
Posts: 158
Joined: Mon 20. Jun 2022, 12:01
Location: Heppen Belgium
Contact:

Re: Antivirus enabled and CPU-load goes nuts.

Post by BasHeijermans »

BTW Alex, I tested on your testing-server, it had 3GB and it went also nuts with your settings.

After I changed it to mine, it went down from overload and swapping to real quite running.

It seems the number of databases you selected is too much.

The Virus-scan is simply too much, as there is also SpamAssasing that is for most scans more then enough.

1.3GB of memory, upping to almost 2.6GB during updates is not ok. The CPU is overloaded on a small system, maybe also on a big system.
I understand that you want to protect users maximal, but this is a bit overboard :o

Server should not protect users into extreme, users should protect themselves, we just help a bit.

Bas.
Greetings Bas.

Ik heb KeyHelp naar het Nederlands vertaald, contacteer me als er translatie fouten zijn.
(I have translated KeyHelp into Dutch, contact me if there are translation errors.)
Next version 24 of KeyHelp 100% translated.
User avatar
Alexander
Keyweb AG
Posts: 3810
Joined: Wed 20. Jan 2016, 02:23

Re: Antivirus enabled and CPU-load goes nuts.  [SOLVED]

Post by Alexander »

Here are the changes regarding this issue for the next KeyHelp update:

For new installations:
-> If the system has less than 4 GB of RAM, the more resource-efficient list of signature URLs will be used

When updating:
-> On systems with less than 4 GB of RAM and without custom signatures enabled, the system will switch to the more resource-efficient signatures.

Within the UI:
-> You can switch between the complete signature set (the prior default), the resource-efficient set, and the custom signature settings.
Mit freundlichen Grüßen / Best regards
Alexander Mahr

**************************************************************
Keyweb AG - Die Hosting Marke
Neuwerkstr. 45/46, 99084 Erfurt / Germany
http://www.keyweb.de - http://www.keyhelp.de
**************************************************************
User avatar
BasHeijermans
KeyHelp Translator
Posts: 158
Joined: Mon 20. Jun 2022, 12:01
Location: Heppen Belgium
Contact:

Re: Antivirus enabled and CPU-load goes nuts.

Post by BasHeijermans »

Nice....that will probably work well for many people.
Greetings Bas.

Ik heb KeyHelp naar het Nederlands vertaald, contacteer me als er translatie fouten zijn.
(I have translated KeyHelp into Dutch, contact me if there are translation errors.)
Next version 24 of KeyHelp 100% translated.
Post Reply