Antivirus enabled and CPU-load goes nuts. [SOLVED]

[BasHeijermans]

Not sure if this is a bug or a feature

I noticed that if I enable Antivirus scanning my tiny-VPS (1 CPU E-2136 @ 3.30Ghz + 2GB ram + 50GB HD) goes nuts.

The normal load without virus scanning is about 27% but with scanner it goes >250%

Also email will become terribly slow, upto 30 seconds to send a simple email.

When I turn the scanner off, the system is quick and resposive.

Swapping is not used and of the 2GB memory there is always 1.4GB free, so no memory problems.

Could this be an issue of the scanner? Or something else?

No matter what I do, as soon as I turn it on, it goes nuts on CPU-power.

Thanks,

PS. I do not really need the Antivirus scanner, never used it before, but wondering why it eats CPU-power for breakfast all the time.

Bas.

[tab-kh]

I'm not sure if it isnt really the 2GB RAM. If you install Keyhelp on a server with less than 4GB, the E-Mail Antivirus scanmner is disabled by default. And I can imagine, that this is so for a reason.

Edit: My still quite small server with 2 Intel Xeon Gold vCPU and 8 GB RAM scans without complaining. A server with 4 GB RAM has been reported to fail with scanning enabled.

[BasHeijermans]

Thanks for putting me on the right track.

After a bit of searching arround, it turns out it loads the virus-database in memory to make scanning faster.
For me this is not needed.

After I added: ConcurrentDatabaseReload no in /etc/clamav/clamd.conf

It didn't overload the server anymore, well so far, have to watch it a few hours.

But it looks promissing.

Edit: That didn't work either, it ate all memory to a stopping point when scanning directories.
Have to look further.

Edit2, changed the default scanning databases to just limited for virus and malware-scanning

Code: Select all

https://ftp.swin.edu.au/sanesecurity/phish.ndb
https://ftp.swin.edu.au/sanesecurity/rogue.hdb
https://ftp.swin.edu.au/sanesecurity/sanesecurity.ftm
https://ftp.swin.edu.au/sanesecurity/sigwhitelist.ign2
https://ftp.swin.edu.au/sanesecurity/spamattach.hdb
https://ftp.swin.edu.au/sanesecurity/foxhole_generic.cdb
https://ftp.swin.edu.au/sanesecurity/foxhole_filename.cdb
https://ftp.swin.edu.au/sanesecurity/malwarehash.hsb
https://ftp.swin.edu.au/sanesecurity/hackingteam.hsb
https://ftp.swin.edu.au/sanesecurity/winnow_malware.hdb
https://ftp.swin.edu.au/sanesecurity/winnow_malware_links.ndb
https://ftp.swin.edu.au/sanesecurity/winnow_extended_malware.hdb
https://ftp.swin.edu.au/sanesecurity/winnow.attachments.hdb
https://ftp.swin.edu.au/sanesecurity/winnow_bad_cw.hdb
https://ftp.swin.edu.au/sanesecurity/bofhland_cracked_URL.ndb
https://ftp.swin.edu.au/sanesecurity/bofhland_malware_URL.ndb
https://ftp.swin.edu.au/sanesecurity/bofhland_phishing_URL.ndb
https://ftp.swin.edu.au/sanesecurity/bofhland_malware_attach.hdb

I removed all others and used the first settings as well.

Then started scanning again, it didn't not use 1.3GB memory as before but ~500MB, it didn't slowdown the server so far and the panel as well as email where responsive.
Looks to me one of the default-config-databases is simply too huge to be run in a 2GB system.

Going to let it run again for a bit, and see what happens.

[Alexander]

Hello,

when you've done your testing, let me know the results so I can also investigate it a bit and maybe make some improvements for an upcoming release.

Thank you!

[BasHeijermans]

Hi Alex,

It is running for about 24 hours with the above settings.

After I added: ConcurrentDatabaseReload no in /etc/clamav/clamd.conf

and these databases:

Code: Select all

https://ftp.swin.edu.au/sanesecurity/phish.ndb
https://ftp.swin.edu.au/sanesecurity/rogue.hdb
https://ftp.swin.edu.au/sanesecurity/sanesecurity.ftm
https://ftp.swin.edu.au/sanesecurity/sigwhitelist.ign2
https://ftp.swin.edu.au/sanesecurity/spamattach.hdb
https://ftp.swin.edu.au/sanesecurity/foxhole_generic.cdb
https://ftp.swin.edu.au/sanesecurity/foxhole_filename.cdb
https://ftp.swin.edu.au/sanesecurity/malwarehash.hsb
https://ftp.swin.edu.au/sanesecurity/hackingteam.hsb
https://ftp.swin.edu.au/sanesecurity/winnow_malware.hdb
https://ftp.swin.edu.au/sanesecurity/winnow_malware_links.ndb
https://ftp.swin.edu.au/sanesecurity/winnow_extended_malware.hdb
https://ftp.swin.edu.au/sanesecurity/winnow.attachments.hdb
https://ftp.swin.edu.au/sanesecurity/winnow_bad_cw.hdb
https://ftp.swin.edu.au/sanesecurity/bofhland_cracked_URL.ndb
https://ftp.swin.edu.au/sanesecurity/bofhland_malware_URL.ndb
https://ftp.swin.edu.au/sanesecurity/bofhland_phishing_URL.ndb
https://ftp.swin.edu.au/sanesecurity/bofhland_malware_attach.hdb

Load doesn't go nuts. It's handling fine under 2GB ram and no swapping.

Still 500MB of ram free. Tiny server doesn't do more then 2 accounts, not even a webpage.
Just handling mail and DNS.

I took all databases that check for Virus and give LOW chance of false positives, not interested in Spam databases as SpamAssasin does that job very well.

[BasHeijermans]

BTW Alex, I tested on your testing-server, it had 3GB and it went also nuts with your settings.

After I changed it to mine, it went down from overload and swapping to real quite running.

It seems the number of databases you selected is too much.

The Virus-scan is simply too much, as there is also SpamAssasing that is for most scans more then enough.

1.3GB of memory, upping to almost 2.6GB during updates is not ok. The CPU is overloaded on a small system, maybe also on a big system.
I understand that you want to protect users maximal, but this is a bit overboard

Server should not protect users into extreme, users should protect themselves, we just help a bit.

Bas.

[Alexander]

Here are the changes regarding this issue for the next KeyHelp update:

For new installations:
-> If the system has less than 4 GB of RAM, the more resource-efficient list of signature URLs will be used

When updating:
-> On systems with less than 4 GB of RAM and without custom signatures enabled, the system will switch to the more resource-efficient signatures.

Within the UI:
-> You can switch between the complete signature set (the prior default), the resource-efficient set, and the custom signature settings.

[BasHeijermans]

Nice....that will probably work well for many people.