Server eats memory like candy....my 6GB was eaten to poin of failures. [SOLVED]

[BasHeijermans]

Rspamd eating memory like candy.

I made some changes to Rspamd and now the server is running normal.
My VPS had 6GB memory and it was eaten to the last bit, making it impossible for the server to run normal.

Solution, goto /etc/rspamd and copy the from modules.d the file neural.conf into override.d

Edit the file so this section looks like this:

Code: Select all

neural {
  #servers = 127.0.0.1:6379; # Redis server to store learning data and ANN
  enabled = false;
  train {
    max_trains = 1k; # Number of trains per epoch
    max_usages = 20; # Number of learn iterations while ANN data is valid
    learning_rate = 0.01; # Rate of learning (Torch only)
    max_iterations = 25; # Maximum iterations of learning (Torch only)
  }

  timeout = 20; # Increase redis timeout

I also turned off Antivirus as it eats a whopping 1.3GB.

And last, logrotate is also a big problem, the files grow too big and then it starts zipping them on rotate.
I suggest you rotate them more often, so they do not grow that much.
That will help the CPU to stay fast when it's rotating.

Hope it helps others.

Bas.

[BasHeijermans]

Update, that didn't work well as it stopped Rspam from working.

So next try, /etc/rdis.conf

I set:

maxmemory 536870912 (=512MB)

maxmemory-policy volatile-lru

No I wait, the memory consumption seems to be fine.

Update, this seems to work and I also disbled the email-virus-scanner, now the server is working fine.

This may have solved the CPU 100% and memory overloading.

[BasHeijermans]

It seems to work....finally....the server is really fast.

And the numbers show it's not eating memory for candy anymore.

Alex, you may want to introduce a setting, how much it should consume.

[Alexander]

Hello,

thanks for sharing!

I am interested in other users who use Debian 12: Are you experience the same behavior?

At least on my test-servers, there are all fine regarding memory consumption. Thou they are only used for mail to an extend.

Anyway, the /etc/redis/redis.conf is not touched by KeyHelp, so feel free to adjust the configuration for your needs.

[BasHeijermans]

I also turned off database compression and checksum:

rdbcompression no
rdbchecksum no

To gain more speed, as it eats CPU-cycles too.

[BasHeijermans]

In the end it turned out to be the VPS at another provider.

So I moved the server to Alex of Keyhelp and it's 100x faster too.

Cinfu turned out to be the problem, their disk-IO is horrible and causeing all problems.

Thanks Alex....for helping me moving. Your VPS servers are 100x better then any other!!!

[Alexander]

Perfect, I am pleased to hear that you are satisfied .

[BasHeijermans]

On your server it started swapping, so I did the things above and turned off email-server Antivirus.

This is what happened

Memory to spare and no more swapping.
Most users run their own mail-AV software on Windows anyway, so scanning on the server is not really needed for me.

I advise to set these parameters I listed above and turn of AV-email on Low-memory systems as it eats a whopping 1.3G

[BasHeijermans]

Little bit longer and it works.....

The server is quiet, fast en not doing much, as it should as we don't use it for many things but a few friends and some email.

The biggest task is NTP-pool-server

[BasHeijermans]

It's running for more then a week now....still humming along and fast as can be.

So yes, the settings I suggest are good.

This server does runs at KeyWeb, best move I did ever.....they do know how to do VPS and do it well.

Works:
rdbcompression no
rdbchecksum no
maxmemory 536870912
maxmemory-policy volatile-lru

Antivirus for Email should be disabled by default on all systems in my opinion, let an admin decide if he wants the 'extra' protection as it does eat memory.
AV also eats CPU-cycles making a server very slow.

As for the maxmemory, maybe a good idea to make it selectable, as more memory means more 'tokes' stored and better spam-protection.
However, a large database eats CPU and memory, not very desired either.