[BlueOnyx:20062] Re: 5209R Sendmail hanging - please read

[Dirk Estenfeld]

Michael,

I do not want to update all AV-SPAM packages on all servers. And I think this is not the right approach.
Sendmail and saslauthd was working fine also with the older AV-Spam versions until the beginning of this week. 
So the question is what did change and what have to change back to get both services stable again.

Today only two servers had issues with sendmail / saslauthd. Did you change something?

Best regard,
Dirk


-----------------------------------------------
blackpoint GmbH - Friedberger Straße 106 - 61118 Bad Vilbel

-----Ursprüngliche Nachricht-----
Von: Blueonyx [mailto:blueonyx-bounces at mail.blueonyx.it] Im Auftrag von Michael Stauber
Gesendet: Mittwoch, 31. August 2016 23:02
An: BlueOnyx General Mailing List <blueonyx at mail.blueonyx.it>
Betreff: [BlueOnyx:20044] Re: 5209R Sendmail hanging - please read

Hi all,

> Every so often, and usually in the early morning, we will find our 5209R
> boxes with hung Sendmail.    Typically if it happens on one, it's
> happening on the others as well.  I wonder if anyone else has had this
> happen?

First of all: Many thanks to everyone for your input. A bit further
below Michael Aronoff suggested a change to the Saslauthd configuration,
which sounds really sensible. I will publish an update which will make
this the new default configuration for Saslauthd.

That should take care of the Saslauthd issue.

Now let's get to Sendmail with the AV-SPAM:

On 16th August I published the AV-SPAM 6.2.1-1, which contains fixes
that are supposed to deal with most of the issues.

Please check which version of the AV-SPAM you all have installed. If
your's is hanging and is not version 6.2.1-1, then please consider
upgrading. Once you have upgraded, disable an AV-SPAM Milter and turn it
back again. That will create a new and impoved sendmail.mc and sendmail.cf.

And I'd love to hear if anyone still has issues although 6.2.1-1 is
already installed.

Here are the essential changes between the 6.2.1-1 and previous versions:

Old AV-SPAM (all services enabled):

[root at 5209r ~]# cat /etc/mail/sendmail.mc|grep INPUT_MAIL_FILTER
INPUT_MAIL_FILTER(`greylist',`S=local:/var/milter-greylist/milter-greylist.sock')
INPUT_MAIL_FILTER(`milter-geoip', `S=local:/var/run/milter-geoip.sock,
F=T, T=S:6m;R:6m;E:15m')dnl
INPUT_MAIL_FILTER(`clamav', `S=local:/var/run/clamav/clamav-milter.sock,
F=T, T=S:4m;R:4m')
INPUT_MAIL_FILTER(`spamassassin', `S=local:/var/run/spamass.sock, F=T,
T=S:6m;R:6m;E:15m')dnl

New AV-SPAM (all services enabled):

[root at 5209r ~]# cat /etc/mail/sendmail.mc|grep INPUT_MAIL_FILTER
INPUT_MAIL_FILTER(`greylist',`S=local:/var/milter-greylist/milter-greylist.sock')
INPUT_MAIL_FILTER(`milter-geoip', `S=local:/var/run/milter-geoip.sock,
T=S:6m;R:6m;E:15m')dnl
INPUT_MAIL_FILTER(`clamav', `S=local:/var/run/clamav/clamav-milter.sock,
T=S:4m;R:4m')
INPUT_MAIL_FILTER(`spamassassin', `S=local:/var/run/spamass.sock,
T=S:6m;R:6m;E:15m')dnl

The difference is that the flag "F=T" has been removed from all Milters.
This flag defines which action Sendmail should take if a Milter stops
working. "F=T" will cause a "tempfail", so Sendmail will not process the
emails and will let the sender know to come back later.

The removal of "F=T" from the Milter configuration will instead cause
that the milter is then ignored (until it becomes responsive again) and
email is still delivered right away. Just without the usual Milter
processing.

As for why the Milters sometime stop working? There is no single cause
and effect that affects all. It all depends. SpamAssasin sometimes
becomes bogged down when more emails are parallel processed than Spamd
has spawned children.

The ClamAV Milter might become unresponsive for a bit when it's rehashed
after virus signature updates.

Milter-GeoIP depends heavily on MySQL for the extended logging, so it's
performance depends on how agile your MySQL server is currently
responding. If there is a large SQL-Dump running (during a backup for
instance), then the Milter might get affected.

Dropping out the "F=T" flag has greatly improved reliability on those
boxes under my direct management.

Active Monitor:

The AV-SPAM adds a "plugin" to that to monitor all the AV-SPAM
components. It does a great job identifying if a Milter is down and
restarts them, the related components and Sendmail in the correct
sequence. Between 6.2.0-1 up to 6.2.0-3 and the latest 6.2.1-1 this also
got some extra fine tuning.

There is also a new shell script that can be used to monitor the AV-SPAM
status and can be used to restart all of it:

[root at 5209r ~]# /usr/sausalito/sbin/avspam_init.pl -status
==================================================
| Service:       | GUI:  | AutoStart: | Running: |
==================================================
| Sendmail       |   1   |      1     |     1    |
| Greylist       |   1   |      1     |     1    |
| GeoIP          |   1   |      1     |     1    |
| Spamassassin   |   1   |      1     |     1    |
| Spamass-Milter |   1   |      1     |     1    |
| ClamAV         |   1   |      1     |     1    |
| ClamAV-Milter  |   1   |      1     |     1    |
==================================================
AV-SPAM Email service are running.

To restart everything that's enabled:

/usr/sausalito/sbin/avspam_init.pl -restart

But there is still one key element that I want to address during one of
the the next updates of base-email on BlueOnyx itself: As is the current
Active Monitor component that checks if email is working just does a
"telnet localhost 25" to see if it gets a "220" reply, which would
indicate that Sendmail is accepting connections.

That perhaps is not good enough. I will modify this component further so
that it actually *tries* to send an email during each Active Monitor
run. Not really sending one (if I can do without that), but just trying.
Because that would reliably determine if Sendmail is willing to process
an email, or if underlying components of it (such as the Milters) would
bail on that.

This check could also monitor Saslauthd if the attempt to send the email
is done via a local user and uses SMTP-Auth. Technically that's a bit
complicated, but it will be well worth to implement. I expect to have
that component ready sometime in the next 8-10 days.

Like said: This will be a base component of BlueOnyx and not part of the
AV-SPAM.

-- 
With best regards

Michael Stauber
_______________________________________________
Blueonyx mailing list
Blueonyx at mail.blueonyx.it
http://mail.blueonyx.it/mailman/listinfo/blueonyx