[BlueOnyx:14969] Vacation blocked by cced down

[Eric Peabody]

This morning messages for a few users on a 5107R were backing up in the 
sendmai queue.  The mail log contained "timeout waiting for input from 
local during Draining Input" messages and "ps" showed sendmail processes 
waiting for vacation to run.  The queued messages were for users who had 
a .forward configured for processing vacation messages.

Eventually I discovered that cced was not running correctly.  The code 
for vacation processing depends on cced and if that process is not 
running, /etc/smrsh/vacation.pl will hang, causing the problem. 
(vacation.pl checks the date range configured in the UI to decide to 
send a vacation message and needs cced running to do so.)

Note that the affected users' vacation messages had expired.  The code 
in vacation.pl checks with cced for that condition, and cced was hung, 
so sendmail waited.  While working on this, I was unable to bring up the 
BlueOnyx UI, a big fat clue.  There were two cced processes listed in 
ps, one a zombie.  After kill -9 these, and restarting cced, the mail 
queue was processed.

So first, it would be good for users to manually disable their vacation 
message once it expires.  This will remove the line from .forward that 
calls vacation.pl and avoid the problem.  Even better would be for the 
system to do that for them.  Once vacation.pl determines that a user's 
message has expired, remove the .forward line.

Also, cced being down is a big problem so a watchdog process seems like 
a good addition.  That is, a separate process that checks that cced is 
running correctly, being sure to detect hangs, and restarts cced and 
alerts the admin if it gets a bad response.

-- 
Eric Peabody
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.blueonyx.it/pipermail/blueonyx/attachments/20140321/45a4e72c/attachment.html>