[BlueOnyx:15024] Re: Vacation blocked by cced down

[Michael Stauber]

Hi Eric,

> You are right that the script will hang if cced is down but nagios has
> the logic to time out a plugin that hangs.  The default for
> service_check_timeout is 60 seconds which seems reasonable for this
> situation.

Yeah, that is reasonable.

> I wonder if you are re-inventing the wheel and perhaps the product would
> be better if nagios was included and configured for common checks. 
> There are rpms for it so the config files would be needed and a few
> custom plugins.  The monitoring would be easy but integrating the UI
> would require some work.

I don't doubt that Nagios can do it better and could also provide some
additional functionality. But our Active Monitor isn't that bad either
for what it does. It's also highly modular and can easily be adjusted.

So it's not really re-inventing the wheel. Active Monitor is already
well integrated with CCE, CODB and both the Sausalito and the new
Chorizo GUI. It just would benefit from a more robust CCEd monitoring
plugin, which is what I'll try to add during the next week or two.

Same goes for a few of the other monitoring components such as for FTP
and the mail services. At this time they just use a telnet connection to
localhost and the respective port of the monitored service and then
analyze the response they get. If the response is as expected, the
service is running. If not, it's failed and a restart attempt is made.

However, there are cases where services such as the email or FTP related
one return the expected initial response, but are still in one failed
state or other and bugger out one step further.

So my plan is to extend these monitoring components a bit. Instead of
using Telnet and interpreting the response with "expect", I'll use Perl
and the respective Perl modules for Mail and FTP to simulate a more
thorough and more complete transaction to determine the state of a service.

-- 
With best regards

Michael Stauber