[BlueOnyx:06639] Re: Server processes slowly grow, admin GUI down, cced hangs on shutdown

[Jeffrey Pellin]

Hi Jeff,

This sounds like a problem we had. It was never actually fixed but turning
off Tomcat had a very positive impact.

It may be nothing, but it's somewhere to start.

Good luck

Jeffrey


On Tue, 08 Mar 2011 14:43:07 +0000, Jeff Jones <jeffrhysjones at mac.com>
wrote:
> Hi all,
> 
> A while back I reported an issue of a BX server that mysteriously hung in
> the night - all websites down, basically ran out of RAM:
> 
> http://www.blueonyx.it/pipermail/blueonyx/2010-September/005484.html
> 
> The only thing I could see beforehand that was fishy, was that the number
> of processes running on the box suddenly started to increase, by 1 or 2 -
> every 15 mins - until the server RAM went.
> 
> So what I did was stick a monitor on the number of processes, and if it
> went over 'X' in the future, to send me an alert.
> 
> Well - I just got my first high process alert!
> 
> The first thing I noticed was that the admserv was not working - I was
> unable to get into GUI admin. I restarted admsrv, but no dice.
> All the websites on the server were fine - this is the same as last time,
> the sites only fail when the number of processes gets so high that the
> server runs out of ram.
> I tried to restart apache, but that hung.
> On a different SSH session, I then attempted a reboot - but the reboot
hung
> when trying to kill cced. 
> 
> So there was something majorly wrong with the server.
> 
> After the hard reset - it rebooted - and fine again.
> 
> It's disappointing that although I can now detect this issue very early
on
> - I still need to perform a hard reset again to 'fix' it. I guess the
> bonus now is that I get early warning, so I don't get woken up at 3am!!!
> 
> So I was wondering - are there any other ideas as to why this might be
> happening? 
> 
> Any suggestions for logs etc to look at?
> 
> Cheers,
> 
> Jeff
> 
> Jeff Jones
> jeffrhysjones at mac.com
> 
> 
> 
> _______________________________________________
> Blueonyx mailing list
> Blueonyx at blueonyx.it
> http://www.blueonyx.it/mailman/listinfo/blueonyx