[BlueOnyx:02317] Re: Protection Against Root Toolkit Attack

[Hans Draaisma]

Micheal, that was a great and informative piece you wrote there!

Regards,

Hans Draaisma
Netvictory B.V. 


-----Oorspronkelijk bericht-----
Van: blueonyx-bounces at blueonyx.it [mailto:blueonyx-bounces at blueonyx.it]
Namens Michael Stauber
Verzonden: woensdag 9 september 2009 1:52
Aan: BlueOnyx General Mailing List
Onderwerp: [BlueOnyx:02316] Re: Protection Against Root Toolkit Attack

Hi Phil,

> Does anyone think that a Root Toolkit Detection system would be beneficial
> on a BO server?

I think there is no good answer to that question that suits everybody.

But in my opinion the typical implementations of Chkrootkit, Portsentry and 
similar tools very often a waste of time and CPU cycles. It's like fighting 
todays threats with the tools of 2001. Or in other words: It's inadequate.

At least in an automated fashion. Nothing speaks against running rootkit 
detectors manually from time to time. But especially people with multiple 
servers will get tired of the daily emails and frequent false alarms or the 
information overload that these tend to generate. Up to the point where you 
simply overlook the one email that should have gotten your attention,
because 
it reported a true and sucessful attack.

> I am looking at Rkdet, chkrootkit, Tripwire, psionic and similar software
> one of which should run ok with a little configuration.

I don't know Rkdet, but over the years I had a lot of (mixed) experience
with 
Tripwire. I'm running it on my border router, so it sees all traffic that
hits 
the pipe facing the outside world. Finding the right mix of Tripwire rules
and 
tresholds that suits your specific environment is almost an art. I'm down to

like 10-15 false alarms a day now - after years of fine tuning. There is a
set 
of rules that I could drop and would get rid of most false alarms, but that 
would also prevent the detection of some stuff that I'd like to keep on top 
of. 

I certainly would NOT build Tripwire into the standard BlueOnyx
distribution, 
because it would be a support nightmare. Everyone would come with exceptions

to the rules that need to be built in to cater his specific needs. Which in 
turn would make the GUI for it overly complex. I also don't want to fall
into 
the same pits that Cobalt did when they built their security stuff into the 
RaQ550 GUI. Some here may remember it. It was pretty much useless. Partially

due to horrible design and implementation flaws (it scanned outbound traffic

only due to a mixup in the code!), partially because due to the "appliance 
idea" they made the GUI for it too simplicistic to really allow you to
adjust 
it to your specific needs. Imperfect tools can create a false sense of 
security and then do more harm then good.

> I have been told (although I can not verify just how true that it is) that
> a good packet sniffer could possibly build a set of usernames and
passwords
> for a linux system.

Well, "sniffing" has always been an issue and always will be. But in order
to 
"sniff" someone needs to have "an ear on the track". Like he must have the 
ability to tie into your network traffic. Like having root access to one of 
the boxes in your network segment. If that's the case, then the child has 
already fallen into the well. Or the attacker must sit somewhere between you

and the person(s) that frequently login to the server through unencrypted 
protocols (POP3, IMAP, SMTP, FTP, etc.).

In that case he can more or less easily sniff packets and can gather the 
usernames and passwords. A good start there is the emphasis on using SSL 
whenever possible. All network services support it - more or less.

Intelligent network design can also prevent some if it. Like don't put all 
eggs in one basket and have all client boxes in the same network segment 
(switched or not).

I've also seen that large ISPs run custom kernels which prevent you from
using 
sniffers as they hacked the kernel in a way that it can no longer switch the

network cards into promiscuous mode through the usual means.

Or there is OpenVZ, Virtuozzo or Aventurin{e}, where the virtualization
layer 
prevents anyone (including "root") in a VPS to see any network traffic but
the 
one that's relevant to him.

None of that provides an all around protection, but it can help.

> I already run fail2ban and use the inbuilt Pam intrusion protection but
> guess that would not afford protection against a root toolkit attack.

Then that's already a good start. You have to keep in mind that rootkits
don't 
infect a system out of the blue. It has been a while since a daemon was
found 
that had a network exploitable vulnerability that - if exploited - instantly

granted you "root" access. The typical exploits you see these days involve 
multiple steps. During the first step the attacker gains unprivileged shell 
access. Through a vulnerable web application (PHP or Perl script) for
example. 
Or by brute force password guessing attempts. To some degree BlueOnyx
already 
protects against these due to the PHP security settings, which limit the 
damage that an attacker can do or which prevent remote code inclusion. Or 
there is the new brute force login attempt blocker, which blocks IPs after 
repeated failed login attempts.

Still, if someone gets past that layer and gets unprivileged local access, 
then that's bad. But he still doesn't have root access yet. So he needs to 
find another locally exploitable vulnerability to get to that point. If you 
don't allow shell access to your boxes, then the attacker is furtherly 
hindered. He'll have to upload exploit code through FTP and has to hope that

the browser allows him to run the commands he needs for the task. The
default 
PHP security settings for sites will certainly p*ss him off quite a bit, but

it's an obstacle that can be overcome with some efforts. So in oder to load 
the rootkit, the attacker needs to find that vulnerability that allows him
to 
gain root access first. The rootkit is then used to hide the intrusion and
to 
mask the changes made to the system and to cloak the hackers presence in the

system.

Personally I haven't seen a thoroughly hacked (i.e.: "root" access was 
established) BlueOnyx yet, but I've heard of two cases. However, when I look

at past experience with hacked boxes (I used to be frequently asked to do 
post-mortem forensics on hacked boxes or give an estimate if it was 
salvageable), then I can say this:

Over the last few years it appears that both BlueQuartz (and BlueOnyx) have 
been subject to the usual (but declining) number of unauthorized and 
unprivileged non-root accesses. Typically for the purpose of sending SPAM or

phishing emails. The drop in numbers there can be explained with the fact
that 
most of this SPAM is nowadays sent through bot-nets of infected home user
PCs, 
which are a much softer and easier target. Also the implementation of SMTP-
Auth a few years ago made it easy for most admins to make sure that they 
unintentionally turned their boxes into open relays. Of course that's a 
different topic as exploiting that isn't a hack in the traditional sense.

But the number of root-compromises or hostile takeovers of servers has
dropped 
considerably. At least in our small and concealed world of BlueQuartz and 
BlueOnyx. On the Cobalts it was MUCH worse during their days of glory. Part
of 
this can be attributed to the OS we use now, which is more modern and better

maintained (although CentOS really slacked a lot with CentOS4!). Part of it 
may be a tribute to the limited "spread" that BlueQuartz and BlueOnyx
servers 
have when you compare the numbers with the enourmous spread that Cobalt
boxes 
had in their days of old, or with todays market share of other target groups

such as Plesk, Ensim or Cpanel. But it may also be a tribute to dropping 
prices for root servers.

A few years ago you had to pay like $250-500 US a month for a well connected

data center hosted server - when this used to be a lot of money. And
typically 
such contracts had a duration of one year.  

Nowadays you can get a small VPS with root access for the price of a pizza
and 
you can cancel that contract pretty much at any time or to the end of the 
month at the latest. Plus the average home user now has access to bandwiths 
which - a few years ago - were only available in data centers or through 
expensive dedicated lines.

Don't get me wrong: I don't want to say that there is no threat, nothing to 
see and move along please <grin>. Not at all. Some well designed extra 
security, an investment in network security and proper network design and 
being prepared and vigilant is always good and recommended. Especially if
you 
host websites which - for political or practical reasons - can be considered

"high value targets". 

But the threats have changed and a lot of recommendations you find on the
net 
in regards to security may no longer be really adequate. They should still
be 
taken into some sort of consideration, but don't use them as main line of 
defense. In a recent (private) security discussion someone praised his 
implementation of Portsentry and I though to myself: "Dude, you just 
disqualified yourself." :o)

-- 
With best regards

Michael Stauber

_______________________________________________
Blueonyx mailing list
Blueonyx at blueonyx.it
http://www.blueonyx.it/mailman/listinfo/blueonyx