[BlueOnyx:21783] Re: Memcached security issue

[Richard Morgan :: Morgan Web]

Thank you Michael for a detailed answer.

While I could stop and remove memcached, I'd appreciate you taking a look in
case something else has caused the issue. Login details from GUI to follow.

When I run the netstat command I get a different response which I think
implies it's listening on the public IP:

[root at s2 admin]# netstat -tupan|grep LISTEN|grep memcached
tcp        0      0 0.0.0.0:11211               0.0.0.0:*
LISTEN      25751/memcached
tcp        0      0 :::11211                    :::*
LISTEN      25751/memcached

Appreciate all your help.

Regards

Richard

> Secondly, has anyone experienced this and is a server-wide fix or 
> extra limitations Memcached available? What is Memcached used for on 
> BX and it is it possible to remove it?

During our "need for speed" initiative for BlueOnyx we looked at certain
ways to improve the speed of CCEd to accelerate the GUI interface. That
development drive has over the years taken various steps that produced
considerable results. However, not all of them were fruitful.

In December 2015 we had published a version of CCEd that used "memcached" to
cache certain frequent CCEd transactions.

Initial tests were promising enough to warrant a release as regular YUM
update, but during the course of the next few days users reported problems
related to multi-user/multi-language caching issues that we had not foreseen
and which put the whole approach of using "memcached" into question.

So early in January 2016 I published a rollback that stopped and disabled
memcached and reinstalled a CCEd that no longer depended on "memcached".

Hence the service "memcached" (if installed at all) should be inactive on a
BlueOnyx by default. There is no legitimate process or service on a fully
updated BlueOnyx that needs it and the RPM can in fact be deinstalled as
well:

rpm -e memcached
... or ...
yum remove memcached

I have no idea why it was still enabled on your server and of course it is
highly regrettable that the service was abused.

Even then it raises the question how *that* could have happened, as
"memcached" by default only binds to 127.0.0.1:11211 and is therefore only
accessible by local services and processes on your server and cannot be
reached remotely:

[root at 5208r ~]# /sbin/service memcached restart
Stopping memcached:                                        [FAILED]
Starting memcached:                                        [  OK  ]

[root at 5208r ~]# netstat -tupan|grep LISTEN|grep memcached
tcp        0      0 127.0.0.1:11211             0.0.0.0:*
   LISTEN      7483/memcached

So in order to exploit "memcached" and to use it to run attacks against
other servers someone would have had to gain elevated access on your
BlueOnyx in first place, because "memcached" could not be manipulated and
tricked into running attacks from the outside as it simply wasn't reachable
from the outside on its own.

This needs further investigation and I'd be glad to lend a hand to see what
was going in. Please use the GUI to submit a "Support Request"
(under "Software Updates") and tick the checkbox "Allow access" to submit
temporary login details to your server in a secure fashion.

Beyond that: As we don't need "memcached" on a BlueOnyx anymore, but some
servers still might have it installed I will publish an immediate YUM update
that will "obsolete" the RPM "memcached" to make sure that it's removed at
once. While this should not be necessary due to the service being configured
for localhost usage only *and* the service being disabled by default I'd
rather like to play it safe here.

--
With best regards

Michael Stauber
_______________________________________________
Blueonyx mailing list
Blueonyx at mail.blueonyx.it
http://mail.blueonyx.it/mailman/listinfo/blueonyx