[BlueOnyx:15144] OpenSSL CVE-2014-0160 (best practices)

[Michael Stauber]

Hi all,

I just finished an end run of securing about 50 servers. First among
them were the BlueOnyx YUM repositories and build boxes, list server,
webserver and what not. Then all the rest.

I'm a bit paranoid when it comes to security, but let me tell you a bit
about what I've done and then think about it which steps you want to
implement on your own servers, too.

The potential impact of CVE-2014-0160 is quite severe. As you know, it
affected only OpenSSL 1.0.1 through 1.0.1f. So for us that meant:

5106R 		=> no problem
5107R/5108R	=> affected

Even after running "yum update" to get the fixed OpenSSL RPMs and
rebooting, there are some things to consider:

A potential attacker could (before the update and reboot) have gained
privileged insight via TLS/SSL traffic and might have collected contends
of the memory of your affected server(s). There seems to be some
disagreement as of what memory contends might have leaked. Just those
memory ranges that the affected process occupied? Or all of it? The
first part can be assumed as certain, the second part is a bit unlikely.
The cautious answer would probably be: "Depends."

Still, the damage is bad enough, as the leaked memory information might
reveal privileged information such as login details of users connected
to the server, the private key of SSL certificates, private SSH keys,
session information and what not.

Like posted yesterday, the site http://heartbleed.com/ has some good
insights on the scope of things.

------------------------------------------------------------------------
Further good reads are these:

http://blog.existentialize.com/diagnosis-of-the-openssl-heartbleed-bug.html

http://www.reddit.com/r/netsec/comments/22gym6/diagnosis_of_the_openssl_heartbleed_bug/

Can Heartbleed be used to obtain memory from other processes?
http://security.stackexchange.com/questions/55097/can-heartbleed-be-used-to-obtain-memory-from-other-processes

What should end-users do about Heartbleed?
http://security.stackexchange.com/questions/55083/what-should-end-users-do-about-heartbleed

Does Heartbleed mean new certificates for every SSL server?
http://security.stackexchange.com/questions/55075/does-heartbleed-mean-new-certificates-for-every-ssl-server

What should one do about the Heartbleed OpenSSL exploit?
http://security.stackexchange.com/questions/55076/what-should-one-do-about-the-heartbleed-openssl-exploit
------------------------------------------------------------------------

My personal take on this prompted a thorough security audit and various
changes of my own infrastructure.

SSL Certificates:
==================

Personally I consider each and any SSL certificate on my own 5107R/5108R
as compromised and I have changed them all or am in the process of
changing them.

Which meant: A login to the BlueOnyx/Aventurin{e} GUI on all boxes. Then
"Security" / "SSL" / "Create Self-Signed Certificate" to create a new
self signed certificate (and/or "Create Signing Request").

I usually only use self signed certificates except for e-commerce sites.
However, the certificate of the GUI (self signed or not) *is* used not
only for HTTPS connections of the GUI, but also for FTPS and the SSL/TLS
secured email services. So creating new SSL certificates is sensible.

In the case of non self-signed certificates this means spending bucks
for new certificates. Which I think is warranted given these
extraordinary and lamentable circumstances.

Aside from replacing the SSL certificate of AdmServ all Vsites which
have self-signed or real certificates also might need new ones.

SSH:
====

I mostly use key based SSH logins and not password authentication. I
also run SSH usually on non-standard ports and/or block access to SSH
from all untrusted networks.

Still, CVE-2014-0160 has prompted me to create a new public/private SSH
key on my workstation and to install it on all servers. Even though SSH
is *not* using the TLS protocol and therefore ought to be in the clear.
At the same time I removed the old SSH public key from all servers
(/root/.ssh/authorized_keys).

As some of the servers are also allowed to connect to each other via SSH
(I extensively use "ssh-fs" for development) I also created new
public/private SSH keys on *all* servers. Which took the better part of
today due to the complex access rights that I maintain between servers.
Not every server is allowed to connect to every other, as that would be
a security risk.

If you want to do the same, here are some tips:

During the creation of a new public/private key-pair I went the extra
length of creating a longer than usual key-pair. I chose 8192 bits
instead of the usual 1024:

ssh-keygen -t rsa -b 8192

To distribute the new public key to ~user/.ssh/authorized_keys on the
target boxes it helps if you create the new private/public key-pair with
a different name first and rename it later. So that you can still SSH to
the server using your old SSH key for which you have granted access
previously.

The command "ssh-copy-id" is available on many Linux distributions and
can be used to copy a public key to a remote box and to install it
directly in the desired users .ssh/authorized_key file.

Example:

ssh-copy-id -i ~mstauber/.sshnew/id_rsa.pub root at over.there.net

See "man ssh-copy-id" for more info.

However, after that is done and you have verified that the new key
works, do not forget to remove the old public key from
.ssh/authorized_keys on the target box.

Login credentials:
==================

I changed all admin/root passwords and all passwords of each and any
account of importance. I advised my users to do the same.

Application related security:
============================

Now it gets shady and I have to generalize. Like everyone else I run
some cross-network services between servers which require authentication
in one form or other. Like server A being allowed to access MySQL on
server B. I have some legacy backup stuff running which uses FTP or
FTPS. Or I use ssh-fs to be able to mount a directory on a remote
storage to a local directory via SSH. Things like that. These services
also requires login credentials, certificates or key-pairs. I thought
this was a good time to change these passwords, certificates and
key-pairs as well.

My suggestion there would be: Think about your infrastructure and how
you have set things up. What uses passwords? What uses certificates?
Might be a good idea to look into that as well and to change things
accordingly.

Yes, this is a lot of work. In the cases where you have to buy new SSL
certificates it'll also cost money. Think about it and draw your own
conclusions if it's worth the effort or not. Your answer might be
different than mine.

But like said: I'm more of the paranoid type.

-- 
With best regards

Michael Stauber