Securing client-node communication in Cassandra Part1

Recently I came across this nosql database stuff, I did some researches and I landed in Cassandra world.

You can find information about Cassandra here: http://cassandra.apache.org/.

One of the issues I encountered working with Cassandra was securing the database, there is an excellent post (http://thelastpickle.com/blog/2015/09/30/hardening-cassandra-step-by-step-part-1-server-to-server.html) about  how to set up node to node encryption using Certificates and CA, but I didn't find a good tutorial for client to node encryption.

In this post I will do my best to show how to setup a secure communication in Cassandra between a client and a node, but first of all let's see how easy it is to sniff a client to node communication.

In the following pictures I sniffed with wireshark cqlsh asking Cassandra cql version (SELECT cql_version FROM system.local;)

and Cassandra answering (3.2.1)

I am too much paranoiac to let it be, I want an encrypted version, otherwise I won't have good dreams.

So first first of all let's create a CA that will sign SSL certificates, then we will make ourselves  sure that Cassandra is using a certificate signed by our "trusted" CA in order to set up a secure (encrypted) connection.

I use Linux so all the commands I will show in the following will work on a Linux system.

For the sake of my mind I prefer to keep separate the user that will act as CA from the user that will manage Cassandra, even if those users are always me.

Create the user that will act as a CA, he will generate the pair of keys, he will publish his public certificate and he will sign the request from the other user:

# useradd user-ca

# passwd user-ca

Create two directory, one to store the private key and one for the certificates:

#mkdir certs private

Protect the private directory so that only the CA user can read, write and enter in it:

#chmod 700 private

Create the serial file that will track the serial number of the certificates and the index.txt file that act as a database of signed certificates

#echo '01' > serial

#touch index.txt

Create a gen_ca_cert.conf like this:

[ ca ]

default_ca ={CAname}

[ {CAname} ]

dir =./

certificate = $dir/cacert.pem

database = $dir/index.txt

serial = $dir/serial

new_certs_dir = $dir/certs

private_key = $dir/private/privkey.pem

default_days = 365

default_md = md5

policy = CAuser_policy

[ CAuser_policy ]

stateOrProvinceName = optional

countryName = supplied

organizationName = supplied

organizationalUnitName =optional

commonName = supplied

[ req ]

distinguished_name = req_distinguished_name

prompt = no

default_bits = 2048

default_keyfile =./private/privkey.pem

[ req_distinguished_name ]

C = {COUNTRY}

ST = {PROVINCE}

L = {CITY}

O = {ORGANITATION}

OU = {ORGANITATION UNIT}

CN = {COMMON NAME}

emailAddress = {EMAIL ADDRESS}

where instead of the words between brackets and brackets themselves put the stuff that make sense for you.

Now it is possible to generate the CA private key and CA certificate:

#openssl req -config gen_ca_cert.conf -new -x509 -out cacert.pem

where

• req: it is used to emit the root self signed certificate
• -config specifies to user the conf file created before
• -new specifies to generate a new certificate request
• -x509 specifies the certificate format
• -out specifies the name of the file that will be generated

A password will be required to protect the private key and it will be prompted to be inserted.

Now private key and certificate are generated, the key is stored in directory named private and the certificate in userCA home:

#ls -l

total 20

-rw-rw-r--. 1 CAuser CAuser 1314 May 25 14:24 cacert.pem

drwxrwxr-x. 2 CAuser CAuser 4096 May 25 11:52 certs

-rw-rw-r--. 1 CAuser CAuser 656 May 25 14:39 gen_ca_cert.conf

-rw-rw-r--. 1 CAuser CAuser 0 May 25 11:58 index.txt

drwx------. 2 CAuser CAuser 4096 May 25 14:23 private

-rw-rw-r--. 1 CAuser CAuser 3 May 25 11:58 serial

#ls -l private/

total 4

-rw-rw-r--. 1 CAuser CAuser 1834 May 25 14:24 privkey.pem

Now we can create the certificate and private key for each node, login as Cassandra administrator and issue the following command:

#keytool -genkeypair -keyalg RSA -alias client -keystore {KEYSTORENAME}.jks -storepass {PASSWORD} -keypass {PASSWORD} -validity 365 -keysize 2048 -dname "CN={IPADDRESS}, OU={ORGANIZATION UNIT}, O={ORGANIZATION}, L={LOCATION}, S={STATE},C={COUNTRY}"

where instead of the words between brackets and brackets themselves you put the stuff that make sense for you.

I use keytool instead of openssl for two reasons, first I found tutorials using it, and I copy from them, second it already generates the keystore in the format  Cassandra uses.

Keytool can be found in the bin directory of JDK distribution, I use jdk1.8.0_77

Now that {KEYSTORENAME}.jks is generated the certificate request can be generated:

#keytool -keystore {KEYSTORENAME}.jks -alias client -certreq -file {CERT-REQ} -keypass {KEYSTOREPASSWORD} -storepass {KEYSTOREPASSWORD}

this will create {CERT-REQ} that can be passed to the CA so that it can sign it.

We are now CAuser:

#openssl ca -config gen_ca_cert.conf -in {CERT-REQ}

Now the signed certificate is ready and stored in the certs directory:

#ls -la certs/

drwxrwxr-x. 2 CAuser CAuser 4096 May 25 17:59 .

drwx------. 13 CAuser CAuser 4096 May 26 09:33 ..

-rw-rw-r--. 1 CAuser CAuser 3963 May 25 17:59 01.pem

Give 01.pem back to Cassandra admin.

Let's change hat again, we are Cassandra admin.

The CA certificate and the signed certificate of the node should now be stored in the {KEYSTORENAME}.jks

#keytool -keystore {KEYSTORENAME}.jks -alias CARoot -import -file cacert.pem -noprompt -keypass {PASSWORD} -storepass {PASSWORD}

for the CA certificate

#keytool -keystore {KEYSTORENAME}.jks -alias client -import -file 01.pem -keypass {KEYSTOREPASSWORD} -storepass {KEYSTOREPASSWORD}

and for the signed certficate.

Copy the {KEYSTORENAME}.jks in a directory accessible from Cassandra, e.g. I copy it in the configuration directory of Cassandra that in my system is in /etc/cassandra/conf/ and as I installed Cassandra as a service I give the ownership of the file to the service/user

Login as root, or use sudo and type:

#cp {KEYSTORENAME}.jks /etc/cassandra/conf/{KEYSTORENAME}.jks

#chown cassandra:cassandra

#exit

Now cassandra.yaml has to be modified as following go to the  client_encryption_options section and change it as in the following:

client_encryption_options:

enabled: true

# If enabled and optional is set to true encrypted and unencrypted connections are handled.

optional: false

keystore: /etc/cassandra/conf/{KEYSTORENAME}.jks

keystore_password: {KEYSTOREPASSWORD}

#truststore:truststore.jks

#truststore_password: cassandra

require_client_auth: false

# Set trustore and truststore_password if require_client_auth is true

# More advanced defaults below:

protocol: TLS

algorithm: SunX509

store_type: JKS

cipher_suites: [TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA]

Restart Cassandra.

Now you can try to connect to Cassandra:

#cqlsh –ssl

Validation is enabled; SSL transport factory requires a valid certfile to be specified. Please provide path to the certfile in [ssl] section as 'certfile' option in /home/ieio/.cassandra/cqlshrc (or use [certfiles] section) or set SSL_CERTFILE environment variable.

Now cqlsh  (the client) doesn't trust Cassandra node anymore, it needs the CA certificate in order to be sure that Cassandra is using a certificate signed by the CA.

Modify or create ~/.cassandra/cqlshrc as in the following:

[connection]

hostname = 127.0.0.1

port = 9042

factory = cqlshlib.ssl.ssl_transport_factory

[ssl]

certfile = ~/.cassandra/cacert.pem

# Optional, true by default

validate = true

and of course, copy the CA certificate in the directory you specify in the certfile and give right permission; as root if needed.

Now cqlsh can trust Cassandra and connect to it:

#cqlsh --ssl

Connected to Test Cluster at 127.0.0.1:9042.

[cqlsh 5.0.1 | Cassandra 2.1.14 | CQL spec 3.2.1 | Native protocol v3]

Use HELP for help.

>

If you trying to sniff the traffic with wireshark you cannot easily understand what is going on... phew now I can sleep.

But we want to always to  move further with security so let's force Cassandra to ask for a username and password each time we want to connect to it, in order to do that we just have to modify cassandra.yaml as in the following:

authenticator: PasswordAuthenticator

Restart Cassandra. If we now try to connect to it we will receive an error:

#cqlsh --ssl

Connection error: ('Unable to connect to any servers', {'127.0.0.1': AuthenticationFailed('Remote end requires authentication.',)})

So we must now add a username to the cqlsh command and a promt asking for a password will appear (default password is cassandra):

#cqlsh --ssl -u cassandra

Password:

Connected to Test Cluster at 127.0.0.1:9042.

[cqlsh 5.0.1 | Cassandra 2.1.14 | CQL spec 3.2.1 | Native protocol v3]

Use HELP for help.

>

Now we have a secure connection and the client can trust the node if we want to move even further we can force Cassandra to ask us for a signed certificate so that Cassandra can trust the client, but this will be the subject of one of the next posts.