Securing client-node communication in Cassandra Part3

This is the last post about securing client-node communication in Cassandra and it will show how to force Cassandra to ask the client for a certificate signed by a trusted CA.
So not only Cassandra nodes must have the right credentials as I showed in the last post but also the client must be trusted.

Let's start creating the certificate and key pairs for the client:

We can now embody Cassandra administrator and we can use Openssl to generate the keys and the request:

#openssl req -newkey rsa:1024 -keyout clientkey.pem -keyform PEM -out clientreq.pem

This will ask a password (I answered “mysupersecretpassword”, change it accordingly in the C code below if you change it), and CN, OU, O, L,C, essentially the same stuff we passed in the command line to keytool when we generated the keys, see the previous posts.

The ouput of the commands will be two files, clientkey.pem which contains the keys and clientreq.pem which will contain the request to be sent to the CA.

Of course instead of clientkey.pem and clientreq.pem you can use names you like.

Now let's act the role of CA:

#openssl ca -config gen_ca_cert.conf -in clientreq.pem

The CA has to use  his password in order to use his private key to sign the certificate, so type the CA password when prompted. At the end of command the certificate is ready and stored in the certs directory, the location of the certs directory is specified in the gen_ca_cert.conf under the section [ {CAname} ], and specifically in $dir/certs where $dir=./ (see my previous posts Securing client-node communication in Cassandra Part1); now we can give the certificate to Cassandra administrator as client_cert.pem, of course you can use another name, but in the following I will use that one.

Now as Cassandra administrator we must create a truststore for the node:

#keytool -keystore {TRUSTSTORE}.jks -alias CARoot -importcert -file ~/cacert.pem -keypass {TRUSTSTOREPASSORD} -storepass {TRUSTSTOREPASSORD} -noprompt

Basically we are storing the CA certificate which will grant for the client certificate, so that that node could trust the client.

Copy the {TRUSTSTORE}.jks in a directory accessible from Cassandra, e.g. I copy it in the configuration directory of Cassandra that in my system is in /etc/cassandra/conf/

As I installed Cassandra as a service I give the ownership of the file to the service/user.

# chown cassandra /etc/cassandra/conf/{TRUSTSTORE}.jks

In order to use the keystore generated let's modify cassandra.yaml, always as Cassandra administrator, as the following:

client_encryption_options:

enabled: true

# If enabled and optional is set to true encrypted and unencrypted connections are handled.

optional: false

keystore: /etc/cassandra/conf/{KEYSTORENAME}.jks

keystore_password: {KEYSTOREPASSWORD}

truststore:{TRUSTSTORE}.jks

truststore_password: {TRUSTSTOREPASSORD}

require_client_auth: true

# Set trustore and truststore_password if require_client_auth is true

# More advanced defaults below:

protocol: TLS

algorithm: SunX509

store_type: JKS

cipher_suites: [TLS_RSA_WITH_AES_128_CBC_SHA,TLS_RSA_WITH_AES_256_CBC_SHA,TLS_DHE_RSA_WITH_AES_128_CBC_SHA,TLS_DHE_RSA_WITH_AES_256_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_128_CBC_SHA,TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA]

As before instead of the words between brackets and brackets themselves you put the stuff that make sense for you.

Reboot Cassandra, on my system:

#service cassandra restart

If we try now to connect to Cassandra we are not trusted anymore:

# cqlsh --ssl

Connection error: ('Unable to connect to any servers', {'127.0.0.1': error(8, "Tried connecting to [('127.0.0.1', 9042)]. Last error: _ssl.c:492: EOF occurred in violation of protocol")})

We must therefore modify .~/.cassandra/cqlshrc as the following:

[connection]

hostname = 127.0.0.1

port = 9042

factory = cqlshlib.ssl.ssl_transport_factory

[ssl]

certfile = ~/.cassandra/cacert.pem

# Optional, true by default

validate = true

userkey = ~/ssl-cassandra/clientkey.pem

usercert = ~/ssl-cassandra/client_cert.pem

where clientkey.pem and client_cert.pem are respectively the client private key and the certificate signed by the CA.

Now set everything to be trusted by the node, we are using a certificate signed by a CA that is in its trusted list, in its truststore.
We can now try to connect to Cassandra node using cqlsh, of course we must give the correct password to ensure that the keys are ours, the password we gave when we generated the keys and the request at the beginning (in my case “mysupersecretpassword”).

#cqlsh --ssl

Enter PEM pass phrase:

Enter PEM pass phrase:

Connected to Test Cluster at 127.0.0.1:9042.

[cqlsh 5.0.1 | Cassandra 2.1.14 | CQL spec 3.2.1 | Native protocol v3]

Use HELP for help.

dle_relay@cqlsh>

we could remove the password from the file that stores the keys:

#openssl rsa -in clientkey.pem -out clientkey1.pem

and modify .~/.cassandra/cqlshrc to use the key without password:

[connection]

hostname = 127.0.0.1

port = 9042

factory = cqlshlib.ssl.ssl_transport_factory

[ssl]

certfile = ~/.cassandra/cacert.pem

# Optional, true by default

validate = true

userkey = ~/ssl-cassandra/clientkey1.pem

usercert = ~/ssl-cassandra/client_cert.pem

Now cqlsh will not prompt for password, but of course we don't like it and will immediately delete  clientkey1.pem and will revert .~/.cassandra/cqlshrc.

We played the Cassandra administrator and the CA role, now it is the time to play the programmer role.

In my previous post I showed how to use DataStax C/C++ driver to connect to a Cassandra DB where the node authenticate itself, now we have also the client that authenticate itself, so we need to modify the code in order to load the client certificate and client private.

We need, first of all, to add the function that loads the client certificate calling the cass_ssl_set_cert_n API:

int  
 load_clientCert_file (const char *file, CassSsl * ssl)  
 {  
  CassError rc;  
  char *cert;  
  long cert_size;  
  size_t bytes_read;  
  FILE *in = fopen (file, "rb");  
  if (in == NULL)  
   {  
    fprintf (stderr, "Error loading certificate file '%s'\n", file);  
    return 0;  
   }  
  fseek (in, 0, SEEK_END);  
  cert_size = ftell (in);  
  rewind (in);  
  cert = (char *) malloc (cert_size);  
  bytes_read = fread (cert, 1, cert_size, in);  
  fclose (in);  
  if (bytes_read == (size_t) cert_size)  
   {  
    rc = cass_ssl_set_cert_n (ssl, cert, cert_size);  
    if (rc != CASS_OK)  
      {  
       fprintf (stderr, "Error loading SSL certificate: %s\n",  
             cass_error_desc (rc));  
       free (cert);  
       return 0;  
      }  
   }  
  free (cert);  
  return 1;  
 }  

and a function that loads the client private key calling cass_ssl_set_private_key_n:

 int  
 load_privateKey_file (const char *file, CassSsl * ssl, const char *password)  
 {  
  CassError rc;  
  char *cert;  
  long cert_size;  
  size_t bytes_read;  
  FILE *in = fopen (file, "rb");  
  if (in == NULL)  
   {  
    fprintf (stderr, "Error loading certificate file '%s'\n", file);  
    return 0;  
   }  
  fseek (in, 0, SEEK_END);  
  cert_size = ftell (in);  
  rewind (in);  
  cert = (char *) malloc (cert_size);  
  bytes_read = fread (cert, 1, cert_size, in);  
  fclose (in);  
  int len = 0;  
  if (bytes_read == (size_t) cert_size)  
   {  
    rc = cass_ssl_set_private_key_n (ssl, cert, cert_size, password, len);  
    if (rc != CASS_OK)  
      {  
       fprintf (stderr, "Error loading SSL certificate: %s\n",  
             cass_error_desc (rc));  
       free (cert);  
       return 0;  
      }  
   }  
  free (cert);  
  return 1;  
 }  

we then eventually modify the main function:

 main ()  
 {  
  CassError rc = CASS_OK;  
  CassCluster *cluster = NULL;  
  CassSession *session = NULL;  
  CassSsl *ssl = NULL;  
  CassFuture *connect_future = NULL;  
  const char *username = "Cassandra";  
  const char *password = "Cassandra";  
  cluster = cass_cluster_new ();  
  cass_cluster_set_credentials (cluster, username, password);  
  session = cass_session_new ();  
  ssl = cass_ssl_new ();  
  cass_cluster_set_contact_points (cluster, "127.0.0.1");  
  cass_ssl_set_verify_flags (ssl,  
                   CASS_SSL_VERIFY_PEER_CERT |  
                   CASS_SSL_VERIFY_PEER_IDENTITY);  
  if (!add_ca_cert_file ("/home/ieio/.cassandra/cacert.pem", ssl))  
   {  
    fprintf (stderr, "Failed to load certificate\n");  
    cass_ssl_free (ssl);  
    cass_session_free (session);  
    cass_cluster_free (cluster);  
    return -1;;  
   }  
  if (!load_clientCert_file ("/home/ieo/.cassandra/client_cert.pem", ssl))  
   {  
    fprintf (stderr, "Failed to load client certificate\n");  
    cass_ssl_free (ssl);  
    cass_session_free (session);  
    cass_cluster_free (cluster);  
    return -1;  
   }  
  if (!load_privateKey_file  
    ("/home/ieio/.cassandra/clientkey.pem", ssl, "mysupersecretpassword"))  
   {  
    fprintf (stderr, "Failed to private key certificate\n");  
    cass_ssl_free (ssl);  
    cass_session_free (session);  
    cass_cluster_free (cluster);  
    return -1;  
   }  
  cass_cluster_set_ssl (cluster, ssl);  
  connect_future = cass_session_connect (session, cluster);  
  cass_future_wait (connect_future);  
  rc = cass_future_error_code (connect_future);  
  if (rc == CASS_OK)  
   {  
    printf ("Cassandra Connected\n");  
   }  
  else  
   {  
    printf ("Cassandra NOT Connected: %s\n", cass_error_desc (rc));  
    if (connect_future != NULL)  
      cass_future_free (connect_future);  
    return rc;  
   }  
  cass_ssl_free (ssl);  
  cass_session_free (session);  
  cass_future_free (connect_future);  
  cass_cluster_free (cluster);  
  return rc;  
 }  

To compile cut and paste the code in a file named Cassandra.c and if you use Linux give the following command:

#gcc Cassandra.c -l cassandra

the output is a.out and you can execute with typing:

#./a.out

and if everything is correct the result will be:

1465723034.040 [WARN] (src/connection.cpp:795:void cass::Connection::notify_error(const string&, cass::Connection::ConnectionError)): Host 127.0.0.1 received invalid protocol response Invalid or unsupported protocol version: 4

1465723034.041 [WARN] (src/control_connection.cpp:213:virtual void cass::ControlConnection::on_close(cass::Connection*)): Lost control connection on host 127.0.0.1

1465723034.041 [WARN] (src/control_connection.cpp:232:virtual void cass::ControlConnection::on_close(cass::Connection*)): Host 127.0.0.1 does not support protocol version 4. Trying protocol version 3...

Cassandra Connected

This concludes my series of posts on Securing client-node communication in Cassandra.

Securing client-node communication in Cassandra Part2

In the previous post I showed how to create a CA, how to use the CA to sign certificate generated by keytool and how to use these certificates in Cassandra to protect node to client communication where the client was CQLSH.

In this new post I show how to use DataStax C/C++ driver instead of CQLSH.

DataStax C/C++ driver is a set of APIs in C/C++ to interface an application to Cassanda, it is available at https://github.com/datastax/cpp-driver and pretty well documented here http://docs.datastax.com/en/latest-cpp-driver/cpp-driver/whatsNew.html.

DataStax C/C++ driver is available also in binary format (http://downloads.datastax.com/cpp-driver/), depends on libuv, which is also available at the same place.

I based this post on the instructions I found here: https://docs.datastax.com/en/developer/cpp-driver/1.0/cpp-driver/reference/ssl.html but instead of using self signed certificates I used CA signed certificates.

So we need first of all a function to load the certificate of the CA using the cass_ssl_add_trusted_cert_n API:

 #include <cassandra.h>  
 #include <stdio.h>   
 #include <stdlib.h>  
 int add_ca_cert_file(const char* file, CassSsl* ssl) {  
  CassError rc;  
  char* cert;  
  long cert_size;  
  size_t bytes_read;  
  FILE *in = fopen(file, "rb");  
  if (in == NULL) {  
   fprintf(stderr, "Error loading certificate file '%s'\n", file);  
   return 0;  
  }  
  fseek(in, 0, SEEK_END);  
  cert_size = ftell(in);  
  rewind(in);  
  cert = (char*)malloc(cert_size);  
  bytes_read = fread(cert, 1, cert_size, in);  
  fclose(in);  
  if (bytes_read == (size_t) cert_size) {  
   rc = cass_ssl_add_trusted_cert_n(ssl, cert, cert_size);  
   if (rc != CASS_OK) {  
    fprintf(stderr, "Error loading SSL certificate: %s\n", cass_error_desc(rc));  
    free(cert);  
    return 0;  
   }  
  }  
  free(cert);  
  return 1;  
 }  

and then we can use it in our main function:

 main ()  
 {  
  CassError rc = CASS_OK;  
  CassCluster *cluster=NULL;  
  CassSession *session =NULL;  
  CassSsl *ssl=NULL;  
  CassFuture *connect_future = NULL;  
  const char *username = "Cassandra";  
  const char *password = "Cassandra";  
  cluster = cass_cluster_new ();  
  cass_cluster_set_credentials (cluster, username, password);  
  session = cass_session_new ();  
  ssl = cass_ssl_new ();  
  cass_cluster_set_contact_points (cluster, "127.0.0.1");  
  cass_ssl_set_verify_flags (ssl,  
                   CASS_SSL_VERIFY_PEER_CERT |  
                   CASS_SSL_VERIFY_PEER_IDENTITY);  
  if (!add_ca_cert_file ("/home/ieio/.cassandra/cacert.pem", ssl))  
   {  
    fprintf (stderr,  
          "Failed to load certificate\n");  
      cass_ssl_free (ssl);  
      cass_session_free(session);  
      cass_cluster_free(cluster);  
      return -1;  
   }  
  cass_cluster_set_ssl (cluster, ssl);  
  connect_future = cass_session_connect (session, cluster);  
  cass_future_wait (connect_future);  
  rc = cass_future_error_code (connect_future);  
  if (rc == CASS_OK)  
   {  
    printf ("Cassandra Connected\n");  
   }  
  else  
   {  
    printf ("Cassandra NOT Connected: %s\n", cass_error_desc (rc));  
    if (connect_future != NULL)  
      cass_future_free (connect_future);  
    return rc;  
   }  
  cass_ssl_free (ssl);  
  cass_session_free(session);  
  cass_future_free (connect_future);  
  cass_cluster_free(cluster);  
  return 0;  
 }   

as you can see add_ca_cert_file is loading the same certificate we user to make CQLSH works, do u remember? ~/.cassandra/cacert.pem

To compile cut and paste the code in a file named Cassandra.c and if you use Linux give the following command:

#gcc Cassandra.c -l cassandra

the output is a.out and you can execute with typing:

#./a.out

and if everything is correct the result will be:

1465492279.189 [WARN] (src/connection.cpp:795:void cass::Connection::notify_error(const string&, cass::Connection::ConnectionError)): Host 127.0.0.1 received invalid protocol response Invalid or unsupported protocol version: 4

1465492279.189 [WARN] (src/control_connection.cpp:213:virtual void cass::ControlConnection::on_close(cass::Connection*)): Lost control connection on host 127.0.0.1

1465492279.189 [WARN] (src/control_connection.cpp:232:virtual void cass::ControlConnection::on_close(cass::Connection*)): Host 127.0.0.1 does not support protocol version 4. Trying protocol version 3...

Cassandra Connected

Warnings are due to the fact I am still using Cassandra 2.1 which does not support protocol 4.

In the next post as promised in the previous one I will show how to force Cassandra to ask us for a signed certificate in order to establish the secure (TLS) communication and how to use CQLSH and DataStax C/C++ driver in this case.