Tag Archives: openssl

OpenSSL v 1.0.2: building / compiling from source (Linux / CentOS / Red Hat)

As a precursor of installing a web server such as Apache (http version 2.4.10) and to avail of certain features as in able to configure the certificate chain on a certificate basis, we need to update to this version.

An earlier post detailed the steps to compile the OpenSSL v1.0.1 series and this one would cover the next version. The steps remain the same however the earlier post is more detailed.

  1. Run “config”
  2. Run “make”
  3. Run “make test”
  4. Run “make install”

A little more granularity:


$ ./config shared --prefix=/installs/openssl-1.0.2

Here: I have specified the installation folder to be explicit. You could point it to any other location or go with the default “/usr/local/ssl”. Note that if you would like to compile in the ECC Optimizations (ECC accelerator) refer to my earlier post.


$ make
$ make test
$ make install

You might face build issues as in “gcc” not being available or an incorrect version. Try “yum update gcc”.

The final step is to set the environment as in:


export LD_LIBRARY_PATH=/installs/openssl-1.0.2/lib:$LD_LIBRARY_PATH
export PATH=/installs/openssl-1.0.2/bin:$PATH

DTLS and OpenSSL: quick setup for server and client

To quickly setup either a DTLS server or client, the “s_client” and the “s_server” utilities can be utilized.

On the server, run “s_server”, provide it the certificate and the private key and specify the port:


$ openssl s_server -cert cert.pem -key pk.pem -dtls1 -accept 4444
Enter pass phrase for pk.pem:
Using default temp DH parameters
Using default temp ECDH parameters
ACCEPT
----

On the client, run “s_client” and you would see something akin to the following:


$ openssl s_client -dtls1 -connect xx.xx.xx.xx:4444 -debug
...
...
New, TLSv1/SSLv3, Cipher is ECDHE-RSA-AES256-SHA
Server public key is 2048 bit
Secure Renegotiation IS supported
Compression: NONE
Expansion: NONE
SSL-Session:
Protocol : DTLSv1
Cipher : ECDHE-RSA-AES256-SHA
Session-ID: ....
Session-ID-ctx:
Master-Key: .....
Key-Arg : None
PSK identity: None
PSK identity hint: None
SRP username: None
TLS session ticket:

CVE-2014-3513 – OPENSSL_NO_SRTP – is it compiled into your version of OpenSSL

If you would want to confirm if your version of OpenSSL that was compiled months ago (and the options specified at that time are forgotten) provides for SRTP support then one way to do that would be to utilize “objmap” on linux. If you see functions such as the following then it was not compiled with the “OPENSSL_NO_SRTP” option.

It seems that SRTP is compiled by default and is supported by default.

The command to use:

SSL 3.0 and POODLE (CVE-2014-3566)

We have a new vulnerability well explained here. The easiest solution is to remove support for SSL 3.0 from the web server – that in itself is a trivial thing to do – be it Apache or Nginx. However there might be clients that support SSL 3.0 exclusively and none of the TLS versions.

As of now I see that Chrome, Firefox 33 and the Google Web Server (the server that powers its sites) supports this extension.

Once support for TLS Fallback Signaling Cipher Suite Value (SCSV) is available in OpenSSL then the web servers would support it as well.

References:

OpenSSL Heartbeat vulnerability – Heartbleed and Java, BouncyCastle, How to write a program to check

To write a program to validate if a particular web server that runs on a version susceptible to “heartbleed”, one could use the plethora of free tests on the web such as the ones linked from the Wikipedia link on the subject. I am the author of the heartbleed test for Symantec SSL at: “https://ssltools.websecurity.symantec.com/checker/views/certCheck.jsp“.  Note: the codebase in that is completely different and follows a completely different approach to what I am going to release to the general public: a sample code that demonstrates heartbeat requests with BouncyCastle.

However, if the server that is to be verified is not accessible from the outside, you would need to write your own or download one. There are quite a few of python, Go and even one that details the changes to be made to OpenSSL s_client and use them to discover the vulnerability. Some of them work, some some of the time and when they do not, one need to understand why.

The following sections would briefly explain the OpenSSL vulnerability and the fix and how to write one of your own.

The vulnerability

An improper  heartbeat (HB) request would lead to a vulnerable web server leaking the content in its memory. This content could be a secret key or its password and so on.

While testing my HB tester program, noticed that there were attacks on my external facing web server at a rate of 2 every 10 minutes. There are a lot of attacks going on at this time.

The HB request

The request is in the following format:


HB request => ContentType (1 byte) : TLS Version (2 bytes: major and minor) : Record Size (2 bytes) : Encrypted or not Encrypted bytes

An example in hex: 18 0303 00cc and then the encrypted or not encrypted HB message follows.

More:

Encrypted or not Encrypted bytes => HB message Type : Payload Size : Payload : Padding

An example in hex for a non encrypted HB request message: 1 00cf and the actual payload and padding follow.

An improper HB request

An improper or attack vector request would create a payload of size less than what it specifies in the “Payload”. As simple as that.

An improper HB request to a vulnerable OpenSSL installation would result in it returning a HB response. The same request to a patched / not vulnerable OpenSSL (or any other web server that is not susceptible to HB) would result in a TLS alert.

The vulnerability in a little more detail

The malformed message results in an affected OpenSSL version returning a payload of the same size that is specified in the HB request irrespective of the real payload in the request. The affected OpenSSL version does not validate this aspect and the response payload is read from the memory going way beyond and leaking memory contents.

 The patch

The patch is a test of the payload length specified in the request and the actual payload size.

if (1 + 2 + payload + 16 > s->s3->rrec.length) return 0;
/* silently discard per RFC 6520 sec. 4 */

If the 1 byte that specifies the size of the HB record type (request in this case) plus the 2 bytes of that specify the payload length plus the size of the payload that is specified plus the minimum size of the padding (that is 16 bytes) is GREATER than the record length then no HB response is to be returned.

Design of a program to test for this vulnerability using Java

  1. Create a Java Socket to the web server
  2. Get a sample of a TLS ClientHello from TCPDUMP (Use s_client to send in a HB request and capture the packets to get the sample) (Make sure that it has the Heartbeat Extension setup to allow for heartbeats).
  3. Write the TLS ClientHello bytes to the socket
  4. Read the ServerHello response bytes till the end
  5. Sent in the malformed HB request (construct it in the way that is described above)
  6. Check the response: if it not an TLS alert then the web server is vulnerable. The server could reset the connection or timeout and we can assume – in these cases – that the server is not vulnerable. However there is a infinitesimal chance of a false negative especially in case of a connection timeout (and consequently the server certified to be not vulnerable) then the connection time out could be due to an actual connection issue! Note that to circumvent the heartbleed issue, network adminstrators have deployed firewall rules that would time out a heartbeat request – valid or not.

There are a huge number of types of web servers out there. The above steps would in all probability work in properly diagnosing this vulnerability in Apache and Nginx on Linux but would fail with IBM HTTP Web Server or IIS. In such a scenario, one would parse the complete ServerHello and check whether it is an extended ServerHello and if it is would check for the existence of the heartbeat extension.

Also note, that the SSL version 3.0 RFC does not allow an extended ClientHello or a ServerHello so the suggestion is to use a TLS 1.0 ClientHello in this case.

The other alternate approach would be to use BouncyCastle and send in heartbeat requests after the TLS session has been established: check out TLSProtocolHandler. If I have time, I would post that piece of code for your perusal. However as of now, I have experimented with BouncyCastle and have been successful in:

  • Establishing a TLS session
  • Sending in an encrypted “valid” TLS heartbeat request and received an encrypted heartbeat response.
  • Sending in an encrypted “invalid” TLS heartbeart request and received an encrypted heartbeat response if the server is vulnerable to heartbleed. Else, we receive an Alert, connection reset or a socket read timeout that points to a patched or a server unaffected by heartbleed.

NB: It seems to me that a valid heartbeat request is only allowed by OpenSSL after a TLS session has been established. I have tested (and you can test either with s_client or your own tool perhaps using BouncyCastle) sending a valid heartbeat request after establishing a TLS session. I established a valid TLS session and sent in an encrypted heartbeat and was able to elicit a heartbeat response using java and bouncycastle. I have not cleaned out the code and once I do will post it. So empirically, it seems that even in OpenSSL versions that are broken, a valid heartbeat request right after ServerHelloDone is disallowed. That would be the reason that a heartbeat response is not forthcoming for a valid heartbeat request send before the TLS handshake is complete.

Happy testing.

 

 

 

OpenSSL verify a certificate chain (chain verification and validation) using the “verify” command

In addition to the verification of the chain through the “s_client” command demonstrated earlier in the series, one can also use the ” verify” command to the same. It is easier in the case when the certificate chain is not already installed on a web server (in that case we can use the verify option with the “s_client” command) or it is a chain for the client certificates.

In the following example, we have an end-entity client certificate (PEM encoded) in 1.pem and the intermediate certificate in 2.pem. The root self-signed CA certificate is in 3.pem. We are verifying the end-entity certificate (1.pem) with the intermediate CA certificate (2.pem).


$ openssl verify -verbose -purpose sslclient -CAfile 2.pem 1.pem
1.pem: C = US, CN = XXX, O = YYY
error 20 at 0 depth lookup:unable to get local issuer certificate

To delve deeper for the failure, we need to add the “issuer_checks” option to display all the checks that are taking place. And we notice that the intermediate certificate does not have the “Certificate Signing” (KeyCertSign) bit set so the verification fails. We need to ask for a intermediate CA certificate with the right key usage bits. Please see “keyCertSign” in RFC 5280.

$ openssl verify -verbose -issuer_checks -purpose sslclient -CAfile 2.pem 1.pem
1.pem: C = US, CN = XXX, O = YYY
error 29 at 0 depth lookup:subject issuer mismatch
...
error 32 at 0 depth lookup:key usage does not include certificate signing
.....
....
error 20 at 0 depth lookup:unable to get local issuer certificate

 

What is the encoding of the SSL certificates on the wire and how is the certificate chain configured?

It is DER and it follows the RFC for TLS v1.2. Opened up WireShark and exported the raw bytes for one of the certificates among the chain transmitted by the server in the SSL / TLS “Certificate” message and decoded it and validated the DER encoding. This was on an HTTPS connection to an Apache Web Server.

The other question that web server administrators and writers of server certificate verification code would need to know is what should be the order of the certificates in the certificate chain that is being sent back by the web server. The RFC provides details on that as well wherein the sender’s certificate must come in first followed by the certificate that would certify it and so on.

Nginx 1.2.x and install Elliptic Curve Crytpography (ECC) support – installation on Linux (.configure options and build for SSL / TLS support and enable HTTPS)

As of writing this and as far as I know, the pre-compiled binaries for nginx for various platforms (RedHat / CentOS or another linux variant) do not come with ECC support so you would not be able to utilize ECC based certificates (ECDHE key exchange or ECDSA  authentication). The solution is to compile the Nginx source code with an OpenSSL version that has ECC support such as OpenSSL 1.0.1c or 1.0.1e. As of writing, 1.0.1c has a vulnerability (please see the OpenSSL web site for more details) and 1.0.1e is the recommended version.

When comparing with Apache HTTPD web server compilation, I found building Nginx to be simpler based on the fact that one needs to specify the OpenSSL source and Nginx build process takes care of building and linking to it. If you are only interested in building OpenSSL from source with ECC support then refer to this post.

After downloading Nginx source, run the following to check the options for configure:

./configure --help

This will list out all options that determine what modules to enable or disable, locations of dependencies such as OpenSSL if they are not in the obvious locations.

Since we have downloaded the OpenSSL source (1.0.1x) to support ECC into a different folder, we need to specify that so the configure option becomes:

./configure --prefix=/app/installs/nginx --with-http_ssl_module --with-openssl=/app/source/openssl/openssl-1.0.1c

This implies that nginx will be installed at “/app/installs/nginx” with the module to add SSL / TLS support and the location of the OpenSSL source is specified as well (this is where the OpenSSL source was extracted).

Thereafter run the following commands:

make

Switch to root if not already so and

make install

Uncomment the HTTPS / SSL sections from the Nginx configuration file and specify the certificates and you are all set.

To check the options for the nginx command line:

nginx --help

To start nginx:

nginx

If you get errors about PCRE at the configure stage or later (error messages replicated below) and if you have previously installed it, update the LD_LIBRARY_PATH environment variable to include the library but if you do not have it installed, there is a section on this blog on installing PCRE. All one has to do is to download the source and simply install that. Another approach is to install the PCRE development libraries. Both approaches are outlined below.

Error Message 1 (at configure time):


./configure: error: the HTTP rewrite module requires the PCRE library.
You can either disable the module by using --without-http_rewrite_module
option, or install the PCRE library into the system, or build the PCRE library
statically from the source with nginx by using --with-pcre= option.

Error Message 2 (later at run time):


nginx: error while loading shared libraries: libpcre.so.1: cannot open shared object file: No such file or directory

Solution 1:

While configuring nginx, one can specify the location of the source of PCRE source (8.3.1 is the version that I used and can be downloaded from the PCRE website) at the configure step:

./configure ..... ..... ... --with-pcre=/app/source/pcre/pcre-8.31

And repeat the “make, make install” steps as outlined earlier.

Solution 2:

Alternatively if PCRE is already installed then simply point to it (you would need the development libraries):

$ yum search pcre
Matched: pcre ==
opensips-regex.x86_64 : RegExp via PCRE library
pcre.i386 : Perl-compatible regular expression library
pcre.x86_64 : Perl-compatible regular expression library
pcre-devel.i386 : Development files for pcre
pcre-devel.x86_64 : Development files for pcre

Then install the development version:

$ yum install pcre-devel

Time to reconfigure and install Nginx:


$ ./configure ...... [same arguments as above]

A successful “./configure” would have something akin to this output:

$ ./configure .........
checking for OS
+ Linux 2.6.18-128.1.6.el5 x86_64
checking for C compiler ... found
+ using GNU C compiler
+ gcc version: 4.1.2 20080704 (Red Hat 4.1.2-44)
checking for gcc -pipe switch ... found
checking for gcc builtin atomic operations ... found
checking for C99 variadic macros ... found
checking for gcc variadic macros ... found
checking for unistd.h ... found
checking for inttypes.h ... found
checking for limits.h ... found
checking for sys/filio.h ... not found
checking for sys/param.h ... found
checking for sys/mount.h ... found
checking for sys/statvfs.h ... found
checking for crypt.h ... found
checking for Linux specific features
checking for epoll ... found
checking for sendfile() ... found
checking for sendfile64() ... found
checking for sys/prctl.h ... found
checking for prctl(PR_SET_DUMPABLE) ... found
checking for sched_setaffinity() ... found
checking for crypt_r() ... found
checking for sys/vfs.h ... found
checking for nobody group ... found
checking for poll() ... found
checking for /dev/poll ... not found
checking for kqueue ... not found
checking for crypt() ... not found
checking for crypt() in libcrypt ... found
checking for F_READAHEAD ... not found
checking for posix_fadvise() ... found
checking for O_DIRECT ... found
checking for F_NOCACHE ... not found
checking for directio() ... not found
checking for statfs() ... found
checking for statvfs() ... found
checking for dlopen() ... not found
checking for dlopen() in libdl ... found
checking for sched_yield() ... found
checking for SO_SETFIB ... not found
checking for SO_ACCEPTFILTER ... not found
checking for TCP_DEFER_ACCEPT ... found
checking for TCP_KEEPIDLE, TCP_KEEPINTVL, TCP_KEEPCNT ... found
checking for TCP_INFO ... not found
checking for accept4() ... not found
checking for int size ... 4 bytes
checking for long size ... 8 bytes
checking for long long size ... 8 bytes
checking for void * size ... 8 bytes
checking for uint64_t ... found
checking for sig_atomic_t ... found
checking for sig_atomic_t size ... 4 bytes
checking for socklen_t ... found
checking for in_addr_t ... found
checking for in_port_t ... found
checking for rlim_t ... found
checking for uintptr_t ... uintptr_t found
checking for system byte ordering ... little endian
checking for size_t size ... 8 bytes
checking for off_t size ... 8 bytes
checking for time_t size ... 8 bytes
checking for setproctitle() ... not found
checking for pread() ... found
checking for pwrite() ... found
checking for sys_nerr ... found
checking for localtime_r() ... found
checking for posix_memalign() ... found
checking for memalign() ... found
checking for mmap(MAP_ANON|MAP_SHARED) ... found
checking for mmap("/dev/zero", MAP_SHARED) ... found
checking for System V shared memory ... found
checking for POSIX semaphores ... not found
checking for POSIX semaphores in libpthread ... found
checking for struct msghdr.msg_control ... found
checking for ioctl(FIONBIO) ... found
checking for struct tm.tm_gmtoff ... found
checking for struct dirent.d_namlen ... not found
checking for struct dirent.d_type ... found
checking for sysconf(_SC_NPROCESSORS_ONLN) ... found
checking for openat(), fstatat() ... found
checking for getaddrinfo() ... found
checking for PCRE library ... found
checking for PCRE JIT support ... not found
checking for OpenSSL library ... found
checking for zlib library ... found
creating objs/Makefile

Configuration summary
+ using system PCRE library
+ using system OpenSSL library [or the source location]
+ md5: using OpenSSL library
+ sha1: using OpenSSL library
+ using system zlib library

nginx path prefix: “/app/…..”
nginx binary file: “/app/….”
nginx configuration prefix: “/app/…..”
nginx configuration file: “/app/…..”
nginx pid file: “/app/…/nginx.pid”
nginx error log file: “/app/…./logs/error.log”
nginx http access log file: “/app/…../logs/access.log”
nginx http client request body temporary files: “client_body_temp”
nginx http proxy temporary files: “proxy_temp”
nginx http fastcgi temporary files: “fastcgi_temp”
nginx http uwsgi temporary files: “uwsgi_temp”
nginx http scgi temporary files: “scgi_temp”

And then proceed with the make and make install.

OpenSSL’ s_time command simple and short tutorial – CPU user time versus real time

A succinct tutorial on s_time and the interpretation of its results

One can install OpenSSL and do a quick check with respect to the performance of a remote server. For instance: the s_time invocation will attempt to make as many connections for a specified period of time. The default period is 30 seconds but one can override that with the appropriate option (“-time”) in this case. With s_time, we can get the numbers of connections per second that are full handshakes as well as resumed handshakes. For details on what “handshake” implies, one could refer to other texts on the web such as the wikipedia page on “Secure Sockets Layer” that has a succinct explanation of the different flavors of handshakes including “resumed” handshakes. Please see the references section below for the link.

The key facet that would like to emphasize is that this command does not invoke the server through concurrent connections but it is sequential and attempts to extract the total time that X connections took in the time (default is 3o secs) specified. For instance, we infer from the run below that for “new” connections, the total number of connections made were 107 and the total time expended in those connections was 1.20 seconds (CPU user time). The test was run for around 30 seconds.

openssl s_time  -cipher 'RSA' -connect host:443 -CAfile chain.pem -www /

Collecting connection statistics for 30 seconds
Collecting connection statistics for 30 seconds
ttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttttt

107 connections in 1.20s; 89.17 connections/user sec, bytes read 44298
107 connections in 31 real seconds, 414 bytes read per connection

Now timing with session id reuse.
starting
trrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrrr

126 connections in 0.07s; 1800.00 connections/user sec, bytes read 52164
126 connections in 31 real seconds, 414 bytes read per connection

 

From the snippet above, one can also realize that the in the “reuse” (session resumption) case, we see that the number of connections has increased to 126 and it can be extrapolated to 1800 connections per second. Please note that the rest of the 31 seconds, the program was busy in network IO etc.

Also note that if SSL session cache is not setup on the server then s_time will display the same result as for “new” connections. This command does not support RFC 5077: TLS Session Resumption without Server-Side State.

References

  • http://en.wikipedia.org/wiki/Secure_Sockets_Layer [Provides information on SSL / TLS handshakes]
  • http://tools.ietf.org/html/rfc5077 [RFC on TLS Session Resumption without Server-Side State]

Apache web server httpd 2.4.3 Build / Compile from source and ECC (Elliptic Curve) support (RHEL or CentOS Linux)

Overview

An earlier post had detailed setting up Apache httpd 2.2.22 with both DSA and RSA support in terms of SSL/TLS authentication. This post will detail setting up Apache httpd 2.4.3 with support for all three ciphers viz: RSA, DSA and ECC. The earlier post also covered the OpenSSL 1.0.x installation that supports all of these ciphers.

Please note that Apache httpd version 2.2.x does not have ECC support built in and it needs to be patched for ECC. However support for ECC is in trunk for the 2.4.x branch and that is the path that we will take.

Building Apache

  1. Download the source.
  2. Build./configure --prefix=/app/install/myinstalls/httpd-2.4 --enable-mods-shared="all ssl deflate disk-cache expires headers info cache proxy proxy-ajp proxy-balancer proxy-connect proxy-ftp proxy-http rewrite" --with-ssl=/usr/local/ssl --with-included-apr --with-pcre=/usr/local

    Note that we are utilizing the provided APR (Apache Portable Runtime) and are also pointing to the PCRE deployment. Please see the Prerequisite section below on the reasons for this.If there are any issues, run the following before retrying:

     

Prerequisites:

APR and APR-UTIL

Apache Portable Runtime (APR) and utils might need to be updated or installed if the following error is printed on the screen while configuring Apache httpd which is the first step in the build process. If while running configure, the following is spewed out then you need to download and install APR and APR-UTIL.
configure: error: APR not found. Please read the documentation

 

Steps for APR (1.4.6) and APR-UTIL (1.4.1)  setup:

  1. Download the source into “[Apache HTTPD build location]/srclib”. Extract it and make sure there are no version numbers in the folders.
    From the Apache httpd documentation (http://httpd.apache.org/docs/2.4/install.html):download the latest versions of both APR and APR-Util from Apache APR, unpack them into ./srclib/apr and ./srclib/apr-util (be sure the domain names do not have version numbers; for example, the APR distribution must be under ./srclib/apr/)
  2. Append the following to the Apache httpd configure command:
    --with-included-apr
  3. Continue with the Apache httpd configure process.

PCRE

If during the Apache httpd build process, the following is spewed out then we need to build and install PCRE.
configure: error: pcre-config for libpcre not found. PCRE is required and available from http://pcre.org/

 

Steps for PCRE (8.31) setup:

  1. Download the PCRE source. Save it at any location.
  2. The PCRE build and install process will generate both shared and static libraries and that implies we do not have to explicitly require the dynamic libraries to be built.
  3. The configure command and the build process then is:
    ./configure --prefix=/usr/local
    make
    make install
    If there is this error during the process then it implies that either libtools or GCC C++ compiler is not available:
    make[1]: Entering directory /app/install/myinstalls/pcre/pcre-8.31'CXX pcrecpp.lolibtool: compile: unrecognized option -DHAVE_CONFIG_H'libtool: compile: Try libtool --help' for more information.make[1]: *** [pcrecpp.lo] Error 1make[1]: Leaving directory /app/install/myinstalls/pcre/pcre-8.31'

    make: *** [all] Error 2


    Consequently, you would need to perform the following installs:

Miscellaneous

Errors related to mod_deflate and zlib

If the following error is spewed during configure:

checking for zlib location... not found

 

checking whether to enable mod_deflate... configure: error: mod_deflate has been requested but can not be built due to prerequisite failures

Then this implies that zlib or zlib-devel packages are missing or might need to be forced to be reinstalled. This should take care of installing them:

yum install zlib

 

yum install zlib-devel