Dissecting TLS Using Wireshark

The primary goal of the Transport Layer Security protocol as defined in RFC 2246 (TLS version 1.0) is “to provide privacy and data integrity between two communicating applications.” The TLS protocol ensures this by encrypting data so that any third party is unable to intercept the communication; it also authenticates the peers to verify their identity. By providing a secure channel of communication between two peers, TLS protocol protects the integrity of the message and ensures it is not being tampered.

History

TLS and SSL are used interchangeably. TLS evolved from SSL protocol (SSL 3.0) that is no longer considered secure; vulnerabilities such as POODLE attack has demonstrated this. TLS has gone through two iterations, RFC 4346 (TLS 1.1) and RFC 5246 (TLS 1.2), with the latest update TLS 1.3 being a working draft.

Architecture

TLS lies in between the application and the transport layer. It is designed to work on top of a reliable transport protocol such as TCP (but has been adapted to UDP, as well) and is divided into two sub-layers:

• TCP Record Protocol Layer – This is the lower layer which lies on top of the TCP layer and is responsible for:
• Fragmenting the message to be transmitted into manageable blocks
• Encrypting/Decrypting the hashed data
• Compressing/decompressing the outgoing/incoming data
• Applying the Message Authentication Code (MAC), a hash to maintain the data integrity
• Transmitting the data from the upper application layer to the lower transport layer and vice versa.
• The higher layer consists of the following sub-protocols:
• Alert: This sub-protocol defines the alert levels and provides a description of the alerts. It is used to notify the peer of any error condition that has occurred.
• Change Cipher Spec: It defines changes in ciphering strategies. The change cipher spec message, transmitted by both the client and the server, defines the re-negotiated cipher spec and keys that will be used for all the messages exchanged henceforth.
• Application Data: This protocol ensures that messages are fragmented, compressed, encrypted and transmitted in a secure manner.
• Handshake: To communicate over a secure channel, two peers must agree on the cryptographic keys and encryption algorithms for that session. TLS protocol describes the steps to authenticate the peers and set up a secure connection with defined parameters.  The entire sequence which involves setting up the session identifier, TLS protocol version, negotiating the cipher suite, certificate authentication of the peers and cryptographic key exchange between peers is called a TLS Handshake. The steps involved in the TLS handshake are shown below:

Analyzing TLS handshake using Wireshark

The below diagram is a snapshot of the TLS Handshake between a client and a server captured using the Wireshark, a popular network protocol analyzer tool.

Let’s analyze each step.

1. Initial Client to Server Communication

Client Hello

Typically, the first message in the TLS Handshake is the client hello message which is sent by the client to initiate a session with the server.

The message contains:

• Version: The TLS protocol version number that the client wants to use for communication with the server. This is the highest version supported by the client.
• Client Random: A 32-byte pseudorandom number that is used to calculate the Master secret (used in the creation of the encryption key).
• Session Identifier: A unique number used by the client to identify a session.
• Cipher Suite: The list of cipher suites supported by the client ordered by the client’s preference. The cipher suite consists of a key exchange algorithm, bulk encryption algorithm, MAC algorithm and a pseudorandom function. An example of a single cipher suite (one of the 28 suites mentioned in the above diagram) is as follows:

where

TLS = protocol version

RSA = Key exchange algorithm determining the peer authentication

3DES_EDE_CBC = bulk encryption algorithm used for data encryption

SHA-1 = Message Authentication Code which is a cryptographic hash.

• Compression Method: Contains a list of compression algorithms ordered by the client’s preference. This is optional.

2. Server response to Client

The server responds to the client with multiple messages:

Server Hello

The Server Hello contains the following information:

• Server Version: The highest TLS protocol version supported by the server which is also supported by the client.
• Server Random: 32-byte pseudorandom number used to generate the Master Secret.
• Session Identifier: Unique number to identify the session for the corresponding connection with the client.If the session ID in the client hello message is not empty, the server will find a match in the session cache. If a match is found and the server wants to use the same session state, it returns the same ID as sent by the client. If the server doesn’t want to resume the same session, then a new ID is generated. The server can also send an empty ID, indicating the session cannot be resumed.
• Cipher Suite: The single strongest cipher suite that both the server and the client support. If there is no supporting cipher suite, then a handshake failure alert is created.
• Compression Method: The compression algorithm agreed by both the server and the client. This is optional.

Server Certificate

The server sends the client a list of X.509 certificates to authenticate itself. The server’s certificate contains its public key. This certificate authentication is either done by a third party (Certificate Authority) that is trusted by the peers, the operating system and the browser which contains the list of well-known Certificate Authorities or by manually importing certificates that the user trusts.

Certificate Status

This message validates whether the server’s X.509 digital certificate is revoked or not, it is ascertained by contacting a designated OCSP (Online Certificate Status Protocol) server. The OCSP response, which is dated and signed, contains the certificate status. The client can ask the server to send the “certificate status” message which contains the OCSP response. This approach is known as OCSP Stapling. The process saves bandwidth on constrained networks as it prevents OCSP servers from getting overwhelmed with too many client requests.

The client, to indicate that it wants a certificate status message, includes an extension of the type “status_request” in the Client Hello request.

Server Key Exchange

The message is optional and sent when the public key present in the server’s certificate is not suitable for key exchange or if the cipher suite places a restriction requiring a temporary key. This key is used by the client to encrypt Client Key Exchange later in the process.

Client Certificate Request

This is optional and is sent when the server wants to authenticate the client, for e.g. a website where the server needs to confirm the client’s identity before providing access to private information. The message contains the type of certificate required and the list of acceptable Certificate Authorities.

Server Hello Done

This message indicates the server is done and is awaiting the client’s response.

3. Client response to Server

Client Key Exchange

This message contains:

• The protocol version of the client which the server verifies if it matches with the original client hello message.
• Pre-master secret – a random number generated by the client and encrypted with the server public key. This along with the client and server random number is used to create the master secret. If the server can decrypt the message using the private key and can create the master secret locally, then the client is assured that the server has authenticated itself.

Change Cipher Spec

This message notifies the server that all the future messages will be encrypted using the algorithm and keys that were just negotiated.

Finished (Encrypted Handshake)

The Finished message is complicated as it is a hash of all the messages exchanged previously along with a label (“client finished”). This message indicates that the TLS negotiation is completed for the client.

Note: Wireshark displays the Finished message as Encrypted Handshake since, unlike the previous messages, this message has been encrypted with the just negotiated keys/algorithms.

4. Server response to Client

Change Cipher Spec

The server informs the client that it the messages will be encrypted with the existing algorithms and keys. The record layer now changes its state to use the symmetric key encryption.

Finished

Like the Client Finished message but contains a different label (“server finished”). Once the client successfully decrypts and validates the message, the server is successfully authenticated.

5. Application Data Flow

Once the entire TLS Handshake is successfully completed and the peers validated, the applications on the peers can begin communicating with each other.

The article gives a brief explanation of how the TLS Protocol works and the analysis of the TLS handshake using a powerful tool like Wireshark. One important thing to note is applications should not rely on TLS to create the strongest secure connection between the peers as it is possible for a hacker to make the peers drop down to the least secure connection. Also, the use of TSL/SSL could impact performance (explained here). Hence, a clear understanding of the protocol will help evaluate its advantages and vulnerabilities.

Read the ebook Monitoring Network Protocols to learn Digital Experience Monitoring can help improve network performance.