Dissecting MQTT using Wireshark

We discussed the MQTT protocol, how it works, and it’s growing role in the digital landscape in a previous blog post. To better understand the content in this article, read up on the basics of Machine to Machine (M2M) in the previous post. We will be using the Catchpoint MQTT monitor to subscribe and publish to a topic from an MQTT broker. The image below illustrates the test set up using Catchpoint.

The metrics from Catchpoint’s MQTT monitor includes publish time, publish size, subscribe time, subscribe size, DNS, and connect time.

To interpret the data collected using the MQTT monitor, you need to see what happens under the hood and this can be done using Wireshark. Sniffing tools such as Wireshark allow us to dissect each step in the process and examine the data packets. In this analysis, we will use the Catchpoint MQTT monitor to set up a sample test and see what is involved in the process.

Wireshark Analysis

MQTT protocol is based on top of TCP/IP and both the client and broker need to have a TCP/IP stack.

Below is the snapshot of a subscribe followed by a publish to the same MQTT broker using Wireshark tool from the Catchpoint MQTT monitor. We will be considering a message with the delivery level of QoS t0 here. Let’s dive into a packet-by-packet analysis of the protocol.

Note: In the image above, the IP 192.168.0.11 is the IP of the MQTT broker and the client is 192.168.0.12 that is acting as both the publishing client and the subscribing client.

1. Connect Message

MQTT connection is between a client and a broker and never directly with another client. The initiation of this connection is using a CONNECT command sent from the client to the broker. The connection, once established, is kept open until the time it does not receive a disconnect command from the client.

The destination port is 1883, which is the default port for MQTT over TCP.

Port 8883 is for MQTT over TLS.

Diving into connect command details:

• Header Flags: Holds information on the MQTT control packet type.
• Connect Flags: The connect flag bytes contains parameters specifying the behavior of the MQTT connection. It denotes the presence or absence of fields in the payload.
• Clean session: Bit 1 of the connection flags. This flag indicates the broker whether the client wants to establish a persistent connection or not. The flag when set to true results in a clean session where the subscriptions are removed on disconnection and when set to false, a durable connection can be achieved where subscriptions remain and messages with high QoS are delivered on reconnection.
• Will flag: Bit 2 of the connect flags. Part of the last will and testament (LWT) feature of MQTT. This flag when set means, if the Connect request is accepted, a Will message must be stored on the server. A Will message is an MQTT message with a Will Topic and a Will Message. This is used while notifying other clients on disconnection. When the client disconnects, the broker sends this message on behalf of it. When the Will flag is set to 1, the Will QoS and Will Retain fields in the Connect flags is used by the server.
• Will QoS: Bit 4 and 3 of the connect flags. Denote the QoS level to be used while publishing the Will message.
• Will retain: Bit 5 of the connect flags. If Will Retain is set to 0, the server must publish the Will message as a non-retained message and when it is set to 1, the Will message is published as a retained message.
• User Name and Password: Bit 7 and 6 of the connect flags respectively. When these fields are set, the credentials are expected in the payload. MQTT allows sending username and password for authenticating a client and for authorization. The password is sent in plaintext if it is not encrypted underneath.
• Keep alive: Keep alive timer is used to know if an MQTT client is on the network where the client sends regular PING request messages to the broker. Broker responds with a PING response.
• Client ID: Is an identifier for each MQTT client connecting to an MQTT broker. It needs to be unique per broker.
• Payload: The payload contains Client ID, Will Topic, Will Message, User Name, and Password fields whose presence is determined by the flags.

2. Connect Acknowledgment Message

On receiving a CONNECT message, the broker responds with a CONNACK message.

• Header Flags: Holds information on the MQTT control packet type.
• Session Present: Bit 0 of the Connection Ack byte is the session present flag. This flag indicates if the broker has an existing persistent session of the client from previous interactions.
• Return Code: Return code values and response in the table below

• Payload: The CONNACK packet has no payload.

3. Subscribe Message

A client sends a SUBSCRIBE message to an MQTT broker to receive relevant messages.

• Header Flags: Holds information on the MQTT control packet type.
• Message Identifier: Identifier between a client and a broker to identify a message in a message flow. This is more relevant for QoS greater than zero.
• Topic and QoS Level: Each subscription is a pair of a topic filter and QoS level. The topic is the subject of interest that the client would like to get messages on.
• Payload: The payload contains a list of subscriptions. A payload is mandatory in a SUBSCRIBE packet.

4. Subscribe Acknowledgement Message

The MQTT broker confirms subscription by sending an acknowledgment back to the client using a SUBACK message.

• Header Flags: Holds information on the MQTT control packet type.
• Message Identifier: Relevant for a message with QoS greater than zero and is the same as in the SUBSCRIBE message.
• Return Code: MQTT broker sends a return code for each Topic/QoS pair received in the SUBSCRIBE message. Return code is in line with the QoS level in case of success. Return code values and response in the table below.

• Payload: The payload contains a list of return codes.

5. Publish Message

Once an MQTT client is connected to the broker, it can publish messages.

• Header Flags: Holds information on the MQTT control packet type.
• DUP flag: When DUP flag is 0, it means this the first attempt at sending this PUBLISH packet. If the flag is 1, it indicates a possible re-attempt at sending the message.
• QoS: The QoS level determines the assurance level of a message.
• Retain Flag: If retain flag is set to 1, the server must store the message and its QoS, so that it can cater to future subscriptions matching the topic. When a PUBLISH packet is sent to a subscribing client, the server must set the retain flag to 1 if the packet is sent because of a new subscription. The server must set the retain flag to 0 when the packet is sent because it matches an established subscription irrespective of whether the flag was set when the message was received.
• Topic Name: A UTF-8 string which can also contain forward slashes when it needs to be hierarchically structured. A publish message must contain a topic which is used by the broker for a subject based filtering. This way the broker will send messages to clients that have subscribed to the topic.
• Message: Message is the payload along with the topic which contains the actual data to be transmitted. Since MQTT is data agnostic, the payload can be structured based on the use case.
• Payload: The payload contains the message that is being published.

6. Disconnect Request Message:

Disconnect message is the final control packet sent by the client to the broker. Means a clean disconnection by the client.

• Header Flags: Holds information on the MQTT control packet type.
• Payload: The disconnect packet has no payload.

7. MQTT Keep Alive

Keep alive functionality assures that connection is open and the client and the broker are connected to each other. When the connection is established the keep-alive time interval (in seconds) is communicated to the broker by the client.

MQTT specification says,

“It is the responsibility of the Client to ensure that the interval between control packets being sent does not exceed the Keep Alive value. In the absence of sending any other control packets, the client must send a PINGREQ packet.”

The Keep Alive flow is maintained by PINGREQ and PINGRESP messages.

A typical keep alive flow looks like the image below

PINGREQ

The PINGREQ is sent by the client to the broker to denote that it is still alive though it has not sent other MQTT control packets. If the broker does not receive a PINGREQ or any other packet, it will close the connection and send out the LWT message, provided the client has specified one.

• Header Flags: Holds information on the MQTT control packet type.
• Payload: The PINGREQ packet has no payload.

PINGRESP

The PINGRESP is the reply that the broker sends to the PINGREQ packet received. This indicates the broker’s availability to the client.

• Header Flags: Holds information on the MQTT control packet type.
• Payload: The PINGRESP packet has no payload.

Conclusion

MQTT has spurred innovation in the IoT landscape and has become an integral part of the digital world. The previous blog post on MQTT gave us an overview of the protocol and in this article, we discussed the different processes involved in the communication between an MQTT client and MQTT broker with the help of Wireshark. Understanding these metrics and the workflow, will help you quickly identify MQTT related issues.