Tags

By year

  1. 2019 (1)
  2. 2018 (2)
  3. 2017 (4)
  4. 2015 (9)

Inspecting the Retransmission Behavior of the Mosquitto MQTT Broker with GDB

GDB Debugging MQTT Mosquitto Network Protocol

Posted on Oct 6


I just wanted to actually see how the Mosquitto MQTT broker retransmit packets when a communication is interrupted and confirm the effect of the retry_interval config variable. Steps are as follows.

Environment

  • Linux 3.13.0-48-generic Ubuntu SMP 86_64 GNU/Linux

Building Mosquitto

Installing Required Packages

$ sudo apt-get install build-essential
$ sudo apt-get install git
$ sudo apt-get install gdb

Configuring and Building

$ cd $DIR_TO_HOST_MOSQUITTO_CLONE
$ mkdir mosquitto
$ git clone http://git.eclipse.org/gitroot/mosquitto/org.eclipse.mosquitto.git mosquitto
$ cd mosquitto
# Read build requirements and steps.
$ less compiling.txt
# Disable some features to reduce dependencies.
$ cp -v config.mk config.mk.orig
$ sed -e 's/^\(WITH_TLS:=\).*$/\1=no/' \
    -e 's/^\(WITH_TLS_PSK:=\).*/\1=no/' \
    -e 's/^\(WITH_SRV:=\).*/\1=no/' \
    -e 's/^\(WITH_UUID:=\).*/\1=no/' \
    config.mk.orig > config.mk
# Build.
$ make

Testing Mosquitto

$ export LD_LIBRARY_PATH=lib
$ sudo ldconfig
# Launch the broker
$ src/mosquitto -v &
# Sending PUBLISH message to the broker with topic being "topic" and QoS being 2.
#    -h hostname-to-connect
#    -q QoS-value
$ client/mosquitto_pub -h localhost -t topic -m message -q 2

Output

xxxxxxxxxx: New connection from 127.0.0.1 on port 1883.
xxxxxxxxxx: New client connected from 127.0.0.1 as mosqpub/xxx (c1, k60).
xxxxxxxxxx: Sending CONNACK to mosqpub/xxx (0, 0)
xxxxxxxxxx: Received PUBLISH from mosqpub/xxx (d0, q2, r0, m1, 'topic', ... (7 bytes))
xxxxxxxxxx: Sending PUBREC to mosqpub/xxx (Mid: 1)
xxxxxxxxxx: Received PUBREL from mosqpub/xxx (Mid: 1)
xxxxxxxxxx: Sending PUBCOMP to mosqpub/xxx (Mid: 1)
xxxxxxxxxx: Received DISCONNECT from mosqpub/xxx
xxxxxxxxxx: Client mosqpub/xxx disconnected.

Testing Retransmission using GDB

Assuming that the mosquitto broker is running background at port 1883 on localhost, we publish a retained message, launch a subscriber to receive that message and then interrupt the conversation with GDB to see the broker resend the same message.

Publishing a Retained Message

The following command publishes a message of topic "topic" with QoS 2 and makes the broker retain it for future subscribers.

# The -r option makes this message retained on the server.
$ client/mosquitto_pub -h localhost -t topic -m message -q 2 -r

Launching a Subscriber in GDB session

$ gdb client/mosquitto_sub
(gdb) # Stop the process from responding to PUBREL messages.
(gdb) b _mosquitto_handle_pubrel
(gdb) # Run the subscriber with debugging output enabled (-d) and client ID being "c1".
(gdb) run -d -h localhost -t topic -q 2 -i c1
Starting program: client/mosquitto_sub -d -h localhost -t topic -q 2 -i c1
xxxxxxxxxx: New connection from 127.0.0.1 on port 1883.
Client c1 sending CONNECT
xxxxxxxxxx: New client connected from 127.0.0.1 as c1 (c1, k60).
xxxxxxxxxx: Sending CONNACK to c1 (0, 0)
Client c1 received CONNACK
Client c1 sending SUBSCRIBE (Mid: 1, Topic: topic, QoS: 2)
xxxxxxxxxx: Received SUBSCRIBE from c1
xxxxxxxxxx:     topic (QoS 2)
xxxxxxxxxx: c1 2 topic
xxxxxxxxxx: Sending SUBACK to c1
xxxxxxxxxx: Sending PUBLISH to c1 (d0, q2, r1, m1, 'topic', ... (7 bytes))
Client c1 received SUBACK
Subscribed (mid: 1): 2
Client c1 received PUBLISH (d0, q2, r1, m1, 'topic', ... (7 bytes))
Client c1 sending PUBREC (Mid: 1)
xxxxxxxxxx: Received PUBREC from c1 (Mid: 1)
xxxxxxxxxx: Sending PUBREL to c1 (Mid: 1)

Breakpoint 1, _mosquitto_handle_pubrel (db=0x0, mosq=0x6070c0)
    at read_handle_shared.c:119

# Wait for 20 seconds ... and the server retransmit the PUBREL message.
xxxxxxxxxx: Sending PUBREL to c1 (Mid: 1)

# If we keep waiting here, the server retransmit the same message repeatedly.
# The retransmission interval can be set with `retry_interval` in a config file.
xxxxxxxxxx: Sending PUBREL to c1 (Mid: 1)
xxxxxxxxxx: Sending PUBREL to c1 (Mid: 1)

# Run the c (continue) command to make the process respond to the first PUBREL.
(gdb) c

# The breakpoint seems to hit as many times as the server has retransmitted.
...

Changing retry_interval

The config file can be supplied to the broker with the -c option.

# Here we use the default config file in the workspace.
# Change the retry_interval variable.
$ vim mosquitto.conf
# Launch the broker.
$ src/mosquitto -v -c mosquitto.conf

2015 My gh-pages