Searching...

Matching results

Using MQTT with AirVantage

This page explains how to use the MQTT protocol with AirVantage: how to send values and timeseries data, and how to debug communications.

Introduction

Supported features

The following MQTT 3.1 features are supported on the AV Reserved topics:

• Publish / Subscribe
• QoS 0, 1 (at least one-time delivery)
• Last Will & Testament: publish a message when a client disconnects

Protocol Configuration

• Port: 1883 or 8883 (SSL/TLS)
• url: tcp://eu.airvantage.net
• Username: Serial Number (but IMEI can be used as well)
• Password: any UTF-8 character. The length is limited to 255 characters.
• ClientId: The ClientId must be unique. Use Serial Number

Topics

• Push data to AirVantage:
  • {IMEI or SN}/messages/json
• Receive AirVantage operation tasks:
  • {IMEI or SN}/tasks/json
• Acknowledge the commands received on the device:
  • {IMEI or SN}/acks/json
• CBOR-encoded messages:
  • {IMEI or SN}/messages/ts (for Time Series).
• CBOR+zlib messages:
  • {IMEI or SN}/messages/tsz

Publish data

MQTT is payload-agnostic, so you can use any format as payload. But if you want to interface with the AirVantage data store and tasks topics, you need to respect the specified formats.

Supported Data type

The supported data type are the following JSON data type: double, int, string, boolean, binary and date link.

Name	Comment
int	integer signed 32bits
double	floating-point number encoded on 64bits following IEE754 standard
string	UTF-8 text
date	date must be the number of milliseconds elapsed since 1 January 1970 GMT
boolean	true/false (0 and 1 are not supported)
binary	Base-64 encoded. Can not be used for a command parameter

Json Format 1

{ <timestamp>: { "<dataname>": <value>, "<dataname>": <value>}, ... }

• timestamp: nb of millisecond since 01/01/1970. Optional.
• dataname: full path starting with the asset-id
• value: string value must be quoted, boolean value must be true or false.

Example:

<asset default-label="My Machine" id="machine">
     <variable default-label="Temperature" path="temperature" type="double"/>
     <variable default-label="Humidity" path="humidity" type="double"/>

     <setting default-label="Threshold" path="threshold" type="int"/>
</asset>

{ "1349907137000": { "machine.temperature": 23.2, "machine.humidity": 70 },
  "1349908137000": { "machine.temperature": 24.5 },
  "1349909137000": { "machine.temperature": 22.9 },
  "1349907137000": { "machine.threshold": 30}
}

As the timestamp value is optional, the format can be:

{ "machine.temperature": 23.2, "machine.humidity": 70 }

When the server receives the values, it adds a timestamp.

Json Format 2

[{ "<dataname>": [{ "timestamp": <timestamp>, "value:" <value>}, ...], "<dataname>": ... }]

• timestamp: nb of milliseconds since 01/01/1970 (string format)
• dataname: full path starting with the asset-id
• value: string value must be quoted, boolean value must be true or false.

Example:

[{
    "machine.temperature": [{
      "timestamp" : 1416324560869,
      "value" : 42.2
    },
    {
      "timestamp" : 1416324560869,
      "value" : 24.5
    },
    {
      "timestamp" : 1416324560869,
      "value" : 42.9
    }]
  },
  {
    "machine.threshold": [{
      "timestamp" : 1416324560869,
      "value" : 30
    }]
  }]

---

MQTT Time Series Format

Introduction

This section explains the AirVantage Time Series format for MQTT. This is a compressed format dedicated to sending time-series such as large sensor collections or GPS traces.

This format is based on CBOR, an IETF standard RFC 7049 for Concise Binary Object Representation.

You can optionally add an additional layer of zlib compression.

To receive task messages from AirVantage (read, write, commands) you need to use the JSON topic.

MQTT Topics

The CBOR-encoded messages will be published on the following MQTT topic: {IMEI or SN}/messages/ts (for Time Series).

For CBOR+zlib messages: {IMEI or SN}/messages/tsz

Representation

This format will let you send a list of data for different timestamps. For example we want to send a car accelerometer X,Y,Z values together with the GPS coordinates (latitude and longitude) and the car speed.

Example:

Timestamp (ms)	X	Y	Z	Latitude	Longitude	Speed
1412320402000	0	2	0	49.455177	0.537743	100
1412320402100	0	3	0	49.455177	0.537743	98.4

Encoding

For basic compression we can use delta encoding , in order to send a smaller number. In place of sending the actual values for each new line, we will just send the delta compared to the previous line. If we look at our previous example, for the second timestamp, rather than sending 1412320402100, we will send the delta: 100.

To reduce the size of floating numbers with a fixed precision (like GPS coordinates), we can apply a factor to the samples. In place of encoding the actual floating value we will encode the integer delta.

To reduce the size of the timestamp, we will remove the last two zeros because we know we are always sampling every 100ms.

CBOR is based on the successful JSON data model: numbers, strings, arrays and maps.

We can represent CBOR object using JSON, but at the end it will be binary encoded on the wire.

The payload for the message is a CBOR map with three entries:

• “h” the list of data path, in our case [“X”,“Y”,“Z”,“Latitude”,“Longitude”,“Speed”] the first column is always timestamps so we don’t need to specify it in this list
• “f” the list of factors to apply, in our case [0.01,1,1,1,1000000,1000000,1] the first value is the factor to apply to timestamps
• “s” the list of samples, we will place the delta encoded values here, in a big list

As a JSON representation:

  {
      "h" : [             "x", "y", "z", "lat",    "long",   , "speed" ],
  
      "f" : [ 0.01,        1,   1 ,  1,   1000000,  1000000,   1       ],
  
      "s" : [ 14123204020, 0,   2,   0,   49455177, 537743,    100,
              1,           0,   1,   0,   0,        0,         -1.6    ]
  }

As a binary CBOR encoded buffer:

a361688661786179617a636c6174646c6f6e67657370656564616687fb3f847a
e147ae147b0101011a000f42401a000f42400161738e1b0000000349cefdb400
02001a02f2a0491a0008348f1864010001000000fbbff999999999999a

You can easily verify your CBOR-encoded message using cbor.me

Extra compression

If your embedded target allows, you can add another optional layer of compression using zlib according to the RFC1950 standard. You can use the popular zlib compression library.

Helpful links

• cbor.io : for an introduction to CBOR and also a list of implementation in various languages (C, Java, Python, Go, ..)
• cn-cbor A constrained node implementation of CBOR in C
• zlib.net : the homepage of one of the most popular zlib implementations in C
• miniz Single C file implementations of zlib

---

Receive Tasks

Read or Write Tasks

[
   {
      "uid" : "<uid>",
      "timestamp" : <timestamp>,
      "read" : [ "<dataname>", ... ]
      OR
      "write" : [ { "<dataname>" : <value> }, ... ]
    }
]

• uid: job universal identifier
• timestamp: nb of milliseconds since 01/01/1970 (string format)
• dataname: full path with asset-id to identify the setting
• value: the new value of the setting. See section “Supported data types”.

You can send either a “read” or a “write” task, but not both at the same time.

A single task can be sent at a given time.

Example:

[
  {
    "uid" : "3c12547b613740adb686271bdc8f097c",
    "timestamp" : 1348836320188,
    "read" : ["machine.temperature"]
  }
]

or

[
  {
    "uid": "8006cc58ba2141f69161a78f1bfdea1d",
    "timestamp": 1348836320566,
    "write" : [{"machine.threshold" : 25}]
  }
]

Customer command Tasks

[
   {
      "uid" : "<uid>",
      "timestamp" : <timestamp>,
      "command" : {"id": "<command-id",
                   "params": {"<param-name>":"<param-value", ...}
                      }
    }
]

• uid:job universal identifier
• timestamp: nb of milliseconds since 01/01/1970 (string format)
• command-id: full path with asset-id to identify the command.
• param-name: name of the parameter.
• param-value: value of the parameter

A single command can be sent at a given time.

Example:

<asset default-label="My Machine" id="machine">
     <command path="sendMessage" default-label="send Message">
          <parameter id="message" default-label="Message" type="string" />
     </command>
</asset>

[
   {
      "uid" : "e87b35c3c2e2417b902277ff9d049d70",
      "timestamp" : 1416324560869,
      "command" : {"id": "machine.sendMessage",
                   "params": {"message":"Hello World!"}
                      }
    },
    ...
]

Acknowledging tasks

Once you process a task from AirVantage you should acknowledge it. You can provide a success or error acknowledgement (with an optional error message).

[
   {
      "uid" : "<uid of the acknownleged task>",
      "status" : "<OK or ERROR>",
      "message" : "Failure details"
    },
    ...
]

A failure message is only needed when the status is ERROR. A timestamp is optional.

Example:

[
    {"uid": "8006cc58ba2141f69161a78f1bfdea1d", "status" : "OK"},
    {"uid": "3c12547b613740adb686271bdc8f097c", "status" : "KO", "message" : "Crashed in flames! Error code: 12345"}
]

---

Troubleshooting communications

This section explains how to receive trace logs of the data you are sending by subscribing to the MQTT Errors topic.

When using the MQTT protocol to connect an embedded application to AirVantage, developers can subscribe to the errors MQTT topic, to get notified when a message contains an invalid payload.

Use a different clientId to connect to AirVantage using your device’s username (serial number or IMEI) and password.

Monitor JSON errors

Suppose you publish a message on topic {IMEI or SN}/messages/json with an invalid JSON format:

// Invalid JSON - timestamp should be a string
{ 12346013564 : { "temperature" : 42 } }

The JSON payload is invalid because the timestamp is expected to be a JSON string (formats explained above).

You can subscribe to errors to receive a message, with a payload containing 3 lines in text format (separated by \n):

• the topic on which the initial message was published
• the payload of the message
• a diagnostic message describing the error

With the previous request, you would receive:

{IMEI or SN}/messages/json
{ 12346013564 : { "temperature" : 42 } }
invalid character '1' looking for beginning of object key string

Monitor CBOR errors

Suppose you publish on {IMEI or SN}/messages/ts the CBOR corresponding to the following JSON message:

    // Invalid CBOR, a timestamp is missing in the sample
  {
      "h" : [             "x", "y", "z", "lat",    "long",   , "speed" ],

      "f" : [ 0.01,        1,   1 ,  1,   1000000,  1000000,   1       ],

      "s" : [ 0,   2,   0,   49455177, 537743,    100 ]
  }

Time Series format is explained earlier on this page.

You can subscribe to errors to receive a message, with a payload containing 3 lines of text (separated by \n):

• the topic on which the message was published
• the payload of the message (encoded as UTF-8)
• a diagnostic message describing the error

With the previous request, you would receive:

{IMEI or SN}/messages/tz
[Encoded payload]
Not enough CBOR samples at line 0 ; found 6, expected 7. 
See https://doc.airvantage.net/av/reference/hardware/protocols/mqtt-ts/

---

More on MQTT

MQTT.org

Eclipse Paho

MQTT standard at OASIS

Understand the MQTT protocol

hiveMQ

Getting started with MQTT

Tutorials and code samples

• Using MQTT with BeagleBone Black in Javascript, refer to tutorial .
• Using MQTT with an raspberry Pi in c, refer to tutorial .
• Using MQTT with an HL6 in ruby, refer to tutorial .
• Using MQTT with a SL808xT in c, refer to tutorial .
• Using MQTT with GX440 in lua, see this tutorial .
• Simulate a single system communicating with AirVantage via MQTT tutorial
• If you don’t find you device in this list, refer to this tutorial .
• You can also learn how to simulate a single system communicating with AirVantage via MQTT via the MQTTLens Chrome application, using this tutorial .