欢迎使用 ESP32-ThingsBoard-MQTT-Client SDK

ESP32-ThingsBoard-MQTT-Client SDK 适用于 C/C++ 开发人员。 此库提供了一些简单的 API 来使用 MQTT API 与 ThingsBoard 平台进行通信。

它被封装成一个ESP-IDF组件,既可以在ESP-IDF和ESP-ADF下使用,也可以很简单移植到Arduino-ESP32。

当前客户端版本基于 ESP-IDF-v4.4.x,兼容 ThingsBoard 3.4.x 及更新版本。

Contents

Usage

Installation

ThingsBoard Overview

TODO: IoT/ThingsBoard 整体架构;简单介绍各个业务功能;简单介绍各组API;两者的对应;协议的层次及图示。

Reprinted articles: https://thingsboard.io/docs/

What is ThingsBoard?

See What is ThingsBoard?

Architecture Overview

_images/367957f14d57b99a38e6876de96492f49bca148fcae9ad4cad513e4da4b776e9.svg

Key concepts & Features

  • Time-series data

  • Attribures

  • Remote commands to devices

  • Claiming Devices

  • Provisoin * Device provisioning * Bulk Provisioning???

  • OTA updates

Data Visualization

ThingsBoard allows you to configure customizable IoT dashboards. Each IoT Dashboard may contain multiple dashboard widgets that visualize data from multiple IoT devices. Once IoT Dashboard is created, you may assign it to one of the customers of you IoT project.

IoT Dashboards are light-weight and you may have millions of dashboards. For example, you may automatically create a dashboard for each new customer based on data from registered customer IoT devices. Or you may modify dashboard via script when a new device is assigned to a customer. All these actions may be done manually or automated via REST API.

You can find useful links to get started below:

  • Dashboards

  • Widgets Library
    • Digital and analog gauges for latest real-time values visualization

    • Highly customizable Bar and Line charts for visualization of historical and sliding-window data points

    • Map widgets for tracking movement and latest positions of IoT devices on Google or OpenStreet maps.

    • GPIO control widgets that allow sending GPIO toggle commands to devices.

    • Card widgets to enhance your dashboards with flexible HTML labels based on static content or latest telemetry values from IoT devices.

Getting Started Guides

These guides provide quick overview of main ThingsBoard features. Designed to be completed in 15-30 minutes.

  • Hello world : Learn how to collect IoT device data using MQTT, HTTP or CoAP and visualize it on a simple dashboard. Provides variety of sample scripts that you can run on your PC or laptop to simulate the device.

  • End user IoT dashboards : Learn how to perform basic operations over Devices, Customers, and Dashboards.

  • Device data management : Learn how to perform basic operations over device attributes to implement practical device management use cases.

Telemetry - Time-series data

遥测(Telemetry)系统的目标, 就是获取时序数据(Time-series data), 进而对其进行监测、处理、分析等。

概念 Concept

时序维度(Time-series axis, Time-series dimension)

被监测的目标。一个装置可以有多个 Time-series axis, 例如, 室内温度、湿度。

采样频率 ———— 采样周期

Time-series axis 的采集周期。

时间戳 Timestamp

采样的时刻, Unix timestamp格式, 精确到毫秒。

小技巧

  • Timestamp 可以由本地装置产生,也可由服务器产生(接收的时刻).

数据点 DataPoints

Time-series axis 每被采集一次,就获得一个 Data Point。

小技巧

  • 一次上传, 可以包括不同 Time-series axis 的多个 DataPoint.

  • 每个 DataPoint 都会在 Server 上保存,直至超过期限。

小技巧

  • Time-series data 对 Server 的网络带宽, CPU 处理能力和磁盘存储空间影响巨大。

  • 需要重点监控的数据,才以 Time-series data 的形式上传。

  • 根据实际需要,确定 Time-series axis 的采样频率,不可太短也不可太长。太短会消耗 Server 的带宽与处理能力, 并在Server形成大量冗余数据, 太长会造成 Server 无法针对 Time-series axis 做出实时响应。例如, 若烟感每10分钟采集一次到数据并上报, 会造成软大延时。

  • Time-series data 以 json 数据格式上传,故其 key 不应太长,会消耗带宽; 若 key 类似密文或数字,也会带来维护难度。

Working with telemetry data

See Working with telemetry data.

_images/c6800c90061a9567231ee58bd15482ffe883b48ca48e4530fc8f13d7c2704457.svg

Data points

Improvement

  • 把 Alarm 机制由 Server 移到 Client/Deivce 端; 或者在 Client/Deivce 端加一个新的较弱 Alarm 机制; 或者只是为 Time-series data 加几个属性, 告诉 Client/Device 该如何更好的上传 Time-series data:

    • 最小值 Alram: 低于该值即刻upload

    • 最大值 Alarm: 高于该值即刻upload

    • 上一次 Upload 后的变化值 Delta: 差值大于该值即刻upload

    • 优势:

      • 减少数据冗余

      • 提升实时响应能力

    • 劣势:

      • 增加 Client/Deivce 端的复杂度

Device Attributes

Working with IoT device attributes

See Working with IoT device attributes.

Attributes are treated key-value pairs. Flexibility and simplicity of the key-value format allow easy and seamless integration with almost any IoT device on the market.

Device specific attributes are separated into two main groups:

  • client-side attributes - attributes are reported and managed by the device application. For example current software/firmware version, hardware specification, etc.

  • shared attributes - attributes are reported and managed by the server-side application. Visible to the device application. For example customer subscription plan, target software/firmware version.

_images/138c4286b0dfd52f6ca1b842e1d1930fa247bf6ab28aa6369cde4cd20a5ad318.svg

TODO: move to api!!!!

Publish attribute update to the server

Request attribute values from the server

Subscribe to attribute updates from the server

RPC

Using RPC capabilities

See Using RPC capabilities.

Thinsboard RPC feature can be divided into two types based on originator: device-originated and server-originated RPC calls. In order to use more familiar names, we will name device-originated RPC calls as a client-side RPC calls and server-originated RPC calls as server-side RPC calls.

Client-side RPC
_images/f6c1cab2136504ab9f579dd0285989b2d92feb4a16c0f589f95e2243a0e10a0a.svg
Server-side RPC

Server-side RPC calls can be divided into one-way and two-way:

  • One-way server-side RPC request is sent to the device without delivery confirmation and obviously, does not provide any response from the device. RPC call may fail only if there is no active connection with the target device within a configurable timeout period.

    _images/cb0452fe943458769fa58056e667aba0daa48c023328511b42324d9d07da0fad.svg
  • Two-way server-side RPC request is sent to the device and expects to receive a response from the device within the certain timeout. The Server-side request is blocked until the target device replies to the request.

    _images/1998ca311f6142613e531914cd22b472dfeed3f74e0ca62d6bc3f8eb361005ef.svg

Claiming Devies

Claiming devices

See Claiming devices.

TODO: Claiming devices.

[MQTT Device API Reference - Claiming devices](https://thingsboard.io/docs/reference/mqtt-api/#claiming-devices)

[Other Features - Claiming devices](https://thingsboard.io/docs/user-guide/claiming-devices/?claimingscenario=deviceside#use-case-description)

OTA updates

Refer to MQTT Device API Reference - Firmware API.

Tips: In this article, client refers to device.

Firmware OTA updates

Tips: Firmware attributes are some shared attributes.

Request firmware attributes values from the server
_images/c95f310cc919cc0da5a625fbc3b5570874988d59ba930bb30146ced0bed9fb46.svg
  1. Prerequisites
    • Control Packet type: SUBSCRIBE (Client subscribe request)

    • Direction of flow: Client to Server

    • Payload-Topic Filter: ???

  2. Send request of firmware attributes

  3. Handle response of firmware attributes

Subscribe to firmware attributes updates from the server
_images/c95f310cc919cc0da5a625fbc3b5570874988d59ba930bb30146ced0bed9fb46.svg
  1. Prerequisites
    • Control Packet type: SUBSCRIBE (Client subscribe request)

    • Direction of flow: Client to Server

    • Payload-Topic Filter: firmware attributes updates

  2. Handle PUBLISH message of firmware attributes updates
    • Control Packet type: SUBSCRIBE (Client subscribe request)

    • Direction of flow: Server to Client

    • Payload-Topic Filter: firmware attributes updates

Request firmware chunk from the server
  1. Prerequisites
    • Control Packet type: SUBSCRIBE (Client subscribe request)

    • Direction of flow: Client to Server

    • Payload-Topic Filter: ???

  2. Send request of firmware chunk

  3. Handle response of firmware chunk

  4. Send telemetry of current firmware status

Software OTA updates

所有应用于Firmware的操作, 也可以应用于 Software。相关的协议, 把 fw_… 替换成 sw_… 即可。

ThingsBoard Operation

  • Device 的 OTA updates 覆盖 Device profile 的 OTA updates.

References

  1. Firmware API - ThingsBoard.

Provision

delay 9 times : 0, 1, 2, 4, 8, 16, 32, 64, 128 senconds

_images/0912ac93456889985d8a58beb8ca9e2a6944c176c5392ebe44869253ec2223c2.svg

ThingsBoard MQTT API reference

Introduction

See ThingsBoard API reference.

ThingsBoard API consists of two main parts: device API and server-side API.

_images/c1bb6839805ea38f74c85698f7f0d2aa3e247a8a647d4a3ea4af80575cfc5958.svg

MQTT API

See MQTT Device API Reference.

Getting started
MQTT basics

MQTT is a lightweight publish-subscribe messaging protocol which probably makes it the most suitable for various IoT devices. You can find more information about MQTT here.

ThingsBoard server nodes act as an MQTT Broker that supports QoS levels 0 (at most once) and 1 (at least once) and a set of predefined topics.

Client libraries setup

You can find a large number of MQTT client libraries on the web. Examples in this article will be based on Mosquitto and MQTT.js. In order to setup one of those tools, you can use instructions in our Hello World guide.

MQTT Connect

We will use access token device credentials in this article and they will be referred to later as $ACCESS_TOKEN. The application needs to send MQTT CONNECT message with username that contains $ACCESS_TOKEN. Possible return codes and their reasons during connect sequence:

  • 0x00 Connected - Successfully connected to ThingsBoard MQTT server.

  • 0x04 Connection Refused, bad user name or password - Username is empty.

  • 0x05 Connection Refused, not authorized - Username contains invalid $ACCESS_TOKEN.

Key-value format

By default, ThingsBoard supports key-value content in JSON. Key is always a string, while value can be either string, boolean, double, long or JSON. Using custom binary format or some serialization framework is also possible. See Protocol customization for more details. For example:

{
   "stringKey":"value1",
   "booleanKey":true,
   "doubleKey":42.0,
   "longKey":73,
   "jsonKey": {
      "someNumber": 42,
      "someArray": [1,2,3],
      "someNestedObject": {"key": "value"}
   }
}
JSON value support

TODO: …

Protocol customization

MQTT transport can be fully customized for specific use-case by changing the corresponding module.

Device MQTT Topic
Device MQTT Topic

Function Topic

Subscribe

Tx

Rx

Telemetry

① v1/devices/me/telemetry

Request attributes

① v1/devices/me/attributes/response/+

② v1/devices/me/attributes/request/$request_id

③ v1/devices/me/attributes/response/$request_id

Publish attributes

① v1/devices/me/attributes

Subscribe attributes update

① v1/devices/me/attributes

② v1/devices/me/attributes

Server-Side RPC

① v1/devices/me/rpc/request/+

③ v1/devices/me/rpc/response/$request_id

② v1/devices/me/rpc/request/$request_id

Client-Side RPC

① v1/devices/me/rpc/response/+

② v1/devices/me/rpc/request/$request_id

③ v1/devices/me/rpc/response/$request_id

Claiming device

① v1/devices/me/claim

Note: ①②③ The order in which topics are performed.

Telemetry upload API

## 前提

### tb是什么

### tb的架构

### xxx是什么

## flow chart

### chart

### description

### note

## msg

### note xxx

### example

## tips

  • key的长度

  • msg的周期

  • 可以有几个telemetry消息

## Next steps

## Relation documents

### Client API

所有功能/通信协的描述,都采用下面的模板:

1. 功能描述 Function description 1. 流程图 Flow chart 1. 编程接口参考 API reference 1. 评论 Review 1. 操作流程 Operation 1. 客户端示例 Client examples

1. Python example Scripts 1. TBMQTH example

  1. 参考 References

Flow Chart

_images/35f6ff74327b801d99b7cbb725eb38cf94f101fef13269aa848dc2d4c948685a.svg

In order to publish telemetry data to ThingsBoard server node, send PUBLISH message to the following topic:

v1/devices/me/telemetry

The simplest supported data formats are:

{"key1":"value1", "key2":"value2"}

or

[{"key1":"value1"}, {"key2":"value2"}]

Please note that in this case, the server-side timestamp will be assigned to uploaded data!

In case your device is able to get the client-side timestamp, you can use following format:

{"ts":1451649600512, "values":{"key1":"value1", "key2":"value2"}}

In the example above, we assume that “1451649600512” is a unix timestamp with milliseconds precision. For example, the value ‘1451649600512’ corresponds to ‘Fri, 01 Jan 2016 12:00:00.512 GMT’

Example

Client library

Shell file

JSON file

Mosquitto

mosquitto-telemetry.sh

MQTT.js

mqtt-js-telemetry.sh

mosquitto-telemetry.sh
# Publish data as an object without timestamp (server-side timestamp will be used)
mosquitto_pub -d -h "127.0.0.1" -t "v1/devices/me/telemetry" -u "$ACCESS_TOKEN" -f "telemetry-data-as-object.json"
# Publish data as an array of objects without timestamp (server-side timestamp will be used)
mosquitto_pub -d -h "127.0.0.1" -t "v1/devices/me/telemetry" -u "$ACCESS_TOKEN" -f "telemetry-data-as-array.json"
# Publish data as an object with timestamp (server-side timestamp will be used)
mosquitto_pub -d -h "127.0.0.1" -t "v1/devices/me/telemetry" -u "$ACCESS_TOKEN" -f "telemetry-data-with-ts.json"
mqtt-js-telemetry.sh
# Publish data as an object without timestamp (server-side timestamp will be used)
cat telemetry-data-as-object.json | mqtt pub -v -h "127.0.0.1" -t "v1/devices/me/telemetry" -u '$ACCESS_TOKEN' -s
# Publish data as an array of objects without timestamp (server-side timestamp will be used)
cat telemetry-data-as-array.json | mqtt pub -v -h "127.0.0.1" -t "v1/devices/me/telemetry" -u '$ACCESS_TOKEN' -s
# Publish data as an object with timestamp (server-side timestamp will be used)
cat telemetry-data-with-ts.json | mqtt pub -v -h "127.0.0.1" -t "v1/devices/me/telemetry" -u '$ACCESS_TOKEN' -s
telemetry-data-as-object.json
{
   "stringKey": "value1",
   "booleanKey": true,
   "doubleKey": 42.0,
   "longKey": 73,
   "jsonKey": {
      "someNumber": 42,
      "someArray": [1,2,3],
      "someNestedObject": {"key": "value"}
   }
}
telemetry-data-as-array.json
[{"key1":"value1"}, {"key2":true}]
telemetry-data-with-ts.json
{
   "ts": 1451649600512,
   "values": {
      "stringKey": "value1",
      "booleanKey": true,
      "doubleKey": 42.0,
      "longKey": 73,
      "jsonKey": {
         "someNumber": 42,
         "someArray": [1, 2, 3],
         "someNestedObject": {
         "key": "value"
         }
      }
   }
}

Attributes API

ThingsBoard attributes API allows devices to

  • Request client-side and shared device attributes from the server.

  • Upload client-side device attributes to the server.

  • Subscribe to shared device attributes from the server.

Request attribute values from the server

_images/1f362cd658b3bd8b94fa8f570f1e6f8796648dbe38aaee3f10aca892a5e2cf2a.svg

Before sending PUBLISH message with the attributes request, client need to subscribe to:

v1/devices/me/attributes/response/+

Once subscribed, the client may request client-side or shared device attributes to ThingsBoard server node, send PUBLISH message to the following topic:

v1/devices/me/attributes/request/$request_id

where $request_id is your integer request identifier.

The client should receive the response to the following topic:

v1/devices/me/attributes/response/$request_id
Example

The following example is written in javascript and is based on mqtt.js. Pure command-line examples are not available because subscribe and publish need to happen in the same mqtt session.

Client library

Shell file

JavaScript file

Result (JSON file)

MQTT.js

mqtt-js-attributes-request.sh

mqtt-js-attributes-request.js

attributes-response.json

mqtt-js-attributes-request.sh
export TOKEN=$ACCESS_TOKEN
node mqtt-js-attributes-request.js
mqtt-js-attributes-request.js
var mqtt = require('mqtt')
var client  = mqtt.connect('mqtt://127.0.0.1',{
   username: process.env.TOKEN
})

client.on('connect', function () {
   console.log('connected')
   client.subscribe('v1/devices/me/attributes/response/+')
   client.publish('v1/devices/me/attributes/request/1', '{"clientKeys":"attribute1,attribute2", "sharedKeys":"shared1,shared2"}')
})

client.on('message', function (topic, message) {
   console.log('response.topic: ' + topic)
   console.log('response.body: ' + message.toString())
   client.end()
})
attributes-response.json
{"key1":"value1"}

Please note, the intersection of client-side and shared device attribute keys is a bad practice! However, it is still possible to have same keys for client, shared or even server-side attributes.

Publish attribute update to the server

_images/6f68ecb5dfceec90754ce65ec6e685129b1da784bdd6d4b37942b9e0d4b6667d.svg

In order to publish client-side device attributes to ThingsBoard server node, send PUBLISH message to the following topic:

v1/devices/me/attributes
Example

Client library

Shell file

JSON file

Mosquitto

mosquitto-attributes-publish.sh

new-attributes-values.json

MQTT.js

mqtt-js-attributes-publish.sh

mosquitto-attributes-publish.sh
# Publish client-side attributes update
mosquitto_pub -d -h "127.0.0.1" -t "v1/devices/me/attributes" -u "$ACCESS_TOKEN" -f "new-attributes-values.json"
mqtt-js-attributes-publish.sh
# Publish client-side attributes update
cat new-attributes-values.json | mqtt pub -d -h "127.0.0.1" -t "v1/devices/me/attributes" -u '$ACCESS_TOKEN' -s
new-attributes-values.json
{
   "stringKey": "value1",
   "booleanKey": true,
   "doubleKey": 42.0,
   "longKey": 73,
   "jsonKey": {
      "someNumber": 42,
      "someArray": [1,2,3],
      "someNestedObject": {"key": "value"}
   }
}

Subscribe to attribute updates from the server

_images/8965056795e4787b33b45e7bd7841222f9efce0bb1f829f3f4f599614c3ef9e6.svg

In order to subscribe to shared device attribute changes, send SUBSCRIBE message to the following topic:

v1/devices/me/attributes

When a shared attribute is changed by one of the server-side components (such as the REST API or the Rule Chain), the client will receive the following update:

{"key1":"value1"}
Example

Client library

Shell file

Mosquitto

mosquitto-attributes-subscribe.sh

MQTT.js

mqtt-js-attributes-subscribe.sh

mosquitto-attributes-subscribe.sh
# Subscribes to attribute updates
mosquitto_sub -d -h "127.0.0.1" -t "v1/devices/me/attributes" -u "$ACCESS_TOKEN"
mqtt-js-attributes-subscribe.sh
# Subscribes to attribute updates
mqtt sub -v "127.0.0.1" -t "v1/devices/me/attributes" -u '$ACCESS_TOKEN'

PRC API

Server-side RPC

_images/b6a3ba27f8bc1a1a6c4ece7317f6407aede9f1bce6a04e278c1473026b26223a.svg

In order to subscribe to RPC commands from the server, send SUBSCRIBE message to the following topic:

v1/devices/me/rpc/request/+

Once subscribed, the client will receive individual commands as a PUBLISH message to the corresponding topic:

v1/devices/me/rpc/request/$request_id

where $request_id is an integer request identifier.

The client should publish the response to the following topic:

v1/devices/me/rpc/response/$request_id
Example

The following example is written in javascript and is based on mqtt.js. Pure command-line examples are not available because subscribe and publish need to happen in the same mqtt session.

Client library

Shell file

JavaScript file

MQTT.js

mqtt-js-rpc-from-server.sh

mqtt-js-rpc-from-server.js

mqtt-js-rpc-from-server.sh
export TOKEN=$ACCESS_TOKEN
node mqtt-js-rpc-from-server.js
mqtt-js-rpc-from-server.js
var mqtt = require('mqtt');
var client  = mqtt.connect('mqtt://127.0.0.1',{
   username: process.env.TOKEN
});

client.on('connect', function () {
   console.log('connected');
   client.subscribe('v1/devices/me/rpc/request/+')
});

client.on('message', function (topic, message) {
   console.log('request.topic: ' + topic);
   console.log('request.body: ' + message.toString());
   var requestId = topic.slice('v1/devices/me/rpc/request/'.length);
   //client acts as an echo service
   client.publish('v1/devices/me/rpc/response/' + requestId, message);
});

Client-side RPC

_images/e979f283a029406684ebebd0d8ecf6284d448cab043562a554b448ff3600b0f3.svg

In order to subscribe to client-side RPC response from the server, send SUBSCRIBE message to the following topic:

v1/devices/me/rpc/response/+

Once subscribed, the client may send PUBLISH message to the following topic:

v1/devices/me/rpc/request/$request_id

where $request_id is an integer request identifier. The response from server will be published to the following topic:

v1/devices/me/rpc/response/$request_id
Example

The following example is written in javascript and is based on mqtt.js. Pure command-line examples are not available because subscribe and publish need to happen in the same mqtt session.

Client library

Shell file

JavaScript file

MQTT.js

mqtt-js-rpc-from-client.sh

mqtt-js-rpc-from-client.js

mqtt-js-rpc-from-client.sh
export TOKEN=$ACCESS_TOKEN
node mqtt-js-rpc-from-client.js
mqtt-js-rpc-from-client.js
var mqtt = require('mqtt');
var client = mqtt.connect('mqtt://127.0.0.1', {
   username: process.env.TOKEN
});

client.on('connect', function () {
   console.log('connected');
   client.subscribe('v1/devices/me/rpc/response/+');
   var requestId = 1;
   var request = {
      "method": "getTime",
      "params": {}
   };
   client.publish('v1/devices/me/rpc/request/' + requestId, JSON.stringify(request));
});

client.on('message', function (topic, message) {
   console.log('response.topic: ' + topic);
   console.log('response.body: ' + message.toString());
});

Claiming Devices API

Please see the corresponding article to get more information about the Claiming devices feature.

In order to initiate claiming device, send PUBLISH message to the following topic:

v1/devices/me/claim

The supported data format is:

{"secretKey":"value", "durationMs":60000}

Please note that the above fields are optional. In case the secretKey is not specified, the empty string as a default value is used. In case the durationMs is not specified, the system parameter device.claim.duration is used (in the file /etc/thingsboard/conf/thingsboard.yml ).

Firmware API

When ThingsBoard initiates an MQTT device firmware update, it sets the fw_title, fw_version, fw_checksum, fw_checksum_algorithm shared attributes.

Device provisioning API

Security

MQTT Transport

Copyrights and Licenses

Copyrights

All original source code & document in this repository is Copyright (C) 2022 Liang Zhuzi. This source code is licensed under the Apache License 2.0 as described in the file LICENSE.

Additional third party copyrighted code & document is included under the following licenses.

Where source code & document headers specify Copyright & License information, this information takes precedence over the summaries made here.

ThingsBoard License

thingsboard/thingsboard.github.io is licensed under the Apache License 2.0.

备注

This project is under active development.