Messaging query telemetry transport (MQTT) has emerged as one of the dominant IoT message transports across multiple industries in the last five years. Considering that most cloud services provide native MQTT capabilities, more device manufacturers, software, and services are implementing MQTT-based products.
The genesis of MQTT
Adoption of MQTT by Facebook, cloud service providers, and many others in the information technology (IT) space might lead one to think that MQTT was invented targeting IT solutions, but the genesis of MQTT was driven by an industrial communication problem.
In 1997, Phillips 66 had installed one of the first transmission control protocol/internet protocol (TCP/IP)-based very-small-aperture-terminal (VSAT) systems in the market for use in its pipeline supervisory control and data acquisition (SCADA) system. Numerous challenges needed to be addressed to use this network infrastructure effectively. Poll/response protocols were the norm for any SCADA system implementation until this system was implemented.
However, due to the propagation delays inherent to VSAT communications, and the cost associated with continuously polling for process variables that may not have changed, Phillips was looking for a better way to optimize its network infrastructure.
By this time, information technology (IT) departments used message-oriented middleware (MOM) software to decouple applications from each other. They were efficient infrastructures that used message brokers to ensure applications that “published” information could be connected to applications that “subscribed” to that information. Information could be published on an exception basis to any application that had interest and was subscribed to that information.
The idea was to use this same type of infrastructure for a real-time SCADA system. The only problem was that MOM products on the market at that time weren’t appropriate for use in the SCADA environment.
Based on these requirements, a project was started to develop a MOM specification that would be appropriate for use in these types of industrial environments. This ultimately led to the design of MQTT.
The original design goals of MQTT were that it would be simple, efficient, stateful, and open.
Simple. When MQTT first was being developed, the hardware platforms available on the market for remote edge computing were minimal; 8-bit microprocessors with 64 KB of memory were the norm. MQTT had to be simple to implement with minimal computing resources. Even in 2018, Arduino microcontrollers can provide complete MQTT communication stacks.
Efficient. Early VSAT system providers charged for every byte of information sent and received. The MQTT transport had to provide minimal overhead on the network. Once an MQTT session is established, there is only a 2-byte overhead in messages being published.
Stateful. If a user is providing infrastructure for mission critical, real-time infrastructure then the “state” of the MQTT TCP/IP connection is critical. MQTT provides a mechanism called “continuous session awareness” that informs all clients that care about the real-time state information of the MQTT connections.
Open. In the late 1990’s SCADA/DCS/Telemetry products were based mainly on proprietary legacy Poll/Response protocols. For MQTT to be useful to the industry as a whole it was understood that when it was released, it needed to be an open specification that anyone could implement for free.
Even with those criteria, it would be easy to assume a few important aspects are left out, including:
Security. A lot of people note the MQTT specification does not define any security. This is because the MQTT specification in based on top of TCP/IP. It always was envisaged that the latest TCP/IP security practices would be applicable to an MQTT infrastructure. This ranges from private networks where security isn’t even required, to full transport layer security (TLS) certificates being used for connections. Since MQTT is a remote-originated connection, edge devices and clients don’t even have to have any TCP/IP ports open, which is a huge reduction in the overall cybersecurity footprint.
Payload data format. MQTT is data agnostic when it comes to the information contained in an MQTT payload. It can be a binary message from a programmable logic controller (PLC), a JPEG image, an extensible markup language (XML) document or a JavaScript object notation (JSON) string. MQTT leaves the encoding and interpretation of the payload to the software provider.
Industrial-strength MQTT
As Internet of Things (IoT) solutions using MQTT started to migrate to more mission-critical Industrial IoT (IIoT) solutions, the market needed a specification that would allow multiple vendors of MQTT-based hardware and software Although the MQTT specification does not dictate any message topic namespace or data representation, one was needed for the IIoT space. The Sparkplug specification does that for the IIoT market.
The Sparkplug specification was developed to help define how best to get started using MQTT in a mission-critical, real-time application. The Sparkplug specification defines:
- A well-known MQTT topic namespace so publishers and subscribers of information can know the topic namespace in advance for interoperability.
- A binary payload optimized for industrial process variables. The Sparkplug specification acknowledges that industrial infrastructures don’t have unlimited bandwidth and must work well over VSAT, radio, and cellular infrastructures.
- How the “state” management in MQTT works and how to effectively use it in SCADA, distributed control system (DCS), and industrial control system (ICS) solutions to know the state of all MQTT clients in real time.
The Sparkplug specification and all of the reference implementation code written in C, Java, JavaScript, Python, and Node Red have been contributed to the Eclipse Foundation and a complete open source project.
Arlen Nipper is president/CTO, Cirrus Link Solutions. Edited by Chris Vavra, production editor, Control Engineering, CFE Media, cvavra@cfemedia.com.
MORE ANSWERS
Keywords: MQTT, Internet of Things, messaging query telemetry transport
Messaging query telemetry transport (MQTT) started as a communication protocol used by Phillips 66 to operate in tough industrial conditions.
MQTT’s usefulness to engineers and manufacturers stems from being easy to use and operate.
MQTT has been enhanced to work in the Industrial Internet of Things (IIoT) era, which takes up a lot of data in terms of volume.
Consider this
For what applications do you use MQTT and how successful and/or useful has it been?
ONLINE extra
About the author
Arlen Nipper is the co-founder and president/CTO of Cirrus Link Solutions. Across his entire career, he has applied embedded computer technology to existing paradigm problems in the industrial controls and automation market sector. In recent years, he has stepped back from just the hardware/software aspects of embedded systems and started to view the entire ecosystem of hardware, software, security, infrastructure, IT, and ultimately the people being served by the this hugely interesting, emerging Internet of Things (IoT).
Nipper has more than 40 years of SCADA experience in the oil & gas industry, starting his career in 1978 with Koch Oil and was key to the implementation on their first pipeline SCADA system installation. Throughout his career he has advanced SCADA technology including hardware design, software design, and overall SCADA infrastructure implementations. He is the co-creator of MQTT, a widely adopted IIoT protocol used across industries, designed to optimize and make use of data for both OT and IT for Phillips 66’s VSAT-based SCADA system in 1998.
Learn more about MQTT’s role with Facebook Messenger:
Learn more about the Eclipse project called “Tahu.”