NATS – Messaging as simple as possible

NATS is a relatively new cloud native Publish/Subscribe system with optional streaming support. It appears to be much simpler than already known systems like ActiveMQ, RabbitMQ or Apache Kafka but also comes with a smaller feature set. The goal of NATS is to be a central integration platform of distributed systems or like authors like to formulate “A Central Nervous System”. Initially, NATS has been developed for Cloud Foundry. In the meantime it has been ported from Ruby to Go and picked up by the Cloud Native Computing Foundation.

Core NATS

Core NATS is a very simple pub/sub system for 1:N communication. The central use case consists of distributing data to connected consumers. For example, showing the locations of available taxis on a map. When consumers should also get messages that have been published when they were not connected, Jetstream, the additional streaming engine, provides solutions.

NATS with Jetstream

With Jetstream, NATS is not only able to deliver messages safely after temporary network or consumer outages but also messages that have been published long time before a consumer has been added to the system landscape. This makes it an easy-to-use competitor in the domain of streaming systems like Apache Kafka or Pulsar. This blog post is focuses Core NATS.

1. via WorkQueuePolicy, is applied to the Stream, not to subscriptions or groups. Means, those messages can be consumed only a single time globally
2. Can be implemented with a custom reply-to-header, but the broker has not a standard for this

Management UI & CLI tools

The built-in http UI is rather minimalistic. It provides data which is interesting from an operations perspective but nothing about the data which is distributed via the broker.

Though, the command line tools provide deep insights for those use cases.

nats is the main cli tool. It allows to manage and browse subjects and streams. Publishing and subscribing to subjects and streams is also possible. On top, it contains useful commands for benchmarking an entire NATS cluster.

nats-top provides live resource metrics such as CPU, memory and data rates.

  nats-server version 0.6.4 (uptime: 31m42s)
Server:
  Load: CPU: 0.8%   Memory: 5.9M  Slow Consumers: 0
  In:   Msgs: 34.2K  Bytes: 3.0M  Msgs/Sec: 37.9  Bytes/Sec: 3389.7
  Out:  Msgs: 68.3K  Bytes: 6.0M  Msgs/Sec: 75.8  Bytes/Sec: 6779.4

Connections: 4
  HOST                 CID      SUBS    PENDING     MSGS_TO     MSGS_FROM   BYTES_TO    BYTES_FROM  LANG     VERSION SUBSCRIPTIONS
  127.0.0.1:56134      2        5       0           11.6K       11.6K       1.1M        905.1K      go       1.1.0   foo, hello
  127.0.1.1:56138      3        1       0           34.2K       0           3.0M        0           go       1.1.0    _INBOX.a96f3f6853616154d23d1b5072
  127.0.0.1:56144      4        5       0           11.2K       11.1K       873.5K      1.1M        go       1.1.0   foo, hello
  127.0.0.1:56151      5        8       0           11.4K       11.5K       1014.6K     1.0M        go       1.1.0   foo, hello

API

You can spin up a local NATS instance with the docker command docker run -p 4222:4222 -p 8222:8222 nats:2.7.4. Now, you can send and receive a first hello world message via the my-hello-subject using the nats cli tool.

Core NATS

> nats pub my-hello-subject "hello world"
Published 11 bytes to "my-hello-subject"

> nats sub "my-hello-subject"
[#1] Received on "my-hello-subject"
hello world

    // rust consumer example

    let nats_connection = nats::connect("localhost")?;
    let subscription = nats_connection.subscribe("my-hello-subject").unwrap();
    while let next_message = subscription.next(){
        match next_message {
            Some(message) => {
                match std::str::from_utf8(&message.data) {
                    Ok(v) => println!("{}", v),
                    Err(e) => panic!("Invalid UTF-8 sequence: {}", e),
                };
            }
            None => {}
        }
    }

    // java consumer example

    Connection natsConnection = Nats.connect();
    Subscription subscription = natsConnection.subscribe("my-hello-subject");
    while (true) {
        Message msg = subscription.nextMessage(Duration.ofMillis(500));
        if (msg != null) {
            String msgString = new String(msg.getData(), StandardCharsets.UTF_8);
            System.out.println(msgString);
        }
    }

Simplicity Comparison

In order to show, why Core NATS is simple, I decided to explain the main steps and decisions which have to be made to connect two applications and compare it with RabbitMQ.

Core NATS

1. Choose a subject name, e.g. orders
2. Make the producer publish data to orders
3. Make the consumer consume from orders

RabbitMQ

1. Think about the exchange type
2. Choose an exchange name
3. Choose a routing key for the producer, e.g. orders
4. Make the producer publish data to the chosen exchange with the chosen routing key orders
5. Make the consumer create a queue with a self-chosen name and a binding key orders

Conclusions

As you can see, connecting applications with NATS requires only a single thing to think about: a subject name. With RabbitMQ, there are more things which have to be considered.

Subject Hierarchies and Wildcards

As also known from RabbitMQ or ActiveMQ, subjects can be set up in a hierarchy. Referring to the example use case of tracking available taxis on a map, the subjects could be set up as:

• germany.duesseldorf.north
• germany.duesseldorf.east
• germany.duesseldorf.south
• germany.duesseldorf.west

A customer which is in the north of duesseldorf can subscribe to the subject germany.duesseldorf.north and will receive only data about taxi positions in the north of Duesseldorf. The fleet manager might want to see all taxis in duesseldorf and will subscribe to the subject germany.duesseldorf.*

Load Balancing

Although, NATS has not been designed as a queuing system, subscribers can join queue groups, also known as consumer groups in Apache Kafka. When this configuration option is set, it is ensured that only a single consumer of a queue group will receive a published message. The next message will be routed to another consumer of the same group and so on. It is also possible to set up multiple queue groups on a subject. This is probably the setup for a microservice architecture with multiple instances per service.

Request-Reply

Another special feature of NATS is the built-in support for the request-reply communication pattern. Here, the focus is not around publishing data but querying data from other systems. In general, it is possible to implement this with any messaging or streaming system by putting information about the response address/subject/topic into the request message. NATS comes with a standard for this which is visible on the API directly. For example, the nats cli tool allows querying data via nats request userdetails 3.

Operations & High Availability

Unfortunately, right now, there is no hosted solution available from Microsoft, AWS or Google. However, nats.io provides a documentation how to setup k8s clusters on the main cloud providers and deploy NATS with Helm. High-Availability deployments are supported by the clustering mode which will form a dynamic mesh network of NATS servers. Clients have to know only a single server of the whole cluster. The other nodes will be discovered automatically. Therefore, adding an additional node does not require all clients to be reconfigured.

From my point of view, primarily a developer, the documentation does not provide easy understandable information about the cluster behavior when nodes start to fail. Indeed, the documentation provides information about this topic but only for NATS with Jetstream – not Core NATS.

Summary

If it is sufficient for the consumers to receive only data that is published while they are connected, Core NATS can shine with its simplicity. There is not much to explain on how to connect services, even when it is the first time a development team is using a messaging system.