The idea behind it is simple, but powerful. We have senders called publishers. Their sole role is to send or publish messages. They don’t care about who is going to receive them or if someone will receive them at all. They just shoot and forget the messages. And they do that via channels.

Think of them as, for example, TV channels. We have Sports channels, Weather Forecasting channels, Cooking channels, and so on. Every publisher sends its messages to a certain channel, and whoever is subscribed for this channel will be able to receive these messages.

Here is where the subscribers come in play. They can subscribe to one or more channels and start receiving the messages broadcasted in there.

As we already mentioned, the messages are to be sent and forgotten. This means that if a subscriber subscribes for a certain channel, all the messages that were sent previously in that channel are not going to be available to this subscriber.

Due to the nature of this kind of architecture, we can easily achieve low coupling between the different components and provide a solid foundation for building robust and easy-to-maintain applications.

For example, imagine a situation where we need to replace or improve the publishing part of our system – say add more publishers, more channels or so on. Since the two parts are isolated, meaning publishers don’t care about subscribers and vice versa, we could easily do that without worrying whether we are breaking some other part of the system. We just add the new publishers. Then later, when a subscriber comes to the relevant channels, it just starts using them.

What is Redis?

The initial idea behind Redis was to serve as an in-memory cache solution, as an alternative to its ancestor Memcached.

But nowadays it's a many-in-one solution, providing an in-memory data structure store, key-value database, message brokering, and so on. This makes it perfect candidate when building an application that needs a really fast caching solution as well as some of the other features mentioned before. Especially if the performance of the app is crucial for its regular usage.

Redis performance comparison (source: google)

One of the biggest advantages when using Redis is the huge community and technical resources you can find online. A lot of these resources are free, and there are online platforms that have free tier offerings.

Redis includes in its arsenal a cloud solution as well. If you want to try it yourself, you may go here and register a free account or use their initial coupon offering.

Redis Enterprise Cloud Sign Up / Sign In page

Pub/Sub in Redis

What is pub/sub?

Publish/Subscribe channels in Redis is one of the features I haven’t mentioned above but it’s included in the last versions of Redis. This is their implementation of the pub/sub messaging pattern, where we have publishers and subscribers that exchange messages via channels.

We'll go briefly through it below and then see it in practice in a small demo app I have prepared for you.

How does Redis pub/sub work?

We have publishers (the producers of messages), channels (that the messages are going through), and subscribers (the receivers of the messages). Who receives what depends solely on who is subscribed to which channel.

Let's see how this works in an example:

If we have created three publishers which will be publishing messages to three different channels. Let’s call them channels 1, 2 and 3. We also have three subscribers, let’s call them subscribers A, B and C.

Now, let’s imagine subscriber A is listening for messages on all three channels, that is, it's subscribed to them. And subscribers B and C are subscribed to channels 2 and 3. This means that when either of the three publishers sends a message, subscriber A will receive it. And subscribers B and C will be receiving messages sent only by publishers 2 and 3, because they are listening only for messages on these channels (2 and 3).

Notice that we have two entities using a channel – one is sending, the other is receiving – but they are totally independent. And the messages being sent are not persisted. Once they are sent by the publisher they are forgotten. The only entities that are subscribed at the moment of sending will get them.

How to use pub/sub in Redis

There are a plethora of client libraries that you can use with Redis. There is a dedicated page where everybody can go and pick one, depending on the specific project needs or just on your preferred programming language.

People at Redis also marked some of these repositories as recommended which makes the choice easier, if you are new to all this.

For our demo below, I used ioredis, a full-featured Redis client for Node.js. I chose this because the demo app UI is built with React and Node.js and my server code goes pretty well with it.

Redis Pub/Sub Demo

Redis Pub/Sub Visualizer app

Show time!

The idea behind the demo application is to show visually how the pattern works.

What you will see when you open it for first the time is three buttons for publishing simple messages (news) in the three imaginary TV channels: Weather, Sports and Music.

The cards below the publish buttons are the subscribers. Once you move your mouse cursor over any of them, it will flip to its back side and you will see three buttons. You may use each of these buttons to subscribe to the relevant channel.

Once a subscriber is signed over a channel and you click on the icon or the publish button for this channel, you will see a sample news appearing on the front side of the card.

Play with different publishers/subscribers combinations and see the result.

I hope this will give you a better understanding of what I explained in the example above.

How to run the demo app locally

In order to install and run the demo application locally, follow the steps below (all commands are considered to be run from the root project directory):

Run frontend:

cd client yarn && yarn dev

Run backend:

cd server && yarn yarn start

And finally, use your local installation of Docker (if don’t have one, you may get it from here) to run this:

docker run -p 6379:6379 redislabs/redismod:preview

That’s probably the easiest way to have a running copy of Redis locally. The other option would be to use Redis Cloud directly and deploy the application online. This is an option I am still investigating and if I manage to do it, I will deploy the whole app and will let you know.

Closing

This article introduced you to the pub/sub messaging pattern subject. It's important to remember that whenever we want to build a highly performant application with a low coupled architecture and real-time like messaging features, consider using the Publish/Subscribe pattern and Redis in particular.

In fact a lot of the real life applications that use Redis are dashboard-based. This means that usually there is a nice dashboard screen, showing different data, often being updated in real time.

Imagine, for example, a system showing traffic in a specific area. This kind of software is a perfect candidate for leveraging the advantages of pub/sub. And in many cases this is achieved by using Redis.

In any case, as developers and engineers, we should always be guided by the specific needs of the project we are working on. Whenever we decide to introduce a new pattern or technology, we should do it carefully and back it up with serious research.

But sometimes, when you're trying to establish a voice call connection, you'll get an error that says “RTC Connecting Discord”. This message will keep showing without any meaningful progress.

So what does this error mean?

Discord uses the Real-time Chat protocol (RTC) to run concurrent communication. So if you are experiencing the “RTC Connecting Discord” problem, it is a network issue.

In this article, I will show you 5 changes you can make to your network configurations to fix the RTC Connecting Discord issue. 3 of the fixes involve your computer network settings, while the remaining 2 happen right in your Discord app.

How to Fix RTC Connecting Discord by Updating your Network Driver

If your device relies on an outdated network adapter driver for internet connections, it could have a negative effect on your internet experience – and could cause this issue as well.

So, updating your network driver can fix the issue.

The steps below take you through how you can update your network adapter driver.

Step 1: Click on Start (Windows logo) and search for "device manager". Hit ENTER to open the first search result – which is always Device Manager. You can click on the "Device Manager" search result, too.

Step 2: Expand "Network Adapters."

Step 3: Look for the adapter in use, right-click on it and select "update driver".

Step 4: Select "Search automatically for drivers".

Windows will now search the internet for an updated driver and install it for you.

How to Fix RTC Connecting Discord by Changing your Domain Name Server

A domain name server (DNS) is assigned to you by your ISP (Internet Service Provider). Domain name servers make it possible to reach websites by typing addresses (URLs) to the browser instead of some unreadable numbers.

Changing this DNS to a widely used DNS like that of Google or Cloudflare can help you fix the RTC Connecting Discord issue.

To change your DNS to Google’s, follow the steps below.

Step 1: Right-click on Start and choose “Run” to open the Run dialogue.

Step 2: Type in “Control ncpa.cpl” (without quotes) and hit ENTER. This will open up your network connection devices.

Step 3: Right-click on your current network and select “Properties”.

Step 4: Look for Internet Protocol Version 4 (TCP/IPv4) and double-click it.

Step 5: Click on the “Use the following DNS server address” radio button and type in the following values:

• 8.8.8.8 for Preferred DNS Server
• 8.8.4.4 for Alternate DNS Server

Step 6: Click Ok.

Complete the setup with the next fix.

How to Fix RTC Connecting Discord by Clearing your Computer Network Cache in the Command Line

If you’re using the web-based Discord, this fix can work for you.

You can clear your network cache on your browser, but a more effective way to do it is to clear it right on your Windows 10 computer in the command line.

The steps below show you how to do it.

Step 1: Hit the WIN (Windows logo) key on your keyboard and search for "cmd".

You have to use the Command Prompt as an administrator, so you should select "Run as Administrator" on the right instead of just hitting ENTER to open it.

Step 2: Enter and execute the following commands one after the other:

• ipconfig /release
• ipconfig /flushdns
• ipconfig /renew

Step 3: Restart your computer.

How to Fix RTC Connecting Discord by Disabling QoS

Discord’s QoS (Quality of Service) communicates to your router that the units of data being transmitted are of high priority. This could make your router misbehave, and cause the RTC Connecting Discord issue.

So, disabling the QoS might fix it for you in case you have it enabled.

Follow the steps below to disable QoS on Discord.

Step 1: Launch Discord, then click on Settings on the bottom left corner.

Step 2: Select Voice and Video on the left panel.

Step 3: Scroll down to the QoS section and disable it.

Step 4: Restart the Discord app.

How to Fix RTC Connecting Discord by Changing the Audio Subsystem in Discord

In Discord, the Legacy audio subsystem has always been suggested as the best because it is the highest quality compared to the Standard and Experimental audio subsystems.

Changing your Audio subsystem to Legacy can make you establish quality audio – which can end up fixing the RTC Connecting Discord issue.

These steps take you through how to change your Audio subsystem to Legacy.

Step 1: Open Discord and click on Settings on the bottom left corner.

Step 2: Select Voice and Video.

Step 3: Scroll down to the Audio Subsystem dropdown and select “Legacy”.

Step 4: Restart Discord.

Conclusion

This article showed you how you can fix the RTC Connecting Discord problem you might experience when you are trying to use Discord's audio call feature.

Apart from the solutions explained in this article, you can try other minor fixes such as:

• Restarting your devices – computer and router
• Double-checking internet connection
• Using a VPN

Thanks a lot for reading this article. If you find it helpful, share it with your friends and loved ones.

In the world of web development, Chrome extensions are a pretty handy set of tools to have around.

Whether you use them to add headers to simple requests or to scrape important data from the DOM, extensions help provide extra functionality that makes life easier.

I started playing around with developing a Chrome extension for a use-case I had in mind at work. It was then that I stumbled upon the way we pass around certain data from a web page to an extension. And the lack of a simplified guide made me write this article.

We do have the Chrome API documentation, and it is indeed very thorough. But I consider myself to be more of a visual learner, and being able to visualize how we pass messages between the extension scripts helped simplify the overall development.

Note: This article makes use of Manifest V2 instead of V3. The major difference between the two is the usage of service workers in V3 instead of background pages and related actions.

Message Passing: Interaction Between Scripts

An extension, as the name suggests, is like a layer on top of the existing webpage you're trying to access. The browser acts as the container.

It mainly comprises the following scripts:

• Popup Script - Local JavaScript file for the extension DOM
• Background Script - Provides persistence and handles background events
• Content Script - Scripts that run in isolation in the context of the web page
• Injected Script - Scripts that are programmatically injected into the web page

Normally, if you have to merely deal with the DOM content, then the way the extension is developed is relatively straightforward.

The raw HTML is already available to the content script and all you need to do is pass it to the popup script.

However, if you need to access the page's variables and functions, the process gets a little tricky.

The variables and functions available in the page context, say in the window object, are not accessible to the content scripts since they tend to run in a special JavaScript environment. They have access to only the DOM of the page but not the variables and functions.

To access a page's variables and functions, we inject scripts by appending them to the DOM. This makes the browser assume that it is run in the context of the web page. This in turn provides the injected script access to the local variables and functions.

Since Chrome extensions are event-driven because of their architecture, once the injected scripts have access to the page's variables and functions, they can pass it to the content script.

The content script then passes the these objects to the background page.

And finally, the popup script is able to call onto the background page using the Extension API and pass it to the Extension DOM.

Message Passing Overview

Now, we will build a simple Performance Watcher extension that reads the performance data from the global window object of a page and maps the essential metrics for the user to see. Let's get into the code then.

Enough Talk, Show Me The Code

You can find the complete code repository for the project here. Let's quickly run through the primary files and the important functionalities they offer.

The Manifest File

Every Chrome Extension needs a manifest file. It is basically a JSON-formatted file that provides a set of metadata so the browser can recognize the permissions that need to be granted and the likely operational reach of the extension.

Here is the manifest used for our application.

manifest.json: metadata for your extension

Some of the important properties we need to focus on are the following:

• background - Takes an array of scripts that would be run in the background page.
• content-scripts - Includes an array of content scripts we wish to run as part of the web page's context.
• web_accessible_resources - An array of packaged resources expected to be used in a web page's context. For example, an image we intend to embed in a page or a custom script we want to inject.
• permissions - Allows your extension to gain access to certain Chrome APIs like tabs in this case.

The Content Script

Content Scripts have easy access to the DOM of the web page. We make use of the content script to append our custom script – inject-script.js – into the DOM.

content-script.js: inject custom script into the DOM

The content script also simultaneously continues to listen for any message being sent upstream from the custom script.

As soon as we get a message from the injected script, we run a quick check on the data received and verify whether our extension is installed. Once done, we simply use Chrome's Runtime API to send the data received forward to the background page.

content-script.js: send the required data to the background page

The Injected Script

The custom script can access global variables and functions like the window object. We map only the properties we require.

inject-script.js: procure required object from the page's JS context

The message from the custom script is communicated safely to the content script using the [window.postMessage](https://developer.mozilla.org/en-US/docs/Web/API/Window/postMessage) function. In this case, a setInterval function is used to dynamically update the properties we are observing.

inject-script.js: send the gathered data to the content-script

The Background Script

The background script listens for any message transmitted by the content script using the Runtime API. The window object of the background page is then updated with the tab.id acting as the identifier.

background.js: listen for incoming message from the content-script

The Popup Script

The popup script is where we finally read the data we had procured from our custom script. It is also the place where we code any necessary JavaScript operations.

The background page is retrieved using the getBackgroundPage method of the Extension API. The active tab's id is queried using the tabs.query method of the Tabs API in order to correctly extract the relevant data.

popup.js: reading the global object stored in the background page context

In this way, we are able to finally receive and map the data we need – performance in our case – efficiently in our extension.

The UI styling and other cosmetic code are available in the repository, for further reference.

Final Thoughts

Message passing is an essential concept when it comes to developing a Chrome extension. This is just one of the multiple ways in which you can communicate between scripts.

I spent a few hours in order to figure out how it would work for my use case. Hopefully, this simple walkthrough and the visual representation saves you some time.

I would suggest playing around with the code for a bit. If you have any questions, feel free to reach out to me on [LinkedIn](https://www.linkedin.com/in/tarique-ejaz/).

In the meantime, keep coding.

Flutter has quickly become one the most popular frameworks for cross-platform mobile application development. It helps developers build native UIs with support for different device sizes, pixel densities, and orientations creating a beautiful pixel-perfect UI/UX.

In this tutorial, we are going to learn how to add push notifications to a Flutter app using Firebase Cloud Messaging. This tutorial will only deal with configuration for the Android platform.

First, what are push notifications?

Push Notifications are a sort of pop-up messaging medium that alerts app users to what's going on in the app. They are also an important way to amplify user engagement in your app.

For example, say a user forgets about the app once they have installed it. Then you can use push notifications as a mechanism to regain and retain their interest. Push notifications also help drive traffic to your app.

Firebase Cloud Messaging is a service offered by Firebase which lets you send these notifications to your users. You can set up various configurations to send different notifications to different audiences based on time and routine.

Because of all these benefits, we are going to use it to send notifications to our Flutter app.

Step 1: Create a Flutter Project

First, we are going to create a flutter project. For that, we must have the Flutter SDK installed in our system. You can find simple steps for flutter installation in the official documentation.

After you've successfully installed Flutter, you can simply run the following command in your desired directory to set up a complete Flutter project:

flutter create pushNotification

After you've set up the project, navigate inside the project directory. Execute the following command in the terminal to run the project in either an available emulator or an actual device:

flutter run

After a successful build, you will get the following result in the emulator screen:

Step 2: Integrate Firebase Configuration with Flutter

In this step, we are going to integrate Firebase services with our Flutter project. But first, we need to create a Firebase project. The setup guidelines are also provided in the official firebase documentation for Flutter.

To create a Firebase project, we need to log in to Firebase and navigate to the console. There we can simply click on 'Add a project' to get our project started.

Then a window will appear asking to input the project's name. Here, I've kept the project name as FlutterPushNotification as shown in the screenshot below:

We can continue to the next step when the project has been created. After the project has been set up, we will get a project console as shown in the screenshot below:

Here, we are going to set up Firebase for the Android platform. So we need to click on the Android icon displayed in the above screenshot. This will lead us to the interface to register Firebase with our Flutter app project.

Step 3: Register Firebase to Your Android App

As the registration process is platform-specific, we are going to register our app for the Android platform. After clicking on the Android icon, we will be directed to an interface asking for the Android package name.

In order to add the package name of our Flutter project, we need to locate it first. The package name will be available in the ./android/app/build.gradle file of your Flutter project. You will see something like this:

com.example.pushNotification

We just need to copy it and paste it to the Android package name input field as shown in the screenshot below:

After that, we can simply click on the 'Register app' button. This will lead us to the interface where we can get the google-services.json file which will link our Flutter app to Firebase Google services.

We need to download the file and move it to the ./android/app directory of our Flutter project. The instructions are also shown in the screenshot below:

Step 4: Add Firebase Configurations to Native Files in your Flutter Project

Now in order to enable Firebase services in our Android app, we need to add the google-services plugin to our Gradle files.

First in our root-level (project-level) Gradle file (android/build.gradle), we need to add rules to include the Google Services Gradle plugin. We need to check if the following configurations are available or not:

buildscript {
  repositories {
    // Check that you have the following line (if not, add it):
    google()  // Google's Maven repository
  }
  dependencies {
    ...
    // Add this line
    classpath 'com.google.gms:google-services:4.3.4'
  }
}

allprojects {
  ...
  repositories {
    // Check that you have the following line (if not, add it):
    google()  // Google's Maven repository
    ...
  }
}

If not, we need to add the configurations as shown in the code snippet above.

Now in our module (app-level) Gradle file (android/app/build.gradle), we need to apply the Google Services Gradle plugin.

For that, we need to add the piece of code highlighted in the following code snippet to the ./android/app/build.gradle file of our project:

// Add the following line:
**apply plugin: 'com.google.gms.google-services'**  // Google Services plugin

android {
  // ...
}

Now, we need to run the following command so that some automatic configurations can be made:

flutter packages get

With that we have successfully integrated Firebase configurations with our Flutter project.

Step 5: Integrate Firebase Messaging with Flutter

First, we need to add the firebase-messaging dependency to the ./android/app/build.gardle file. In the file, we need to add the following dependencies:

dependencies {
    implementation "org.jetbrains.kotlin:kotlin-stdlib-jdk7:$kotlin_version"
    implementation 'com.google.firebase:firebase-messaging:20.1.0'
}

Next, we need to add an action and a category as an intent-filter within the activity tag in the ./android/app/src/main/AndroidManifest.xml file:

<intent-filter>
    <action android:name="FLUTTER_NOTIFICATION_CLICK" />
    <category android:name="android.intent.category.DEFAULT" />
</intent-filter>

Now, we need to create a Java file called Application.java in the path /android/app/src/main/java/.

Then, we need to add the code from the following code snippet inside it:

package io.flutter.plugins.pushNotification;

import io.flutter.app.FlutterApplication;
import io.flutter.plugin.common.PluginRegistry;
import io.flutter.plugin.common.PluginRegistry.PluginRegistrantCallback;
import io.flutter.plugins.GeneratedPluginRegistrant;
import io.flutter.plugins.firebasemessaging.FirebaseMessagingPlugin;
import io.flutter.plugins.firebasemessaging.FlutterFirebaseMessagingService;

public class Application extends FlutterApplication implements PluginRegistrantCallback {
    @Override
    public void onCreate() {
        super.onCreate();
        FlutterFirebaseMessagingService.setPluginRegistrant(this);
    }

    @Override
    public void registerWith(PluginRegistry registry) {
        FirebaseMessagingPlugin.registerWith(registry.registrarFor("io.flutter.plugins.firebasemessaging.FirebaseMessagingPlugin"));
    }
}

Now, we need to assign this Application activity to the application tag of the AndroidManifest.xml file as shown in the code snippet below:

<application
        android:name=".Application"

This completes our setup of the Firebase messaging plugin in the native Android code. Now, we'll move on to the Flutter project.

Step 6: Install the Firebase Messaging Package

Here, we are going to use the [firebase_messaging] package, which you can find here. For that, we need to add the plugin to the dependency option of the pubspec.yaml file.

We need to add the following line of code to the dependencies option:

firebase_messaging: ^7.0.3

Step 7: Implement a Simple UI Screen

Now, inside the MyHomePage stateful widget class of the main.dart file, we need to initialize the FirebaseMessaging instance and some constants as shown in the code snippet below:

String messageTitle = "Empty";
String notificationAlert = "alert";

FirebaseMessaging _firebaseMessaging = FirebaseMessaging();

The messageTitle variable will receive the notification message title and notificationAlert will be assigned the action that's been completed once the notification comes up.

Now, we need to apply these variables to the build function inside the Scaffold widget body as shown in the code snippet below:

Widget build(BuildContext context) {
    return Scaffold(
      appBar: AppBar(
        title: Text(widget.title),
      ),
      body: Center(
        child: Column(
          mainAxisAlignment: MainAxisAlignment.center,
          children: <Widget>[
            Text(
              notificationAlert,
            ),
            Text(
              messageTitle,
              style: Theme.of(context).textTheme.headline4,
            ),
          ],
        ),
      ),
    );
  }

Next, we need to run the Flutter application by executing the following command in the project terminal:

flutter run

We will get the result you see in the image below:

For now, the notification title is empty, and the alert is also as defined. We need to assign a proper value to them once we receive the notification message.

So we need to configure the code to receive the notification and use the notification message to display it on the screen.

For that, we need to add the code from the following code snippet in the initiState function:

@override
  void initState() {
    // TODO: implement initState
    super.initState();

    _firebaseMessaging.configure(
      onMessage: (message) async{
        setState(() {
          messageTitle = message["notification"]["title"];
          notificationAlert = "New Notification Alert";
        });

      },
      onResume: (message) async{
        setState(() {
          messageTitle = message["data"]["title"];
          notificationAlert = "Application opened from Notification";
        });

      },
    );
  }

Here, we have used the configure method provided by _firebaseMessaging instance which in turn provides the onMessage and onResume callbacks. These callbacks provide the notification message as a parameter. The message response will hold the notification object as a map object.

The onMessage function triggers when the notification is received while we are running the app. The onResume function triggers when we receive the notification alert in the device notification bar and opens the app through the push notification itself. In this case, the app can be running in the background or not running at all.

Now we are all equipped with the Flutter app. We just need to configure a message in Firebase Cloud Messaging and send it to the device.

Step 8: Create a Message from the Firebase Cloud Messaging Console

First, we need to go back to the Cloud Messaging console in the Firebase site as shown in the image below:

Here, we can see the 'Send your first message' option in the window, as we have not configured any messages before. We need to click on it which will lead us to the following window:

Here, we can enter the title, text, image, and name of the notification. The title we set here will be provided as the title in the message object on the callbacks we set before in the Flutter project.

After setting the required fields, we can click on 'Next' which will lead us to the following window:

Here, we need to provide our target app and click on 'Next'.

For Scheduling we can keep the default option:

Next, the Conversion window will appear which we can keep as default as well, and then click on the 'Next' button.

Lastly, a window where we need to enter the custom data will appear in which we can set the title and click_action. This click action event is triggered whenever we click on the notification that appears in the notification bar of the device.

After clicking on the notification message from the notification bar, the app will open and the onResume callback will trigger, setting title as assigned in the custom data in the screenshot below:

Now, we are ready to send the first notification message to the device. First, let's try it with the device running in the emulator.

As we click on the 'Review' button and send the message, we will get the following result in the Cloud Messaging console as well as the emulator:

Here, we can see that the title and the notification alert on the emulator screen are updated as soon as we send a message from the console. We can be sure that the onMessage callback was triggered in the app after receiving the notification message.

Now let's try with the app running in the background. As we send the message from the console, we will get the result as shown in the demo below:

Here, as soon as we send the message, we receive a push notification in the notification bar of the device. Then, as we drag down the notification bar, we can see the notification message title and text. And, by clicking on the notification message, we can launch the application and display the custom data on the screen. This ensures that our onResume callback was triggered.

And we're done! We have successfully added a push notification feature in our Flutter application using Firebase Cloud Messaging.

Conclusion

Push notifications are essential in any app. They can be used to alert users to what's going on in the app, and can help drive users' interest back to the app.

Additionally, Firebase Cloud Messaging makes sending notification alerts much simpler and easier.

In this tutorial, we started by configuring the Firebase app and then moved on to the setup and implementation of the Firebase messaging configuration in the Flutter app. Lastly, we were able to send notification alerts to the app using Firebase Cloud Messaging.

The tutorial was meant to be simple and easy to understand. Hope that it helps you add push notification to your Flutter apps. Want to see some examples of how you can implement all this? Check out these powerful Flutter templates.

In this guide, we’re going to explore the basics of using RabbitMQ with Node.js.

Theory

At its most basic level, you’d ideally have two different services interacting with one another through Rabbit - a publisher and a subscriber.

A publisher typically pushes messages to Rabbit, and a subscriber listens to these messages, and executes code on the basis of those messages.

Note that they can be both at the same time - a service can publish messages to Rabbit and consume messages at the same time, which allows really powerful systems to be designed.

Now a publisher typically publishes messages with a routing key to something called an exchange. A consumer listens to a queue on the same exchange, bound to the routing key.

In architectural terms, your platform would use one Rabbit exchange, and different kinds of jobs/services would have their own routing keys and queues, in order for pub-sub to work effectively.

Messages can be strings; they can also be native objects - AMQP client libraries do the heavy lifting of converting objects from one language to another. And yes, that does mean services can be written in different languages, so long as they’re able to understand AMQP.

Getting started

We’re going to cook up a very simple example where a publisher script publishes a message to Rabbit, containing a URL, and a consumer script listens to Rabbit, takes the published URL, calls it and displays the results. You can find the finished sample up on Github.

First, let’s initialize a npm project:

$ npm init

You can always just hit Enter all the way and use the default options - or you can fill them up.

Now, let’s install the packages we need. We’re going to use Tortoise to interact with RabbitMQ. We’re also going to use node-cron to schedule the actual message publishing.

$ npm install --save tortoise node-cron

Now your package.json should look a lot like this:

{
  "name": "freecodecamp-guides-rabbitmq-tortoise",
  "version": "1.0.0",
  "description": "Sample code to accompany the FreeCodeCamp guide on async messaging with RabbitMQ and Tortoise.",
  "main": "index.js",
  "scripts": {
    "test": "echo \"Error: no test specified\" && exit 1"
  },
  "repository": {
    "type": "git",
    "url": "git+https://github.com/rudimk/freecodecamp-guides-rabbitmq-tortoise.git"
  },
  "keywords": [
    "rabbitmq",
    "tortoise",
    "amqp"
  ],
  "author": "Rudraksh M.K.",
  "license": "MIT",
  "bugs": {
    "url": "https://github.com/rudimk/freecodecamp-guides-rabbitmq-tortoise/issues"
  },
  "homepage": "https://github.com/rudimk/freecodecamp-guides-rabbitmq-tortoise#readme",
  "dependencies": {
    "node-cron": "^1.2.1",
    "tortoise": "^1.0.1"
  }
}

Now we’re all set. Let’s create a publisher first.

const Tortoise = require('tortoise')
const cron = require('node-cron')

const tortoise = new Tortoise(`amqp://rudimk:YouKnowWhat@$localhost:5672`)

After importing tortoise and node-cron, we’ve gone ahead and initialized a connection to RabbitMQ. Next, let’s write a quick and dirty function that publishes a message to Rabbit:

function scheduleMessage(){
    let payload = {url: 'https://randomuser.me/api'}
    tortoise
    .exchange('random-user-exchange', 'direct', { durable:false })
    .publish('random-user-key', payload)
}

That’s simple enough. We’ve defined a dictionary containing a URL to the RandomUser.me API, which is then published to the random-user-exchange exchange on RabbitMQ, with the random-user-key routing key.

As mentioned earlier, the routing key is what determines who gets to consume a message. Now, let’s write a scheduling rule, to publish this message every 60 seconds.

cron.schedule('60 * * * * *', scheduleMessage)

And our publisher’s ready! But it’s really no good without a consumer to actually consume these messages! But first, we do need a library that can call the URL in these messages. Personally, I use superagent: npm install --save superagent.

Now, in consumer.js:

const Tortoise = require('tortoise')
const superagent = require('superagent')

const tortoise = new Tortoise(`amqp://rudimk:YouKnowWhat@$localhost:5672`)

Next, let’s write an async function that calls a URL and displays the result:

async function getURL(url){
    let response = await superagent.get(url)
    return response.body
}

Time to write code to actually consume messages:

tortoise
.queue('random-user-queue', { durable: false })
// Add as many bindings as needed 
.exchange('random-user-exchange', 'direct', 'random-user-key', { durable: false })
.prefetch(1)
.subscribe(function(msg, ack, nack) {
  // Handle 
  let payload = JSON.parse(msg)
  getURL(payload['url']).then((response) => {
    console.log('Job result: ', response)
  })
  ack() // or nack()
})

Here, we’ve told tortoise to listen to the random-user-queue, that’s bound to the random-user-key on the random-user-exchange. Once a message is received, the payload is retrieved from msg, and is passed along to the getURL function, which in turn returns a Promise with the desired JSON response from the RandomUser API.

Conclusion

The simplicity associated with using RabbitMQ for messaging is unparalleled, and it’s very easy to come up with really complex microservice patterns with just a few lines of code.

The best part is that the logic behind messaging doesn’t really change across languages - Crystal or Go or Python or Ruby work with Rabbit in pretty much the same way. This means you can have services written in different languages all communicating with each other effortlessly, enabling you to use the best language for the job.

AWS offers a wealth of options for imple­ment­ing mes­sag­ing pat­terns such as Publish/Subscribe (often short­ened to pub/sub) with AWS Lamb­da. In this article, we’ll com­pare and con­trast some of these options.

The pub/sub pattern

Pub/Sub is a mes­sag­ing pat­tern where pub­lish­ers and sub­scribers are decou­pled through an inter­me­di­ary message bro­ker (ZeroMQ, Rab­bit­MQ, SNS, and so on).

_Source: [Publish Subscribe Pattern (Wikipedia)](https://en.wikipedia.org/wiki/Publish%E2%80%93subscribe_pattern" rel="noopener" target="blank" title=")

In the AWS ecosys­tem, the obvi­ous can­di­date for the bro­ker role is Simple Notification Service (SNS).

SNS will make three attempts for your Lambda func­tion to process a mes­sage before send­ing it to a Dead Let­ter Queue (DLQ) if a DLQ is spec­i­fied for the func­tion. How­ev­er, accord­ing to an analy­sis by the folks at Ops­Ge­nie, the number of retries can be as many as six.

Anoth­er thing to con­sid­er is the degree of par­al­lelism this set­up offers. For each mes­sage, SNS will cre­ate a new invo­ca­tion of your func­tion. So if you pub­lish 100 mes­sages to SNS, then you can have 100 con­cur­rent exe­cu­tions of the sub­scribed Lamb­da func­tion.

This is great if you’re opti­mis­ing for through­put.

How­ev­er, we’re often con­strained by the max through­put our down­stream depen­den­cies can han­dle — data­bas­es, S3, internal/external ser­vices, and so on.

If the burst in through­put is short, then there’s a good chance the retries would be suf­fi­cient (there’s a ran­domized, expo­nen­tial back off between retries too) and you won’t miss any mes­sages.

Erred messages are retried 2 times with exponential back off. If the burst is short-lived then the retry is likely to succeed, resulting in no message loss.

If the burst in through­put is sus­tained over a long peri­od of time, then you can exhaust the max number of retries. At this point you’ll have to rely on the DLQ and pos­si­bly human inter­ven­tion in order to recov­er the mes­sages that couldn’t be processed the first time around.

Erred messages are retried 2 times with exponential back off. But the burst in message rate overlaps with the retries, further exacerbating the problem and eventually the max number of retries are exhausted and erred messages have to be delivered to the DLQ instead (if one is specified).

Sim­i­lar­ly, if the down­stream depen­den­cy expe­ri­ences an out­age, then all mes­sages received and retried dur­ing the out­age are bound to fail.

Any message received or retried during the downstream message will fail and be sent to the DLQ.

You can also run into the Lamb­da lim­it on the number of con­cur­rent exe­cu­tions in a region. Since this is an account-wide lim­it, it will also impact your oth­er sys­tems within the account that rely on AWS Lamb­da: APIs, event pro­cess­ing, cron jobs, and so on.

SNS is also prone to suf­fer from tem­po­ral issues, like bursts in traf­fic, down­stream out­age, and so on. Kine­sis, on the oth­er hand, deals with these issues much bet­ter as described below:

• The degree of par­al­lelism is con­strained by the number of shards, which can be used to amor­tize bursts in the mes­sage rate

Bursts in message rate is amortised, as the max throughput is determined by no. of shards max batch size 5 reads per second. Which gives you two levers to adjust the max throughput with.

• Records are retried until suc­cess is achieved, unless the out­age lasts longer than the reten­tion pol­i­cy you have on the stream (the default is 24 hours). You will even­tu­al­ly be able to process the records

The impact of a downstream outage is absorbed by the retry-until-success invocation policy.

But Kine­sis Streams is not with­out its own prob­lems. In fact, from my expe­ri­ence using Kine­sis Streams with Lamb­da, I have found a number of caveats that need­ to be under­stood in order to make effec­tive use of the service.

You can read about these caveats in another article I wrote here.

Inter­est­ing­ly, Kine­sis Streams is not the only stream­ing option avail­able on AWS. There is also DynamoDB Streams.

DynamoDB Streams can be used as a like-for-like replacement for Kinesis Streams.

By and large, DynamoDB Streams + Lamb­da works the same way as Kine­sis Streams + Lamb­da. Oper­a­tional­ly, it does have some inter­est­ing twists:

• DynamoDB Streams auto scales the number of shards
• If you’re pro­cess­ing DynamoDB Streams with AWS Lamb­da, then you don’t pay for the reads from DynamoDB Streams (but you still pay for the read and write capac­i­ty units for the DynamoDB table itself)

_Source: [DynamoDB Pricing](https://aws.amazon.com/dynamodb/pricing/" rel="noopener" target="blank" title=")

• Kine­sis Streams offers the option to extend data reten­tion to 7 days, but DynamoDB Streams doesn’t offer such an option

_Source: [Working with DynamoDB Streams](https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/Streams.html" rel="noopener" target="blank" title=")

The fact that DynamoDB Streams auto scales the number of shards can be a dou­ble-edged sword. On one hand, it elim­i­nates the need for you to man­age and scale the stream (or come up with home-baked auto scal­ing solu­tions). But on the oth­er hand, it can also dimin­ish the abil­i­ty to amor­tize spikes in the load you pass on to down­stream sys­tems.

As far as I know, there is no way to lim­it the number of shards a DynamoDB stream can scale up to, which is some­thing you’d sure­ly con­sid­er when imple­ment­ing your own auto scal­ing solu­tion.

I think the most per­ti­nent ques­tion is, “what is your source of truth?”

Does a row being writ­ten in DynamoDB make it canon to the state of your system? This is cer­tain­ly the case in most N-tier sys­tems that are built around a data­base, regard­less of whether it’s an RDBMS or NoSQL database.

In an event-sourced sys­tem where state is mod­eled as a sequence of events (as opposed to a snap­shot), the source of truth might well be the Kine­sis stream. For example, as soon as an event is writ­ten to the stream, it’s con­sid­ered canon to the state of the sys­tem.

Then, there’re oth­er con­sid­er­a­tions around cost, auto-scal­ing, and so on.

From a devel­op­ment point of view, DynamoDB streams also have some limitations and short­com­ings:

• Each stream is lim­it­ed to events from one table
• The records describe DynamoDB events and not events from your domain, which I’ve always felt cre­ates a sense of dis­so­nance when work­ing with these events

Exclud­ing the cost of Lamb­da invo­ca­tions for pro­cess­ing the mes­sages, here are some cost pro­jec­tions for using SNS vs Kine­sis Streams vs DynamoDB Streams as the bro­ker. I’m mak­ing the assump­tion that through­put is con­sis­tent, and that each mes­sage is 1KB in size.

Month­ly cost at 1 msg/s

Month­ly cost at 1,000 msg/s

These pro­jec­tions should not be tak­en at face val­ue. For starters, the assump­tion about a per­fect­ly con­sis­tent through­put and mes­sage size is unre­al­is­tic, and you’ll need some head­room with Kine­sis and DynamoDB streams even if you’re not hit­ting the throt­tling lim­its.

That said, what these pro­jec­tions do tell me is that:

1. You get an awful lot with each shard in Kine­sis streams
2. While there’s a base­line cost for using Kine­sis streams, the cost goes down when usage scales up as com­pared to SNS and DynamoDB streams, thanks to the sig­nif­i­cant­ly low­er cost per mil­lion requests

Whilst SNS, Kine­sis, and DynamoDB streams are your basic choic­es for the bro­ker, Lamb­da func­tions can also act as bro­kers in their own right and prop­a­gate events to oth­er ser­vices.

This is the approach used by the aws-lamb­da-fanout project from awslabs. It allows you to prop­a­gate events from Kine­sis and DynamoDB streams to oth­er ser­vices that can­not direct­ly sub­scribe to the three basic choice of bro­kers (either because of account/region lim­i­ta­tions or that they’re just not sup­port­ed).

The aws-lambda-fanout project from awslabs propagates events from Kinesis and DynamoDB Streams to other services across multiple accounts and regions.

While it’s a nice idea and def­i­nite­ly meets some spe­cif­ic needs, it’s worth bear­ing in mind the extra com­plex­i­ties it intro­duces, such as han­dling par­tial fail­ures, deal­ing with down­stream out­ages, mis­con­fig­u­ra­tions, and so on.

Conclusion

So what is the best event source for doing pub-sub with AWS Lamb­da? Like most tech deci­sions, it depends on the prob­lem you’re try­ing to solve, and the con­straints you’re work­ing with.

In this post, we looked at SNS, Kine­sis Streams, and DynamoDB Streams as can­di­dates for the bro­ker role. We walked through a num­ber of sce­nar­ios to see how the choice of event source affects scal­a­bil­i­ty, par­al­lelism, and resilience against tem­po­ral issues and cost.

You should now have a much bet­ter under­stand­ing of the trade­offs between various event sources when work­ing with Lamb­da.

Until next time!

Korean translation
Portuguese translation
Spanish translation
Mongolian translation
Persian translation
Vietnamese translation

Cryptography can be a hard subject to understand. It’s full of mathematical proofs. But unless you are actually developing cryptographic systems, much of that complexity is not necessary to understand what is going on at a high level.

If you opened this article hoping to create the next HTTPS protocol, I’m sorry to say that pigeons won’t be enough. Otherwise, brew some coffee and enjoy the article.

Alice, Bob and … pigeons?

Any activity you do on the Internet (reading this article, buying stuff on Amazon, uploading cat pictures) comes down to sending and receiving messages to and from a server.

This can be a bit abstract so let’s imagine that those messages were delivered by carrier pigeons. I know that this may seem very arbitrary, but trust me HTTPS works the same way, albeit a lot faster.

Also instead of talking about servers, clients and hackers, we will talk about Alice, Bob and Mallory. If this isn’t your first time trying to understand cryptographic concepts you will recognize those names, because they are widely used in technical literature.

A first naive communication

If Alice wants to send a message to Bob, she attaches the message on the carrier pigeon’s leg and sends it to Bob. Bob receives the message, reads it and it’s all is good.

But what if Mallory intercepted Alice’s pigeon in flight and changed the message? Bob would have no way of knowing that the message that was sent by Alice was modified in transit.

This is how HTTP works. Pretty scary right? I wouldn’t send my bank credentials over HTTP and neither should you.

A secret code

Now what if Alice and Bob are very crafty. They agree that they will write their messages using a secret code. They will shift each letter by 3 positions in the alphabet. For example D → A, E → B, F → C. The plain text message “secret message” would be “pbzobq jbppxdb”.

Now if Mallory intercepts the pigeon she won’t be able to change the message into something meaningful nor understand what it says, because she doesn’t know the code. But Bob can simply apply the code in reverse and decrypt the message where A → D, B → E, C → F. The cipher text “pbzobq jbppxdb” would be decrypted back to “secret message”.

Success!

This is called symmetric key cryptography, because if you know how to encrypt a message you also know how to decrypt it.

The code I described above is commonly known as the Caesar cipher. In real life, we use fancier and more complex codes, but the main idea is the same.

How do we decide the key?

Symmetric key cryptography is very secure if no one apart from the sender and receiver know what key was used. In the Caesar cipher, the key is an offset of how many letters we shift each letter by. In our example we used an offset of 3, but could have also used 4 or 12.

The issue is that if Alice and Bob don’t meet before starting to send messages with the pigeon, they would have no way to establish a key securely. If they send the key in the message itself, Mallory would intercept the message and discover the key. This would allow Mallory to then read or change the message as she wishes before and after Alice and Bob start to encrypt their messages.

This is the typical example of a Man in the Middle Attack and the only way to avoid it is to change the encryption system all together.

Pigeons carrying boxes

So Alice and Bob come up with an even better system. When Bob wants to send Alice a message she will follow the procedure below:

• Bob sends a pigeon to Alice without any message.
• Alice sends the pigeon back carrying a box with an open lock, but keeping the key.
• Bob puts the message in the box, closes the locks and sends the box to Alice.
• Alice receives the box, opens it with the key and reads the message.

This way Mallory can’t change the message by intercepting the pigeon, because she doesn’t have the key. The same process is followed when Alice wants to send Bob a message.

Alice and Bob just used what is commonly known as asymmetric key cryptography. It’s called asymmetric, because even if you can encrypt a message (lock the box) you can’t decrypt it (open a closed box).
In technical speech the box is known as the public key and the key to open it is known as the private key.

How do I trust the box?

If you paid attention you may have noticed that we still have a problem. When Bob receives that open box how can he be sure that it came from Alice and that Mallory didn’t intercept the pigeon and changed the box with one she has the key to?

Alice decides that she will sign the box, this way when Bob receives the box he checks the signature and knows that it was Alice who sent the box.

Some of you may be thinking, how would Bob identify Alice’s signature in the first place? Good question. Alice and Bob had this problem too, so they decided that, instead of Alice signing the box, Ted will sign the box.

Who is Ted? Ted is a very famous, well known and trustworthy guy. Ted gave his signature to everyone and everybody trusts that he will only sign boxes for legitimate people.

Ted will only sign an Alice box if he’s sure that the one asking for the signature is Alice. So Mallory cannot get an Alice box signed by Ted on behalf of her as Bob will know that the box is a fraud because Ted only signs boxes for people after verifying their identity.

Ted in technical terms is commonly referred to as a Certification Authority and the browser you are reading this article with comes packaged with the signatures of various Certification Authorities.

So when you connect to a website for the first time you trust its box because you trust Ted and Ted tells you that the box is legitimate.

Boxes are heavy

Alice and Bob now have a reliable system to communicate, but they realize that pigeons carrying boxes are slower than the ones carrying only the message.

They decide that they will use the box method (asymmetric cryptography) only to choose a key to encrypt the message using symmetric cryptography with (remember the Caesar cipher?).

This way they get the best of both worlds. The reliability of asymmetric cryptography and the efficiency of symmetric cryptography.

In the real world there aren’t slow pigeons, but nonetheless encrypting messages using asymmetric cryptography is slower than using symmetric cryptography, so we only use it to exchange the encryption keys.

Now you know how HTTPS works and your coffee should also be ready. Go drink it you deserved it ?