In this walkthrough, you'll first learn what Server-Side Request Forgery is and how it differs from Client-Side Request Forgery. We will create a sample application to gain a better understanding of how Server-Side Request Forgery attacks work, and explore various methods to safeguard our application against SSRF vulnerabilities.

Table of Contents:

• Prerequisites

• What is Server-Side Request Forgery?

• How Does SSRF Differ from CSRF?

• Identifying Code Smells

• Understanding the Pain Points

• Project Setup

• How to Exploit the Vulnerability

• How to Defend Against SSRF Attacks

• Wrapping Up

Prerequisites

1. Node and Express: We'll create a JavaScript sample application using the Express framework. A basic understanding of the framework would be helpful. You will need the Node Runtime Environment to execute the scripts.

2. Postman Client: To make an API request and to exploit the vulnerability, you will need a tool to make HTTP Requests. You may use your web browser's "Edit and Send" feature under the Networks tab, but since not all browsers allow this, it's best to use a tool like Postman which provides a better UI to observe responses.

What is Server-Side Request Forgery?

Server-Side Request Forgery, or SSRF, is a security vulnerability that allows malicious actors to manipulate the server into making unintended requests on behalf of the server itself.

SSRF provides a window for such malicious actors to make requests "from" the server when they should be making requests "to" the server.

To appreciate what this means, let's look at a normal request execution using the sequence diagrams below:

UML Sequence Diagram for normal request execution

In a typical scenario, a server processes incoming requests from clients. Users or external systems initiate these requests, and the server responds accordingly. This is a standard client-server interaction where the server acts upon the requests it receives.

Now let's look at what SSRF looks like:

UML Sequence Diagram for SSRF attacks

In applications vulnerable to SSRF, attackers exploit the server's ability to make HTTP requests to resources that should not be directly accessible from the public internet. These resources may include internal protected resources, APIs, websites, or databases that can only be accessed from the server.

Attackers achieve this by tricking the server into making unintended requests to various destinations, including internal APIs, internal HTML pages, and internal databases.

How Does SSRF Differ from CSRF?

SSRF is an attack where an attacker can make the server perform requests on their behalf. This involves manipulating the server to make requests to internal resources, which can result in unauthorized actions or information disclosure.

On the other hand, in CSRF, or Client-Side Request Forgery, the attacker tricks a user's browser into making unintended requests to a specific web application for which the user is already authenticated. This means that actions are performed on behalf of the user without their consent.

Backend Developers must be aware of SSRF to make secure applications. In contrast, front-end developers must be mindful of and implement client-side security measures to prevent CSRF attacks.

Identifying Code Smells

SSRF attacks often occur when web applications improperly mishandle user-controlled input, leading to network requests based on inadequately sanitized user input. Processing un-sanitized URLs in API requests is a common entry point for SSRF attacks.

Another common giveaway to identifying SSRF vulnerabilities in your applications is to check for instances where XML parsing occurs without adequate validation of external entities. Applications that fail to validate and secure their XML parsers properly may inadvertently expose themselves to SSRF risks.

In this walkthrough, you will make a server that takes a URL and uses it to make network requests without proper validation and sanitization. You will then see ways to mitigate this issue.

Understanding the Pain Points

To better understand the issue of SSRF attacks, lets create a sample application using Express and JavaScript. Below is a Mermaid Sequence Diagram where we explain what the code base does:

UML Sequence Diagram for the sample application

We will create an Express app with two endpoints — /fetch, a GET request designed to fetch content from a specified URL, and /admin, another GET request, which is an internal API within the organization that accesses an internally protected resource.

We will discover a security vulnerability associated with Server-Side Request Forgery (SSRF) in implementing the first GET request.

We will also create another helper function at the /uploads endpoint to allow our clients to fetch and view their recently uploaded content.

Project Setup

To get started, let's quickly set up our repository and install all the required packages. In the root of your workspace, install Express and Axios using the following command:

npm init -y | npm i axios express

Executing this command will create a package.json file with default settings and install the specified packages.

To simulate the internal protected resource, let's create a data.json in the root of your workspace:

{   
    "name": "Hamdaan Ali Quatil",
    "password": "violinblackeye"
}

Now, create a file called app.js in the root of your repository. Here, we will define all of our endpoints. Import all required packages like this:

const express = require('express');
const axios = require('axios');
const fs = require('fs').promises;

We use the fs (File System) module to interact with the local file system. Within the Express application, we use fs.promises to read the contents of a file. The fetchPrivateResource function asynchronously reads the contents of the data.json file, which is an internal resource.

Let's create an instance of the Express app to handle HTTP requests and define the fetchPrivateResource method. In the sample application, only the admin should be able to fetch this internal resource, but you will observe how a malicious actor can access this using an SSRF attack.

const app = express();
const port = 3000;

// Function to fetch private resource
const fetchPrivateResource = async () => {
  try {
    const content = await fs.readFile('data.json', 'utf-8');
    return content;
  } catch (error) {
    console.error('Error reading private resource:', error.message);
    throw error;
  }
};

The Fetch Endpoint

Now, let's define our first endpoint, /fetch which expects a query parameter url containing the target URL. Upon receiving a request, the server uses the Axios library to make a GET request to the specified URL.

app.get("/fetch", async (req, res) => {
  const url = req.query.url;

  try {
    const response = await axios.get(url);
    const responseData = JSON.stringify(response.data);

    const filename = path.basename(url);
    const textFilePath = path.join(__dirname, "uploads", "upload-data.txt");

    await fs.writeFile(textFilePath, responseData, "utf-8");

    res.send("Upload Successful");
  } catch (error) {
    console.error("Error:", error.message);
    res.status(500).send("Internal Server Error");
  }
});

The axios.get method is used to perform the HTTP GET request, and the response data is then converted to a JSON string. The resulting string is written to a text file named upload-data.txt in the uploads folder of the server. Finally, a success message or an error message is sent back to the client, depending on the outcome of the operation.

The Uploads Endpoint

With that done, let's create an endpoint to allow users to access and verify their uploaded files. The server will check if the requested file exists, and if so, it sends the file to the client. When a file cannot be found, the server returns a 404 error.

app.get("/uploads/:filename", async (req, res) => {
  const filename = req.params.filename;
  const filePath = path.join(__dirname, "uploads", filename);
  console.log(filePath);

  try {
    // Check if file exists
    await fs.access(filePath);

    // If file exists, send it to the client
    res.sendFile(filePath);

  } catch (error) {
    res.status(404).send("File not found: " + error);
  }
});

The Admin Endpoint

Now, we need to make an internal API – the /admin route – which is intentionally shielded from public access. The objective is to ensure this API is only accessible from localhost or the local machine (127.0.0.1).

We can do this by implementing a middleware that acts as a protective barrier, permitting requests to proceed to the /admin route only if they originate from the local host.

The middleware checks whether the req.hostname property, which represents the hostname specified in the HTTP request, matches localhost or 127.0.0.1. If the request is from a different host, the middleware responds with a 403 Forbidden status, thereby restricting access.

// middleware to protect admin API
app.use('/admin', async (req, res, next) => {
  const isLocalhost = req.hostname === 'localhost' || req.hostname === '127.0.0.1';

  if (isLocalhost) {
    next();
  } else {
    res.status(403).send('Forbidden');
  }
});

// Route to access the admin API
app.get('/admin', async (req, res) => {
  try {
    const content = await fetchPrivateResource();
    res.send(content);
  } catch (error) {
    res.status(500).send('Internal Server Error');
  }
});

Once all routes are configured, we start the server using the app.listen method, and it begins listening on port 3000 for incoming requests.

app.listen(port, () => {
  console.log(`Server is running on http://localhost:${port}`);
});

With our app.js now set up to process incoming requests, let's run the sample application using nodemon:

npm i -D nodemon | nodemon app.js

The server has started on the port 3000. Now, we are ready to test our sample application and look for code smells that may lead to SSRF attacks. You may find the complete code here — GitHub Gist | HamdaanAliQuatil.

How to Exploit the Vulnerability

Let's try to make a GET request to the fetch API. We are simulating the process of uploading a text file using the URL to the file. In this demonstration, we will fetch the contents of an example file and save it on our servers. Here is the link to the text file.

Open your Postman Client and execute a GET request with the URL http://localhost:3000/fetch?url=https://example-files.online-convert.com/document/txt/example.txt. We are adding the link to the file as a Query Parameter in the /fetch endpoint. When you hit send, you will see a response "Upload Successful".

Postman Client: Fetch Endpoint

You'll see that your repository now has a newly created file in the uploads directory. Clients can now access their uploaded information using the /uploads API endpoint to view their files.

Postman Client: Uploads Endpoint

Now, let's send a malicious request by changing our Query param to http://120.0.07/admin in the same request to the /fetch endpoint. The updated URL will now look like this: http://localhost:3000/fetch?url=http://127.0.0.1:3000/admin.

In the Query parameter, 127.0.0.1 is a Loopback Address. A loopback address is a reserved IP address used to establish network connections with the same host (the local machine) for testing and communication within the device.

The malicious actor is attempting is to make a request to the server's /admin route from the server itself using the loopback address. This simulates an internal resource access scenario.

Postman Client: Admin Endpoint

You'll notice that an "Upload Successful" message comes as a response to this request. Now try accessing your uploaded file again using the GET request at the /upload endpoint.

Postman Client: Uploads Endpoint

You'll see that the contents of the uploaded file have been altered. This alteration highlights a successful SSRF (Server-Side Request Forgery) attack, where a malicious actor took advantage of the server's capability to initiate internal requests.

The file, which initially contained specific data, has now been tampered with. This showcases the potential for unauthorized access and manipulation of sensitive information through SSRF exploits.

How to Defend Against SSRF Attacks

Now, let’s see the ways in which we can fix our application's vulnerability to SSRF. The most intuitive solution that comes to your mind could be to never allow a client to enter a URL. This is certainly the most powerful defense. The server should create a URL it needs.

But many times, allowing URLs in your business logic becomes an absolute necessity. In such cases, our goal is to prevent the attack or at least reduce the risk if an attack occurs.

If you really must allow a URL as it is, here are some precautionary steps you can take:

Sanitization and Validation

As with most vulnerabilities, a pain-point in SSRF attacks is the use of untrusted data. Always treat any data coming from the client side as untrusted.

Sanitizing and validating the client-supplied data should go a long way to defend against SSRF attacks. A very intuitive validation is to restrict any URL containing localhost or the loopback address.

Let's create a helper function isValidUrl and call it in the function for the /fetch endpoint.

function isValidUrl(url) {
  // Restrict URLs to HTTP only. This blocks FTP and other protocols
  const validUrlRegex = /^http:\/\/\S+$/;

  if (!validUrlRegex.test(url)) {
    return false;
  }

  try {
    const parsedUrl = new URL(url);

    // Check if the host is localhost or a loopback IP address
    const isLocalhost = parsedUrl.hostname === 'localhost';
    const isLocalIP = /^127\.\d+\.\d+\.\d+$/g.test(parsedUrl.hostname);

    return !(isLocalhost || isLocalIP);
  } catch (error) {
    return false;
  }
}

Your updated function for /fetch endpoint should look like this:

app.get("/fetch", async (req, res) => {
  const url = req.query.url;

  if (!isValidUrl(url)) {
    res.status(400).send("Loopback URLs are not allowed");
    return;
  }

  try {
    ...
    res.send("Upload Successful");
  } catch (error) {
    ...
  }
});

Now, go back to the Postman Client and resend the malicious request. You will observe that previously uploaded file is not tampered and you receive "Loopback URLs are not allowed" in the response.

Whitelisting via an Allow List

You may create a positive allow list to only allow certain trusted IP Addresses, URL Schema, and Port. Let's implement an allow list and improve our isValidUrl function:

const whitelist = ["boost.com", "boost.in", "trustedDomain3.com"];
const allowedPorts = ['80', '443'];

Now use your declared whitelist in the isValidUrl function:

function isValidUrl(url) {
  try {
    const parsedUrl = new URL(url);

    if (!whitelist.includes(parsedUrl.hostname)) {
      return false;
    }

    if (!allowedPorts.includes(parsedUrl.port)) {
      return false;
    }

    return true;
  } catch (error) {
    return false;
  }
}

Notice how we've removed the need for regex. This brings us to another mitigation technique that you must avoid:

Don't Use a Deny List

You must never mitigate SSRF vulnerabilities using a deny list or regex. Restricting the use of IP Addresses is not straightforward. To understand why we must avoid a deny list, look at the following example.

A Loopback Address is typically represented using 127.0.0.1 . Its quite easy to spot this address and reject it. But a problem arises when a malicious request is sent using any other forms of this Loopback address that also points to the local machine. For example, 127.1, ::1, localhost ,::ffff:7f00:1 all point to the local machine.

A regular expression to spot all such variations is much more complex. Malicious actors can easily bypass a deny list by passing an octal representation of decimal encoding of the IP address.

Enforce a URL Scheme

In absence of this measure, a client might send requests that use any protocols other than the intended ones. To replace our validUrlRegex, we will use a allowedSchemes list. We will restrict our application to only process requests when the protocols are either https: or http. Not allowing any requests with protocols file: and ftp: will safe-guard our sample application.

const allowedSchemes = ['http:', 'https:'];

The updated isValidUrl function will look like this:

function isValidUrl(url) {
  try {
    const parsedUrl = new URL(url);

    if (!whitelist.includes(parsedUrl.hostname)) {
      return false;
    }

    if (!allowedPorts.includes(parsedUrl.port)) {
      return false;
    }

    if (!allowedSchemes.includes(parsedUrl.protocol)) {
      return false;
    }

    return true;
  } catch (error) {
    return false;
  }
}

Disable Redirects

Redirects are a mechanism used by web applications to forward a user's browser from one URL to another. If a server follows redirects automatically, an attacker could exploit this behavior to make the server inadvertently access internal resources, leading to data exposure or unauthorized actions.

To restrict redirects in Axios, pass in an Axios Configuration object in the second parameter:

const response = await axios.get(url, { maxRedirects: 0 });

To learn more about Axios Config, check this guide: Axios | Request Config.

Send Filtered Data to the Client

Avoid sending raw response bodies directly from your server to the client. Ensure that the responses reaching the client are carefully curated and conform to expected formats.

By implementing this practice, you shield your application from potential security vulnerabilities associated with exposing unfiltered or sensitive information. Always validate, filter, and format responses to align with your application's anticipated data structures.

Wrapping Up

And there you have it: by implementing a few well-established methodologies and best practices, you can effectively detect and mitigate SSRF attacks in your applications and create secure APIs as developers.

Find the complete code snippets here — GitHub Gist | HamdaanAliQuatil.
You may find me on X (formerly Twitter) - Hamdaan Ali Quatil.

But what exactly are full-stack engineers? What do they do on a day-to-day basis? And how can you become a full-stack engineer yourself?

In this article, I will go over the definition of full-stack engineering and cover some of the tasks that full-stack engineers work on. I will also mention some of the skills you will need to learn to become a software engineer.

What Is A Full Stack Engineer? Definition of Full-Stack Engineering

Modern web applications consist of two layers: the front-end and the back-end.

The front-end, also known as the client side, consists of the content, the presentation and layout of that content, and elements of interactivity. It includes all the visible parts a user views on a screen and can interact with.

The back-end, also known as the server side, consists of a server running the code with the necessary logic to receive, handle, and process requests, as well as a database for storing user data securely. It includes all the behind-the-scenes processes a user is not directly aware of.

A stack is a collection of technologies and refers to the combination of software, tools, programming languages, frameworks, and data storage technologies that work together to build and run web applications.

There are many technology stacks, and each stack uses the same programming language throughout.

One of the most popular stacks for the JavaScript language is the MERN stack, which stands for MongoDB, Express, React, and NodeJs.

MongoDB is a document database, Express is a back-end web application framework for Node.js, React is a front-end JavaScript library, and NodeJS is a back-end JavaScript runtime environment.

A full-stack engineer is an engineer that knows how to build a web application from start to finish, including the front-end parts, the back-end parts, and the infrastructure it lives on. They work across the entire stack and understand each part of it.

What Does A Full Stack Engineer Do? Tasks and Responsibilities of Full Stack Engineers

Full-stack engineers work on a broad set of problems and typically have end-to-end ownership of projects – from conceptualization to deployment.

Some of the everyday responsibilities may include:

• Gather project requirements by communicating with clients and stakeholders to understand the vision of the software product.
• Brainstorm and collaborate with the design team and review design prototypes before coding them into a product.
• Solve HTML, CSS, and JavaScript issues for the client-facing side of the application and work with front-end frameworks used within the application.
• Ensure the web application is responsive for end-users and works on most devices.
• Follow accessibility best practices.
• Manage and maintain databases and servers to ensure the client-side works efficiently and optimally.
• Ensure application security, maintenance, performance, uptime, and scalability.
• Monitor software and write tests to ensure the code is working as intended. Find and fix bugs in the code to keep the software optimized.
• Write clean, well-designed, and efficient code that adheres to industry best practices.
• Keep up to date with new technological advances to improve business needs.
• Create a Minimal Viable Product to showcase to stakeholders and communicate with decision-makers.
• Implement new features after gathering feedback.
• Review code written by other engineers and provide constructive feedback.
• Read and write documentation that outlines the software development process.

With that said, the tasks and responsibilities will depend on the size of the company.

For example, a small company may have only one full-stack developer who handles the entire application. A larger company may have front-end and back-end developers to work on specific tasks within their domain and expertise.

What Is the Average Salary for Full Stack Engineers?

Software engineers, including full-stack engineers, generally command comfortable salaries.

According to the U.S Bureau of Labor Statistics, the median salary for software engineers is $109,020 per year.

Specifically for full-stack engineers, Glassdoor lists the average salary at around $120,300 per year.

Indeed lists the average salary as $120,749. And the Stack Overflow developer survey lists that the average salary for full-stack developers is $140,000.

Note that these salaries are for full-stack engineers based in the USA. The compensation will depend on your location and years of experience.

Google the average salary for software engineers in your location with the same experience as you to help you get a better idea.

How To Become A Software Engineer – Skills Required for Full Stack Engineers

In the following sections, I will go over just a few of the technologies you would need to learn to become a full-stack engineer.

Becoming a full-stack engineer takes time, and you can't be an expert in all the tools and technologies available – it is more about having well-rounded knowledge of technology in general and knowing enough to solve problems using code.

Learn Internet and Web Fundamentals

As a full-stack engineer, you will benefit from understanding how the internet works and being familiar with some web terms, such as DNS and IP addresses.

To learn more about DNS, IP addresses, and how the internet works, check out the following resources:

• How the Web Works: A Primer for Newcomers to Web Development (or anyone, really)
• How the Web Works Part II: Client-Server Model & the Structure of a Web Application
• What is DNS? Domain Name System, DNS Server, and IP Address Concepts Explained

You will also need to know HTTP (HyperText Transfer Protocol), the foundation of the World Wide Web, as it manages the communication between clients (such as web browsers) and servers on the web.

Specifically, you need to know about HTTP request methods (such as GET, POST, PUT, PATCH, and DELETE) and HTTP response codes (such as 200, 404, and 500).

To learn more about HTTP, check out the following resources:

• What is HTTP? Protocol Overview for Beginners
• How HTTP Works and Why it's Important – Explained in Plain English
• Master the HTTP Networking Protocol
• HTTP Request Methods – Get vs Put vs Post Explained with Code Examples

Learn Front-end Web Development Fundamentals

Only three front-end web languages run in all modern web browsers: HTML, CSS, and JavaScript.

HTML (short for HyperText Markup Language) defines the structure and content on a web page, such as text, links, forms, images, videos, and so on.

To learn HTML, check out this course.

CSS (short for Cascading Style Sheets) styles the HTML content and makes it aesthetically pleasing – it determines the look and feel of a web page.

CSS is responsible for sizing, displaying, laying out, and presenting elements on a page. CSS is also responsible for web pages being usable on all screen sizes.

To learn more about CSS, check out this course, which covers Flexbox and Grid – two important CSS topics. Once you understand the basics, you could learn a CSS framework, like Tailwind CSS.

JavaScript is a dynamic scripting programming language designed to run in the browser.

It is the only programming language you can use for front-end web development, making it an essential part of web development. It is used together with HTML and CSS to create interactive web pages.

To learn more about JavaScript, checkout out this course.

Learn Git and GitHub

Git and GitHub are a core part of the development workflow and are tools used in every software development job.

Git is a distributed version control system that offers a way to make changes to your projects, back up those changes, keep track of them, and even go back to them if needed. It also lets you collaborate with other team members at the same time.

GitHub is an online hosting service that makes it easier to use Git and is a place for you and your team to upload and review code.

To learn more about Git and GitHub, check out this course.

Learn A Front-end Library and Framework

Once you understand the core concepts of JavaScript, you can move on to learning one of JavaScript's front-end libraries and frameworks.

A front end library is pre-written reusable code containing various functions, methods, or objects you can use in your JavaScript project to perform tasks and implement specific functionalities. And a web framework is a tool that makes it easier and faster to create web applications.

According to the Stack Overflow Survey for 2022, the most popular and commonly used JavaScript library to learn is ReactJS.

To learn more about ReactJS, check out this course.

With that said, there are other frameworks to work with and consider learning, such as Vue, Angular and Svelte. Each has its way of organizing and writing code, as well as its own benefits and limitations.

Make sure you have a solid foundation in JavaScript before learning these tools.

Learn Back-end Web Development

As a full-stack web developer, you need to know front-end technologies and back-end tools, so you will also need to be able to work with server-side scripting programming languages.

There are many to choose from, such as Python, Ruby, and Java, to name a few.

And although JavaScript is used widely in front-end development, in recent years, it is also used for back-end web development with the help of NodeJS.

NodeJS is a JavaScript runtime that provides back-end functionality and is designed to build dynamic scalable web applications.

First, learn how to use NPM, a Node package manager for installing and managing local dependencies for JavaScript packages. It is designed specifically for use with NodeJS.

To learn the basics of NodeJS, check out this course.

You can pair NodeJS with the ExpressJS server-side web framework to create full-stack web applications. To learn back-end development with NodeJS and ExpressJS, check out this course.

Learn Database Management Systems and SQL

As a full-stack engineer, you will be working with databases as most web applications you will be working with will have a database. A lot of your time will be spent writing database queries to fetch the data you need.

A database is a storage container – a place to store all the data used in your project, such as user data.

There are two main types of databases:

• SQL or Relational databases, also referred to as SQL databases, which store data in a structured, organized, tabular format.
• Non-relational or NoSQL databases, which don't store data in tables.

A database has a program called a database management system (DBMS), which serves as an interface between the database, allowing users or programs to retrieve, update, and manage the data.

To learn about Relational Databases, check out freeCodeCamp's Relational Database Course.

To communicate with relational databases and manipulate stored data, you query them using a query language such as SQL (short for Structured Query Language). To learn more about SQL, check out this resource

And if you want to get started with a NoSQL database management system, MongoDB is a great place to start. To get started with MongoDB, check out these courses.

Conclusion

Hopefully, you found this article helpful and have a better understanding of what full-stack engineers do.

In this article, we went over the definition of full-stack engineering and covered some of the tasks full-stack engineers work on.

We also covered some of the tools and languages you would need to know to become a full-stack engineer yourself.

Thank you for reading!

Hello folks! 👋 Recently, I have been learning about Node.js. So I decided to share my learnings with you here. 👨‍💻

In this tutorial, we'll take a high-level look at Node.js – what it is, and what you can do with it.

We will be covering all the important Node concepts with hands-on examples and a lot of code snippets. This will give you the foundational knowledge required to get started with backend development with NodeJS.

This article is heavily inspired by the NodeJS and Express - Full Course on freeCodeCamp taught by John Smilga. You can check that out here if you'd like.

Here's what we'll cover in this guide:

• What Is NodeJS?
• Global Variables
• Modules
• Short Note On module.exports
• Types Of Modules
• Event Driven Programming
• Creating our first server
• Let's Serve Something Interesting!
• Conclusion

What is Node?

Node is an environment in which you can run JavaScript code "Outside the web browser". Node be like – "Hey y'all, you give your JS code to me and I'll run it 😎". It uses Google's V8 Engine to convert the JavaScript code to Machine Code.

Since Node runs JavaScript code outside the web browser, this means that it doesn't have access to certain features that are only available in the browser, like the DOM or the window object or even the localStorage.

This means that at any point in your code, you can't type in document.querySelector() or alert() as these will produce errors (This is what is shown in the below image).

Remember: Node is meant for server-side programming, while those browser features are meant for client-side programming.

Front-end folks don't be sad – there's more to it! Node provides you with lots of API's and Modules with which you can perform a variety of operations like File Handling, Creating Servers, and much more. Before diving into the NodeJS, first let's install it in our machine.

How to Install NodeJS

Installing NodeJS is straightforward. If you already have Node installed in your machine, you can skip this section. If not, then follow along.

Here are the steps to download NodeJS on your machine:

1. Navigate to https://nodejs.org/
2. Download the LTS Version of NodeJS for your operating system
3. Run the installer and follow the installation wizard. Simply answer Yes to all the questions.
4. Once the installation is complete, open a new terminal or command prompt window and run the following command to verify that NodeJS is installed correctly: node -v. If you see the version of NodeJS printed in your terminal, Congratulations! You have now successfully installed NodeJS on your machine.

Note: If you encounter any issues during the installation process, you can refer to the official NodeJS documentation for more detailed instructions and troubleshooting tips.

Global Variables

Let's start this article by learning about some variables present in NodeJS called Global Variables. These are basically variables which store some data and can be accessed from anywhere in your code – doesn't matter how deeply nested the code is.

You should know about these commonly used Global variables:

• __dirname: This variable stores the path to the current working directory.
• __filename: This variable stores the path to the current working file.

Let's use them and see what value they contain. For this, let's create a new folder called NodeJSTut in your Desktop and open it up with your favorite text editor (In the entire tutorial, we will be using VS Code). Create a new file called app.js and open up a new integrated VS Code Terminal.

Paste the following code in the app.js file and save it:

// __dirname Global Variable
console.log(__dirname);

// __filename Global Variable
console.log(__filename);

To run this code using Node, type in the following command in the terminal and press Enter: node app.js. You will see the absolute path to the present working directory and the path to the current file is printed in the terminal. This is what the output looks like in my case:

C:\Desktop\NodeJSTut
C:\Desktop\NodeJSTut\app.js

You can go ahead and create your own global variables which can be accessed from anywhere in your code. You can do so, like this:

// Define a global variable in NodeJS
global.myVariable = 'Hello World';

// Access the global variable
console.log(myVariable); // Output: Hello World

Modules in NodeJS

In Node.js, a module is essentially a reusable block of code that can be used to perform a specific set of tasks or provide a specific functionality. A module can contain variables, functions, classes, objects, or any other code that can be used to accomplish a particular task or set of tasks.

The primary purpose of using modules in Node.js is to help organize code into smaller, more manageable pieces. A modules can then be imported at any time and used flexibly which helps in creating reusable code components that can be shared across multiple projects.

To understand this, consider this example: Let's say you have defined lots of functions in your code that works with a huge volume of JSON data.

Losing your sleep and increased anxiety levels are common side effects of keeping all this stuff (functions + data + some other logic) in one single file.

So you, being a clever programmer, thought of making a separate file for the JSON data and a separate file for storing all the functions. Now, you can simply import the data and the functions whenever you want and use them accordingly. This method increases efficiency as your file size reduces drastically. This is the concept of modules!

Let's see how we can make our own modules. For this, we are going to write some code where we will be defining a function called sayHello() in a file called hello.js. This function will accept a name as the parameter and simply print a greeting message in the console.

We will then import it in another file called app.js and use it there. How interesting, right 😂? Let's check out the code:

This is the code in hello.js file:

function sayHello(name){
    console.log(`Hello ${name}`);
}

module.exports = sayHello

This is the code in app.js file:

const sayHello = require('./hello.js');

sayHello('John');
sayHello('Peter');
sayHello('Rohit');

The file hello.js can be called the module in this case. Every module has an object called exports which should contain all the stuff you want to export from this module like variables or functions. In our case, we are defining a function in the hello.js file and directly exporting it.

The app.js file imports the sayHello() function from hello.js and stores it in the sayHello variable. To import something from a module, we use the require() method which accepts the path to the module. Now we can simply invoke the variable and pass a name as a parameter. Running the code in app.js file will produce the following output:

Hello John
Hello Peter
Hello Rohit

Short Note on module.exports

In the previous section of the article, we saw how to use module.exports but I felt that it is important to understand how it works in a bit more detail. Hence, this section of the article is like a mini tutorial where we will see how we can export one variable/function as well as multiple variables and functions using module.exports. So, Let's get started:

module.exports is a special object in NodeJS that allows you to export functions, objects, or values from a module, so that other modules can access and use them. Here's an example of how to use module.exports to export a function from a module:

// myModule.js

function myFunction() {
  console.log('Hello from myFunction!');
}

module.exports = myFunction;

In this example, we define a function myFunction and then export it using module.exports. Other modules can now require this module and use the exported function:

// app.js

const myFunction = require('./myModule');

myFunction(); // logs 'Hello from myFunction!'

Everything seems fine now and life is good. But the problem arises when we have to export multiple functions and variables from a single file. The point is when you use module.exports multiple times in a single module, it will replace the previously assigned value with the new one. Consider this code:

// module.js

function myFunction() {
  console.log('Hello from myFunction!');
}

function myFunction2() {
  console.log('Hello from myFunction2!');
}

// First Export
module.exports = myFunction;

// Second Export
module.exports = myFunction2;

In this example, we first export myFunction(). But we then overwrite module.exports with a new function - myFunction2(). As a result, only the second export statement will take effect, and the myFunction() function will not be exported.

This problem can be solved if you assign module.exports to an object which contains all the functions you want to export, like this:

// myModule.js

function myFunction1() {
  console.log('Hello from myFunction1!');
}

function myFunction2() {
  console.log('Hello from myFunction2!');
}

module.exports = {
  foo: 'bar',
  myFunction1: myFunction1,
  myFunction2: myFunction2
};

In this example, we export an object with three properties: foo, myFunction1, and myFunction2. Other modules can require this module and access these properties:

// app.js

const myModule = require('./myModule');

console.log(myModule.foo); // logs 'bar'
myModule.myFunction1(); // logs 'Hello from myFunction1!'
myModule.myFunction2(); // logs 'Hello from myFunction2!'

To summarize, you can use module.exports as many times as you want in your NodeJS code, but you should be aware that each new assignment will replace the previous one. You should use an object to group multiple exports together.

Types Of Modules in Node

There are 2 types of modules in NodeJS:

• Built In Modules: These are modules included in Node by default, so you can use them without installation. You just need to import them and get started.
• External Modules: These are modules created by other developers which are not included by default. So you need to install them first before using them.

Here is an image of popular built-in modules in NodeJS and what can you do using them:

Let's go over each of these in more detail so you can learn more about what they do.

The OS Module

The OS Module (as its name implies) provides you methods/functions with which you can get information about your Operating System.

To use this module, the first step is to import it like this:

const os = require('os');

This is how you can use the OS Module to get information about the Operating System:👇

const os = require('os')

// os.uptime()
const systemUptime = os.uptime();

// os.userInfo()
const userInfo = os.userInfo();

// We will store some other information about my WindowsOS in this object:
const otherInfo = {
    name: os.type(),
    release: os.release(),
    totalMem: os.totalmem(),
    freeMem: os.freemem(),
}

// Let's Check The Results:
console.log(systemUptime);
console.log(userInfo);
console.log(otherInfo);

This is the output of the above code:

Note that the output shows information about the Windows Operating System running on my system. The output could be different from yours.

521105
{
    uid: -1,
    gid: -1,
    username: 'krish',
    homedir: 'C:\\Users\\krish',
    shell: null
}
{
    name: 'Windows_NT',
    release: '10.0.22621',
    totalMem: 8215212032,
    freeMem: 1082208256
}

Let's break down the above code and output:

• os.uptime() tells the system uptime in seconds. This function returns the number of seconds the system has been running since it was last rebooted. If you check the first line of the output: 521105 is the number of seconds, my system has been running since it was last rebooted. Of course, it will be different for you.
• os.userInfo() gives the information about the current user. This function returns an object with information about the current user including the user ID, group ID, username, home directory, and default shell. Below is the breakdown of the output in my case:

    {
        uid: -1,
        gid: -1,
        username: 'krish',
        homedir: 'C:\\Users\\krish',
        shell: null
    }

The uid and gid is set to -1 in Windows, because Windows does not have the concept of user IDs like Unix-based systems. The username of my OS is krish and the home directory is 'C:\\Users\\krish'. The shell is set to null because the concept of a default shell does not exist on Windows. Windows has a default command interpreter program called Command Prompt (cmd.exe), which runs commands and manages the system.

The other methods related to OS Module like os.type(), os.release() and so on, which you saw in the above code has been used within the otherInfo object. Here is a breakdown of what these methods do:

• os.type() - Tells the name of the Operating System
• os.release() - Tells the release version of the Operating System
• os.totalMem() - Tells the total amount of memory available in bytes
• os.freeMem() - Tells the total amount of free memory available in bytes

This is the information which the above methods display about my OS:

{
    name: 'WindowsNT', // Name of my OS
    release: '10.0.22621', // Release Version of my OS
    totalMem: 8215212032, // Total Memory Available in bytes (~ 8 GB)
     freeMem: 1082208256 // Free Memory Available in bytes (~ 1 GB) 
}

The PATH Module

The PATH module comes in handy while working with file and directory paths. It provides you with various methods with which you can:

• Join path segments together
• Tell if a path is absolute or not
• Get the last portion/segment of a path
• Get the file extension from a path, and much more!

You an see the PATH Module in action in the code below.

Code:

// Import 'path' module using the 'require()' method:
const path = require('path')

// Assigning a path to the myPath variable
const myPath = '/mnt/c/Desktop/NodeJSTut/app.js'

const pathInfo = {
    fileName: path.basename(myPath),
    folderName: path.dirname(myPath),
    fileExtension: path.extname(myPath),
    absoluteOrNot: path.isAbsolute(myPath),
    detailInfo: path.parse(myPath),
}

// Let's See The Results:
console.log(pathInfo);

Output:

{
  fileName: 'app.js',
  folderName: '/mnt/c/Desktop/NodeJSTut',
  fileExtension: '.js',
  absoluteOrNot: true,
  detailInfo: {
    root: '/',
    dir: '/mnt/c/Desktop/NodeJSTut',
    base: 'app.js',
    ext: '.js',
    name: 'app'
  }
}

Let's have a detailed breakdown of the above code and its output:

The first and foremost step to work with path module is to import it in the app.js file using the require() method.

Next, we are assigning a path of some file to a variable called myPath. This can be a path to any random file. For the purpose of understanding the path module, I chose this: /mnt/c/Desktop/NodeJSTut/app.js.

Using the myPath variable, we will understand the path module in detail. Let's check out the functions which this module has to offer and what can we do with it:

• path.basename(myPath): The basename() function accepts a path and returns the last part of that path. In our case, the last part of myPath is: app.js.
• path.dirname(myPath): The dirname() function selects the last part of the path provided to it and returns the path to it's parent's directory. In our case, since the last part of myPath is app.js. The dirname() function returns the path to the parent directory of app.js (the folder inside which app.js file lies), i.e, /mnt/c/Desktop/NodeJSTut. It can be also thought as: the dirname() function simply excludes the last part of the path provided to it and returns the leftover path.
• path.extname(myPath): This function checks for any extension on the last part of the provided path and it returns the file extension (if it exists), otherwise it returns an empty string: ''. In our case, since the last part is app.js and a file extension exists, we get '.js' as the output.
• path.isAbsolute(myPath): This tells whether the provided path is absolute or not. On Unix-based systems (such as macOS and Linux), an absolute path always starts with the forward slash (/). On Windows systems, an absolute path can start with a drive letter (such as C:) followed by a colon (:), or with two backslashes (\\). Since the value stored in myPath variable starts with /, therefore isAbsolute() returns true.

However, if you just change the myPath variable to this: Desktop/NodeJSTut/app.js (converting it to a relative path), isAbsolute() returns false.

• path.parse(myPath): This function accepts a path and returns an object which contains a detailed breakdown of the path provided to it. Here is what it returns when we provide the myPath variable to it:
• root: The root of the path (in this case, /).
• dir: The directory of the file (in this case, /mnt/c/Desktop/NodeJSTut).
• base: The base file name (in this case, app.js).
• ext: The file extension (in this case, .js).
• name: The base name of the file, without the extension (in this case, app).

Before continuing with the other functions of the path module, we need to understand something called path separator and the path structure.

You must have seen that the path to a same file looks different in different Operating Systems. For example, consider the path to a file named example.txt located in a folder called Documents on the desktop of a Windows user:

C:\Users\username\Desktop\Documents\example.txt

On the other hand, the file path to the same file for a user on a macOS system would look like this:

/Users/username/Desktop/Documents/example.txt

2 differences are to be noted here:

1. Difference in path separators: In Windows, file paths use the backslash (\) as the separator between directories, while in macOS/Linux (which is a Unix-based system), file paths use the forward slash (/) as the separator.
2. Difference in root directory of the users files: On Windows, the root directory for a user's files is commonly found at C:\Users\username, whereas on macOS and Linux, it is located at /Users/username/. While this holds true for most Windows, macOS, and Linux systems, there may be some variations in the exact location of the user's home directory based on the system's configuration.

With this in mind, let's move ahead and understand some other functions provided by the path module:

• path.sep: sep is a variable which contains the system specific path separator. For Windows machine: console.log(path.sep) prints \ in the console while in case of macOS or Linux, path.sep returns a forward slash ( / ).
• path.join(<paths>): The path.join() function accepts path(s) as strings. It then joins those paths using the system specific path separator and returns the joined path. For example, consider this code:

console.log(path.join('grandParentFolder', 'parentFolder', 'child.txt'))

The above code prints different results for different Operating Systems.
In Windows, it will give this output: grandParentFolder\parentFolder\child.txt while in macOS/Linux, it will give this output: grandParentFolder/parentFolder/child.txt. Note that the difference is only in the path separators - backward slash and forward slash.

• path.resolve(<paths>): This function works in a similar way as compared to path.join(). The path.resolve() function just joins the different paths provided to it using the system specific path separator and then appends the final output to the absolute path of the present working directory.

Suppose you are a Windows user and the absolute path to your present working directory is this: C:\Desktop\NodeJSTut, If you run this code:

console.log(path.resolve('grandParentFolder', 'parentFolder', 'child.txt'));

You will see the following output in the console:

C:\Desktop\NodeJSTut\grandParentFolder\parentFolder\child.txt

The same is applicable to a macOS or a Linux user. It's just the difference in the absolute path of the present working directory and the path separator.

The FS Module

This module helps you with file handling operations such as:

• Reading a file (sync or async way)
• Writing to a file (sync or async way)
• Deleting a file
• Reading the contents of a director
• Renaming a file
• Watching for changes in a file, and much more

Let's perform some of these tasks to see the fs (File System) module in action below:

How to create a directory using fs.mkdir()

The fs.mkdir() function in Node.js is used to create a new directory. It takes two arguments: the path of the directory to be created and an optional callback function that gets executed when the operation is complete.

• path: Here, path refers to the location where you want to create a new folder. This can be an absolute or a relative path. In my case, the path to the present working directory (the folder I am currently in), is: C:\Desktop\NodeJSTut. So, Let's create a folder in the NodeJSTut directory called myFolder.
• callback function: The purpose of the callback function is to notify that the directory creation process has completed. This is necessary because the fs.mkdir() function is asynchronous, meaning that it does not block the execution of the rest of the code while the operation is in progress. Instead, it immediately returns control to the callback function, allowing it to continue executing other tasks. Once the directory has been created, the callback function is called with an error object (if any) and any other relevant data related to the operation. In the below code, we are just using it to display a success message in the console or any error.

// Import fs module
const fs = require('fs');

// Present Working Directory: C:\Desktop\NodeJSTut
// Making a new directory called ./myFolder:

fs.mkdir('./myFolder', (err) => {
    if(err){
        console.log(err);
    } else{
        console.log('Folder Created Successfully');
    }
})

After executing the above code, you will see a new folder called myFolder created in the NodeJSTut directory.

How to create and write to a file asynchronously using fs.writeFile()

After the myFolder directory is created successfully, it's time to create a file and write something to it by using the fs module.

There are basically 2 ways of doing this:

• Synchronous Approach: In this approach, we create a file and write the data to it in a blocking manner, which means that NodeJS waits for the creation and write operation to complete before moving on to the next line of code. If an error occurs during this process, it throws an exception that must be caught using try...catch.
• Asynchronous Approach: In this approach, we create and write data to a file in a non-blocking manner, which means that NodeJS does not wait for the write operation to complete before moving on to the next line of code. Instead, it takes a callback function that gets called once the entire process is completed. If an error occurs during the write operation, the error object is passed to the callback function.

In this tutorial, we will be using the fs.writeFile() function which follows the asynchronous approach.

writeFile() is a method provided by the fs (file system) module in Node.js. It is used to write data to a file asynchronously. The method takes three arguments:

1. The path of the file to write to (including the file name and extension)
2. The data to write to the file (as a string or buffer)
3. An optional callback function that is called once the write operation is complete or an error occurs during the write operation.

When writeFile() is called, Node.js creates a new file or overwrites an existing file at the specified path. It then writes the provided data to the file and closes it. Since the method is asynchronous, the write operation does not block the event loop, allowing other operations to be performed in the meantime.

Below is the code where we create a new file called myFile.txt in the myFolder directory and write this data to it: Hi,this is newFile.txt.

const fs = require('fs');

const data = "Hi,this is newFile.txt";

fs.writeFile('./myFolder/myFile.txt', data, (err)=> {
    if(err){
        console.log(err);
        return;
    } else {
        console.log('Writen to file successfully!');
    }
})

Since newFile.txt didn't exist previously, Hence the writeFile() function created this file for us on the provided path and then wrote the value in the data variable to the file. Suppose this file already existed. In that case, writeFile() will just open the file, erase all the existing text present in it and then write the data to it.

The problem with this code is: when you run the same code multiple times, it erases the previous data that is already present in newFile.txt and writes the data to it.

In case you do not want the original data to get deleted and just want the new data to be added/appended at the end of the file, you need to make a little change in the above code by adding this "options object": {flag: 'a'} as the third parameter to writeFile() – like this:

const fs = require('fs');

const data = 'Hi,this is newFile.txt';

fs.writeFile('./myFolder/myFile.txt', data, {flag: 'a'}, (err) => {
    if(err){
        console.log(err);
        return;
    } else {
        console.log('Writen to file successfully!');
    }
})

Once you run the above code again and again, you will see that the myFile.txt has the value of the data variable written to it multiple times. This is because the object (3rd parameter): {flag: 'a'} indicates the writeFile() method to append the data at the end of the file instead of erasing the previous data present in it.

How to read a file asynchronously using fs.readFile()

After creating and writing to the file, it's time we learn how to read the data present in the file using the fs module.

Again there are 2 ways of doing this: Synchronous approach and the Asynchronous approach (just like the previous function). Here we are going to use the readFile() function provided by fs module which performs the reading operation asynchronously.

The readFile() function takes 3 parameters:

1. The path to the file which is to be read.
2. The encoding of the file.
3. The callback function that gets executed once the reading operation is completed or if any error occurs during the reading operation. It accepts 2 parameters: first parameter stores the file data (if read operation is successful) and the second parameter stores the error object (if read operation fails due to some error).

The readFile() function is very intuitive and once called, it reads the data present in the provided file according to the given encoding. If the read operation is successful, it returns the data to the callback function and if not, it will return the error occurred.

In the below code, we read the contents of the file - myFile.txt which we had created while learning the previous function and then log the data stored in it in the console.

const fs = require('fs');

fs.readFile('./myFolder/myFile.txt', {encoding: 'utf-8'}, (err, data) => {
    if(err){
        console.log(err);
        return;
    } else {
        console.log('File read successfully! Here is the data');
        console.log(data);
    }
})

It is to be noted here that the encoding property is set to 'utf-8'. At this point, some of you may not know about the encoding property, So Let's understand it in a bit more detail:

The encoding parameter in the fs.readFile() method of Node.js is used to specify the character encoding used to interpret the file data. By default, if no encoding parameter is provided, the method returns a raw buffer.

If the readFile() method is called without providing an encoding parameter, you will see a result similar to this printed in the console:

<Buffer 54 68 69 73 20 69 73 20 73 6f 6d 65 20 64 61 74 61 20 69 6e 20 61 20 66 69 6c 65>

This raw buffer is difficult to read and interpret as it represents the contents of the file in binary form. To convert the buffer to a readable string, you can specify an encoding parameter when calling readFile().

In our case, we specified the 'utf8' encoding as the second parameter of the readFile() method. This tells Node.js to interpret the file contents as a string using the UTF-8 character encoding, thus you see the original data printed in the console. Other common encodings that can be used with readFile() include:

• 'ascii': Interpret the file contents as ASCII-encoded text.
• 'utf16le': Interpret the file contents as 16-bit Unicode text in little-endian byte order.
• 'latin1': Interpret the file contents as ISO-8859-1 (also known as Latin-1) encoded text.

Reading and Writing to a File Synchronously

Up until now, you have learned how to write to and read the data from a file asynchronously. But there are synchronous alternatives to the 2 functions we learnt above, namely: readFileSync() and writeFileSync().

Note that since these are synchronous operations, they need to be wrapped in a try...catch block. In case the operations fail for some reason, the errors thrown will be caught by the catch block.

In the below code, we first create a new file: ./myFolder/myFileSync.txt and write to it using the writeFileSync() method. Then we read the contents of the file using the readFileSync() method and print the data in the console:

const fs = require('fs');

try{
    // Write to file synchronously
    fs.writeFileSync('./myFolder/myFileSync.txt', 'myFileSync says Hi');
    console.log('Write operation successful');

    // Read file synchronously
    const fileData = fs.readFileSync('./myFolder/myFileSync.txt', 'utf-8');
    console.log('Read operation successful. Here is the data:');
    console.log(fileData);

} catch(err){
    console.log('Error occurred!');
    console.log(err);
}

When you run the above code, a new file called myFileSync.txt is created in the myFolder directory and it contains the following text in it: myFileSync says Hi. This is the output printed in the console:

Write operation successful
Read operation successful. Here is the data:
myFileSync says Hi

How to read the contents of a directory using fs.readdir()

If you have been following along until now, you will see that we currently have 2 files in the myFolder directory, i.e, myFile.txt and myFileSync.txt. The fs module provides you with readdir() function using which you can read the contents of a directory (the files and folders present in the directory).

The readdir() function accepts 2 parameters:

• The path of the folder whose contents are to be read.
• Callback function which gets executed once the operation is completed or if any error occurs during the operation. This function accepts 2 parameters: The first one which accepts the error object (if any error occurs) and the second parameter which accepts an array of the various files and folders present in the directory whose path has been provided.

In the code below, we are reading the contents of the myFolder directory and printing the result in the console.

const fs = require('fs');

fs.readdir('./myFolder', (err, files) => {
    if(err){
        console.log(err);
        return;
    }
    console.log('Directory read successfully! Here are the files:');
    console.log(files);
})

This is what we get as output when we run the above code:

[ 'myFile.txt', 'myFileSync.txt' ]

How to rename a file using fs.rename()

The fs.rename() method in Node.js is used to rename a file or directory. The method takes two arguments, the current file path and the new file path, and a callback function that is executed when the renaming is complete.

Here's the syntax for the fs.rename() method:

fs.rename(oldPath, newPath, callback);

where:

• oldPath (string) - The current file path
• newPath (string) - The new file path
• callback (function) - A callback function to be executed when the renaming is complete. This function takes an error object as its only parameter.

Let's rename the newFile.txt file to newFileAsync.txt:

const fs = require('fs');

fs.rename('./newFolder/newFile.txt', './newFolder/newFileAsync.txt', (err)=>{
    if(err){
        console.log(err);
        return;
    }
    console.log('File renamed successfully!')
})

Once you run the above code, you will see that the newFile.txt gets renamed to newFileAsync.txt.

Note that you should only provide valid paths (absolute or relative) to the rename() function and not just the names of the files. Remember it's oldPath and newPath and NOT oldName and newName.

For example, consider this code: fs.rename('./newFolder/newFile.txt', 'newFileAsync.txt', ...rest of the code). In this case, since we did not provide a proper path in the 2nd parameter, rename() assumes that the path to the newly named file should be: ./newFileAsync.txt. Thus, it basically removes the newFile.txt from the newFolder directory, renames the file to newFileAsync.txt and moves it to the current working directory.

How to delete a file using fs.unlink()

Last but not the least, we have the fs.unlink() function which is used to delete a file. It takes in 2 parameters:

• The path of the file which you want to delete, and
• The callback function which gets executed once the delete operation is over or if any error occurs during the operation.

Running the following code deletes the newFileSync.txt file present in the myFolder directory:

const fs = require('fs');

fs.unlink('./myFolder/myFileSync.txt', (err) => {
    if(err){
        console.log(err);
        return;
    }
    console.log('File Deleted Successfully!')
})

Event-Driven Programming

Ok, before we move forward to learn the HTTP Module and create our own servers, it's important to know about something called "Event driven programming".

Event-driven programming is a programming paradigm where program flow is largely determined by events or user actions, rather than by the program's logic.

In this type of programming, the program listens for events, and when they occur, it executes some code/function that should run in response to that event.

An event could be anything from a mouse click or a button press to the arrival of new data in the system.

The below image demonstrates how event driven programming works. In this form of programming, we write some code which constantly listens for a particular event and once that event occurs, we run some code in response to it.

In this section of the tutorial we will be learning about events in NodeJS. Although we may not be using events directly for our day-to-day coding tasks, but a lot of NodeJS modules use the concept of events under the hood. This is why it becomes important to be aware about it.

To implement Event Driven Programming in NodeJS, You need to remember 2 things:

• There is a function called emit() which causes an event to occur.
  For example, emit('myEvent') emits/causes an event called myEvent.

• There is another function called on() which is used to listen for a particular event and when this event occurs, the on() method executes a listener function in response to it. For example, Consider this code: on('myEvent', myFunction): Here we are listening for an event called myEvent and when this event takes place, we run the myFunction listener function in response to it.

We can access the on() and emit() functions by creating an instance of the EventEmitter class. The EventEmitter class can be imported from a built-in package called events.

In the below code, we are listening for the userJoined event and once this event takes place, we run the welcomeUser() function using the on() method and we emit the userJoined event using the emit() method:

// Importing 'events' module and creating an instance of the EventEmitter Class
const EventEmitter = require('events');
const myEmitter = new EventEmitter();

// Listener Function - welcomeUser()
const welcomeUser = () => {
    console.log('Hi There, Welcome to the server!');
}

// Listening for the userJoined event using the on() method
myEmitter.on('userJoined', welcomeUser);

// Emitting the userJoined event using the emit() method
myEmitter.emit('userJoined');

Points to Note:

There are 3 points you should note while working with events in Node.
Each point in shown in action in the corresponding code snippets:

• There can be multiple on()'s for a single emit():

Check out the following code, where multiple on() functions are listening for a single event to happen (userJoined event) and when this event is emitted in the last line of the code using the emit() function, you will see that the all the listener functions which were attached to the on() function get's executed:

// Importing `events` module and creating an instance of EventEmitter class
const EventEmitter = require('events');
const myEmitter = new EventEmitter();

// Listener Function 1: sayHello
const sayHello = () => {
    console.log('Hello User');
}

// Listener Function 2: sayHi
const sayHi = () => {
    console.log('Hi User');
}

// Listener Function 3: greetNewYear
const greetNewYear = () => {
    console.log('Happy New Year!');
}

// Subscribing to `userJoined` event
myEmitter.on('userJoined', sayHello);
myEmitter.on('userJoined', sayHi);
myEmitter.on('userJoined', greetNewYear);

// Emiting the `userJoined` Event
myEmitter.emit('userJoined');

You can think of it this way: Each time the userJoined event is emitted, a notification is sent to all the on() functions listening for the event and then all of them will run their corresponding listener functions: sayHello, sayHi, greetNewYear.

Thus, when you run the code, you will see the following output printed in the console:

Hello User
Hi User
Happy New Year!

• The emit() can also contain arguments which will be passed to the listener functions:

In the following code, We are using the on() method to subscribe to an event called birthdayEvent and when this event is emitted, we run the greetBirthday() function in response to it.

The extra parameters mentioned in the emit() function, gets passed as parameters to all the listener functions which will run in response to the birthdayEvent. Therefore John and 24 gets passed as parameters to the greetBirthday() function.

const EventEmitter = require('events');
const myEvent = new EventEmitter();

// Listener function
const greetBirthday = (name, newAge) => {
    // name = John
    // newAge = 24
    console.log(`Happy Birthday ${name}. You are now {newAge}!`);
}

// Listening for the birthdayEvent
myEmitter.on('birthdayEvent', greetBirthday);

// Emitting the birthdayEvent with some extra parameters
myEmitter.emit('birthdayEvent', 'John', '24');

The following output will be seen printed in the console: Happy Birthday John, You are now 24!.

• The emit() function should always be defined after the on() function(s):

The entire process of communication between on() and emit() works like this: Before emitting any event, you need to make sure that all the listener functions have subscribed/registered to that event. Any function which is registered as a listener after the event has been emitted, will not be executed.

Check out the following code where this process is studied in detail:

const EventEmitter = require('events');
const myEmitter = new EventEmitter();

// Listener Function 1 - sayHi
const sayHi = () => {
    console.log('Hi User');
}

// Listener Function 2 - sayHello
const sayHello = () => {
    console.log('Hello User');
}

// Registering sayHi function as listener
myEmitter.on('userJoined', sayHi);

// Emitting the event
myEmitter.emit('userJoined');

// Registering sayHello function as listener
myEmitter.on('userJoined', sayHello);

When the above code is executed, we see Hi User printed in the console but Hello User is not printed.

The reason behind this is: First we had registered the sayHi function as the listener using myEmitter.on('userJoined', sayHi). Next we emit the userJoined event which results in execution of the sayHi function. In the next line we are registering the sayHello function as the listener, but it's already too late to do this because userJoined event has been emitted. This signifies the importance of defining all the on() functions before we emit the event using emit().

You can also think of it this way: When you use emit() to trigger an event, NodeJS looks for any corresponding on() methods that have been defined in your code above the emit() method. If it finds any, it will execute them in order to handle the event.

The HTTP Module

Let's move forward and learn the HTTP Module which helps you create Web Servers.

HTTP stands for Hypertext Transfer Protocol. It is used to transfer data over the internet which allows communication between clients and servers.

Suppose you want to watch some YouTube videos, you go to your web browser and type in: https://youtube.com, and then YouTube's home page gets displayed on your screen. This entire process happened because of communication between your machine (client) and YouTube's Server. The client, in this case, your machine requested for YouTube's home page and the server sent back the HTML, CSS and JS Files as the response.

The client sends a request to the server in the form of a URL with some additional information, such as headers and query parameters.

The server processes the request, performs necessary operations, and sends a response back to the client. The response contains a status code, headers, and the response body with the requested data.

_Source: https://course-api.com/slides/_

Components Of Request-Response

Both the Request (sent by client to the server) and the Response (sent by server to the client) comprises of 3 parts:

1. The Status Line: This is the first line of the request or response. It contains information about the message, such as the method used, URL, protocol version, and so on.
2. The Header: This is a collection of key-value pairs, separated by colon.
   The headers include additional information about the message such as the content type, content length, caching information, and so on.
3. The Body: The Body contains the actual data being sent or received. In the case of requests, it might contain form data or query parameters. In the case of responses, it could be HTML, JSON, XML, or any other data format.

The 3 components of a Request and Response are described in much more detail in the below image:

_Source: https://course-api.com/slides/_

What are HTTP Methods?

HTTP methods, also known as HTTP verbs, are actions that a Client can perform on a Server. The 4 HTTP Methods are:

• GET: Retrieves a resource from the server
• POST: Inserts a resource in the server
• PUT: Updates an existing resource in the server
• DELETE: Deletes a resource from the server

This might sound complicated, but let's try to understand these methods with the help of an example:

1. GET: Retrieves a resource from the server
   When you enter http://www.google.com in your web browser's address bar and press enter, your browser sends a HTTP GET request to the Google server asking for the HTML content of the Google homepage. That's then rendered and displayed by your browser.
2. POST: Inserts a resource in the server
   Imagine you're filling out a registration form to create an account on Google. When you submit the form, your browser sends a POST request to Google's server with the data you typed in the form fields like: Username, Age, Birthdate, Address, Phone Number, Email, Gender and so on.

The server will then create a new user account in its database storing all the information sent to it using the POST Request. So a POST request is used to add/insert a resource in the server.

3. PUT: Updates an existing resource in the server
   Now imagine you want to update your Google account's password. You would send a PUT request to the server with the new password. The server would then update your user account in its database with the new password.
4. DELETE: Deletes a resource from the server
   Finally, imagine you want to delete your Google user account. You would send a DELETE request to the server indicating that you want your account to be deleted. The server would then delete your user account from its database.

Note that these are just examples. The actual requests and their purposes may vary.

To see more examples of HTTP Methods, you can refer this image:

_Source: https://course-api.com/slides/_

What is a Status Code?

HTTP status codes are three-digit numbers that indicate the status of a HTTP request made to a server. They are server responses that provide information about the request's outcome. Here are some of the most common HTTP status codes and what they represent:

Let's Create a Server

Finally let's move to the good part 🥳🔥 and learn how to create a Web Server using the http module:

Step 1: Import the http module like this:

const http = require('http');

Step 2: The http module provides you with http.createServer() function which helps you create a server. This function accepts a callback function with 2 parameters – req (which stores the incoming request object) and res which stands for the response to be sent by the server. This callback function gets executed every time someone hits the server.

This is how we can create a server using the createServer() function:

const http = require('http');

const server = http.createServer((req, res) => {
    res.end('Hello World');
})

Note: res.send() is a function attached on the res object using which we can send some data back to the client. Here once we are done setting up the server, you will see a Hello World message in your web browser.

Step 3: Listening the server at some port using the listen() method.

The listen() function in Node.js http module is used to start a server that listens for incoming requests. It takes a port number as an argument and binds the server to that port number so that it can receive incoming requests on that port.

In the below code, we use the listen() function to start the server and bind it to port 5000. The second argument to the listen() function is a callback function that is executed when the server starts listening on the specified port. We are using this callback function just to display a success message in the console.

const http = require('http');

const server = http.createServer((req, res) => {
    res.end('Hello World');
})

server.listen(5000, () => {
    console.log('Server listening at port 5000');
})

You are likely to see a Hello World message when you visit this URL: http://localhost:5000/.

If you try to visit some other port like 5001 (http://localhost:5001/), which is not bound to your server, you won't see any response because your server is not listening on that port. You will likely receive an error message stating that the connection to the server could not be established.

At this point, we've made a server that renders a simple Hello World message every time someone tries to access it. This is quite good but there is a problem....

The problem is that for every route, the server sends the same message. For example, if I try to access the about page or the contact page, still the server shows the same message:

• http://localhost:5000/ -> Hello World
• http://localhost:5000/about -> Hello World
• http://localhost:5000/contact -> Hello World

There is a simple way to fix this: there's a property called url in the req object which gives the URL of the request or in other words it tells you about the resource the client is trying to access.

Suppose if I type in: http://localhost:5000/about in my web browser's search bar, this means I am performing a GET Request on the server and I am trying to access the /about page. So In this case the value of req.url will be /about.

Similarly for the below requests, the value of req.url will be:

We can use some conditionals if...else along with the req.url property to make our server respond to different requests differently. This is how we can achieve this:

const http = require('http');

const server = http.createServer((req, res) => {
    if(req.url === '/'){
        res.end('This is my Home Page');
    } else if(req.url === '/about'){
        res.end('This is my About Page');
    } else if(req.url === '/contact'){
        res.end('This is my Contact Page');
    } else {
        res.end('404, Resource Not Found');
    }
})

server.listen(5000, () => {
    console.log('Server listening at port 5000');
})

Now we have a perfect server that responds to different requests differently. We are sending back responses using a method called res.end(). But there is an even better way of sending back a response in which we can add on 2 more methods along with res.end():

1. res.writeHead() – This method is used to send the response headers to the client. The status code and headers like content-type can be set using this method.
2. res.write() – This method is used to send the response body to the client.
3. res.end() – This method is used to end the response process.

Below is the modified code where we added the writeHead() and write() methods along with end() method:

const http = require('http');

const server = http.createServer((req, res) => {
    if(req.url === '/'){
        res.writeHead(200, {'content-type': 'text/html'});
        res.write('<h1>Home Page</h1>');
        res.end();
    } else if(req.url === '/about'){
        res.writeHead(200, {'content-type': 'text/html'});
        res.write('<h1>About Page</h1>');
        res.end();
    } else if(req.url === '/contact'){
        res.writeHead(200, {'content-type': 'text/html'});
        res.write('<h1>Contact Page</h1>');
        res.end();
    } else {
        res.writeHead(404, {'content-type': 'text/html'});
        res.write('<h1>404, Resource Not Found <a href="/">Go Back Home</a></h1>');
        res.end();
    }
})

server.listen(5000, () => {
    console.log('Server listening at port 5000');
})

The below image shows what the server sends back as response when we visit multiple URL's:

Let's break down what's happening in the above code:

1. Here we are responding differently to different incoming requests by using the req.url property.
2. In every response, we are doing 3 things:
   – Setting the response header using the res.writeHead() method. Here we provide 2 parameters to res.writeHead(): the status code and an object which has the content-type property set to text/html
   – Setting the response body using the res.write() method. Note that instead of sending simple messages, we are actually sending some HTML code in this case,
   – And closing the response process using the res.end() method.
3. In case of resources like: /, /about and /contact the status code is set to 200 which means that the request to access a resource was successful. But if the client tries to access some other resource, they simply get back an error message and the status code is set to 404.
4. Here the 'content-type': 'text/html' is a way of telling the browser how it should interpret and display the response. In this case, we are telling the browser to interpret the response as some HTML code. There are different content-type's for different types of responses:
   – To send back JSON data as a response, we need to set the content-type to application/json
   – To send back CSS as a response, the content-type should be text/css
   – To send back JavaScript code as a response, the content-type should be text/javascript, and so on...

Setting the content type is very important as it determines how the web browser interprets the response. For example: if we just change the content-type from text/html to text/plain, this is how the response will be displayed in the web browser:

Let's Serve Something Interesting

Up until now you've learned how to set up web servers but we haven't built anything interesting. So let's add some fun to our lives.

In the last section of this tutorial, we will be serving this navbar:

Since this is not a front-end related tutorial, we will not be building this navbar from scratch. Instead, you can head over to this GitHub repo and copy the contents of the navbar-app directory and set it up locally: John Smilga's GitHub repo. The idea is to:

1. Set up the navbar-app folder locally
2. Use the fs module to read the contents of the HTML, CSS, JS file and the Logo
3. Using the http Module to render the files when someone tries to access the / route or the home page. So Let's Get Started:

In the below code, we are using the fs module's readFileSync() method to read the contents of the HTML, CSS, JS file and the Logo.

Note that we are going to serve the contents of the file and not the file itself. So readFileSync() comes into picture.

Then, we serve the contents of the HTML file (stored in homePage variable) using the res.write() method. Remember to set the content-type as text/html as we are serving HTML content. We have also set up responses for /about route and also a 404 page.

const http = require('http');
const fs = require('fs');

// Get the contents of the HTML, CSS, JS and Logo files
const homePage = fs.readFileSync('./navbar-app/index.html');
const homeStyles = fs.readFileSync('./navbar-app/style.css');
const homeLogo = fs.readFileSync('./navbar-app/logo.svg');
const homeLogic = fs.readFileSync('./navbar-app/browser-app.js');

// Creating the Server
const server = http.createServer((req, res) => {
    const url = req.url;
    if(url === '/'){
        res.writeHead(200, {'content-type': 'text/html'});
        res.write(homePage);
        res.end();
    } else if(url === '/about'){
        res.writeHead(200, {'content-type': 'text/html'});
        res.write(<h1>About Page</h1>);
        res.end();
    } else{
        res.writeHead(200, {'content-type': 'text/html'});
        res.write(<h1>404, Resource Not Found</h1>);
        res.end();
    }
})

server.listen(5000, () => {
    console.log('Server listening at port 5000');
})

When you run this code using node app.js command, you will see these responses sent by the server for the following routes:

We see that the other routes work fine, but the home page doesn't look as we expected. The problem is that we only see the HTML Structure of the navbar being displayed and not the other stuff like the CSS, logo, and JavaScript.

Let's see what the bug is. We can check what requests are being made by the web browser to the server by modifying the above code like this:

// ... above code
const server = http.createServer((req, res) => {
    const url = req.url;
    console.log(url);

    // ... rest of the code
})

Here we are simply printing the url of the request being made by the client to the server.

Once we refresh the page, we see that initially the browser asks for the home page and makes a GET request with the / URL. Afterward it makes 3 more requests:

• /style.css – asking for the CSS file
• /browser-app.js – asking for the JS file
• /logo.svg – asking for the logo

From this, we can infer how browsers work.

The browser makes request for the contents of the / path and the server just sends back the HTML content. Once the browser receives the HTML content, it interprets it and starts displaying the elements. While parsing HTML, if the browser encounters any additional resource like a CSS page or JS page, it will make a request to the server for the same.

Since we are not sending the CSS, JS and Logo in the response, we do not see them on the screen. We can fix this by adding some more if()s in the code and sending those resources which the browser asks for and BOOM – this bug is fixed.

const http = require('http');
const fs = require('fs');

// Get the contents of the HTML, CSS, JS and Logo files
const homePage = fs.readFileSync('./navbar-app/index.html');
const homeStyles = fs.readFileSync('./navbar-app/style.css');
const homeLogo = fs.readFileSync('./navbar-app/logo.svg');
const homeLogic = fs.readFileSync('./navbar-app/browser-app.js');

// Creating the Server
const server = http.createServer((req, res) => {
    const url = req.url;
    if(url === '/'){
        res.writeHead(200, {'content-type': 'text/html'});
        res.write(homePage);
        res.end();
    } else if(url === '/style.css'){
        res.writeHead(200, {'content-type': 'text/css'});
        res.write(homeStyles);
        res.end();
    } else if(url === '/browser-app.js'){
        res.writeHead(200, {'content-type': 'text/javascript'});
        res.write(homeLogic);
        res.end();
    } else if(url === '/logo.svg'){
        res.writeHead(200, {'content-type': 'image/svg+xml'});
        res.write(homeLogo);
        res.end();
    } else if(url === '/about'){
        res.writeHead(200, {'content-type': 'text/html'});
        res.write(<h1>About Page</h1>);
        res.end();
    } else{
        res.writeHead(200, {'content-type': 'text/html'});
        res.write(<h1>404, Resource Not Found</h1>);
        res.end();
    }
})

server.listen(5000, () => {
    console.log('Server listening at port 5000');
})

Now we can see the HTML, CSS, Logo and the JS Functionality present:

Conclusion

With this we come to the end of this tutorial – I hope you liked it and learned a lot about Node.

Do share your learnings from this guide on Twitter and LinkedIn (#LearnInPublic) and follow freeCodeCamp for more such informative coding articles.

Connect with me on Twitter: Twitter - Krish4856, DMs Open.

See you next time 👋 ❤️ ✨

In some previous tutorials, I covered how to upload files using HTML and JavaScript. It requires sending HTTP requests with the [Content-Type](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Type) header set to multipart/form-data.

Today, we are going to the back end to receive those multipart/form-data requests and access the binary data from those files.

Some Background

Most of the concepts in this tutorial should broadly apply across frameworks, runtimes, and languages, but the code examples will be more specific.

I’ll be working within a Nuxt.js project that runs in a Node.js environment. Nuxt has some specific ways of defining API routes which require calling a global function called defineEventHandler.

/**
 * @see https://nuxt.com/docs/guide/directory-structure/server
 * @see https://nuxt.com/docs/guide/concepts/server-engine
 * @see https://github.com/unjs/h3
 */
export default defineEventHandler((event) => {
  return { ok: true };
});

The event argument provides access to work directly with the underlying Node.js request object (a.k.a. IncomingMessage) through event.node.req. So we can write our Node-specific code in an abstraction, like a function called doSomethingWithNodeRequest that receives this Node request object and does something with it.

export default defineEventHandler((event) => {
  const nodeRequestObject = event.node.req;

  doSomethingWithNodeRequest(event.node.req);

  return { ok: true };
});

/**
 * @param {import('http').IncomingMessage} req
 */
function doSomethingWithNodeRequest(req) {
  // Do not specific stuff here
}

Working directly with Node in this way means the code and concepts should apply regardless of whatever higher-level framework you’re working with. Ultimately, finish things up working in Nuxt.js.

How to Deal with multipart/form-data in Node.js

In this section, we’ll dive into some low-level concepts that are good to understand, but not strictly necessary. Feel free to skip this section if you are already familiar with chunks and streams and buffers in Node.js.

Uploading a file requires sending a multipart/form-data request. In these requests, the browser will split the data into little “chunks” and send them through the connection, one chunk at a time. This is necessary because files can be too large to send in as one massive payload.

Chunks of data being sent over time make up what’s called a “stream”. Streams are kind of hard to understand the first time around, at least they were for me. They deserve a full article (or many) on their own, so I’ll share web.dev’s excellent guide in case you want to learn more.

Basically, a stream is sort of like a conveyor belt of data, where each chunk can be processed as it comes in. In terms of an HTTP request, the backend will receive parts of the request, one bit at a time.

Node.js provides us with an event handler API through the request object’s on method, which allows us to listen to “data” events as they are streamed into the backend.

/**
 * @param {import('http').IncomingMessage} req
 */
function doSomethingWithNodeRequest(req) {
  req.on("data", (data) => {
    console.log(data);
  }
}

For example, when I upload a photo of Nugget making a cute yawny face, then look at the server’s console, I’ll see some weird things that look like this:

I used a screenshot here to prevent assistive technology from reading that gibberish out loud. Could you imagine?

These two pieces of garbled nonsense are called “buffers” and they represent the two chunks of data that made up the request stream containing the cute photo of Nugget.

A buffer is a storage in physical memory used to temporarily store data while it is being transferred from one place to another. – MDN

Buffers are another weird, low-level concept I have to explain when talking about working with files in JavaScript.

JavaScript doesn’t work directly on binary data, so we get to learn about buffers. It’s also OK if these concepts still feel a little vague. Understanding everything completely is not the important part right now, and as you continue to learn about file transfers, you’ll gain a better knowledge of how it all works together.

Working with one partial chunk of data is not super useful. What we can do instead is rewrite our function into something we can work with:

1. Return a [Promise](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise) to make the async syntax easy to work with.
2. Provide an [Array](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Array) to store the chunks of data to use later on.
3. Listen for the “data” event and add the chunks to our collection as they arrive.
4. Listen to the “end” event and convert the chunks into something we can work with.
5. Resolve the Promise with the final request payload.
6. We should also remember to handle “error” events.

/**
 * @param {import('http').IncomingMessage} req
 */
function doSomethingWithNodeRequest(req) {
  return new Promise((resolve, reject) => {
    /** @type {any[]} */
    const chunks = [];
    req.on('data', (data) => {
      chunks.push(data);
    });
    req.on('end', () => {
      const payload = Buffer.concat(chunks).toString()
      resolve(payload);
    });
    req.on('error', reject);
  });
}

And every time that the request receives some data, it pushes that data into the array of chunks.

So with that function set up, we can actually [await](https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Operators/await) that returned Promise until the request has finished receiving all the data from the request stream, and log the resolved value to the console.

export default defineEventHandler((event) => {
  const nodeRequestObject = event.node.req;

  const body = await doSomethingWithNodeRequest(event.node.req);
  console.log(body)

  return { ok: true };
});

This is the request body. Isn’t it beautiful?

I honestly don’t even know what a screen reader would do with if this was plain text.

If you upload an image file, it’ll probably look like an alien has hacked your computer. Don’t worry, it hasn’t. That’s literally what the text contents of that file look like. You can even try opening up an image file in a basic text editor and see the same thing.

If I upload a more basic example, like a .txt file with some plain text in it, the body might look like this:

Content-Disposition: form-data; name="file"; filename="dear-nugget.txt"
Content-Type: text/plain

I love you!
------WebKitFormBoundary4Ay52hDeKB5x2vXP--

Notice that the request is broken up into different sections for each form field. The sections are separated by the “form boundary”, which the browser will inject by default.

I’ll skip going into excess details, so if you want to read more, check out [Content-Disposition](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Content-Disposition) on MDN. The important thing to know is that multipart/form-data requests are much more complex than just key/value pairs.

Most server frameworks provide built-in tools to access the body of a request. So we’ve actually reinvented the wheel. For example, Nuxt provides a global readBody function. So we could have accomplished the same thing without writing our own code:

export default defineEventHandler((event) => {
  const nodeRequestObject = event.node.req;

  const body = await readBody(event.node.req);
  console.log(body)

  return { ok: true };
});

This works fine for other content types, but for multipart/form-data, it has issues. The entire body of the request is being read into memory as one giant string of text. This includes the Content-Disposition information, the form boundaries, and the form fields and values. Never mind the fact that the files aren’t even being written to disk.

The big issue here is if a very large file is uploaded, it could consume all the memory of the application and cause it to crash.

The solution is, once again, working with streams.

When our server receives a chunk of data from the request stream, instead of storing it in memory, we can pipe it to a different stream. Specifically, we can send it to a stream that writes data to the file system using [createWriteStream](https://nodejs.org/api/fs.html#filehandlecreatewritestreamoptions). As the chunks come in from the request, that data gets written to the file system, then released from memory.

That’s about as far down as I want to go into the low-level concepts. Let’s go back up to solving the problem without reinventing the wheel.

How to Use a Library to Stream Data onto Disk

Probably my best advice for handling file uploads is to reach for a library that does all this work for you:

• Parse multipart/form-data requests
• Separate the files from the other form fields
• Stream the file data into the file system
• Provide you with the form field data as well as useful data about the files

Today, I’m going to be using this library called formidable. You can install it with npm install formidable, then import it into your project.

import formidable from 'formidable';

Formidable works directly with the Node request object, which we conveniently already grabbed from the Nuxt event (“Wow, what amazing foresight!!” 🤩).

So we can modify our doSomethingWithNodeRequest function to use formidable instead. It should still return a promise because formidable uses callbacks, but promises are nicer to work with. Otherwise, we can mostly replace the contents of the function with formidable.

We’ll need to create a formidable instance and use it to parse the request object. As long as there isn’t an error, we can resolve the promise with a single object that contains both the form fields and the files.

/**
 * @param {import('http').IncomingMessage} req
 */
function doSomethingWithNodeRequest(req) {
  return new Promise((resolve, reject) => {
    /** @see https://github.com/node-formidable/formidable/ */
    const form = formidable({ multiples: true })
    form.parse(req, (error, fields, files) => {
      if (error) {
        reject(error);
        return;
      }
      resolve({ ...fields, ...files });
    });
  });
}

This provides us with a handy function to parse multipart/form-data using promises and access the request’s regular form fields, as well as information about the files that were written to disk using streams.

Now, we can examine the request body:

export default defineEventHandler((event) => {
  const nodeRequestObject = event.node.req;

  const body = await doSomethingWithNodeRequest(event.node.req);
  console.log(body)

  return { ok: true };
});

We should see an object containing all the form fields and their values, but for each file input, we’ll see an object that represents the uploaded file, and not the file itself. This object contains all sorts of useful information including its path on disk, name, mimetype, and more.

{
  file-input-name: PersistentFile {
    _events: [Object: null prototype] { error: [Function (anonymous)] },
    _eventsCount: 1,
    _maxListeners: undefined,
    lastModifiedDate: 2023-03-21T22:57:42.332Z,
    filepath: '/tmp/d53a9fd346fcc1122e6746600',
    newFilename: 'd53a9fd346fcc1122e6746600',
    originalFilename: 'file.txt',
    mimetype: 'text/plain',
    hashAlgorithm: false,
    size: 13,
    _writeStream: WriteStream {
      fd: null,
      path: '/tmp/d53a9fd346fcc1122e6746600',
      flags: 'w',
      mode: 438,
      start: undefined,
      pos: undefined,
      bytesWritten: 13,
      _writableState: [WritableState],
      _events: [Object: null prototype],
      _eventsCount: 1,
      _maxListeners: undefined,
      [Symbol(kFs)]: [Object],
      [Symbol(kIsPerformingIO)]: false,
      [Symbol(kCapture)]: false
    },
    hash: null,
    [Symbol(kCapture)]: false
  }
}

You’ll also notice that the newFilename is a hashed value. This is to ensure that if two files are uploaded with the same name, you will not lose data. You can, of course, modify how files are written to disk.

Note that in a standard application, it’s a good idea to store some of this information in a persistent place, like a database, so you can easily find all the files that have been uploaded. But that’s not the point of this post.

Now there’s one more thing I want to fix. I only want to process multipart/form-data requests with formidable. Everything else can be handled by a built-in body parser like the one we saw above.

So I’ll create a “body” variable first, then check the request headers, and assign the value of the body based on the “Content-Type”. I’ll also rename my function to parseMultipartNodeRequest to be more explicit about what it does.

Here’s what the whole thing looks like (note that getRequestHeaders is another built-in Nuxt function):

import formidable from 'formidable';

/**
 * @see https://nuxt.com/docs/guide/concepts/server-engine
 * @see https://github.com/unjs/h3
 */
export default defineEventHandler(async (event) => {
  let body;
  const headers = getRequestHeaders(event);

  if (headers['content-type']?.includes('multipart/form-data')) {
    body = await parseMultipartNodeRequest(event.node.req);
  } else {
    body = await readBody(event);
  }
  console.log(body);

  return { ok: true };
});

/**
 * @param {import('http').IncomingMessage} req
 */
function parseMultipartNodeRequest(req) {
  return new Promise((resolve, reject) => {
    /** @see https://github.com/node-formidable/formidable/ */
    const form = formidable({ multiples: true })
    form.parse(req, (error, fields, files) => {
      if (error) {
        reject(error);
        return;
      }
      resolve({ ...fields, ...files });
    });
  });
}

This way, we have an API that is robust enough to accept multipart/form-data, plain text, or URL-encoded requests.

📯📯📯 Finishing up

There’s no emoji rave horn, so those will have to do. We covered kind of a lot, so let’s do a little recap.

When we upload a file using a multipart/form-data request, the browser will send the data one chunk at a time, using a stream. That’s because we can’t put the entire file in the request object at once.

In Node.js, we can listen to the request’s “data” event to work with each chunk of data as it arrives. This gives us access to the request stream.

Although we could capture all of that data and store it in memory, that’s a bad idea. A large file upload could consume all the server’s memory, causing it to crash.

Instead, we can pipe that stream somewhere else, so each chunk is received, processed, then released from memory. One option is to use [fs.createWriteStream](https://nodejs.org/api/fs.html#fscreatewritestreampath-options) to create a [WritableStream](https://developer.mozilla.org/en-US/docs/Web/API/WritableStream) that can write to the file system.

Instead of writing our own low-level parser, we should use a tool like formidable. But we need to confirm that the data is coming from a multipart/form-data request. Otherwise, we can use a standard body parser.

We covered a lot of low-level concepts, and landed on a high-level solution. Hopefully, it all made sense and you found this useful.

If you have any questions or if something was confusing, please go ahead and reach out to me. I’m always happy to help.

Thank you so much for reading. If you liked this article, and want to support me, the best ways to do so are to share it, sign up for my newsletter, and follow me on Twitter.

One common thing in every organisation is policy. Policies define how an organisation operates.

They are essential to the long-term success of an organisation. They preserve significant knowledge about how to comply with matters such as legal requirements, work within technical constraints, and avoid repeating mistakes.

Softwares follow the same pattern by adhering to rules that govern its behavior. These rules (or policies) may specify the application's environments, permitted network routes, dependencies versions allowed, and when micro-services receive API requests. Usually, developers create them manually using documents like spreadsheets.

The issue with this method is that it gradually becomes bulky. If each part of an application has its policy, things like authorization will be hard to manage across the whole application. There might also be the unnecessary repetition of policies across different parts of the application.

Aside from that, updating any policy will require the redeployment of the whole application. Fortunately, Open Policy Agent(OPA) found a way to fix these issues.

This article will explain what OPA is, how it works, what the OPA plugin entails, and how to use it.

Let’s get started!

What is OPA?

OPA is an open-source general-purpose policy engine. It can replace built-in policy function modules in software and help users decouple services from the policy engine.

OPA provides a way to build applications separate from their policies and for them to be reusable in many applications.

The OPA policy handling method reduces complexities and gives more control to the application owner. OPA allows users to integrate it with other services, such as program libraries, and HTTP APIs.

How OPA Works

OPA mediates between applications and policies to decide the rule to apply in handling a request. The image below describes its operation:

Here is a breakdown of the image above:

1. A service (let’s say it is an authentication micro-service) receives a request (like a login request). For the service to decide how to handle the request, it needs to get the policy guiding authentication. That takes us to the next step.
2. The service sends a query (this can be in any JSON format) to OPA requesting for the policy to be adhered to in handling the request received.
3. OPA now compares the data and policies it has access to and makes the right decision.
4. Finally, OPA returns the policy decision (this can be in any JSON format) reached to the service.

That is a summary of how OPA works. You can imagine many services attached to OPA and OPA helping them decide how to handle requests or events instead of each service managing its policies. It provides a more robust system that is easy to maintain.

Apache APISIX decided to integrate with OPA by providing the OPA plugin. That's what we'll discuss now.

Apache APISIX OPA Plugin

The plugin allows Apache APISIX users to conveniently introduce the policy capabilities provided by OPA when using Apache APISIX. It enables flexible authentication and access control features.

How It Works

Apache APISIX OPA Plugin follows two main steps to carry out its task:

First, APISIX re-constructs any request data it receives into acceptable JSON data and makes a policy query to OPA with it. The query is usually referred to as an APISIX to OPA service request. See the following example:

{
    "type": "http",
    "request": {
        "scheme": "http",
        "path": "\/get",
        "headers": {
            "user-agent": "curl\/7.68.0",
            "accept": "*\/*",
            "host": "127.0.0.1:9080"
        },
        "query": {},
        "port": 9080,
        "method": "GET",
        "host": "127.0.0.1"
    },
    "var": {
        "timestamp": 1701234567,
        "server_addr": "127.0.0.1",
        "server_port": "9080",
        "remote_port": "port",
        "remote_addr": "ip address"
    },
    "route": {},
    "service": {},
    "consumer": {}
}

The JSON data above tells OPA that a user has made an HTTP request using the GET method via 127.0.0.1:9080/get at 1701234567 timestamp (Wednesday, 29 November 2023 05:09:27).

OPA now has to help Apache APISIX decide how to handle the request.

Next, OPA checks the policies and data available, compares them, and reaches the decision in JSON format below:

{
    "result": {
        "allow": true,
        "reason": "test",
        "headers": {
            "an": "header"
        },
        "status_code": 401
    }
}

The policy decision above is an OPA service to APISIX response. It tells APISIX to accept the request due to the reason (test) given. When allow is false, Apache APISIX rejects it.

The following is an explanation of some of the keys in the request and response above:

• type indicates the request type (HTTP or stream).
• request is used when the type is HTTP and contains the basic request information like URL and headers.
• var holds the basic information about the requested connection (IP, port, server details, and request timestamp).
• route, service, and consumer contain the same data stored in APISIX. They require configuration for a user to see them after a transaction.
• allow is required and indicates whether the request is authorised to pass through APISIX.
• reason, headers, and status_code are optional and are returned when you configure a custom response.

How to Use the Plugin

This section will introduce you to some of the features of the plugin. You will see how to use Docker to build OPA services, create policy, create users’ data, create a custom route, test requests, and enable and disable the plugin.

How to use docker to build OPA services

Use the command below to launch the OPA environment on port 8181

docker run -d --name opa -p 8181:8181 openpolicyagent/opa:0.35.0 run -s

We will be using CURL for the rest of this article. If you are new to it or you are coming from other programming languages, copy the requests or response code and paste the code here to convert it to your preferred language.

We will also stick to the -H and -d flags instead of --header and --data-raw respectively.

How to create a policy

Creating a policy follows the format below:

curl -X PUT '127.0.0.1:8181/v1/policies/example1' \
    -H 'Content-Type: text/plain' \
    -d 'package example

import input.request

default allow = false

allow {
    # HTTP method must GET
    request.method == "GET"
}'

The code above came about through the following steps:

• State the route: 127.0.0.1:8181/v1/policies/example1.
• Import Request: import input.request.
• State that no request is allowed: default allow = false.
• Specify what is permissible:

allow {
    # HTTP method must GET
    request.method == "GET"
}

The code above instructs that the only acceptable HTTP method is GET. Every line in the allow object gets implemented as policies asides from the lines that begin with a # because they are comments.

You can add as many rules as you want based on the policies you have in mind. For example, the code below contains five rules that must be adhered to:

# Create policy
curl -X PUT '127.0.0.1:8181/v1/policies/example1' \
    -H 'Content-Type: text/plain' \
    -d 'package example

import input.request
import data.users

default allow = false

allow {
    # has the name test-header with the value only-for-test request header
    request.headers["test-header"] == "only-for-test"

    # The request method is GET
    request.method == "GET"

    # The request path starts with /get
    startswith(request.path, "/get")

    # GET parameter test exists and is not equal to abcd
    request.query["test"] != "abcd"

    # GET parameter user exists
    request.query["user"]
}'

With the configuration we have made so far, everything will work fine. But what happens when our users get something wrong and an error they don’t understand is returned to them? They will become frustrated and left with a bad user experience. We can avoid that by adding a custom response.

A custom response provides extra details (body, header, and status code) about the result of a transaction. Our request now becomes:

# Create policy
curl -X PUT '127.0.0.1:8181/v1/policies/example1' \
    -H 'Content-Type: text/plain' \
    -d 'package example

import input.request
import data.users

default allow = false

allow {
    # has the name test-header with the value only-for-test request header
    request.headers["test-header"] == "only-for-test"
    # The request method is GET
    request.method == "GET"
    # The request path starts with /get
    startswith(request.path, "/get")
    # GET parameter test exists and is not equal to abcd
    request.query["test"] != "abcd"
    # GET parameter user exists
    request.query["user"]
}

# custom response body (Accepts a string or an object, the object will respond as JSON format)
reason = users[request.query["user"]].reason {
    not allow
    request.query["user"]
}

# custom response header (The data of the object can be written in this way)
headers = users[request.query["user"]].headers {
    not allow
    request.query["user"]
}

# custom response status code
status_code = users[request.query["user"]].status_code {
    not allow
    request.query["user"]
}'

When a user gets an error, it becomes easier to debug because the error comes with a reason, headers details, and status_code.

How to create users’ data

The users' data is an object of objects. Each user data is an object of custom details (body, header, and status code) that help with user authorization.

The code below is an example of users data containing four (4) users with different details:

# Create test user data
curl -X PUT '127.0.0.1:8181/v1/data/users' \
    -H 'Content-Type: text/plain' \
    -d '{

    "alice": {
        "headers": {
            "Location": "http://example.com/auth"
        },
        "status_code": 302
    },

    "bob": {
        "headers": {
            "test": "abcd",
            "abce": "test"
        }
    },

    "carla": {
        "reason": "Give you a string reason"
    },

    "dylon": {
        "headers": {
            "Content-Type": "application/json"
        },
        "reason": {
            "code": 40001,
            "desc": "Give you a object reason"
        }
    }
}'

Notice that each user’s custom details are optional and may differ for every user.

How to create a custom route and enable the plugin

The APISIX OPA plugin's flexibility makes it possible for users to customize their route like in the code below:

curl -X PUT 'http://127.0.0.1:9080/apisix/admin/routes/r1' \
    -H 'X-API-KEY: <api-key>' \
    -H 'Content-Type: application/json' \
    -d '{
    "uri": "/*",
    "methods": [
        "GET",
        "POST",
        "PUT",
        "DELETE"
    ],
    "plugins": {},
    "upstream": {
        "nodes": {
            "httpbin.org:80": 1
        },
        "type": "roundrobin"
    }
}'

For this to work, the plugin has to be enabled. Enter the needed configuration into the plugins object to turn it on. So we have:

curl -X PUT 'http://127.0.0.1:9080/apisix/admin/routes/r1' \
    -H 'X-API-KEY: <api-key>' \
    -H 'Content-Type: application/json' \
    -d '{
    "uri": "/*",
    "methods": [
        "GET",
        "POST",
        "PUT",
        "DELETE"
    ],
    "plugins": {
        "opa": {
            "host": "http://127.0.0.1:8181",
            "policy": "example1"
        }
    },
    "upstream": {
        "nodes": {
            "httpbin.org:80": 1
        },
        "type": "roundrobin"
    }
}'

Now that the plugin is enabled, you can use your route as you see fit.

How to test the requests

We have been able to create policies, users’ data, and custom routes and enabled the Apache APISIX OPA plugin so far. Let’s now test these setups and see the response we get for different scenarios:

Here's a test for when a request is allowed:

Request:

curl -XGET '127.0.0.1:9080/get?test=none&user=dylon' \
    --header 'test-header: only-for-test'

Response:

{
    "args": {
        "test": "abcd1",
        "user": "dylon"
    },
    "headers": {
        "Test-Header": "only-for-test",
        "with": "more"
    },
    "origin": "127.0.0.1",
    "url": "http://127.0.0.1/get?test=abcd1&user=dylon"
}

Here's a test for when a request is rejected and the status code and response headers are re-written:

Request:

curl -XGET '127.0.0.1:9080/get?test=abcd&user=alice' \
    --header 'test-header: only-for-test'

Response:

HTTP/1.1 302 Moved Temporarily
Date: Mon, 20 Dec 2021 09:37:35 GMT
Content-Type: text/html
Content-Length: 142
Connection: keep-alive
Location: http://example.com/auth
Server: APISIX/2.11.0

Here's a test for when a request is rejected and a custom response header is returned:

Request:

curl -XGET '127.0.0.1:9080/get?test=abcd&user=bob' \
    --header 'test-header: only-for-test'

Response:

HTTP/1.1 403 Forbidden
Date: Mon, 20 Dec 2021 09:38:27 GMT
Content-Type: text/html; charset=utf-8
Content-Length: 150
Connection: keep-alive
abce: test
test: abcd
Server: APISIX/2.11.0

Here's a test for when a request is rejected and a custom response (string) is returned:

Request:

curl -XGET '127.0.0.1:9080/get?test=abcd&user=carla' \
    --header 'test-header: only-for-test'

Response:

HTTP/1.1 403 Forbidden
Date: Mon, 20 Dec 2021 09:38:58 GMT
Content-Type: text/plain; charset=utf-8
Transfer-Encoding: chunked
Connection: keep-alive
Server: APISIX/2.11.0

Give you a string of reason

And here's a test for when a request is rejected and a custom response (JSON) is returned:

Request:

curl -XGET '127.0.0.1:9080/get?test=abcd&user=dylon' \
    --header 'test-header: only-for-test'

Response:

HTTP/1.1 403 Forbidden
Date: Mon, 20 Dec 2021 09:42:12 GMT
Content-Type: application/json
Transfer-Encoding: chunked
Connection: keep-alive
Server: APISIX/2.11.0

{"code":40001,"desc":"Give you a object reason"}

How to disable the plugin

To disable the APISIX OPA plugin, remove all the configurations we added when we set up a custom route and enabled the plugin. We now have:

curl -X PUT 'http://127.0.0.1:9080/apisix/admin/routes/r1' \
    -H 'X-API-KEY: <api-key>' \
    -H 'Content-Type: application/json' \
    -d '{
    "uri": "/*",
    "methods": [
        "GET",
        "POST",
        "PUT",
        "DELETE"
    ],
    "plugins": {},
    "upstream": {
        "nodes": {
            "httpbin.org:80": 1
        },
        "type": "roundrobin"
    }
}'

The plugins object being empty indicates that the plugin cannot work. It is that easy because of Apache APISIX’s dynamic nature.

Conclusion

This article aimed to introduce you to the Apache APISIX OPA plugin and walk you through some of its features.

We began by looking at what OPA is and why APISIX adopted it by employing a plugin. Then we discussed how the plugin works and how we can use it.

Apache APISIX currently has more than ten authentication and authorization-related plugins that support interfacing with mainstream authentication/authorization services in the industry.

If you need to interface with other authentication authorities, you can visit Apache APISIX's GitHub and leave your suggestions via an issue or subscribe to Apache APISIX's mailing list to express your ideas.

I hope this article helps you understand how to use OPA in Apache APISIX so you can start adopting it yourself. I also encourage you to take the time to visit the Apache APISIX OPA plugin documentation to see other use cases for the plugin. The more you practice with it, the better you get at using it.

Happy Policy Making!

With Node.js, developers can create server side applications, command line tools, and more.

This article is not a crash course on how to use Node.js (you'll find resources for that in the last section of this article). Rather, it's an introduction to what Node.js is, its features, and what it is used for.

What is Node.js?

Node.js is an open source JavaScript runtime environment that lets developers run JavaScript code on the server.

If that's too complex for you to understand then you should think of it this way: Node.js is JavaScript that runs outside the browser — on the server.

Note that Node.js is not a programming language - it's a tool.

What Is So Special About Node.js?

In this section, we'll discuss some of the features that make Node.js cool to use.

The aim is not to compare Node.js to other backend technologies, but to help you understand some of its functionalities.

Single Threaded and Asynchronous

Node.js is fast at executing tasks (receiving requests and sending back responses) because of its single threaded and asynchronous nature.

Let's explain some of the terms above.

By single threaded, this means that Node.js has a single source for handling requests. Multiple threaded backend technologies allocate a new thread for every new request.

You can think of a thread as someone who renders a service to multiple people. A very popular real life example would be a restaurant. We'll explain this example further along with the asynchronous part of Node.js.

Node.js is asynchronous because it can handle multiple requests simultaneously. Let's get back to the restaurant example.

A customer gets to a restaurant and sits down waiting for a server. The server gets to the customer's table and takes their order. The order is then taken to the kitchen.

But the server doesn't wait for the order to be ready before proceeding to the next customer. They'll return with what the customer ordered for when its ready – in the meantime, the server proceeds to the next customer and repeats the same process.

The example above is similar to how Node.js works under the hood. It is able to process multiple requests using a single thread asynchronously (without waiting for one request's completion before moving to the next).

So when the response for a request is ready, it is sent back to the client.

The single threaded and asynchronous nature of Node.js makes it very fast and ideal for building data-intensive and real-time applications.

JavaScript Everywhere

Another advantage of using Node.js as a web developer is the possibility of using JavaScript on the frontend and backend of your web app.

Before the release of Node.js, web developers had to learn a different programming language to build the backend of their web apps.

Of course, some developers still use different languages for their backend but Node.js makes it easy to use just one language — JavaScript – if you want.

Fast Execution Time

Node.js is built on Google's V8 JavaScript engine which has a very high performance. This lets Node execute requests quickly.

Cross Platform Compatibility

Node.js supports many major platforms. So you can write your code and it will run on Windows, MacOS, LINUX, UNIX and even some mobile devices.

What is Node Used For?

Here are some of the cool things you can do with Node.js:

• Create HTTP web servers.
• Generate web pages dynamically.
• Collect and send form data to a database.
• Create, read, update, and delete data stored in a database.
• Create APIs.
• Build command line tools.
• Read, write, move, delete, and open/close files on a server.

Summary

In this article, we talked about Node.js. We first had a look at what it really is.

We then talked about some the features that make Node.js stand out.

Lastly, we saw a list of how you can use Node.js.

How to Learn Node.js

Now that you've had a brief introduction to what Node.js is, its features, and what it is used for, here are some resources that you can use to learn how to use Node.js:

• freeCodeCamp's Back End Development and APIs certification. You'll learn how to write back end apps with Node.js and npm. You'll also build web applications with the Express framework, along with MongoDB and the Mongoose library.
• An 8-hour course on the freeCodeCamp.org YouTube channel that'll teach you Node.js and Express.
• A 10-hour project based course on the freeCodeCamp.org YouTube channel. You'll build four projects from the knowledge gained from the 8-hour course above.

Happy Coding!

But what exactly is full-stack web development, and how do you become a full-stack developer?

In this article, you will first learn what the term full-stack means.

Then, you will see some suggested technologies to learn accompanied by learning resources to start your full-stack web development journey.

What is Full Stack Development?

The web applications you use daily consist of:

• The front-end, also known as the client-side part of the application,
  • and the back-end, also known as the server-side part of the application.

What is front end development?

The front end is the user interface of the application.

It’s the presentation of the content, the parts the user views and interacts with, and the look and feel of the web page. It’s how information displays on both web browsers but also mobile devices.

It includes different components such as navigation bars, dropdown menus, buttons, and links.

The front end consists of all the visible parts responsible for the user experience.

What is back end development?

The back end consists of a server that receives and processes requests and a database used to store data.

The back-end is all the behind-the-scenes processes happening in a web application.

It is the business logic, the handling and manipulation of data, and the algorithms.

Essentially, it is all the hidden parts that a user is not directly aware of when viewing and interacting with a webpage. These parts are processes running in the background.

The back end is considered the logical side or the ‘brain’ of a web application.

Both the front-end and the back-end together make up a full-stack web application.

So, full-stack web development refers to the knowledge of all the parts responsible for the front-end and the back-end side of a web application.

Full Stack Engineer Guide

When you are a beginner just starting out on your learning journey, you may spend more time searching for what to learn instead of actually learning. And there is a lot to learn when it comes to full-stack web development - which can be overwhelming and intimidating.

In the sections to come, we'll discuss a lot of different technologies you'll need to learn, and there are many more which are not listed.

It's impossible to learn everything at once, just as it is also impossible to become an expert in everything.

This guide is a suggestion and a starting point. It provides the basics on which you can later expand.

Remember that it is going to take time to become a full stack developer. It may take some trial and error to find a learning schedule and routine that suits you and works for you, as you may have commitments such as taking care of family, working a full-time job, or any other life obligations.

Success is not linear.

Sketch from author and comedian Demetri Martin

The key to success is staying consistent and beating procrastination.

Map out and set aside some time every day - whether that is for only half or one hour. An hour a day is better than not doing any coding at all.

Remember to rest and take time away from the screen to not burn yourself out in the process.

Now, let’s see some of the technologies you should learn to become a full-stack web developer in 2022.

Basic Understanding of How the Internet and the Web Work

You will be spending a lot of time on the Internet, designing and developing for the Web.

Instead of just learning how to create web applications, products, and services, it is wise to have a well-rounded knowledge and good understanding of how these tools you are using to achieve your own goals work underneath the hood and interact with each other.

For example, it is a good use of your time to learn:

• What is HTTP?
• What are the different HTTP methods, and what does each mean and do?
• What is the difference between HTTP and HTTPS?
• What is an IP address, a Domain Name, and DNS?
• An understanding of clients and servers, and how they interact using what is called the request-response cycle.
• How the Internet works as a whole.

Here is some suggested reading to get you started:

• How HTTP Works and Why it's Important – Explained in Plain English
• HTTP Request Methods – Get vs Put vs Post Explained with Code Examples
• What is HTTPS? HTTP vs HTTPS Meaning and How it Works
• What is DNS? Domain Name System, DNS Server, and IP Address Concepts Explained
• How the Web Works: A Primer for Newcomers to Web Development (or anyone, really)
• How the Web Works Part II: Client-Server Model & the Structure of a Web Application
• How the Web Works Part III: HTTP & REST

Frontend Web Development Fundamentals

Start off learning to code by practicing HTML and CSS.

HTML and CSS are two languages that serve a different purpose but come together to create static web pages. For this reason, it is often the case that you'll learn them both in parallel.

First, learn HTML fundamentals, specifically HTML5, which is the latest HTML version and supports many more features.

What is HTML?

HTML (which stands for HyperText Markup Language) is used for defining the structure and content on a web page. So, the paragraphs, headings, lists, images, forms, and links you see on a page, along with their hierarchies, are all HTML code.

Key HTML concepts to learn are:

• There are many HTML tags (as a matter of fact, there are over a hundred), but you only need to learn the most commonly used ones that you will end up using the most.
• Learn to write semantic HTML. Semantic HTML describes the tag and the content it will hold instead of using a tag that has no meaning. For example, if you want to create a section on a webpage to hold specific content, you could use the <section> tag that specifies and describes the tag instead of the <div> tag that doesn't have any meaning behind it. All-in-all, avoid using too many <div> tags and learn the alternatives you can use instead.
• The above point ties in well with learning to write HTML with accessibility in mind. Designing and developing with accessibility in mind means creating websites for everyone. People with visual impairments rely on assistive technologies such as screen readers to read the content out loud. People with other disabilities may depend on keyboard-only navigation. So, learning to write accessible HTML will lead you to create more user-friendly web pages.
• Learn how to create HTML forms. Forms are present on almost all websites. It's how you log in or sign up and register to a website. It's also how websites gather information and data from users.

What is CSS?

Next, it's time to learn CSS.

CSS (short for Cascading Style Sheets) styles HTML elements. It is responsible for presenting content in a visually appealing way.

CSS code enables all the different colors and fonts. The sizing of elements and how those elements get displayed on the page. The page layout and how items get arranged next to one another.

Start learning CSS fundamentals to apply styles to the plain-text HTML content by covering topics such as:

• CSS selectors. Learn the different ways you can select HTML elements, how each of them works, and how to choose which one to use. For example, you can use simple selectors that select an HTML element by its tag name, class name, or id name. In addition, a way to select HTML elements is by using pseudo-element selectors and pseudo-class selectors, to name a few.
• Learn CSS specificity. When there are two or more selectors with the same matching CSS rules, how does the browser decide which styles get applied to the HTML element? Every selector has a different specificity value, so learning about specificity will help you avoid any confusion as to why styles are not applied.
• Learn the Box Model. HTML elements are considered boxes, with each box consisting of the content, padding, border, and margin.
• Learn the different ways to position elements on a page. Learn the differences between the relative, absolute, fixed, and sticky position properties.
• Learn about the display and how block elements differ from inline and inline-block ones.
• Learn layout techniques such as Flexbox for creating one-dimensional layouts or CSS Grid for creating two-dimensional ones.
• Learn Media Queries and responsive web design. Websites are viewed on many devices nowadays, such as laptops, mobiles, and tablets, and each device has a different size and resolution. It's paramount that the site looks good and is usable no matter the device.

How to Learn HTML and CSS

The best place to start learning HTML and CSS is with freeCodeCamp's Responsive Web Design Certification.

It's a free, structured, and well thought-out interactive curriculum. You learn in a practical way by building projects.

You will learn all the concepts listed above (and more) by building 15 practice projects and five certification projects.

Once you've completed the certification projects, you can claim the certification and add it to your LinkedIn profile. It's a way to showcase your achievement to your network and prospective employers.

Know Your Way Around The Command Line

Knowing at least the basics of the command line will make you more productive by saving you a significant amount of time when performing repetitive and time-consuming tasks.

You can achieve the same things (and more!) as you can when using a GUI (Graphical User Interface). But with the command line, you'll use keyboard-only navigation and enter text commands instead of clicking and dragging icons.

You can create files and folders, view the contents of files and folders, copy or move the contents of a file to another one, and delete files and folders altogether, to name a few.

You type and enter the commands in a window or CLI (short for Command Line Interface).

Depending on your Operating System, this will be a different application. On a Mac, it’s called Terminal.app. On Windows, it’s called the Command Prompt.

The application runs a shell, such as Bash or Zsh, which acts as an intermediary between you and your computer’s Operating System. You type commands and essentially give orders to your computer. The shell running in the CLI reads the commands and instructs the Operating System. The Operating System carries out the instructions.

No matter the Operating System you are using, whether it's MacOS or Microsoft Windows, as a full stack developer, it will help if you learn Linux. Linux powers the majority of the servers on the Internet.

A way to get started with Linux is by installing it on your Operating System.

You can do this by setting up a Virtual Machine. A Virtual Machine acts as a separate computer inside your computer.

Once you set up a Virtual Machine, you can install one of the Linux distributions, such as Ubuntu, which is a version of Linux.

Here are some helpful resources to get you started:

• Learn the Basics of the Linux Operating System
• Learn the 50 Most-Used Linux & Terminal Commands
• What is a Virtual Machine And How to Setup a VM on Windows, Linux, and Mac

Learn To Use A Code Editor

To start developing your projects locally, you will need a designated place for writing code.

It is not advised to code using a word processor such as Google Docs or Microsoft Word but instead set up a code editor to write source code.

There are many to choose from, with some of the most popular ones being Atom, Sublime Text, and Visual Studio Code. Many developers also use a command-line text editor called VIM. It is not recommended for beginners since there is a steep learning curve.

Visual Studio Code is the editor of choice for many developers, which you will also see referred to as VS Code.

It is free, open-source, and has many features available.

VS Code has powerful tooling and features that resemble an IDE (which stands for Integrated Development Environment).

When using Visual Studio Code, you can write and edit source code, save time with code autocompletion, and use the built-in debugger and terminal. You can compile and run source code, all under the same roof.

Additionally, it is easy to customise your workspace with the different extensions available in Visual Studio Code Marketplace to further enhance your productivity.

Below are links to download Visual Studio Code for your Operating System and for learning how to use it:

• Download VS Code
• Visual Studio Code Course – How to Increase Your Productivity in VS Code
• VS Code Extensions to Increase Developer Productivity

Learn A Version Control System

A Version Control System is a way to back up and save your projects and collaborate with other team members. It is a tool used in every software development job.

The most popular Version Control System is Git, which is free and open-source.

You can view a project’s entire history and keep track of all the changes. If needed, you can also revert to a previous version of it.

Git is not only practical for your personal projects, but it is necessary when you are part of a team.

For example:

• You can download (or clone) a working copy of the entire project’s codebase source code on your local machine. Codebase refers to all the folders and files that make up the project.
• You can stay up to date by pulling all the recent changes that have taken place in the codebase.
• You can make changes safely, without affecting the original codebase, by creating what is known as a branch. Branches allow you to work on bugs or specific features.
• You can review the modifications you made and merge them with the content in the original codebase.

Now, you may have heard of the term GitHub and may be confused about the differences between Git and GitHub.

Git is a tool you install locally to manage your projects, whereas GitHub is an online hosting service. GitHub makes it easier to use Git and is a place for you and your team to upload code.

Here are some helpful resources for you to learn Git and GitHub:

• Git and GitHub Crash Course
• Git and GitHub Tutorial – Version Control for Beginners
• The beginner’s guide to Git & GitHub
• How to Use Git and GitHub in a Team like a Pro – Featuring Harry and Hermione 🧙

Now that you are familiar with the command line, Git, and GitHub, you can deploy a static HTML and CSS webpage and publish it to the Internet. You can do this using either GitHub pages or Netlify.

Learn a Programming Language

Programming languages act as the translator between humans and machines.

Programming languages are similar to human languages. They have syntactical elements like nouns, verbs, and phrases. And you group these elements to form something that resembles a sentence to create meaning.

Some of these languages resemble and look a lot like English. But they offer a shorter, more precise, and less verbose way to create instructions that the computer can understand.

A spoken/natural language like English, on the other hand, leaves a lot of room for ambiguity and different interpretations from different people. With programming languages, you don't have that ambiguity.

How to Make Web Pages Dynamic with JavaScript

You may have noticed that I used the word static to describe the type of webpages created when using just HTML and CSS.

To add dynamic behaviors and interactivity to the web pages, you need to use JavaScript alongside HTML and CSS.

JavaScript is a scripting language designed to run in the browser. It adds interactive features to otherwise static web pages.

Some interactive features JavaScript adds are:

• A map with a user’s current location
• A message shown to a user once they land on the page
• A change on the page based on user input
• An animation when a user clicks a button
• A smooth scroll effect when there is a button to scroll back up to the top of the page.

To start learning JavaScript, begin studying the basic syntax, or what is known as Vanilla JavaScript.

There are a lot of concepts to learn, and this is not a complete list, but some of the topics to pay attention to are:

• Data Types
• Variables and Scope
• Conditional statements
• Loops
• Iterators
• Functions
• Objects
• DOM Manipulation
• Async JS
• Event Handling
• ES6 syntax
• Arrays
• Array Methods
• Call backs
• Promises

To start learning JavaScript, you can go through freeCodeCamp’s JavaScript Algorithms and Data Structures certification.

You will start with the fundamentals and progress to more advanced topics such as Object-Oriented Programming and Data Structures.

In the end, you will build five certification projects, some of which include a telephone number validator and a Roman numeral converter.

Another fun way to sharpen your JavaScript skills is by building games.

Check out the following videos from freeCodeCamp’s YouTube channel to get started:

• Learn JavaScript by Building 7 Games - Full Course
• Easy JavaScript Game Development with Kaboom.js (Mario, Zelda, and Space Invaders) - Full Course
• JavaScript Tutorial: Build Flappy Bird and Doodle Jump

JavaScript Libraries and Frameworks

Before moving on to different technologies, you need to be comfortable with the basic JavaScript concepts. Take time to practice and build some projects using the language.

Then, you can learn one of the JavaScript libraries or frameworks.

JavaScript libraries and frameworks provide pre-written code that implements specific functionalities. They help you develop web applications much easier and faster and with more consistency across the different parts by creating what are known as UI components.

The most popular and commonly used JavaScript library to learn is ReactJS.

To learn React, here are some helpful resources to help you get started:

• Learn React JS in This Free 7-Hour Course
• Free React Course for 2022
• Learn ReactJS – Complete Roadmap
• If you are a Spanish speaker, or have a friend/ family member that is a Spanish speaker and wants to learn React JS, check out this 8-hour free course on React in Spanish.

Server-side Scripting Languages

As a full-stack web developer, you will need to know both front-end technologies and back-end tools. You will also need to be able to work with server-side scripting programming languages.

There are many to choose from, such as PHP, Ruby, and Java, to name a few.

And just a note about JavaScript – it's used widely in front-end development, but in recent years many developers have started using it for back-end development too.

This is all thanks to NodeJS (a JavaScript runtime environment) which makes that possible by providing back-end functionality.

Pairing NodeJS with the ExpressJS server-side web framework, you are now able to create full-stack web applications.

To learn Node JS and Express JS, check out the following resources:

• Build Four Node.js and Express.js Projects
• Learn Node.js and Express with This Free 8-hour Back End Development Course

Knowledge of SQL and Relational Databases

As a full-stack developer, you will need to know how to interact with databases.

Databases fall into two categories, relational and non-relational.

Relational databases (also referred to as SQL databases) store data in a structured, organized, tabular format.

There are clearly defined relationships between the data points, which makes searching for data easy.

Some relational databases include:

• MySQL
• Postgresql
• Oracle
• Microsoft SQL Server
• SQLlite

The way to communicate with relational databases and manipulate stored data is to query them using a query language such as SQL (short for Structured Query Language).

It helps to have an understanding of how relational databases and SQL work. A great place to start your learning is with freeCodeCamp's Relational Database Course.

You will learn relational databases through interactive tutorials and projects using SQL and PostgreSQL.

Non-relational databases (also referred to as NoSQL databases) don’t store data in tables.

They instead store data in a less structured manner, and items are not related to one another, which allows for more flexibility.

The most popular non-relational database is MongoDB.

MongoDB, along with some of the technologies mentioned in previous sections, is part of the MERN stack. The MERN stack is a set of tools used for developing full-stack web applications.

So putting it all together, this is what MERN stands for:

• MongoDB is the non-relational database for storing data.
• ExpressJS is a server-side web framework layered on top of Node JS.
• ReactJS is a JavaScript client-side web library for building user interfaces
• NodeJS is the JavaScript runtime environment for running JavaScript on a server and not just the browser.

To learn more about MongoDB, here are some resources to get you started:

• Back End Development and APIs Certification
• MongoDB Full Course w/ Node.js, Express, & Mongoose
• Learn the MERN stack by building an exercise tracker app (MERN Tutorial)

Conclusion

Hopefully, you found this article helpful, and you now have a better understanding of what full-stack web development entails.

In this article, we went over the definition of full-stack development, and you saw a suggested learning path to get you started with full-stack development.

Thank you for reading!

But what do these terms mean exactly?

In this article, you will learn about the differences between the two by examining the frontend and backend separately.

Here is what we will cover:

1. Defining frontend
   1. What is condidered frontend?
   2. Why is frontend important?
   3. Common frontend tasks
   4. Frontend tools and technologies
   5. Is frontend development right for you?
2. Defining backend
   1. How the backend works
   2. Examples of backend functionality
   3. Common backend tasks
   4. Backend tools and technologies
   5. Is backend development right for you?

What is Frontend?

The frontend is everything a user sees and interacts with when they click on a link or type in a web address. The web address is also known as at URL, or Uniform Resource Locator, and it tells what webpage should load and appear in your browser.

It is the client-side part of a web application.

For example, take this article. There is an associated cover picture, and you are now reading through the text.

At the top of the page, you can also see freeCodeCamp's logo. There is a navigation bar with a link to freeCodeCamp's forum and a button to donate.

There is also a search box. You can enter a keyword to search for an article on a topic that interests you.

There is a link to click, which will take you to lessons and certifications that will help you learn to code for free.

All the above represent the frontend of the web page you're viewing.

What Parts of a Web Application are Considered Frontend?

The frontend is anything and everything visual that a user comes in contact with.

It's all the parts with which they directly interact.

It's all the content and styles.

It's the buttons and the different hover effects before a user clicks on a link.

It's the contact forms with various input fields, the search boxes and the dropdown menus.

The layouts, text, and colors.

It's the images and videos.

However, it is not just the styles.

It is also how fast the website loads, how easy it is to navigate through it, and how accessible it is to people with disabilities.

It's how usable and responsive it is on a variety of different devices and browsers.

Essentially, the frontend is all the parts of a web application that create the look and feel of it.

Why Frontend is Important

The frontend represents someone's (or a team's) idea, vision, and a dream come to life, available for the world to see and witness.

The idea needs to be represented in the right way and be visually appealing. It will be the determining factor of how others will perceive it.

A company or organization with a site that is difficult to use and navigate through, which doesn't look good, and does not present its services nicely, will lose customers and clients.

Users will become frustrated due to the lack of usability, the unpleasant user experience, and the lack of good design. They will end up searching elsewhere for what they were looking for and will most likely not return to that site again.

What Are Some of a Frontend Developer's Tasks?

The tasks a frontend developer completes and works on day-to-day will vary. They will heavily depend on the company they work for and the role.

As a frontend dev, you may do a lot of design work.

That could be creating a style guide to create consistent styles and the brand's identity and overall look and image.

It includes the typography (fonts) used for all text, the color scheme, the company's logo, and layouts, to name a few.

You may work on building User Interfaces using design tools to make sure the website has all the necessary visual elements it needs, displayed and organized pleasantly.

On the other hand, frontend developers may not do any design work.

Instead, you could collaborate with program managers, graphic designers, and UX/UI designers to solve problems.

You could take the created visuals, graphics, and findings from the conducted user research.

You would already know the target audience and demographic, common customer problems, and information on how usable the site is for users.

Then, your job as a frontend developer would be to translate the prototypes and static visual designs to something tangible, to a real-world, pixel-perfect, and functional website that users can easily interact with.

Using frontend technologies, you would implement the layouts already designed and build all the visual elements.

A big part of a frontend developer's job is to create responsive websites.

You need to make sure websites look good, are optimally displayed, and are usable on a variety of devices and screen widths.

Nowadays, users don't use only desktop computers, but they instead view websites mostly from mobile and tablet devices.

In addition, frontend developers design and develop with many of the most popular and most used browsers in mind.

Websites need to look good and need to function optimally on all browsers.

Here is where knowing your target audience and demographic comes to play and is vital information.

Depending on your service/product and your users/customers, you may need to develop with older or more modern browsers in mind.

Another big part of a frontend developer's job is to make sure websites are accessible to all users. This entails creating web accessibility functions that would make browsing the web a pleasant experience for everyone.

Important accessibility features a frontend developer can implement are:

• text-to-speech
• keyboard-only navigation
• accessible color combinations with sufficient contrast
• large buttons

Frontend developers also work on improving the site's performance and load time.

The majority of users become impatient and click off if a website takes too long to load its content and images.

Improving the time it takes to load the website, even by a few seconds, makes a big difference and will most likely retain users.

Another part of a frontend developer's role is to design and create usable web forms.

Web forms bridge the communication gap between the company/organization and users. They are an easy way for users to make inquiries, contact customer service, submit data, and create accounts.

An Overview of Frontend Tools and Technologies

As a frontend developer, you will constantly learn new technologies, depending on what is popular at the time. Technologies change fast.

However, the most fundamental frontend technologies are HTML, CSS, and JavaScript.

They run in the user's browser and are the building block of all websites on the World-Wide Web.

Let's talk a bit more about these technologies:

• HTML stands for HyperText Markup Language. You create all the content and structure of a webpage using semantic HTML.
• CSS stands for Cascading Style Sheets. CSS styles the HTML elements. It adds colors, fonts, it determines the size of text and the layouts of web pages.
• Vanilla JavaScript is how you add functionality and interactivity to elements. It's how you make web pages have dynamic behavior.

Besides those three, there are many frontend libraries and frameworks.

These help ensure consistency across each webpage in a website and provide out of the box functionalities for repetitive tasks.

Some of the most popular ones are:

• React
• Redux
• Vue
• Angular

Depending on the role and job description, you may need design skills.

Some commonly used tools used for design work are:

• Figma
• Sketch
• Adobe XD
• and Photoshop for image processing

Is Frontend Development Right for You?

Frontend development is a creative and artistic endeavor.

There is a blend of both visual art and programming.

A way to decide whether frontend development is for you is to consider whether you enjoy seeing the visual results of your hard work.

It is possible to see the results of your creation instantly on the screen of your computer.

Another thing to consider is whether you care deeply about users having the best possible user experience when browsing the web.

User experience and creating usable websites are a big part of the job.

The only way to find out if frontend development is right path for you is to give it a go.

A great place to start your learning is with freeCodeCamp's Responsive Web Design Certification, where you will learn by building 20 projects and earn a certification at the end.

You will learn HTML and modern CSS techniques alongside the best accessibility practices. You will also learn to create responsive web pages that accommodate each screen size.

Following the Responsive Web Design Certification, learn the JavaScript programming language with the JavaScript Algorithms and Data Structures Certification, where you will learn how to make web pages interactive.

You will start from the basics, learning the syntax and necessary fundamental building blocks that will allow you to progress to more advanced topics such as Object-Oriented Programming.

What is Backend?

Backend development deals with the technologies responsible for storing and securely manipulating user data.

It is the part associated with all the hidden logic that powers the applications users interact with.

Backend is considered the server-side part of an application.

Backend is all the hidden inner workings and the behind-the-scenes processes in a web application.

It refers to everything going on underneath the hood and all the necessary components that make the front-end function properly and smoothly.

It makes sure everything is working optimally.

Essentially, the backend is what the users don't have direct access to or don't directly interact with and are most likely unaware of when using an application.

An Overview of How Backend Works for Beginners

The back-end is responsible for receiving requests from a client.

It processes the incoming request and makes sure it fetches the appropriate data associated with the correct user.

Then, it sends a response back to the client. The necessary user data is presented nicely visually, with frontend code created by a frontend developer, to the right user that has access to it.

The backend consists of:

• The server, which is a program that waits and listens for incoming requests from the client. It communicates with the database by submitting database queries. When the database responds with information, the server fetches the correct necessary data.

It achieves this with the help of an application running on the server that contains the necessary logic that gathers the requested information and resources for the right user. Backend server-side scripts, written in a backend scripting programming language, are responsible for processing the requests.

Lastly, the server sends a response back to the client.

• The database, which is like the 'brain' of an application. Databases are programs used to organize and store all of an application's assets, content, and information in a way that can be easily accessed and retrieved, managed, and updated.

Examples of Backend Functionality

Let's take a real-world example to illustrate the different backend functionalities available on modern web applications.

Say that you are a member of a paid workout subscription platform.

To login:

• You would need to enter your email address and password in a user input field.
• There would be basic validation checks that would need to pass. Validations ensure that the required input fields are not empty and the email address has the correct format.
• Then, there are checks to see if you exist as a user in the database by sumbmitting database queries, to see if your email address and password are correct. There would also be checks to see whether the email address and password you provided match the email and password combination stored in the database when you created your account.
• If the email and password are incorrect, you would see an error message saying that the combination doesn't exist.
• If everything is correct, you would be directed to your Home page, with a visual cue indicating your login. It could be a 'Welcome' message that includes your name or username.

To pick a workout:

• The database stores all your raw data. Workouts already completed are stored in the database. The same goes with any saved in a playlist you created or marked as favorites that you would like to revisit.
• You can also view all the available workouts and filter through them to only show a particular category. For example, you could type a keyword in the search box such as abs, arms, lower body, or upper body. Another way could be to pick from one of the available categories on the site, such as yoga, stretch, HIIT, or strength. The database will go through every workout stored there, but you will view only the filtered results of the particular category you specified in your search.

To pay for the subscription:

• The application would be able to accept credit card information or other payment options and store that information securely in the database.
• It would also update the payment at the right time. Depending on the subscription, there would be a reoccurring monthly or yearly update, making sure each time it withdraws the right amount of money from your card.

All this is handled by the back end of an application.

What Are Some of a Backend Developer's Tasks?

It's safe to say that the role of a backend developer varies from company to company and from job to job.

The main focus of a backend developer's work is to create and maintain services and programs that help the front end function.

In general, here are some things a backend developer would work on:

• Creating, managing, and maintaining the type of database the product/service uses.
  • Building, interacting with, and maintaining servers.
  • Building internal or external functionalities and server-side software using server-side technologies and web frameworks to provide solutions to problems.
  • Working with APIs (Application Programming Interface). Designing, developing, implementing, maintaining, and managing APIs that support CRUD (Create Read Update Delete) operations.
  • Performing data validation to ensure data has the correct format before being stored in the database.
  • Users need to interact safely with their accounts. Backend developers create systems that secure every user's data - especially when dealing with payment processing systems.
  • Dealing with third-party services such as authentication and external payment services, to name a few.
  • Organizing system logic.
  • Developing site architecture.

An Overview of Backend Tools and Technologies

There are different tools and technologies backend developers use on a day-to-day basis to implement logic into web applications.

A key component of backend programming is using a server-side scripting programming language.

Some of the most frequently used ones are:

• PHP
• Ruby
• Python
• Java
• JavaScript. Yes, JavaScript is used widely in frontend development, but in recent years is used for backend development too. Node.js (a JavaScript runtime) makes that possible by providing backend functionality.

Besides backend programming languages, backend frameworks and libraries are used to provide extra functionality to create web applications.

Some of the most popular ones are:

• Ruby on Rails
• Django
• Flask
• Express

When developers are part of a team, they use Git - a version control system that tracks changes across different project files.

It's an open-source collaboration tool for programmers.

Interacting with databases is a big part of a backend developer's job, so knowledge of databases is paramount.

Databases fall into two categories, relational and non-relational.

Some of the most oftenly used ones are:

• MySQL
• PostgreSQL
• MongoDB
• Oracle

When working with relational databases (such as MySQL, PostgreSQL, and Oracle), you will use SQL.

SQL stands for Structured Query Language. It is used for performing database queries. SQL communicates with relational databases to manipulate data.

You can learn more about SQL from this helpful list of resources.

Is Backend Development Right for You?

Backend developers are in charge of the logic behind web applications and making the frontend function optimally.

If you enjoy solving problems by breaking them down into smaller pieces, implementing algorithms, and finding an optimal and most effective solution, maybe backend development is for you.

If you consider yourself an analytical and methodical person, backend development involves working with, analyzing, and organizing large amounts of data. It requires performing manipulations and calculations with that data.

Lastly, if you don't consider yourself a particularly artistic person and are more concerned with the logic side of things and what goes on underneath the hood, maybe you are more suited to backend development.

That said, folks may start with the frontend and then transition to the backend side of things. This way, they can figure out which they prefer working with the most.

If you want to start learning backend development, learn with freeCodeCamp's Relational Database Course.

You will learn about relational databases using SQL and PostgreSQL, the Git version control system, to name a few. These will get you off to a great start in your backend journey.

Then, you can progress to the Back End Development and APIs Certification.

Here, you will learn Node.js and build web applications using the Express framework.

Conclusion

Hopefully, now you have more insight into what frontend and backend development entails and can tell the differences between the two.

To summarize, the frontend involves all the parts a user interacts with, whereas the backend is the logic that powers the frontend interactions.

Thanks for reading!

In this tutorial, we will be learning how to build and deploy an image management application backend.

It will be able to store a record of an image in the database, get the image's record back from the database, update the record, and even delete the record completely as the case may be.

To achieve all of this, we will be using Express (a Node.js framework), Postgres (a database), Cloudinary (a cloud based image storage), GitHub (for version control/storage) and Heroku (a hosting platform).

These tools are all free. So you don’t have to bother about how to go about paying for them. Thanks to these great innovators.

Prerequisites

If you are new to most of these technologies, I would advise you go through my other tutorial on how to create a server and upload images to Cloudinary.

If you are totally to Postgres, then check out this tutorial.

Whenever you are ready, let’s get to work!

How to Store and Retrieve an Image Record

Create Database and Table

So you'll want to start by cloning this project if you don't already have it.

In your pgAdmin:

• Create a database and name it tutorial
• Create a table and name it tutorial
• Create a Login/Group Role and name it tutorial. (Do not forget to give it all privileges.)

Back in your project directory, install the node-postgres (npm i pg) and make-runnnable (npm i make-runnable) packages.

In your package.json file, replace the contents of the "scripts" with "create": "node ./services/dbConnect createTables". We will use this to execute the dbConnect file we are about to create.

Create a services/dbConnect file to contain the following code:

const pg = require("pg");

const config = {
  user: "tutorial",
  database: "tutorial",
  password: "tutorial",
  port: 5432,
  max: 10, // max number of clients in the pool
  idleTimeoutMillis: 30000,
};

const pool = new pg.Pool(config);

pool.on("connect", () => {
  console.log("connected to the Database");
});

const createTables = () => {
  const imageTable = `CREATE TABLE IF NOT EXISTS
    images(
      id SERIAL PRIMARY KEY,
      title VARCHAR(128) NOT NULL,
      cloudinary_id VARCHAR(128) NOT NULL,
      image_url VARCHAR(128) NOT NULL
    )`;
  pool
    .query(imageTable)
    .then((res) => {
      console.log(res);
      pool.end();
    })
    .catch((err) => {
      console.log(err);
      pool.end();
    });
};

pool.on("remove", () => {
  console.log("client removed");
  process.exit(0);
});

//export pool and createTables to be accessible from anywhere within the application
module.exports = {
  createTables,
  pool,
};

require("make-runnable");

Now we are all set to create the table in our database. If you are ready, let's rock and roll!

Execute the following code in your terminal:

  npm run create

If the image below is your result, then you are good to go:

Check your pgAdmin, and you should have your table seated properly in your database like in the image below:

Alright, it's been a long road. It's time to unite Node, Postgres, and Cloudinary.

How to Create Endpoints to Store and Retrieve Image Records

Endpoint 1: Persist Image

First, require the dbConnect.js file on the top of the app.js file like so:

  const db = require('services/dbConnect.js');

Then in the app.js file, make a new endpoint (persist-image) with the following code:

// persist image
app.post("/persist-image", (request, response) => {
  // collected image from a user
  const data = {
    title: request.body.title,
    image: request.body.image,
  }

  // upload image here
  cloudinary.uploader.upload(data.image)
  .then().catch((error) => {
    response.status(500).send({
      message: "failure",
      error,
    });
  });
})

Replace the then block with the following code:

.then((image) => {
    db.pool.connect((err, client) => {
      // inset query to run if the upload to cloudinary is successful
      const insertQuery = 'INSERT INTO images (title, cloudinary_id, image_url) 
         VALUES($1,$2,$3) RETURNING *';
      const values = [data.title, image.public_id, image.secure_url];
    })
  })

The image.public_id and image.secure_url are gotten as part of the details returned for an image after the image has been successfully uploaded to Cloudinary.

We are now keeping a record of the image.public_id and image.secure_url (as you can see in the code above) in order to use it to retrieve, update, or delete the image record when we see fit.

Alright, let's move forward!

Still in the then block, add the following code under the query we created:

// execute query
client.query(insertQuery, values)
      .then((result) => {
        result = result.rows[0];

        // send success response
        response.status(201).send({
          status: "success",
          data: {
            message: "Image Uploaded Successfully",
            title: result.title,
            cloudinary_id: result.cloudinary_id,
            image_url: result.image_url,
          },
        })
      }).catch((e) => {
        response.status(500).send({
          message: "failure",
          e,
        });
      })

So our persist-image endpoint now looks like this:

// persist image
app.post("/persist-image", (request, response) => {
  // collected image from a user
  const data = {
    title: request.body.title,
    image: request.body.image
  }

  // upload image here
  cloudinary.uploader.upload(data.image)
  .then((image) => {
    db.pool.connect((err, client) => {
      // inset query to run if the upload to cloudinary is successful
      const insertQuery = 'INSERT INTO images (title, cloudinary_id, image_url) 
         VALUES($1,$2,$3) RETURNING *';
      const values = [data.title, image.public_id, image.secure_url];

      // execute query
      client.query(insertQuery, values)
      .then((result) => {
        result = result.rows[0];

        // send success response
        response.status(201).send({
          status: "success",
          data: {
            message: "Image Uploaded Successfully",
            title: result.title,
            cloudinary_id: result.cloudinary_id,
            image_url: result.image_url,
          },
        })
      }).catch((e) => {
        response.status(500).send({
          message: "failure",
          e,
        });
      })
    })  
  }).catch((error) => {
    response.status(500).send({
      message: "failure",
      error,
    });
  });
});

Now let's test out all our hard work:

Open your postman and test out your endpoint like the image below. Mine was successful. Hope yours had no errors too?

Open your Cloudinary console/dashboard and check your media Library. Your new image should be sitting there comfortably like mine below:

And now to the main reason why we are here: check the images table in your pgAdmin. Mine is what you see below:

Oohlala! We made it this far. Please take a break if you need one. I will be here waiting when you return. :)

If you are ready, then let's retrieve the image we persisted a moment ago.

Endpoint 2: Retrieve Image

Start with this code:

app.get("/retrieve-image/:cloudinary_id", (request, response) => {

});

Next, we will need to collect a unique ID from the user to retrieve a particular image. So add const { id } = request.params; to the code above like so:

app.get("/retrieve-image/:cloudinary_id", (request, response) => {
  // data from user
  const { cloudinary_id } = request.params;

});

Add the following code just below the code above:

db.pool.connect((err, client) => {
      // query to find image
    const query = "SELECT * FROM images WHERE cloudinary_id = $1";
    const value = [cloudinary_id];
    });

Under the query, execute the query with the following code:

// execute query
    client
      .query(query, value)
      .then((output) => {
        response.status(200).send({
          status: "success",
          data: {
            id: output.rows[0].cloudinary_id,
            title: output.rows[0].title,
            url: output.rows[0].image_url,
          },
        });
      })
      .catch((error) => {
        response.status(401).send({
          status: "failure",
          data: {
            message: "could not retrieve record!",
            error,
          },
        });
      });

Now our retrieve-image API looks like this:

app.get("/retrieve-image/:cloudinary_id", (request, response) => {
  // data from user
  const { cloudinary_id } = request.params;

  db.pool.connect((err, client) => {
    // query to find image
    const query = "SELECT * FROM images WHERE cloudinary_id = $1";
    const value = [cloudinary_id];

    // execute query
    client
      .query(query, value)
      .then((output) => {
        response.status(200).send({
          status: "success",
          data: {
            id: output.rows[0].cloudinary_id,
            title: output.rows[0].title,
            url: output.rows[0].image_url,
          },
        });
      })
      .catch((error) => {
        response.status(401).send({
          status: "failure",
          data: {
            message: "could not retrieve record!",
            error,
          },
        });
      });
  });
});

Let's see how well we did:

In your postman, copy the cloudinary_id and add it to the URL like in the image below:

YEEESSS! We can also retrieve our image.

If you are here, then you deserve a round of applause and a standing ovation for your industriousness.

Congratulations! You just reached a great milestone.

The code for storing and retrieving image records is here.

How to Update and Delete an Image Record

We will now see how to delete and update an image record as the case maybe. Let's begin with the delete endpoint.

Delete Endpoint

In the app.js file, start with the following code:

// delete image
app.delete("delete-image/:cloudinary_id", (request, response) => {

});

Next, we want to get the unique ID of the image we want to delete from the URL, that is cloudinary_id. So inside the code above add:

const { cloudinary_id } = request.params;

We now start the deleting process.

First, we delete from Cloudinary. Add the following code to delete the image from Cloudinary:

cloudinary.uploader
    .destroy(cloudinary_id)
    .then((result) => {
      response.status(200).send({
        message: "success",
        result,
      });
    })
    .catch((error) => {
      response.status(500).send({
        message: "Failure",
        error,
      });
    });

At this point, our API can delete the image from Cloudinary only (you can check it out in postman). But we also want to get rid of the record we have in our Postgres database.

Second, we delete from our Postgres database. To do so, replace the code in the then block with the following query:

db.pool.connect((err, client) => {

      // delete query
      const deleteQuery = "DELETE FROM images WHERE cloudinary_id = $1";
      const deleteValue = [cloudinary_id];

})

Execute the query with the following code underneath it:

// execute delete query
      client.query(deleteQuery, deleteValue)
      .then((deleteResult) => {
        response.status(200).send({
          message: "Image Deleted Successfully!",
          deleteResult
        });
      }).catch((e) => {
        response.status(500).send({
          message: "Image Couldn't be Deleted!",
          e
        });
      });

So our Endpoint should look like this:

// delete image
app.delete("/delete-image/:cloudinary_id", (request, response) => {
  // unique ID
  const { cloudinary_id } = request.params;

  // delete image from cloudinary first
  cloudinary.uploader
    .destroy(cloudinary_id)

    // delete image record from postgres also
    .then(() => {
      db.pool.connect((err, client) => {

      // delete query
      const deleteQuery = "DELETE FROM images WHERE cloudinary_id = $1";
      const deleteValue = [cloudinary_id];

      // execute delete query
      client
        .query(deleteQuery, deleteValue)
        .then((deleteResult) => {
          response.status(200).send({
            message: "Image Deleted Successfully!",
            deleteResult,
          });
        })
        .catch((e) => {
          response.status(500).send({
            message: "Image Couldn't be Deleted!",
            e,
          });
        });
      })
    })
    .catch((error) => {
      response.status(500).send({
        message: "Failure",
        error,
      });
    });
});

The time has arrived for us to put our Endpoint to the test.

The following is my Cloudinary media library with two images I uploaded already. Take note of their unique ID (public_id).

If you don't already have that, please use the persist-image endpoint to upload some images.

Now let's proceed to postman:

Notice, the unique ID as it matches one of the image in my Cloudinary media library.

From the output, we executed the DELETE command and that deleted one ROW from our image TABLE in our database.

Now this is my media library with one of the images remaining:

Walahhhh... We are now able to get rid of an image.

Do take a break if you need one. ✌🏾

If you are ready, I am ready to update images.

Update Image API

Below the delete-image API, let's start creating the update-image API with the following code:

// update image
app.put("/update-image/:cloudinary_id", (request, response) => {

});

All codes will live in there.

Collect the unique Cloudinary ID and new image details from the user with the following code:

// unique ID
  const { cloudinary_id } = request.params;

// collected image from a user
  const data = {
    title: request.body.title,
    image: request.body.image,
  };

Delete the image from Cloudinary with the following code:

// delete image from cloudinary first
  cloudinary.uploader
    .destroy(cloudinary_id)
      // upload image here
    .then()
    .catch((error) => {
      response.status(500).send({
        message: "failed",
        error,
      });
    });

Next, upload another image to Cloudinary. To do that, enter the following code into the then block:

() => {
      cloudinary.uploader
        .upload(data.image)
        .then()
        .catch((err) => {
          response.status(500).send({
            message: "failed",
            err,
          });
        });
    }

Now let's replace our initial record with the new image details. Replace the content of the then block with the following:

(result) => {
          db.pool.connect((err, client) => {

            // update query
            const updateQuery =
              "UPDATE images SET title = $1, cloudinary_id = $2, image_url = $3 WHERE cloudinary_id = $4";
            const value = [
              data.title,
              result.public_id,
              result.secure_url,
              cloudinary_id,
            ];
          });
        }

We execute the query using the following code just beneath the query declaration:

// execute query
            client
              .query(updateQuery, value)
              .then(() => {

                // send success response
                response.status(201).send({
                  status: "success",
                  data: {
                    message: "Image Updated Successfully"
                  },
                });
              })
              .catch((e) => {
                response.status(500).send({
                  message: "Update Failed",
                  e,
                });
              });

At this point, this is what I have:

// update image
app.put("/update-image/:cloudinary_id", (request, response) => {
  // unique ID
  const { cloudinary_id } = request.params;

  // collected image from a user
  const data = {
    title: request.body.title,
    image: request.body.image,
  };

    // delete image from cloudinary first
    cloudinary.uploader
      .destroy(cloudinary_id)

      // upload image here
      .then(() => {
        cloudinary.uploader
          .upload(data.image)

          // update the database here
          .then((result) => {
            db.pool.connect((err, client) => {
            // update query
            const updateQuery =
              "UPDATE images SET title = $1, cloudinary_id = $2, image_url = $3 WHERE cloudinary_id = $4";
            const value = [
              data.title,
              result.public_id,
              result.secure_url,
              cloudinary_id,
            ];

            // execute query
            client
              .query(updateQuery, value)
              .then(() => {

                // send success response
                response.status(201).send({
                  status: "success",
                  data: {
                    message: "Image Updated Successfully"
                  },
                });
              })
              .catch((e) => {
                response.status(500).send({
                  message: "Update Failed",
                  e,
                });
              });
            });
          })
          .catch((err) => {
            response.status(500).send({
              message: "failed",
              err,
            });
          });
      })
      .catch((error) => {
        response.status(500).send({
          message: "failed",
          error,
        });
      });

});

It's testing time!

This is my postman in the image below:

Take note of the unique cloudinary ID which matches the image left in my Cloudinary media library.

Now take a look at my Cloudinary media library in the image that follows:

Take note of the new image replacing the initial one in my media library above.

Also, see that the unique Cloudinary ID matches that in my database with the new title. See image below:

Yayeh! You did awesomely great! 💪

We just completed an image management application with Node.js, Cloudinary and Postgres.

Code Optimisation With Express Routing

Express Routing enables us to make our Node.js code more optimized or give it a more modular structure by separating the business logic from the controllers. We want to use that to clean up our code so far.

We'll begin by creating a new folder with the name routes in the root directory:

mk dir routes

In the routes folder, create a file with the name: routes.js.

For Windows:

echo . > routes.js

For Mac:

touch routes.js

Empty the routes.js file if anything is there and enter the following code:

const express = require('express');

const router = express.Router();



module.exports = router;

Add the following code above the last line:

const cloudinary = require("cloudinary").v2;
require("dotenv").config();
const db = require("../services/dbConnect.js");

// cloudinary configuration
cloudinary.config({
  cloud_name: process.env.CLOUD_NAME,
  api_key: process.env.API_KEY,
  api_secret: process.env.API_SECRET,
});

Back in the App.js file, delete the following code:

const cloudinary = require("cloudinary").v2;
require("dotenv").config();
const db = require("./services/dbConnect.js");

// cloudinary configuration
cloudinary.config({
  cloud_name: process.env.CLOUD_NAME,
  api_key: process.env.API_KEY,
  api_secret: process.env.API_SECRET,
});

Move all the APIs to routes.js.

Change all occurence of app to router carefully.

My routes.js file now looks like this.

Back in the app.js file, import the routes.js file like so:

// import the routes file
const routes = require("./routes/routes")

Now register the routes like so:

// register the routes 
app.use('/', routes);

This is my app.js file at the moment:

const express = require("express");
const app = express();

// import the routes file
const routes = require("./routes/routes")

// body parser configuration
const bodyParser = require("body-parser");
app.use(bodyParser.json());
app.use(bodyParser.urlencoded({ extended: true }));

// register the routes 
app.use('/', routes);

module.exports = app;

It's time to test and see if our routes are still working like before.

Make sure yours are working like mine below:

persist-image

retrieve-image

update-image

delete-image

Wow! We have been able to separate our routes from our app.js file.

The code for this is here.

Even though our routes.js file is still lengthy, we have a good basis to separate our business logic from our controllers. The time has arrived to do just that.

How to Move Each Endpoint to a Different File

Begin by creating a new folder in the routes folder and name it controllers.

In the controllers folder, create 5 files and name them after the 5 endpoints.

Our folder and files should be structured as follows:

Back in the routes.js file, let's work on the image-upload API. Cut the following code:

(request, response) => {
  // collected image from a user
  const data = {
    image: request.body.image,
  };

  // upload image here
  cloudinary.uploader
    .upload(data.image)
    .then((result) => {
      response.status(200).send({
        message: "success",
        result,
      });
    })
    .catch((error) => {
      response.status(500).send({
        message: "failure",
        error,
      });
    });
}

In the imageUpload file, equate the code you already cut from the image-upload endpoint to exports.imageUpload like so:

exports.imageUpload = (request, response) => {
    // collected image from a user
    const data = {
      image: request.body.image,
    };

    // upload image here
    cloudinary.uploader
      .upload(data.image)
      .then((result) => {
        response.status(200).send({
          message: "success",
          result,
        });
      })
      .catch((error) => {
        response.status(500).send({
          message: "failure",
          error,
        });
      });
  }

Now let's import what is necessary for this code to work. So this is my imageUpload file right now:

const cloudinary = require("cloudinary").v2;
require("dotenv").config();

// cloudinary configuration
cloudinary.config({
  cloud_name: process.env.CLOUD_NAME,
  api_key: process.env.API_KEY,
  api_secret: process.env.API_SECRET,
});

exports.imageUpload = (request, response) => {
    // collected image from a user
    const data = {
      image: request.body.image,
    };

    // upload image here
    cloudinary.uploader
      .upload(data.image)
      .then((result) => {
        response.status(200).send({
          message: "success",
          result,
        });
      })
      .catch((error) => {
        response.status(500).send({
          message: "failure",
          error,
        });
      });
  }

Let's import and register the imageUpload API in the routes.js file like so:

const imageUpload = require("./controllers/imageUpload");

// image upload API
router.post("image-upload", imageUpload.imageUpload);

Now we have this line of code pointing to the imageUpload API in the imageUpload.js file from the routes.js file.

How awesome! Our code is more readable.

Make sure to test the API to be sure it's working properly. Mine works perfectly. See image below:

Now, it's your turn!

Apply what you have learnt to the other APIs. Let's see what you have got.

I will be waiting on the other side...

If you are here, then I believe you have done yours and they're working perfectly – or at least, you already gave it your best shot. Kudos!

Checkout mine here.

Congratulations. You are awesome :)

The code optimisation code is here.

Alright, on to the next step.

How to Deploy to GitHub And Heroku

Now that we've completed our application, let's deploy it on Heroku so that we can access it even without being on our laptop where the code was written.

I will be walking you through uploading our application to GitHub and deploying it to Heroku.

Without further ado, let's get our hands dirty.

How to Upload the Code to GitHub

Uploading or pushing to GitHub is as easy as eating your favorite meal. Check out this resource to learn how to push your project from you local machine to GitHub.

How to Deploy to Heroku

Let's begin by creating an account on Heroku.

If you have created an account, you may have been prompted to create an app (that is a folder where your app will be housed). You can do that, but I will do mine using my terminal since the terminal comes with a few added functionalities that we will need later.

Open your project in a terminal if you have not done so already. I will be using the VS Code integrated terminal.

Install Heroku CLI:

npm install heroku

Login to Heroku CLI. This will open a browser window, which you can use to log in.

heroku login

Create an app. It can have any name. I am using node-postgres-cloudinary.

heroku create node-postgres-cloudinary

Go to your Heroku dashboard and you will find the newly created app.

Waalaah!

That is how mine looks in the image above. I have some apps there already but you can see the one I just created.

Let's now add the PostgreSQL database to the app.

How to Add Heroku Postgres

Click on the app you just created. It will take you to the app's dashboard.

Click on the Resources tab/menu.

In the Add-ons Section, search and select Heroku Postgres.

Make sure you select the Hobby Dev - Free plan in the pop-up window that follows:

Click on the provision button to add it to the app like so:

Click on the Heroku Postgres to take you to the Heroku Postgres dashboard.

Click on the settings tab:

Click on View Credentials:

In the Credentials, we are interested in the Heroku CLI. We will be using it in a bit.

Back to the terminal.

Let's confirm if the Heroku Postgres was added successfully. Enter the following in the terminal:

heroku addons

Yeeeeaaaah! It was added successfully.

Before we proceed, make sure that your PostgreSQL path is set correctly if you are on Windows. Follow this link to learn how to set a path. The path should be like this: C:\Program Files\PostgreSQL\<VERSION>\bin.

The version will depend on the one installed on you machine. Mine is: C:\Program Files\PostgreSQL\12\bin since I am using the version 12.

The following image might be helpful:

You may have to navigate to the folder where PostgreSQL is installed on your machine to find out your own path.

Login into the Heroku Postgres using the Heroku CLI from our Heroku Postgres credentials. This is mine – yours will be different:

heroku pg:psql postgresql-slippery-19135 --app node-postgres-cloudinary

If you got an error, it is most likely because your path is not set properly.

How to Prepare our Database Connection to Match Heroku's

At the moment, my database looks like this:

const pg = require("pg");

const config = {
  user: "tutorial",
  database: "tutorial",
  password: "tutorial",
  port: 5432,
  max: 10, // max number of clients in the pool
  idleTimeoutMillis: 30000,
};

const pool = new pg.Pool(config);

pool.on("connect", () => {
  console.log("connected to the Database");
});

If you try connecting Heroku to this, you are going to get an error. This is because Heroku has a connection string setup already. So we have to setup our connection such that Heroku can easily connect.

I am going to refactor my database connection file (dbConnect.js) and .env file to make this happen.

• dbConnect.js

const pg = require('pg');
require('dotenv').config();

// set production variable. This will be called when deployed to a live host
const isProduction = process.env.NODE_ENV === 'production';

// configuration details
const connectionString = `postgresql://${process.env.DB_USER}:${process.env.DB_PASSWORD}@${process.env.DB_HOST}:${process.env.DB_PORT}/${process.env.DB_DATABASE}`;

// if project has been deployed, connect with the host's DATABASE_URL
// else connect with the local DATABASE_URL
const pool = new pg.Pool({
  connectionString: isProduction ? process.env.DATABASE_URL : connectionString,
  ssl: isProduction,
});

// display message on success if successful
pool.on('connect', () => {
  console.log('Teamwork Database connected successfully!');
});

• .env file

DB_USER="tutorial"
DB_PASSWORD="tutorial"
DB_HOST="localhost"
DB_PORT="5432"
DB_DATABASE="tutorial"

With the setup of the dbconnect and .env file, we are ready to export our database and tables from our local machine to heroku postgres.

How to Export Database and Tables

Go to your pgAdmin and locate the database for this tutorial. Mine is tutorial.

Right-Click on it and select Backup. This will bring up a new window.

Enter a name for the SQL file like I did. Select the plain format. Then click Backup. This will save the file to your documents folder.

Locate the file and move it into the project directory. It can be anywhere in the directory but I choose to move mine into the services directory because it holds the database related files.

Back in the terminal, navigate to the folder containing the SQL file and run the following code to add the tables we just exported to the heroku postgres database:

cat <your-SQL-file> | <heroku-CLI-from-the-heroku-posgres-credentials>

This is what mine looks like:

cat tutorial.sql | heroku pg:psql postgresql-slippery-19135 --app node-postgres-cloudinary

Did you notice that I changed directory to services (cd services)? That is where my sql file is located.

Wow! We have just successfully exported our database and tables to Heroku.

It is almost over...

How to Tell GitHub that We Made Changes

Add the files we have made changes to:

$ git add .

The period (.) adds all files.

Commit your latest changes:

$ git commit -m "refactored the dbConnect and .env file to fit in to heroku; Added the database SQL file"

Push the committed files:

$ git push origin -u master

Finally deploying our App

Go to you app's dashboard:

Select the GitHub Deployment method:

Search and select a repo, and click on connect:

Select the branch you want to deploy (in my own case, it is the master branch):

Enable automatic deployment by clicking the Enable automatic deployment button as in the image above.

Click on the Deploy button in the manual deploy

We will not have to do all this for subsequent deployments.

Now you have a button telling you to "view site" after build is completed. Click it. This will open your app in a new tab.

Oh no! A bug? Application error??

Don't worry, it just a small issue. Something you should never forget to do while making deployments. Most hosting service will require it.

How to Fix the Heroku Application Error

Get back to the root directory of your project.

Create a file and name it Procfile (it has no extension).

In the file, enter the following code:

web: node index.js

This directs Heroku to the server file (index.js) which is the entry point of the application. If your server is in a different file, please modify as required.

Save the file and push the new changes to GitHub.

Wait 2 to 5 minutes for Heroku to automatically detect changes in your GitHub repo and render the changes on the app.

You can now refresh that error page and see your hard work paying off:

You can also test the retrieve image route and see it working.

Congratulations! What a feat you have attained.

Other routes (persist-image, update-image, and delete-image) will not be working because we have not provisioned or added cloudinary add-on. It is as simple as the PostgreSQL one we just did. So you can give it a shot.

Conclusion

We started this tutorial with the aim of learning how to build a backend application using Express, Postgres, Cloudinary, Github and Heroku.

We learned how to store, retrieve, delete, and update an image record. We then organised our code with Express Routing, pushed it to GitHub, and deployed it on Heroku. That was a lot.

I hope you will agree that it was worth it because we learnt a lot. You should try adding the Cloudinary add-on yourself to sharpen your knowledge even more.

Thanks for reading!

There are so many different options to pick from, how do you know whether the backend you choose is the right one?

In this guide, you'll learn how to pick the appropriate backend for the type of React application you're making in the simplest and least expensive way possible.

Let's dive in!

Does My App Need a Backend?

As React developers, building our project largely focuses on what the user sees, which is known as the frontend.

In every React project, we manage data on the client through state and user interactions. However, many apps are not possible without data that comes from the backend.

The backend takes care of getting or updating data in our application and it is hidden away from the user.

Most backends consists of two parts:

1. A place to store our data (often a database)
2. A method for retrieving the data (often an API)

But here's the good news: based off of the type of application you're making, you may not need either.

Stage 1: No Backend

If you are building an app where your data changes very infrequently, you probably don't need a database or an API.

For example, if you are building a personal blog that you update a few times a week at best and that is built as a static site using Next.js or Gatsby, you don't need a backend.

Static sites built with Gatsby or Next.js may not need a backend

Instead, you could simply write your blog posts as markdown files, which are stored and tracked (using Git) within a project folder.

If you have an e-commerce application where the product data rarely changes, you could store all of app data in JSON files that you simply import and use within your React application.

If you are fine updating files manually and re-deploying your project, that may be all that you need.

What type of backend you choose depends upon some key features of your data you need to ask yourself:

• Does my data change often?
• Am I fine managing my data as local files and folders?
• Can my app data or files be tracked in version control (Git)?
• Are other people going to be updating the data?
• Will my application need authentication?

Depending on your answers to these questions, you might be able to get away with using static files as your data source.

Going this route will ultimately save a lot of money on database and hosting costs, since static sites can be hosted on a free tier of many hosting providers.

Stage 2: Content Management Systems

If you're need more features than static files alone can provide, content management systems are the next step

Content management systems or (CMS) give us tools to more easily manage our content. They often give us dedicated applications with built-in editors to more easily view, update, and structure our data.

What we specifically need for our React application is a headless CMS.

A headless CMS does not have a visible interface, since React will be serve as the user interface for our app.

A CMS is ideal for your application if you simply have too much data to manage as separate files or want other potentially non-technical users to edit or add content to your app.

Some of the simplest CMSs range from Excel-like sheets like Google Sheets and Airtable, to note-taking apps like Notion.

You can use tools like Google Sheet, Airtable and Notion to function as your app's CMS

The benefit of these solutions is that they are easy to get started with, have a generous free tier, and have their own built-in API to fetch data.

There are other CMSs that offer developer-friendly features such as image and media asset management as well as more expansive API features.

Some of the more developer-friendly CMSs include:

• Sanity
• GraphCMS
• Contentful

Sanity, GraphCMS, and Contentful are more powerful, dev-friendly CMSs

And if you are looking for content management systems that are the most powerful, with features like built-in authentication and updating data from your React client, check out:

• Strapi
• KeystoneJS

Want a CMS that is basically a complete backend? Check out Strapi or KeystoneJS

Stage 3: Backend as a Service

The limitation with content management systems is that they are great for managing and accessing data.

However, when you need to add more complex, custom features such as updating data from your React client, authenticating users, protecting content, and real-time data, a standard CMS falls short.

Managing a database and building a complete API to interact with that database is a daunting challenge, especially if you've only worked on the frontend.

If this is the case for you, it may be worth looking into a backend as a service (BaaS). It will give you much of the power of a custom-built backend without the domain-specific knowledge.

The most popular BaaS is Firebase, and for good reason. It gives you a ton of features that you simply could not build on your own, including dozens of authentication strategies, real-time NoSQL databases, cloud storage, machine learning tools, and much more.

There are many other BaaSs that give you the productivity of Firebase, with little to no code required:

• Supabase
• Hasura
• Appwrite

Firebase, Supabase, and Hasura are all great backends to use if you aren't comfortable building your own

Caveat: The speed of development for all of the services can help you build and ship your applications faster. But be aware that all of them have their own associated costs, such as the cloud storage you use and number of database operations you perform (reads/writes).

Stage 4: Build Your Own Backend

Before considering this stage, you should look intently at whether you could potentially use options 1 through 3.

This is the most advanced option to choose as a React developer because it requires the most knowledge, time, and coding skills.

With that being said, it is also the most customizable, considering that you can build exactly what you need to power your app.

Entire books have been written on just parts of building your own backend, but here is what I would recommend as someone who has built many production applications using a custom backend.

I recommend using a SQL database such as Postgres or MySQL. Services such as Heroku, Render.com, and PlanetScale offer hosted databases (often with free daily backups) at great prices.

Heroku, Render.com and PlanetScale are some of the best places for your database to live

Unless you are very comfortable and confident writing raw SQL statements and know all of the security precautions to take to avoid nasty things like SQL injection, use an object relational mapper (an ORM) to create a database schema and interact with it.

I highly recommend using Prisma as your ORM. It generates all the code required to perform every kind of operation against your database as well as types for each.

Use Prisma as your ORM

While you can certainly build a custom Node backend using your favorite Node library or framework (Express, Fastify, Nest.js), I would advise you to start small and use a feature like Next.js' API routes.

Tools like Next API routes allow you to build your API fast without the need to run and manage your server code in a separate repository.

Become a Professional React Developer

React is hard. You shouldn't have to figure it out yourself.

I've put everything I know about React into a single course, to help you reach your goals in record time:

Introducing: The React Bootcamp

It’s the one course I wish I had when I started learning React.

Click below to try the React Bootcamp for yourself:

Click to get started

In this article, I will walk you through step by step how to setup the backend challenges on your local computer and submit the localhost link.

How to clone the freeCodeCamp GitHub repo

For the Managing Packages with NPM section, you will need to use this GitHub repository.

On the GitHub repo, click on the green Code button located on the right hand side of the page.

Copy the URL the address here:

If you are using a Mac, open up your Terminal app. If you are using Windows, open up your Command Prompt.

In the command line, run cd Desktop and hit enter which will change directories to your Desktop.

jessicawilkins@Dedrias-MacBook-Pro-2 ~ % cd Desktop

You should now be in the Desktop directory and see this result in the command line.

jessicawilkins@Dedrias-MacBook-Pro-2 Desktop %

Then run git clone [https://github.com/freeCodeCamp/boilerplate-npm.git](https://github.com/freeCodeCamp/boilerplate-npm.git). That was the URL you copied earlier from GitHub.

jessicawilkins@Dedrias-MacBook-Pro-2 Desktop % git clone https://github.com/freeCodeCamp/boilerplate-npm.git

You should see this result in the command line for successfully cloning the folder.

Cloning into 'boilerplate-npm'...
remote: Enumerating objects: 46, done.
remote: Total 46 (delta 0), reused 0 (delta 0), pack-reused 46
Unpacking objects: 100% (46/46), done.
jessicawilkins@Dedrias-MacBook-Pro-2 Desktop %

You should be able to see the new folder on your Desktop.

How to install node_modules

In the command line, run cd boilerplate-npm to change directories to the folder we just cloned from GitHub.

jessicawilkins@Dedrias-MacBook-Pro-2 Desktop % cd boilerplate-npm

Then run npm install to install the node_modules folder. The node_modules folder contains all of the dependencies needed to run your project.

Without this folder, you will not be able to run any of the challenges. That is why we need to install it inside the project folder.

jessicawilkins@Dedrias-MacBook-Pro-2 boilerplate-npm % npm install

Once installed, you should see this result in the command line:

added 50 packages, and audited 51 packages in 2s

found 0 vulnerabilities
jessicawilkins@Dedrias-MacBook-Pro-2 boilerplate-npm %

How to submit the localhost link

Go to your code editor of choice and open up the project folder.

Then complete the first challenge which is to add an author to your package.json file.

Go back to the command line, and run npm start which will start the local server.

jessicawilkins@Dedrias-MacBook-Pro-2 boilerplate-npm % npm start

You should see this result in the command line:

> start
> node server.js

Node.js listening on port 3000

Go to your browser and open up a new tab. Type in http://localhost:3000/.

You should see this result in the browser.

This is the local server that you will use for all of the backend challenges.

While the server is still running, go to the first challenge and submit the localhost link.

Once you have completed the challenges, you can stop the server by using Ctrl+C in the command line.

Those are the steps for completing the freeCodeCamp backend challenges using the localhost.

For the other sections of the Back End Development and APIs certification, you will need to use the corresponding GitHub repository.

For the Basic and Express section, you need to clone this GitHub repo.

For the MongoDB and Mongoose section, you need to clone this GitHub repo.

REST and GraphQL are both standard ways to develop backend APIs. But over the past decade REST APIs have dominated as a choice for developing backend API's. And many companies and developers use it actively in their projects.

But REST has some limitations, and there's another alternative available – GraphQL. GraphQL is a great choice for developing APIs in large codebases.

What is GraphQL?

GraphQL was developed by Facebook in 2012 for internal usage and made public in 2015. It is a query language for APIs and a runtime for fulfilling those queries with your existing data. Many companies use it in production.

The official website introduces GraphQL like this:

GraphQL provides a complete and understandable description of the data in your API, gives clients the power to ask for exactly what they need and nothing more, makes it easier to evolve APIs over time, and enables powerful developer tools.

And we'll see how all this works in this blog.

Issues With REST APIs

• Querying Multiple Endpoints
• OverFetching
• UnderFetching and n+1 Request Problem
• Not super fast to cope up with changing client end requirements
• High Coupling between Backend Controllers and Frontend Views

So, what are these issues and how does GraphQL solve them? Well, we'll learn more going forward. But first we'll need to make sure you're comfortable with the basic concepts of GraphQL like the type system, schema, queries, mutations, and so on.

Now we'll look at some examples to better understand the disadvantages of using REST APIs.

Overfetching

Let's assume that we need to show this User Card in the UI.

With REST, the request is going to be a GET at /users/1.

The problem here is that the server returns a fixed data-structure, something like this:

{
    "_id": "1",
    "name": "Aagam Vadecha",
    "username": "aagam",
    "email": "testemail@gmail.com",
    "currentJobTitle": "Software Engineer",
    "phone": "9876543210",
    "intro": "As a software engineer my daily routine revolves around writing cleancode, maintaining infrastructure, and building scalable softwaresystems. In my free time I love to write tech blogs, freelance, listenmusic, and watch thrillers.",
    "website": "https://www.aagam.tech",
    "gender": "MALE",
    "city": "Surat",
    "state": "Gujarat",
    "country": "India",
    "display_picture": "8ba58af0-1212-4938-8b4a-t3m9c4371952",
    "phone_verified": true,
    "email_verified": true,
    "_created_at": "2021-03-08T14:13:41Z",
    "_updated_at": "2021-03-08T14:13:41Z",
    "_deleted": false
}

The server returns extra data (aside from the Name, Intro, and Job Designation) which is not required at the client-end to build the card at this point – but the response still has it. This is called overfetching.

Overfetching brings extra data in each request which is not required by the client. And this increases the payload size and eventually that has an effect on the overall response time of the query.

What's worse, the situation escalates when a query is bringing data from multiple tables even though the client doesn't require it right then. So if we can avoid it, we definitely should

With GraphQL, the query inside the request body would look something like this:

It will only return the name, intro, and currentJobTitle as required by the client, so the overfetching problem is solved.

Underfetching and the n+1 request problem

Now let's assume this UserList needs to be shown in the UI.

With REST, considering "experience" is a table which has a foreign key of user_id, there are three possible options

1. One is to send some exact fixed data-structure from all foreign-key linked tables to the users table in the GET /users request, and many frameworks provide this option.

{
    "_id": "1",
    "name": "Aagam Vadecha",
    "username": "aagam",
    "email": "testemail@gmail.com",
    "currentJobTitle": "Software Engineer",
    "phone": "9876543210",
    "intro": "As a software engineer my daily routine revolves around writing cleancode, maintaining infrastructure, and building scalable softwaresystems. In my free time I love to write tech blogs, freelance, listenmusic, and watch thrillers.",
    "website": "https://www.aagam.tech",
    "gender": "MALE",
    "city": "Surat",
    "state": "Gujarat",
    "country": "India",
    "display_picture": "8ba58af0-1212-4938-8b4a-t3m9c4371952",
    "phone_verified": true,
    "email_verified": true,
    "_created_at": "2021-03-08T14:13:41Z",
    "_updated_at": "2021-03-08T14:13:41Z",
    "_deleted": false,
    "experience": [
        {
            "organizationName": "Bharat Tech Labs",
            "jobTitle": "Software Engineer",
            "totalDuration": "1 Year"
        }
    ],
    "address": [
        {
            "street": "Kulas Light",
            "suite": "Apt. 556",
            "city": "Gwenborough",
            "zipcode": "929983874",
            "geo": {
                "lat": "-37,3159",
                "lng": "81.1496"
            }
        }
    ]
}

But this method makes expensive queries, overfetches all other /users requests as well, and ends up bringing back a lot of data from all foreign tables (address, experience) which is not required in most cases.

For example, you want user data somewhere else in the frontend where you just need to show user's website so you make a GET user/1 request. But it overfetches data from the experience table as well as the address table, which you don't need at all.

2. The second option is that the client can make multiple trips to the server like this:

GET /users 
GET users/1/experience

This is an example of underfetching, as one endpoint doesn't have enough data. But multiple network calls slows down the process and affects the user experience. :(

Also, in this specific case of a List, underfetching escalates and we run into the n+1 request problem.

You need to make an API call to get all users and then individual API calls for each user to get their experience, something like this:

 GET /users 
 GET /users/1/experience 
 GET /users/2/experience 
 ...
 GET /users/n/experience.

This is known as the n+1 request problem. To solve this issue, what we generally do is the third option, which we'll discuss now.

3. Another option is to make a custom controller on the server which returns the data-structure that meets the client requirements at that point in time.

   GET /user-experience
   

   This is what is done in major real world REST API cases.

On the other hand, a simple GraphQL request which would seamlessly work without any development required on the server end, would look something like this:

No overfetching, no underfetching, and no development on the server. Simply fantastic!

Tight coupling between front end views and back end APIs

Okay, so you might argue that you can just use REST, make the server controller with a one time initial development effort, and be happy – right?

But there's a major drawback which comes along with using custom controllers.

We have formed a tight coupling with the front-end view and the back-end controller, so overall it needs more effort to cope with changes on the client end. It also gives us less flexibility.

As Software Engineers, we know how often requirements change. The custom controller at this point GET /user-experience is returning data depending on what the front-end view wants to display (user-name and current-role). So when a requirement changes, we need to refactor both the client and server.

For example, after a considerable amount of time, requirements change and instead of the experience data the UI needs to show a user's last transaction info.

With Rest, we would need to make the relevant changes in the front end layer. Additionally, in order to return transaction data instead of experience data from the back end, the custom controller will either need to be refactored (data sent, route, etc) or we'll need to make a new controller in case we want to preserve the old one.

So basically, a change in client requirements greatly influences what is to be returned by the server – which means we have tight coupling between the front end and the back end!

It would be better not to have to make any changes on the server but only the front end.

With GraphQL we won't need to make any changes in the server side. The change in the front end query would be as minimal as this:

No server API refactoring, deployment, testing – this means time and effort saved!

Conclusion

As I hope you can see from this article, GraphQL has a number of advantages over REST in many areas.

It might take more time to set up GraphQL initially, but there are many scaffolders which make that job easier. And even if it takes more time in the beginning, it gives you long term advantages and is totally worth it.

Thanks for reading!
I hope you liked this article, and that it gave you more insights to help you make your next choice. If you need some help, feel free to reach out to me on LinkedIn.

If you're a beginner in tech and you don't know much about the field, this might be a confusing term that brings up lots of questions like: What is a full-stack developer? What exactly do they do? What skills are needed to become one? And so on.

Now, to help you answer this burning question, in this article we will cover all this and more including resources and tips to help you become a Full-Stack dev.

Let's get started.

What Does "Full Stack" Mean?

Just like any other beginner starting out, when I began to learn about coding, I knew that you were either a web developer or a mobile developer – at least, that's what I thought back then.

So with this in mind it was easier for me to choose a track. I decided to first learn Android development then later transitioned to web development. I knew that with these skills, I was set and ready to apply for jobs.

After a year of dedicated learning, there I was already applying for jobs. During this time of trying to find a role that aligned with my skills, something stood out in all the descriptions: they either wanted a front-end developer, back-end developer, or a full-stack developer.

For a moment I was confused – all I ever wanted was to be a web developer, or at least I thought that was where everybody began.

Seeing those new terms made me feel confused and for a moment I thought I had chosen the wrong path. But no, all this was because I wasn't well informed when I was starting out. I just needed to learn more about different terms/roles and I was good to go.

It's also worth pointing out that, over the past few years, the web has grown and opened many opportunities for developers with different skills. So it does not matter what skills you have, the web is diverse it can always accommodate all of us 😊.

So first, let's highlight the most common terms you are likely to encounter, and depending on your skills you will always classify yourself between these roles. They include:

• Front-end Developer - They deal with what we can see and interact with in a website.
• Back-end Developer - They deal with what the user can not see, they create pathways to deliver information to users.
• Full-Stack Developer - They are able to do both front and back end development (which applies for both mobile and web).

Let's step it up a notch and understand these roles. We'll also see what skills and knowledge you need to become a front end, back end, or full stack developer.

What is a Front End Developer?

The main role of front-end developers is to develop the visible parts of a website, mainly what the users can see and interact with. It is also known as client-side or customer facing side.

In a real world example, let's take an example of a widely known meme: you can think of front-end as the complete setup in a wedding reception or party:

In order to become a front-end developer you need to know the basics of web development. In this case which are HTMl, CSS, and JavaScript. Let's break these skills down a little bit. To get a better understanding, we are going to use our body as an example – cool, right?

Skills Needed for Front-End Development

HTML (HyperText Markup Language) – This is like the skeleton of the body, because it gives it a website structure. Every site you visit is built using HTML, which handles the structure and the content of the site.

CSS (Cascading Style Sheets) – Think of CSS as the aesthetics and muscles of the body! CSS makes a site beautiful and interesting to look at. Just like in your body where muscles and skin make you beautiful, CSS does the same for a website. It sets the fonts and colors, adds images, and even helps make the site responsive.

JavaScript – like the brain of the operation that tells everything how to behave. JS is a gamechanger in web development as it gives a site all its functionality.

By having this skills alone you are able to develop static websites as part of your project and use them in your resume. To lead by example take a look at my portfolio that I made using this skills alone. MY PORTFOLIO.

As I stated earlier, changes continue to happen daily and we have to embrace them. Speaking of changes JavaScript has evolved and changed for the better with addition of many great features which have made it easier for developers to use the language.

Some of the important improvements include introduction of frameworks like React, Angular, Node, Vue etc... that has made it easier for front-end developers to build projects.

Just like any other role out there, front end devs also has some duties that are mandated to them. Let's look at the responsibilities for a typical front-end dev in a day to day activity:

Responsibilities of a Front-End Developer

They are expected to:

1. Ensure that web designs are responsive and work well on all devices.
2. Collaborate with designers and application developers to develop customized websites.
3. work with creative teams to design new features, and maintain old features.
4. Collaborate with back-end developers and improve code usability.
5. Write a detailed functional requirement documentation and guides, To help you write a good documentation as a front-end developer, this guide on The Do's and Don'ts of front-end documentation will come in handy.
6. Stay up-to-date with the latest trends in their field of work.

How to Get Started with Front End Development

By this point you have a rough idea of what is associated with the front-end developer role, Perhaps you are wondering how do I get the necessary resources to start learning? Well, worry not because I've created this list of excellent resources.

Out there we have a bunch of platform that might be a little confusing for beginners, We have both paid and free resources but this is not the issue the main point is for you to get what you learn and use it for your benefit.

My top recommendations include:
For learning

• FreeCodeCamp 🆓
• Udacity 🆓 (free & paid)
• Codecademy (paid)
• Skillshare (paid)
• Frontend Masters 🆓

For Challenges

• Frontend Mentor
• CSSBattle
• Coderbyte
• Javascript30

GitHub Repos with learning materials

• I have compiled a repo to help beginners get started with front-end development – you can check it out here: HTML,CSS & JS
• Front-End-Checklist
• Frontend Roadmap
• Frontend Developer Roadmap

What next 🤷‍ after studying all this much, Well if you are able to get through the above and learn it, you are qualified to own the title Front-End Developer, meaning you are now ready to apply for jobs associated with this role.

To give you motivation to apply for jobs, according to Glassdoor, the average salary associated with role is $86,148 per year, good money there.

During the process of learning, though, you will need to practice to perfect your new skills.

And for me, practicing is all about building projects and hosting them somewhere so that you can showcase your work to potential employers.

Below is a list of free hosting services where you can host your front end projects:

• Vercel
• Netify
• GitHub Pages
• Heroku
• Firebase

What is a Back End Developer?

Back end developers deal with the parts of a website that users don't interact with directly or see. Their work is much more behind the scenes. They mainly create pathways to deliver information to users. It's mainly about servers and database.

Referencing to our earlier example, now we will be talking about the minds behind all the good staff guests enjoy, the people who prepared the delicacies.

Some of the most commonly used languages for back-end work are PHP, Ruby, and Python. If you know these languages you can create algorithms to manipulate data delivered from the front-end side of the site.

Back-end developers work with front-end developers to make their projects work. One thing to note in job listings is that they will often not directly mention back end development as a required skill – instead the company might be looking for a Python developer, a PHP developer, and so on.

Skills Needed for Back-End Development

By choosing this path, it assures you of familiarizing yourself with several if not one programming language which will help you along you career path.

Some of the thing to keep in mind if you choose this path include:

1. Know and understand the basics of data structure and algorithms on the language you choose to learn.
2. Learn a programming language that you are comfortable with, some of the most common include Python, Java, PHP, Go, Ruby & Rails, etc...
3. You need to have the knowledge of databases, some of the most popular include MYSQL, MONGODB, POSTGRESQL, REDIS, OCACLES, etc...
4. Find a framework associated withe the language you are learning and learn it, an example: If you are learning Python then choose between Flask or Django.
5. Practice and start creating projects. Then you can show off those projects to your future employers.

Responsibilities of a Back-End Developer

1. Database creation and management.
2. Understand the goals of a website and create effective solutions.
3. Research and introduce new and relevant technologies.
4. Back-up and restore the website's files and data.
5. Collaborate with teams of designers, front end developers, and system admins.
6. Troubleshoot and debug applications.

How to Get Started With Back-End Development

It will be fair to note that, the resources under this section are most likely to be resources to studying a particular language or it's associated framework.

My recommendation include:

• Beginning PHP
• Mastering Backend
• The FreeCodeCamp YouTube channel has tons of resources on back-end development – just visit the channel and search what you want to learn.

GitHub Repos

• Backend-best-practises
• Backend-dev-textbook
• Mentorship-backend

Just like we host projects in front-end development the same applies to back-end. Some of the platforms that you can use to host your projects include:

• AWS
• Azure
• Digital Ocean
• Linode
• Google Cloud

Now, that we have covered both the front-end and back-end paths that are essential for one to become a full-stack developer, we can finally discuss our main topic – What is a full-stack developer?

What is a Full Stack Developer?

In simple words, we can say that a full-stack developer is able to address both sides of the equation, meaning they can work with both the front and back ends of a website or an application. They are able to handle projects that involve building user interfaces and databases.

In order to become a full-stack developer you will need to put in some years of hard-work and get some experience in the industry. You'll need to learn both the front-end and back-end technologies.

Skills Needed to Be a Full-Stack Developer

It is obvious that in order to get the title of a full-stack developer you will need a wide variety of skills. Just a recap of all of it:

Basic UI/UX design skills
This skills will help you be able to develop a visual prototype that will enable you get to know and feel how users will interact with you end product.

We have many resoursec and softwares that can help with the mockup design, the common ones include: AdobeXD, Invision and Figma.

Having the softwares without knowing how to use them won't be of any help, right? To help learn how to go about these tools we have Youtube videos, online courses, and UX bootcamps just to mention a few:

Basics of front-end development – HTML, CSS, and JavaScript.
These are the fundamental skills you need for web development, They will help you create beautiful interfaces for your sites. If you want to become a full-stack developer be sure that you clearly understand the basics before moving to the next step. Also find a front-end framework of choice and study it.

Programming languages like Java, Python, Ruby, or PHP.
This step might get confusing or make you feel overwhelmed when it comes to choosing a language to learn. You can always pick one and learn it well, then move into another once you've gotten the basics down. After picking a language find a corresponding framework and master it too:
Most common: Django/Flask for Python, Laravel for PHP, Spring for Java etc...

At this point it will also be important to note that:

Before you start looking for other resources to learn from, check out the main documentations of the language you're trying to learn.

Learn about development frameworks.
Learning a framework helps make your development work easier. It's a good idea to learn at least one framework for both the front-end and back-end because:

• It eases debugging and maintenance of applications and sites.
• It is a good use of code re-usability.
• It helps reduce the length of your code.
• It improves proficiency and reinforces security.

Now, to help you get started with studying frameworks the right way, here are quick links to official documentation of the most common ones:
Front-End

• React Docs
• VueJS Docs
• AngularJS docs
• NodeJS

Back-End

• Spring Docs
• Django Docs
• Laravel

Learn about Databases
This are the brains that make websites/applications dynamic. You'll need to understand how the client side and server side communicate. Databases provide an interface to let a user save data in a persistent way to memory. This lets us retrieve data whenever we need it.

Depending on the type of project you are working on, your database needs might vary.

Learn Version control**
It is considered to be a very important tool by most developers, this is because of the advantageous role it plays in the development field. Some of it's importance include:

• It helps keep track of project progress
• It makes working with team and collaboration easier.
• It helps recover lost project files incase of a deletion.

The most common and widely used version control is Git. To help you get started to learn it, check out the official Git Documentation.

Other Important Skills:

• Time management
• Creativity
• Good communication
• Problem solving

There you have it all. By this point you have the concept of what a full-stack developer is all about and the skills needed. What are the responsibilities that come with the role?

Responsibilities of a Full-stack Developer

1. Have the front-end knowledge and any related frameworks.
2. Have the back-end design and development knowledge, including at least one framework associated with language of choice.
3. Have the database and server management knowledge for the website functionality.
4. Ensure that the projects run smoothly on all platforms like Windows, MacOS, and Linux.
5. Meet both the technical and consumer requirements.
6. Design and Develop APIs which are responsible for fetching data.

By choosing to be a full-stack developer, it means that you have a big mandate coming your way. This means we will have some pros and cons associated with the role. Some of them include:

Advantages of being a Full-Stack Developer

1. Adaptability – you are well versed in both front and back end technologies which means you can easily switch projects when required to.
2. You can carry out a wide range of tasks because of your diverse skill set which lets you work on different types of projects.
3. It reduces project communication because you can implement changes independently in most cases.

Disadvantages

1. Under-productivity and time management issues may happen when your company/team depends on only one or two full-stack developers.
2. It's difficult to keep up with trends – since you are working across different areas of tech, it's a challenge to keep up with emerging trends.
3. Issues with balancing work, as a full-stack developer it tends to take on too much time. As a result, you might forget to divide your workload into smaller tasks and get confused about your responsibilities on a certain project.

One of the main reason why people study and later work is to be able to pay bills, so you're probably thinking that all these requirements should come with a very good salary, right?

Well, that's true – according to Indeed the average salary of a full-stack developer ranges at $107,750/yr.

How to Get Started with Full Stack Development

In order to learn all that we have discussed here, you'll probably want some guidance. Fortunately, you won't have to search for the resources yourself because I did that for you. All you need is to pick one and get started learning.

Online Courses

1. Full Stack Web Development: HTML, CSS, React & Node (Udemy)
2. Full-Stack Web Development Course with React (Coursera)
3. Professional Full Stack Developer – Nanodegree Certification (Udacity).
4. Free Full-Stack Development Training (LinkedIn Learning – Lynda)

GitHub Repos

1. Full Stack Developer Course
2. Full Stack Web Developer Path

Wrap Up

A full-stack developer can do pretty much everything. If you enjoy being a creative and a problem solver, and trying to understand how different things work together, then this is a path you might want to pursue.

Although many job descriptions lists Computer Science Degree as a requirement to get hired as a full-stack developer, I tend to disagree with this. I believe that all you need are skills and projects which you can gain by building projects.

In the recent past, we have had many polls whether a CS degree is necessary to be hired as a developer but many who have gotten the job tend to disagree. The tweet below proves it all.

My final question for you, Which type of developer are you or Which type of developer are you aiming to be?

I will go first: I have always preferred to call my self a Growing Developer as there is room to learn every day. I can't really specify exactly one level, because I believe I can pretty much do anything.

If you have read this far I really appreciate it.

Connect with me on: Twitter | LinkedIn | GitHub | YouTube.

Enjoy Coding ❤.

These two languages are very popular and powerful, but they do have key differences. We will cover them in detail here.

In this article, you will learn:

• The different real-world applications of Python and JavaScript.
• The key syntactic and functional differences between Python and JavaScript.

Let's begin! ✨

🔹 Python VS JavaScript: Real-World Applications

We will start with a quick tour of their real-world applications.

Python

Python has become an essential tool in virtually every scientific application around the world because of its power and versatility. It is a general-purpose programming language that supports different programming paradigms.

It is widely used in scientific and specialized applications, including data science, artificial intelligence, machine learning, computer science education, computer vision and image processing, medicine, biology, and even astronomy.

It is also used for web development. This is where we can start to compare its applications to the applications of JavaScript. Python is used for back-end development, which is the area of web development in charge of creating the elements that users don't see, such as the server side of an application.

JavaScript

While Python can be used to develop the back-end part of a web application, JavaScript can be used to develop both the back-end and the front-end of the application.

The front-end is the part of the application that the user sees and interacts with. Whenever you see or interact with a website or web application, you are using JavaScript "behind the scenes".

Similarly, when you interact with a mobile app, you might be using JavaScript because frameworks like React Native let us write applications that adapt to different platforms.

JavaScript is so widely used in web development because it is a versatile language that gives us the tools we need to develop the components of a web application.

Differences between the applications of Python and JavaScript

In short, developers use Python for a range of scientific applications. They use JavaScript for web development, user-facing functionality, and servers

🔸 Python VS JavaScript: Syntax

Now that you know what they are used for, let's see how they are written and the differences in their syntax.

We will cover the differences in their main elements:

• Code Blocks
• Variable Definitions
• Variable Naming Conventions
• Constants
• Data Types and Values
• Comments
• Built-in Data Structures
• Operators
• Input/Output
• Conditional Statements
• For Loops and While Loops
• Functions
• Object-Oriented Programming

Code Blocks in Python and JavaScript

Each programming language has its own style to define code blocks. Let's see their differences in Python and JavaScript:

How Python defines code blocks

Python relies on indentation to define code blocks. When a series of continuous lines of code are indented at the same level, they are considered part of the same code block.

We use this to define conditionals, functions, loops, and basically every compound statement in Python.

These are some examples:

Use of indentation to define code blocks in Python

💡 Tip: We will see the specific differences between these elements in Python and JavaScript in just a moment. At this moment, please focus on the indentation.

How JavaScript defines code blocks

In contrast, in JavaScript we use curly braces (**{}**) to group statements that belong to the same code block.

These are some examples:

Use of curly braces to define code blocks in JavaScript

Variable Definitions in Python and JavaScript

The assignment statement is one of the most fundamental statements in any programming language. Let's see how we can define a variable in Python and JavaScript.

How to define a variable in Python

To define a variable in Python, we write the name of the variable followed by an equal sign (**=**) and the value that will be assigned to the variable.

Like this:

<variable_name> = <value>

For example:

x = 5

How to define a variable in JavaScript

The syntax is very similar in JavaScript, but we just need to add the keyword **var** or **let** before the name of the variable and end the line with a semicolon (**;**).

Like this:

var <variable_name> = <value>;

💡 Tip: When you define a variable using **var**, the variable has function scope.

For example:

var x = 5;

We can also use the keyword **let**:

let <variable_name> = <value>;

For example:

let x = 5;

💡 Tip: In this case, when we use **let**, the variable will have block scope. It will only be recognized in the code block where it was defined.

Variable definitions in Python and JavaScript

💡 Tip: In JavaScript, the end of a statement is marked with a semicolon (;) but in Python, we just start a new line to mark the end of a statement.

Variable Naming Conventions in Python and JavaScript

Python and JavaScript follow two different variable naming conventions.

How to name variables in Python

In Python, we should use the **snake_case** naming style.

According to the Python Style Guide:

Variable names follow the same convention as function names.

Function names should be lowercase, with words separated by underscores as necessary to improve readability.

Therefore, a typical variable name in Python would look like this:

first_name

💡 Tip: The style guide also mentions that "**mixedCase** is allowed only in contexts where that's already the prevailing style, to retain backwards compatibility."

How to name variables in JavaScript

In contrast, we should use the **lowerCamelCase** naming style in JavaScript. The name starts with a lowercase letter and then every new word starts with an uppercase letter.

According to the JavaScript guidelines article by the MDN Web Docs:

For variable names use lowerCamelCasing, and use concise, human-readable, semantic names where appropriate.

Therefore, a typical variable name in JavaScript should look like this:

firstName

Constants in Python and JavaScript

Great. Now you know more about variables, so let's talk a little bit about constants. Constants are values that cannot be changed during the execution of the program.

How to define constants in Python

In Python, we rely on naming conventions to define constants because there are no strict rules in the language to prevent changes to their values.

According to the Python Style Guide:

Constants are usually defined on a module level and written in all capital letters with underscores separating words.

This is the naming style that we should use to define a constant in Python:

CONSTANT_NAME

For example:

TAX_RATE_PERCENTAGE = 32

💡 Tip: This serves as a red warning for ourselves and for other developers indicating that this value should not be modified in the program. But technically, the value can still be modified.

How to define constants in JavaScript

In contrast, in JavaScript we can define constants that cannot be changed in the program, and the variable identifier cannot be reassigned.

But this does not mean that the value itself cannot be changed.

According to the article **const** in MDN Web Docs:

The const declaration creates a read-only reference to a value. It does not mean the value it holds is immutable—just that the variable identifier cannot be reassigned. For instance, in the case where the content is an object, this means the object's contents (e.g., its properties) can be altered.

To define a constant in JavaScript, we add the keyword **const** before the name of the variable:

const TAX_RATE_PERCENTAGE = 32;

If we try to change the value of the constant, we will see this error:

Therefore, the value cannot be changed.

💡 Tip: To run and test small code snippets of JavaScript code, you can use the console in Chrome Developer Tools.

Data Types and Values in Python and JavaScript

Let's see the main differences between Python and JavaScript data types.

Numeric Data Types

Python has three numeric types to help us perform precise calculations for scientific purposes. These numeric types include: int (integers), float (floating-point numbers), and complex. Each one of them has its own properties, characteristics, and applications.

In contrast, JavaScript has only two numeric types: **Number** and **BigInt**. Integers and floating-point numbers are both considered to be of type **Number**.

According to the article Number in MDN Web Docs:

A number literal like 37 in JavaScript code is a floating-point value, not an integer. There is no separate integer type in common everyday use. (JavaScript now has a BigInt type, but it was not designed to replace Number for everyday uses. 37 is still a Number, not a BigInt.)

None vs. null

In Python, there is a special value called **None** that we typically use to indicate that a variable doesn't have a value at a particular point in the program.

The equivalent value in JavaScript is **null**, which "represents the intentional absence of any object value" (source).

The undefined Value

In JavaScript, we have a special value that is assigned automatically when we declare a variable without assigning an initial value.

This is an example:

As you can see, the value of the variable **x** is **undefined**.

In Python, you have to assign an initial value to the variable. We can't declare it without an initial value.

💡 Tip: You can assign **None** as the initial value of a variable in Python to represent the absence of a value.

Primitive Data Types in Python and JavaScript

Primitive data types represent the most fundamental values that we can work with in a programming language. Let's compare the primitive data types of these two languages:

• Python has four primitive data types: Integers (int), Floats (float), Booleans (bool), and strings (str).
• JavaScript has seven primitive data types: undefined, Boolean, String, Number, BigInt, Symbol, and null (source).

How to Write Comments in Python and JavaScript

Comments are very important to write clean and readable code. Let's see how you can use them in Python and JavaScript:

Single-Line Comments

• In Python, we use a hashtag (**#**) to write a comment. All the characters on the same line after this symbol are considered part of the comment.
• In JavaScript, we write two slashes (**//**) to start a single-line comment.

This is a graphical example:

In Python:

# Comment

In JavaScript:

// Comment

Multi-Line Comments

• In Python, to write a multi-line comment we start each line with a hashtag.
• In JavaScript, multi-line comments start with a **/*** and end with a ***/**. All the characters between these symbols are considered part of the comment.

In Python:

# Multi-line comment 
# in Python to explain
# the code in detail.

In JavaScript:

/* 
Multi-line comment 
in JavaScript to explain 
the code in detail.
*/

Built-in Data Structures in Python and JavaScript

The built-in data structures in Python and JavaScript also have key differences.

Tuples

• In Python, we have a built-in data structure called tuple that is very similar to a list but immutable. Therefore, it cannot be changed during the execution of the program, so it is used to store data that should not be modified.
• In JavaScript, there isn't a built-in data structure with these characteristics. Although you can implement a similar data structure with certain features of the language.

Lists vs. Arrays

• In Python, lists are used to store a sequence of values in the same data structure. They can be modified, indexed, sliced, and used in the program.
• In JavaScript, an equivalent version of this data structure is called array.

This is an example:

Hash Tables

• In Python, there is a built-in data structure called dictionary that helps us map certain values to other values and create key-value pairs. This works as a hash table.
• JavaScript doesn't have this type of built-in data structure, but there are certain ways to reproduce its functionality with certain elements of the language.

Operators in Python and JavaScript

Operators are essential to write expressions in any programming language. Let's see their key differences in Python and JavaScript.

Floor Division

While most of the arithmetic operators work exactly the same in Python and JavaScript, the floor division operator is a little bit different.

• In Python, the floor division operation (also called "integer division") is represented with a double slash (**//**).
• In JavaScript, we don't have a particular floor division operator. Instead, we call the **Math.floor()** method to round down the result to the nearest integer.

Comparing Values and Types

In Python, we use the **==** operator to compare if two values and their data types are equal.

For example:

# Comparing Two Integers
>>> 0 == 0     
True
# Comparing Integer to String
>>> 0 == "0"
False

In JavaScript, we also have this operator but it works a little bit differently because it converts the two objects to the same type before actually performing the comparison.

If we check the result of the "integer vs. string" comparison from the previous example using JavaScript (0 == "0"), the result is **True** instead of **False** because the values are converted to the same data type before being compared:

In JavaScript, to check if the value and the data type are both equal, we need to use this operator **===** (a triple equal sign).

Now we get the result that we were expecting:

Awesome, right?

💡 Tip: The **==** operator in Python works like the **===** operator in JavaScript.

Logical Operators

• In Python, the three logical operators are: **and**, **or**, and **not**.
• In JavaScript, these operators are: **&&**, **||**, and **!** (respectively).

Type Operators

• In Python, to check the type of an object we use the **type()** function.
• In JavaScript, we use the **typeof** operator.

This is a graphical description of their syntax:

Input and Output in Python and JavaScript

Asking for user input and displaying values to the user are very common operations. Let's see how you can do this in Python and JavaScript:

Input

• In Python, we use the **input()** function to ask for user input. We write the message within parentheses.
• In JavaScript, one alternative (if you are running the code on a browser) is to display a small prompt with **window.prompt(message)** and assign the result to a variable.

The main difference between these two approaches is that in Python, the user will be prompted to enter a value in the console while in JavaScript, a small prompt will be displayed on the browser and it will ask the user to enter a value.

💡 Tip: You will see this in the Python console to enter a value:

In JavaScript, if you open Chrome Developer tools and enter this line of code in the console:

This prompt will be displayed:

Prompt displayed when window.prompt() is called

Output

• In Python, we print a value to the console with the **print()** function, passing the value within parentheses.
• In JavaScript, we print a value to the console using **console.log()**, passing the value within parentheses as well.

💡 Tip: If you are working on a browser, you can also call **alert()** to display a small prompt with the message (or value) passed within parentheses.

Conditional Statements in Python and JavaScript

With conditionals, we can choose what happens in the program based on whether a specific condition is **True** or **False**. Let's see their differences in Python and JavaScript.

if Statement

• In Python, we rely on indentation to indicate which lines of code belong to the conditional.
• In JavaScript, we have to surround the condition with parentheses and the code with curly braces. The code should also be indented.

Conditional in Python (left) and JavaScript (right)

if/else Statement

The else clause is very similar in both languages. The only difference is that:

• In Python, we write a colon (**:**) after the **else** keyword
• In JavaScript, we surround the code that belongs to this clause with curly braces (**{}**) .

Multiple Conditions

To write multiple conditions:

• In Python, we write the keyword **elif** followed by the condition. After the condition, we write a colon (:) and the code indented on the next line.
• In JavaScript, we write the keywords **else if** followed by the condition (surrounded by parentheses). After the condition, we write curly braces and the code indented within the braces.

Conditional in Python (left) and JavaScript (right)

Switch in JavaScript

• In JavaScript, we have an additional control structure that we can use to choose what happens based on the value of an expression. This statement is called **switch**.
• Python doesn't have this type of built-in control structure.

This is the general syntax of this statement:

Switch statement in JavaScript

In JavaScript:

switch (expression) {
    case value1:
        // Code
        break;
    case value2:
        // Code
        break;
    case value3:
        // Code
        break;
    default:
        // Code
}

💡 Tip: We can add as many cases as we need and the expression can be a variable.

For Loops and While Loops in Python and JavaScript

Now let's see how we can define different types of loops in Python and JavaScript and their main differences.

For Loops

The syntax to define a for loop in Python is relatively simpler than the syntax in JavaScript.

• In Python, we write the keyword for followed by the name of the loop variable, the keyword in, and a call to the range() function specifying the necessary parameter(s). Then, we write a colon (:) followed by the body of the loop indented.
• In JavaScript, we have to specify several values explicitly. We start with the for keyword followed by parentheses. Within those parentheses, we define the loop variable with its initial value, the condition that must be False to stop the loop, and how the variable will be updated on every iteration. Then, we write curly braces to create a code block and within the braces we write the body of the loop indented.

For Loop in Python (left) and JavaScript (right)

Iterating Over Iterables

We can use a for loop in Python and JavaScript to iterate over the elements of an iterable.

• In Python, we write the keyword for followed by the loop variable, the in keyword, and the iterable. Then, we write a colon (:) and the body of the loop (indented).
• In JavaScript, we can use a **for .. of** loop. We write the for keyword followed by parentheses and within those parentheses, we write the keyword var or let followed by the loop variable, the keyword of, and the iterable. We surround the body of the loop with curly braces and then we indent it.

For Loop in Python (left) and JavaScript (right)

In JavaScript, we also have **for .. in** loops to iterate over the properties of an object.

According to MDN Web Docs:

The for...in statement iterates over all enumerable properties of an object that are keyed by strings (ignoring ones keyed by Symbols), including inherited enumerable properties.

This is an example:

const object = { a: 1, b: 2, c: 3 };

for (const property in object) {
  console.log(property, object[property]);
}

The output when we run this code in the console of Chrome Developer Tools is:

While Loops

While loops are very similar in Python and JavaScript.

• In Python, we write the keyword while followed by the condition, a colon (:), and in a new line, the body of the loop (indented).
• In JavaScript, the syntax is very similar. The differences are that we have to surround the condition with parentheses and the body of the loop with curly braces.

While Loop in Python (left) and JavaScript (right)

do..while Loops in JavaScript

In JavaScript, we also have a type of loop that doesn't exist in Python.

This type of loop is called a **do..while** loop because it does something at least once and it continues running while a condition is True.

This is the basic syntax:

do {
    // Code
} while (condition);

💡 Tip: This type of loop guarantees that the code will be executed at least once.

This is particularly helpful when we ask for user input because the user will be prompted to enter the input. If the input is valid, we can continue with the program. But if it's not valid, we can prompt the user to enter the value again until it is valid.

Functions in Python and JavaScript

Functions are incredibly important to write concise, maintainable, and readable programs. The syntax is very similar in Python and JavaScript, but let's analyze their key differences:

• In Python, we write the keyword **def** followed by the name of the function, and within parentheses the parameters list. After this list, we write a colon (:) and the body of the function (indented).
• In JavaScript, the only differences are that we define a function using the **function** keyword and surround the body of the function with curly braces.

Function in Python (top) and JavaScript (bottom)

In addition, there is a very important difference between Python and JavaScript functions:

Number of Function Arguments

• In Python, the number of arguments passed to the function call has to match the number of parameters defined in the function definition (unless default values have been assigned for them in the function definition). If this is not the case, an exception will occur.

This is an example:

>>> def foo(x, y):
    print(x, y)


>>> foo(3, 4, 5)
Traceback (most recent call last):
  File "<pyshell#3>", line 1, in <module>
    foo(3, 4, 5)
TypeError: foo() takes 2 positional arguments but 3 were given

• In JavaScript, this is not necessary since parameters are optional. You can call a function with fewer or more arguments than the parameters that were defined in the function definition. Missing arguments are assigned the value **undefined** by default and extra arguments can be accessed with the **arguments** object.

This is an example in JavaScript:

Notice how the function was called with three arguments but only two parameters were included in the parameters list of the function definition.

💡 Tip: To get the number of arguments passed to the function, you can use arguments.length within the function.

Object-Oriented Programming in Python and JavaScript

Both Python and JavaScript support Object-Oriented Programming, so let's see how you can create and use the main elements of this programming paradigm.

Classes

The first line of a class definition is very similar in Python and JavaScript. We write the keyword **class** followed by the name of the class.

The only difference is that:

• In Python, after the name of the class, we write a colon (**:**)
• In JavaScript, we surround the content of the class with curly braces (**{}**)

Class definition in Python (left) and JavaScript (right)

💡 Tip: In Python and JavaScript, class names should start with an uppercase letter and each word should start with an uppercase letter as well.

Constructor and Attributes

The constructor is a special method that is called when a new instance of the class (a new object) is created. Its main purpose is to initialize the attributes of the instance.

• In Python, the constructor that initializes the new instance is called **init** (with two leading and trailing underscores). This method is called automatically when an instance of the class is created to initialize its attributes. Its parameters list defines the values that we have to pass to create the instance. This list starts with **self** as the first parameter.
• In JavaScript, the constructor method is called **constructor** and it has a parameters list as well.

💡 Tip: In Python, we use **self** to refer to the instance while in JavaScript we use **this**.

To assign values to the attributes in Python, we use this syntax:

self.attribute = value

In contrast, we use this syntax in JavaScript:

this.attribute = value;

Class Example in Python (left) and JavaScript (right)

Methods in Python and JavaScript

• In Python, we define methods with the **def** keyword followed by their name and the parameters list within parentheses. This parameters list starts with the **self** parameter to refer to the instance that is calling the method. After this list, we write a colon (**:**) and the body of the method indented.

This is an example:

class Circle:

    def __init__(self, radius, color):
        self.radius = radius
        self.color = color

    def calc_diameter(self):
        return self.radius * 2

• In JavaScript, methods are defined by writing their name followed by the parameters list and curly braces. Within the curly braces, we write the body of the method.

class Circle {

    constructor(radius, color) {
        this.radius = radius;
        this.color = color;
    }

    calcDiameter() {
        return this.radius * 2;
    }
}

Instances

To create instances of a class:

• In Python, we write the name of the class and pass the arguments within parentheses.

my_circle = Circle(5, "Red")

• In JavaScript, we need to add the **new** keyword before the name of the class.

my_circle = new Circle(5, "Red");

🔹 To Summarize

Python and JavaScript are very powerful languages with different real-world applications.

Python can be used for web development and for a wide range of applications, including scientific purposes. JavaScript is mainly used for web development (front-end and back-end) and for mobile app development.

They have important differences, but they both have the same basic elements that we need to write powerful programs.

I hope you liked this article and found it helpful. Now you know the key differences between Python and JavaScript.

⭐ Subscribe to my YouTube channel and follow me on Twitter to find more coding tutorials and tips.

MongoDB is undoubtedly one of the most popular NoSQL database choices today. And it has a great community and ecosystem.

In this article, we'll review some of the best practices to follow when you're setting up MongoDB and Mongoose with Node.js.

Pre-requisites for this article

This article is one of the part codedamn's backend learning path, where we start from backend basics and cover them in detail. Therefore I assume you have some experience with JavaScript (and Node.js) already.

Currently we are here:

If you have very little experience with Node.js/JavaScript or the back end in general, this is probably a good place to start. You can also find a free course on Mongoose + MongoDB + Node.js here. Let's dive in.

Why do you need Mongoose?

To understand why we need Mongoose, let's understand how MongoDB (and a database) works on the architecture level.

• You have a database server (MongoDB community server, for example)
• You have a Node.js script running (as a process)

MongoDB server listens on a TCP socket (usually), and your Node.js process can connect to it using a TCP connection.

But on the top of TCP, MongoDB also has its own protocol for understanding what exactly the client (our Node.js process) wants the database to do.

For this communication, instead of learning the messages we have to send on the TCP layer, we abstract that away with the help of a "driver" software, called MongoDB driver in this case. MongoDB driver is available as an npm package here.

Now remember, the MongoDB driver is responsible for connecting and abstracting the low level communication request/responses from you – but this only gets you so far as a developer.

Because MongoDB is a schemaless database, it gives you way more power than you need as a beginner. More power means more surface area to get things wrong. You need to reduce your surface area of bugs and screw-ups you can make in your code. You need something more.

Meet Mongoose. Mongoose is an abstraction over the native MongoDB driver (the npm package I mentioned above).

The general rule of thumb with abstractions (the way I understand) is that with every abstraction you lose some low-level operation power. But that doesn't necessarily mean it is bad. Sometimes it boosts productivity 1000x+ because you never really need to have full access to the underlying API anyway.

A good way to think about it is you technically create a realtime chat app both in C and in Python.

The Python example would be much easier and faster for you as a developer to implement with higher productivity.

C might be more efficient, but it'll come at a huge cost in productivity/speed of development/bugs/crashes. Plus, for the most part you don't need to have the power C gives you to implement websockets.

Similarly, with Mongoose, you can limit your surface area of lower level API access, but unlock a lot of potential gains and good DX.

How to connect Mongoose + MongoDB

Firstly, let's quickly see how you should connect to your MongoDB database in 2020 with Mongoose:

mongoose.connect(DB_CONNECTION_STRING, {
    useNewUrlParser: true,
    useUnifiedTopology: true,
    useCreateIndex: true,
    useFindAndModify: false
})

This connection format makes sure that you're using the new URL Parser from Mongoose, and that you are not using any deprecated practices. You can read in depth about all these deprecation messages here if you like.

How to perform Mongoose operations

Let's now go ahead and quickly discuss operations with Mongoose, and how you should perform them.

Mongoose gives you options for two things:

1. Cursor-based querying
2. Full fetching query

Cursor-based querying

Cursor-based querying means that you work with a single record at a time while you fetch a single or a batch of documents at a time from the database. This is an efficient way of working with huge amounts of data in a limited memory environment.

Imagine that you have to parse documents of 10GB in total size on a 1GB/1core cloud server. You cannot fetch the whole collection because that will not fit on your system. Cursor is a good (and the only?) option here.

Full fetching querying

This is the type of query where you get the full response of your query all at once. For the most part, this is what you'll be using. Therefore, we'll be focusing mostly on this method here.

How to use Mongoose Models

Models are the superpower of Mongoose. They help you enforce "schema" rules and provide a seamless integration of your Node code into database calls.

The very first step is to define a good model:

import mongoose from 'mongoose'

const CompletedSchema = new mongoose.Schema(
    {
        type: { type: String, enum: ['course', 'classroom'], required: true },
        parentslug: { type: String, required: true },
        slug: { type: String, required: true },
        userid: { type: String, required: true }
    },
    { collection: 'completed' }
)

CompletedSchema.index({ slug: 1, userid: 1 }, { unique: true })

const model = mongoose.model('Completed', CompletedSchema)
export default model

This is one trimmed down example directly from codedamn's codebase. A few interesting things you should note here:

1. Try to keep required: true on all fields which are required. This can be a huge pain saver for you if you don't use a static type checking system like TypeScript to assist you with correct property names while creating an object. Plus the free validation is super cool, too.
2. Define indexes and unique fields. unique property can also be added within a schema. Indexes are a broad topic, so I will not go into depth here. But on a large scale they can really help you to speed up your queries a lot.
3. Define a collection name explicitly. Although Mongoose can automatically give a collection name based on the name of model (Completed here, for example), this is way too much abstraction in my opinion. You should at least know about your database names and collections in your codebase.
4. Restrict values if you can, using enums.

How to perform CRUD Operations

CRUD means Create, Read, Update and Delete. These are the four fundamental options with which you can perform any sort of data manipulation in a database. Let's quickly see some examples of these operations.

The Create Operation

This simply means creating a new record in a database. Let's use the model we defined above to create a record:

try {
    const res = await CompletedSchema.create(record)
} catch(error) {
    console.error(error)
    // handle the error
}

Again, a few pointers here:

1. Use async-await instead of callbacks (nice on the eyes, no ground breaking performance benefit as such)
2. Use try-catch blocks around queries because your query can fail for a number of reasons (duplicate record, incorrect value, and so on)

The Read Operation

This means reading existing values from the database. it's simple just like it sounds, but there are a couple of gotchas you should know with Mongoose:

const res = await CompletedSchema.find(info).lean()

1. Can you see the lean() function call there? It is super useful for performance. By default, Mongoose processes the returned document(s) from the database and adds its magical methods on it (for example .save)
2. When you use .lean(), Mongoose returns plain JSON objects instead of memory and resource heavy documents. Makes queries faster and less expensive on your CPU, too.
3. However, you can omit .lean() if you are actually thinking of updating data (we'll see that next)

The Update Operation

If you already have a Mongoose document with you (without firing with .lean()), you can simply go ahead and modify the object property, and save it using object.save():

const doc = await CompletedSchema.findOne(info)
doc.slug = 'something-else'
await doc.save()

Remember that here, there are two database calls made. The first one is on findOne and the second one is on doc.save.

If you can, you should always reduce the number of requests hitting the database (because if you're comparing memory, network, and disk, network is almost always the slowest).

In the other case, you can use a query like this:

const res = await CompletedSchema.updateOne(<condition>, <query>).lean()

and it will only make a single call to the database.

The Delete Operation

Delete is also straightforward with Mongoose. Let's see how you can delete a single document:

const res = await CompletedSchema.deleteOne(<condition>)

Just like updateOne, deleteOne also accepts the first argument as the matching condition for the document.

There is also another method called deleteMany which should be used only when you know you want to delete multiple documents.

In any other case, always use deleteOne to avoid accidental multiple deletes, especially when you're trying to execute queries yourself.

Conclusion

This article was a simple introduction to the Mongoose and MongoDB world for Node.js developers.

If you enjoyed this article, you can step up your game even more as a developer by following the codedamn backend learning path. Please feel free to reach out to me on Twitter for any feedback!