To get the most out of this guide, you should have a background in JavaScript, understand how Mongoose works, and know Object-Oriented Programming basics.

Here's what we'll cover:

1. What is a Mongoose Schema?

2. Discriminator

3. Statics

4. Methods

5. Query Builder

6. Hooks

7. Summary

What is a Mongoose Schema?

Mongoose schemas provide a structured way to model data in a MongoDB database, allowing you to define the properties and behavior of the documents. Schemas serve as a blueprint for a document that gets saved in the database. They enables developers to enforce data integrity and work with MongoDB in a more intuitive and organized manner.

Within a MongoDB collection, a schema outlines the fields of the documents, their data types, validation rules, default values, constraints, and more.

Programmatically, a Mongoose schema is a JavaScript object. Actually, it is an instance of a built-in class called Schema inside the mongoose module. For this reason, you can add more methods to its prototype. This will help you implement many features as middleware, methods, statics, and more. You will learn about some of them in this tutorial.

Features you'll learn how to implement:

• Discriminator

• Statics

• Methods

• Query Builder

• Hooks

Discriminator

A discriminator is a feature that enables you to create multiple models (subtypes) that inherit from a base model (parent). This happens by defining a base schema and then extending it with extra fields specific to each subtype or each child schema.

All documents, regardless of their specific model, are stored in the same MongoDB collection. This keeps your data organized in a single collection while allowing for flexible querying and data management. Also, each document includes a special field that indicates its specific model type, allowing Mongoose to distinguish between the different subtypes.

How to use discriminator:

1. Start by defining a base schema, which will have the common fields among the subtypes. After that, create a model from it.

    import mongoose from 'mongoose';
   
    const baseSchema = new mongoose.Schema({
        name: { type: String, required: true },
    }, { discriminatorKey: 'kind' }; // defaults to '__t');
   
    const BaseModel = mongoose.model('Base', baseSchema);
   

2. Create the subtypes that extend the base schema by defining the discriminator for each one.

    const catSchema = new mongoose.Schema({
        meow: { type: Boolean, default: true }
    });
    // subtype
    const Cat = BaseModel.discriminator('Cat', catSchema);
   
    const dogSchema = new mongoose.Schema({
        bark: { type: Boolean, default: true }
    });
    // subtype
    const Dog = BaseModel.discriminator('Dog', dogSchema);
   

3. You can then create documents in the regular way. All the documents will be stored in the same collection, but each has its own type depending on its subtype model.

    const fluffy = await Cat.create({ name: 'Fluffy' });
    const rover = await Dog.create({ name: 'Rover' });
   

discriminator use case:

Let's say that you're building a multi-user Ecommerce web application which accommodates three main user roles: admins, clients, and sellers. Each of these roles plays a crucial part in the ecosystem of online shopping.

If you try to build a class for each role, you'll find that all the three have common fields and methods. You may decide to create a parent schema (user) and some other children schemas (client, seller, admin) that inherit from it.

You can use the discriminator to achieve this.

In your user.model.js file, add the following code:

import mongoose from "mongoose";

const userSchema = mongoose.Schema(
  {
    name: String,
    profilePic: String,
    email: String,
    password: String,
    birthDate: Date,
    accountAcctivated: { type: Boolean, default: false },
  },
  {
    timestamps: true,
    discriminatorKey: "role",
  }
);

const User = mongoose.model("User", userSchema);
export default User;

Now you have the base model (User) from which other subtypes will inherit. In this parent schema, you define the common fields that all users will share regardless of their roles.

In your client.model.js file:

import mongoose from "mongoose";
import User from "./user.model.js";

const clientSchema = mongoose.Schema(
  {
    products: Array,
    address: String,
    phone: String,
  }
);

const Client = User.discriminator("Client", clientSchema);
export default Client;

In your seller.model.js file:

import mongoose from "mongoose";
import User from "./user.model.js";

export const sellerSchema = mongoose.Schema(
  {
    rating: Number,
    businessType: { type: String, enum: ["individual", "corporation"] },
  }
);

const Seller = User.discriminator("Seller", sellerSchema);
export default Seller;

In your admin.model.js file:

import mongoose from "mongoose";
import User from "./user.model.js";

export const adminSchema = mongoose.Schema(
  {
    permissions: Array,
    assignedTasks: Array,
    department: String,
  }
);

const Admin = User.discriminator("Admin", adminSchema);
export default Admin;

The subtypes or children will be the Client, Seller, and Admin. In each subtype schema, you should add any extra fields or behaviors specific to this subtype only. By creating the child model using the discriminator, the child model will inherit all the fields and methods of its parent model User.

So the previous code will create a user collection in the database with each document having a role field either Client, or Seller, or Admin. All documents are now sharing the parent (user) fields, and depending on the role of each document, each has another extra field.

Although all the documents will be saved in one single collection, models are fully separated while coding. What does this mean?

For instance, If you need to retrieve all clients from the User collection, you should write Client.find({}). This statement uses the discriminator key to find all documents whose role is Client. This way, any operations or queries that refer to one of the child models will still be written separately from the parent model.

Note: Before diving into the next sections, just keep in mind that any statics, methods, query builders, or hooks should be defined before creating the model itself (that is, before const User = mongoose.model("User", userSchema);).

Statics

Statics are useful for defining functions that operate on the model level. They allow you to define reusable functions for operations related to the entire model. They help encapsulate logic that applies to the model rather than individual documents, making your code cleaner, more organized and maintainable

Methods like find, findOne, findById and others all are methods attached to the model. By using the statics property of Mongoose schemas, you will be able to build your own model method.

Statics are powerful. By using them, you can encapsulate complex queries that you might want to reuse. Also, you can create statics for operations that modify or aggregate data, such as counting documents or finding documents based on specific criteria.

statics use case

Statics are easy to build. You define a static method on your schema using the statics object.

In your user.model.js file, add these static methods, countUsers and findByEmail:

// model method
userSchema.statics.countUsers = function () {
    return this.countDocuments({});
};

// model method
userSchema.statics.findByEmail = async function (email) {
  return await this.findOne({ email });
};

Inside any static method, this refers to the model itself. In this example, this in this.findOne({ email }) refers to the User model.

Example usage:

const user = await User.findByEmail("foo@bar.com");
//or
const client = await Client.findByEmail("foo@bar.com");
//or
const seller = await Seller.findByEmail("foo@bar.com");
//or
const admin = await Admin.findByEmail("foo@bar.com");

When you call the static method on your model, the method gets called and this is replaced by the model you called the statics on. This line performs a query to find a single document in the MongoDB collection where the email field matches the provided email argument.

Methods

Methods are functions that you can define on a schema and that can be called on instances of documents created from this schema. They help encapsulate logic within the document itself, making your code cleaner and more modular.

By using instance methods, you can easily interact with and manipulate the data associated with specific documents.

methods use case

You can define methods on the schema using the methods object.

In your user.model.js file, add a document method through which you can check the password of a user:

// instance or document method
userSchema.methods.getProfile = function () {
    return `${this.name} (${this.email})`;
};

// instance or document method
userSchema.methods.checkPassword = function (password) {
    return password === this.password ? true : false;
};

Inside any document method, this refers to the document itself. In this example, this in this.password refers to the user document at which the method will get called on. This means that you can access all the fields of this document. This is so valuable because you can retrieve, modify, and check for anything related to this document.

Example usage:

const client = await Client.findById(...)
client.checkPassword("12345")
//or
const seller = await Seller.findById(...)
seller.checkPassword("12345")
//or
const admin = await Admin.findById(...)
admin.checkPassword("12345")

Since methods are instance-level functions, they are called on the documents. await Client.findById(...) will return a document that has all the built-in methods as well as your own predefined methods checkPassword and getProfile. So by calling, for example client.checkPassword("12345"), the this keyword in the checkPassword function definition will get replaced with the client document. This in turn will compare the user password with the password saved earlier in the database.

Query Builder

A query builder in Mongoose is a custom method that you can define on the query object to simplify and encapsulate common query patterns. These query builders allow you to create reusable and readable query logic, making it easier to work with your data.

One of the most valuable usages of query builders is chaining. They can be chained with other query builders that you've built or with standard query methods like find, sort, and so on.

Query builder use case

You define query builders by adding them to the query property of a Mongoose schema.

In your user.model.js file, add a query helper method that lets you implement pagination.

// query helper
userSchema.query.paginate = function ({ page, limit }) {
    // some code
    const skip = limit * (page - 1);
    return this.skip(skip).limit(limit);
};

To implement pagination, you need two important variables: first, the page number, and second, the number of items you will retrieve per page.

To query the database for a specific count of documents, you will always use the skip and limit built-in query methods in mongoose. skip is used to set a cursor after a certain number of documents, after which the query will get implemented. limit is used to retrieve a specific number of documents.

Inside any query builder method, this refers to the query itself. And since query builders are chainable, you can call any of them after each other.

Finally, any query builder method should return a mongoose query object, which is why you must write return this.skip(skip).limit(limit).

Example usage:

const results = await Client.find().paginate({ page: 2, limit: 5 });
//or
const results = await Seller.find().paginate({ page: 2, limit: 5 });
//or
const results = await Admin.find().paginate({ page: 2, limit: 5 });

You can then call it on any query, and await Client.find().paginate({ page: 2, limit: 5 }) will invoke the paginate function and replace the this keyword with Client.find() using the query builder.

You can implement pagination with certain conditions, but you'll always call skip and limit. By defining the paginate query builder you won't repeat yourself and you'll be able to encapsulate the logic in one single function.

Hooks

Hooks (also known as middleware) are functions that are executed at specific points in the lifecycle of a document. They allow you to add custom behavior before or after certain operations, such as saving, updating, or removing documents.

Types of Hooks

• Pre Hooks: Executed before an operation.

• Post Hooks: Executed after an operation.

Hooks use case

In your user.model.js file, add a post save middleware through which you can send an email for account activation once the user document is saved in the database.

// post hook
userSchema.post("save", async function (doc, next) {
  // send email logic
  // if succeeded
  return next();
  // if failed
  return next(new Error("Failed to send email!"));
});

The callback function will get invoked once you create a user through model.create() or any time you call save() method on the user document.

In this this example, if you need to avoid sending emails on save, you should write a condition to be sure that this save is for a new user only. You can write something like if (doc.createdAt.getTime() === doc.updatedAt.getTime()).

Summary

In this overview of Mongoose features, we've explored four key concepts: discriminators, statics, methods, and hooks.

Discriminators allow you to create multiple models that share a common schema enabling different document types to be stored in a single collection. This facilitates data management and querying.

Statics are model-level methods that provide reusable functionality applicable to the entire model. They encapsulate complex queries and data manipulation logic, helping to keep your codebase clean and maintainable.

Methods are instance-level functions that operate on individual document instances. They allow for custom behaviors and data manipulations specific to each document, so you can modify the document’s data in a specific way, such as formatting or calculating values based on its fields.

Hooks (or middleware) enable you to run functions at specific points in the document lifecycle, such as before or after saving, updating or deleting a document. This is useful for implementing validation, logging, or any other side effects related to database operations.

Together, these features enhance the versatility and organization of your Mongoose models, making it easier to build robust and maintainable applications with MongoDB.

Here can will find a repository where you can learn more about Mongoose schemas and use cases.

Testing is a vital part of software development. The sooner you start testing, the better.

In this article, I'll show you how to write tests for your NodeJs/ExpressJS and MongoDB/Mongoose applications with Jest and Supertest.

Let's get started

First let's set up a demo Express.js app.

Let's say we are building a backend REST API for an eCommerce application.

This app should:

• Get all the products
• Get a product by id
• Add product(s) to the database
• Delete product(s) from the database
• Update product information

Express.js App Set Up

Step 1: Project set up

First, create a folder and start a blank application with npm.

npm init

Fill all the details it asks for.

Then, install express, mongoose, axios and dotenv with the following command:

npm i express mongoose axios dotenv

Here's a link to the package.json on my GitHub.

Step 2: Create the boilerplate

Let's create all the folders and files and then fill them with some boilerplate code.

This is how your folder hierarchy should look:

.
├── controllers
│   └── product.controller.js
├── models
│   └── product.model.js
├── routes
│   └── product.route.js
├── package-lock.json
├── package.json
├── .env
├── app.js
└── server.js

Use these files' code by copying and pasting. Analyze the code and flow as best you can.

• [product.controller.js](https://github.com/itsrakeshhq/jest-tests-demo/blob/main/controllers/product.controller.js)
• [product.model.js](https://github.com/itsrakeshhq/jest-tests-demo/blob/main/models/product.model.js)
• [product.route.js](https://github.com/itsrakeshhq/jest-tests-demo/blob/main/routes/product.route.js)
• [app.js](https://github.com/itsrakeshhq/jest-tests-demo/blob/main/app.js)
• [server.js](https://github.com/itsrakeshhq/jest-tests-demo/blob/main/server.js)

Step 3: Database setup

I advise using two databases for a project—one for testing, the other for development. But just one database will be sufficient for learning purposes.

First, create a MongoDB account or log in.

Then create a new project. Give it a name and press the Next button.

Naming the project

Then click Create Project after that.

We must create a database in the following window by selecting a cloud provider, a location, and specs. So press Build a Database to get going.

Build a database

Choose "Shared" because it is sufficient for learning purposes. And then click Create.

Choose a deployment option

Next, select "aws" as your cloud provider and the region that is closest to you. Following your selection, click Create Cluster.

The cluster's formation will take some time. Create a user to access your database in the meanwhile.

Create Superuser

Choose "My Local Environment" because we are developing our application. You can then add an IP addresses. To conclude, click Close.

Add IP addresses

You will receive a URI string after the database is set up, which we'll use to connect to the database. The string appears as follows:

mongodb+srv://<YOUR_USERNAME>:<YOUR_PASSWORD>@<YOUR_CLUSTER_URL>/<DATABASE_NAME>?retryWrites=true&w=majority

Put this string in the .env file.

MONGODB_URI=your database string

Now we're ready to start testing our app.

How to Write Tests with Jest and SuperTest

Step 1: Install packages

You need three npm packages to begin writing tests: jest, supertest, and cross-env. You can install them like this:

npm i jest supertest cross-env

• jest: Jest is a framework for testing JavaScript code. Unit testing is the main usage of it.
• supertest: Using Supertest, we can test endpoints and routes on HTTP servers.
• cross-env: You can set environmental variables inline within a command using cross-env.

Step 2: Add test script

Open your package.json file and add the test script to the scripts.

"scripts": {
    "test": "cross-env NODE_ENV=test jest --testTimeout=5000",
    "start": "node server.js",
    "dev": "nodemon server.js"
},

In this case, we're using cross-env to set environment variables, jest to execute test suites, and testTimeout is set to 5000 because certain requests might take a while to finish.

Step 3: Start writing tests

First, create a folder called tests at the application's root, and then create a file there called product.test.js. Jest searches for the folder tests at the project's root when you do npm run test. As a result, you must place your test files in the tests folder.

Next, import the supertest and mongoose packages into the test file.

const mongoose = require("mongoose");
const request = require("supertest");

Import dotenv to load environment variables, and import app.js as that is where our application starts.

const mongoose = require("mongoose");
const request = require("supertest");
const app = require("../app");

require("dotenv").config();

You'll need to connect and disconnect the database before and after each test (because we don't require the database once testing is complete).

/* Connecting to the database before each test. */
beforeEach(async () => {
  await mongoose.connect(process.env.MONGODB_URI);
});

/* Closing database connection after each test. */
afterEach(async () => {
  await mongoose.connection.close();
});

Now you can write your first unit test.

describe("GET /api/products", () => {
  it("should return all products", async () => {
    const res = await request(app).get("/api/products");
    expect(res.statusCode).toBe(200);
    expect(res.body.length).toBeGreaterThan(0);
  });
});

In the above code,

• We use describe to describe the unit test. Even though it is not required, it will be useful to identify tests in test results.
• In it, we write the actual test code. Tell what the test performs in the first argument, and then in the second argument, write a callback function that contains the test code.
• In the callback function, the request is sent to the endpoint first, and the expected and actual responses are then compared. The test passes if both answers match, else, it fails. ✨ As simple as that ✨.

You can write tests for all the endpoints in the same manner.

describe("GET /api/products/:id", () => {
  it("should return a product", async () => {
    const res = await request(app).get(
      "/api/products/6331abc9e9ececcc2d449e44"
    );
    expect(res.statusCode).toBe(200);
    expect(res.body.name).toBe("Product 1");
  });
});

describe("POST /api/products", () => {
  it("should create a product", async () => {
    const res = await request(app).post("/api/products").send({
      name: "Product 2",
      price: 1009,
      description: "Description 2",
    });
    expect(res.statusCode).toBe(201);
    expect(res.body.name).toBe("Product 2");
  });
});

describe("PUT /api/products/:id", () => {
  it("should update a product", async () => {
    const res = await request(app)
      .patch("/api/products/6331abc9e9ececcc2d449e44")
      .send({
        name: "Product 4",
        price: 104,
        description: "Description 4",
      });
    expect(res.statusCode).toBe(200);
    expect(res.body.price).toBe(104);
  });
});

describe("DELETE /api/products/:id", () => {
  it("should delete a product", async () => {
    const res = await request(app).delete(
      "/api/products/6331abc9e9ececcc2d449e44"
    );
    expect(res.statusCode).toBe(200);
  });
});

Then run npm run test to run the test suites (suite - test file).

Test results

And that's it! You now know how to test your Express/Mongoose apps with Jest and SuperTest.

Now go forth and create new tests for your apps. :)

If you have any questions, feel free to message me on Twitter.

MongoDB is undoubtedly one of the most popular NoSQL database choices today. And it has a great community and ecosystem.

In this article, we'll review some of the best practices to follow when you're setting up MongoDB and Mongoose with Node.js.

Pre-requisites for this article

This article is one of the part codedamn's backend learning path, where we start from backend basics and cover them in detail. Therefore I assume you have some experience with JavaScript (and Node.js) already.

Currently we are here:

If you have very little experience with Node.js/JavaScript or the back end in general, this is probably a good place to start. You can also find a free course on Mongoose + MongoDB + Node.js here. Let's dive in.

Why do you need Mongoose?

To understand why we need Mongoose, let's understand how MongoDB (and a database) works on the architecture level.

• You have a database server (MongoDB community server, for example)
• You have a Node.js script running (as a process)

MongoDB server listens on a TCP socket (usually), and your Node.js process can connect to it using a TCP connection.

But on the top of TCP, MongoDB also has its own protocol for understanding what exactly the client (our Node.js process) wants the database to do.

For this communication, instead of learning the messages we have to send on the TCP layer, we abstract that away with the help of a "driver" software, called MongoDB driver in this case. MongoDB driver is available as an npm package here.

Now remember, the MongoDB driver is responsible for connecting and abstracting the low level communication request/responses from you – but this only gets you so far as a developer.

Because MongoDB is a schemaless database, it gives you way more power than you need as a beginner. More power means more surface area to get things wrong. You need to reduce your surface area of bugs and screw-ups you can make in your code. You need something more.

Meet Mongoose. Mongoose is an abstraction over the native MongoDB driver (the npm package I mentioned above).

The general rule of thumb with abstractions (the way I understand) is that with every abstraction you lose some low-level operation power. But that doesn't necessarily mean it is bad. Sometimes it boosts productivity 1000x+ because you never really need to have full access to the underlying API anyway.

A good way to think about it is you technically create a realtime chat app both in C and in Python.

The Python example would be much easier and faster for you as a developer to implement with higher productivity.

C might be more efficient, but it'll come at a huge cost in productivity/speed of development/bugs/crashes. Plus, for the most part you don't need to have the power C gives you to implement websockets.

Similarly, with Mongoose, you can limit your surface area of lower level API access, but unlock a lot of potential gains and good DX.

How to connect Mongoose + MongoDB

Firstly, let's quickly see how you should connect to your MongoDB database in 2020 with Mongoose:

mongoose.connect(DB_CONNECTION_STRING, {
    useNewUrlParser: true,
    useUnifiedTopology: true,
    useCreateIndex: true,
    useFindAndModify: false
})

This connection format makes sure that you're using the new URL Parser from Mongoose, and that you are not using any deprecated practices. You can read in depth about all these deprecation messages here if you like.

How to perform Mongoose operations

Let's now go ahead and quickly discuss operations with Mongoose, and how you should perform them.

Mongoose gives you options for two things:

1. Cursor-based querying
2. Full fetching query

Cursor-based querying

Cursor-based querying means that you work with a single record at a time while you fetch a single or a batch of documents at a time from the database. This is an efficient way of working with huge amounts of data in a limited memory environment.

Imagine that you have to parse documents of 10GB in total size on a 1GB/1core cloud server. You cannot fetch the whole collection because that will not fit on your system. Cursor is a good (and the only?) option here.

Full fetching querying

This is the type of query where you get the full response of your query all at once. For the most part, this is what you'll be using. Therefore, we'll be focusing mostly on this method here.

How to use Mongoose Models

Models are the superpower of Mongoose. They help you enforce "schema" rules and provide a seamless integration of your Node code into database calls.

The very first step is to define a good model:

import mongoose from 'mongoose'

const CompletedSchema = new mongoose.Schema(
    {
        type: { type: String, enum: ['course', 'classroom'], required: true },
        parentslug: { type: String, required: true },
        slug: { type: String, required: true },
        userid: { type: String, required: true }
    },
    { collection: 'completed' }
)

CompletedSchema.index({ slug: 1, userid: 1 }, { unique: true })

const model = mongoose.model('Completed', CompletedSchema)
export default model

This is one trimmed down example directly from codedamn's codebase. A few interesting things you should note here:

1. Try to keep required: true on all fields which are required. This can be a huge pain saver for you if you don't use a static type checking system like TypeScript to assist you with correct property names while creating an object. Plus the free validation is super cool, too.
2. Define indexes and unique fields. unique property can also be added within a schema. Indexes are a broad topic, so I will not go into depth here. But on a large scale they can really help you to speed up your queries a lot.
3. Define a collection name explicitly. Although Mongoose can automatically give a collection name based on the name of model (Completed here, for example), this is way too much abstraction in my opinion. You should at least know about your database names and collections in your codebase.
4. Restrict values if you can, using enums.

How to perform CRUD Operations

CRUD means Create, Read, Update and Delete. These are the four fundamental options with which you can perform any sort of data manipulation in a database. Let's quickly see some examples of these operations.

The Create Operation

This simply means creating a new record in a database. Let's use the model we defined above to create a record:

try {
    const res = await CompletedSchema.create(record)
} catch(error) {
    console.error(error)
    // handle the error
}

Again, a few pointers here:

1. Use async-await instead of callbacks (nice on the eyes, no ground breaking performance benefit as such)
2. Use try-catch blocks around queries because your query can fail for a number of reasons (duplicate record, incorrect value, and so on)

The Read Operation

This means reading existing values from the database. it's simple just like it sounds, but there are a couple of gotchas you should know with Mongoose:

const res = await CompletedSchema.find(info).lean()

1. Can you see the lean() function call there? It is super useful for performance. By default, Mongoose processes the returned document(s) from the database and adds its magical methods on it (for example .save)
2. When you use .lean(), Mongoose returns plain JSON objects instead of memory and resource heavy documents. Makes queries faster and less expensive on your CPU, too.
3. However, you can omit .lean() if you are actually thinking of updating data (we'll see that next)

The Update Operation

If you already have a Mongoose document with you (without firing with .lean()), you can simply go ahead and modify the object property, and save it using object.save():

const doc = await CompletedSchema.findOne(info)
doc.slug = 'something-else'
await doc.save()

Remember that here, there are two database calls made. The first one is on findOne and the second one is on doc.save.

If you can, you should always reduce the number of requests hitting the database (because if you're comparing memory, network, and disk, network is almost always the slowest).

In the other case, you can use a query like this:

const res = await CompletedSchema.updateOne(<condition>, <query>).lean()

and it will only make a single call to the database.

The Delete Operation

Delete is also straightforward with Mongoose. Let's see how you can delete a single document:

const res = await CompletedSchema.deleteOne(<condition>)

Just like updateOne, deleteOne also accepts the first argument as the matching condition for the document.

There is also another method called deleteMany which should be used only when you know you want to delete multiple documents.

In any other case, always use deleteOne to avoid accidental multiple deletes, especially when you're trying to execute queries yourself.

Conclusion

This article was a simple introduction to the Mongoose and MongoDB world for Node.js developers.

If you enjoyed this article, you can step up your game even more as a developer by following the codedamn backend learning path. Please feel free to reach out to me on Twitter for any feedback!

Building a full stack web app is no mean feat. Learning to deploy one to production so that you can share it with the world can add an additional layer of complexity.

In this new tutorial, we'll learn how to deploy a full stack MEVN app to Heroku!

Follow along (46 minute watch):

You can follow the original tutorial to build this app here.

Happy coding!

If you enjoyed this article, please consider checking out my games and books, subscribing to my YouTube channel, or joining the Entromancy Discord.

M. S. Farzan, Ph.D. has written and worked for high-profile video game companies and editorial websites such as Electronic Arts, Perfect World Entertainment, Modus Games, and MMORPG.com, and has served as the Community Manager for games like Dungeons & Dragons Neverwinter and Mass Effect: Andromeda. He is the Creative Director and Lead Game Designer of Entromancy: A Cyberpunk Fantasy RPG and author of The Nightpath Trilogy. Find M. S. Farzan on Twitter @sominator.

Performance is an essential parameter to consider when you're designing any piece of software. It is particularly important when it comes to what happens behind-the-scenes.

We, as developers and technologists, adopt multiple tweaks and implementations in order to improve performance. This is where caching comes into play.

Caching is defined as a mechanism to store data or files in a temporary storage location from where it can be instantly accessed whenever required.

Caching has become a must have in web applications nowadays. We can use Redis to supercharge our web APIs - which are built using Node.js and MongoDB.

"Caching would apparently still play a super important role 100 to 200 years down the line."

Redis: A Layman's Overview

Redis, according to the official documentation, is defined as an in-memory data structure store which is used as a database, message broker, or cache storage. It supports data structures such as strings, hashes, lists, sets, sorted sets with range queries, bitmaps, hyperloglogs, geospatial indexes with radius queries and streams.

Okay, that is quite a lot of data structures right there. Just to make it simple, almost all the data structures supported can be condensed into one form of string or the other. You will get more clarity as we run through the implementation.

But one thing is clear. Redis is powerful, and when used properly can make our applications not only faster but amazingly efficient. Enough talk. Let's get our hands dirty.

Let's Talk Code

Before we start off, you will need to get redis setup in your local system. You can follow this quick setup process to get redis up and running.

Done? Cool. Let's start. We have a simple application created in Express which makes use of an instance in MongoDB Atlas to read and write data from.

We have two major APIs created in the /blogs route file.

...

// GET - Fetches all blog posts for required user
blogsRouter.route('/:user')
    .get(async (req, res, next) => {
        const blogs = await Blog.find({ user: req.params.user });

        res.status(200).json({
            blogs,
        });
    });

// POST - Creates a new blog post
blogsRouter.route('/')
    .post(async (req, res, next) => {
        const existingBlog = await Blog.findOne({ title: req.body.title });

        if (!existingBlog) {
            let newBlog = new Blog(req.body);

            const result = await newBlog.save();

            return res.status(200).json({
                message: `Blog ${result.id} is successfully created`,
                result,
            });
        }

        res.status(200).json({
            message: 'Blog with same title exists',
        });
    });

...

Sprinkling Some Redis Goodness

We start off by downloading the npm package redis to connect to the local redis server.

const mongoose = require('mongoose');
const redis = require('redis');
const util = require('util');

const redisUrl = 'redis://127.0.0.1:6379';
const client = redis.createClient(redisUrl);
client.hget = util.promisify(client.hget);

...

We make use of the utils.promisify function to transform the client.hget function to return a promise instead of a callback. You can read more about promisification here.

The Redis connection is in place. Before we start writing any more caching code, let us take a step back and try to understand what are the requirements we need to fulfill and the likely challenges we might face.

Our caching strategy should be able to address the following points.

• Cache the request for all blog posts for a particular user

• Clear cache every time a new blog post is created

  The likely challenges we should be careful of as we go about our strategy are:

• The right way to handle key creation for storing cache data

• Cache expiration logic and forced expiration for maintaining cache freshness
• Reusable implementation of caching logic

All right. We have our points jotted down and redis connected. On to the next step.

Overriding the Default Mongoose Exec Function

We want our caching logic to be reusable. And not only reusable, we also want it to be the first checkpoint before we make any query to the database. This can easily be done by using a simple hack of piggy-backing onto the mongoose exec function.

...

const exec = mongoose.Query.prototype.exec;

...

mongoose.Query.prototype.exec = async function() {
    ...

     const result = await exec.apply(this, arguments);

    console.log('Data Source: Database');
    return result;
}

...

We make use of the prototype object of mongoose to add our caching logic code as the first execution in the query.

Adding Cache as a Query

In order to denote which queries should be up for caching, we create a mongoose query. We provide the ability to pass the user to be used as a hash-key through the options object.

Note: Hashkey serves as an identifier for a hash data structure which, in layman terms, can be stated as the parent key to a set of key-value pairs. Thereby, enabling caching of a larger number of query-value set. You can read more about hashes in redis here.

...

mongoose.Query.prototype.cache = function(options = {}) {
    this.enableCache = true;
    this.hashKey = JSON.stringify(options.key || 'default');

    return this;
};

...

Having done so, we can easily use the cache(<options argument>) query along with the queries we want to cache in the following manner.

...

const blogs = await Blog
                    .find({ user: req.params.user })
                    .cache({ key: req.params.user });

...

Crafting The Cache Logic

We have set up a common reusable query to denote which queries need to be cached. Let's go ahead and write the central caching logic.

...

mongoose.Query.prototype.exec = async function() {
    if (!this.enableCache) {
        console.log('Data Source: Database');
        return exec.apply(this, arguments);
    }

    const key = JSON.stringify(Object.assign({}, this.getQuery(), {
        collection: this.mongooseCollection.name,
    }));

    const cachedValue = await client.hget(this.hashKey, key);

    if (cachedValue) {
        const parsedCache = JSON.parse(cachedValue);

        console.log('Data Source: Cache');

        return Array.isArray(parsedCache) 
                ?  parsedCache.map(doc => new this.model(doc)) 
                :  new this.model(parsedCache);
    }

    const result = await exec.apply(this, arguments);

    client.hmset(this.hashKey, key, JSON.stringify(result), 'EX', 300);

    console.log('Data Source: Database');
    return result;
};

...

Whenever we use the cache() query along with our main query, we set the enableCache key to be true.

If the key is false, we return the main exec query as default. If not, we first form the key for fetching and storing/refreshing the cache data.

We use the collection name along with the default query as the key name for the sake of uniqueness. The hash-key used is the name of the user which we have already set earlier in the cache() function definition.

The cached data is fetched using the client.hget() function which requires the hash-key and the consequent key as parameters.

Note: We always use JSON.parse() while fetching any data from redis. And similarly, we use JSON.stringify() on the key and data before storing anything into redis. This is done since redis does not support JSON data structures.

Once we have obtained the cached data, we have to transform each of the cached objects into a mongoose model. This can be done by simply using new this.model(<object>).

If the cache does not contain the required data, we make a query to the database. Then, having returned the data to the API, we refresh the cache using client.hmset(). We also set a default cache expiration time of 300 seconds. This is customizable based on your caching strategy.

The caching logic is in place. We have also set a default expiration time. Next up, we look at forcing cache expiration whenever a new blog post is created.

Forced Cache Expiration

In certain cases, such as when a user creates a new blog post, the user expects that the new post should be available when they fetche all the posts.

In order to do so, we have to clear the cache related to that user and update it with new data. So we have to force expiration. We can do that by invoking the del() function provided by redis.

...

module.exports = {
    clearCache(hashKey) {
        console.log('Cache cleaned');
        client.del(JSON.stringify(hashKey));
    }
}

...

We also have to keep in mind that we will be forcing expiration on multiple routes. One extensible way is to use this clearCache() as a middleware and call it once any query related to a route has finished execution.

const { clearCache } = require('../services/cache');

module.exports = async (req, res, next) => {
    // wait for route handler to finish running
    await next(); 

    clearCache(req.body.user);
}

This middleware can be easily called on a particular route in the following way.

...

blogsRouter.route('/')
    .post(cleanCache, async (req, res, next) => {

    ...

    }

...

And we are done. I agree that was a quite a lot of code. But with that last part, we have set up redis with our application and taken care of almost all the likely challenges. It is time to see our caching strategy in action.

Redis in Action

We make use of Postman as the API client to see our caching strategy in action. Here we go. Let's run through the API operations, one by one.

1. We create a new blog post using the /blogs route

New Blog Post Creation

2. We then fetch all the blog posts related to user tejaz

Fetching all Blog Posts for User tejaz

3. We fetch all the blog posts for user tejaz once more.

Fetch all Blog Posts for User tejaz Once More

You can clearly see that when we fetch from the cache, the time taken has gone down from 409ms to 24ms. This supercharges your API by decreasing the time taken by almost 95%.

Plus, we can clearly see that cache expiration and update operations work as expected.

You can find the complete source code in the redis-express folder here.

Conclusion

Caching is a mandatory step for any performance-efficient and data-intensive application. Redis helps you easily achieve this in your web applications. It is a super powerful tool, and if used properly it can definitely provide an excellent experience to developers as well as users all around.

You can find the complete set of redis commands here. You can use it with redis-cli to monitor your cache data and application processes.

The possibilities offered by any particular technology is truly endless. If you have any queries, you can reach out to me on [LinkedIn](https://www.linkedin.com/in/tarique-ejaz/).

In the mean time, keep coding.

Building a full stack app is no small endeavor, and deploying it comes with its own host of things to consider.

I'm a tabletop game developer, and recently deployed a simple roleplaying game tracker that uses the M-E-V-N stack (you can follow my tutorial to create your own app here).

In deploying the app, I came across three key takeaways that may be useful as you begin considering the best way to bring your projects from development to production.

You can check out the code to my app on GitHub, and I should mention that it includes Chad Carteret's very cool CSS statblock in prettifying what's otherwise very basic HTML.

If you're thinking of following the same path to deployment as I have here, be sure to check out the official documentation on Heroku, the Vue CLI, and this tutorial by Nick Manning.

You'll also want to take a look at Will Abramson's article on a similar topic.

On to deployment!

Your front end and back end can be deployed together or separately, depending on your app's complexity.

One snag that seems to appear immediately when considering production is the structural question of how to deploy the front and back ends of your app.

Should the client (or static files) live in the same place as the server and database? Or should they be separate, with the front end making HTTP requests from elsewhere to the back end using CORS?

The answer is yes! Or no. Maybe??

For better or worse, there's no one-size-fits-all solution to this question, as it will likely depend on your app's architecture and complexity. In the roleplaying game tracker that I linked to above, I have the whole stack running on a single Heroku dyno, with the following folder structure:

All of the front and back end files live in the same place, with the Vue client built for production in a folder located at /client/dist.

In server.js, along with a bunch of database and routing code, there's a little line that says:

server.use(serveStatic(__dirname + "/client/dist"));

In Express, this tells the app to serve my static client files from a particular folder, and enables me to run the front and back ends all within the same environment. If you're deploying a similarly simple app, this type of solution might work for you as well.

Conversely, and depending on your project's complexity, you may have to separate the front and back ends and treat them as separate applications, which is no big deal. In the app above, my client is making calls to static API endpoints that are handled by the server, like this:

getQuests: function () {
    axios
        .get('https://mevn-rpg-app.herokuapp.com/quests')
        .then(response => (this.questData = response.data))                   
 }

Technically, my client could be making those requests from anywhere - even a static GitHub Pages site. This type of solution can help separate your app into two distinct entities to tackle, which is sometimes better than trying to cram the whole project into one location.

One note: if you're going to be making cross-origin HTTP requests - that is, requests from a client that lives on a separate domain from the API or server - you'll need to become familiar with CORS. You can read more about it in this article.

Your code will need to change to support a production environment.

When you're knee deep in the development process, it can be easy to lose track of how much of your code depends on local files or other data.

Consider the following in a locally-running server.js:

server.listen(3000, () => console.log("Server started!"));

On a local machine, the code just asks the server to listen on port 3000 and log to the console that we're ready for liftoff.

In a production environment, the server has no concept of where the "localhost" should be, or to whose port 3000 it should be listening. With this example, you'd have to change your code to something like:

const port = process.env.PORT || 3000;

server.listen(port, () => console.log("Server started!"));

The above instructs the server to instead listen at port 3000 of the process that's currently running, wherever that might be (check out this article for further reading on this topic).

Similarly, in my app, I have several modules that need to be imported by one another to function:

In /routes/Quests.js, for example, I have a router that tells the server what to do when receiving API requests to interact with quest-related items in the database. The router needs to import a Mongoose schema from /models/quest.js to function properly. If the application were running locally, we could just say:

const Quest = require('../models/quest');

Pretty simple! Yet, unfortunately, our server won't know where to find the root directory of our project once deployed. In Express, we'd change our code to something like:

const path = require('path');
const Quest = require(path.join(__dirname, '../models/quest'));

Your particular case might be different, depending on your language and framework(s), but you'll need to get specific about what your code looks like in a production environment rather than in your local development environment.

Additionally, you're probably already familiar with whatever bundler you're using for your front end (e.g., webpack), and will want to build your client for production to optimize it for deployment.

You have a multitude of deployment platforms from which to choose.

If you've deployed a front end website or some other type of static app, you might be familiar with just pushing your files to some remote repository and calling it a day.

Deploying a full stack app (or even just a back end) is eminently more complex. You'll need a dedicated server, or something that emulates one, to respond to the HTTP requests that it will be receiving and work with an online database.

There are a number of services that will do this very thing for you, and the spectrum ranges based on price, scalability, complexity, and other factors.

There's a bunch of articles out there that compare PaaS options for deployment, but here are some thoughts as you consider platforms for your first project:

• Heroku: If you have a small project like mine or just want to learn about deployment, a good first step could be Heroku.
• AWS, Docker, and Kubernetes: If you're seeking a career in full stack web development or DevOps, now's a good time to familiarize yourself with Amazon Web Services and/or container platforms like Docker and Kubernetes.
• Azure: If you're a C# or .NET developer, Azure appears to be a seamless way to deploy your apps without having to leave the safety of the Microsoft ecosystem.

There are, of course, several other options out there, and your particular use-case scenario might depend on pricing or the specific feature sets that are on offer.

Additionally, you'll want to consider any addons that will be necessary to replicate your app's functionality in a production environment. My roleplaying game tracker, for example, uses MongoDB, but the production version certainly can't use the little database on my local machine! Instead, I've used the mLab Heroku addon to get the live site up and running with the same functionality as in my development environment.

Your app's success, as well as your own progress as a full stack web developer, depend on your ability to consider deployment options and create a successful pipeline for production. With a little research, I'm certain that you can find the best solution that fits all of your app's needs.

Happy coding!

If you enjoyed this article, please consider checking out my games and books, subscribing to my YouTube channel, or joining the Entromancy Discord.

M. S. Farzan, Ph.D. has written and worked for high-profile video game companies and editorial websites such as Electronic Arts, Perfect World Entertainment, Modus Games, and MMORPG.com, and has served as the Community Manager for games like Dungeons & Dragons Neverwinter and Mass Effect: Andromeda. He is the Creative Director and Lead Game Designer of Entromancy: A Cyberpunk Fantasy RPG and author of The Nightpath Trilogy. Find M. S. Farzan on Twitter @sominator.

I'm a tabletop game developer, and enjoy making apps that have the potential to perform some service related to gaming. In this article, we'll walk through the steps to create a roleplaying game character generator using MongoDB, Express, Vue, and Node (also known as the "MEVN" stack).

Prerequisites: this tutorial presumes that you have Node/NPM and MongoDB installed and configured, with a code editor and CLI (or IDE) ready to go.

If you'd prefer to follow along with a visual tutorial, you can check out the companion video to this article below:

I should also mention that this tutorial would not have been possible without Bennett Dungan's article on building a REST API, Aneeta Sharma's tutorial on full stack MEVN web apps, and Matt Maribojoc's article on the same topic.

I used each of these articles in addition to official documentation (for Vue, Express, and a whole lot more) in learning to create my own MEVN apps (you can read more about my journey with web APIs here).

You can access the entire repository for this tutorial on GitHub.

The Front End

Our app is going to allow us to create new roleplaying game characters and view them altogether, with the following stack:

• Vue Client
• Node/Express Server
• MongoDB Database

The Vue Client will make HTTP requests to the Node/Express Server (or "API"), which will in turn communicate with our MongoDB Database to send data back up the stack.

We'll begin by opening a command line, creating a new directory for our project, and navigating into that directory:

mkdir mevn-character-generator
cd mevn-character-generator

We'll then install the Vue CLI globally to help us scaffold a basic app:

npm install -g @vue/cli

Next, we'll use the Vue CLI to create a new app called "Client" within our mevn-character-generator directory:

vue create client

You can just hit "enter" at the prompt to keep going.

We can run our app by first navigating into the /client folder:

cd client
npm run serve

When the script has completed running, we can now open a browser page and navigate to the URL indicated by our terminal (usually http://localhost:8080 or 8081). We should see something like this:

Nice! The Vue CLI has scaffolded a basic app for us, and is rendering it right into the browser. It'll also reload the page automatically upon file changes, and throw errors if something in the code looks amiss.

Let's open the project directory in our code editor to take a look at the file structure, which should look like this:

If you're OCD like I am, you can go ahead and delete the "favicon.ico" file and "/assets" folder as we won't need them for this project.

Diving into /src/main.js, we see:

import Vue from 'vue'
import App from './App.vue'

Vue.config.productionTip = false

new Vue({
  render: h => h(App),
}).$mount('#app')

This file is the main entry point for our client. It tells the browser to mount our App.vue file to the div with id "#app" in /public/index.html.

Let's look at /src/App.vue (I've omitted some code for readability):

<template>
  <div id="app">
    <img alt="Vue logo" src="./assets/logo.png">
    <HelloWorld msg="Welcome to Your Vue.js App"/>
  </div>
</template>

<script>
import HelloWorld from './components/HelloWorld.vue'

export default {
  name: 'App',
  components: {
    HelloWorld
  }
}
</script>

<style>
#app {
...
}
</style>

App.vue is a typical Vue component, with ,

1. It gives structure to MongoDB Collections
2. It gives you helpful methods to use

In this article, we'll go through:

1. The basics of using Mongoose
2. Mongoose subdocuments
3. Mongoose population

By the end of the article, you should be able to use Mongoose without problems.

Prerequisites

I assume you have done the following:

1. You have installed MongoDB on your computer
2. You know how to set up a local MongoDB connection
3. You know how to see the data you have in your database
4. You know what "collections" are in MongoDB

If you don't know any of these, please read "How to set up a local MongoDB connection" before you continue.

I also assume you know how to use MongoDB to create a simple CRUD app. If you don't know how to do this, please read "How to build a CRUD app with Node, Express, and MongoDB" before you continue.

Mongoose Basics

Here, you'll learn how to:

1. Connect to the database
2. Create a Model
3. Create a Document
4. Find a Document
5. Update a Document
6. Delete a Document

Connecting to a database

First, you need to download Mongoose.

npm install mongoose --save

You can connect to a database with the connect method. Let's say we want to connect to a database called street-fighters. Here's the code you need:

const mongoose = require('mongoose')
const url = 'mongodb://127.0.0.1:27017/street-fighters'

mongoose.connect(url, { useNewUrlParser: true })

We want to know whether our connection has succeeded or failed. This helps us with debugging.

To check whether the connection has succeeded, we can use the open event. To check whether the connection failed, we use the error event.

const db = mongoose.connection
db.once('open', _ => {
  console.log('Database connected:', url)
})

db.on('error', err => {
  console.error('connection error:', err)
})

Try connecting to the database. You should see a log like this:

Creating a Model

In Mongoose, you need to use models to create, read, update, or delete items from a MongoDB collection.

To create a Model, you need to create a Schema. A Schema lets you define the structure of an entry in the collection. This entry is also called a document.

Here's how you create a schema:

const mongoose = require('mongoose')
const Schema = mongoose.Schema

const schema = new Schema({
  // ...
})

You can use 10 different kinds of values in a Schema. Most of the time, you'll use these six:

• String
• Number
• Boolean
• Array
• Date
• ObjectId

Let's put this into practice.

Say we want to create characters for our Street Fighter database.

In Mongoose, it's a normal practice to put each model in its own file. So we will create a Character.js file first. This Character.js file will be placed in the models folder.

project/
    |- models/
        |- Character.js

In Character.js, we create a characterSchema.

const mongoose = require('mongoose')
const Schema = mongoose.Schema

const characterSchema = new Schema({
  // ...
})

Let's say we want to save two things into the database:

1. Name of the character
2. Name of their ultimate move

Both can be represented with Strings.

const mongoose = require('mongoose')
const Schema = mongoose.Schema

const characterSchema = new Schema({
  name: String,
  ultimate: String
})

Once we've created characterSchema, we can use mongoose's model method to create the model.

module.exports = mongoose.model('Character', characterSchema)

Creating a document

Let's say you have a file called index.js. This is where we'll perform Mongoose operations for this tutorial.

project/
    |- index.js
    |- models/
        |- Character.js

First, you need to load the Character model. You can do this with require.

const Character = require('./models/Character')

Let's say you want to create a character called Ryu. Ryu has an ultimate move called "Shinku Hadoken".

To create Ryu, you use the new, followed by your model. In this case, it's new Character.

const ryu = new Character ({
  name: 'Ryu',
  ultimate: 'Shinku Hadoken'
})

new Character creates the character in memory. It has not been saved to the database yet. To save to the database, you can run the save method.

ryu.save(function (error, document) {
  if (error) console.error(error)
  console.log(document)
})

If you run the code above, you should see this in the console.

Promises and Async/await

Mongoose supports promises. It lets you write nicer code like this:

// This does the same thing as above
function saveCharacter (character) {
  const c = new Character(character)
  return c.save()
}

saveCharacter({
  name: 'Ryu',
  ultimate: 'Shinku Hadoken'
})
  .then(doc => { console.log(doc) })
  .catch(error => { console.error(error) })

You can also use the await keyword if you have an asynchronous function.

If the Promise or Async/Await code looks foreign to you, I recommend reading "JavaScript async and await" before continuing with this tutorial.

async function runCode() {
  const ryu = new Character({
    name: 'Ryu',
    ultimate: 'Shinku Hadoken'
  })

  const doc = await ryu.save()
  console.log(doc)
}

runCode()
  .catch(error => { console.error(error) })

Note: I'll use the async/await format for the rest of the tutorial.

Uniqueness

Mongoose adds a new character to the database each time you use new Character and save. If you run the code(s) above three times, you'd expect to see three Ryus in the database.

We don't want to have three Ryus in the database. We want to have ONE Ryu only. To do this, we can use the unique option.

const characterSchema = new Schema({
  name: { type: String, unique: true },
  ultimate: String
})

The unique option creates a unique index. It ensures that we cannot have two documents with the same value (for name in this case).

For unique to work properly, you need to clear the Characters collection. To clear the Characters collection, you can use this:

await Character.deleteMany({})

Try to add two Ryus into the database now. You'll get an E11000 duplicate key error. You won't be able to save the second Ryu.

Let's add another character into the database before we continue the rest of the tutorial.

const ken = new Character({
  name: 'Ken',
  ultimate: 'Guren Enjinkyaku'
})

await ken.save()

Finding a document

Mongoose gives you two methods to find stuff from MongoDB.

1. findOne: Gets one document.
2. find: Gets an array of documents

findOne

findOne returns the first document it finds. You can specify any property to search for. Let's search for Ryu:

const ryu = await Character.findOne({ name: 'Ryu' })
console.log(ryu)

find

find returns an array of documents. If you specify a property to search for, it'll return documents that match your query.

const chars = await Character.find({ name: 'Ryu' })
console.log(chars)

If you did not specify any properties to search for, it'll return an array that contains all documents in the collection.

const chars = await Character.find()
console.log(chars)

Updating a document

Let's say Ryu has three special moves:

1. Hadoken
2. Shoryuken
3. Tatsumaki Senpukyaku

We want to add these special moves into the database. First, we need to update our CharacterSchema.

const characterSchema = new Schema({
  name: { type: String, unique: true },
  specials: Array,
  ultimate: String
})

Then, we use one of these two ways to update a character:

1. Use findOne, then use save
2. Use findOneAndUpdate

findOne and save

First, we use findOne to get Ryu.

const ryu = await Character.findOne({ name: 'Ryu' })
console.log(ryu)

Then, we update Ryu to include his special moves.

const ryu = await Character.findOne({ name: 'Ryu' })
ryu.specials = [
  'Hadoken',
  'Shoryuken',
  'Tatsumaki Senpukyaku'
]

After we modified ryu, we run save.

const ryu = await Character.findOne({ name: 'Ryu' })
ryu.specials = [
  'Hadoken',
  'Shoryuken',
  'Tatsumaki Senpukyaku'
]

const doc = await ryu.save()
console.log(doc)

findOneAndUpdate

findOneAndUpdate is the same as MongoDB's findOneAndModify method.

Here, you search for Ryu and pass the fields you want to update at the same time.

// Syntax
await findOneAndUpdate(filter, update)

// Usage
const doc = await Character.findOneAndUpdate(
  { name: 'Ryu' },
  {
    specials: [
      'Hadoken',
      'Shoryuken',
      'Tatsumaki Senpukyaku'
    ]
  })

console.log(doc)

Difference between findOne + save vs findOneAndUpdate

Two major differences.

First, the syntax for findOne` + `save is easier to read than findOneAndUpdate.

Second, findOneAndUpdate does not trigger the save middleware.

I'll choose findOne + save over findOneAndUpdate anytime because of these two differences.

Deleting a document

There are two ways to delete a character:

1. findOne + remove
2. findOneAndDelete

Using findOne + remove

const ryu = await Character.findOne({ name: 'Ryu' })
const deleted = await ryu.remove()

Using findOneAndDelete

const deleted = await Character.findOneAndDelete({ name: 'Ken' })

Subdocuments

In Mongoose, subdocuments are documents that are nested in other documents. You can spot a subdocument when a schema is nested in another schema.

Note: MongoDB calls subdocuments embedded documents.

const childSchema = new Schema({
  name: String
});

const parentSchema = new Schema({
  // Single subdocument
  child: childSchema,

  // Array of subdocuments
  children: [ childSchema ]
});

In practice, you don't have to create a separate childSchema like the example above. Mongoose helps you create nested schemas when you nest an object in another object.

// This code is the same as above
const parentSchema = new Schema({
  // Single subdocument
  child: { name: String },

  // Array of subdocuments
  children: [{name: String }]
});

In this section, you will learn to:

1. Create a schema that includes a subdocument
2. Create documents that contain subdocuments
3. Update subdocuments that are arrays
4. Update a single subdocument

Updating characterSchema

Let's say we want to create a character called Ryu. Ryu has three special moves.

1. Hadoken
2. Shinryuken
3. Tatsumaki Senpukyaku

Ryu also has one ultimate move called:

1. Shinku Hadoken

We want to save the names of each move. We also want to save the keys required to execute that move.

Here, each move is a subdocument.

const characterSchema = new Schema({
  name: { type: String, unique: true },
  // Array of subdocuments
  specials: [{
    name: String,
    keys: String
  }]
  // Single subdocument
  ultimate: {
    name: String,
    keys: String
  }
})

You can also use the childSchema syntax if you wish to. It makes the Character schema easier to understand.

const moveSchema = new Schema({
  name: String,
  keys: String
})

const characterSchema = new Schema({
  name: { type: String, unique: true },
  // Array of subdocuments
  specials: [moveSchema],
  // Single subdocument
  ultimate: moveSchema
})

Creating documents that contain subdocuments

There are two ways to create documents that contain subdocuments:

1. Pass a nested object into new Model
2. Add properties into the created document.

Method 1: Passing the entire object

For this method, we construct a nested object that contains both Ryu's name and his moves.

const ryu = {
  name: 'Ryu',
  specials: [{
    name: 'Hadoken',
    keys: '↓ ↘ → P'
  }, {
    name: 'Shoryuken',
    keys: '→ ↓ ↘ → P'
  }, {
    name: 'Tatsumaki Senpukyaku',
    keys: '↓ ↙ ← K'
  }],
  ultimate: {
    name: 'Shinku Hadoken',
    keys: '↓ ↘ → ↓ ↘ → P'
  }
}

Then, we pass this object into new Character.

const char = new Character(ryu)
const doc = await char.save()
console.log(doc)

Method 2: Adding subdocuments later

For this method, we create a character with new Character first.

const ryu = new Character({ name: 'Ryu' })

Then, we edit the character to add special moves:

const ryu = new Character({ name: 'Ryu' })
const ryu.specials = [{
  name: 'Hadoken',
  keys: '↓ ↘ → P'
}, {
  name: 'Shoryuken',
  keys: '→ ↓ ↘ → P'
}, {
  name: 'Tatsumaki Senpukyaku',
  keys: '↓ ↙ ← K'
}]

Then, we edit the character to add the ultimate move:

const ryu = new Character({ name: 'Ryu' })

// Adds specials
const ryu.specials = [{
  name: 'Hadoken',
  keys: '↓ ↘ → P'
}, {
  name: 'Shoryuken',
  keys: '→ ↓ ↘ → P'
}, {
  name: 'Tatsumaki Senpukyaku',
  keys: '↓ ↙ ← K'
}]

// Adds ultimate
ryu.ultimate = {
  name: 'Shinku Hadoken',
  keys: '↓ ↘ → ↓ ↘ → P'
}

Once we're satisfied with ryu, we run save.

const ryu = new Character({ name: 'Ryu' })

// Adds specials
const ryu.specials = [{
  name: 'Hadoken',
  keys: '↓ ↘ → P'
}, {
  name: 'Shoryuken',
  keys: '→ ↓ ↘ → P'
}, {
  name: 'Tatsumaki Senpukyaku',
  keys: '↓ ↙ ← K'
}]

// Adds ultimate
ryu.ultimate = {
  name: 'Shinku Hadoken',
  keys: '↓ ↘ → ↓ ↘ → P'
}

const doc = await ryu.save()
console.log(doc)

Updating array subdocuments

The easiest way to update subdocuments is:

1. Use findOne to find the document
2. Get the array
3. Change the array
4. Run save

For example, let's say we want to add Jodan Sokutou Geri to Ryu's special moves. The keys for Jodan Sokutou Geri are ↓ ↘ → K.

First, we find Ryu with findOne.

const ryu = await Characters.findOne({ name: 'Ryu' })

Mongoose documents behave like regular JavaScript objects. We can get the specials array by writing ryu.specials.

const ryu = await Characters.findOne({ name: 'Ryu' })
const specials = ryu.specials
console.log(specials)

This specials array is a normal JavaScript array.

const ryu = await Characters.findOne({ name: 'Ryu' })
const specials = ryu.specials
console.log(Array.isArray(specials)) // true

We can use the push method to add a new item into specials,

const ryu = await Characters.findOne({ name: 'Ryu' })
ryu.specials.push({
  name: 'Jodan Sokutou Geri',
  keys: '↓ ↘ → K'
})

After updating specials, we run save to save Ryu to the database.

const ryu = await Characters.findOne({ name: 'Ryu' })
ryu.specials.push({
  name: 'Jodan Sokutou Geri',
  keys: '↓ ↘ → K'
})

const updated = await ryu.save()
console.log(updated)

Updating a single subdocument

It's even easier to update single subdocuments. You can edit the document directly like a normal object.

Let's say we want to change Ryu's ultimate name from Shinku Hadoken to Dejin Hadoken. What we do is:

1. Use findOne to get Ryu.
2. Change the name in ultimate
3. Run save

const ryu = await Characters.findOne({ name: 'Ryu' })
ryu.ultimate.name = 'Dejin Hadoken'

const updated = await ryu.save()
console.log(updated)

Population

MongoDB documents have a size limit of 16MB. This means you can use subdocuments (or embedded documents) if they are small in number.

For example, Street Fighter characters have a limited number of moves. Ryu only has 4 special moves. In this case, it's okay to use embed moves directly into Ryu's character document.

But if you have data that can contain an unlimited number of subdocuments, you need to design your database differently.

One way is to create two separate models and combine them with populate.

Creating the models

Let's say you want to create a blog. And you want to store the blog content with MongoDB. Each blog has a title, content, and comments.

Your first schema might look like this:

const blogPostSchema = new Schema({
  title: String,
  content: String,
  comments: [{
    comment: String
  }]
})

module.exports = mongoose.model('BlogPost', blogPostSchema)

There's a problem with this schema.

A blog post can have an unlimited number of comments. If a blog post explodes in popularity and comments swell up, the document might exceed the 16MB limit imposed by MongoDB.

This means we should not embed comments in blog posts. We should create a separate collection for comments.

const comments = new Schema({
  comment: String
})

module.exports = mongoose.model('Comment', commentSchema)

In Mongoose, we can link up the two models with Population.

To use Population, we need to:

1. Set type of a property to Schema.Types.ObjectId
2. Set ref to the model we want to link too.

Here, we want comments in blogPostSchema to link to the Comment collection. This is the schema we'll use:

const blogPostSchema = new Schema({
  title: String,
  content: String,
  comments: [{ type: Schema.Types.ObjectId, ref: 'Comment' }]
})

module.exports = mongoose.model('BlogPost', blogPostSchema)

Creating a blog post

Let's say you want to create a blog post. To create the blog post, you use new BlogPost.

const blogPost = new BlogPost({
  title: 'Weather',
  content: `How's the weather today?`
})

A blog post can have zero comments. We can save this blog post with save.

const doc = await blogPost.save()
console.log(doc)

Creating comments

Now let's say we want to create a comment for the blog post. To do this, we create and save the comment.

const comment = new Comment({
  comment: `It's damn hot today`
})

const savedComment = await comment.save()
console.log(savedComment)

Notice the saved comment has an _id attribute. We need to add this _id attribute into the blog post's comments array. This creates the link.

// Saves comment to Database
const savedComment = await comment.save()

// Adds comment to blog post
// Then saves blog post to database
const blogPost = await BlogPost.findOne({ title: 'Weather' })
blogPost.comments.push(savedComment._id)
const savedPost = await blogPost.save()
console.log(savedPost)

Searching blog posts and their comments

If you tried to search for the blog post, you'll see the blog post has an array of comment IDs.

const blogPost = await BlogPost.findOne({ title: 'Weather' })
console.log(blogPost)

There are four ways to get comments.

1. Mongoose population
2. Manual way #1
3. Manual way #2
4. Manual way #3

Mongoose Population

Mongoose allows you to fetch linked documents with the populate method. What you need to do is call .populate when you execute with findOne.

When you call populate, you need to pass in the key of the property you want to populate. In this case, the key is comments. (Note: Mongoose calls this key a "path").

const blogPost = await BlogPost.findOne({ title: 'Weather' })
  .populate('comments')
console.log(blogPost)

Manual way (method 1)

Without Mongoose Populate, you need to find the comments manually. First, you need to get the array of comments.

const blogPost = await BlogPost.findOne({ title: 'Weather' })
  .populate('comments')
const commentIDs = blogPost.comments

Then, you loop through commentIDs to find each comment. If you go with this method, it's slightly faster to use Promise.all.

const commentPromises = commentIDs.map(_id => {
  return Comment.findOne({ _id })
})
const comments = await Promise.all(commentPromises)
console.log(comments)

Manual way (method 2)

Mongoose gives you an $in operator. You can use this $in operator to find all comments within an array. This syntax takes a little effort to get used to.

If I had to do the manual way, I'd prefer Manual #1 over this.

const commentIDs = blogPost.comments
const comments = await Comment.find({
    '_id': { $in: commentIDs }
})

console.log(comments)

Manual way (method 3)

For the third method, we need to change the schema. When we save a comment, we link the comment to the blog post.

// Linking comments to blog post
const commentSchema = new Schema({
  comment: String
  blogPost: [{ type: Schema.Types.ObjectId, ref: 'BlogPost' }]
})

module.exports = mongoose.model('Comment', commentSchema)

You need to save the comment into the blog post, and the blog post id into the comment.

const blogPost = await BlogPost.findOne({ title: 'Weather' })

// Saves comment
const comment = new Comment({
  comment: `It's damn hot today`,
  blogPost: blogPost._id
})
const savedComment = comment.save()

// Links blog post to comment
blogPost.comments.push(savedComment._id)
await blogPost.save()

Once you do this, you can search the Comments collection for comments that match your blog post's id.

// Searches for comments
const blogPost = await BlogPost.findOne({ title: 'Weather' })
const comments = await Comment.find({ _id: blogPost._id })
console.log(comments)

I'd prefer Manual #3 over Manual #1 and Manual #2.

And Population beats all three manual methods.

Quick Summary

You learned to use Mongoose on three different levels in this article:

1. Basic Mongoose
2. Mongoose subdocuments
3. Mongoose population

That's it!

Thanks for reading. This article was originally posted on my blog. Sign up for my newsletter if you want more articles to help you become a better frontend developer.

Are you building a full-stack app and want to let users upload an image but you’re not sure how? In my experience, this is always achieved by having users input a link and saving this string to your database. This works great and is quick and easy, but it’s also kind of cheating. What kind of app makes you first go to another site and upload your image, and then come back and link to it?

So, what’s the solution?

Allow the user to input a file, then on your server, upload this file to a cloud service and save this in your database. Cloudinary is great for this. It’s dedicated to media uploads. It has great documentation. It allows for transformations. And has a huge free plan (10 GB storage). You can sign up for Cloudinary here (I get nothing for this).

Let’s get started on the front-end

<form action='/api/images' method="post" enctype="multipart/form-data">
  <input type='file' name='image' />
</form>

This should look familiar. All you need is a form which will submit the information to the server. Enctype is required for submitting files to the server.

That’s the front-end solved.

The back-end

Now, the back-end is where all the magic happens. You will need all the usual dependencies for working with Express and Mongoose. In addition, we will utilise Multer, Cloudinary, and multer-storage-cloudinary. Multer will allow access to files submitted through the form. Cloudinary is used for configuration and uploading. multer-storage-cloudinary will make the process of combining these easy.

const multer = require("multer");
const cloudinary = require("cloudinary");
const cloudinaryStorage = require("multer-storage-cloudinary");

Once the dependencies are required you need to configure them. When you sign up to Cloudinary, you will be provided your API credentials. I recommend storing these in a “.env” file to keep them secure.

Below we are also:

• setting a folder to keep all the images organised on Cloudinary for this project
• ensuring only “.jpg” and “.png” files are uploaded
• adding a transformation to keep the height and width consistent and to manage file size.

There’s a lot more you can do in regards to transformations — you can take a look here if you are interested.

cloudinary.config({
cloud_name: process.env.CLOUD_NAME,
api_key: process.env.API_KEY,
api_secret: process.env.API_SECRET
});
const storage = cloudinaryStorage({
cloudinary: cloudinary,
folder: "demo",
allowedFormats: ["jpg", "png"],
transformation: [{ width: 500, height: 500, crop: "limit" }]
});
const parser = multer({ storage: storage });

Now that your server is all set up to receive and process these images, we can move onto setting up the route.

In your post route, you simply add the parser we set up before as a middleware. This will take in the file, upload it to Cloudinary, and return an object with the file information. You can access this information in the request object.

I like to extract just the information I want from this, to keep my database organised. At the least, you will want:

• the URL which can be used to display the image on the front-end
• the public_id which will allow you to access and delete the image from Cloudinary.

app.post('/api/images', parser.single("image"), (req, res) => {
  console.log(req.file) // to see what is returned to you
  const image = {};
  image.url = req.file.url;
  image.id = req.file.public_id;
  Image.create(image) // save image information in database
    .then(newImage => res.json(newImage))
    .catch(err => console.log(err));
});

Your image will probably be part of a larger object in your database. The image URL and id can be saved as strings as a part of this.

*Image is an example placeholder for your database collection. Substitute it for your own.

Displaying the image

When you want to display the image on the front-end, perform a database query, and then use the URL inside your image tags <img src=imageURL />.

I hope this will help you in adding that little extra to your websites. It’s not that difficult once you break down each step in the process. It will give your website the professional touch and will make it stand out.

If you have any questions, please ask in the comments.

Introducing schm, a functional and highly composable library for creating schemas in JavaScript and Node.js

_Background photo by [Willi Heidelbach](https://www.flickr.com/photos/wilhei/" rel="noopener" target="blank" title="Vá para a galeria de Willi Heidelbach)

I’ve been working with HTML, CSS, and JavaScript since 2002. The first time I needed some sort of schema in JavaScript was just a few years ago.

After using many different libraries and even authoring one and another, I decided to create schm. That’s the result of all my experience with schemas in JavaScript.

What is schm?

schm is a group of npm packages to help developers deal with schemas in JavaScript and Node.js.

It’s highly inspired by Mongoose Schemas. Actually, they’re so similar that you can use schm parameters within Mongoose Schemas and vice-versa. It's not MongoDB specific, though. You can use it for everything in JavaScript.

What kind of problems does schm solve?

? Parsing and validation of form values

On the client, you can use schemas to define models for HTML forms. It makes it easier to transform and validate values. Also, if you’re using Node.js on the server, you can use the same schema. The result is a consistent behavior between client and server validations.

? Parsing and validation of query string

Consider the following query string: /?term=John&page=2&limit=10 . By combining packages such as schm-koa, schm-express and/or schm-mongo, you will be able to parse and validate query strings with search terms and pagination with ease.

☊ Communication between client and server

If you have an app that consumes resources from a REST API, for example, you can use schemas to define, on the client, the object structure your client expects to receive from the server. If something changes on the server (properties have been renamed, for example), you can just update your schema so your entire application will continue to work.

Creating a schema

A simple schema is just a map of keys and types:

That’s the same as using a type property:

A schema can also be a map between keys and default values. Types will be automatically inferred:

That’s equivalent to doing the following:

To learn more about how to write schemas, take a look at Mongoose Schemas.

Parsing values

After defining the schema, you can use it to parse values. This process will convert values to the proper types, as well as applying other parsers defined in the schema. These are the available parsers: type, default , set , get , trim, uppercase , lowercase .

The output will be something like this:

{  name: "HAZ",  birthdate: Tue Apr 10 1990 00:00:00 GMT,}

Validating values

Just like in Mongoose Schemas, you can define validation rules within your schemas. These are the available validators: validate , required , match , enum , max , min , maxlength , minlength .

You can also create custom parsers and validators by extending the schema. We’ll talk about it later in this article.

The validation error will be, by default, an array of objects describing the errors.

[  {    message: "age must be greater than or equal 18",    min: 18,    param: "age",    validator: "min",    value: 17,  }]

If the validation passes, it will return the parsed values, just like parse().

Composing multiple schemas

Say you have separate schemas describing a body, identity and other things, and want to compose them to build a human schema. That's as easy as it sounds:

Another way to compose schemas is through nesting. A schema can be used as a type within another schema:

Extending schemas

This is the part where schm really shines. You can add custom parsers and validators or even replace the default behavior of parse and validate methods by creating schema groups.

A schema group is a function that receives the previous schema as the only argument. Besides previous params , parsers and validators , the schema object has a merge method, which is useful for schema group functions to merge new functionality into the previous schemas.

The output of the above snippet will be something like this:

{  name: "Haz!!!",  age: 27,}

If you want to go further and learn more about how to create custom parsers, take a look at how parsers are written inside the core library here.

By extending schemas, we can create many kinds of things. That’s how most of the schm satellite libraries, such as schm-translate, schm-computed and schm-mongo, are written.

We’re going to talk about one of them now.

Renaming values keys

schm-translate is one of the simplest, yet powerful, satellite libraries of schm . It's a few more than 10 lines of code compressed into one function which lets you translate values keys to your schema keys.

Say you’re working on a webapp that consumes resources from a REST API. Suddenly, developers change things on the API, which makes the response body return a slightly different model than the one the client expected. Instead of an email property, it returns now an array of emails .

This will probably make your app to break. If you don’t have a schema or any other centralized way to handle that object, you will need to update every part of the application to conform to the server changes.

With schm and schm-translate , it can be solved by changing a few lines of code in just one place:

The output will be exactly the one your app expected before the change:

{  name: "Haz",  email: "hazdiego@gmail.com",}

Click here to see the list of all packages

How is this different from other schema libraries?

A common question is the difference between schm and other libraries, such as Joi and ajv (which follows JSON Schema spec).

Comparing to ajv, schm doesn't follow any particular spec. Instead, it tries to mimic the Mongoose Schema API. Also, even though ajv has some parsing features, they're limited to default values and type coercion.

In schm, the ability to parse values based on the schema is what makes it possible to transform a query string into a MongoDB query, for example.

That said, both Joi and ajv can be combined with schm . You can easily extend it to use a different validation method:

Thank you for reading this!

If you like it and find it useful, here are some things you can do to show your support:

• Hit the clap ? button on this article a few times (up to 50)
• Give a star ⭐️ on GitHub: https://github.com/diegohaz/schm
• Follow me on GitHub: https://github.com/diegohaz
• Follow me on Twitter: https://twitter.com/diegohaz

Mongoose is an Object Data Modeling (ODM) library for MongoDB and Node.js. It manages relationships between data, provides schema validation, and is used to translate between objects in code and the representation of those objects in MongoDB.

Object Mapping between Node and MongoDB managed via Mongoose

MongoDB is a schema-less NoSQL document database. It means you can store JSON documents in it, and the structure of these documents can vary as it is not enforced like SQL databases. This is one of the advantages of using NoSQL as it speeds up application development and reduces the complexity of deployments.

Below is an example of how data is stored in Mongo vs. SQL Database:

NoSQL Documents vs. Relational Tables in SQL

Terminologies

Collections

Collections in Mongo are equivalent to tables in relational databases. They can hold multiple JSON documents.

Documents

Documents are equivalent to records or rows of data in SQL. While a SQL row can reference data in other tables, Mongo documents usually combine that in a document.

Fields

Fields, also known as properties or attributes, are similar to columns in a SQL table. In the image above, FirstName, LastName, Email, and Phone are all fields.

Schema

While Mongo is schema-less, SQL defines a schema via the table definition. A Mongoose schema is a document data structure (or shape of the document) that is enforced via the application layer.

SchemaTypes

While Mongoose schemas define the overall structure or shape of a document, SchemaTypes define the expected data type for individual fields (String, Number, Boolean, and so on).

You can also pass in useful options like required to make a field non-optional, default to set a default value for the field, and many more.

Models

Models are higher-order constructors that take a schema and create an instance of a document equivalent to records in a relational database.

Example

Here's a small code snippet to illustrate some of the terminology above:

const puppySchema = new mongoose.Schema({
  name: {
    type: String,
    required: true
  },
  age: Number
});

const Puppy = mongoose.model('Puppy', puppySchema);

In the code above, puppySchema defines the shape of the document which has two fields, name, and age.

The SchemaType for name is String and for age is Number. Note that you can define the SchemaType for a field by using an object with a type property like with name. Or you can apply a SchemaType directly to the field like with age.

Also, notice that the SchemaType for name has the option required set to true. To use options like required and lowercase for a field, you need to use an object to set the SchemaType.

At the bottom of the snippet, puppySchema is compiled into a model named Puppy, which can then be used to construct documents in an application.

Getting Started

Mongo Installation

Before we get started, let’s setup Mongo. You can choose from one of the following options (we are using option #1 for this article):

1. Download the appropriate MongoDB version for your Operating System from the MongoDB Website and follow their installation instructions
2. Create a free sandbox database subscription on mLab
3. Install Mongo using Docker if you prefer to use Docker

Let’s navigate through some of the basics of Mongoose by implementing a model that represents data for a simplified address book.

I am using Visual Studio Code, Node 8.9, and NPM 5.6. Fire up your favorite IDE, create a blank project, and let’s get started! We will be using the limited ES6 syntax in Node, so we won’t be configuring Babel.

NPM Install

Let’s go to the project folder and initialize our project

npm init -y

Let’s install Mongoose and a validation library with the following command:

npm install mongoose validator

The above install command will install the latest version of the libraries. The Mongoose syntax in this article is specific to Mongoose v5 and beyond.

Database Connection

Create a file ./src/database.js under the project root.

Next, we will add a simple class with a method that connects to the database.

Your connection string will vary based on your installation.

let mongoose = require('mongoose');

const server = '127.0.0.1:27017'; // REPLACE WITH YOUR DB SERVER
const database = 'fcc-Mail'; // REPLACE WITH YOUR DB NAME

class Database {
  constructor() {
    this._connect();
  }

  _connect() {
    mongoose
      .connect(`mongodb://${server}/${database}`)
      .then(() => {
        console.log('Database connection successful');
      })
      .catch((err) => {
        console.error('Database connection error');
      });
  }
}

module.exports = new Database();

The require(‘mongoose’) call above returns a Singleton object. It means that the first time you call require(‘mongoose’), it is creating an instance of the Mongoose class and returning it. On subsequent calls, it will return the same instance that was created and returned to you the first time because of how module import/export works in ES6.

Module import/require work-flow

Similarly, we have turned our Database class into a singleton by returning an instance of the class in the module.exports statement because we only need a single connection to the database.

ES6 makes it very easy for us to create a singleton (single instance) pattern because of how the module loader works by caching the response of a previously imported file.

Mongoose Schema vs. Model

A Mongoose model is a wrapper on the Mongoose schema. A Mongoose schema defines the structure of the document, default values, validators, etc., whereas a Mongoose model provides an interface to the database for creating, querying, updating, deleting records, etc.

Creating a Mongoose model comprises primarily of three parts:

1. Referencing Mongoose

let mongoose = require('mongoose');

This reference will be the same as the one that was returned when we connected to the database, which means the schema and model definitions will not need to explicitly connect to the database.

2. Defining the Schema

A schema defines document properties through an object where the key name corresponds to the property name in the collection.

let emailSchema = new mongoose.Schema({
  email: String
});

Here we define a property called email with a schema type String which maps to an internal validator that will be triggered when the model is saved to the database. It will fail if the data type of the value is not a string type.

The following Schema Types are permitted:

• Array
• Boolean
• Buffer
• Date
• Mixed (A generic / flexible data type)
• Number
• ObjectId
• String

Mixed and ObjectId are defined under require(‘mongoose’).Schema.Types.

3. Exporting a Model

We need to call the model constructor on the Mongoose instance and pass it the name of the collection and a reference to the schema definition.

module.exports = mongoose.model('Email', emailSchema);

Let’s combine the above code into ./src/models/email.js to define the contents of a basic email model:

let mongoose = require('mongoose');

let emailSchema = new mongoose.Schema({
  email: String
});

module.exports = mongoose.model('Email', emailSchema);

A schema definition should be simple, but its complexity is usually based on application requirements. Schemas can be reused and they can contain several child-schemas too. In the example above, the value of the email property is a simple value type. However, it can also be an object type with additional properties on it.

We can create an instance of the model we defined above and populate it using the following syntax:

let EmailModel = require('./email');

let msg = new EmailModel({
  email: 'ada.lovelace@gmail.com'
});

Let’s enhance the Email schema to make the email property a unique, required field and convert the value to lowercase before saving it. We can also add a validation function that will ensure that the value is a valid email address. We will reference and use the validator library installed earlier.

let mongoose = require('mongoose');
let validator = require('validator');

let emailSchema = new mongoose.Schema({
  email: {
    type: String,
    required: true,
    unique: true,
    lowercase: true,
    validate: (value) => {
      return validator.isEmail(value);
    }
  }
});

module.exports = mongoose.model('Email', emailSchema);

Basic Operations

Mongoose has a flexible API and provides many ways to accomplish a task. We will not focus on the variations because that is out of scope for this article, but remember that most of the operations can be done in more than one way either syntactically or via the application architecture.

Create Record

Let’s create an instance of the email model and save it to the database:

let EmailModel = require('./email');

let msg = new EmailModel({
  email: 'ADA.LOVELACE@GMAIL.COM'
});

msg
  .save()
  .then((doc) => {
    console.log(doc);
  })
  .catch((err) => {
    console.error(err);
  });

The result is a document that is returned upon a successful save:

{ 
  _id: 5a78fe3e2f44ba8f85a2409a,
  email: 'ada.lovelace@gmail.com',
  __v: 0 
}

The following fields are returned (internal fields are prefixed with an underscore):

1. The _id field is auto-generated by Mongo and is a primary key of the collection. Its value is a unique identifier for the document.
2. The value of the email field is returned. Notice that it is lower-cased because we specified the lowercase: true attribute in the schema.
3. __v is the versionKey property set on each document when first created by Mongoose. Its value contains the internal revision of the document.

If you try to repeat the save operation above, you will get an error because we have specified that the email field should be unique.

Fetch Record

Let’s try to retrieve the record we saved to the database earlier. The model class exposes several static and instance methods to perform operations on the database. We will now try to find the record that we created previously using the find method and pass the email as the search term.

EmailModel.find({
  email: 'ada.lovelace@gmail.com' // search query
})
  .then((doc) => {
    console.log(doc);
  })
  .catch((err) => {
    console.error(err);
  });

The document returned will be similar to what was displayed when we created the record:

{ 
  _id: 5a78fe3e2f44ba8f85a2409a,
  email: 'ada.lovelace@gmail.com',
  __v: 0 
}

Update Record

Let’s modify the record above by changing the email address and adding another field to it, all in a single operation. For performance reasons, Mongoose won’t return the updated document so we need to pass an additional parameter to ask for it:

EmailModel.findOneAndUpdate(
  {
    email: 'ada.lovelace@gmail.com' // search query
  },
  {
    email: 'theoutlander@live.com' // field:values to update
  },
  {
    new: true, // return updated doc
    runValidators: true // validate before update
  }
)
  .then((doc) => {
    console.log(doc);
  })
  .catch((err) => {
    console.error(err);
  });

The document returned will contain the updated email:

{ 
  _id: 5a78fe3e2f44ba8f85a2409a,
  email: 'theoutlander@live.com',
  __v: 0 
}

Delete Record

We will use the findOneAndRemove call to delete a record. It returns the original document that was removed:

EmailModel.findOneAndRemove({
  email: 'theoutlander@live.com'
})
  .then((response) => {
    console.log(response);
  })
  .catch((err) => {
    console.error(err);
  });

Helpers

We have looked at some of the basic functionality above known as CRUD (Create, Read, Update, Delete) operations, but Mongoose also provides the ability to configure several types of helper methods and properties. These can be used to further simplify working with data.

Let’s create a user schema in ./src/models/user.js with the fieldsfirstName and lastName:

let mongoose = require('mongoose');

let userSchema = new mongoose.Schema({
  firstName: String,
  lastName: String
});

module.exports = mongoose.model('User', userSchema);

Virtual Property

A virtual property is not persisted to the database. We can add it to our schema as a helper to get and set values.

Let’s create a virtual property called fullName which can be used to set values on firstName and lastName and retrieve them as a combined value when read:

userSchema.virtual('fullName').get(function () {
  return this.firstName + ' ' + this.lastName;
});

userSchema.virtual('fullName').set(function (name) {
  let str = name.split(' ');

  this.firstName = str[0];
  this.lastName = str[1];
});

Callbacks for get and set must use the function keyword as we need to access the model via the this keyword. Using fat arrow functions will change what this refers to.

Now, we can set firstName and lastName by assigning a value to fullName:

let model = new UserModel();

model.fullName = 'Thomas Anderson';

console.log(model.toJSON()); // Output model fields as JSON
console.log();
console.log(model.fullName); // Output the full name

The code above will output the following:

{ _id: 5a7a4248550ebb9fafd898cf,
  firstName: 'Thomas',
  lastName: 'Anderson' }

Thomas Anderson

Instance Methods

We can create custom helper methods on the schema and access them via the model instance. These methods will have access to the model object and they can be used quite creatively. For instance, we could create a method to find all the people who have the same first name as the current instance.

In this example, let’s create a function to return the initials for the current user. Let’s add a custom helper method called getInitials to the schema:

userSchema.methods.getInitials = function () {
  return this.firstName[0] + this.lastName[0];
};

This method will be accessible via a model instance:

let model = new UserModel({
  firstName: 'Thomas',
  lastName: 'Anderson'
});

let initials = model.getInitials();

console.log(initials); // This will output: TA

Static Methods

Similar to instance methods, we can create static methods on the schema. Let’s create a method to retrieve all users in the database:

userSchema.statics.getUsers = function () {
  return new Promise((resolve, reject) => {
    this.find((err, docs) => {
      if (err) {
        console.error(err);
        return reject(err);
      }

      resolve(docs);
    });
  });
};

Calling getUsers on the Model class will return all the users in the database:

UserModel.getUsers()
  .then((docs) => {
    console.log(docs);
  })
  .catch((err) => {
    console.error(err);
  });

Adding instance and static methods is a nice approach to implement an interface to database interactions on collections and records.

Middleware

Middleware are functions that run at specific stages of a pipeline. Mongoose supports middleware for the following operations:

• Aggregate
• Document
• Model
• Query

For instance, models have pre and post functions that take two parameters:

1. Type of event (‘init’, ‘validate’, ‘save’, ‘remove’)
2. A callback that is executed with this referencing the model instance

Example of Middleware (a.k.a. pre and post hooks)

Let’s try an example by adding two fields called createdAt and updatedAt to our schema:

let mongoose = require('mongoose');

let userSchema = new mongoose.Schema({
  firstName: String,
  lastName: String,
  createdAt: Date,
  updatedAt: Date
});

module.exports = mongoose.model('User', userSchema);

When model.save() is called, there is a pre(‘save’, …) and post(‘save’, …) event that is triggered. For the second parameter, you can pass a function that is called when the event is triggered. These functions take a parameter to the next function in the middleware chain.

Let’s add a pre-save hook and set values for createdAt and updatedAt:

userSchema.pre('save', function (next) {
  let now = Date.now();

  this.updatedAt = now;
  // Set a value for createdAt only if it is null
  if (!this.createdAt) {
    this.createdAt = now;
  }

  // Call the next function in the pre-save chain
  next();
});

Let’s create and save our model:

let UserModel = require('./user');

let model = new UserModel({
  fullName: 'Thomas Anderson'
});

msg
  .save()
  .then((doc) => {
    console.log(doc);
  })
  .catch((err) => {
    console.error(err);
  });

You should see values for createdAt and updatedAt when the record that is created is printed:

{ _id: 5a7bbbeebc3b49cb919da675,
  firstName: 'Thomas',
  lastName: 'Anderson',
  updatedAt: 2018-02-08T02:54:38.888Z,
  createdAt: 2018-02-08T02:54:38.888Z,
  __v: 0 }

Plugins

Suppose that we want to track when a record was created and last updated on every collection in our database. Instead of repeating the above process, we can create a plugin and apply it to every schema.

Let’s create a file ./src/model/plugins/timestamp.js and replicate the above functionality as a reusable module:

module.exports = function timestamp(schema) {
  // Add the two fields to the schema
  schema.add({
    createdAt: Date,
    updatedAt: Date
  });

  // Create a pre-save hook
  schema.pre('save', function (next) {
    let now = Date.now();

    this.updatedAt = now;
    // Set a value for createdAt only if it is null
    if (!this.createdAt) {
      this.createdAt = now;
    }
    // Call the next function in the pre-save chain
    next();
  });
};

To use this plugin, we simply pass it to the schemas that should be given this functionality:

let timestampPlugin = require('./plugins/timestamp');

emailSchema.plugin(timestampPlugin);
userSchema.plugin(timestampPlugin);

Query Building

Mongoose has a very rich API that handles many complex operations supported by MongoDB. Consider a query where we can incrementally build query components.

In this example, we are going to:

1. Find all users
2. Skip the first 100 records
3. Limit the results to 10 records
4. Sort the results by the firstName field
5. Select the firstName
6. Execute that query

UserModel.find()               // find all users
  .skip(100)                   // skip the first 100 items
  .limit(10)                   // limit to 10 items
  .sort({ firstName: 1 })      // sort ascending by firstName
  .select({ firstName: true }) // select firstName only
  .exec()                      // execute the query
  .then((docs) => {
    console.log(docs);
  })
  .catch((err) => {
    console.error(err);
  });

Closing

We have barely scratched the surface exploring some of the capabilities of Mongoose. It is a rich library full of useful and and powerful features that make it a joy to work with data models in the application layer.

While you can interact with Mongo directly using Mongo Driver, Mongoose will simplify that interaction by allowing you to model relationships between data and validate them easily.

Fun Fact: Mongoose is created by Valeri Karpov who is an incredibly talented engineer! He coined the term The MEAN Stack.

If this article was helpful, follow me on Twitter.

_[How to Build a REST API with Node | Express | Mongo](https://twitter.com/intent/follow?screen_name=theoutlander" rel="noopener" target="_blank" title="">You may also like my workshop on youtube: <a href="https://youtu.be/egeHq-lYyxo" rel="noopener" target="blank" title=")