Well, you can save your time and energy by writing tests for your application that run whenever you add a new feature, all without leaving your IDE during development. This will help you be confident that each feature works as expected.

In this guide, I'll help you build confidence through code: you'll learn how to validate APIs with tests first, then confirm the results in Postman or any other API testing tool.

Prerequisites

• Node.js & Express basics: You should have a working knowledge of Node.js and Express (or a similar library like fastify), including how to build and run a simple API.

• Working knowledge of TypeScript: The code snippets in this guide are written using TypeScript, so you should have a solid understanding of TypeScript basics.

• Basic familiarity with MongoDB: Helpful but not required. The examples in this guide use MongoDB for demonstration purposes, but the underlying logic applies to any database or data layer. Only the tooling differs.

• Hands-on API experience: Prior experience writing at least one backend API with Express will help you follow along more effectively.

• Curiosity and motivation: A willingness to deepen your backend skills by learning how to write and run tests for your APIs.

• Environment setup: Node.js version 20 or higher installed on your machine.

Table of Contents

• Prerequisites

• Testing Concept

• Unit testing

• Comparison between unit and integration tests

• Summary

You can find all the code snippets used in this guide in this Git repository. Each example has its own branch, and they're combined into a main branch if you want a full example version. Please consider starring the repository if you find it helpful.

Key Testing Concepts

Testing helps you build confidence in your codebase. You wrie code that tests other code in your application, or code that checks if a function or feature behaves the way that it should.

In this guide, we'll cover two types of testing:

• Unit testing: This is the most basic type of testing and, in my opinion, the easiest. You test a single piece of code in isolation to see how it behaves.

• Integration testing: This approach is used to test how different parts of the application are integrated to make sure they're working together as intended.

Key Testing Terms

Throughout this guide, I'll be using some technical terms are related to testing which I want to explain up front:

• Mocking: mocking is a technique used to make a fake implementation of a real function call.

• Spying: spying is a way of inspecting a function to see, for example, if the function is called with a certain type of argument or how many times it's been called.

• Assertion: assertions check to see if the output you're getting matches the expected output.

Alright now that we have that covered, let's go over some basics of testing so you have the best practices down before we start writing tests.

How to Name a Test File

Make sure your test files follow one of these naming conventions:

• filename.test.ts

• filename.spec.ts

• filename.test.js

• filename.spec.js

Both the spec and test keyword in a file name makes it possible to run the test file. They also help testing frameworks run the right file.

You can choose either the TypeScript or JavaScript extension on a file based on which language you're using to write the test. For this guide I'm using TypeScript so I'll be using the TypeScript (.ts) test file version.

Parts of a Testing File

Typically, a test file will have four main parts that you should be familiar with, which are:

• describe: Used to describe which test you're going to write.

• it: Used to specify a condition that a function must pass when tested against.

• expect: Used to determine what type of results you're expecting when you call a function that's being tested

• matchers: These are different method available on the expect keyword that we use to evaluate if the function we're testing returns a value that meets the expected data or value.

Example of a test file:

import { describe, it, expect } from "vitest";
import validateEmail from "../utils/email-validation";

describe("email validation test suites", () => {
  it("must define email validation function", () => {
    expect(validateEmail).toBeDefined();
  });
});

The above snippet tests if the validateEmail function is defined.

In this test file:

• We use describe to specify the description of the test. describe receives a string description of the test and a function to handle different test cases.

• The it keyword specifies and defines a test for a function you're testing, it receives a string describing a specific test description and a callback function to execute and handle test assertion.

• Then expect uses the function return type to check if it matches a specific condition through the toBeDefined matcher method.

List of Matchers:

There are many matchers available to you. Below are a few of them:

• toBe: compares the passed-in value to see if it matches the function's returned value. It's used on primitive data types like strings, numbers, and so on.

• toEqual: compares the passed-in value to see if it matches the function's returned value. It's used on non-primitive data types like objects, arrays, and so on.

• toThrow: used on a function that threw an error to check if the function threw expected error object or instance.

• toBeCalledWith: used to check if a function is called with a given parameter.

• toBecalledOnce: used to check if a function is called only once.

• toBeDefined: used to check if a function is defined.

• toBeUndefined: used to check if a function returns an undefined value.

• toBeTruthy: used to check if a function returns a true Boolean value.

• toBeFalsy: used to check if a function returns a false Boolean value.

These are few of the many matchers out there.

Vitest Installation

Now that you know some testing basics, we can get into the actual tests. We'll start by installing vitest, the framework that we'll use to run and write tests for our application.

You can choose your preferred package manager to use to install vitest from the list below:

pnpm add -D vitest # for pnpm package manager
npm install -D vitest # for npm package manager
yarn add -D vitest # for yarn package manager
bun add -D vitest # for bun package manager

Unit Testing

A unit test focuses on testing a small piece of code in isolation in your application. A simple example could be if you have a function that adds contact info to a database. For the test, you could check if the email is valid before adding the contact to the database.

Let's start by testing a simple email validation function so you can get comfortable with how unit tests works and how to write one:

export default function validateEmail(email: string) {
  const regex = /^[^\s@]+@[^\s@]+\.[^\s@]+$/;

  if (regex.test(email)) {
    return true;
  } else {
    throw new Error("Invalid email format");
  }
}

The above snippet exports a function that receives an email and then uses regex to validate if the email is valid. It throws an error if the email is invalid.

Tests for the validateEmail Function

Let's start by checking if validateEmail returns true for correct emails:

import { describe, it, expect } from "vitest";
import validateEmail from "../utils/email-validation";

describe("email validation test suites", () => {

  it("returns true for valid email", () => {
    const sampleEmail = "arnoldjabo@gmail.com";
    expect(validateEmail(sampleEmail)).toBeTruthy();
  });

});

In the above test, we're creating a variable sampleEmail to be used as a sample email in the validateEmail function. Save this and then run npx vitest in your terminal. You should see a terminal with the results of your test. It should look like the below screenshot:

When you run tests, Vitest shows the list of test files you're testing, how many tests were executed, and how many passed and failed tests you have.

Let's create another test that detects an invalid email:

import { describe, it, expect } from "vitest";
import validateEmail from "../utils/email-validation";

describe("email validation test suites", () => {

    it("throws error for invalid email", () => {
    const sampleEmail = "verymasd.com";
    const invalidEmailResults = () => validateEmail(sampleEmail);
    expect(invalidEmailResults).toThrow("Invalid email format");
  });

});

For functions that throw errors, you need to wrap them inside another function to prevent them from stopping the test before the test reaches the assertion or expect section.

In our example above, the validateEmail function is wrapped inside another function which will hold whatever the error validateEmail throws is. It then assigns it to the invalidEmailResults variable. Next we use the toThrow matcher on the expect assertion to match a type of error validateEmail expects to be thrown for an invalid email.

If your run the test, you'll have two passed tests now:

If you didn't wrap the validateEmail function inside another function when it throws an error, you'll see something like this when you run the test:

As you can see, the error fires before reaching the test final assertion. So when you have a function that throws errors, remember to wrap your function inside another function to avoid throwing errors mid-test.

Tip: When writing unit tests, focus on the expected input and output of a function. You don't have to worry or need to think about the implementation of the function: the input and output are the key elements here.

Testing API Calls

By now you should understand how a unit test works. So next, let's see how you can unit test an API that makes a call to a database.

import Contacts from "../../schema/contactList";
import { Request, Response } from "express";
import validateEmail from "../../utils/email-validation";

async function addContacts(req: Request, res: Response) {
  const { contactName, phoneNumber, email } = req.body;
  validateEmail(email);

  const contact = await Contacts.create({ contactName, phoneNumber, email });

  return res.status(201).json({ message: `successfully created ${contact.contactName}` });
}

export default addContacts;

In the code snippet above, we have a function that creates a contact with the database. It also validates if the passed email is valid.

In this example, we're using mongo DB for the database and Mongoose for connecting the codebase with our mongo DB instance.

Unit tests for non-pure functions (that is, functions that are dependent on external services, like making API calls or calling other functions) are tested a bit differently.

For these cases, you create a mock or fake version of the original function that makes a call to the external service and then define its behavior to match the expected return value that you'd have if you used its real version.

We'll start by mocking the implementation of the email validation function. This will help solidify the understanding on how mocking works in unit testing and testing in general.

Note: mocking the validateEmail function isn't that important because it doesn't make a big difference from using the email validation function directly here. But for learning purposes, we'll mock it to help you understand how it works.

When mocking the modules import, we use the vi.mock function which helps transform the imports of a given module into mocks or fake versions of the real ones.

vi.mock(filepath,callback);

vi.mock receives two arguments: the file path location of the module you want to mock, and a callback function called a factory function which we'll use to transform the module imports into mocks.

Let's start by mocking the validateEmail path and creating a factory function to transform the module into mocks:

vi.mock("../utils/email-validation", () => {
  return { default: vi.fn()};
});
import validateEmail from "../utils/email-validation";

In the callback function (factory function), we then return an object of the exported module, with a key of default and a value of vi.fn. For function mocking we use vi.fn() which automatically replaces the function's return value with undefined.

We use default as a key because the validateEmail function is exported as a default export. If it was a named export, we would have used the actual export name instead of default in the return object.

vi.mock("../utils/email-validation", () => {
  return { validateEmail: vi.fn() };
});
import { validateEmail } from "../utils/email-validation";

Always import your module after the mock module operation to avoid using the real module.

There are methods on vi.fn() that help define the implementation and behaviors of the mocked function. Some of these methods include:

• mockReturnValue: Used to define a return value for a mocked function

• mockRejectsValue: Used for promise-based functions to define the error the function will return.

• mockResolveValue: Used for promise-based functions to define the data the function will return.

• mockImplementation: Used to define a new function behavior of a mocked function.

• mockReturnThis: Used to return the actual instance of a function you're mocking.

These are the methods that you'll likely use most of the time when defining mock implementation and setting mock return value. Just keep in mind that there are many others.

Here, for emailValidate, we'll be using the mockReturnValue and mockImplementation methods.

Let's use mockReturnValue to make the validateEmail function return true by default, assuming the email will be formatted correctly:

vi.mock("../utils/email-validation", () => {
  return { default: vi.fn().mockReturnValue(true) };
});
import validateEmail from "../utils/email-validation";

We define a mock function with vi.fn and then chain on the mockReturnValue(true) to change the mock function default return value (undefined) to true in our case.

Defining the Mocking Implementation

You can do a lot with a mocked function, like defining a new implementation for the mocked function that replaces existing logic in the original function.

Let's create a test suite with a fake email that throws an error when you pass the wrong email while creating contacts.

import { describe, expect, it, vi } from "vitest";
import mockinggoose from "mockingoose";
import Contacts from "../schema/contactList";
import addContacts from "../src/controllers/add-contacts.controller";

import { type Response, type Request } from "express";

vi.mock("../utils/email-validation", () => {
  return { default: vi.fn().mockReturnValue(true) };
});

import validateEmail from "../utils/email-validation";

const fakeContact = {
  contactName: "arnold",
  phoneNumber: 798600102,
  email: "arnoldjabo@gmail.com",
};

describe("Add contacts to the database", async () => {
  it("throws error for the wrong email address", async () => {
    const req = {
      body: { ...fakeContact, email: "fakemail" },
    } as Request;  

    const res = {
      status: vi.fn().mockReturnThis(),
      json: vi.fn(),
    } as any as Response;

    (validateEmail as ReturnType<typeof vi.fn>).mockImplementation(() => {
      throw new Error("invalid email!");
    });
 
    await expect(addContacts(req, res)).rejects.toThrow();
  });
});

In the snippet above, we're mocking or creating a fake request object that's cast as request type of express. We do the same with the response object – but the difference here is that with response we're also creating mocks for common methods that you'd use on an Express response (which are status and a json object).

We then turn the return type of the validateEmail function into the vitest mocking function type to avoid TypeScript warnings. Then we use the mockImplementation method to throw a new error inside validateEmail.

The assertion works differently because now we're throwing a promise-based error. We use rejects on the assertion and then chain on another matcher that stimulates which type of error the function will throw.

Tip: when working with TypeScript, the response object can't be cast like we did on the request object because the response object is much stricter than request. So you'll first need to cast it as any and then cast back to the response object. That way you avoid the TypeScript warning while still keeping the type in play for your test code.

Mocking a Mongoose Model

With unit tests, we don't want to save test data to a real database. Instead we can fake the implementation of the service that calls the database service – in our case, we can use the create method from Mongoose. It'll save the record to a Mongo database. We can then define what it should return on success (and it should look identical to what it would return if we were using a real database).

We'll start by installing a library for mocking a Mongoose model called mockingoose:

pnpm add -D mockingoose # for pnpm package manager
npm install -D mockingoose # for npm package manager
yarn add -D mockingoose # for yarn package manager
bun add -D mockingoose # for bun package manager

After installation we'll create a mock for our contacts model:

import { describe, expect, it, vi } from "vitest";
import mockinggoose from "mockingoose";
import Contacts from "../schema/contactList";

const fakeContact = {
  contactName: "arnold",
  phoneNumber: 798600102,
  email: "arnoldjabo@gmail.com",
};

describe("Add contacts to the database", async () => {
  it("successfully create a new contact to the database", async () => {
    mockinggoose(Contacts).toReturn(fakeContact, "save");
  });

});

To mock a Mongoose model, we call the mockinggoose() function and pass the model to mock. Then we use the toReturn matcher to describe what it should return, toReturn matcher expects two arguments.

Those arguments are a fake dataset for the model and a Mongo method that we'll use to work with the data. For our example we'll use save because we're creating records in the document.

Unit Testing the API

We can start by writing the first test for the add contact API call like this:

import { describe, expect, it, vi } from "vitest";
import mockinggoose from "mockingoose";
import Contacts from "../schema/contactList";
import addContacts from "../src/controllers/add-contacts.controller";
import { type Response, type Request } from "express";

vi.mock("../utils/email-validation", () => {
  return { default: vi.fn().mockReturnValue(true) };
});

import validateEmail from "../utils/email-validation";

const fakeContact = {
  contactName: "arnold",
  phoneNumber: 798600102,
  email: "arnoldjabo@gmail.com",
};

describe("Add contacts to the database", async () => {
  it("successfully create a new contact to the database", async () => {
    mockinggoose(Contacts).toReturn(fakeContact, "save");

    const req = {
      body: fakeContact,
    } as Request;

    const res = {
      status: vi.fn().mockReturnThis(),
      json: vi.fn(),
    } as any as Response;

    await addContacts(req, res);
    expect(res.status).toHaveBeenCalledWith(201);
    expect(res.json).toHaveBeenCalledWith({
      message: `successfully created ${fakeContact.contactName}`,
    });
  });
 });

For this test assertion, we're using different matchers called spies. These are used on a function to inspect how many times it's been called or which parameters were used to call it (and so on).

Here we're expecting the status function of response to be called with a status of 201 as its argument. Then the JSON object is called with a message argument that we're using to send out the response.

Integration Tests

Integration tests test the communication and integration of different parts of an application. For example, they might check if your database integrates well with the function that makes the API call to the database.

Unlike unit tests (where we don't need to have our test making API calls), with integration tests we're testing if parts of the application integrate together and works as expected. We don't need to mock anything, because we want to make sure that we're successfully sending a request to the backend and connecting to the database.

Creating Integration Test Data Storage

When you're running integration tests, there are two ways to create a testing environment that acts as a database. They include:

• Creating a duplicate schema of your real database and using the copy as a testing database environment. Whenever you're running tests, you point your database connectivity to the test DB.

• Creating in-memory database storage. This approach doesn't require you to have two separate schemas (one for testing and another for production). Instead you construct the same schema shape in your codebase memory and use it as your testing environment.

Using the first approach is complicated because you have to set up and configure which database to use for which environment. But for the second approach, you can just set up the right schema structure as the original schema and use it for testing without needing to configure it in the database and remove it after use.

For Mongo DB there's a package that simplifies the in-memory storage option for us called mongodb-memory-server. It deletes all the data that was used for testing after the tests have finished running.

How to Set Up the Environment for the Integration Tests

You'll need to install:

• supertest: a package that helps you make API calls/requests and returns back the response when testing.

• mongodb-memory-server: a package that makes in memory database storage for testing data.

# command for pnpm package manager
pnpm add -D mongodb-memory-server supertest @types/supertest

# command for npm package manager
npm install --save-dev mongodb-memory-server supertest @types/supertest

# command for yarn package manager
yarn add --dev mongodb-memory-server supertest @types/supertest

# command for bun package manager
bun add -d mongodb-memory-server supertest @types/supertest

Before moving on, we need to change the setup of our server entry file.

If you've been using Express with Node or any other framework, you may be familiar with the following type of setup for the server entry file where everything is added into a single file:

import express from "express";
import { loadEnvFile } from "node:process";
import connectToDB from "../config/dbConfig";
import addContacts from "./controllers/add-contacts.controller";

const app = express();
loadEnvFile();
async function dbConnection() {
  await connectToDB();
}
dbConnection();

app.use(express.json());
app.post("/add-contacts", addContacts);
app.listen(5000, () => console.log("the server successfully connected"));

This setup works fine and it's valid in certain cases. But when working with integration tests, it can be problematic. This is because in integration tests, we'll need an instance of Express to use when making the request. If we export the app variable here inside the main file when we make a request while testing, the production DB connection will conflict with the testing DB connection. This'll cause the tests to stop working.

The solution here is create another file, define an Express instance, and export it. Then we'll use the exported Express instance in the server to start a server. The setup looks like this:

app.ts

import express from "express";
import addContacts from "./controllers/add-contacts.controller";

const app = express()

app.use(express.json())
app.post("/add-contacts", addContacts);

export default app;

Then the main file server.ts or main.ts uses the app variable like this:

import { loadEnvFile } from "node:process";
import connectToDB from "../config/dbConfig";
import app from "./app";

loadEnvFile();

async function bootsrap() {
  await connectToDB();
  app.listen(5000, () => console.log("the server successfully connected"));
}
bootsrap();

We're importing the Express instance from the app file and then using the bootstrap() function to set up the database and start the server. With this in place, we can start writing integration tests for the addContact module.

How to Write the Integration Tests

Tip: With integration test(s) you can name your file like filename.integration.test.ts this is the most commonly used naming convention for integration tests, but it is not mandatory it just a naming convention.

You first need to set up the database testing data storage using the mongoose and mongdb-memory-server packages and the Express instance for making requests.

import { afterAll, beforeAll, describe, expect, it } from "vitest";

import { MongoMemoryServer } from "mongodb-memory-server";
import mongoose from "mongoose";
import app from "../src/app";


describe("intergration test setup for add contact api", () => {
  let mongoServer: MongoMemoryServer;
  let server: any;

  beforeAll(async () => {
    mongoServer = await MongoMemoryServer.create();
    const uri = mongoServer.getUri();
    await mongoose.connect(uri);
    server = app.listen(0);
  });

  afterAll(async () => {
    mongoServer.stop();
    mongoose.disconnect();
    server.close();
  });
});

The beforeAll and afterAll functions are vitest functions. beforeAll runs before any test starts executing and afterAll will run after all tests are done executing.

In the test, we set up the database and Express instance before any test runs.

First, we created the mongoServer variable. Then, inside the beforeAll block, we initialize it with MongoMemoryServer.create to create an in-memory database for testing data storage. We get the connection string using the uri variable using the getUri method. Finally we use Mongoose to connect to the generated in-memory connection string.

The server variable is assigned to the Express instance listening to port 0, but you can use any port number of your choice – it's just for demonstration purposes. This creates an Express instance for our testing environment.

In afterAll, after all tests have finished executing, we close the server and in-memory DB and then also disconnect our Mongoose instance.

Within the same describe block, we then add the test description and assertion (same as we did in unit testing):

import { afterAll, beforeAll, describe, expect, it } from "vitest";
import { MongoMemoryServer } from "mongodb-memory-server";
import mongoose from "mongoose";
import app from "../src/app";
import request from "supertest";
import Contacts from "../schema/contactList";

describe("intergration test for add contact api", () => {
  /*
    Here we do server setup and database in memory setup that was discussed,
    in the previous snippets for setting up integration data storage testing environment 
   */
  const fakeContact = {
    contactName: "arnold",
    phoneNumber: 798600102,
    email: "arnoldjabo@gmail.com",
  };

  describe("POST /add-contacts", () => {
    it("creates a new record to the database", async () => {
      const response = await request(app)
        .post("/add-contacts")
        .send(fakeContact);

       const contactList = await Contacts.findOne({
        email: "arnoldjabo@gmail.com",
      })!;
      expect(contactList?.email).toBe("arnoldjabo@gmail.com");
      console.log(contactList);

      expect(response.status).toBe(201);
      expect(response.body).toEqual({
        message: `successfully created ${fakeContact.contactName}`,
      });
    });
  });
});

Here in the test file, we're describing the test as a post method test for the add-contact endpoint. Then we test if it adds data to the database.

Within the it body, we use request from supertest to make a request to the server we've created. We also chain on an HTTP method with the endpoint we want to test.

For methods that send data to the backend like POST, PATCH, or PUT, we use the send() method on request to add an object of the data that we're sending.

We'll use the response to assert what the response could look like. For example, we're expecting the server to give a status code of 201 on successful data entry and a JSON object with a message property that confirms that it has added a contact.

We're using an assertion to check if the response's status matches what we expect, as well as if the response body matches the expected message we should be getting.

To test if the data are really being added to the database, I've added contactList to get the contact we just added by finding it by email. Then we log the contactList to the console to show how in-memory works. It's pretty much the same as a real Mongo DB instance. If we were to run the tests, we would have something that looks like this:

You can see from the console the in-memory stores and retrieves data as a regular Mongo database does.

When to Use Unit vs Integration Tests

So when do you use each type of test?

Use unit tests when you have pure functions like the email validation example we had earlier.

And use Integration tests for functions that makes external API calls that are dependent on external service like database calls to avoid mocking every function that you're importing.

Summary

This guide explains how two types of testing work: unit tests and integration tests.

Unit tests are code that tests specific pieces of your codebase in isolation, and are best for pure functions. Integration test are code that tests successful integration and communication between parts of your application, and they're best for non-pure functions.

If you found the article helpful, you can buy me coffee.

Coming from traditional software testing, I was used to a familiar pattern.

A requirement tells us what the system should do. We create a test case, provide an input, define an expected result, execute the test, and compare the actual result with what we expected.

For example:

Then I started learning conversational AI. Suddenly, the same approach didn't fit quite as neatly.

If I ask an AI agent "How can I reset my password?", it might answer, "You can reset your password using the Forgot Password option on the login page."

Ask the same question again and it might say: "Select Forgot Password from the sign-in screen and follow the instructions sent to your registered email."

The wording is different, but both responses may be perfectly acceptable. So how do we test something when the exact response can change?

That question changed the way I approached conversational AI testing.

In this article, I'll walk through the testing areas I found most important while learning how conversational systems behave, and show how traditional QA techniques can be adapted for AI agents.

Table of Contents

• 1. Start With Intent, Not Exact Wording

• 2. Don't Use Exact Text Matching for Every Response

• 3. Evaluate Response Quality Across Multiple Dimensions

• 4. Test the Conversation, Not Just the Response

• 5. Test Whether the AI Can Handle Corrections

• 6. Test Ambiguity

• 7. Test the Knowledge Behind the Answer

• 8. Test for Hallucinations

• 9. Test Fallback Behavior

• 10. Test Human Escalation

• 11. Test Integrations Like You Would in Any Other Application

• 12. Build a Golden Dataset

• 13. Don't Only Measure Pass Rate

• 14. Create Risk-Based Conversational Tests

• 15. A Practical Conversational AI Test Strategy

• What Traditional QA Engineers Already Bring to AI Testing

• Wrapping Up

1. Start With Intent, Not Exact Wording

Consider these three messages:

1. "How do I reset my password?"

2. "I can't access my account."

3. "Forgot password."

They look different. But depending on the application, they may all represent the same underlying user goal:

PASSWORD_RESET

In conversational AI, the sentence a user types is often called an utterance, while the goal behind that message can be represented as an intent.

This creates an important testing question: Can the system understand the same intent when users express it differently?

A simple test set could look like this:

Let's look at a small example:

test_cases = [ 
          { "message": "I forgot my password", 
            "expected_intent": "PASSWORD_RESET", 
          }, 
          { "message": "How do I change my password?", 
            "expected_intent": "PASSWORD_RESET", 
          }, 
          { "message": "Can't get into my account",  
            "expected_intent": "PASSWORD_RESET", 
          }, 
]

for test in test_cases: 
    response = ai_agent.send(test["message"])
    assert response.intent == test["expected_intent"]

Before running this test, you need to know the expected intent. It usually comes from the application's approved intent definitions or a reviewed test dataset. The automation isn't deciding what the correct intent should be. It's checking whether the AI agent classified the user's message according to the behavior the team already defined:

Input: "Can't get into my account"

Expected intent: PASSWORD_RESET Actual intent: PASSWORD_RESET

PASS

Don't stop with clean sentences.

Real users make spelling mistakes, use abbreviations, provide incomplete information, and sometimes type only a few words.

So I would also test:

"forgot pwd"

"cant login"

"password help"

"locked out"

This is where conversational AI testing starts becoming interesting. We're not testing whether the system recognizes one predefined sentence. We're testing whether it understands variations of a user's goal.

2. Don't Use Exact Text Matching for Every Response

One of the first habits I had to reconsider was comparing actual and expected responses word for word.

Suppose the expected answer is: "You can reset your password using the Forgot Password link."

But the AI responds: "Select Forgot Password on the login screen to begin resetting your password."

An exact string comparison fails. But from a user's perspective, the response may be completely correct.

Instead of defining the expected result as one sentence, define the properties a good response must contain.

For example, the response should:

• explain how to start the password-reset process

• provide an actionable next step

• not ask the user to reveal their password

• avoid inventing account information

• stay relevant to password recovery

Now multiple responses can pass without being identical. This was one of the biggest changes for me.

For deterministic applications, expected output is often a value. For conversational AI, the expected result may need to be a set of evaluation criteria.

3. Evaluate Response Quality Across Multiple Dimensions

Correctness is important, but it shouldn't be the only thing you evaluate. I find it useful to break response quality into several dimensions.

1. Accuracy: is the information correct? If the AI says customers can reset passwords through email when the actual process requires contacting support, the response fails even if it sounds convincing.

2. Relevance: did the AI answer what the user actually asked? A response can contain accurate information and still be irrelevant.

3. Completeness: did the response include the information required for the user to move forward?

4. Clarity: can the user easily understand the answer?

5. Helpfulness: does the response actually help the user accomplish their goal?

A simple scoring rubric could look like this:

You can then define a threshold appropriate for your application.

The exact scoring system isn't the important part. What's important is making the evaluation criteria explicit instead of relying on: "This answer looks good to me."

4. Test the Conversation, Not Just the Response

Single-turn testing is useful, but users rarely interact with an AI agent using perfectly isolated questions.

Consider this conversation:

User: I need to update my address.

The AI explains the process.

Then:

User: Can I do that online?

What does "that" mean? The second message depends entirely on the first.

Now imagine:

User: I need to update my address.

AI: Sure. I can help with that.

User: Actually, before that, can you tell me when my next payment is due?

AI: Your next payment is due on September 15.

User: Thanks. Now back to the address.

Can the system return to the original topic?

That's a different type of test.

Your multi-turn test suite should include scenarios such as:

• follow-up questions

• references to earlier messages

• topic switching

• returning to a previous topic

• user corrections

• incomplete information

• ambiguous questions

• repeated questions

This changed the unit of testing for me. Sometimes you're testing a response, while other times you're testing the entire conversation.

Here's an example:

ai_agent.send("My order number is A10245")
response = ai_agent.send("When will it arrive?")

assert response.order_id == "A10245"

The second message doesn't contain the order number. This check verifies that the AI agent retained information from the earlier turn instead of treating “When will it arrive?” as an unrelated question.

5. Test Whether the AI Can Handle Corrections

People change their minds and make mistakes.

For example:

User: My account number ends in 4567.

Then:

User: Sorry, I meant 4576.

What happens next?

The system should ideally use the corrected information rather than continuing with the original value.

The same principle applies to other conversational details.

"I'm traveling to Boston."

followed by:

"Actually, make that Chicago."

Or:

"I need the report for June."

followed by:

"Sorry, July."

These are useful tests because they expose whether the agent is genuinely maintaining conversational context or simply accumulating information without understanding which information is current.

6. Test Ambiguity

Users don't always provide enough information.

Imagine someone types: "I want to change it."

Change what? Their address? Password? Payment method? Notification preference?

A poor conversational system may guess. A better one may ask: "What would you like to change?"

This gives us another important test category: clarification behavior.

Create intentionally ambiguous utterances such as:

• "How do I update it?"

• "It's not working."

• "Can you change that?"

• "I need help with my account."

Then evaluate whether the AI recognizes that information is missing, avoids making unsupported assumptions, and asks an appropriate clarification question.

Sometimes the best AI response isn't an answer, it's another question.

7. Test the Knowledge Behind the Answer

At first, I focused almost entirely on what the AI said. Then I realized that a poor answer doesn't automatically mean the language model itself is the problem.

The system may be using a knowledge base, retrieval system, documentation, APIs, or other enterprise data sources.

If that information is wrong, incomplete, conflicting, or outdated, the AI may produce a poor response even when the model is behaving as designed.

Suppose the official policy says: Customers have 30 days to return a product. But an outdated knowledge article says that customers have 60 days.

If the AI retrieves the outdated article and confidently answers "60 days," the response is wrong.

But the investigation shouldn't end with: "The AI hallucinated." The tester needs to determine where the wrong information came from.

Questions I would investigate include:

• What source did the response use?

• Was the source approved?

• Is the information current?

• Were multiple sources contradictory?

• Did retrieval return the correct document?

• Did the final response accurately represent the retrieved information?

This becomes particularly important in systems using retrieval-augmented generation, or RAG.

8. Test for Hallucinations

One of the most important conversational AI tests is surprisingly simple: Ask about something the system doesn't know.

Suppose an internal support assistant contains documentation for Products A, B, and C. Ask: "What is the cancellation policy for Product Z?"

Product Z doesn't exist.

So what should happen? The worst outcome is for the system to confidently invent a cancellation policy.

Depending on the application, better behavior might be:

"I don't have information about Product Z."

or:

"I couldn't find that information. Would you like me to connect you with support?"

A useful hallucination test suite should include:

• nonexistent products

• fake policy names

• unsupported features

• deliberately incorrect assumptions

• questions outside the knowledge domain

• requests for information unavailable to the system

You're testing whether the AI knows when not to answer.

9. Test Fallback Behavior

Every conversational system will eventually receive something it doesn't understand. But that isn't necessarily a failure.

The important question is what happens next?

Imagine a scenario like this:

User: I need help with my ZXP adjustment.

The system doesn't recognize "ZXP."

A poor fallback might repeatedly say: "Sorry, I don't understand." A better fallback might ask: "Could you tell me a little more about what you mean by ZXP adjustment?"

If the system still can't understand the request, it may need to offer another path.

Fallback tests should cover:

• unknown intents

• misspellings

• incomplete requests

• unsupported topics

• conflicting requests

• repeated misunderstanding

Also test what happens after multiple failures. An AI agent shouldn't trap the user in an endless loop of: "Sorry, I didn't understand that."

10. Test Human Escalation

Sometimes the AI agent should recognize that it can no longer handle the conversation reliably. This might happen when the user explicitly asks for a person, the request falls outside the agent's capabilities, or the situation requires human judgment. In those cases, continuing to generate answers may be worse than handing the conversation over

Consider situations involving:

• repeated misunderstanding

• unsupported account problems

• user requests for a human

• sensitive workflows

• exceptions the automated process cannot handle

If escalation is part of the product design, test the entire transition.

For example:

User: I want to speak to someone.

Does the AI recognize the request? Does it transfer the conversation correctly? Does the human agent receive the relevant conversation history? Does the user have to explain everything again? Does the AI continue trying to answer after escalation should have occurred?

A technically successful transfer can still create a poor experience if all the context is lost.

11. Test Integrations Like You Would in Any Other Application

Conversational interfaces can make complex systems look simple.

The user sees:

"What's the status of my order?"

But behind that sentence, the agent might:

1. identify the user's intent

2. authenticate the customer

3. call an order API

4. retrieve the order

5. interpret the response

6. generate a natural-language answer

Traditional testing skills become extremely valuable here.

If the API returns:

{
  "order_id": "A10245",
  "status": "SHIPPED"
}

the AI shouldn't tell the user: "Your order is still processing."

To check these integrations, make sure you test:

• correct API mapping

• authentication failures

• timeouts

• empty responses

• malformed responses

• unavailable services

• incorrect status codes

• partial data

AI doesn't eliminate conventional integration testing. It adds another layer on top of it.

12. Build a Golden Dataset

Manual exploratory testing is useful when you're learning how an AI behaves, but eventually you need repeatability. This is where a golden dataset becomes useful.

A golden dataset is a curated collection of representative test inputs and expected behaviors that can be rerun as the system changes.

For example:

Then expand each important intent with paraphrases, edge cases, negative cases, and multi-turn scenarios.

Whenever prompts, knowledge, models, integrations, or conversation logic change, rerun the dataset.

Now you have something much closer to traditional regression testing.

13. Don't Only Measure Pass Rate

Suppose you execute 1,000 conversational tests and 950 pass. A 95% pass rate sounds good.

But what failed? Five hundred harmless FAQ questions? Or five critical account-security scenarios?

Aggregate pass rate alone doesn't tell the whole story. Depending on the application, useful metrics might include:

• Intent recognition accuracy: How often was the user's goal understood correctly?

• Fallback rate: How often did the system fail to understand the user?

• Task completion rate: How often did users successfully accomplish the intended task?

• Escalation success rate: When human help was required, did the transition succeed?

• Grounding failures: How often did responses conflict with approved knowledge?

• Context failures: How often did the system lose important information during multi-turn conversations?

• Critical hallucinations: How often did the system confidently provide unsupported information?

The right metrics depend on the product and its risks. A customer-service FAQ bot and an AI system supporting financial decisions shouldn't necessarily have the same quality thresholds.

14. Create Risk-Based Conversational Tests

This is another traditional QA principle that transfers very well.

Not every AI failure has the same impact. If an AI responds awkwardly to: "What are your business hours?", that's inconvenient.

If it gives incorrect information about a payment, account security, healthcare instruction, or financial policy, the impact could be much greater.

So categorize scenarios by risk.

For example:

Then concentrate regression coverage on the scenarios where incorrect AI behavior would cause the greatest harm.

15. A Practical Conversational AI Test Strategy

If I were starting a conversational AI QA effort today, I would organize it into these layers:

Layer 1: Intent Testing

Can the system understand what the user wants despite variations in language?

Layer 2: Response Evaluation

Are responses accurate, relevant, complete, clear, and useful?

Layer 3: Conversation Testing

Can the system maintain context across multiple turns?

Layer 4: Knowledge and Grounding

Are answers supported by approved and current information?

Layer 5: Negative and Hallucination Testing

Does the system avoid confidently answering when it doesn't have enough information?

Layer 6: Fallback and Escalation

Can the system recover when it doesn't understand, and can it hand off to a human when necessary?

Layer 7: Integration Testing

Are APIs, authentication, databases, and downstream systems behaving correctly?

Layer 8: Regression Testing

Can important behaviors be rerun after model, prompt, knowledge, or application changes?

That gives QA teams a much more structured starting point than simply opening a chatbot and asking random questions.

What Traditional QA Engineers Already Bring to AI Testing

When I first started learning conversational AI, I thought I needed to forget everything I knew about traditional testing. But I don't believe that anymore.

A lot of our existing skills transfer extremely well. We already know how to:

• question assumptions

• explore edge cases

• design negative tests

• trace failures through multiple systems

• validate integrations

• prioritize by risk

• build regression suites

• investigate unexpected behavior

What changes is the definition of the expected result.

For some AI scenarios, the expected result isn't:

Response = X

It's closer to:

The response must satisfy X, Y, and Z while avoiding A and B.

Once I understood that distinction, conversational AI testing started making much more sense to me.

Wrapping Up

My first instinct when learning conversational AI was to search for the test cases.

Now I think that's the wrong place to start. Start with the user.

• What are they trying to accomplish?

• What are the different ways they might ask for it?

• What information does the AI need?

• What would a useful answer contain?

• What should the system never say?

• What happens if the AI doesn't know?

• What happens when the conversation changes direction?

• What evidence would make you confident enough to release that experience to real users?

Those questions eventually become your test cases.

Conversational AI may be less deterministic than the applications many QA engineers are used to testing. But that doesn't make it untestable. It simply means we need to move beyond asking:

"Did I get the exact output I expected?"

and start asking:

"Did the system behave correctly, safely, and usefully across the different ways a real person might interact with it?"

For me, that was the biggest shift. The tools are changing, and so is the interface. Even the definition of an expected result is changing. But the fundamental responsibility of quality engineering hasn't changed very much at all: understand how the system can fail before the user has to discover it for you.

It's a common perception, and to be fair, finding bugs is an important part of the job. But if you spend even a few weeks working on a modern software team, you'll quickly realize that's only a small part of what QA engineers actually do.

Software development has changed dramatically over the last decade. Teams no longer wait months to release new features. Many organizations deploy updates every week, every day, or even several times a day. Applications have become more complex, with cloud services, APIs, microservices, mobile apps, and third-party integrations all working together behind the scenes.

As software has evolved, the role of QA has evolved with it.

Today's QA engineers are involved long before a feature reaches testing. They help review requirements, identify risks, clarify business expectations, verify APIs and databases, automate repetitive tests, investigate production issues, and work closely with developers throughout the entire development lifecycle.

In other words, QA isn't just about finding problems after software has been built. It's about helping prevent those problems from happening in the first place.

Whether you're thinking about becoming a QA engineer, transitioning from manual testing into automation, or simply curious about what quality engineering looks like today, understanding how the role has changed is an important first step.

In this article, we'll explore what QA engineers actually do, why the profession has evolved, and the skills that have become essential for building reliable software in today's fast-moving development environments.

Table of Contents

• How the QA Role Has Changed

• Good Testing Starts Before the First Test Case

• Real-world Example

• Modern QA Is About More Than Automation

• Where Different Types of Testing Fit

• Skills Every QA Engineer Should Develop

• Common Misconceptions About QA

• If You're Considering a Career in QA

• Wrapping Up

How the QA Role Has Changed

Many software usually teams followed a simple workflow. Business analysts gathered requirements, developers built the application, and once development was complete, the software was handed over to QA for testing. If defects were found, the application went back to the development team before eventually being released.

In that model, QA was often viewed as the final checkpoint before production. Testing happened after most of the important technical decisions had already been made.

That approach worked reasonably well when software releases happened only a few times a year. Teams had enough time to finish development, perform weeks of manual testing, fix defects, and prepare for a scheduled release.

Todays software development looks very different.

Many organizations use Agile methodologies, Continuous Integration (CI), and Continuous Delivery (CD). Instead of delivering software every few months, teams continuously add features, fix bugs, and release improvements in short development cycles.

Because development moves much faster, quality can no longer be treated as the final phase of a project.

Instead, QA engineers work alongside developers, product owners, business analysts, UX designers, and DevOps engineers from the beginning of the development process. They participate in sprint planning, review user stories, discuss acceptance criteria, identify potential risks, and help ensure new features are designed with testing in mind before development even starts.

This shift has changed the role of QA from testing completed software to helping teams build quality into the software from the start.

For example, imagine a team is building an online banking application that allows customers to transfer money between accounts. A traditional testing approach might focus on verifying whether the transfer succeeds after the feature has been developed.

An experienced QA engineer starts much earlier by asking questions such as:

• What happens if the network connection is interrupted during the transfer?

• What should happen if the customer has insufficient funds?

• Can the same transfer request be submitted twice accidentally?

• How should the application respond if the receiving bank is temporarily unavailable?

Questions like these help uncover potential problems before developers spend time writing code. Addressing these scenarios early is often much less expensive than discovering them during testing—or worse, after the application has been released to customers.

This is one of the biggest reasons QA has evolved from a role focused primarily on testing into one that contributes throughout the entire software development lifecycle.

Quality is no longer something that's checked at the end of a project.

It's something the entire team builds together, one decision at a time.

Traditional Development

Requirements -> Development -> QA Testing -> Production

Modern Development

Requirement -> DEV+QA+Product Owner -> Continuous Development -> Continuous Testing -> Production

Good Testing Starts Before the First Test Case

One of the biggest surprises for people entering software testing is discovering that a significant part of a QA engineer's work happens before a single test case is written.

Many people picture testing as something that begins only after developers finish building a feature. In reality, experienced QA engineers become involved much earlier because that's often where they can have the greatest impact.

Consider a simple requirement:

A user must create a password containing at least eight characters.

At first glance, the requirement seems complete. A developer can implement it, and a tester can verify that passwords shorter than eight characters are rejected.

But software requirements are rarely that straightforward.

A QA engineer naturally starts looking beyond the obvious by asking questions such as:

• Should spaces or count as characters? Or Should leading or trailing spaces be removed automatically?

• Is there a maximum password length?

• Are special or Unicode characters required?

• Will the same validation rules apply on both the web application and the mobile app?

• What error message should users see if the password is invalid?

None of these questions are about trying to catch developers making mistakes.

They're about making sure the entire team has the same understanding of how the feature should work before development moves forward.

This process is often called requirement clarification, and it's one of the most valuable contributions a QA engineer can make.

Without these discussions, developers may implement one interpretation of the requirement while testers validate another. Business stakeholders may expect something entirely different. Even when everyone is working hard, unclear requirements can lead to unnecessary defects, rework, and frustration.

Asking thoughtful questions early helps avoid those situations.

It also saves time.

Finding an unclear requirement during a planning meeting usually takes a few minutes to resolve. Discovering the same issue after development, testing, and deployment can take hours—or even days—to investigate and fix.

Good QA isn't only about verifying that software works correctly.

It's also about helping ensure the team is building the right software in the first place.

Real-world Example

Imagine an online shopping website that offers discount coupons.

The requirement says:

"Users can apply one coupon during checkout."

At first, it sounds simple.

A QA engineer might ask:

• What happens if the coupon has expired?

• Can two browser tabs apply the same coupon simultaneously?

• What if the customer removes an item after applying the coupon?

• Can multiple coupons be combined?

• What happens if the payment fails after the coupon has been applied?

These questions often uncover situations that weren't considered during the initial discussion.

Instead of becoming production defects, they become design decisions that the team can address before writing code.

That's one of the reasons experienced QA engineers spend so much time asking questions. They're not slowing development down. They're helping the team avoid expensive surprises later.

Modern QA Is About More Than Automation

If you browse job postings for QA engineers, you'll probably notice one thing almost immediately.

Many of them mention automation tools such as Selenium, Plawright, Cypress, Appium, Javascript, Java, Python, and so on - the list goes on

Because of that, it's easy to assume that QA engineers spend their entire day writing automated tests. Automation is certainly an important part of the job, but it isn't the job itself.

Think about a new feature being added to a food delivery app. Customers can now save multiple delivery addresses and choose one during checkout. Before any automation script is written, a QA engineer is already involved.

They review the requirement to understand how the feature is expected to work. They discuss different scenarios with developers and product owners, such as what should happen if a customer deletes their default address or enters an invalid ZIP code. They identify edge cases that may not have been considered during planning and think about how the feature interacts with existing functionality.

Only after those discussions do they begin designing test scenarios.

Some of those tests may eventually become automated, especially if the feature will be used in future regression testing. Others may be better suited for exploratory testing because they require human observation and judgment.

QA engineers choose the right testing approach based on the problem they're trying to solve rather than trying to automate everything.

A typical week for a QA engineer might include a variety of responsibilities, such as:

• Reviewing new requirements before development begins.

• Designing functional and edge-case test scenarios.

• Verifying REST API responses using tools like Postman or Bruno.

• Validating backend data with SQL queries.

• Building or maintaining automated regression tests.

• Investigating issues reported from production.

• Working with developers to reproduce and understand defects.

• Confirming bug fixes before a release.

Some weeks involve more automation than others. Some involve more investigation, collaboration, or exploratory testing.

That's one of the reasons QA engineering is such a diverse field. No two projects are exactly the same, and the work changes depending on the product, the team, and the stage of development.

Automation simply helps QA engineers spend less time repeating predictable tasks so they can focus on the work that requires critical thinking.

It's a tool that supports quality engineering—not the definition of it.

Where Different Types of Testing Fit

Another common misconception is that QA engineers only test the user interface. In reality, software can be tested at many different levels.

For example:

QA engineers don't necessarily perform all of these types of testing every day, but understanding when and why each approach is useful helps them choose the right strategy for different situations.

Quality isn't achieved by relying on a single testing technique.

It's achieved by combining multiple testing approaches to build confidence that the software behaves correctly under real-world conditions.

Skills Every QA Engineer Should Develop

If you're just starting your QA journey, it's easy to focus on learning specific tools. While tools are important, they're only part of the picture. The most successful QA engineers build a combination of technical skills, problem-solving abilities, and effective communication.

Here are some of the core skills that will help you grow in QA role.

We don't need to masters all of these skills overnight. The important thing is to build a strong foundation, stay curious, and keep learning as the industry evolves.

Common Misconceptions About QA

If you're new to software testing, you've probably heard some of these statements before. While they may sound reasonable, they don't reflect how modern QA teams actually work.

Myth: QA engineers Only Find Bugs

Reality: Finding bugs is only one part of the job. QA engineers also review requirements, identify risks, design test strategies, automate repetitive testing, validate APIs and databases, and help teams prevent defects before software reaches users.

Myth: Automation Will Replace QA Engineers.

Reality: Automation is a tool, not a replacement for human thinking. Automated tests can execute repetitive tasks, but they can't decide what should be tested, identify unclear requirements, or evaluate whether a feature delivers a good user experience.

Myth: QA is Responsible for Quality.

Reality: Quality is a shared responsibility. Developers write reliable code, product owners define clear requirements, designers focus on usability, DevOps engineers build dependable deployment pipelines, and QA engineers help ensure everything works together as expected.

Myth: Manual Testing is No Longer Useful.

Reality: Automation is excellent for repetitive regression testing, but manual testing remains valuable for exploratory testing, usability checks, and investigating unexpected behavior. The two approaches complement each other rather than compete.

Myth: QA is Easier than Software Development.

Reality: Modern QA requires strong analytical thinking, technical knowledge, communication skills, and a solid understanding of how software systems work. While the responsibilities differ from software development, both roles play an equally important part in delivering high-quality software.

If You're Considering a Career in QA

If you're thinking about becoming a QA engineer, don't worry if you don't know every testing tool or automation framework yet. Every experienced QA professional started by learning the fundamentals.

Begin by understanding how software works. Learn how web applications communicate with APIs, how data is stored in databases, and how different components interact to deliver a feature. These concepts will help you understand why you're testing something, not just how to test it.

At the same time, practice thinking like a user. Ask questions, explore different scenarios, and look beyond the "happy path." Some of the most valuable defects are discovered simply because someone asked, "What happens if this doesn't go as expected?"

As you grow, gradually build technical skills such as SQL, API testing, automation, version control, and CI/CD. Focus on continuous learning and improving one skill at a time.

Most importantly, remember that software quality isn't created by one person or one team. The best QA engineers work collaboratively, communicate effectively, and help everyone build better software together.

Wrapping Up

Software testing has come a long way from being viewed as the final step before a release.

Today's QA engineers contribute throughout the software development lifecycle. They ask thoughtful questions, clarify requirements, automate repetitive testing, validate APIs and databases, investigate production issues, and help teams deliver reliable software with confidence.

Finding bugs will always be an important part of the role, but it's no longer what defines a successful QA engineer.

What truly makes a difference is the ability to prevent problems before they happen, think critically about how software is used, and collaborate with the entire team to deliver a better product.

If you're just starting your journey in software quality, don't measure your progress by how many testing tools you've learned. Focus on building strong fundamentals, staying curious, and continuously improving your problem-solving skills.

Technology will continue to evolve, but those qualities will always be valuable.

For me personally, there is a deep satisfaction that comes with seeing your product come alive under load to find out how it’ll actually work in production when thousands of users will be using it.

Performance testing is about discovering how your system performs under real-world pressure in terms of load, concurrency, and throughput. One of the key aspects of performance testing is ensuring that the APIs can endure the expected load. You can do this using tools like Apache JMeter and K6.

In this tutorial, we’ll explore how you can build your first end-to-end performance test in Apache JMeter. You will be learning to create a test suite that is dynamic (the test can be run with any test data) and that’s one-click executable (the test execution can be done through the GUI as well as the CLI).

Table of Contents

1. Prerequisites

2. Introduction to Apache JMeter

   • Step 1: Create a New Test Plan

   • Step 2: Configure the Thread Group

   • Step 3: Add HTTP Request Defaults

   • Step 4: Add a CSV Data Set Config (Dynamic Input)

   • Step 5: Add the HTTP Request Sampler

   • Step 6: Add a JSON Extractor

   • Step 7: Add an Assertion

   • Step 8: Add Listeners

   • Step 9: Run Your Test

   • Step 10: Chain Another Request (Optional)

   • Step 11: Analyze the Results

   • Pro Tips

   • Example Folder Structure:

3. Conclusion

Prerequisites

Before you start, make sure you have:

• Apache JMeter (5.5 or above) installed.

• Java 8 or later configured on your system.

You can check if JMeter is installed by running the command below:

jmeter -v

Note: This tutorial will use the JSONPlaceholder public API. You’ll learn how you can get a post_id and use it in a chain request to get user details.

Let’s get started.

Introduction to Apache JMeter

Apache JMeter is an open-source API load and stress testing tool. It’s a powerful testing tool that supports a wide range of protocols, including HTTP, HTTPS, FTP, JDBC, SOAP, and REST.

JMeter helps you answer critical questions about your APIs, like:

• How does my API perform under heavy load?

• What’s the maximum number of users it can handle before it starts failing?

• Which requests or endpoints are slowing things down?

Let’s go through the step-by-step process of building a dynamic load testing suite with JMeter.

Step 1: Create a New Test Plan

Once JMeter opens, you’ll see an empty Test Plan. Think of this as your main workspace, which holds everything: Test configuration, users, requests, assertions, and results.

Right-click on Test Plan → Add → Threads (Users) → Thread Group to add a thread group. A thread group is essentially a test suite containing our test cases.

Step 2: Configure the Thread Group

To configure the thread group, fill out the following input fields:

You’ve now created a small, controlled load test of 10 total requests (5 users × 2 loops).

Step 3: Add HTTP Request Defaults

When you’re creating a suite of 100s of APIs, you don’t need to add your request details to all the API samplers in JMeter. JMeter lets you set it once globally by using a config element called HTTP Request Defaults. To add this element, follow the steps below:

1. Right-click on Thread Group → Add → Config Element → HTTP Request Defaults.

2. Enter the following:

   • Protocol: https

   • Server Name or IP: jsonplaceholder.typicode.com

This means all requests in this test will automatically use this base URL.

Step 4: Add a CSV Data Set Config (Dynamic Input)

In real projects, APIs rarely use static inputs. Take as an example a login API that you want to run for 100 concurrent users. In a real-world scenario, every login request will have a different username and password.

To replicate this on JMeter, you need to run your test for 100 different login credentials. This means that your test should be test data-driven. We can build a data-driven test in JMeter using a CSV file:

1. Create a file named data.csv with the following content:

    post_id
    1
    2
    3
    4
    5
   

2. Save it in your JMeter project folder.

3. In JMeter, right-click on Thread Group → Add → Config Element → CSV Data Set Config.

   

4. Fill in the following fields:

   • Filename: data.csv

   • Variable Names: post_id

   • Recycle on EOF: True

   • Stop thread on EOF: False

     

Now each user will pick a new post_id for every iteration from the CSV file.

Step 5: Add the HTTP Request Sampler

Now let’s add the actual API call we'll test under load. To do this, follow the steps below:

1. Right-click on Thread Group → Add → Sampler → HTTP Request.

   

2. Rename it to Get Post Data.

3. Set the following fields:

   • Method: GET

   • Path: /posts/${post_id}

Here ${post_id} dynamically takes its value from your CSV file. The Protocol and Server IP fields will automatically get data from the ‘HTTP Request default’ config element that we added in Step #3.

Step 6: Add a JSON Extractor

When the API returns a response, we can extract a value (like userId) from it and use it later. This is used to implement an end-to-end flow where data is gotten (with GET) from an API and sent to the next POST/DELETE API.

For our API, below is the example response:

{
  "userId": 1,
  "id": 3,
  "title": "fugiat veniam minus",
  "body": "This is an example post body"
}

To extract userId:

1. Right-click on Get Post Data → Add → Post Processors → JSON Extractor.

   

2. Set the variables below in the JSON Extractor:

   • Name: Extract User ID

   • Variable Name: user_id

   • JSON Path Expression: $.userId

Now you can use ${user_id} in the next request, making your test fully dynamic.

Step 7: Add an Assertion

Assertions help you verify that your API responds correctly even under load. You can assert on the API response code, response time, or even the response payload. To add an assertion, follow the steps below:

1. Right-click Get Post Data → Add → Assertions → Response Assertion.

   

2. Configure as:

   • Response Field to Test: Response Code – This will add an assertion for the response code.

   • Pattern Matching Rules: Contains

   • Pattern to Test: 200

This ensures JMeter only counts the request as successful if the word fugiat appears in the response.

Step 8: Add Listeners

We’ll add listeners to display our test results in different forms, such as visually or in a summary. Let’s add two essential ones:

1. View Results Tree: to view and debug individual requests.

2. Summary Report: to view performance metrics like response time, error rate, and throughput.

Add them via Thread Group → Add → Listener → [Choose Listener]

Step 9: Run Your Test

Hit the green Start button at the top. JMeter will start sending requests to your API using the dynamic post IDs from your CSV file.

As the test runs:

• Green checkmarks in View Results Tree mean successful responses.

• Assertion failures will appear in red.

• Summary Report will aggregate key metrics.

Step 10: Chain Another Request (Optional)

Let’s take it one step further: we’ll use the extracted user_id from the first response to get user details from the GET users call. To do this, follow the steps below:

1. Right-click Thread Group → Add → Sampler → HTTP Request.

2. Rename to Get User Details.

3. Set:

   • Method: GET

   • Path: /users/${user_id}

     

Step 11: Analyze the Results

Once the test completes, open the Summary Report. You’ll see:

If your error percentage is 0% and throughput is stable, your system handled the load well.

Pro Tips

• Parameterize everything. Use multiple CSVs for realistic test flows (users, IDs, tokens).

• Add timers (like Constant Timer) to simulate think time between user actions.

• Use Assertions wisely. Don’t add extra assertions; focus on key validations such as response time and API status code.

• Generate HTML reports using the command below:

    jmeter -n -t test-plan.jmx -l results.jtl -e -o report
  

Example Folder Structure:

Follow the folder structure below for an organized test suite.

performance-test/
├── data.csv
├── test-plan.jmx
└── results/
    ├── summary.csv
    └── report.html

Conclusion

Performance testing is an essential element of a production readiness checklist for any product. It helps you ensure that your product can handle the expected user load and scale gracefully.

This guide is your first step towards writing end-to-end performance test cases and bridging the gap between being a functional test engineer and a full-stack QA Engineer who understands both quality and scalability.

I hope you found this tutorial helpful. If you want to stay connected or learn more about performance testing, follow me on LinkedIn.

Usually, a QA tester creates a report after discovering a bug. The next step is that they send a report to the bug handler. Basically, a bug handler is a developer that fixes a bug according to the detailed report. The bug handler ensures a clean and efficient QA process and makes it easier to have a clean conversation between the fixers and the testers.

But you might be wondering – why do we need QA documentation?

QA testing is essential in IT products today. Tight deadlines, unique skill sets within the organization, and the demand to develop products stress the need for a structured methodology. QA documentation guides testers through levels of clarity and overall coverage.

I've written this article to make your life a bit easier. So here it is, your ultimate guide on how to write software QA documentation that will work.

This is what we'll cover:

• Make a Test Plan and a Test Progress Report
• Create Test Cases
• Defect Reports
• Useful Tips for Defect Report Writing
• Submit a Defect Report
• Conclusion

Make a Test Plan and a Test Progress Report

Developing great software requires a thorough and documented approach to testing. Everything begins in the development phase with a complete checkout plan as a template for the entire QA process. It outlines the general requirements, defines the followed route, identifies the necessary assets, and sets a simple timeline for achieving the objectives.

Draw up a roadmap before you start

Before you start implementing your test plan, take some time to think about the big picture. Ask yourself:

What problem does this software solve? As soon as you understand the main purpose of the software, you'll be able to manage the checkout process and identify the features that are most important to your goals.

I promise that once you see that you can do this, you quickly prioritize your efforts. They are based primarily on the importance of the functionality and its impact on the user.

Create a plan

The test plan is the central part of the QA technique. The test plan should contain the following key elements:

• Imaginary test: Clearly identify the expected impact of the payment method. Will it confirm that core functionality is running smoothly, or will it alert you to capability gaps?
• Test method: Outline how the test will be performed. Will you do targeted testing, useless testing, or a mix of the two?
• Resource allocation: Identify the equipment and technology required to perform a complete control. What control framework will you use? Are specific hardware or software configurations required for proper control?
• Timetable and remaining dates: Establish a realistic timetable for the trial. Set clear deadlines to keep the project on track.

Understand the 'why' and 'how'

A well-developed roadmap will ensure that key issues are addressed throughout the improvement approach:

• Acceptance requirements: Clear fail/success criteria should be defined for all control cases. So, criteria allow the customer to understand that the product is high quality and ready for the end-user.
• Resource management: Identify the assets required for the testing. This includes preferred machines, copies of software in various forms, required expertise, and more.
• Team dynamics: Ensure that simple roles and tasks are defined within the test team. Who is responsible for particular test cases? Who documents bugs and who talks to developers?
• Time management: I advise setting realistic checkout times, taking into account project deadlines and the availability of useful help.

The test progress report is another part of the QA documentation, similar to the test plan, but with additional data on current progress. This document allows you and your development team to monitor project progress and identify any organizational issues.

A test plan and a progress report

Create Test Cases

Think of each test case as a detailed recipe for finding potential defects in the functionality of a software program. By following these "recipes" and evaluating the expected and actual results, defects can be accurately identified and addressed before preventive action is taken.

Each test case acts as an independent unit, outlining the steps required to evaluate a particular element of a software program.

Below is a breakdown of the key elements that make up a well-defined test case Here is my step-by-step guide for creating test cases:

• ID: This is an identifier field to help distinguish between test cases and track them easily.
• Priority: This indicates the severity of the test case based on the functionality of the program and its impact on the normal overall performance of the known software program application.
• Test Requirements: These are requirements for testing software successfully, this can include reference documents also.
• Software Module: This shows the feature under testing. It also refers to the software requirements specifications (SRS) document explaining the software feature in detail.
• Test Context: This details the test plan to clarify how the daily tests will be carried out. It also identifies the test data required for a successful case study and includes unique and important statistical information.
• Expected Output: This describes the expected output to be displayed if the test is successful.
• Actual output: This indicates the actual result in the event of failure and shows the developer the errors in the application module of the software program.
• Comment: This is an optional section where the tester can give a description of the observations or add additional records.

All QAs typically include the above elements, but can also be designed specifically for the task selected by the QA group. Also, each control case follows a lifecycle that defines the phases of creation, testing (pass, fail), completion, and so on.

In the next section, we’ll check out another important element of QA documentation: the defect report.

Test cases

Defect Reports

Defect reporting is an important element of QA documentation. Issue tracking is the detailed reporting of sudden problems that arise in a software product. Careful documentation of these issues lays the groundwork for a complex and bug-free final product.

Sounds simple, right? Yes, but only until you start documenting.

A defect report

The bug report consists of the following sections: Identifier, Summary, Description, Steps to Reproduce, Reproducibility, Severity, Priority, Environment, and Attachments.

• Identifier: Each software program problem is assigned a completely unique identifier that acts like a customized nameplate. This makes the QA documentation less complicated to navigate and allows verbal communication between the installer, tester, and project manager (PM).
• Summary: This is an opportunity to provide brief and informative answers to three basic questions: What was the problem? Where did the problem appear? Under what specific conditions did the problem appear?
• Explanation: Examine the failure log in more detail. Summarize the identified action (the result completed) and check it against the planned action (the predicted end result). Including a link to the app requirements of the software program can serve as a useful reference element.
• Reproduction method (STR): This phase should be considered as a step-by-step recipe for reproducing the defect. It should be strict and cover all the steps that caused the problem. Skipping critical steps can make it difficult for you to reproduce the problem and cause delays.
• Reproducibility: In this section, you clarify if the bug appears every time you follow the STR. You should use numbers to show approximate chances, for example, 7 times out of 10.
• Severity: Here you explain how much harm the bug may bring to the project. Essentially, this is a measure of the severity of the technical disruption that a bug will cause to the whole project. Remember that problems that are known to be minor can grow and cause you extreme problems throughout the software.
• Priority: Each error log assigns a problem priority that indicates its urgency. Common priorities are letters (A: highest priority, Z: highest priority), numbers (1: highest priority, 9: highest priority), or descriptive terms (high, medium, low).
• Environment: Specify the gadget or browser model in which the bug appeared. This will help you to put the problem in context and narrow down a valid cause.
• Attachments: If possible, enrich the documentation with screenshots, screen recordings, console log documents, and others. This will help to provide visual documentation of the error.

My structure provides detailed information, so you can empower developers to effectively diagnose, address, and eliminate any software bugs. In this way, leading to more user-friendly and robust products.

Now, in the next section, we'll see some useful tips you can use for defect report writing.

Useful Tips for Defect Report Writing

1. Write a sufficient summary. It does not matter if it is long or short. What matters is that it should be clear.
2. Have a look at the summary and the description. Do they look pretty much the same? You must have forgotten to outline the expected and actual results in the description and to add the link to the requirements.
3. Capture the issue with the help of a screenshot. It may save you and the development team a lot of time. Sometimes one glance at the picture is just enough to understand the situation.
4. Before reporting the issue, try to reproduce it at least 3 times to be sure that it exists.
5. Report the issue ASAP and notify your project manager or product owner if the issue is major.
6. Check for grammar mistakes in your QA documentation so you're not taken down by the grammar police.
7. However comical it sounds, make sure that the issue is not a feature – review the documentation again!
8. Do not miss any important information in your Steps to Reproduce.

Submit a Defect Report

The final and one of the most important elements of QA documentation is the defect report. You may understand it covers the entire lifecycle of a problem, from its initial discovery to its final closure.

Let's now examine the key areas of this process:

a defect report lifecycle

We'll go over the defect report lifestyle piece by piece:

Problem Reporting:

This adventure begins with the careful compilation and submission of a report on the entire program. This serves as a roadmap for developers and provides a clear assessment of the problem.

Triage and tasks:

The task manager or technical lead takes on the role of gatekeeper at this stage. They carefully compare the files. If the file contains enough elements to work with, it is assigned to the developer and repaired. However, if the file is missing essential elements, it may be rejected for further improvement.

Bug fixing:

The assigned developer takes the initiative and works diligently to eliminate the annoying bug.

Verification and completion:

Once the developer claims to have fixed the problem, it's your turn as QA. Carefully verify the fix by retesting the functionality in question. If everything works as it should, you are done.

The documentation has come to a happy end. Ideally, this will happen within a reasonable period of one to two weeks.

Reboot and keep going:

But it is not always that simple. If it is known that bugs are still being made in the validation system, there is no need to despair!

Re-open the bug documentation and send it to the developers for further attention. Sometimes the process of fixing bugs is repetitive and requires patience. But by being careful and effective, you can guarantee that all bug reports will eventually reach their final destination, resulting in a more polished and reliable software application product.

Conclusion

Quality Assurance is a process you simply cannot avoid. Each airplane prior to departure undergoes a technical check. If there is any issue, the aircraft is grounded until the problem is solved. The same goes for any software.

But QA documentation is not always "write and ignore". At some point in the software development lifecycle, QA documentation needs to be continually updated and improved as requirements change, new features are introduced, and feedback is received from deployment and production use.

There are also a growing number of styles that use synthetic intelligence and machine learning to partially automate the creation of QA documentation.

For example, natural language processing (NLP) is used to analyze requirements documents and generate draft control examples. Test bots can use NLP to interpret and routinely execute directed test cases. Predictive evaluation can also be used to identify the most dangerous areas of a software program that require more detailed control.

While these strategies are still new and not mature enough to replace human testers, they can help with growth and augment manual exploration, especially for large and complex builds. By making QA documentation a collaborative and ongoing hobby, your team can deliver better software faster and with fewer defects.

Do you need to improve the quality of your software?

My company KeenEthics provides solid development and quality assurance services. In case you need an estimate for a similar project, feel free to get in touch.

If you have enjoyed the article, you should continue with How IT Outsourcing Saves Costs for Your Company and Avoiding Pitfalls in IT Outsourcing: Tips for Minimizing Risks.

But how do you become a QA Engineer? In this article, I will talk about the qualifications and process for becoming a QA engineer.

What is Quality Assurance and Quality Control?

When a product is being built, stakeholders have expectations and requirements. It is important that there is a process put in place to ensure that those high quality standards are being adhered to and mistakes in the product development are spotted and reported.

Quality Assurance focuses more on proactive processes while Quality Control focuses more on reactive corrections that need to be made.

Quality control also focuses on inspection and testing whereas Quality Assurance involves documentation, audits, and management.

Core skills needed for becoming a good QA engineer

Attention to detail

The role of a QA engineer involves being detail-oriented and thorough in order to spot issues or discrepancies in quality standards set for the product.

Good communication

QA engineers will be communicating with stakeholders, management, and developers for the product. Effective and clear communication between all parties involved is really important for the development process of a product.

Good time management

Since a QA engineer will have to balance multiple tasks throughout the day, it is important to have strong time management skills to ensure that you stay productive and are able to meet your deadlines.

Problem solving skills

Throughout the life cycle of a product, issues will arise. It is important that QA engineers have strong problem solving skills and are able to develop a plan to solve those issues.

Strong listening skills

A good QA engineer not only needs to be able to communicate but also listen to their development team, management, and stakeholders to ensure the successful launch of the product.

The ability to write a test plan

A test plan is a document detailing the objectives and testing processes that will be applied to the software product. QA engineers will work with other developers and stakeholders to develop this testing strategy.

An understanding of test automation tools and manual testing practices

It is helpful for QA engineers to be familiar with different automated testing tools like Selenium. They will also need to know how to effectively manually test products in order to identify bugs.

Strong understanding of the Software Development Life Cycle

A good QA engineer should understand the full life cycle of software products in order to effectively plan and test them.

Here are the 7 phases of the Software Development Life Cycle:

1. Planning Stage
2. Analysis Stage
3. Design Stage
4. Software Development Stage
5. Software Testing Stage
6. Implementation and Integration
7. Operations and Maintenance Stage

Do you need to learn how to code become a QA engineer?

Learning programming fundamentals can help QA engineers build automated test suites to save time on manual testing and provide a better understanding of the development implementations by the software developers.

Some widely used programming languages to learn would be Java, C#, JavaScript, and Python.

Choose one language to learn to get an understanding of the programming fundamentals.

Do you need a Computer Science degree to become a QA engineer?

Having a Computer Science degree can help you when it comes to finding your first Quality Assurance job because a lot of employers still prefer a degree. If you have the time and money to pursue that path, that can be a viable option.

Other options would be to go to a bootcamp or teach yourself through online resources. If you are considering a bootcamp, please read through this helpful resource first.

Do you need certifications to become a good QA engineer?

QA certifications can help boost your resume and knowledge base. A lot of these certifications do require that you have degree from an accredited university or a few years of experience.

Here are some QA certifications you can look into:

• Certified Associate in Software Testing (CAST)
• Certified Test Engineer (CSTE)
• Certified Software Quality Analyst (CSQA)

How to get a job as a QA engineer

Once you have learned programming, testing, and QA fundamentals then you can start looking for junior level positions. Make sure to build out a strong résumé and prepare for the QA interview questions using free online interview guides.

If you are self taught, it would help to have a little bit of experience through small contract work. If you are in a degree program, students will have access to internship opportunities and other career resources.

Salaries for QA engineers

Salaries for QA engineers will range depending on experience and location. The average base US salary according to payscale is $72,000.

I hope you enjoyed this article and best of luck on your software journey.

First we're going to talk about what testing is, why is it a good idea to test our code, and the different kinds of tests that can be implemented.

Then we're going to introduce each of the tools we will be using, and finally we'll give practical examples for vanilla JS code, a front-end React app, and a back-end Node app.

Andiamo via!

Table of Contents

• What is testing and why is it valuable

• Different types of tests

  • Manual vs Automated testing

  • Functional vs Non-functional testing

  • Unit vs Integration vs End-to-end testing

  • White box vs Black box vs Grey box testing

• When to test

• Our toolset

  • What is Jest

  • What is Testing library

  • What is Cypress

  • What is Supertest

  • Tools roundup

• How to test vanilla JS code

• How to test a front-end React app with Jest and React testing library

• How to test a front-end React app with Cypress

• How to test a back-end Node app

• Wrap up

What is Testing and Why is it Valuable

Testing is the practice of checking if a piece of software runs as expected. This is often recognized as QA or quality assurance, and aims to reduce to a bare minimum the amount of bugs that get to production.

We test software to identify errors, gaps or missing requirements and fix those things before shipping code to production.

Testing our code thoroughly improves our project's reliability, saves us later bug-fixing time and hence reduces costs, and improves the chance of our customer being fully satisfied with our product.

Here's a nice short video by Fireship introducing some of the concepts we'll talk about later.

Different Types of Tests

Testing practices can be classified in different types according to many factors. Personally I think there's a lot of mumbo jumbo about this topic, with hundreds of terms that often refer to very similar things. So let's keep it simple and review only the most popular terms and what they mean.

This will help clarify the many ways in which a software can be tested and better understand the tools we're going to present later on.

Manual vs Automated testing

Depending on the tools we use to test our software, we can classify testing into manual or automated testing.

Manual testing is the practice of "clicking around" and manually checking all the features our product has, simulating what an actual user would do.

Automated testing is done through code, writing programs that check how our application runs.

There're many testing frameworks and libraries we can use for this. When it comes to functional testing (we're going to see what that means in a sec), most libraries work in a similar way:

• First we define what piece of code we want to test.

• Then we provide that piece of code some sort of input or execute an action on it.

• Then we define what that piece of code should do given the input/action we performed.

• And finally we will compare what that piece of code actually did against what we said it should do.

If it did what we said it should, the test passed. If it didn't, it failed.

Functional vs Non-functional testing

Functional testing refers to the actual features of our product. For example, if we have a blog platform, functional testing should assure the users can create new articles, edit those articles, browse through articles written by other people, and so on.

Non-functional testing refers to anything that's not strictly related to the core features of our product. And that again can be classified into different categories, for example:

• Stress testing checks how infrastructure responds to heavy usage.

• Security testing checks if an application is vulnerable to common hacking attacks.

• Accessibility testing checks if an application is coded in a way that is accessible for people with different disabilities.

Unit vs Integration testing vs End-to-end testing

Another way to classify testing is depending how broad or comprehensive it is.

Unit testing aims to test individual functions, methods or small chunks of code in an independent way. In unit testing, small pieces of code are checked in an isolated way.

Integration testing checks how individual pieces of code interact with each other and work together. In integration testing, we put pieces together and see if they interact correctly.

End-to-end testing, also known as E2E, executes programs in a simulated environment that emulates actual user behavior. Having a website as an example, our code would open in an actual browser and all the features would be executed in the same way a user would use them. E2E testing is a lot like manual testing in that sense, but fully automated.

E2E testing is the most broad or comprehensive type of these three, as it evaluates whole features and behaviors, not specific parts of our code.

White box vs Black box vs Grey box testing

The last classification we're going to see depends on how much our tests focus on implementation details or user experience.

Let's say we have a simple website with a button that, when it gets clicked, it opens a modal. In our code, the button has a click event listener that executes a function. That function changes the CSS class of our modal HTML element, and that gets the modal rendered in the screen.

We talk about "white box" testing when we test implementation details. Following the example, under this paradigm we could test that the button click executes the corresponding function, and that after the function execution, the CSS class of our modal element is changed accordingly.

Another way to do this is to forget about implementation all together and just check if the modal is rendered after the button click. We don't care what the CSS class is, or if the corresponding function is executed or not. We just focus on testing what the user should perceive. That's "black box" testing.

And, as you may have guessed, "grey box" testing is just a combination of the previous two.

One last thing to mention here is that these different types of tests aren't necessarily mutually exclusive. I mean, they can and often are implemented at the same time on the same projects.

It's very common to have both manual and automated testing, functional and non-functional testing, unit and E2E testing ... The idea will always be to try to anticipate and solve the greatest possible number of problems in reasonable time and effort.

When to Test

This may seem like a simple question at first, but there are actually different approaches to this, too.

Some people like to test their app once it's been fully developed. Others like to write tests at the same time they code the application, and test each feature as it's being developed.

Others like to write tests first before anything else, defining in this way the minimum requirements for the program to accomplish. And then they code the app in a way that passes those tests as fast as possible (this is called test driven development or TDD).

Once you have an app or a whole feature developed, and you have a test suite in place (a test suite is a group of tests that check a particular feature or an entire app), another common practice is to run your tests each time you make any kind of modification to the codebase, to verify nothing gets broken.

Lastly, if you have a CI/CD system in place, it's common to automate the execution of tests before any deployment. So that if any test fails, the deployment is stopped and some kind of alert is dispatched (which of course is always better than seeing your app catch fire on prod 🔥😱).

Same as with test types, it's common to test applications at different times. Each company normally has its own testing schedule or practice to follow, tailored to their needs.

Our Toolset

Ok, now that we have a clearer idea of what testing is and the types of tests we can perform, let's review the tools we're going to use in our examples.

As mentioned before, there are a lot of different libraries to choose to run our tests. I chose these four because they are some of the most popular when it comes to JavaScript apps, but know there are more options out there. I'll be naming alternatives for most of the tools we'll use in case you'd like to investigate more. 😉

What is Jest

Jest is a JavaScript test-runner. A test-runner is a piece of software that allows you to run tests to evaluate your app. It's an open-source project maintained by Meta (formerly Facebook), and was first open-sourced in 2014.

Side comment: Every time I say "test runner" I picture this. Am I the only one? 🤔

Test runner, not Blade runner!

Anyway...you can use Jest in projects that use Babel, TypeScript, Node.js, React, Angular, Vue.js, Svelte and other technologies too. You can install Jest through NPM just like any library and it requires very little configuration to start off.

Jest comes installed by default when setting up React apps with create-react-app.

Jest is often also called a testing framework, as it comes with many other built-in features besides just running tests (which is not the case with all test runners). Some of those features are:

• Assertion library: Jest comes with a lot of built-in functions and methods you can use to assert your code (asserting basically means checking if a piece of code behaves like it's expected).

• Snapshot testing: Jest allows you to use snapshots, which are a way of capturing a large object and storing it in memory so you can later on compare it with something else.

• Code coverage: Jest allows you to get code coverage reports of your tests. These reports show what percentage of your code is currently being tested, and you can even see the exact lines of code that aren't currently being covered.

• Mocking library: Jest also works like a mocking library in the sense that it allows you to mock data (like a function or a module) and use that in your tests.

Some well known alternatives to Jest are Mocha, Jasmine, and Karma.

Here's a nice little video explaining what Jest is.

What is Testing Library?

Testing library is not a test runner, but a set of utilities that will work together with a test runner like Jest or Mocha. This utilities are tools we can use to test our code easily and with a deeper focus on user experience (black box testing).

Testing library was developed by Kent C Dodds (who also happens to be one of the best JS teachers on earth, so I recommend that you follow him).

Quoting the official docs:

"The Testing Library family of libraries is a very light-weight solution for testing without all the implementation details.

The main utilities it provides involve querying for nodes similarly to how users would find them. In this way, testing-library helps ensure your tests give you confidence in your UI code."

In plain English, with the testing library we can test UI elements (like a paragraph, a button, a div...) instead of testing the code responsible for rendering the UI.

The principle behind the library is:

"The more your tests resemble the way your software is used, the more confidence they can give you."

... and that's exactly what we mean by "black box" testing. 😉

The testing library is actually a set of libraries, each created to achieve the same objective but adapted to work with different technologies such as React, Angular, Vue, Svelte, React Native and more... That's why you might hear "React-testing-library" or "Vue-testing-library". It's the same thing but adapted to work with different technologies.

React-testing-library comes installed by default when setting up React apps with create-react-app.

An alternative to testing library is Enzyme (a UI testing set of utilities developed by Airbnb).

What is Cypress?

Cypress is an open source test-runner that allows you to execute your projects in an automated browser, in the same way a user would.

With Cypress, we can program what the browser will do (like visit a URL, click a button, complete and submit a form...) and check that each action is matched with the corresponding response.

What's sweet about this is that the testing resembles A LOT to what the user will experience. And since the whole point of making software is the user, the closer we are to their perspective, the closer we should be to catching the most meaningful bugs in our code. (Plus it's really cool to see an automated browser go through your entire app in just a few seconds... 🤓)

Another nice feature of Cypress is "time travel". On Cypress's automated browser we can see all the test's we've written, and simply hover over them to see a graphical snapshot of its result. It's a very useful thing to better understand what's breaking and when.

Even though it can be used for unit and integration testing, Cypress is mostly used for end-to-end testing as it can easily evaluate complete features in a matter of seconds.

You can use Cypress to test anything that runs in a browser, so you can easily implement it on React, Angular, Vue, and so on.

Unlike Jest and React-Testing-Library, Cypress doesn't come pre-installed with create-react-app. But we can easily install it with NPM or your package manager of choice.

Some alternatives to Cypress are Selenium and Puppeteer.

Here's a sweet video by Fireship explaining what Cypress is and how it works.

Side comment: ...and every time I talk about Cypress this plays in my mind. 😎

What is Supertest?

Supertest is a library that simulates HTTP requests. It's super handy to test back-end Node apps together with Jest (as we will see in the coming examples).

Tools roundup

As a quick round-up about this topic:

• Jest is the library that we'll use to write and run tests for JavaScript.

• Testing library works together with Jest, and provides us with functions and methods to test the UI directly, forgetting about the code behind it.

• Cypress runs your app in a simulated browser and checks if actions performed in the UI respond as expected.

• Supertest is a library that mocks HTTP requests and it can be used together with Jest to test back-end apps.

Now let's begin with the fun part ...

Let the testing begin!!

How to Test Vanilla JS Code

Ok, let's start by testing some simple vanilla JS code. The idea here is to see how we can implement Jest in our project and learn the basics of how it works.

Let's start by creating a new directory in our machine and creating a Node app with npm init -y. Then install Jest by running npm i -D jest (-D saves it as a development dependency).

Now you should see something like this in your package.json file: "devDependencies": { "jest": "^27.5.1" } .

And speaking about it, in your package.json, replace your test script with "test": "jest". This will allow us to later run our tests by running npm test. ;)

Your entire package.json file should look something like this:

{
  "name": "vanillatesting",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "jest"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "jest": "^27.5.1"
  }
}

Cool, we're all set to write some JS we can actually test! Create an index.js file and put this code in it:

// index.js
function isPalindrome(string) {
    // O(n)
    // Put a pointer at each extreme of the word and iterate "inwards"
    // At each iteration, check if the pointers represent equal values
    // If this condition isn't accomplished, the word isn't a palindrome
    let left = 0
    let right = string.length-1

    while (left < right) {
        if (string[left] === string[right]) {
            left += 1
            right -= 1
        }
        else return false
    }

    return true
}

This function is a palindrome checker. It receives a string as parameter, and returns true if the string is a palindrome and false if it's not. (This is a classic technical interview question btw, but that's stuff for another article.🤫)

See that we're exporting the function too. Side comment: If you'd like to know more about this and how JS modules work, I recently wrote an article about it.

Great, so now let's test this function and see if it works as expected. Let's create a file called index.test.js.

This file is where we'll write our tests. The suffix we're using (.test.js) is important here, as Jest will automatically identify the .test files and execute them when we order Jest to test our project.

Jest also identifies files with the .spec suffix, like index.spec.js (for "specification", which refers to the requirements of your project). Personally I prefer .test as it feels more explicit to me, but both work the same.

Now let's write our first tests! Put this within your index.test.js file.

// index.test.js
isPalindrome = require('./index.js')

test('neuquen is palindrom', () => {
    expect(isPalindrome("neuquen")).toBe(true)
})

test('bariloche is not palindrom', () => {
    expect(isPalindrome("bariloche")).toBe(false)
})

Let's recap what we're actually doing:

1. Require the function we want to test: isPalindrome = require('./index.js')

2. The test() function is provided by Jest and within it we will put the code we want Jest to execute.

3. test() takes two parameters. The first is a test description, which is a distinctive name that will show on our console when the test is run. We'll see an example in a sec.

4. The second parameter is a callback, which contains the actual testing code.

5. Within this callback we're calling the expect() function (also provided by Jest). expect() takes our function as parameter, which itself is receiving a parameter we made up.

6. Last, we chain the .toBe() function (provided by Jest too) and as parameter we pass it the value we expect isPalindrome() to return for each case. ("neuquen" is a palindrome so our function should return true, and "bariloche" is not, so it should return false.)

One of the things I like the most about Jest is how easy it is to set it up. Another thing I like a lot is how self explanatory its syntax is. Notice that we can easily understand what our tests will evaluate by just reading them.👌

Now let's try this! If we run npm test in our console, we should get the following:

// console
> jest PASS 
./index.test.js
✓ neuquen is palindrom (1 ms)
✓ bariloche is not palindrom

Test Suites: 1 passed, 1
total Tests:       2 passed, 2
total Snapshots:   0
total Time:        0.244 s
Ran all test suites.

Congratulations, you just passed your first Jest test ever.

To see how a failing test looks too, let's change our function by editing the return lines.

// index.js
function isPalindrome(string) {
    // O(n)
    // Put a pointr at each extreme of the word and iterate "inwards"
    // At each iteration, check if the pointers represent equal values
    // If this condition isn't accomplished, the word isn't a palindrome
    let left = 0
    let right = string.length-1

    while (left < right) {
        if (string[left] === string[right]) {
            left += 1
            right -= 1
        }
        else return 1
    }

    return 2
}

Now you should get something like this:

// console
> vanillatesting@1.0.0 test
> jest

 FAIL  ./index.test.js
  ✕ neuquen is palindrom (4 ms)
  ✕ bariloche is not palindrom

  ● neuquen is palindrom

    expect(received).toBe(expected) // Object.is equality

    Expected: true
    Received: 2

      3 | // describe('isPalindrome function', () => {
      4 |   test('neuquen is palindrom', () => {
    > 5 |     expect(isPalindrome("neuquen")).toBe(true)
        |                                     ^
      6 |   })
      7 |
      8 |   test('bariloche is not palindrom', () => {

      at Object.<anonymous> (index.test.js:5:37)

  ● bariloche is not palindrom

    expect(received).toBe(expected) // Object.is equality

    Expected: false
    Received: 1

       7 |
       8 |   test('bariloche is not palindrom', () => {
    >  9 |     expect(isPalindrome("bariloche")).toBe(false)
         |                                       ^
      10 |   })
      11 | // })
      12 |

      at Object.<anonymous> (index.test.js:9:39)

Test Suites: 1 failed, 1 total
Tests:       2 failed, 2 total
Snapshots:   0 total
Time:        0.28 s, estimated 1 s
Ran all test suites.

See that you get a nice description of what tests failed and at which point they failed. In our case they failed when we asserted (checked) the return values.

This is very useful and we should always pay attention to these descriptions, as some times our tests may fail because they're not written correctly. And we don't normally write tests for our tests, yet... 😅 So when you see a failing test, first check that it's working as expected and then go review your actual code.

Now let's add and test another function to show some more Jest features:

// index.js
function twoSum(nums, target) {
    // O(n)
    // Iterate the array once
    // At each iteration, calculate the value needed to get to the target, which is target - currentValue
    // If the neededValue exists in the array, return [currentValue, neededValue], else continue iteration
    for (let i = 0; i < nums.length; i++) {
        const neededNum = target - nums[i]
        if (nums.indexOf(neededNum) !== -1 && nums.indexOf(neededNum) !== i) return [nums[i], nums[nums.indexOf(neededNum)]]
    }
    return false
}

module.exports = { isPalindrome, twoSum }

This is another classic interview question. The function takes two parameters, an array of numbers and a target value number. What it does is to identify if there are two numbers in the array that add up to the second parameter value. If the two values exist in the array, it returns them in an array, and if they don't, it returns false.

Now let's write some tests for this:

({ isPalindrome, twoSum } = require('./index.js'))

...

test('[2,7,11,15] and 9 returns [2, 7]', () => {
    expect(twoSum([2,7,11,15], 9)).toEqual([2,7])
})

test('[3,2,4] and 6 returns [2, 4]', () => {
    expect(twoSum([3,2,4], 6)).toEqual([2,4])
})

test('[3,2,4] and 10 returns false', () => {
    expect(twoSum([3,2,4], 10)).toBe(false)
})

See that the structure is almost the same, except we're using a different matcher in two of the tests, toEqual().

Matchers are the functions Jests provides us with to evaluate values. There are many types of matchers that can be used for many different occasions.

For example, .toBe() is used to evaluate primitives like strings, numbers, or booleans. toEqual() is used to evaluate objects (which covers pretty much everything else in Javascript).

If you need to compare the return value with a number you could use .toBeGreaterThan() or toBeGreaterThanOrEqual() and so on...

To see a full list of the available matchers, check the docs.

If we run our tests now, we will get the following:

> vanillatesting@1.0.0 test
> jest

 PASS  ./index.test.js
  ✓ neuquen is palindrom (2 ms)
  ✓ bariloche is not palindrom
  ✓ [2,7,11,15] and 9 returns [2, 7] (1 ms)
  ✓ [3,2,4] and 6 returns [2, 4]
  ✓ [3,2,4] and 10 returns false (1 ms)

Test Suites: 1 passed, 1 total
Tests:       5 passed, 5 total
Snapshots:   0 total
Time:        0.256 s, estimated 1 s
Ran all test suites.

That's cool, but our tests results look a bit messy. And as our tests suite grows, it will probably be harder to identify each separate result.

To help us with this, Jest provides us with a describe() function, which we can use to group tests together and show results in a more schematic way. We can use it like this:

({ isPalindrome, twoSum } = require('./index.js'))

describe('isPalindrome function', () => {
  test('neuquen is palindrom', () => {
    expect(isPalindrome("neuquen")).toBe(true)
  })

  test('bariloche is not palindrom', () => {
    expect(isPalindrome("bariloche")).toBe(false)
  })
})

describe('twoSum function', () => {
  test('[2,7,11,15] and 9 returns [2, 7]', () => {
    expect(twoSum([2,7,11,15], 9)).toEqual([2,7])
  })

  test('[3,2,4] and 6 returns [2, 4]', () => {
    expect(twoSum([3,2,4], 6)).toEqual([2,4])
  })

  test('[3,2,4] and 10 returns false', () => {
    expect(twoSum([3,2,4], 10)).toBe(false)
  })
})

The first parameter is the description we want to show for the given group of tests, and the second is a callback that contains our tests. Now if we run npm test again, we get this 😎:

// console
> vanillatesting@1.0.0 test
> jest

 PASS  ./index.test.js
  isPalindrome function
    ✓ neuquen is palindrom (2 ms)
    ✓ bariloche is not palindrom
  twoSum function
    ✓ [2,7,11,15] and 9 returns [2, 7] (1 ms)
    ✓ [3,2,4] and 6 returns [2, 4]
    ✓ [3,2,4] and 10 returns false

Test Suites: 1 passed, 1 total
Tests:       5 passed, 5 total
Snapshots:   0 total
Time:        0.216 s, estimated 1 s
Ran all test suites.

How to Test a Front-end React App with Jest and React Testing Library

Now that we know the basics of Jest, let's hop on to see how we can combine it with Testing library to test a React app.

For this we're going to use a dead simple example. Just a page with random text, a button that toggles another piece of text, a text input, and a button that toggles the rendering of the input.

Take into account we'll be using create-react-app to create this app (which has Jest and Testing library installed by default). If you're not using create-react-app, you might need to install both libraries and add some extra config.

We're not going to see any React code here, we're just going to focus on the tests.

The folder structure of our project is the following:

> src
    > components
        - About.jsx
    - App.jsx
    - Index.js
    - setupTests.js

The setupTests.js file is important here. It's created by default with create-react-app with this content:

// jest-dom adds custom jest matchers for asserting on DOM nodes.
// allows you to do things like:
// expect(element).toHaveTextContent(/react/i)
// learn more: https://github.com/testing-library/jest-dom
import '@testing-library/jest-dom';

It globally imports the jest-dom library provided by Testing library, which gives us additional Jest matchers we can use to test the DOM (like toHaveTextContent(), toBeInTheDocument(), etc).

We're going to see examples in a bit, but know that some of the functions and matchers we'll use come from here.

Regarding our tests files, the common practice is to have a different test file for each component we're testing.

Regarding where to place them, two common practices are to have them all together in a single folder, like __tests__ or similar, or to have each test file in the same folder as the component it's testing.

I prefer the later as I'll often be hopping from the component code to the test code, and it's nice to have them nearby. But trully it doesn't matter. As long as we use the .test or .spec suffixes, Jest will identify and run the files anyway.

Having created our tests files, our folder structure should look like this:

> src
    > components
        - About.jsx
        - About.test.jsx
    - App.jsx
    - Index.js
    - setupTests.js

Cool! Let's start by testing our About component.

First lets test that it's rendering correctly, like this:

// About.test.jsx
import { render, screen } from '@testing-library/react'
import About from './About'

describe('About', () => {

  test('About renders correctly', () => {
    render( <About/> )
    expect(screen.getByText("I'm the about page!")).toBeInTheDocument()
  })

})

• See that we start by importing two things from Testing library: import { render, screen } from '@testing-library/react'.

The render function takes a React component as a parameter and it will render it so we can test it.

screen is an object that comes with lots of queries we can use to test the UI directly, skipping implementation details and focusing on what the user will actually see.

• Then we import our About component: import About from './About'

• We use the describe and test Jest functions previously mentioned.

• We render the About component: render( <About/> )

• We use the expect Jest function, and as a parameter we use the screen object provided by Testing library. We use its getByText query, which scans the React component for the text we pass as parameter.

• To end, we use the Testing library's .toBeInTheDocument() matcher, which just checks if the previous query result is being rendered.

Then we can test that the "Switch state" toggle button works correctly, like this:

// About.test.jsx
import { render, screen } from '@testing-library/react'
import userEvent from '@testing-library/user-event'
import About from './About'

describe('About', () => {

  ...

  test('Switch state works correctly', async () => {
    render( <About/> )

    expect(screen.getByText("It's on!")).toBeInTheDocument()
    userEvent.click(screen.getByText('Switch state'))
    expect(screen.getByText("It's rolling!")).toBeInTheDocument()
    userEvent.click(screen.getByText('Switch state'))
    expect(screen.getByText("It's on!")).toBeInTheDocument()
  })

})

See that we import an additional utility called userEvent. This is an object that contains many methods we can use to simulate user fired events, like clicks, hovers, writting in an input, and so on.

• We first check that the default string is rendered: expect(screen.getByText("It's on!")).toBeInTheDocument()

• Then we simulate a click and check that the string changes in the screen:

userEvent.click(screen.getByText('Switch state'))
expect(screen.getByText("It's rolling!")).toBeInTheDocument()

• And last we simulate another click and check that the string reverses back to default:

userEvent.click(screen.getByText('Switch state'))
expect(screen.getByText("It's on!")).toBeInTheDocument()

To finish, we're going to write another test to verify that the text input and its toggle work correctly.

import { render, screen } from '@testing-library/react'
import userEvent from '@testing-library/user-event'
import About from './About'

describe('About', () => {

  ...

  test('Input works correctly', async () => {
    render( <About/> )

    userEvent.type(screen.getByTestId("testInput"), "Testing the test")
    userEvent.click(screen.getByText("Print input"))

    expect(screen.getByText("Testing the test")).toBeInTheDocument()

    userEvent.click(screen.getByText("Print input"))
    expect(screen.queryByText("Testing the test")).not.toBeInTheDocument()
  })


})

• Again we use the userEvent to simulate text being writen into our input element: userEvent.type(screen.getByTestId("testInput"), "Testing the test")

• Then we simulate a click on the toggle button, and check for the input text to be in the document:

userEvent.click(screen.getByText("Print input"))
expect(screen.getByText("Testing the test")).toBeInTheDocument()

• And we close by simulating another click and checking that the test is no longer present:

userEvent.click(screen.getByText("Print input"))
expect(screen.getByText("Testing the test")).toBeInTheDocument()

You can see how nice the utilities provided by Testing libraries are, and how easy it is to combine them with Jest. 🤓

We can run this specific test file by running npm test -- About.test.jsx and this is the result we get:

// console
PASS  src/components/About.test.jsx
  About
    ✓ About renders correctly (34 ms)
    ✓ Switch state works correctly (66 ms)
    ✓ Input works correctly (67 ms)

Test Suites: 1 passed, 1 total
Tests:       3 passed, 3 total
Snapshots:   0 total
Time:        0.997 s, estimated 1 s
Ran all test suites matching /About.test.jsx/i.

The last Jest feature I'd like to show you is test coverage. You can obtain a coverage report by running npm test -- --coverage.

This will run your tests normally and at the end of the results report you should see something like this:

// console
...

----------------|---------|----------|---------|---------|-------------------
File            | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s 
----------------|---------|----------|---------|---------|-------------------
All files       |      75 |      100 |   85.71 |      70 |                   
 src            |       0 |      100 |       0 |       0 |                   
  App.jsx       |       0 |      100 |       0 |       0 | 7                 
  App.t.js      |       0 |        0 |       0 |       0 |                   
  index.js      |       0 |      100 |     100 |       0 | 5-6               
 src/components |     100 |      100 |     100 |     100 |                   
  About.jsx     |     100 |      100 |     100 |     100 |                   
----------------|---------|----------|---------|---------|-------------------

In the report we can see that our About.jsx component is completely covered, but our App.jsx and index.js files are not being tested.

This feature is very handy when working on big projects and you want to quickly know if most of your code is being tested correctly.

How to Test a Front-end React App with Cypress

We've talked a lot about Jest, so now let's take a look at how we can test our app using Cypress.

We'll start off by installing Cypress by running npm i -D cypress.

This should add this to our package.json:

"devDependencies": {
    "cypress": "^9.5.4"
}

Then we'll run npx cypress open. This will open the Cypress browser, and create a cypress directory within our project. Within this directory you'll find examples, documentation, and config options.

You'llo also find an "integration" folder, in which we have to put our tests. So let's create our About.test.js file in that folder and replicate the same test examples we've seen with Jest:

// About.test.js
describe('AboutPage', () => {
    it('Renders correctly', () => {
        cy.visit('http://localhost:3000/about')
        cy.contains("I'm the about page!")
    })

    it('switch btn toggles text', () => {
        cy.contains("It's on!")
        cy.get('.switchBtn').click()
        cy.contains("It's rolling!")
        cy.get('.switchBtn').click()
        cy.contains("It's on!")
    })

    it('Input works correctly', () => {
        cy.get(".testInput").type("Testing the test")
        cy.get('.printInputBtn').click()
        cy.contains("Testing the test")

        cy.get('.printInputBtn').click()
        cy.contains("Testing the test").should('not.exist')
    })
})

• The describe function works the same as in jest.

• it() is the same as the test() function we've previously seen.

• In the first test we tell the browser to visit our app's URL and check that the corresponding text is rendered:

cy.visit('http://localhost:3000/about')
cy.contains("I'm the about page!")

• Then we check that the default toggle text is rendered, simulate a click and check that it changes accordingly:

cy.contains("It's on!")
cy.get('.switchBtn').click()
cy.contains("It's rolling!")
cy.get('.switchBtn').click()
cy.contains("It's on!")

• And to end we simulate a text input, simulate a click, and check that the input text is rendered:

cy.get(".testInput").type("Testing the test")
cy.get('.printInputBtn').click()
cy.contains("Testing the test")

cy.get('.printInputBtn').click()
cy.contains("Testing the test").should('not.exist')

The syntaxt is slightly different than Jest, but the idea and structure are pretty much the same.🤙

Now if we run npx cypress open again, a window should open with this content:

We can click on "Run integration spec" and our test will run automatically in the mock browser. After the tests have run, on the left panel we'll see the results:

We can open those results to see each step the test executed. If we hover over each step, we'll see it executed in the browser in real time. A real sweet feature of Cypress.👌👌

As you can see, it's very easy to set up tests with Cypress. And if you're already familiar with Jest, you can quickly pick it up as the syntax is not that different.

If you're wondering if it makes sense to use both Jest and Cypress as test runners in the same project, I think this stack-overflow answer sums it up quite nicely.

How to Test a Back-end Node App

Now that we have a basic understanding of the ways we can tests a front-end app, let's cross the river and see how we can use similar tools to test a back end app.

For this we'll use a simple Node and Express API with just 3 endpoints.

Create a directory and run npm init -y to create a Node app. Run npm i express to install Express, and then run npm i -D jest supertest to install both Jest and Supertest as development dependencies.

Inside your package.json, add "scripts": { "test": "jest" }. Your entire package.json should look like this:

{
  "dependencies": {
    "express": "^4.17.3"
  },
  "devDependencies": {
    "jest": "^27.5.1",
    "supertest": "^6.2.2"
  },
    "scripts": {
    "test": "jest"
  }
}

Then create an app.js file and put this code in it:

// app.js
/* Import and initialize express */
const express = require('express')
const app = express()
const server = require('http').Server(app)
/* Global middlewares */
app.use(express.json())

/* Endpoint 1 */
app.get('/', async (req, res) => {

    try {
        res.status(200).json({ greeting: "Hello there!" })
    } catch (err) {
        res.status(500).send(err)
    }
})

/* Endpoint 2 */
app.get('/isPalindrome', async (req, res) => {

    try {
        const string = req.body.string
        let result = true        
        let left = 0
        let right = string.length-1

        while (left < right && result) {
            if (string[left] === string[right]) {
                left += 1
                right -= 1
            }
            else result = false
        }

        res.status(200).json({ result: result })

    } catch (err) {
        res.status(500).send(err)
    }
})

/* Endpoint 3 */
app.get('/twoSum', async (req, res) => {

    try {
        const nums = JSON.parse(req.body.nums)
        const target = JSON.parse(req.body.target)

        let result = false

        for (let i = 0; i < nums.length; i++) {
            const neededNum = target - nums[i]
            if (nums.indexOf(neededNum) !== -1 && nums.indexOf(neededNum) !== i) result = [nums[i], nums[nums.indexOf(neededNum)]]
        }

        res.status(200).json({ result: result })

    } catch (err) {
        res.status(500).send(err)
    }
})

/* Export server object */
module.exports = server

/* Initialize server */
server.listen(3001, () => console.log('Server is listening.') )
server.on('error', error => console.error(error) )

As you can see, endpoint 1 just returns a greeting message. Endpoint 2 and 3 are adaptations of the functions we've seen in our vanilla JS examples. They now receive the parameters within the request and the return values go in the response. 😉

Now the testing! Create an app.test.js file and put this code within it:

// app.test.js
const supertest = require('supertest') // Import supertest
const server = require("./app") // Import the server object
const requestWithSupertest = supertest(server) // We will use this function to mock HTTP requests

afterEach(done => { // afterEach function is provided by Jest and executes once all tests are finished
    server.close() // We close the server connection once all tests have finished
    done()
})

test('GET "/" returns greeting', async () => {
    const res = await requestWithSupertest.get('/')
    expect(res.status).toEqual(200)
    expect(res.type).toEqual(expect.stringContaining('json'))
    expect(res.body).toEqual({ greeting: "Hello there!" })
})

describe("/isPalindrome", () => {
    test('GET "/isPalindrome" neuquen returns true', async () => {
        const res = await requestWithSupertest.get('/isPalindrome').set('Content-type', 'application/json').send({ "string":"neuquen" })
        expect(res.status).toEqual(200)
        expect(res.type).toEqual(expect.stringContaining('json'))
        expect(res.body).toEqual({ result: true })
    })

    test('GET "/isPalindrome" bariloche returns true', async () => {
        const res = await requestWithSupertest.get('/isPalindrome').set('Content-type', 'application/json').send({ "string":"bariloche" })
        expect(res.status).toEqual(200)
        expect(res.type).toEqual(expect.stringContaining('json'))
        expect(res.body).toEqual({ result: false })
    })
})

describe("/twoSum", () => {
    test('GET "/twoSum" [2,7,11,15] and 9 returns [7, 2]', async () => {
        const res = await requestWithSupertest.get('/twoSum').set('Content-type', 'application/json').send({ "nums":"[2,7,11,15]", "target": "9" })
        expect(res.status).toEqual(200)
        expect(res.type).toEqual(expect.stringContaining('json'))
        expect(res.body).toEqual({ result: [7, 2] })
    })

    test('GET "/twoSum" [3,2,4] and 6 returns [4, 2]', async () => {
        const res = await requestWithSupertest.get('/twoSum').set('Content-type', 'application/json').send({ "nums":"[3,2,4]", "target": "6" })
        expect(res.status).toEqual(200)
        expect(res.type).toEqual(expect.stringContaining('json'))
        expect(res.body).toEqual({ result: [4, 2] })
    })

    test('GET "/twoSum" [3,2,4] and 10 returns false', async () => {
        const res = await requestWithSupertest.get('/twoSum').set('Content-type', 'application/json').send({ "nums":"[3,2,4]", "target": "10" })
        expect(res.status).toEqual(200)
        expect(res.type).toEqual(expect.stringContaining('json'))
        expect(res.body).toEqual({ result: false })
    })
})

Let's analyze what we're doing:

• We mock the request with requestWithSupertest.get('/')

• Then we "break" the res object in pieces and assert each part of it:

  • Check the response status: expect(res.status).toEqual(200)

  • Check the response format: expect(res.type).toEqual(expect.stringContaining('json'))

  • Check the response body content: expect(res.body).toEqual({ greeting: "Hello there!" })

The other tests are really similar, except we're sending data in the mock requests bodies, like this:

const res = await requestWithSupertest.get('/isPalindrome').set('Content-type', 'application/json').send({ "string":"bariloche" })

As you can see, testing in this way is really simple once you're familiar with Jest. We just need a little help by Supertest to mock the HTTP request and the rest is just asserting the response. 👏👏

We can run our tests with npm test and we should get the following response:

// console
 PASS  ./app.test.js
  ✓ GET "/" returns greeting (46 ms)
  /isPalindrome
    ✓ GET "/isPalindrome" neuquen returns true (18 ms)
    ✓ GET "/isPalindrome" bariloche returns true (3 ms)
  /twoSum
    ✓ GET "/twoSum" [2,7,11,15] and 9 returns [7, 2] (4 ms)
    ✓ GET "/twoSum" [3,2,4] and 6 returns [4, 2] (3 ms)
    ✓ GET "/twoSum" [3,2,4] and 10 returns false (2 ms)

Test Suites: 1 passed, 1 total
Tests:       6 passed, 6 total
Snapshots:   0 total
Time:        0.552 s, estimated 1 s
Ran all test suites.

Wrap up

And that's it! We've covered the basics of four very popular tools that will allow you to test both the front-end and back-end of your JS apps.

Of course there's much more to all the tools we've seen and many features we haven't covered. But the idea was to give you an introduction so you can take your first steps in the testing world.

As always, I hope you enjoyed the article and learned something new. If you want, you can also follow me on linkedin or twitter.

Cheers and see you in the next one! =D

Stakeholders are always pushing searching for faster solutions. Project managers are rushing products to the delivery table. And there is isn't necessarily anything wrong with doing it this way.

But it can be challenging for QA testing teams who have to embrace this super quick timetable, and then convince the developer team to get onboard.

Software teams know how badly test management teams want to reach deadlines when they are coping with constant shifts and changes.

In this scenario, all blame falls on the test team, who might quickly become a convenient target when production bugs are shipped. And any delays regarding specifications and implementation in the upstream software development lifecycle (SDLC) phase will make matters worse.

Beyond that, the demands of the stakeholders who want the product available ASAP add fuel to the fire. All of this puts the testing team under immense pressure.

A QA team must have a rational and empirical response to this challenge. The answer to this problem lies in risk-based testing (RBT) which can help the testing team deliver on their deadlines.

Your test team can save a ton of effort with the RBT strategy, and it produces great savings while delivering the project on time.

What is Risk-Based Testing? RBT Basics

In risk-based test management, you find the greatest market threats (that would have a detrimental effect on the enterprise as defined by the consumer) early in the development cycle and then you can counter them by taking preventive steps.

These market threats may include increased costs, consumer disappointment, poor user interface, and lost customers. And RBT can help you mitigate them by conducting testing in such a way that, even if a customer gets an error, they can go on using the application (and the organization isn't greatly affected).

RBT involves testing that's performed on the basis of product risks. It's useful in helping you discover how likely a specific feature or capability is to fail in final output/production. It'll also help you determine this failure's effect on the company in terms of cost and other damages, well ahead of time. And it does this by using a prioritization strategy for test cases.

So Risk-Based evaluation works by prioritizing the testing of a product or software's functionality, components, and features. This prioritization is based on the risk of the possibility of failure in the development of the feature or functionality and its effect on clients.

How to Assess Risk with RBT

RBT includes risk management and test prioritization depending on the risk factor of each test. The risk factor is the result of the likelihood and possible effect of future risk.

But how are we able to come up with those values? To allocate a risk factor to a specific risk, here are three requirements to consider.

Code Complexity

Cyclomatic Complexity

The more ambiguous code is, the more likely flaws will be discovered. Simply put, it is more likely that more complicated code contains glitches. The reverse is also true: the cleaner and simpler the code, the less likely it is to contain errors.

The problem then is how you should calculate code complexity. One measure is cyclomatic complexity which refers to the number of potential paths. The minimal number of test cases you require to thoroughly test a feature is determined by that number. In fact, code with high cyclomatic complexity, which may expand the number of errors, can be more difficult to read.

Churn or number of modifications to the code

Churn here implies the number of modifications that a specified file or module experiences. An application area that developers alter more frequently is more likely to have bugs than one which they hardly ever touch.

A great example of this is an application for e-commerce that has been in development for many years. The software team is likely to make frequent adjustments to the offers, introduce new features, and try experiments to boost purchases. The affected sections of the code shift regularly and are thus more likely to produce errors.

When churn is significant, extensive testing should be proposed for risk-based planning: more UI tests, more unit tests, more integration tests, and so on.

Criticality

Criticality involves the calculation of a software defect's influence on your application. Criticality is not spread equally through a codebase. Many systems are expected to have a high-critical code core on which the remainder of the application relies.

If the code does not perform properly, several parts of the software could be compromised and it would probably create defects. If it applies to safety-related software, a critical flaw may cause data loss, sensitive data disclosure, and maybe even endanger lives.

When measuring the effect portion of the risk factor, criticality may be useful.

If a bug arises in the application's most important environment, the effects could be much more extreme than if it existed in some area that included utility code only.

We rely on our algorithms in our test management frameworks to classify site components. If web components can not be detected by the algorithm and related core code, the UI checks if our customers will make mistakes or fail, creating frustration, reworks, service calls, and a lack of faith in our solution.

Worse still, if false positives are generated by our application, it will interrupt the CI/CD process and impede the implementation of a legitimate application.

Smart Testing Is the Solution

Software development companies need a better strategy when it comes to software testing than "just test everything." Not all code is equal, not all threats are credible, and not all bugs trigger equal harm.

In order to make informed decisions about how to distribute capital and test them more effectively, tech companies must take these RBT considerations into account.

If you want to remain on top of your Risk-Based Software Testing, you can check out Kualitee – we're a software testing and information security company.

Code reviews bring many benefits to the process of writing and delivering software:

• Ensures consistency through your codebase.
• Teaches all members of the review (helps knowledge transfer).
• Builds contextual awareness regarding what might affect other parts of the team
• Helps avoid breaking builds
• Casts fresh eyes over a code change to search for optimisations and simplifications in the change.
• Promotes quality and helps make sure no one forgot or missed anything.

Google has 1,918 repositories on their GitHub across multiple languages, and even more that aren't open source.

One of their codebases is shared by over 25,000 Googlers, and typically has 40,000 commits daily (16,000 human changes, and 24,000 by automated systems). Each day it has to serve around 800,000 queries per second during peak times.

Google has published their engineering practices, so let's see how they review code at their scale, with so many commits a day.

Google refers to changes in their codebases as CLs standing for change lists. It's just a unit of work / piece of code going through review.

How to Prepare for Code Reviews

Firstly, before you even think about the code review process, try and focus on who will do the review. Try to pick a subject matter expert. Pick someone who is familiar with this codebase or area of the codebase.

Sometimes this may even mean having different people review different parts of the code. But less than 25% of code review's at Google have more than one reviewer.

Getting someone to respond in a timely manner is also important. To avoid bottle necks and overloading individuals, just CC reviewers onto the change to review it at their convenience if they want to.

How to Run Fast Code Reviews

Code reviews should be completed quickly. The maximum length of time for a review should be one business day.

Why the urgency? I've personally had QA's sometimes take weeks or longer.

• It becomes a blocker. Although the author of the code moves onto new work, new changes start to form a back log, and the delays can build up to weeks or months.
• Developers feel frustrated. If the reviewer asks for major changes but only responds every 3 days, it is frustrating for the developer working on that change. But with quick responses, whenever you require explanation of exactly what you need to do, the frustration fades away.
• Code quality can degrade. If your reviews are always slow, developers are less likely to do code clean ups, refactoring work or general code improvement ("If my reviewer won't reply for 4 days what's even the point?"), and the code quality submitted in the reviews is more likely to go down.

The main reason code reviews can be fast is because they're small. Rather than sending a 1,000 line CL, they're separated into multiple CL's – pushing 10 smaller changes of 100 lines for example.

Google also has emergencies where code changes have to be made quickly to resolve major problems in production. In these cases, code quality is relaxed. This review should immediately become the reviewers first priority. There are some examples of emergencies Google has faced here if you're curious.

Google's Code Review Standards

The key rule Google emphasizes is:

Reviewers should generally approve a change once it definitely improves the overall code health, even if it isn't perfect.

The key point Google seem to be making is that perfect code doesn't exist. If it makes it better, that is enough.

It's definitely a balancing act between something being better, and how much better it could be. If you could add more feedback to make significant improvements, there may need to be more work done on the code.

What Googlers Look for in Code Reviews

When doing code reviews, Google focuses on these elements according to their engineering practices documentation:

Design: Is the code well-designed and appropriate for your system? Does this change belong in your codebase, or in a library? Does it integrate well with the rest of your system?

Functionality: Does the code behave as the author likely intended? Is the way the code behaves good for its users? Mostly, we expect developers to test CLs well-enough that they work correctly by the time they get to code review.

Complexity: Could the code be made simpler? Would another developer be able to easily understand and use this code when they come across it in the future? Is it over-engineered for its current use case?

Tests: Does the code have correct and well-designed automated tests? Will the tests actually fail when the code is broken? Will they start producing false positives? Does each test make simple and useful assertions?

Naming: Did the developer choose clear names for variables, classes, methods, etc.?

Comments: Are the comments clear and useful? Ensure where sensible comments explain why something is being done, rather than how.

Style & Consistency: Does the code follow our style guides?

Documentation: Did the developer also update relevant documentation?

Google has style guides for multiple languages like C++, Swift, HTML/CSS, Lisp, JavaScript and more.

Having a document that everyone can refer to helps standardise the code. It also helps clarify what the expectations are in the review process.

How Googlers Review Code

There is a three-stage process for code review in Google's engineering practices. Here's a list of things you'd need to cover if you were to review:

1. Get a high level overview of the change
2. Examine the main parts of the change
3. Look through the rest of the code, in a sensible order

Let's go over each step in more detail.

1. Get a High Level Overview of the Code Change

Look at the code change's description/summary. Does it all make sense? For example, if someone is changing a codebase that is being deleted next week, push back on the change courteously, and explain why the change doesn't look needed anymore.

It's inefficient if people are spending time on work that isn't actually needed, so have a look at your development life cycle and see why this might be happening. The earlier you can catch these issues the better.

In order to get a high level overview of the code, you may want to briefly scan the code components to see how it all works together.

If you see a serious architectural problem or something majorly wrong, you should share your observations immediately at this stage. Even if you don't have time to review every single other element of the review. Reviewing the rest may well even end up being a waste of time if the architectural problems are severe enough.

Aside from this, there are 2 major reasons why you should send your review comments immediately:

• Googlers will often email a change, and then immediately start new work based on that change. If the change in review need serious adjustments, they may be building on something that needs to be significantly changed.
• Big changes to the CL may take a while to finish. Developers all have deadlines and it's courteous to try and help them meet them by letting them start rework as soon as possible.

2. Examine the Main Parts of the Code Change

Once you have had an overview, review the "main" parts of the change.

Find the file or files that are central to the CL. Often, there is one file that has the largest number of changes, and it’s the major piece of the CL.

Spend most of your focus reviewing these pieces. This provides context to all the smaller pieces, and generally makes it faster to review.

If the CL is too large for you to get a good overview and understand the flow, it's a good sign the developer should be splitting up their CL into smaller changes.

3. Look Through the Rest of the Code, in a Sensible Order

Once you have a good overview of the change, it's time to drill down into the details and go file by file.

Each reviewer generally does this differently. Some go in the order that the version control presents to them and some pick a particular order. Choose what makes sense to you. It's important to just review everything, and miss nothing.

Review Every Line of Code

Don't miss or skim important details. Sometimes there might be config files, or generated code you can scan briefly looking for irregularities. But your job here is to be very thorough and carefully scrutinise the code.

Make sure you think about bugs, race conditions, alternate approaches, conciseness, readability, and so on.

If you can't understand the code, it's highly likely other developers won't be able to either, and should ask the developer to clarify it.

Understand the Context

Often times the version control software will only show the changed lines. But it's important to read the lines before and after, or the whole file, to ensure you understand exactly what the change is doing.

You may only see someone adding 2 more if blocks in a change. But if you look at the context you might see that the if else block now has 15 different if's inside it. Then you can reject the code change and properly improve the code quality in this area, rather than making it worse.

Remember code degradation happens slowly, with every change that goes in. Our main aim is to ensure the quality trends consistently up, and we never go backwards.

How Googlers Write Code Feedback

Google has some specific sections covering how to do code reviews courteously, and respectfully. This isn't in opposition to being clear and as helpful as you can to the developer. The golden rule here is to make comments about the code and not the developer.

What to Do:

####

Ask for the why: If it is unclear why the author is doing things a certain way, feel free to ask why they made a particular change. Not knowing is OK, and asking “Why” leaves a written record that will help answer this question in the future. (And sometimes, “I'm curious, why did you decide to do it that way?” can help the author to rethink their decision.)

Find an end: If you like things neat, it’s tempting to go over a code review over and over until it’s perfect, dragging it out for longer than necessary. It’s soul-deadening for the recipient, though. Keep in mind that “LGTM” does not mean “I vouch my immortal soul this will never fail”, but “looks good to me”. If it looks good, move on. (That doesn’t mean you shouldn’t be thorough. It’s a judgment call.) And if there are bigger refactorings to be done, move them to a new CL. (Source)

What Not to Do

Don't use extreme or very negative language: Please don't say things like “no sane person would ever do this” or “this algorithm is terrible”, whether it’s about the change you're reviewing or about the surrounding code. While it might intimidate the reviewee into doing what you want, it’s not helpful in the long run - they will feel incapable, and there is not much info in there to help them improve. “This is a good start, but it could use some work” or “This needs some cleanup” are nicer ways of saying it. Discuss the code, not the person.

Don't bikeshed: Always ask yourself if this decision really matters in the long run, or if you're enforcing a subjective preference. It feels good to be right, but only one of the two participants can win that game. If it’s not important, agree to disagree, and move on. (Source)

Remember code reviews can sometimes trend towards finding issues or discussions with code and not praise.

If your reviewer has addressed your comments, or they have done some optimisation or smart code you like, thank them. Say you like the approach / how they have solved an issue.

Conversely, thank the reviewer if they thoroughly explain what they want and promptly answer your questions.

I have had some reviews where reviewers have put huge comments explaining something I misunderstood and I can still remember who it was and what they said.

How to Handle Pushback in Code Reviews

Sometimes the developer who is receiving the review might disagree with the changes that you're proposing. Here's how to handle that.

Consider That They Might Be Right

Sometimes the developer is closer to the code and how it works, and they may be right. If they are, move on from that discussion point and leave that code as it is.

But if they're not, or they've misunderstood something, you should insist on the change. Respond to what the developer initially challenged you on, and explain your reasoning courteously.

Code quality improvements tend to happen in small incremental steps, and review is one way it happens. Insist on the code improvement, even if it means extra work for the developer.

Try to Avoid "I'll fix this later..."

One of the most common issues in reviews is not that people disagree whether the code change is necessary, but rather that they want to do it under a different ticket or do it later. So if you pass this review, they assure you that they will follow up later and fix the issues.

Generally speaking, following up later rarely happens. This doesn't mean there's something wrong with the developer or their work ethic. But it's easy to forget, have priorities change, or even just get lost in their queue of work.

Insist that the work gets done now. There is no real fantastic gain to merging changes in the codebase you have to immediately fix. You are just adding technical debt that might be followed up on later, or might be forgotten. Fixing lots of tickets later is an easy way for quality to drop.

The only exception to this is emergencies, which involve the highest priority bugs Google deals with where there is something seriously wrong. This could be services going down, or errors crashing people's pages in production.

There is naturally an eagerness to merge changes ASAP, and the review might accept slightly lower quality, with a follow up fix immediately being written. The main point is that the first change removes the emergency, and the second change fixes it properly.

Conclusion

I hope this has explained some helpful concepts Google makes use of in their code review process. I wrote this to better cement it into my own mind, and was curious about how other companies handle the QA process.

I found that there are quite a few points that I can take away and apply to the reviews I conduct.

The document I referred to throughout this article can be found here.

I share my writing on Twitter if you enjoyed this article and want to see more.

So how can we use Postman to both test our existing APIs and understand how they work?

• What is Postman?
• What are we going to build / learn?
• Part 0: Getting set up with Postman
• Part 1: An introduction to Postman
• Part 2: Creating a new Postman request to GET info about Squirtle
• Part 3: Creating a collection of requests in Postman for the PokéAPI
• Part 4: Making POST requests with Postman to translate sentences to sound like Yoda
• Part 5: Authenticating requests to the Lord of the Rings API with an API Key

What is Postman?

Postman is a tool teams can use to reliably test APIs using easy to use configurations. It comes stocked with features you would expect when dealing with APIs, including authentication, setting headers, customizing the payload, and a bunch more that help reduce the friction of using an API.

And it’s not just for testing. The beauty is that this can be used for many aspects of working with APIs for many different members of the team. Maybe a Project Manager wants to verify that things work or might find it easier to make a change straight with the API, or a QA Engineer needs to make sure everything still works, or a developer wants to actively make changes while working on the API itself.

The best part about it – Postman provides collaboration features. The free tier includes exporting and importing collections of saved API requests as well as creating shared links. If you're part of a team, they have paid tiers that allow you to sync up your collections to make sure everyone has the most recent and up to date collection.

What are we going to build / learn?

We’re going to walk through two different example APIs to cover the concepts of Postman.

First, we’ll walk through some simple HTTP requests with a public API for Pokémon.

We’ll then use the Yoda Translator API for one part to demonstrate how to make specific HTTP requests.

Once we understand how the basics work, we’ll use the Lord of the Rings API to learn how authentication works with APIs. For this, you’ll need to register for a free account for an API key.

Part 0: Getting set up with Postman

Before we get started, you’ll need Postman in order to follow along with this walkthrough. The good news, is Postman is available for free on Mac, Windows, and Linux, so you should be able to find a version that works for you.

Get Postman: https://www.postman.com/downloads/

Once downloaded, go through the standard installation instructions, open it up, and we should be ready to go!

Part 1: An introduction to Postman

The first time you open up Postman you’ll immediately be shown a launchpad with a bunch of options to get started.

It might seem a bit overwhelming, but let’s break down some of the key concepts that we’ll need to know.

Requests

A request is kind of what it sounds like, it’s a specific API request. This will be a single type of request, whether it’s a GET or POST to a specific endpoint. You’ll want to create new requests for each type of endpoint which will allow you to move between them when testing.

Collections

A collection is a group of requests. This is handy for organizing your requests into different groups. This could be as simple as two totally different APIs (ie. Twitter vs Slack) or it could be two different groups of APIs for a single API (ie. Twitter Tweets API vs Twitter Accounts API).

Authorization

Authorization is how requests are authenticated with an API, whether by a person making a request or by a computer making that request on your behalf. This commonly comes in the form of an API key which can be a static value assigned to your account or dynamically generated with tools like OAuth.

Environments

Environments will allow you to configure your endpoints to use specific variables that make it easier to use the same endpoints between different environments. For instance, you might have the same /profile endpoint on both your production and development environments, but they have different domains. Environments lets you manage a single request with a variable domain.

Workspaces

We won’t go too far into workspaces in this post, but it allows you to manage and organize different sets of collections. Imagine if you want to use Postman for both work and a personal project, you might have a Work workspace as well as a Personal workspace.

For the purposes of this article, we’ll be covering Requests, Collections, and Authorization.

Part 2: Creating a new Postman request to GET info about Squirtle

Now that we have a better understanding of the different terminology, let’s actually create a request.

At the top left of the UI you should see a but orange button that says New. Go ahead and click that and then select Request.

Before we get into the request itself, it requests a few things.

This first thing requires is a name. We’re going to start off by requesting information about the Pokémon Squirtle, so let’s name this “Pokémon - Squirtle”.

It also requires a collection, so click Create Collection and let’s name the collection “My Favorite Pokémon”.

Click the orange checkmark button next to the collection name then hit Save.

At this point we’ll have a new request, so let’s build that request.

There are two things we’ll first need to fill out for our first request:

• Request type: GET, POST, PUT, etc - we’ll use GET
• Request URL: The endpoint for your API request - for our request we’ll use https://pokeapi.co/api/v2/pokemon/squirtle/

And once you make sure those are correct, you can simply hit the blue Send button on the right and we’ve successfully made our first request!

We immediately get a few things we can see:

• Body: at the bottom we should now see the response body of the API request. For our Squirtle API, we should have a JSON object with data like abilities, base_experience, and forms.
• Status: on the right, we should see the HTTP status code. “200 Ok” is a good sign and it means it was successful!
• Time: simply how long the request took to finish
• Size: the size in KB (in our example) of the response data

You can also hover over Status, Time, and Size and get a more in depth look at each option.

So we made our first request!

Once thing to notice before we move on is that our request looks like it’s in a browser tab. If we’re done with that particular request, we can close the tab and click Save to make sure all of our changes are there for next time!

Part 3: Creating a collection of requests in Postman for the PokéAPI

Now that we’ve created a request, let’s create a collection of them. Technically we already had to create a new collection for Part 2, but we’ll create a new one to learn how collections themselves work.

At the top left of the UI, click the orange New button again and select Collection.

Similar to a request, it asks for a name so let’s call this “PokéAPI”. Optionally you can add a description, then click Create at the bottom.

On the left, you’ll now see your collection. You can select and expand the folder since we’ll be working with it.

Before we add a request, the PokéAPI has different types of requests, so it makes sense to organize it a little more thoroughly. So let’s click the three dots next to the PokéAPI collection and select Add Folder.

Similar to the others, this asks for a name. Folders are kind of like collections inside of a collection, so you get similar options. Let’s name this one “Pokémon” and click the orange Save button like before.

Now let’s add our requests! First, click the three dots next to the Pokémon folder, similar to how we added a folder to the collection, but this time select Add Request.

Let’s name this request “Pokemon”. While it might be confusing that we have a Pokemon request inside of the Pokémon folder, Pokemon is just one of the endpoints of the Pokémon group.

Now, let’s use the same exact API that we used with our Squirtle request before:

• Request Type: GET
• Request URL: https://pokeapi.co/api/v2/pokemon/squirtle/

And similar to before, when we hit the blue Send button, we should see a successful request!

Now let’s add another request. Follow the same process as before to create a new request under the PokéAPI Pokémon folder and let’s name this request “Abilities”.

If you scroll through the response from the first Squirtle endpoint, you see a lot of other API urls. At the top, we have abilities and we have two different ones — “torrent” and “rain-dish”.

Choose your favorite Squirtle ability and copy the url value into the new Abilities request we just created, I’m going to use rain-dish.

We can leave the Request Type as GET, hit the blue Send button, and we can again see a successful response!

Here we get a lot of information about our Squirtle ability Rain Dish and some of the details come in different languages which is cool!

So now we have a new PokéAPI collection with a Pokémon folder representing the group of Pokémon API endpoints including Pokemon and abilities.

We’re going to stop Part 3 with those 2 requests, but feel free to continue on and add as many of the PokéAPI requests as you’d like!

Part 4: Making POST requests with Postman to translate sentences to sound like Yoda

So far we’ve only made GET requests, but what if we wanted to make a POST request where we need to actually send some data?

For making a POST request, we’re going to use the Yoda Translator API from funtranslations.com. While this API only takes a single parameter, it’s still a good public endpoint we can use to understand the concept.

First, let’s create a new collection with a new request:

• Collection: Fun Translations
• Request: Yoda

This time, instead of a GET request, our request configuration will be:

• Request Type: POST
• Request URL: https://api.funtranslations.com/translate/yoda

Now this time, if we hit the blue Send button, we’ll notice we don’t get a successful 200 response, we get a 400!

We never actually set up any data to be posted to the API and it requires that data, so let’s add it.

Right below the Request URL, click Body. Then instead of none, select raw as the body type. Finally, on the far right of the types, change Text to JSON.

Then, in the space below it, you can add the following:

{
    "text": "Hello, I am learning how to test APIs with Postman!"
}

And now click the blue Send button again and we get a successful response!

We can apply this concept to pretty much any API. Postman doesn’t only permit you to post JSON, it allows you to use the other formats that we see listed in the Body Type section, meaning you have a lot of options depending on what the API you’re using requires.

Part 5: Authenticating requests to the Lord of the Rings API with an API Key

For the rest of the walkthrough, we’re going to use the Lord of the Rings API.

First up, the Lord of the Rings API requires authentication in order to make requests using an API key. So to start, you’ll before we dive in, you’ll need to go create a free account.

https://the-one-api.herokuapp.com/sign-up

Once you sign up and log in, the first thing you’ll see is your API key! Either copy this key down or remember where you can find it for later. If you leave the page, you can always grab it by navigating to Welcome and then Account in the navigation of the API website.

To get started, let’s first create a new collection and request:

• Collection: Lord of the Rings
• Folder: Movie
• Request: All Movies
• Request Type: GET
• Request URL: https://the-one-api.herokuapp.com/v1/movie

Once you’re set with the above, click Send, and you’ll notice immediately it gives a response that says 401 and that it’s unauthenticated.

Because this API requires the API key, this is exactly what we expected. So let’s click on the Authorization tab. We can then select a Type of Bearer Token, and on the right, we can paste in our key that we just set up with the Lord of the Rings API.

And as soon as we hit Send, we now see a successful response!

This worked really great, but what if we have a bunch of requests that use a single key. Do we have to manage that on each request?

Instead of managing it on each individual request, we can manage it on the collection. Let’s first build another request.

Under our Lord of the Rings collection and in the Movie folder, create a new request:

• Request: Quote by Movie ID
• Request Type: GET
• Request URL: https://the-one-api.herokuapp.com/v1/movie/{id}

In this request, let’s use an ID from the response of the first request, I’m going to use 5cd95395de30eff6ebccde5b which is the ID of The Two Towers, so the request URL will look like:

https://the-one-api.herokuapp.com/v1/movie/5cd95395de30eff6ebccde5b

Now, instead of setting our token in the request Authorization, we’re going to leave the type as Inherit auth from parent. Click on the three dots next to the collection and select Edit.

Here, we’re going to do the same exact thing we did with the first request but on the Collection configuration. Select the Authorization tab, under type select Bearer Token, and in the Token field again paste your token.

Finally, click Update and hit the blue Send button again and we can see a successful request!

We can now go back to our All Movies request and update the Authorization to use a Type of Inherit auth from parent and it should still continue to work!

What else can we do with Postman?

While I covered a lot of the basics, there’s quite a lot more you can do with Postman. Here are a few of my favorites.

Environment Variables

If you’re working as a developer on a project, it’s likely that your team uses multiple environments, such as a development and production environment. Instead of creating and maintaining completely separate requests, you can add an environment variable and instead change that variable when switching between environments!

Variables apply to many scenarios, but that’s a common use. Check out Postman’s docs to learn how.

https://learning.postman.com/docs/postman/variables-and-environments/variables/

Import and Export Collections and Data

A great thing about Postman is once you have your requests all organized, you can export them for others to use. This also means that you can import collections from other team members. This makes it much easier to make sure everyone’s using the same collection.

Bonus: you can even store these files in a Git repository, as they’re just JSON.

But keep in mind - if you’re using Authorization on the collection like we went over in this guide, you’ll want to make sure you don’t include that when exporting your collection.

https://learning.postman.com/docs/postman/collections/importing-and-exporting-data/

Automated testing

Once you have a set of requests in a collection and even better, if you’re storing them in Github, you can begin to use those requests as part of a way to manage automated testing of your API.

While there are a few solutions for doing this, Postman includes a Collection runner built right into the app and Newman is a command line tool that lets you run tests right from the terminal.

https://www.postman.com/use-cases/api-testing-automation/

What’s your favorite way to test and play with APIs?

Share with me on Twitter!

Test Driven Development (TDD) is sometimes described as “writing tests first”. The TDD mantra states that we should not write code before we have written automated tests that exercise that code. Writing code first is considered suboptimal.

And of course, writing code first is how we develop software following the so-called waterfall model. In that model, we divide software development activities into stages. For example, we have a ‘requirements gathering’ stage, we have an ‘application building’ stage, we have an ‘application testing’ stage, we have an ‘application deployment’ stage and so on.

But how is that different from the agile methodology? Don’t we have the exact same stages in agile?

Agile Methodology vs Waterfall Methodology

Of course we do. The crucial difference is that in agile, those stages are not gated.

In waterfall, we gate the stages and execute them in strict sequence. This means we won’t begin building the shipping application until such time as the requirements have been gathered, completed, signed off on and frozen. Once requirements are frozen (and controlled by our change management policies), we move into the next stage (or phase) – application building.

And similarly, we won’t move into the testing stage until the entire application has been built and we have reached the code complete milestone, at which point code changes have been frozen.

Once code gets frozen (and code freeze is then controlled by our change management policies), we hand it off to the testers. The testing phase begins, and only once all testing has completed (and provided that no significant defects have been detected), do we move into the deployment phase.

In agile, we do all the above activities in parallel. At the same time. We keep working on user stories (specs) while simultaneously building the shipping application. As we’re building the application we are also testing it. And as we are building and testing the application, we are also deploying it.

We learn from the shipping application deployed to production and use that validated learning as the feedback that will inform new user stories. That way, the loop gets closed, and we’re iterating, improving the value incrementally.

The only way to enable such iterative value stream delivery is by relying on automated tests. And as we’ve described, those tests are being written very early in the game. Actually, tests must be written before we write shipping code.

Why then is the title of this article “Don’t Write All Your Tests First, Just Write One”? It sounds a bit confusing. Let’s unpack the meaning of this title. But first, here's an overview of what tech we'll be using:

The technology stack used for this exercise

In the attempt to keep the exercise simple and easy to follow, I have chosen .NET Core platform, together with xUnit.net testing platform. To follow the coding examples, please install .NET Core and xUnit.net.

In order to be able to run the sample code, please open ./tests/tests.csproj file and add this line to the ItemGroup:

<ProjectReference Include="../app/app.csproj" />

You’re now all set for following the coding exercises.

A simple example

To understand the difference between writing all tests first and writing one test first, it may be better to show rather than just tell.

So let’s try to build a simple example – for this exercise I’ve chosen a trivial case of calculating a tip at a restaurant. Often times we find ourselves in a position where we want to tip the restaurant for the service, but it’s tough to calculate percentages in our head. So a nifty little Tip Calculator could come in handy.

Here are the expectations:

As a patron
I want to calculate the total bill (total plus the tip)
Because I want to compliment the restaurant for the service

Scenario #1: Patron calculates the total for terrible service

Given that the restaurant total is $100.00
And the service was terrible
When the tip calculator calculates the total charge
Then tip calculator shows $100.00 total charge

Scenario #2: Patron calculates the total for poor service

Given that the restaurant total is $100.00
And the service was poor
When the tip calculator calculates the total charge
Then tip calculator shows $105.00 total charge

Scenario #3: Patron calculates the total for good service

Given that the restaurant total is $100.00
And the service was good
When the tip calculator calculates the total charge
Then tip calculator shows $110.00 total charge

Scenario #4: Patron calculates the total for great service

Given that the restaurant total is $100.00
And the service was great
When the tip calculator calculates the total charge
Then tip calculator shows $115.00 total charge

Scenario #5: Patron calculates the total for excellent service

Given that the restaurant total is $100.00
And the service was excellent
When the tip calculator calculates the total charge
Then tip calculator shows $120.00 total charge

Let’s now implement the above user story.

We see that the story has 5 acceptance criteria (a.k.a. scenarios). Now we move into the analysis phase – think about what should be the first functionality that our Tip Calculator application should implement. But first, let’s open the command line terminal and create the new directory:

md TipCalculator
cd TipCalculator

and create app and tests directories inside the TipCalculator directory.

Now cd tests and run:

dotnet new xunit

Then cd .. and cd app, then run:

dotnet new classlib

We’re now ready to boogie!

Open your favourite text editor (mine is Visual Studio Code) and set your mind on the expectations. What behaviour are we expecting from the Tip Calculator?

To narrow the scope of our expectations, it usually helps to take one acceptance criteria (i.e. one scenario) and focus on it first. Let’s take scenario #1:

Scenario #1: Patron calculates the total for terrible service

Given that the restaurant total is $100.00
And the service was terrible
When the tip calculator calculates the total charge
Then tip calculator shows $100.00 total charge

In case the service was terrible, we are not adding any tips, and Tip Calculator is calculating a $0.00 tip. So how do we automate that scenario?

My first expectation would be that we need to somehow inform the Tip Calculator that the service was terrible. We either type the word ‘Terrible’ into the input field, or we select ‘Terrible’ from the list of available service ratings.

So the first thing to do here is to articulate some expectations regarding Tip Calculator’s ability to get notified that the service was terrible.

I like to always start with the expectation that what the user inputs is valid. So I’d first write a test that checks if the rating ‘Terrible’ is recognized by the Tip Calculator as a valid rating.

Go to the tests directory, rename the autogenerated UnitTest1.cs file to TipCalculatorTests.cs and add the following test:

[Fact]
public void CheckIfRatingTerribleIsValid(){    
  var expectedResponseForValidRating = true;    
  var actualResponseForValidRating = false;    
  Assert.Equal(expectedResponseForValidRating, 
  actualResponseForValidRating);
}

Now go to the command line, cd tests, and run:

dotnet test

Of course, the above trivial test will fail, because we have hardcoded the values. But it’s always a good practice to make sure we see our tests fail before we proceed. Not observing a test fail may give us false sense of safety later on, if no tests fail and we end up thinking that everything works as expected.

A few more observations about the above test:

• It helps if the test name is descriptive. I chose CheckIfRatingTerribleIsValid to communicate the fact that we must make sure our application is capable of recognizing our commands.
• It also helps if the expected and actual variable names are descriptive. I chose expectedResponseForValidRating and actualResponseForValidRating as fairly indicative of what our expectation in this test is, and also what actual value will the Tip Calculator produce.
• Test is a first-class source code and must be approached with equal care lavished upon the production code.

First design decision

At this point, we are forced to make a decision – how will our nascent Tip Calculator know if the service rating provided by the user is valid or not?

The design decision that comes to mind is that Tip Calculator must be able to store and retrieve some data. In this case, the data we’re interested in is the service rating.

If we go back to the user story and review the five acceptance criteria, we will see that the expectations are that Tip Calculator must be able to recognize five different service ratings:

1. Terrible
2. Poor
3. Good
4. Great
5. Excellent

So the simplest way to get Tip Calculator to store that information would be to endow it with an array, or a list.

But rather than rushing in to implement that list, we should examine the expectations again, to see if there’s anything else we may have missed. And there is – not only must Tip Calculator be able to recognize valid service ratings, it also must be able to associate each rating with a percentage value.

Our analysis shows the following associations:

1. Terrible => 0%
2. Poor => 5%
3. Good => 10%
4. Great => 15%
5. Excellent => 20%

In this case, a simple array or a simple list won’t be sufficient for holding the above associations. What’s the next simplest data structure that will allow us to implement these associations? After doing a little bit of research, we figure out that Hashtable is probably the most fitting data structure that can cover our needs with the least amount of ceremony.

We now navigate to the app directory and rename autogenerated Class1.cs file to TipCalculator.cs. We now want to add a Hashtable that will hold service ratings and the associated percentage values:

System.Collections.Hashtable ratingPercentages = new System.Collections.Hashtable();

Now is a good time to recall that TDD is focused on coupling the expectations to the application’s behaviour, not to the application’s structure. Knowing that, we need to modify our test to make Tip Calculator exhibit some behaviour. The test codifies some expectations with regards to how the application must behave, and the running application provides the evidence of the expected behaviour.

But what is the evidence of the application’s behaviour? There is no other way for us to assess and evaluate application’s behaviour other than through examining the values that the running application produces.

In this case, we are expecting the running application to produce values true or false (Boolean values) after we ask the application if certain value (i.e. service rating) is valid.

To teach the application how to behave in the expected fashion, we need to endow it with an API. In this case, we design the API as follows:

public bool CheckIfRatingIsValid(string rating)

In our test, we will modify the actual expected value to exercise the running application and collect the output value:

As you can see from the screenshot above, we have instantiated TipCalculator but when attempting to ask the instance to check if the supplied rating (“Terrible”) is valid, the editor is complaining that it cannot find that method.

Well of course, the method hasn’t been implemented yet. Now’s the time to go ahead and do it:

public bool CheckIfRatingIsValid(string rating) {    
  return false;
}

Now that the method is implemented, the test works; here is the complete listing:

using Xunit;
using app;

namespace tests {    
  public class TipCalculatorTests {        
    TipCalculator tipCalculator = new TipCalculator();        

    [Fact]        
    public void CheckIfRatingTerribleIsValid(){            
      var expectedResponseForValidRating = true;            
      var actualResponseForValidRating = 
      tipCalculator.CheckIfRatingIsValid("Terrible");                  
      Assert.Equal(expectedResponseForValidRating, 
      actualResponseForValidRating);        
    }    
  }
}

We see from the above example that we’re cheating again (we have hardcoded return false; in our newly minted method). What’s the point of beating around the bush and merely creating skeletons and scaffoldings instead of rolling up our sleeves and doing actual coding? Let’s discuss this important topic.

Discussion about our first design decision

We’re illustrating here how to do TDD step-by-step. The funny part is that this step-by-step illustration is actually the exact way how we do TDD: step-by-step. There is no other way to do TDD than by doing it step-by-step. One step at a time.

How’s that different from any other way of doing software development? Don’t we also do everything step-by-step even when not following TDD methodology? Well, not really. Let me explain:

TDD to me feels like riding a galloping horse. We’re moving swiftly toward our goal, but we’re frequently touching the ground (the galloping horse is every now and then hitting the ground in order to bounce off and run fast).

In comparison, when I’m doing software development without TDD, it feels to me like I’m flying a kite. I’m making swift moves with the kite, but I never touch the ground, not even once. By the time I land the kite, the landing place may not be where I intended the kite to go (it’s very hard to control the direction of a kite if it’s flying in a strong wind).

With TDD, any time we make a change to the code (both the test code and the shipping application code), we run the tests and so we touch the ground. We are galloping, but at the same time we need this frequent grounding. We need to see whether we’re going in the right direction and also whether we’ve broken anything during our galloping. Our tests are the Oracle who keeps telling us if everything works as expected or if something started misbehaving.

Making changes to the code is a risky business. TDD provides a nice harness that is both guiding our design decisions and ensuring we don’t mess up something that we’ve already confirmed works to our expectations.

Replace hardcoded value with actual processing logic

Let’s now replace the hardcoded value with actual running code. We first teach our Tip Calculator that there is a service rating called “Terrible” and that tip percentage associated with this rating is 0:

public bool CheckIfRatingIsValid(string rating) {      
  ratingPercentages.Add("Terrible", 0);    
  return false;
}

Our Tip Calculator is now knowledgeable about the fact that there is a service rating labeled “Terrible” and the tip percentage associated with terrible service is 0%. Great, but we’re still returning hardcoded value false. Time to replace it with actual calculation:

public bool CheckIfRatingIsValid(string rating) {    
  ratingPercentages.Add("Terrible", 0);    
  return ratingPercentages.ContainsKey(rating);
}

Run the test again:

Great, but the code still looks contrived. We are loading the “Terrible” value into the instance of Hashtable ratingPercentages and then immediately checking to see if that value exists in the Hashtable. Now that we have moved from the failing test (Red) to the passing test (Green), it’s time to perform the third step in the TDD loop – Refactor.

Refactoring is basically the activity of modifying the code structure without affecting the code behaviour. Our task here is simple: extract the code responsible for populating of the Hashtable ratingPercentages into a separate block of code.

The most natural place for this loading is in the block of code that is doing the initialization of the Tip Calculator – the constructor method. After refactoring, our shipping application source code looks like this:

using System.Collections;

namespace app {    
  public class TipCalculator {        
    private Hashtable ratingPercentages = new Hashtable();        
    public TipCalculator() {            
      ratingPercentages.Add("Terrible", 0);        
    }        

    public bool CheckIfRatingIsValid(string rating) {            
      return ratingPercentages.ContainsKey(rating);        
    }    
  }
}

Run the test again, and it passes (we’re in green). We have modified the structure of the code without modifying its behaviour! Good job.

Flip the coin

Any time we satisfy a positive expectation, it is a prudent practice to turn things on their head and describe the negative expectation.

At this point, since we’ve satisfied that a legitimate service rating value is found in the Tip Calculator, we want to ensure that non-legitimate values are not found in the Tip Calculator.

What do we mean by non-legitimate values? Any value other than “Terrible”, “Poor”, “Good”, “Great” and “Excellent”. Time to write the new expectation (i.e. test):

[Fact]
public void CheckIfRatingWhateverIsValid() {    
  var expectedResponseForValidRating = true;    
  var actualResponseForValidRating =                        
  tipCalculator.CheckIfRatingIsValid("Whatever");
  Assert.Equal(expectedResponseForValidRating, 
  actualResponseForValidRating);
}

Run the tests:

Fails. As expected. (We specified that our expectation when supplying the service rating as “Whatever” should be true. In reality, it is false, because our Tip Calculator does not contain value “Whatever”.)

Fix the test (change the expectedResponseForValidRating from true to false) and run it again:

A moment of reflection: why did we fake the first test run and made it fail? Because we always want to make sure we observe our new test failing. That way, we’ll know that in the future any successful passing of the test is not merely a false positive.

In praise of steady state

Software engineering is a balancing act between steady state and periods of unstable state. What do we mean by steady state?

If we have a system (a running application) that behaves the way we expect it to behave (i.e. it produces values we have specified as expected values), we declare that the system is in a steady state. It is running, and delivering some value.

That value delivery is still partial. In our case, the only value to the users this Tip Calculator delivers is its ability to recognize service rating “Terrible” as a legitimate rating. In addition, it is capable of informing us that service rating “Whatever” is not a legitimate rating.

That’s not much, but still is better than nothing. And good news – our running application is currently in a steady state. Now we want to look into how to add more valuable behaviour to our Tip Calculator. And the only way to add more value is by making some changes.

Any time we make a change to our application, we disturb its steady state. This disturbance is risky. It may mean our changes could break something that is already working. Because of that concern, we strive to make the duration of this unstable state as short as possible.

Remember how we compared TDD to riding a galloping horse? When the horse is in flight (i.e. not touching the ground) it is advancing toward our goal, but it’s not in the steady state. Only when the horse touches the ground does its state stabilize.

TDD encourages making small changes (in flight) and immediately grounding the system by verifying that it is back in the steady state. We value steady state despite the fact that we eagerly embrace changes. Without changes, we won’t be able to deliver value, but we must do it in a very deliberate, careful fashion.

When doing TDD, we treat changes to steady state like walking on eggshells. No matter how sure we may be in knowing what and how we’re doing software engineering, it is prudent to still let failing tests guide our decisions.

Check if correct tip percentage is associated with service rating

Let’s now introduce another change into our application – a test to verify if correct tip percentage is associated with service rating “Terrible”. Remember that we populated the instance of Hashtable ratingPercentages with the following values:

ratingPercentages.Add("Terrible", 0);

We have written a test that verifies that our Hashtable ratingPercentages does contain legitimate service rating “Terrible”. Now we need a test that verifies that service rating “Terrible” means that the tip percentage for that rating is 0.

[Fact]
public void CheckIfRatingTerribleHasZeroPercentTip() {    
  var expectedZeroPercentForTerribleRating = 0;    
  var actualZeroPercentForTerribleRating = 10;    
  Assert.Equal(expectedZeroPercentForTerribleRating, 
  actualZeroPercentForTerribleRating);
}

The new test CheckIfRatingTerribleHasZeroPercentTip should fail:

Again, we’re purposefully hard coding wrong actual values just so that we could observe our brand new test fail. Now we must replace hard coded value with the actual call to the Tip Calculator’s method that returns tip percentage for the service rating:

[Fact]
public void CheckIfRatingTerribleHasZeroPercentTip() {    
  var expectedZeroPercentForTerribleRating = 0;    
  var actualZeroPercentForTerribleRating =                 
  tipCalculator.GetPercentageTipForRating("Terrible");
  Assert.Equal(expectedZeroPercentForTerribleRating, 
  actualZeroPercentForTerribleRating);
}

As in the previous case, we have invented a new API for Tip Calculator. We call this new capability GetPercentageTipForRating("Terrible"). It takes the value of the service rating and returns the tip percentage for that rating.

Flip over to the app/TipCalculator.cs and add the hard coded skeleton of the new method:

public int GetPercentageTipForRating(string rating) {    
  return 10;
}

Running the test fails again, because we have hard coded the return value. Let’s replace it with actual processing:

public int GetPercentageTipForRating(string rating) {    
  int tipPercentage = Int32.Parse(ratingPercentages[rating].ToString());
  return tipPercentage;
}

Run the test again:

All three tests pass – we’re in green, we’re back to steady state!

What tip percentage do we expect for non-legitimate service ratings?

Many years of experience in the field taught me to be a bit pessimistic. Now that we have our application back in the steady state and delivering value (answering questions about legitimate service ratings and also giving us correct tip percentage for the “Terrible” rating), we need to see what happens when we run our application by giving it non-legitimate service rating value (for example, by giving it service rating “Whatever”).

Time for leaving the steady state yet again. We will write another test:

[Fact]
public void CheckIfRatingWhateverHasNegativeOnePercentTip() {    
  var expectedZeroPercentForWhateverRating = -1;    
  var actualZeroPercentForWhateverRating =       
  tipCalculator.GetPercentageTipForRating("Whatever");
  Assert.Equal(expectedZeroPercentForWhateverRating, 
  actualZeroPercentForWhateverRating);
}

We are describing our expectation when Tip Calculator is asked to return tip percentage for service rating “Whatever”. Because service rating “Whatever” is a non-legitimate rating, we are expecting Tip Calculator to return tip percentage of value -1.

This test now precipitates one improvement to our shipping code. We need to add some logic to first check whether the supplied service rating is legitimate or not. Only if it is legitimate do we ask Hashtable ratingPercentages to tell us what the associated value of the tip percentage is. If the supplied service rating is non-legitimate (for example, if it is “Whatever”) we bypass talking to Hashtable ratingPercentages and simply return -1.

public int GetPercentageTipForRating(string rating) {    
  int tipPercentage = -1;    
  if(CheckIfRatingIsValid(rating)) {        
    tipPercentage = Int32.Parse(ratingPercentages[rating].ToString());    
  }    
  return tipPercentage;
}

Run the tests, and all 4 tests pass:

We are back to the steady state. Another short excursion into the volatile area, and another swift victory and a safe return to steady, imperturbable state.

Populate other service rating tip percentages

Now is a good time to take a breather and make less risky changes, following the already established pattern. Leave the safety of the steady state and make short trips into the volatile territory by adding a new test to verify if service rating “Poor” is a valid, legitimate rating:

[Fact]
public void CheckIfRatingPoorIsValid() {    
  var expectedResponseForValidRating = true;    
  var actualResponseForValidRating =       
  tipCalculator.CheckIfRatingIsValid("Poor");
  Assert.Equal(expectedResponseForValidRating, 
  actualResponseForValidRating);
}

Running this test will fail:

Service rating “Poor” hasn’t been implemented yet. To make the test pass, implement service rating “Poor” by adding this line to the TipCalculator constructor:

ratingPercentages.Add("Poor", 5);

Run the tests, and we’re back to safety:

We’re enjoying steady state with 6 tests successfully passing.

Now that we have added service rating “Poor” associated with the 5% tip, let’s write a test that will describe that expectation:

[Fact]
public void CheckIfRatingPoorHasFivePercentTip() {    
  var expectedZeroPercentForPoorRating = 5;    
  var actualZeroPercentForPoorRating =       
  tipCalculator.GetPercentageTipForRating("Poor");
  Assert.Equal(expectedZeroPercentForPoorRating, 
  actualZeroPercentForPoorRating);
}

The tests run successfully, and we’re back to being safe in the steady state.
I will leave it to the reader to make the changes that will drive the implementation of the service ratings “Good”, “Great” and “Excellent”. At the end of the exercise you should have your system back in the steady state with 12 tests successfully passing:

Calculate grand total given the total and the service rating

We are now ready for the final step – given the total bill and the service rating, we expect Tip Calculator to calculate tip percentage and add it to the total, producing the grand total to be paid to the restaurant.

As we always do, first we describe the expectation:

[Fact]
public void CalculateTotalWithTip() {    
  var expectedTotalWithTip = 135.7;    
  var actualTotalWithTip = 200.0;    
  Assert.Equal(expectedTotalWithTip, actualTotalWithTip);
}

As usual, we first hard code some expectations that we know are going to fail. This is so that we observe our new test failing:

Time to implement processing logic that will calculate correct total with tip. Given the total of $118.0, and the service rating “Great” (15% tip), we’re expecting the total to be $135.7:

[Fact]
public void CalculateTotalWithTip() {    
  var rating = "Great";    
  var total = 118;    
  var expectedTotalWithTip = 135.7;    
  var actualTotalWithTip = tipCalculator.CalculateTotalWithTip(total, 
  rating);    
  Assert.Equal(expectedTotalWithTip, actualTotalWithTip);
}

We have designed a new API the Tip Calculator – a method called CalculateTotalWithTip(total, rating). It takes the total value and the service rating and returns the total with tip. The implementation of the method looks like this:

public double CalculateTotalWithTip(double total, string rating) {    
  double totalWithTip = -1;    
  if(CheckIfRatingIsValid(rating)) {        
    int percentage = GetPercentageTipForRating(rating);        
    totalWithTip = total + ((total/100) * percentage);    
  }    
  return totalWithTip;
}

Run the tests, and we’re back to steady state:

Are we done here?

No, not yet. Even when all tests are in green and we’re back to the steady state, there are still a couple of things we need to do.

To begin with, we need to add a pessimistic expectation for our Tip Calculator calculation of total with the tip based on the service rating:

[Fact]
public void CalculateTotalWithTipForNonlegitimateRating() {    
  var rating = "Meh";    
  var total = 118;    
  var expectedTotalWithTip = 135.7;    
  var actualTotalWithTip = tipCalculator.CalculateTotalWithTip(total, 
  rating);    
  Assert.Equal(expectedTotalWithTip, actualTotalWithTip);
}

Running the tests produces one failing test:

Our expectation for non-legitimate service rating (“Meh”) was incorrect. The actual total is -1, so we need to adjust our expectation by replacing 135.7 with -1. Run the tests again, and we’re back to the steady state!

We now have 14 tests, they all successfully pass, and our Tip Calculator works according to our expectations and satisfies the acceptance criteria.

We’re almost done. One more sanity check before we can confidently ship our shiny new Tip Calculator – we must run mutation testing.

Our mutation testing framework will mutate the shipping code, one line at a time, and will run all tests for each individual mutation.

If the tests complain about the mutated code, all is good, we have killed the mutant. If the tests don’t complain, we’re in trouble. We have a surviving mutant in our codebase, which means there are lines of code in our repo that are doing something for which we haven’t provided any expectations.

Let’s run mutation testing to see how solid our solution is. Good news – our solution has killed 100% of mutants!

Mutation testing has given our shipping application a clean bill of health. Our Tip Calculator seems to be in good shape.

Red-Green-Refactor-Reflect

Let’s review our Tip Calculator building exercise. We started the process by describing our expectations using the classical user story format. User story (as the name implies) is focused on describing scenarios that fulfill end user’s goals.

In this case, the simple goal is to calculate the tip amount from the supplied service rating and the restaurant bill total. The calculated tip amount is then automatically added to the total.

From there we proceeded to build our shipping application by following the TDD methodology. As we’ve demonstrated, the methodology consists of writing a failing test, observing it fail (the Red phase of TDD), then immediately making code changes that ensure the test passes (the Green phase of TDD). Once the test passes, we move into the Refactor phase (we restructure the code without affecting its behaviour). That way, we make sure our code is not expensive to change.

A proper TDD practice also mandates frequent retrospective – we call it reflection. We stop and think about the things we’ve accomplished thus far, to see if we could learn from our recent experiences. This reflection fortifies the process, as it relies on frequent and tight feedback provided by the failing, then succeeding tests.

I have already compared Test Driven Development to the experience of riding a galloping horse. While riding a horse, we’re alternating between flying through the air (i.e. speed achieved when the horse leaps from the ground) and steering the horse. It is impossible to steer the horse while we’re off the ground, up in the air. At that point, we gain speed, but we cannot make any changes of the direction. It is only once the horse touches the ground that we can make a change in direction.

In TDD, we strive to touch the ground as frequently as possible. The longer the leaps we make without touching the ground, the less chance we have for correcting the course.

I also compared software development practices that don’t follow TDD principles to the experience of flying a kite. When flying a kite, we never touch the ground. It is an exhilarating feeling of letting the wind pick the kite up and bounce it up in the air. We can achieve considerable speed that way. But we struggle in such situations to maintain desired course. And after we eventually land the kite, it usually does not land in the spot we originally wanted it to land.

Why is the emphasis of this article on “don’t write tests first”? Many software engineers who are not familiar with agile practices as implemented in TDD usually either claim that writing automated tests isn’t necessary, or claim that automated tests should be written after the code is complete.

Once they start learning about agile and TDD, they may reconsider their practices and decide that writing tests before writing implementation code may make more sense. Still, because of the ingrained waterfall mentality, some of those engineers make the mistake of writing all tests first, and only then move into writing the code.

That approach is completely wrong. It is equivalent to the traditional waterfall approach where we go through the development process by respecting gated phases.

First we write the requirements (in this case, requirements would be expectations written in the form of automated tests). Only once all the requirements (i.e. automated tests) have been written, signed off and frozen, do we move into the next gated phase – write the code for the shipping application.

TDD is the exact opposite of the “write tests first” approach. In TDD, we always write only one test. That test describes a desired behaviour. The desired behaviour does not exist yet (that’s why it is desired), and the test fails.

We then immediately move into making changes to the code in the attempt to create the desired behaviour. Once desired behaviour is created, it gets validated by the test, and if the expectations of the test are satisfied, we move into refactoring the code (to satisfy nonfunctional requirements, such as cost of change).

We practice a rigorous discipline to never succumb to the temptation to write more than one test at a time. That way, we ensure that we keep touching the ground as frequently as possible.

We prefer to remain ‘in flight’ for the shortest possible time. We are ‘in flight’ during that period when the desired behaviour described in the test has not materialized yet. The smaller the expected and desired behaviour is, the shorter will be our ‘in flight’ trajectory. That way, we keep touching the ground often, which gives us a chance to adjust the steering.

Conclusion

Building a simple Tip Calculator is a toy sized problem, and using that exercise to illustrate TDD methodology is not necessarily providing a convincing argument in favour of TDD. Still, within the constraints of a technical article, going over this hands-on exercise may provide valuable insights into the benefits of adopting TDD.

We would still argue that the real benefits of TDD only become apparent when dealing with much larger, more complex software engineering efforts. The ability to remain grounded while making potentially risky changes to a large, complex system is often a life saver.

In addition to that, building software using TDD methodology results in much less rework. TDD drives high degree of modularization, which results in high cohesiveness of the modules and low coupling between the modules.

All these characteristics produce a shipping application whose codebase is easy and inexpensive to change. And lowering the cost of change has proven to be the best way on the path to embracing changes and abandoning the concept known as ‘scope creep’.

Bottom line, TDD enables software engineering teams to deliver high degree of flexibility to the business.

Quality Assurance (commonly known as QA) is the means by which a product in development is checked to make sure it works as it’s supposed to. The actual methods used in QA processes vary hugely depending on the size and nature of the product.

For a personal project you will probably just test as you go, asking others to provide feedback at appropriate stages. By contrast, a banking application must have every aspect of every feature exhaustively checked and documented to ensure it is both functional and safe.

Regardless of how formal or detailed a QA process is, its aim is to identify bugs so they can be resolved before the product is released.

Methodologies

Agile

In an Agile approach to development, the aim is that each cycle of work (‘sprint’) produces working software that can be added to and improved upon iteratively. This makes the QA processes an intrinsic part of the development cycle.

By testing software components at each stage of their production you reduce the risk of bugs being present at release.

Terminology

Automation Testing

Testing done with pre-written scripts designed to control the execution of tests.

Black Box

These test do not look inside the system under test, but treat it as ‘closed’ in the same way that the end user will experience it.

Defect

Any deviation from an application’s specifications; often referred to as a “bug”.

Exploratory Testing

An unscripted approach to testing, which relies on the tester’s unique creativity in an effort to find unknown bugs and identify regressions.

Integration Testing

Testing individual components/modules together to ensure they connect and interact well with one another.

Negative Path Testing

A testing scenario designed to produce an error state in a feature/application and verify that the error is handled gracefully. An example of this is inputing a series of numbers in the email field in a user registration form and checking to make sure the registration is not accepted until an actual email address is provided.

Regression Testing

Testing done on a new build to ensure that new functionality has not unintentionally broken previously tested functionality.

Smoke Tests

A minimalistic approach to testing intended to ensure basic functionality is working before more in-depth testing takes place. Typically occurs at the beginning of the testing process.

Test Case

Specified preconditions, steps, and expected results referred to by a QA tester/engineer to determine whether or not a feature performs its task as expected.

White Box

Refers to tests performed at a structural level, within the codebase. Programmers checking that the inputs to and outputs from specific functions or components would be white box testing.

Also known as ‘Glass Box’, ‘Clear Box’, ‘Transparent Box’ because the tester can ‘see inside’ the system under test.

Main categories are

• Unit tests (individual units of code do what they should)
• Integration tests (units/components interact with each other properly)
• Regression tests (re-applying tests at later stages of development to ensure they still work)

There are three main techniques:

• Equivalence partitioning (the tested input values are representative of larger input datasets)
• Boundary Value Analysis (the system is tested with chosen inputs where behaviour and therefore output should change)
• Cause-Effect Graphing (tests are designed from a visualization of the input-output relations)

Other Resources

• How to write QA documentation that actually works
• Test Driven Development
• Unit tests

Isn’t Test Driven Development (TDD) twice the work? Should you do it anyway?

The short answer to the first question is NO. On the surface, it may seem like without TDD, time is only required to create the feature. With TDD, you need time to create the test AND create the feature, thus doubling the development time required.

What you’re not considering is the amount of time required for QA testing and debugging when the feature isn’t performing properly.

Case studies were conducted with three development teams at Microsoft and one at IBM that adopted TDD. The results of the case studies indicated that the pre-release defect density of the four products decreased between 40% and 90% relative to similar projects that did not use the TDD practice.

Subjectively, the teams experienced a 15–35% increase in initial development time after adopting TDD. (source)

That 40–90% decrease in pre-release defects means that QA teams and customers weren’t finding and reporting those issues. Engineering wasn’t trying to recreate bugs and develop patches, all of which have associated costs.

Number of Iterations per task

When discussing TDD, we consider a task to be a subset of a requirement that can be implemented in a few days or less. TDD software engineers develop production code through rapid iterations, as shown in the figure above.

What Is TDD?

Test-Driven Development is an approach to writing software in which the developer uses specifications to shape the way they implement a feature. For short, we describe it as the “red-green-refactor cycle”.

Before writing any code that adds new functionality to an application, the developer first writes an automated test describing how the new code should behave, and watches it turn red (fail to pass). They then write the code to the specification, and the test turns green (it passes). Finally, the developer takes a little time to make sure that the code just written is as clean as possible (refactoring).

Why you should care about TDD

Automated testing gives your software developers the confidence to make changes to the software and to know that no bugs were created as a byproduct.

Additionally, it allows more agility for developers who aren’t familiar with the details of the software to confidently modify the source code without introducing errors.

Let’s discuss some really cool advantages of TDD.

1. TDD Helps You Prevent Bugs

First, test suites ensure comprehensive test coverage of the codebase, so bugs are less likely to pop up unnoticed. Second, test suites allow developers to work out potential issues before the application is ready to go into production. Finally, because test suites are constantly maintained, they guarantee software quality.

2. Self Explanatory code (well-documented)

Because refactoring code is a built-in step in TDD, you end up with a much cleaner codebase as you go. Apps built with TDD tend to have less duplication, fewer edge cases that aren’t thought through, and a better overall architecture.

The test serves as a specification for what the code that will be written should do. As long as you’re writing good stories, your development team should be able to build exactly what you asked for. If your team agrees to use Acceptance Test-Driven Development, you can even write tests that describe how you want it to work in plain English!

3. Avoid the bugger debugger problem

Typically, when talking tech about software development, there are two main types of testing that can be integrated: functional and non-functional. These two types of testing practices are further divided into numerous types of testing techniques as you can see here:

And the list goes on

It becomes really important to strategize the testing plan before project commencement, since that helps in clearly defining the roles and responsibilities of developers with respect to testers when it comes to testing.

For example:

• Unit and Integration tests to be performed by developers before handing out builds to testers
• User acceptance testing to be performed by the testers
• Performance testing and UI testing should be done by both

A brief description of a few very important testing methodologies from the diagram above, that should be included in almost every test plan, are covered below.

Unit Testing involves testing individual units of source code to determine if they are fit for use. Intuitively, one can view a unit as the smallest testable part of an application. Faking, Mocking, and Stubbing are indispensable while writing unit tests for code which has API interactions.

Integration testing involves a combination of two or more “units” being tested. Integration tests verify that the components of the software all work together or “integrate” appropriately.

Performance testing is used to ensure that software applications will perform well under their expected workload. The features and functionality supported by a software system are not the only concerns. A software application’s performance, like its response time, reliability, resource usage, and scalability, matters. The goal of performance testing is not to find bugs, but to eliminate performance bottlenecks.

4. You can forecast troubles

A solid TDD approach alarms you about troubles upfront

The benefit of a comprehensive test suite is that it alerts you to changes early. For example, if your checkout flow stops charging users’ credit cards, you’ll know it right away, because the tests will fail. It also means that if someone makes a mistake and something doesn’t work the way it was supposed to, it will be obvious.

This is good, because it will give you a chance to fix it before it goes to production. If it becomes necessary down the road, you can even start a campaign of deep refactoring without fear, because you’ll have an ironclad test suite that will remain green.

5. Save Money

When code is complicated, it gets much harder to get anything done — one little change over here can result in a big problem over there. When following TDD, developers can make changes with confidence and your QA team will catch fewer regressions. In development speak, “time saved is equal to money earned.”

6. Invest the saved time in innovations and research

Spend the saved time in innovation

If we just adopted a TDD approach, much of this money (time saved as expressed in point 5) could be spent on new innovations instead.

7. TDD Helps You Avoid Scope Creep

The nightmare of any project manager is scope creep — any unexpected growth in the scope of work which leads to delays in project delivery.

Scope creep can happen for various reasons: poorly defined tasks, misinterpretation of project requirements, lack of documentation, and so on. There are many methods aimed at mitigating scope creep, and TDD is one of them.

Thanks for reading! Please share it if you found it useful :)