For me personally, there is a deep satisfaction that comes with seeing your product come alive under load to find out how it’ll actually work in production when thousands of users will be using it.

Performance testing is about discovering how your system performs under real-world pressure in terms of load, concurrency, and throughput. One of the key aspects of performance testing is ensuring that the APIs can endure the expected load. You can do this using tools like Apache JMeter and K6.

In this tutorial, we’ll explore how you can build your first end-to-end performance test in Apache JMeter. You will be learning to create a test suite that is dynamic (the test can be run with any test data) and that’s one-click executable (the test execution can be done through the GUI as well as the CLI).

Table of Contents

1. Prerequisites

2. Introduction to Apache JMeter

   • Step 1: Create a New Test Plan

   • Step 2: Configure the Thread Group

   • Step 3: Add HTTP Request Defaults

   • Step 4: Add a CSV Data Set Config (Dynamic Input)

   • Step 5: Add the HTTP Request Sampler

   • Step 6: Add a JSON Extractor

   • Step 7: Add an Assertion

   • Step 8: Add Listeners

   • Step 9: Run Your Test

   • Step 10: Chain Another Request (Optional)

   • Step 11: Analyze the Results

   • Pro Tips

   • Example Folder Structure:

3. Conclusion

Prerequisites

Before you start, make sure you have:

• Apache JMeter (5.5 or above) installed.

• Java 8 or later configured on your system.

You can check if JMeter is installed by running the command below:

jmeter -v

Note: This tutorial will use the JSONPlaceholder public API. You’ll learn how you can get a post_id and use it in a chain request to get user details.

Let’s get started.

Introduction to Apache JMeter

Apache JMeter is an open-source API load and stress testing tool. It’s a powerful testing tool that supports a wide range of protocols, including HTTP, HTTPS, FTP, JDBC, SOAP, and REST.

JMeter helps you answer critical questions about your APIs, like:

• How does my API perform under heavy load?

• What’s the maximum number of users it can handle before it starts failing?

• Which requests or endpoints are slowing things down?

Let’s go through the step-by-step process of building a dynamic load testing suite with JMeter.

Step 1: Create a New Test Plan

Once JMeter opens, you’ll see an empty Test Plan. Think of this as your main workspace, which holds everything: Test configuration, users, requests, assertions, and results.

Right-click on Test Plan → Add → Threads (Users) → Thread Group to add a thread group. A thread group is essentially a test suite containing our test cases.

Step 2: Configure the Thread Group

To configure the thread group, fill out the following input fields:

You’ve now created a small, controlled load test of 10 total requests (5 users × 2 loops).

Step 3: Add HTTP Request Defaults

When you’re creating a suite of 100s of APIs, you don’t need to add your request details to all the API samplers in JMeter. JMeter lets you set it once globally by using a config element called HTTP Request Defaults. To add this element, follow the steps below:

1. Right-click on Thread Group → Add → Config Element → HTTP Request Defaults.

2. Enter the following:

   • Protocol: https

   • Server Name or IP: jsonplaceholder.typicode.com

This means all requests in this test will automatically use this base URL.

Step 4: Add a CSV Data Set Config (Dynamic Input)

In real projects, APIs rarely use static inputs. Take as an example a login API that you want to run for 100 concurrent users. In a real-world scenario, every login request will have a different username and password.

To replicate this on JMeter, you need to run your test for 100 different login credentials. This means that your test should be test data-driven. We can build a data-driven test in JMeter using a CSV file:

1. Create a file named data.csv with the following content:

    post_id
    1
    2
    3
    4
    5
   

2. Save it in your JMeter project folder.

3. In JMeter, right-click on Thread Group → Add → Config Element → CSV Data Set Config.

   

4. Fill in the following fields:

   • Filename: data.csv

   • Variable Names: post_id

   • Recycle on EOF: True

   • Stop thread on EOF: False

     

Now each user will pick a new post_id for every iteration from the CSV file.

Step 5: Add the HTTP Request Sampler

Now let’s add the actual API call we'll test under load. To do this, follow the steps below:

1. Right-click on Thread Group → Add → Sampler → HTTP Request.

   

2. Rename it to Get Post Data.

3. Set the following fields:

   • Method: GET

   • Path: /posts/${post_id}

Here ${post_id} dynamically takes its value from your CSV file. The Protocol and Server IP fields will automatically get data from the ‘HTTP Request default’ config element that we added in Step #3.

Step 6: Add a JSON Extractor

When the API returns a response, we can extract a value (like userId) from it and use it later. This is used to implement an end-to-end flow where data is gotten (with GET) from an API and sent to the next POST/DELETE API.

For our API, below is the example response:

{
  "userId": 1,
  "id": 3,
  "title": "fugiat veniam minus",
  "body": "This is an example post body"
}

To extract userId:

1. Right-click on Get Post Data → Add → Post Processors → JSON Extractor.

   

2. Set the variables below in the JSON Extractor:

   • Name: Extract User ID

   • Variable Name: user_id

   • JSON Path Expression: $.userId

Now you can use ${user_id} in the next request, making your test fully dynamic.

Step 7: Add an Assertion

Assertions help you verify that your API responds correctly even under load. You can assert on the API response code, response time, or even the response payload. To add an assertion, follow the steps below:

1. Right-click Get Post Data → Add → Assertions → Response Assertion.

   

2. Configure as:

   • Response Field to Test: Response Code – This will add an assertion for the response code.

   • Pattern Matching Rules: Contains

   • Pattern to Test: 200

This ensures JMeter only counts the request as successful if the word fugiat appears in the response.

Step 8: Add Listeners

We’ll add listeners to display our test results in different forms, such as visually or in a summary. Let’s add two essential ones:

1. View Results Tree: to view and debug individual requests.

2. Summary Report: to view performance metrics like response time, error rate, and throughput.

Add them via Thread Group → Add → Listener → [Choose Listener]

Step 9: Run Your Test

Hit the green Start button at the top. JMeter will start sending requests to your API using the dynamic post IDs from your CSV file.

As the test runs:

• Green checkmarks in View Results Tree mean successful responses.

• Assertion failures will appear in red.

• Summary Report will aggregate key metrics.

Step 10: Chain Another Request (Optional)

Let’s take it one step further: we’ll use the extracted user_id from the first response to get user details from the GET users call. To do this, follow the steps below:

1. Right-click Thread Group → Add → Sampler → HTTP Request.

2. Rename to Get User Details.

3. Set:

   • Method: GET

   • Path: /users/${user_id}

     

Step 11: Analyze the Results

Once the test completes, open the Summary Report. You’ll see:

If your error percentage is 0% and throughput is stable, your system handled the load well.

Pro Tips

• Parameterize everything. Use multiple CSVs for realistic test flows (users, IDs, tokens).

• Add timers (like Constant Timer) to simulate think time between user actions.

• Use Assertions wisely. Don’t add extra assertions; focus on key validations such as response time and API status code.

• Generate HTML reports using the command below:

    jmeter -n -t test-plan.jmx -l results.jtl -e -o report
  

Example Folder Structure:

Follow the folder structure below for an organized test suite.

performance-test/
├── data.csv
├── test-plan.jmx
└── results/
    ├── summary.csv
    └── report.html

Conclusion

Performance testing is an essential element of a production readiness checklist for any product. It helps you ensure that your product can handle the expected user load and scale gracefully.

This guide is your first step towards writing end-to-end performance test cases and bridging the gap between being a functional test engineer and a full-stack QA Engineer who understands both quality and scalability.

I hope you found this tutorial helpful. If you want to stay connected or learn more about performance testing, follow me on LinkedIn.

Usually, a QA tester creates a report after discovering a bug. The next step is that they send a report to the bug handler. Basically, a bug handler is a developer that fixes a bug according to the detailed report. The bug handler ensures a clean and efficient QA process and makes it easier to have a clean conversation between the fixers and the testers.

But you might be wondering – why do we need QA documentation?

QA testing is essential in IT products today. Tight deadlines, unique skill sets within the organization, and the demand to develop products stress the need for a structured methodology. QA documentation guides testers through levels of clarity and overall coverage.

I've written this article to make your life a bit easier. So here it is, your ultimate guide on how to write software QA documentation that will work.

This is what we'll cover:

• Make a Test Plan and a Test Progress Report
• Create Test Cases
• Defect Reports
• Useful Tips for Defect Report Writing
• Submit a Defect Report
• Conclusion

Make a Test Plan and a Test Progress Report

Developing great software requires a thorough and documented approach to testing. Everything begins in the development phase with a complete checkout plan as a template for the entire QA process. It outlines the general requirements, defines the followed route, identifies the necessary assets, and sets a simple timeline for achieving the objectives.

Draw up a roadmap before you start

Before you start implementing your test plan, take some time to think about the big picture. Ask yourself:

What problem does this software solve? As soon as you understand the main purpose of the software, you'll be able to manage the checkout process and identify the features that are most important to your goals.

I promise that once you see that you can do this, you quickly prioritize your efforts. They are based primarily on the importance of the functionality and its impact on the user.

Create a plan

The test plan is the central part of the QA technique. The test plan should contain the following key elements:

• Imaginary test: Clearly identify the expected impact of the payment method. Will it confirm that core functionality is running smoothly, or will it alert you to capability gaps?
• Test method: Outline how the test will be performed. Will you do targeted testing, useless testing, or a mix of the two?
• Resource allocation: Identify the equipment and technology required to perform a complete control. What control framework will you use? Are specific hardware or software configurations required for proper control?
• Timetable and remaining dates: Establish a realistic timetable for the trial. Set clear deadlines to keep the project on track.

Understand the 'why' and 'how'

A well-developed roadmap will ensure that key issues are addressed throughout the improvement approach:

• Acceptance requirements: Clear fail/success criteria should be defined for all control cases. So, criteria allow the customer to understand that the product is high quality and ready for the end-user.
• Resource management: Identify the assets required for the testing. This includes preferred machines, copies of software in various forms, required expertise, and more.
• Team dynamics: Ensure that simple roles and tasks are defined within the test team. Who is responsible for particular test cases? Who documents bugs and who talks to developers?
• Time management: I advise setting realistic checkout times, taking into account project deadlines and the availability of useful help.

The test progress report is another part of the QA documentation, similar to the test plan, but with additional data on current progress. This document allows you and your development team to monitor project progress and identify any organizational issues.

A test plan and a progress report

Create Test Cases

Think of each test case as a detailed recipe for finding potential defects in the functionality of a software program. By following these "recipes" and evaluating the expected and actual results, defects can be accurately identified and addressed before preventive action is taken.

Each test case acts as an independent unit, outlining the steps required to evaluate a particular element of a software program.

Below is a breakdown of the key elements that make up a well-defined test case Here is my step-by-step guide for creating test cases:

• ID: This is an identifier field to help distinguish between test cases and track them easily.
• Priority: This indicates the severity of the test case based on the functionality of the program and its impact on the normal overall performance of the known software program application.
• Test Requirements: These are requirements for testing software successfully, this can include reference documents also.
• Software Module: This shows the feature under testing. It also refers to the software requirements specifications (SRS) document explaining the software feature in detail.
• Test Context: This details the test plan to clarify how the daily tests will be carried out. It also identifies the test data required for a successful case study and includes unique and important statistical information.
• Expected Output: This describes the expected output to be displayed if the test is successful.
• Actual output: This indicates the actual result in the event of failure and shows the developer the errors in the application module of the software program.
• Comment: This is an optional section where the tester can give a description of the observations or add additional records.

All QAs typically include the above elements, but can also be designed specifically for the task selected by the QA group. Also, each control case follows a lifecycle that defines the phases of creation, testing (pass, fail), completion, and so on.

In the next section, we’ll check out another important element of QA documentation: the defect report.

Test cases

Defect Reports

Defect reporting is an important element of QA documentation. Issue tracking is the detailed reporting of sudden problems that arise in a software product. Careful documentation of these issues lays the groundwork for a complex and bug-free final product.

Sounds simple, right? Yes, but only until you start documenting.

A defect report

The bug report consists of the following sections: Identifier, Summary, Description, Steps to Reproduce, Reproducibility, Severity, Priority, Environment, and Attachments.

• Identifier: Each software program problem is assigned a completely unique identifier that acts like a customized nameplate. This makes the QA documentation less complicated to navigate and allows verbal communication between the installer, tester, and project manager (PM).
• Summary: This is an opportunity to provide brief and informative answers to three basic questions: What was the problem? Where did the problem appear? Under what specific conditions did the problem appear?
• Explanation: Examine the failure log in more detail. Summarize the identified action (the result completed) and check it against the planned action (the predicted end result). Including a link to the app requirements of the software program can serve as a useful reference element.
• Reproduction method (STR): This phase should be considered as a step-by-step recipe for reproducing the defect. It should be strict and cover all the steps that caused the problem. Skipping critical steps can make it difficult for you to reproduce the problem and cause delays.
• Reproducibility: In this section, you clarify if the bug appears every time you follow the STR. You should use numbers to show approximate chances, for example, 7 times out of 10.
• Severity: Here you explain how much harm the bug may bring to the project. Essentially, this is a measure of the severity of the technical disruption that a bug will cause to the whole project. Remember that problems that are known to be minor can grow and cause you extreme problems throughout the software.
• Priority: Each error log assigns a problem priority that indicates its urgency. Common priorities are letters (A: highest priority, Z: highest priority), numbers (1: highest priority, 9: highest priority), or descriptive terms (high, medium, low).
• Environment: Specify the gadget or browser model in which the bug appeared. This will help you to put the problem in context and narrow down a valid cause.
• Attachments: If possible, enrich the documentation with screenshots, screen recordings, console log documents, and others. This will help to provide visual documentation of the error.

My structure provides detailed information, so you can empower developers to effectively diagnose, address, and eliminate any software bugs. In this way, leading to more user-friendly and robust products.

Now, in the next section, we'll see some useful tips you can use for defect report writing.

Useful Tips for Defect Report Writing

1. Write a sufficient summary. It does not matter if it is long or short. What matters is that it should be clear.
2. Have a look at the summary and the description. Do they look pretty much the same? You must have forgotten to outline the expected and actual results in the description and to add the link to the requirements.
3. Capture the issue with the help of a screenshot. It may save you and the development team a lot of time. Sometimes one glance at the picture is just enough to understand the situation.
4. Before reporting the issue, try to reproduce it at least 3 times to be sure that it exists.
5. Report the issue ASAP and notify your project manager or product owner if the issue is major.
6. Check for grammar mistakes in your QA documentation so you're not taken down by the grammar police.
7. However comical it sounds, make sure that the issue is not a feature – review the documentation again!
8. Do not miss any important information in your Steps to Reproduce.

Submit a Defect Report

The final and one of the most important elements of QA documentation is the defect report. You may understand it covers the entire lifecycle of a problem, from its initial discovery to its final closure.

Let's now examine the key areas of this process:

a defect report lifecycle

We'll go over the defect report lifestyle piece by piece:

Problem Reporting:

This adventure begins with the careful compilation and submission of a report on the entire program. This serves as a roadmap for developers and provides a clear assessment of the problem.

Triage and tasks:

The task manager or technical lead takes on the role of gatekeeper at this stage. They carefully compare the files. If the file contains enough elements to work with, it is assigned to the developer and repaired. However, if the file is missing essential elements, it may be rejected for further improvement.

Bug fixing:

The assigned developer takes the initiative and works diligently to eliminate the annoying bug.

Verification and completion:

Once the developer claims to have fixed the problem, it's your turn as QA. Carefully verify the fix by retesting the functionality in question. If everything works as it should, you are done.

The documentation has come to a happy end. Ideally, this will happen within a reasonable period of one to two weeks.

Reboot and keep going:

But it is not always that simple. If it is known that bugs are still being made in the validation system, there is no need to despair!

Re-open the bug documentation and send it to the developers for further attention. Sometimes the process of fixing bugs is repetitive and requires patience. But by being careful and effective, you can guarantee that all bug reports will eventually reach their final destination, resulting in a more polished and reliable software application product.

Conclusion

Quality Assurance is a process you simply cannot avoid. Each airplane prior to departure undergoes a technical check. If there is any issue, the aircraft is grounded until the problem is solved. The same goes for any software.

But QA documentation is not always "write and ignore". At some point in the software development lifecycle, QA documentation needs to be continually updated and improved as requirements change, new features are introduced, and feedback is received from deployment and production use.

There are also a growing number of styles that use synthetic intelligence and machine learning to partially automate the creation of QA documentation.

For example, natural language processing (NLP) is used to analyze requirements documents and generate draft control examples. Test bots can use NLP to interpret and routinely execute directed test cases. Predictive evaluation can also be used to identify the most dangerous areas of a software program that require more detailed control.

While these strategies are still new and not mature enough to replace human testers, they can help with growth and augment manual exploration, especially for large and complex builds. By making QA documentation a collaborative and ongoing hobby, your team can deliver better software faster and with fewer defects.

Do you need to improve the quality of your software?

My company KeenEthics provides solid development and quality assurance services. In case you need an estimate for a similar project, feel free to get in touch.

If you have enjoyed the article, you should continue with How IT Outsourcing Saves Costs for Your Company and Avoiding Pitfalls in IT Outsourcing: Tips for Minimizing Risks.

But how do you become a QA Engineer? In this article, I will talk about the qualifications and process for becoming a QA engineer.

What is Quality Assurance and Quality Control?

When a product is being built, stakeholders have expectations and requirements. It is important that there is a process put in place to ensure that those high quality standards are being adhered to and mistakes in the product development are spotted and reported.

Quality Assurance focuses more on proactive processes while Quality Control focuses more on reactive corrections that need to be made.

Quality control also focuses on inspection and testing whereas Quality Assurance involves documentation, audits, and management.

Core skills needed for becoming a good QA engineer

Attention to detail

The role of a QA engineer involves being detail-oriented and thorough in order to spot issues or discrepancies in quality standards set for the product.

Good communication

QA engineers will be communicating with stakeholders, management, and developers for the product. Effective and clear communication between all parties involved is really important for the development process of a product.

Good time management

Since a QA engineer will have to balance multiple tasks throughout the day, it is important to have strong time management skills to ensure that you stay productive and are able to meet your deadlines.

Problem solving skills

Throughout the life cycle of a product, issues will arise. It is important that QA engineers have strong problem solving skills and are able to develop a plan to solve those issues.

Strong listening skills

A good QA engineer not only needs to be able to communicate but also listen to their development team, management, and stakeholders to ensure the successful launch of the product.

The ability to write a test plan

A test plan is a document detailing the objectives and testing processes that will be applied to the software product. QA engineers will work with other developers and stakeholders to develop this testing strategy.

An understanding of test automation tools and manual testing practices

It is helpful for QA engineers to be familiar with different automated testing tools like Selenium. They will also need to know how to effectively manually test products in order to identify bugs.

Strong understanding of the Software Development Life Cycle

A good QA engineer should understand the full life cycle of software products in order to effectively plan and test them.

Here are the 7 phases of the Software Development Life Cycle:

1. Planning Stage
2. Analysis Stage
3. Design Stage
4. Software Development Stage
5. Software Testing Stage
6. Implementation and Integration
7. Operations and Maintenance Stage

Do you need to learn how to code become a QA engineer?

Learning programming fundamentals can help QA engineers build automated test suites to save time on manual testing and provide a better understanding of the development implementations by the software developers.

Some widely used programming languages to learn would be Java, C#, JavaScript, and Python.

Choose one language to learn to get an understanding of the programming fundamentals.

Do you need a Computer Science degree to become a QA engineer?

Having a Computer Science degree can help you when it comes to finding your first Quality Assurance job because a lot of employers still prefer a degree. If you have the time and money to pursue that path, that can be a viable option.

Other options would be to go to a bootcamp or teach yourself through online resources. If you are considering a bootcamp, please read through this helpful resource first.

Do you need certifications to become a good QA engineer?

QA certifications can help boost your resume and knowledge base. A lot of these certifications do require that you have degree from an accredited university or a few years of experience.

Here are some QA certifications you can look into:

• Certified Associate in Software Testing (CAST)
• Certified Test Engineer (CSTE)
• Certified Software Quality Analyst (CSQA)

How to get a job as a QA engineer

Once you have learned programming, testing, and QA fundamentals then you can start looking for junior level positions. Make sure to build out a strong résumé and prepare for the QA interview questions using free online interview guides.

If you are self taught, it would help to have a little bit of experience through small contract work. If you are in a degree program, students will have access to internship opportunities and other career resources.

Salaries for QA engineers

Salaries for QA engineers will range depending on experience and location. The average base US salary according to payscale is $72,000.

I hope you enjoyed this article and best of luck on your software journey.

First we're going to talk about what testing is, why is it a good idea to test our code, and the different kinds of tests that can be implemented.

Then we're going to introduce each of the tools we will be using, and finally we'll give practical examples for vanilla JS code, a front-end React app, and a back-end Node app.

Andiamo via!

Table of Contents

• What is testing and why is it valuable

• Different types of tests

  • Manual vs Automated testing

  • Functional vs Non-functional testing

  • Unit vs Integration vs End-to-end testing

  • White box vs Black box vs Grey box testing

• When to test

• Our toolset

  • What is Jest

  • What is Testing library

  • What is Cypress

  • What is Supertest

  • Tools roundup

• How to test vanilla JS code

• How to test a front-end React app with Jest and React testing library

• How to test a front-end React app with Cypress

• How to test a back-end Node app

• Wrap up

What is Testing and Why is it Valuable

Testing is the practice of checking if a piece of software runs as expected. This is often recognized as QA or quality assurance, and aims to reduce to a bare minimum the amount of bugs that get to production.

We test software to identify errors, gaps or missing requirements and fix those things before shipping code to production.

Testing our code thoroughly improves our project's reliability, saves us later bug-fixing time and hence reduces costs, and improves the chance of our customer being fully satisfied with our product.

Here's a nice short video by Fireship introducing some of the concepts we'll talk about later.

Different Types of Tests

Testing practices can be classified in different types according to many factors. Personally I think there's a lot of mumbo jumbo about this topic, with hundreds of terms that often refer to very similar things. So let's keep it simple and review only the most popular terms and what they mean.

This will help clarify the many ways in which a software can be tested and better understand the tools we're going to present later on.

Manual vs Automated testing

Depending on the tools we use to test our software, we can classify testing into manual or automated testing.

Manual testing is the practice of "clicking around" and manually checking all the features our product has, simulating what an actual user would do.

Automated testing is done through code, writing programs that check how our application runs.

There're many testing frameworks and libraries we can use for this. When it comes to functional testing (we're going to see what that means in a sec), most libraries work in a similar way:

• First we define what piece of code we want to test.

• Then we provide that piece of code some sort of input or execute an action on it.

• Then we define what that piece of code should do given the input/action we performed.

• And finally we will compare what that piece of code actually did against what we said it should do.

If it did what we said it should, the test passed. If it didn't, it failed.

Functional vs Non-functional testing

Functional testing refers to the actual features of our product. For example, if we have a blog platform, functional testing should assure the users can create new articles, edit those articles, browse through articles written by other people, and so on.

Non-functional testing refers to anything that's not strictly related to the core features of our product. And that again can be classified into different categories, for example:

• Stress testing checks how infrastructure responds to heavy usage.

• Security testing checks if an application is vulnerable to common hacking attacks.

• Accessibility testing checks if an application is coded in a way that is accessible for people with different disabilities.

Unit vs Integration testing vs End-to-end testing

Another way to classify testing is depending how broad or comprehensive it is.

Unit testing aims to test individual functions, methods or small chunks of code in an independent way. In unit testing, small pieces of code are checked in an isolated way.

Integration testing checks how individual pieces of code interact with each other and work together. In integration testing, we put pieces together and see if they interact correctly.

End-to-end testing, also known as E2E, executes programs in a simulated environment that emulates actual user behavior. Having a website as an example, our code would open in an actual browser and all the features would be executed in the same way a user would use them. E2E testing is a lot like manual testing in that sense, but fully automated.

E2E testing is the most broad or comprehensive type of these three, as it evaluates whole features and behaviors, not specific parts of our code.

White box vs Black box vs Grey box testing

The last classification we're going to see depends on how much our tests focus on implementation details or user experience.

Let's say we have a simple website with a button that, when it gets clicked, it opens a modal. In our code, the button has a click event listener that executes a function. That function changes the CSS class of our modal HTML element, and that gets the modal rendered in the screen.

We talk about "white box" testing when we test implementation details. Following the example, under this paradigm we could test that the button click executes the corresponding function, and that after the function execution, the CSS class of our modal element is changed accordingly.

Another way to do this is to forget about implementation all together and just check if the modal is rendered after the button click. We don't care what the CSS class is, or if the corresponding function is executed or not. We just focus on testing what the user should perceive. That's "black box" testing.

And, as you may have guessed, "grey box" testing is just a combination of the previous two.

One last thing to mention here is that these different types of tests aren't necessarily mutually exclusive. I mean, they can and often are implemented at the same time on the same projects.

It's very common to have both manual and automated testing, functional and non-functional testing, unit and E2E testing ... The idea will always be to try to anticipate and solve the greatest possible number of problems in reasonable time and effort.

When to Test

This may seem like a simple question at first, but there are actually different approaches to this, too.

Some people like to test their app once it's been fully developed. Others like to write tests at the same time they code the application, and test each feature as it's being developed.

Others like to write tests first before anything else, defining in this way the minimum requirements for the program to accomplish. And then they code the app in a way that passes those tests as fast as possible (this is called test driven development or TDD).

Once you have an app or a whole feature developed, and you have a test suite in place (a test suite is a group of tests that check a particular feature or an entire app), another common practice is to run your tests each time you make any kind of modification to the codebase, to verify nothing gets broken.

Lastly, if you have a CI/CD system in place, it's common to automate the execution of tests before any deployment. So that if any test fails, the deployment is stopped and some kind of alert is dispatched (which of course is always better than seeing your app catch fire on prod 🔥😱).

Same as with test types, it's common to test applications at different times. Each company normally has its own testing schedule or practice to follow, tailored to their needs.

Our Toolset

Ok, now that we have a clearer idea of what testing is and the types of tests we can perform, let's review the tools we're going to use in our examples.

As mentioned before, there are a lot of different libraries to choose to run our tests. I chose these four because they are some of the most popular when it comes to JavaScript apps, but know there are more options out there. I'll be naming alternatives for most of the tools we'll use in case you'd like to investigate more. 😉

What is Jest

Jest is a JavaScript test-runner. A test-runner is a piece of software that allows you to run tests to evaluate your app. It's an open-source project maintained by Meta (formerly Facebook), and was first open-sourced in 2014.

Side comment: Every time I say "test runner" I picture this. Am I the only one? 🤔

Test runner, not Blade runner!

Anyway...you can use Jest in projects that use Babel, TypeScript, Node.js, React, Angular, Vue.js, Svelte and other technologies too. You can install Jest through NPM just like any library and it requires very little configuration to start off.

Jest comes installed by default when setting up React apps with create-react-app.

Jest is often also called a testing framework, as it comes with many other built-in features besides just running tests (which is not the case with all test runners). Some of those features are:

• Assertion library: Jest comes with a lot of built-in functions and methods you can use to assert your code (asserting basically means checking if a piece of code behaves like it's expected).

• Snapshot testing: Jest allows you to use snapshots, which are a way of capturing a large object and storing it in memory so you can later on compare it with something else.

• Code coverage: Jest allows you to get code coverage reports of your tests. These reports show what percentage of your code is currently being tested, and you can even see the exact lines of code that aren't currently being covered.

• Mocking library: Jest also works like a mocking library in the sense that it allows you to mock data (like a function or a module) and use that in your tests.

Some well known alternatives to Jest are Mocha, Jasmine, and Karma.

Here's a nice little video explaining what Jest is.

What is Testing Library?

Testing library is not a test runner, but a set of utilities that will work together with a test runner like Jest or Mocha. This utilities are tools we can use to test our code easily and with a deeper focus on user experience (black box testing).

Testing library was developed by Kent C Dodds (who also happens to be one of the best JS teachers on earth, so I recommend that you follow him).

Quoting the official docs:

"The Testing Library family of libraries is a very light-weight solution for testing without all the implementation details.

The main utilities it provides involve querying for nodes similarly to how users would find them. In this way, testing-library helps ensure your tests give you confidence in your UI code."

In plain English, with the testing library we can test UI elements (like a paragraph, a button, a div...) instead of testing the code responsible for rendering the UI.

The principle behind the library is:

"The more your tests resemble the way your software is used, the more confidence they can give you."

... and that's exactly what we mean by "black box" testing. 😉

The testing library is actually a set of libraries, each created to achieve the same objective but adapted to work with different technologies such as React, Angular, Vue, Svelte, React Native and more... That's why you might hear "React-testing-library" or "Vue-testing-library". It's the same thing but adapted to work with different technologies.

React-testing-library comes installed by default when setting up React apps with create-react-app.

An alternative to testing library is Enzyme (a UI testing set of utilities developed by Airbnb).

What is Cypress?

Cypress is an open source test-runner that allows you to execute your projects in an automated browser, in the same way a user would.

With Cypress, we can program what the browser will do (like visit a URL, click a button, complete and submit a form...) and check that each action is matched with the corresponding response.

What's sweet about this is that the testing resembles A LOT to what the user will experience. And since the whole point of making software is the user, the closer we are to their perspective, the closer we should be to catching the most meaningful bugs in our code. (Plus it's really cool to see an automated browser go through your entire app in just a few seconds... 🤓)

Another nice feature of Cypress is "time travel". On Cypress's automated browser we can see all the test's we've written, and simply hover over them to see a graphical snapshot of its result. It's a very useful thing to better understand what's breaking and when.

Even though it can be used for unit and integration testing, Cypress is mostly used for end-to-end testing as it can easily evaluate complete features in a matter of seconds.

You can use Cypress to test anything that runs in a browser, so you can easily implement it on React, Angular, Vue, and so on.

Unlike Jest and React-Testing-Library, Cypress doesn't come pre-installed with create-react-app. But we can easily install it with NPM or your package manager of choice.

Some alternatives to Cypress are Selenium and Puppeteer.

Here's a sweet video by Fireship explaining what Cypress is and how it works.

Side comment: ...and every time I talk about Cypress this plays in my mind. 😎

What is Supertest?

Supertest is a library that simulates HTTP requests. It's super handy to test back-end Node apps together with Jest (as we will see in the coming examples).

Tools roundup

As a quick round-up about this topic:

• Jest is the library that we'll use to write and run tests for JavaScript.

• Testing library works together with Jest, and provides us with functions and methods to test the UI directly, forgetting about the code behind it.

• Cypress runs your app in a simulated browser and checks if actions performed in the UI respond as expected.

• Supertest is a library that mocks HTTP requests and it can be used together with Jest to test back-end apps.

Now let's begin with the fun part ...

Let the testing begin!!

How to Test Vanilla JS Code

Ok, let's start by testing some simple vanilla JS code. The idea here is to see how we can implement Jest in our project and learn the basics of how it works.

Let's start by creating a new directory in our machine and creating a Node app with npm init -y. Then install Jest by running npm i -D jest (-D saves it as a development dependency).

Now you should see something like this in your package.json file: "devDependencies": { "jest": "^27.5.1" } .

And speaking about it, in your package.json, replace your test script with "test": "jest". This will allow us to later run our tests by running npm test. ;)

Your entire package.json file should look something like this:

{
  "name": "vanillatesting",
  "version": "1.0.0",
  "description": "",
  "main": "index.js",
  "scripts": {
    "test": "jest"
  },
  "keywords": [],
  "author": "",
  "license": "ISC",
  "devDependencies": {
    "jest": "^27.5.1"
  }
}

Cool, we're all set to write some JS we can actually test! Create an index.js file and put this code in it:

// index.js
function isPalindrome(string) {
    // O(n)
    // Put a pointer at each extreme of the word and iterate "inwards"
    // At each iteration, check if the pointers represent equal values
    // If this condition isn't accomplished, the word isn't a palindrome
    let left = 0
    let right = string.length-1

    while (left < right) {
        if (string[left] === string[right]) {
            left += 1
            right -= 1
        }
        else return false
    }

    return true
}

This function is a palindrome checker. It receives a string as parameter, and returns true if the string is a palindrome and false if it's not. (This is a classic technical interview question btw, but that's stuff for another article.🤫)

See that we're exporting the function too. Side comment: If you'd like to know more about this and how JS modules work, I recently wrote an article about it.

Great, so now let's test this function and see if it works as expected. Let's create a file called index.test.js.

This file is where we'll write our tests. The suffix we're using (.test.js) is important here, as Jest will automatically identify the .test files and execute them when we order Jest to test our project.

Jest also identifies files with the .spec suffix, like index.spec.js (for "specification", which refers to the requirements of your project). Personally I prefer .test as it feels more explicit to me, but both work the same.

Now let's write our first tests! Put this within your index.test.js file.

// index.test.js
isPalindrome = require('./index.js')

test('neuquen is palindrom', () => {
    expect(isPalindrome("neuquen")).toBe(true)
})

test('bariloche is not palindrom', () => {
    expect(isPalindrome("bariloche")).toBe(false)
})

Let's recap what we're actually doing:

1. Require the function we want to test: isPalindrome = require('./index.js')

2. The test() function is provided by Jest and within it we will put the code we want Jest to execute.

3. test() takes two parameters. The first is a test description, which is a distinctive name that will show on our console when the test is run. We'll see an example in a sec.

4. The second parameter is a callback, which contains the actual testing code.

5. Within this callback we're calling the expect() function (also provided by Jest). expect() takes our function as parameter, which itself is receiving a parameter we made up.

6. Last, we chain the .toBe() function (provided by Jest too) and as parameter we pass it the value we expect isPalindrome() to return for each case. ("neuquen" is a palindrome so our function should return true, and "bariloche" is not, so it should return false.)

One of the things I like the most about Jest is how easy it is to set it up. Another thing I like a lot is how self explanatory its syntax is. Notice that we can easily understand what our tests will evaluate by just reading them.👌

Now let's try this! If we run npm test in our console, we should get the following:

// console
> jest PASS 
./index.test.js
✓ neuquen is palindrom (1 ms)
✓ bariloche is not palindrom

Test Suites: 1 passed, 1
total Tests:       2 passed, 2
total Snapshots:   0
total Time:        0.244 s
Ran all test suites.

Congratulations, you just passed your first Jest test ever.

To see how a failing test looks too, let's change our function by editing the return lines.

// index.js
function isPalindrome(string) {
    // O(n)
    // Put a pointr at each extreme of the word and iterate "inwards"
    // At each iteration, check if the pointers represent equal values
    // If this condition isn't accomplished, the word isn't a palindrome
    let left = 0
    let right = string.length-1

    while (left < right) {
        if (string[left] === string[right]) {
            left += 1
            right -= 1
        }
        else return 1
    }

    return 2
}

Now you should get something like this:

// console
> vanillatesting@1.0.0 test
> jest

 FAIL  ./index.test.js
  ✕ neuquen is palindrom (4 ms)
  ✕ bariloche is not palindrom

  ● neuquen is palindrom

    expect(received).toBe(expected) // Object.is equality

    Expected: true
    Received: 2

      3 | // describe('isPalindrome function', () => {
      4 |   test('neuquen is palindrom', () => {
    > 5 |     expect(isPalindrome("neuquen")).toBe(true)
        |                                     ^
      6 |   })
      7 |
      8 |   test('bariloche is not palindrom', () => {

      at Object.<anonymous> (index.test.js:5:37)

  ● bariloche is not palindrom

    expect(received).toBe(expected) // Object.is equality

    Expected: false
    Received: 1

       7 |
       8 |   test('bariloche is not palindrom', () => {
    >  9 |     expect(isPalindrome("bariloche")).toBe(false)
         |                                       ^
      10 |   })
      11 | // })
      12 |

      at Object.<anonymous> (index.test.js:9:39)

Test Suites: 1 failed, 1 total
Tests:       2 failed, 2 total
Snapshots:   0 total
Time:        0.28 s, estimated 1 s
Ran all test suites.

See that you get a nice description of what tests failed and at which point they failed. In our case they failed when we asserted (checked) the return values.

This is very useful and we should always pay attention to these descriptions, as some times our tests may fail because they're not written correctly. And we don't normally write tests for our tests, yet... 😅 So when you see a failing test, first check that it's working as expected and then go review your actual code.

Now let's add and test another function to show some more Jest features:

// index.js
function twoSum(nums, target) {
    // O(n)
    // Iterate the array once
    // At each iteration, calculate the value needed to get to the target, which is target - currentValue
    // If the neededValue exists in the array, return [currentValue, neededValue], else continue iteration
    for (let i = 0; i < nums.length; i++) {
        const neededNum = target - nums[i]
        if (nums.indexOf(neededNum) !== -1 && nums.indexOf(neededNum) !== i) return [nums[i], nums[nums.indexOf(neededNum)]]
    }
    return false
}

module.exports = { isPalindrome, twoSum }

This is another classic interview question. The function takes two parameters, an array of numbers and a target value number. What it does is to identify if there are two numbers in the array that add up to the second parameter value. If the two values exist in the array, it returns them in an array, and if they don't, it returns false.

Now let's write some tests for this:

({ isPalindrome, twoSum } = require('./index.js'))

...

test('[2,7,11,15] and 9 returns [2, 7]', () => {
    expect(twoSum([2,7,11,15], 9)).toEqual([2,7])
})

test('[3,2,4] and 6 returns [2, 4]', () => {
    expect(twoSum([3,2,4], 6)).toEqual([2,4])
})

test('[3,2,4] and 10 returns false', () => {
    expect(twoSum([3,2,4], 10)).toBe(false)
})

See that the structure is almost the same, except we're using a different matcher in two of the tests, toEqual().

Matchers are the functions Jests provides us with to evaluate values. There are many types of matchers that can be used for many different occasions.

For example, .toBe() is used to evaluate primitives like strings, numbers, or booleans. toEqual() is used to evaluate objects (which covers pretty much everything else in Javascript).

If you need to compare the return value with a number you could use .toBeGreaterThan() or toBeGreaterThanOrEqual() and so on...

To see a full list of the available matchers, check the docs.

If we run our tests now, we will get the following:

> vanillatesting@1.0.0 test
> jest

 PASS  ./index.test.js
  ✓ neuquen is palindrom (2 ms)
  ✓ bariloche is not palindrom
  ✓ [2,7,11,15] and 9 returns [2, 7] (1 ms)
  ✓ [3,2,4] and 6 returns [2, 4]
  ✓ [3,2,4] and 10 returns false (1 ms)

Test Suites: 1 passed, 1 total
Tests:       5 passed, 5 total
Snapshots:   0 total
Time:        0.256 s, estimated 1 s
Ran all test suites.

That's cool, but our tests results look a bit messy. And as our tests suite grows, it will probably be harder to identify each separate result.

To help us with this, Jest provides us with a describe() function, which we can use to group tests together and show results in a more schematic way. We can use it like this:

({ isPalindrome, twoSum } = require('./index.js'))

describe('isPalindrome function', () => {
  test('neuquen is palindrom', () => {
    expect(isPalindrome("neuquen")).toBe(true)
  })

  test('bariloche is not palindrom', () => {
    expect(isPalindrome("bariloche")).toBe(false)
  })
})

describe('twoSum function', () => {
  test('[2,7,11,15] and 9 returns [2, 7]', () => {
    expect(twoSum([2,7,11,15], 9)).toEqual([2,7])
  })

  test('[3,2,4] and 6 returns [2, 4]', () => {
    expect(twoSum([3,2,4], 6)).toEqual([2,4])
  })

  test('[3,2,4] and 10 returns false', () => {
    expect(twoSum([3,2,4], 10)).toBe(false)
  })
})

The first parameter is the description we want to show for the given group of tests, and the second is a callback that contains our tests. Now if we run npm test again, we get this 😎:

// console
> vanillatesting@1.0.0 test
> jest

 PASS  ./index.test.js
  isPalindrome function
    ✓ neuquen is palindrom (2 ms)
    ✓ bariloche is not palindrom
  twoSum function
    ✓ [2,7,11,15] and 9 returns [2, 7] (1 ms)
    ✓ [3,2,4] and 6 returns [2, 4]
    ✓ [3,2,4] and 10 returns false

Test Suites: 1 passed, 1 total
Tests:       5 passed, 5 total
Snapshots:   0 total
Time:        0.216 s, estimated 1 s
Ran all test suites.

How to Test a Front-end React App with Jest and React Testing Library

Now that we know the basics of Jest, let's hop on to see how we can combine it with Testing library to test a React app.

For this we're going to use a dead simple example. Just a page with random text, a button that toggles another piece of text, a text input, and a button that toggles the rendering of the input.

Take into account we'll be using create-react-app to create this app (which has Jest and Testing library installed by default). If you're not using create-react-app, you might need to install both libraries and add some extra config.

We're not going to see any React code here, we're just going to focus on the tests.

The folder structure of our project is the following:

> src
    > components
        - About.jsx
    - App.jsx
    - Index.js
    - setupTests.js

The setupTests.js file is important here. It's created by default with create-react-app with this content:

// jest-dom adds custom jest matchers for asserting on DOM nodes.
// allows you to do things like:
// expect(element).toHaveTextContent(/react/i)
// learn more: https://github.com/testing-library/jest-dom
import '@testing-library/jest-dom';

It globally imports the jest-dom library provided by Testing library, which gives us additional Jest matchers we can use to test the DOM (like toHaveTextContent(), toBeInTheDocument(), etc).

We're going to see examples in a bit, but know that some of the functions and matchers we'll use come from here.

Regarding our tests files, the common practice is to have a different test file for each component we're testing.

Regarding where to place them, two common practices are to have them all together in a single folder, like __tests__ or similar, or to have each test file in the same folder as the component it's testing.

I prefer the later as I'll often be hopping from the component code to the test code, and it's nice to have them nearby. But trully it doesn't matter. As long as we use the .test or .spec suffixes, Jest will identify and run the files anyway.

Having created our tests files, our folder structure should look like this:

> src
    > components
        - About.jsx
        - About.test.jsx
    - App.jsx
    - Index.js
    - setupTests.js

Cool! Let's start by testing our About component.

First lets test that it's rendering correctly, like this:

// About.test.jsx
import { render, screen } from '@testing-library/react'
import About from './About'

describe('About', () => {

  test('About renders correctly', () => {
    render( <About/> )
    expect(screen.getByText("I'm the about page!")).toBeInTheDocument()
  })

})

• See that we start by importing two things from Testing library: import { render, screen } from '@testing-library/react'.

The render function takes a React component as a parameter and it will render it so we can test it.

screen is an object that comes with lots of queries we can use to test the UI directly, skipping implementation details and focusing on what the user will actually see.

• Then we import our About component: import About from './About'

• We use the describe and test Jest functions previously mentioned.

• We render the About component: render( <About/> )

• We use the expect Jest function, and as a parameter we use the screen object provided by Testing library. We use its getByText query, which scans the React component for the text we pass as parameter.

• To end, we use the Testing library's .toBeInTheDocument() matcher, which just checks if the previous query result is being rendered.

Then we can test that the "Switch state" toggle button works correctly, like this:

// About.test.jsx
import { render, screen } from '@testing-library/react'
import userEvent from '@testing-library/user-event'
import About from './About'

describe('About', () => {

  ...

  test('Switch state works correctly', async () => {
    render( <About/> )

    expect(screen.getByText("It's on!")).toBeInTheDocument()
    userEvent.click(screen.getByText('Switch state'))
    expect(screen.getByText("It's rolling!")).toBeInTheDocument()
    userEvent.click(screen.getByText('Switch state'))
    expect(screen.getByText("It's on!")).toBeInTheDocument()
  })

})

See that we import an additional utility called userEvent. This is an object that contains many methods we can use to simulate user fired events, like clicks, hovers, writting in an input, and so on.

• We first check that the default string is rendered: expect(screen.getByText("It's on!")).toBeInTheDocument()

• Then we simulate a click and check that the string changes in the screen:

userEvent.click(screen.getByText('Switch state'))
expect(screen.getByText("It's rolling!")).toBeInTheDocument()

• And last we simulate another click and check that the string reverses back to default:

userEvent.click(screen.getByText('Switch state'))
expect(screen.getByText("It's on!")).toBeInTheDocument()

To finish, we're going to write another test to verify that the text input and its toggle work correctly.

import { render, screen } from '@testing-library/react'
import userEvent from '@testing-library/user-event'
import About from './About'

describe('About', () => {

  ...

  test('Input works correctly', async () => {
    render( <About/> )

    userEvent.type(screen.getByTestId("testInput"), "Testing the test")
    userEvent.click(screen.getByText("Print input"))

    expect(screen.getByText("Testing the test")).toBeInTheDocument()

    userEvent.click(screen.getByText("Print input"))
    expect(screen.queryByText("Testing the test")).not.toBeInTheDocument()
  })


})

• Again we use the userEvent to simulate text being writen into our input element: userEvent.type(screen.getByTestId("testInput"), "Testing the test")

• Then we simulate a click on the toggle button, and check for the input text to be in the document:

userEvent.click(screen.getByText("Print input"))
expect(screen.getByText("Testing the test")).toBeInTheDocument()

• And we close by simulating another click and checking that the test is no longer present:

userEvent.click(screen.getByText("Print input"))
expect(screen.getByText("Testing the test")).toBeInTheDocument()

You can see how nice the utilities provided by Testing libraries are, and how easy it is to combine them with Jest. 🤓

We can run this specific test file by running npm test -- About.test.jsx and this is the result we get:

// console
PASS  src/components/About.test.jsx
  About
    ✓ About renders correctly (34 ms)
    ✓ Switch state works correctly (66 ms)
    ✓ Input works correctly (67 ms)

Test Suites: 1 passed, 1 total
Tests:       3 passed, 3 total
Snapshots:   0 total
Time:        0.997 s, estimated 1 s
Ran all test suites matching /About.test.jsx/i.

The last Jest feature I'd like to show you is test coverage. You can obtain a coverage report by running npm test -- --coverage.

This will run your tests normally and at the end of the results report you should see something like this:

// console
...

----------------|---------|----------|---------|---------|-------------------
File            | % Stmts | % Branch | % Funcs | % Lines | Uncovered Line #s 
----------------|---------|----------|---------|---------|-------------------
All files       |      75 |      100 |   85.71 |      70 |                   
 src            |       0 |      100 |       0 |       0 |                   
  App.jsx       |       0 |      100 |       0 |       0 | 7                 
  App.t.js      |       0 |        0 |       0 |       0 |                   
  index.js      |       0 |      100 |     100 |       0 | 5-6               
 src/components |     100 |      100 |     100 |     100 |                   
  About.jsx     |     100 |      100 |     100 |     100 |                   
----------------|---------|----------|---------|---------|-------------------

In the report we can see that our About.jsx component is completely covered, but our App.jsx and index.js files are not being tested.

This feature is very handy when working on big projects and you want to quickly know if most of your code is being tested correctly.

How to Test a Front-end React App with Cypress

We've talked a lot about Jest, so now let's take a look at how we can test our app using Cypress.

We'll start off by installing Cypress by running npm i -D cypress.

This should add this to our package.json:

"devDependencies": {
    "cypress": "^9.5.4"
}

Then we'll run npx cypress open. This will open the Cypress browser, and create a cypress directory within our project. Within this directory you'll find examples, documentation, and config options.

You'llo also find an "integration" folder, in which we have to put our tests. So let's create our About.test.js file in that folder and replicate the same test examples we've seen with Jest:

// About.test.js
describe('AboutPage', () => {
    it('Renders correctly', () => {
        cy.visit('http://localhost:3000/about')
        cy.contains("I'm the about page!")
    })

    it('switch btn toggles text', () => {
        cy.contains("It's on!")
        cy.get('.switchBtn').click()
        cy.contains("It's rolling!")
        cy.get('.switchBtn').click()
        cy.contains("It's on!")
    })

    it('Input works correctly', () => {
        cy.get(".testInput").type("Testing the test")
        cy.get('.printInputBtn').click()
        cy.contains("Testing the test")

        cy.get('.printInputBtn').click()
        cy.contains("Testing the test").should('not.exist')
    })
})

• The describe function works the same as in jest.

• it() is the same as the test() function we've previously seen.

• In the first test we tell the browser to visit our app's URL and check that the corresponding text is rendered:

cy.visit('http://localhost:3000/about')
cy.contains("I'm the about page!")

• Then we check that the default toggle text is rendered, simulate a click and check that it changes accordingly:

cy.contains("It's on!")
cy.get('.switchBtn').click()
cy.contains("It's rolling!")
cy.get('.switchBtn').click()
cy.contains("It's on!")

• And to end we simulate a text input, simulate a click, and check that the input text is rendered:

cy.get(".testInput").type("Testing the test")
cy.get('.printInputBtn').click()
cy.contains("Testing the test")

cy.get('.printInputBtn').click()
cy.contains("Testing the test").should('not.exist')

The syntaxt is slightly different than Jest, but the idea and structure are pretty much the same.🤙

Now if we run npx cypress open again, a window should open with this content:

We can click on "Run integration spec" and our test will run automatically in the mock browser. After the tests have run, on the left panel we'll see the results:

We can open those results to see each step the test executed. If we hover over each step, we'll see it executed in the browser in real time. A real sweet feature of Cypress.👌👌

As you can see, it's very easy to set up tests with Cypress. And if you're already familiar with Jest, you can quickly pick it up as the syntax is not that different.

If you're wondering if it makes sense to use both Jest and Cypress as test runners in the same project, I think this stack-overflow answer sums it up quite nicely.

How to Test a Back-end Node App

Now that we have a basic understanding of the ways we can tests a front-end app, let's cross the river and see how we can use similar tools to test a back end app.

For this we'll use a simple Node and Express API with just 3 endpoints.

Create a directory and run npm init -y to create a Node app. Run npm i express to install Express, and then run npm i -D jest supertest to install both Jest and Supertest as development dependencies.

Inside your package.json, add "scripts": { "test": "jest" }. Your entire package.json should look like this:

{
  "dependencies": {
    "express": "^4.17.3"
  },
  "devDependencies": {
    "jest": "^27.5.1",
    "supertest": "^6.2.2"
  },
    "scripts": {
    "test": "jest"
  }
}

Then create an app.js file and put this code in it:

// app.js
/* Import and initialize express */
const express = require('express')
const app = express()
const server = require('http').Server(app)
/* Global middlewares */
app.use(express.json())

/* Endpoint 1 */
app.get('/', async (req, res) => {

    try {
        res.status(200).json({ greeting: "Hello there!" })
    } catch (err) {
        res.status(500).send(err)
    }
})

/* Endpoint 2 */
app.get('/isPalindrome', async (req, res) => {

    try {
        const string = req.body.string
        let result = true        
        let left = 0
        let right = string.length-1

        while (left < right && result) {
            if (string[left] === string[right]) {
                left += 1
                right -= 1
            }
            else result = false
        }

        res.status(200).json({ result: result })

    } catch (err) {
        res.status(500).send(err)
    }
})

/* Endpoint 3 */
app.get('/twoSum', async (req, res) => {

    try {
        const nums = JSON.parse(req.body.nums)
        const target = JSON.parse(req.body.target)

        let result = false

        for (let i = 0; i < nums.length; i++) {
            const neededNum = target - nums[i]
            if (nums.indexOf(neededNum) !== -1 && nums.indexOf(neededNum) !== i) result = [nums[i], nums[nums.indexOf(neededNum)]]
        }

        res.status(200).json({ result: result })

    } catch (err) {
        res.status(500).send(err)
    }
})

/* Export server object */
module.exports = server

/* Initialize server */
server.listen(3001, () => console.log('Server is listening.') )
server.on('error', error => console.error(error) )

As you can see, endpoint 1 just returns a greeting message. Endpoint 2 and 3 are adaptations of the functions we've seen in our vanilla JS examples. They now receive the parameters within the request and the return values go in the response. 😉

Now the testing! Create an app.test.js file and put this code within it:

// app.test.js
const supertest = require('supertest') // Import supertest
const server = require("./app") // Import the server object
const requestWithSupertest = supertest(server) // We will use this function to mock HTTP requests

afterEach(done => { // afterEach function is provided by Jest and executes once all tests are finished
    server.close() // We close the server connection once all tests have finished
    done()
})

test('GET "/" returns greeting', async () => {
    const res = await requestWithSupertest.get('/')
    expect(res.status).toEqual(200)
    expect(res.type).toEqual(expect.stringContaining('json'))
    expect(res.body).toEqual({ greeting: "Hello there!" })
})

describe("/isPalindrome", () => {
    test('GET "/isPalindrome" neuquen returns true', async () => {
        const res = await requestWithSupertest.get('/isPalindrome').set('Content-type', 'application/json').send({ "string":"neuquen" })
        expect(res.status).toEqual(200)
        expect(res.type).toEqual(expect.stringContaining('json'))
        expect(res.body).toEqual({ result: true })
    })

    test('GET "/isPalindrome" bariloche returns true', async () => {
        const res = await requestWithSupertest.get('/isPalindrome').set('Content-type', 'application/json').send({ "string":"bariloche" })
        expect(res.status).toEqual(200)
        expect(res.type).toEqual(expect.stringContaining('json'))
        expect(res.body).toEqual({ result: false })
    })
})

describe("/twoSum", () => {
    test('GET "/twoSum" [2,7,11,15] and 9 returns [7, 2]', async () => {
        const res = await requestWithSupertest.get('/twoSum').set('Content-type', 'application/json').send({ "nums":"[2,7,11,15]", "target": "9" })
        expect(res.status).toEqual(200)
        expect(res.type).toEqual(expect.stringContaining('json'))
        expect(res.body).toEqual({ result: [7, 2] })
    })

    test('GET "/twoSum" [3,2,4] and 6 returns [4, 2]', async () => {
        const res = await requestWithSupertest.get('/twoSum').set('Content-type', 'application/json').send({ "nums":"[3,2,4]", "target": "6" })
        expect(res.status).toEqual(200)
        expect(res.type).toEqual(expect.stringContaining('json'))
        expect(res.body).toEqual({ result: [4, 2] })
    })

    test('GET "/twoSum" [3,2,4] and 10 returns false', async () => {
        const res = await requestWithSupertest.get('/twoSum').set('Content-type', 'application/json').send({ "nums":"[3,2,4]", "target": "10" })
        expect(res.status).toEqual(200)
        expect(res.type).toEqual(expect.stringContaining('json'))
        expect(res.body).toEqual({ result: false })
    })
})

Let's analyze what we're doing:

• We mock the request with requestWithSupertest.get('/')

• Then we "break" the res object in pieces and assert each part of it:

  • Check the response status: expect(res.status).toEqual(200)

  • Check the response format: expect(res.type).toEqual(expect.stringContaining('json'))

  • Check the response body content: expect(res.body).toEqual({ greeting: "Hello there!" })

The other tests are really similar, except we're sending data in the mock requests bodies, like this:

const res = await requestWithSupertest.get('/isPalindrome').set('Content-type', 'application/json').send({ "string":"bariloche" })

As you can see, testing in this way is really simple once you're familiar with Jest. We just need a little help by Supertest to mock the HTTP request and the rest is just asserting the response. 👏👏

We can run our tests with npm test and we should get the following response:

// console
 PASS  ./app.test.js
  ✓ GET "/" returns greeting (46 ms)
  /isPalindrome
    ✓ GET "/isPalindrome" neuquen returns true (18 ms)
    ✓ GET "/isPalindrome" bariloche returns true (3 ms)
  /twoSum
    ✓ GET "/twoSum" [2,7,11,15] and 9 returns [7, 2] (4 ms)
    ✓ GET "/twoSum" [3,2,4] and 6 returns [4, 2] (3 ms)
    ✓ GET "/twoSum" [3,2,4] and 10 returns false (2 ms)

Test Suites: 1 passed, 1 total
Tests:       6 passed, 6 total
Snapshots:   0 total
Time:        0.552 s, estimated 1 s
Ran all test suites.

Wrap up

And that's it! We've covered the basics of four very popular tools that will allow you to test both the front-end and back-end of your JS apps.

Of course there's much more to all the tools we've seen and many features we haven't covered. But the idea was to give you an introduction so you can take your first steps in the testing world.

As always, I hope you enjoyed the article and learned something new. If you want, you can also follow me on linkedin or twitter.

Cheers and see you in the next one! =D

Stakeholders are always pushing searching for faster solutions. Project managers are rushing products to the delivery table. And there is isn't necessarily anything wrong with doing it this way.

But it can be challenging for QA testing teams who have to embrace this super quick timetable, and then convince the developer team to get onboard.

Software teams know how badly test management teams want to reach deadlines when they are coping with constant shifts and changes.

In this scenario, all blame falls on the test team, who might quickly become a convenient target when production bugs are shipped. And any delays regarding specifications and implementation in the upstream software development lifecycle (SDLC) phase will make matters worse.

Beyond that, the demands of the stakeholders who want the product available ASAP add fuel to the fire. All of this puts the testing team under immense pressure.

A QA team must have a rational and empirical response to this challenge. The answer to this problem lies in risk-based testing (RBT) which can help the testing team deliver on their deadlines.

Your test team can save a ton of effort with the RBT strategy, and it produces great savings while delivering the project on time.

What is Risk-Based Testing? RBT Basics

In risk-based test management, you find the greatest market threats (that would have a detrimental effect on the enterprise as defined by the consumer) early in the development cycle and then you can counter them by taking preventive steps.

These market threats may include increased costs, consumer disappointment, poor user interface, and lost customers. And RBT can help you mitigate them by conducting testing in such a way that, even if a customer gets an error, they can go on using the application (and the organization isn't greatly affected).

RBT involves testing that's performed on the basis of product risks. It's useful in helping you discover how likely a specific feature or capability is to fail in final output/production. It'll also help you determine this failure's effect on the company in terms of cost and other damages, well ahead of time. And it does this by using a prioritization strategy for test cases.

So Risk-Based evaluation works by prioritizing the testing of a product or software's functionality, components, and features. This prioritization is based on the risk of the possibility of failure in the development of the feature or functionality and its effect on clients.

How to Assess Risk with RBT

RBT includes risk management and test prioritization depending on the risk factor of each test. The risk factor is the result of the likelihood and possible effect of future risk.

But how are we able to come up with those values? To allocate a risk factor to a specific risk, here are three requirements to consider.

Code Complexity

Cyclomatic Complexity

The more ambiguous code is, the more likely flaws will be discovered. Simply put, it is more likely that more complicated code contains glitches. The reverse is also true: the cleaner and simpler the code, the less likely it is to contain errors.

The problem then is how you should calculate code complexity. One measure is cyclomatic complexity which refers to the number of potential paths. The minimal number of test cases you require to thoroughly test a feature is determined by that number. In fact, code with high cyclomatic complexity, which may expand the number of errors, can be more difficult to read.

Churn or number of modifications to the code

Churn here implies the number of modifications that a specified file or module experiences. An application area that developers alter more frequently is more likely to have bugs than one which they hardly ever touch.

A great example of this is an application for e-commerce that has been in development for many years. The software team is likely to make frequent adjustments to the offers, introduce new features, and try experiments to boost purchases. The affected sections of the code shift regularly and are thus more likely to produce errors.

When churn is significant, extensive testing should be proposed for risk-based planning: more UI tests, more unit tests, more integration tests, and so on.

Criticality

Criticality involves the calculation of a software defect's influence on your application. Criticality is not spread equally through a codebase. Many systems are expected to have a high-critical code core on which the remainder of the application relies.

If the code does not perform properly, several parts of the software could be compromised and it would probably create defects. If it applies to safety-related software, a critical flaw may cause data loss, sensitive data disclosure, and maybe even endanger lives.

When measuring the effect portion of the risk factor, criticality may be useful.

If a bug arises in the application's most important environment, the effects could be much more extreme than if it existed in some area that included utility code only.

We rely on our algorithms in our test management frameworks to classify site components. If web components can not be detected by the algorithm and related core code, the UI checks if our customers will make mistakes or fail, creating frustration, reworks, service calls, and a lack of faith in our solution.

Worse still, if false positives are generated by our application, it will interrupt the CI/CD process and impede the implementation of a legitimate application.

Smart Testing Is the Solution

Software development companies need a better strategy when it comes to software testing than "just test everything." Not all code is equal, not all threats are credible, and not all bugs trigger equal harm.

In order to make informed decisions about how to distribute capital and test them more effectively, tech companies must take these RBT considerations into account.

If you want to remain on top of your Risk-Based Software Testing, you can check out Kualitee – we're a software testing and information security company.

Code reviews bring many benefits to the process of writing and delivering software:

• Ensures consistency through your codebase.
• Teaches all members of the review (helps knowledge transfer).
• Builds contextual awareness regarding what might affect other parts of the team
• Helps avoid breaking builds
• Casts fresh eyes over a code change to search for optimisations and simplifications in the change.
• Promotes quality and helps make sure no one forgot or missed anything.

Google has 1,918 repositories on their GitHub across multiple languages, and even more that aren't open source.

One of their codebases is shared by over 25,000 Googlers, and typically has 40,000 commits daily (16,000 human changes, and 24,000 by automated systems). Each day it has to serve around 800,000 queries per second during peak times.

Google has published their engineering practices, so let's see how they review code at their scale, with so many commits a day.

Google refers to changes in their codebases as CLs standing for change lists. It's just a unit of work / piece of code going through review.

How to Prepare for Code Reviews

Firstly, before you even think about the code review process, try and focus on who will do the review. Try to pick a subject matter expert. Pick someone who is familiar with this codebase or area of the codebase.

Sometimes this may even mean having different people review different parts of the code. But less than 25% of code review's at Google have more than one reviewer.

Getting someone to respond in a timely manner is also important. To avoid bottle necks and overloading individuals, just CC reviewers onto the change to review it at their convenience if they want to.

How to Run Fast Code Reviews

Code reviews should be completed quickly. The maximum length of time for a review should be one business day.

Why the urgency? I've personally had QA's sometimes take weeks or longer.

• It becomes a blocker. Although the author of the code moves onto new work, new changes start to form a back log, and the delays can build up to weeks or months.
• Developers feel frustrated. If the reviewer asks for major changes but only responds every 3 days, it is frustrating for the developer working on that change. But with quick responses, whenever you require explanation of exactly what you need to do, the frustration fades away.
• Code quality can degrade. If your reviews are always slow, developers are less likely to do code clean ups, refactoring work or general code improvement ("If my reviewer won't reply for 4 days what's even the point?"), and the code quality submitted in the reviews is more likely to go down.

The main reason code reviews can be fast is because they're small. Rather than sending a 1,000 line CL, they're separated into multiple CL's – pushing 10 smaller changes of 100 lines for example.

Google also has emergencies where code changes have to be made quickly to resolve major problems in production. In these cases, code quality is relaxed. This review should immediately become the reviewers first priority. There are some examples of emergencies Google has faced here if you're curious.

Google's Code Review Standards

The key rule Google emphasizes is:

Reviewers should generally approve a change once it definitely improves the overall code health, even if it isn't perfect.

The key point Google seem to be making is that perfect code doesn't exist. If it makes it better, that is enough.

It's definitely a balancing act between something being better, and how much better it could be. If you could add more feedback to make significant improvements, there may need to be more work done on the code.

What Googlers Look for in Code Reviews

When doing code reviews, Google focuses on these elements according to their engineering practices documentation:

Design: Is the code well-designed and appropriate for your system? Does this change belong in your codebase, or in a library? Does it integrate well with the rest of your system?

Functionality: Does the code behave as the author likely intended? Is the way the code behaves good for its users? Mostly, we expect developers to test CLs well-enough that they work correctly by the time they get to code review.

Complexity: Could the code be made simpler? Would another developer be able to easily understand and use this code when they come across it in the future? Is it over-engineered for its current use case?

Tests: Does the code have correct and well-designed automated tests? Will the tests actually fail when the code is broken? Will they start producing false positives? Does each test make simple and useful assertions?

Naming: Did the developer choose clear names for variables, classes, methods, etc.?

Comments: Are the comments clear and useful? Ensure where sensible comments explain why something is being done, rather than how.

Style & Consistency: Does the code follow our style guides?

Documentation: Did the developer also update relevant documentation?

Google has style guides for multiple languages like C++, Swift, HTML/CSS, Lisp, JavaScript and more.

Having a document that everyone can refer to helps standardise the code. It also helps clarify what the expectations are in the review process.

How Googlers Review Code

There is a three-stage process for code review in Google's engineering practices. Here's a list of things you'd need to cover if you were to review:

1. Get a high level overview of the change
2. Examine the main parts of the change
3. Look through the rest of the code, in a sensible order

Let's go over each step in more detail.

1. Get a High Level Overview of the Code Change

Look at the code change's description/summary. Does it all make sense? For example, if someone is changing a codebase that is being deleted next week, push back on the change courteously, and explain why the change doesn't look needed anymore.

It's inefficient if people are spending time on work that isn't actually needed, so have a look at your development life cycle and see why this might be happening. The earlier you can catch these issues the better.

In order to get a high level overview of the code, you may want to briefly scan the code components to see how it all works together.

If you see a serious architectural problem or something majorly wrong, you should share your observations immediately at this stage. Even if you don't have time to review every single other element of the review. Reviewing the rest may well even end up being a waste of time if the architectural problems are severe enough.

Aside from this, there are 2 major reasons why you should send your review comments immediately:

• Googlers will often email a change, and then immediately start new work based on that change. If the change in review need serious adjustments, they may be building on something that needs to be significantly changed.
• Big changes to the CL may take a while to finish. Developers all have deadlines and it's courteous to try and help them meet them by letting them start rework as soon as possible.

2. Examine the Main Parts of the Code Change

Once you have had an overview, review the "main" parts of the change.

Find the file or files that are central to the CL. Often, there is one file that has the largest number of changes, and it’s the major piece of the CL.

Spend most of your focus reviewing these pieces. This provides context to all the smaller pieces, and generally makes it faster to review.

If the CL is too large for you to get a good overview and understand the flow, it's a good sign the developer should be splitting up their CL into smaller changes.

3. Look Through the Rest of the Code, in a Sensible Order

Once you have a good overview of the change, it's time to drill down into the details and go file by file.

Each reviewer generally does this differently. Some go in the order that the version control presents to them and some pick a particular order. Choose what makes sense to you. It's important to just review everything, and miss nothing.

Review Every Line of Code

Don't miss or skim important details. Sometimes there might be config files, or generated code you can scan briefly looking for irregularities. But your job here is to be very thorough and carefully scrutinise the code.

Make sure you think about bugs, race conditions, alternate approaches, conciseness, readability, and so on.

If you can't understand the code, it's highly likely other developers won't be able to either, and should ask the developer to clarify it.

Understand the Context

Often times the version control software will only show the changed lines. But it's important to read the lines before and after, or the whole file, to ensure you understand exactly what the change is doing.

You may only see someone adding 2 more if blocks in a change. But if you look at the context you might see that the if else block now has 15 different if's inside it. Then you can reject the code change and properly improve the code quality in this area, rather than making it worse.

Remember code degradation happens slowly, with every change that goes in. Our main aim is to ensure the quality trends consistently up, and we never go backwards.

How Googlers Write Code Feedback

Google has some specific sections covering how to do code reviews courteously, and respectfully. This isn't in opposition to being clear and as helpful as you can to the developer. The golden rule here is to make comments about the code and not the developer.

What to Do:

####

Ask for the why: If it is unclear why the author is doing things a certain way, feel free to ask why they made a particular change. Not knowing is OK, and asking “Why” leaves a written record that will help answer this question in the future. (And sometimes, “I'm curious, why did you decide to do it that way?” can help the author to rethink their decision.)

Find an end: If you like things neat, it’s tempting to go over a code review over and over until it’s perfect, dragging it out for longer than necessary. It’s soul-deadening for the recipient, though. Keep in mind that “LGTM” does not mean “I vouch my immortal soul this will never fail”, but “looks good to me”. If it looks good, move on. (That doesn’t mean you shouldn’t be thorough. It’s a judgment call.) And if there are bigger refactorings to be done, move them to a new CL. (Source)

What Not to Do

Don't use extreme or very negative language: Please don't say things like “no sane person would ever do this” or “this algorithm is terrible”, whether it’s about the change you're reviewing or about the surrounding code. While it might intimidate the reviewee into doing what you want, it’s not helpful in the long run - they will feel incapable, and there is not much info in there to help them improve. “This is a good start, but it could use some work” or “This needs some cleanup” are nicer ways of saying it. Discuss the code, not the person.

Don't bikeshed: Always ask yourself if this decision really matters in the long run, or if you're enforcing a subjective preference. It feels good to be right, but only one of the two participants can win that game. If it’s not important, agree to disagree, and move on. (Source)

Remember code reviews can sometimes trend towards finding issues or discussions with code and not praise.

If your reviewer has addressed your comments, or they have done some optimisation or smart code you like, thank them. Say you like the approach / how they have solved an issue.

Conversely, thank the reviewer if they thoroughly explain what they want and promptly answer your questions.

I have had some reviews where reviewers have put huge comments explaining something I misunderstood and I can still remember who it was and what they said.

How to Handle Pushback in Code Reviews

Sometimes the developer who is receiving the review might disagree with the changes that you're proposing. Here's how to handle that.

Consider That They Might Be Right

Sometimes the developer is closer to the code and how it works, and they may be right. If they are, move on from that discussion point and leave that code as it is.

But if they're not, or they've misunderstood something, you should insist on the change. Respond to what the developer initially challenged you on, and explain your reasoning courteously.

Code quality improvements tend to happen in small incremental steps, and review is one way it happens. Insist on the code improvement, even if it means extra work for the developer.

Try to Avoid "I'll fix this later..."

One of the most common issues in reviews is not that people disagree whether the code change is necessary, but rather that they want to do it under a different ticket or do it later. So if you pass this review, they assure you that they will follow up later and fix the issues.

Generally speaking, following up later rarely happens. This doesn't mean there's something wrong with the developer or their work ethic. But it's easy to forget, have priorities change, or even just get lost in their queue of work.

Insist that the work gets done now. There is no real fantastic gain to merging changes in the codebase you have to immediately fix. You are just adding technical debt that might be followed up on later, or might be forgotten. Fixing lots of tickets later is an easy way for quality to drop.

The only exception to this is emergencies, which involve the highest priority bugs Google deals with where there is something seriously wrong. This could be services going down, or errors crashing people's pages in production.

There is naturally an eagerness to merge changes ASAP, and the review might accept slightly lower quality, with a follow up fix immediately being written. The main point is that the first change removes the emergency, and the second change fixes it properly.

Conclusion

I hope this has explained some helpful concepts Google makes use of in their code review process. I wrote this to better cement it into my own mind, and was curious about how other companies handle the QA process.

I found that there are quite a few points that I can take away and apply to the reviews I conduct.

The document I referred to throughout this article can be found here.

I share my writing on Twitter if you enjoyed this article and want to see more.

So how can we use Postman to both test our existing APIs and understand how they work?

• What is Postman?
• What are we going to build / learn?
• Part 0: Getting set up with Postman
• Part 1: An introduction to Postman
• Part 2: Creating a new Postman request to GET info about Squirtle
• Part 3: Creating a collection of requests in Postman for the PokéAPI
• Part 4: Making POST requests with Postman to translate sentences to sound like Yoda
• Part 5: Authenticating requests to the Lord of the Rings API with an API Key

What is Postman?

Postman is a tool teams can use to reliably test APIs using easy to use configurations. It comes stocked with features you would expect when dealing with APIs, including authentication, setting headers, customizing the payload, and a bunch more that help reduce the friction of using an API.

And it’s not just for testing. The beauty is that this can be used for many aspects of working with APIs for many different members of the team. Maybe a Project Manager wants to verify that things work or might find it easier to make a change straight with the API, or a QA Engineer needs to make sure everything still works, or a developer wants to actively make changes while working on the API itself.

The best part about it – Postman provides collaboration features. The free tier includes exporting and importing collections of saved API requests as well as creating shared links. If you're part of a team, they have paid tiers that allow you to sync up your collections to make sure everyone has the most recent and up to date collection.

What are we going to build / learn?

We’re going to walk through two different example APIs to cover the concepts of Postman.

First, we’ll walk through some simple HTTP requests with a public API for Pokémon.

We’ll then use the Yoda Translator API for one part to demonstrate how to make specific HTTP requests.

Once we understand how the basics work, we’ll use the Lord of the Rings API to learn how authentication works with APIs. For this, you’ll need to register for a free account for an API key.

Part 0: Getting set up with Postman

Before we get started, you’ll need Postman in order to follow along with this walkthrough. The good news, is Postman is available for free on Mac, Windows, and Linux, so you should be able to find a version that works for you.

Get Postman: https://www.postman.com/downloads/

Once downloaded, go through the standard installation instructions, open it up, and we should be ready to go!

Part 1: An introduction to Postman

The first time you open up Postman you’ll immediately be shown a launchpad with a bunch of options to get started.

It might seem a bit overwhelming, but let’s break down some of the key concepts that we’ll need to know.

Requests

A request is kind of what it sounds like, it’s a specific API request. This will be a single type of request, whether it’s a GET or POST to a specific endpoint. You’ll want to create new requests for each type of endpoint which will allow you to move between them when testing.

Collections

A collection is a group of requests. This is handy for organizing your requests into different groups. This could be as simple as two totally different APIs (ie. Twitter vs Slack) or it could be two different groups of APIs for a single API (ie. Twitter Tweets API vs Twitter Accounts API).

Authorization

Authorization is how requests are authenticated with an API, whether by a person making a request or by a computer making that request on your behalf. This commonly comes in the form of an API key which can be a static value assigned to your account or dynamically generated with tools like OAuth.

Environments

Environments will allow you to configure your endpoints to use specific variables that make it easier to use the same endpoints between different environments. For instance, you might have the same /profile endpoint on both your production and development environments, but they have different domains. Environments lets you manage a single request with a variable domain.

Workspaces

We won’t go too far into workspaces in this post, but it allows you to manage and organize different sets of collections. Imagine if you want to use Postman for both work and a personal project, you might have a Work workspace as well as a Personal workspace.

For the purposes of this article, we’ll be covering Requests, Collections, and Authorization.

Part 2: Creating a new Postman request to GET info about Squirtle

Now that we have a better understanding of the different terminology, let’s actually create a request.

At the top left of the UI you should see a but orange button that says New. Go ahead and click that and then select Request.

Before we get into the request itself, it requests a few things.

This first thing requires is a name. We’re going to start off by requesting information about the Pokémon Squirtle, so let’s name this “Pokémon - Squirtle”.

It also requires a collection, so click Create Collection and let’s name the collection “My Favorite Pokémon”.

Click the orange checkmark button next to the collection name then hit Save.

At this point we’ll have a new request, so let’s build that request.

There are two things we’ll first need to fill out for our first request:

• Request type: GET, POST, PUT, etc - we’ll use GET
• Request URL: The endpoint for your API request - for our request we’ll use https://pokeapi.co/api/v2/pokemon/squirtle/

And once you make sure those are correct, you can simply hit the blue Send button on the right and we’ve successfully made our first request!

We immediately get a few things we can see:

• Body: at the bottom we should now see the response body of the API request. For our Squirtle API, we should have a JSON object with data like abilities, base_experience, and forms.
• Status: on the right, we should see the HTTP status code. “200 Ok” is a good sign and it means it was successful!
• Time: simply how long the request took to finish
• Size: the size in KB (in our example) of the response data

You can also hover over Status, Time, and Size and get a more in depth look at each option.

So we made our first request!

Once thing to notice before we move on is that our request looks like it’s in a browser tab. If we’re done with that particular request, we can close the tab and click Save to make sure all of our changes are there for next time!

Part 3: Creating a collection of requests in Postman for the PokéAPI

Now that we’ve created a request, let’s create a collection of them. Technically we already had to create a new collection for Part 2, but we’ll create a new one to learn how collections themselves work.

At the top left of the UI, click the orange New button again and select Collection.

Similar to a request, it asks for a name so let’s call this “PokéAPI”. Optionally you can add a description, then click Create at the bottom.

On the left, you’ll now see your collection. You can select and expand the folder since we’ll be working with it.

Before we add a request, the PokéAPI has different types of requests, so it makes sense to organize it a little more thoroughly. So let’s click the three dots next to the PokéAPI collection and select Add Folder.

Similar to the others, this asks for a name. Folders are kind of like collections inside of a collection, so you get similar options. Let’s name this one “Pokémon” and click the orange Save button like before.

Now let’s add our requests! First, click the three dots next to the Pokémon folder, similar to how we added a folder to the collection, but this time select Add Request.

Let’s name this request “Pokemon”. While it might be confusing that we have a Pokemon request inside of the Pokémon folder, Pokemon is just one of the endpoints of the Pokémon group.

Now, let’s use the same exact API that we used with our Squirtle request before:

• Request Type: GET
• Request URL: https://pokeapi.co/api/v2/pokemon/squirtle/

And similar to before, when we hit the blue Send button, we should see a successful request!

Now let’s add another request. Follow the same process as before to create a new request under the PokéAPI Pokémon folder and let’s name this request “Abilities”.

If you scroll through the response from the first Squirtle endpoint, you see a lot of other API urls. At the top, we have abilities and we have two different ones — “torrent” and “rain-dish”.

Choose your favorite Squirtle ability and copy the url value into the new Abilities request we just created, I’m going to use rain-dish.

We can leave the Request Type as GET, hit the blue Send button, and we can again see a successful response!

Here we get a lot of information about our Squirtle ability Rain Dish and some of the details come in different languages which is cool!

So now we have a new PokéAPI collection with a Pokémon folder representing the group of Pokémon API endpoints including Pokemon and abilities.

We’re going to stop Part 3 with those 2 requests, but feel free to continue on and add as many of the PokéAPI requests as you’d like!

Part 4: Making POST requests with Postman to translate sentences to sound like Yoda

So far we’ve only made GET requests, but what if we wanted to make a POST request where we need to actually send some data?

For making a POST request, we’re going to use the Yoda Translator API from funtranslations.com. While this API only takes a single parameter, it’s still a good public endpoint we can use to understand the concept.

First, let’s create a new collection with a new request:

• Collection: Fun Translations
• Request: Yoda

This time, instead of a GET request, our request configuration will be:

• Request Type: POST
• Request URL: https://api.funtranslations.com/translate/yoda

Now this time, if we hit the blue Send button, we’ll notice we don’t get a successful 200 response, we get a 400!

We never actually set up any data to be posted to the API and it requires that data, so let’s add it.

Right below the Request URL, click Body. Then instead of none, select raw as the body type. Finally, on the far right of the types, change Text to JSON.

Then, in the space below it, you can add the following:

{
    "text": "Hello, I am learning how to test APIs with Postman!"
}

And now click the blue Send button again and we get a successful response!

We can apply this concept to pretty much any API. Postman doesn’t only permit you to post JSON, it allows you to use the other formats that we see listed in the Body Type section, meaning you have a lot of options depending on what the API you’re using requires.

Part 5: Authenticating requests to the Lord of the Rings API with an API Key

For the rest of the walkthrough, we’re going to use the Lord of the Rings API.

First up, the Lord of the Rings API requires authentication in order to make requests using an API key. So to start, you’ll before we dive in, you’ll need to go create a free account.

https://the-one-api.herokuapp.com/sign-up

Once you sign up and log in, the first thing you’ll see is your API key! Either copy this key down or remember where you can find it for later. If you leave the page, you can always grab it by navigating to Welcome and then Account in the navigation of the API website.

To get started, let’s first create a new collection and request:

• Collection: Lord of the Rings
• Folder: Movie
• Request: All Movies
• Request Type: GET
• Request URL: https://the-one-api.herokuapp.com/v1/movie

Once you’re set with the above, click Send, and you’ll notice immediately it gives a response that says 401 and that it’s unauthenticated.

Because this API requires the API key, this is exactly what we expected. So let’s click on the Authorization tab. We can then select a Type of Bearer Token, and on the right, we can paste in our key that we just set up with the Lord of the Rings API.

And as soon as we hit Send, we now see a successful response!

This worked really great, but what if we have a bunch of requests that use a single key. Do we have to manage that on each request?

Instead of managing it on each individual request, we can manage it on the collection. Let’s first build another request.

Under our Lord of the Rings collection and in the Movie folder, create a new request:

• Request: Quote by Movie ID
• Request Type: GET
• Request URL: https://the-one-api.herokuapp.com/v1/movie/{id}

In this request, let’s use an ID from the response of the first request, I’m going to use 5cd95395de30eff6ebccde5b which is the ID of The Two Towers, so the request URL will look like:

https://the-one-api.herokuapp.com/v1/movie/5cd95395de30eff6ebccde5b

Now, instead of setting our token in the request Authorization, we’re going to leave the type as Inherit auth from parent. Click on the three dots next to the collection and select Edit.

Here, we’re going to do the same exact thing we did with the first request but on the Collection configuration. Select the Authorization tab, under type select Bearer Token, and in the Token field again paste your token.

Finally, click Update and hit the blue Send button again and we can see a successful request!

We can now go back to our All Movies request and update the Authorization to use a Type of Inherit auth from parent and it should still continue to work!

What else can we do with Postman?

While I covered a lot of the basics, there’s quite a lot more you can do with Postman. Here are a few of my favorites.

Environment Variables

If you’re working as a developer on a project, it’s likely that your team uses multiple environments, such as a development and production environment. Instead of creating and maintaining completely separate requests, you can add an environment variable and instead change that variable when switching between environments!

Variables apply to many scenarios, but that’s a common use. Check out Postman’s docs to learn how.

https://learning.postman.com/docs/postman/variables-and-environments/variables/

Import and Export Collections and Data

A great thing about Postman is once you have your requests all organized, you can export them for others to use. This also means that you can import collections from other team members. This makes it much easier to make sure everyone’s using the same collection.

Bonus: you can even store these files in a Git repository, as they’re just JSON.

But keep in mind - if you’re using Authorization on the collection like we went over in this guide, you’ll want to make sure you don’t include that when exporting your collection.

https://learning.postman.com/docs/postman/collections/importing-and-exporting-data/

Automated testing

Once you have a set of requests in a collection and even better, if you’re storing them in Github, you can begin to use those requests as part of a way to manage automated testing of your API.

While there are a few solutions for doing this, Postman includes a Collection runner built right into the app and Newman is a command line tool that lets you run tests right from the terminal.

https://www.postman.com/use-cases/api-testing-automation/

What’s your favorite way to test and play with APIs?

Share with me on Twitter!

Test Driven Development (TDD) is sometimes described as “writing tests first”. The TDD mantra states that we should not write code before we have written automated tests that exercise that code. Writing code first is considered suboptimal.

And of course, writing code first is how we develop software following the so-called waterfall model. In that model, we divide software development activities into stages. For example, we have a ‘requirements gathering’ stage, we have an ‘application building’ stage, we have an ‘application testing’ stage, we have an ‘application deployment’ stage and so on.

But how is that different from the agile methodology? Don’t we have the exact same stages in agile?

Agile Methodology vs Waterfall Methodology

Of course we do. The crucial difference is that in agile, those stages are not gated.

In waterfall, we gate the stages and execute them in strict sequence. This means we won’t begin building the shipping application until such time as the requirements have been gathered, completed, signed off on and frozen. Once requirements are frozen (and controlled by our change management policies), we move into the next stage (or phase) – application building.

And similarly, we won’t move into the testing stage until the entire application has been built and we have reached the code complete milestone, at which point code changes have been frozen.

Once code gets frozen (and code freeze is then controlled by our change management policies), we hand it off to the testers. The testing phase begins, and only once all testing has completed (and provided that no significant defects have been detected), do we move into the deployment phase.

In agile, we do all the above activities in parallel. At the same time. We keep working on user stories (specs) while simultaneously building the shipping application. As we’re building the application we are also testing it. And as we are building and testing the application, we are also deploying it.

We learn from the shipping application deployed to production and use that validated learning as the feedback that will inform new user stories. That way, the loop gets closed, and we’re iterating, improving the value incrementally.

The only way to enable such iterative value stream delivery is by relying on automated tests. And as we’ve described, those tests are being written very early in the game. Actually, tests must be written before we write shipping code.

Why then is the title of this article “Don’t Write All Your Tests First, Just Write One”? It sounds a bit confusing. Let’s unpack the meaning of this title. But first, here's an overview of what tech we'll be using:

The technology stack used for this exercise

In the attempt to keep the exercise simple and easy to follow, I have chosen .NET Core platform, together with xUnit.net testing platform. To follow the coding examples, please install .NET Core and xUnit.net.

In order to be able to run the sample code, please open ./tests/tests.csproj file and add this line to the ItemGroup:

<ProjectReference Include="../app/app.csproj" />

You’re now all set for following the coding exercises.

A simple example

To understand the difference between writing all tests first and writing one test first, it may be better to show rather than just tell.

So let’s try to build a simple example – for this exercise I’ve chosen a trivial case of calculating a tip at a restaurant. Often times we find ourselves in a position where we want to tip the restaurant for the service, but it’s tough to calculate percentages in our head. So a nifty little Tip Calculator could come in handy.

Here are the expectations:

As a patron
I want to calculate the total bill (total plus the tip)
Because I want to compliment the restaurant for the service

Scenario #1: Patron calculates the total for terrible service

Given that the restaurant total is $100.00
And the service was terrible
When the tip calculator calculates the total charge
Then tip calculator shows $100.00 total charge

Scenario #2: Patron calculates the total for poor service

Given that the restaurant total is $100.00
And the service was poor
When the tip calculator calculates the total charge
Then tip calculator shows $105.00 total charge

Scenario #3: Patron calculates the total for good service

Given that the restaurant total is $100.00
And the service was good
When the tip calculator calculates the total charge
Then tip calculator shows $110.00 total charge

Scenario #4: Patron calculates the total for great service

Given that the restaurant total is $100.00
And the service was great
When the tip calculator calculates the total charge
Then tip calculator shows $115.00 total charge

Scenario #5: Patron calculates the total for excellent service

Given that the restaurant total is $100.00
And the service was excellent
When the tip calculator calculates the total charge
Then tip calculator shows $120.00 total charge

Let’s now implement the above user story.

We see that the story has 5 acceptance criteria (a.k.a. scenarios). Now we move into the analysis phase – think about what should be the first functionality that our Tip Calculator application should implement. But first, let’s open the command line terminal and create the new directory:

md TipCalculator
cd TipCalculator

and create app and tests directories inside the TipCalculator directory.

Now cd tests and run:

dotnet new xunit

Then cd .. and cd app, then run:

dotnet new classlib

We’re now ready to boogie!

Open your favourite text editor (mine is Visual Studio Code) and set your mind on the expectations. What behaviour are we expecting from the Tip Calculator?

To narrow the scope of our expectations, it usually helps to take one acceptance criteria (i.e. one scenario) and focus on it first. Let’s take scenario #1:

Scenario #1: Patron calculates the total for terrible service

Given that the restaurant total is $100.00
And the service was terrible
When the tip calculator calculates the total charge
Then tip calculator shows $100.00 total charge

In case the service was terrible, we are not adding any tips, and Tip Calculator is calculating a $0.00 tip. So how do we automate that scenario?

My first expectation would be that we need to somehow inform the Tip Calculator that the service was terrible. We either type the word ‘Terrible’ into the input field, or we select ‘Terrible’ from the list of available service ratings.

So the first thing to do here is to articulate some expectations regarding Tip Calculator’s ability to get notified that the service was terrible.

I like to always start with the expectation that what the user inputs is valid. So I’d first write a test that checks if the rating ‘Terrible’ is recognized by the Tip Calculator as a valid rating.

Go to the tests directory, rename the autogenerated UnitTest1.cs file to TipCalculatorTests.cs and add the following test:

[Fact]
public void CheckIfRatingTerribleIsValid(){    
  var expectedResponseForValidRating = true;    
  var actualResponseForValidRating = false;    
  Assert.Equal(expectedResponseForValidRating, 
  actualResponseForValidRating);
}

Now go to the command line, cd tests, and run:

dotnet test

Of course, the above trivial test will fail, because we have hardcoded the values. But it’s always a good practice to make sure we see our tests fail before we proceed. Not observing a test fail may give us false sense of safety later on, if no tests fail and we end up thinking that everything works as expected.

A few more observations about the above test:

• It helps if the test name is descriptive. I chose CheckIfRatingTerribleIsValid to communicate the fact that we must make sure our application is capable of recognizing our commands.
• It also helps if the expected and actual variable names are descriptive. I chose expectedResponseForValidRating and actualResponseForValidRating as fairly indicative of what our expectation in this test is, and also what actual value will the Tip Calculator produce.
• Test is a first-class source code and must be approached with equal care lavished upon the production code.

First design decision

At this point, we are forced to make a decision – how will our nascent Tip Calculator know if the service rating provided by the user is valid or not?

The design decision that comes to mind is that Tip Calculator must be able to store and retrieve some data. In this case, the data we’re interested in is the service rating.

If we go back to the user story and review the five acceptance criteria, we will see that the expectations are that Tip Calculator must be able to recognize five different service ratings:

1. Terrible
2. Poor
3. Good
4. Great
5. Excellent

So the simplest way to get Tip Calculator to store that information would be to endow it with an array, or a list.

But rather than rushing in to implement that list, we should examine the expectations again, to see if there’s anything else we may have missed. And there is – not only must Tip Calculator be able to recognize valid service ratings, it also must be able to associate each rating with a percentage value.

Our analysis shows the following associations:

1. Terrible => 0%
2. Poor => 5%
3. Good => 10%
4. Great => 15%
5. Excellent => 20%

In this case, a simple array or a simple list won’t be sufficient for holding the above associations. What’s the next simplest data structure that will allow us to implement these associations? After doing a little bit of research, we figure out that Hashtable is probably the most fitting data structure that can cover our needs with the least amount of ceremony.

We now navigate to the app directory and rename autogenerated Class1.cs file to TipCalculator.cs. We now want to add a Hashtable that will hold service ratings and the associated percentage values:

System.Collections.Hashtable ratingPercentages = new System.Collections.Hashtable();

Now is a good time to recall that TDD is focused on coupling the expectations to the application’s behaviour, not to the application’s structure. Knowing that, we need to modify our test to make Tip Calculator exhibit some behaviour. The test codifies some expectations with regards to how the application must behave, and the running application provides the evidence of the expected behaviour.

But what is the evidence of the application’s behaviour? There is no other way for us to assess and evaluate application’s behaviour other than through examining the values that the running application produces.

In this case, we are expecting the running application to produce values true or false (Boolean values) after we ask the application if certain value (i.e. service rating) is valid.

To teach the application how to behave in the expected fashion, we need to endow it with an API. In this case, we design the API as follows:

public bool CheckIfRatingIsValid(string rating)

In our test, we will modify the actual expected value to exercise the running application and collect the output value:

As you can see from the screenshot above, we have instantiated TipCalculator but when attempting to ask the instance to check if the supplied rating (“Terrible”) is valid, the editor is complaining that it cannot find that method.

Well of course, the method hasn’t been implemented yet. Now’s the time to go ahead and do it:

public bool CheckIfRatingIsValid(string rating) {    
  return false;
}

Now that the method is implemented, the test works; here is the complete listing:

using Xunit;
using app;

namespace tests {    
  public class TipCalculatorTests {        
    TipCalculator tipCalculator = new TipCalculator();        

    [Fact]        
    public void CheckIfRatingTerribleIsValid(){            
      var expectedResponseForValidRating = true;            
      var actualResponseForValidRating = 
      tipCalculator.CheckIfRatingIsValid("Terrible");                  
      Assert.Equal(expectedResponseForValidRating, 
      actualResponseForValidRating);        
    }    
  }
}

We see from the above example that we’re cheating again (we have hardcoded return false; in our newly minted method). What’s the point of beating around the bush and merely creating skeletons and scaffoldings instead of rolling up our sleeves and doing actual coding? Let’s discuss this important topic.

Discussion about our first design decision

We’re illustrating here how to do TDD step-by-step. The funny part is that this step-by-step illustration is actually the exact way how we do TDD: step-by-step. There is no other way to do TDD than by doing it step-by-step. One step at a time.

How’s that different from any other way of doing software development? Don’t we also do everything step-by-step even when not following TDD methodology? Well, not really. Let me explain:

TDD to me feels like riding a galloping horse. We’re moving swiftly toward our goal, but we’re frequently touching the ground (the galloping horse is every now and then hitting the ground in order to bounce off and run fast).

In comparison, when I’m doing software development without TDD, it feels to me like I’m flying a kite. I’m making swift moves with the kite, but I never touch the ground, not even once. By the time I land the kite, the landing place may not be where I intended the kite to go (it’s very hard to control the direction of a kite if it’s flying in a strong wind).

With TDD, any time we make a change to the code (both the test code and the shipping application code), we run the tests and so we touch the ground. We are galloping, but at the same time we need this frequent grounding. We need to see whether we’re going in the right direction and also whether we’ve broken anything during our galloping. Our tests are the Oracle who keeps telling us if everything works as expected or if something started misbehaving.

Making changes to the code is a risky business. TDD provides a nice harness that is both guiding our design decisions and ensuring we don’t mess up something that we’ve already confirmed works to our expectations.

Replace hardcoded value with actual processing logic

Let’s now replace the hardcoded value with actual running code. We first teach our Tip Calculator that there is a service rating called “Terrible” and that tip percentage associated with this rating is 0:

public bool CheckIfRatingIsValid(string rating) {      
  ratingPercentages.Add("Terrible", 0);    
  return false;
}

Our Tip Calculator is now knowledgeable about the fact that there is a service rating labeled “Terrible” and the tip percentage associated with terrible service is 0%. Great, but we’re still returning hardcoded value false. Time to replace it with actual calculation:

public bool CheckIfRatingIsValid(string rating) {    
  ratingPercentages.Add("Terrible", 0);    
  return ratingPercentages.ContainsKey(rating);
}

Run the test again:

Great, but the code still looks contrived. We are loading the “Terrible” value into the instance of Hashtable ratingPercentages and then immediately checking to see if that value exists in the Hashtable. Now that we have moved from the failing test (Red) to the passing test (Green), it’s time to perform the third step in the TDD loop – Refactor.

Refactoring is basically the activity of modifying the code structure without affecting the code behaviour. Our task here is simple: extract the code responsible for populating of the Hashtable ratingPercentages into a separate block of code.

The most natural place for this loading is in the block of code that is doing the initialization of the Tip Calculator – the constructor method. After refactoring, our shipping application source code looks like this:

using System.Collections;

namespace app {    
  public class TipCalculator {        
    private Hashtable ratingPercentages = new Hashtable();        
    public TipCalculator() {            
      ratingPercentages.Add("Terrible", 0);        
    }        

    public bool CheckIfRatingIsValid(string rating) {            
      return ratingPercentages.ContainsKey(rating);        
    }    
  }
}

Run the test again, and it passes (we’re in green). We have modified the structure of the code without modifying its behaviour! Good job.

Flip the coin

Any time we satisfy a positive expectation, it is a prudent practice to turn things on their head and describe the negative expectation.

At this point, since we’ve satisfied that a legitimate service rating value is found in the Tip Calculator, we want to ensure that non-legitimate values are not found in the Tip Calculator.

What do we mean by non-legitimate values? Any value other than “Terrible”, “Poor”, “Good”, “Great” and “Excellent”. Time to write the new expectation (i.e. test):

[Fact]
public void CheckIfRatingWhateverIsValid() {    
  var expectedResponseForValidRating = true;    
  var actualResponseForValidRating =                        
  tipCalculator.CheckIfRatingIsValid("Whatever");
  Assert.Equal(expectedResponseForValidRating, 
  actualResponseForValidRating);
}

Run the tests:

Fails. As expected. (We specified that our expectation when supplying the service rating as “Whatever” should be true. In reality, it is false, because our Tip Calculator does not contain value “Whatever”.)

Fix the test (change the expectedResponseForValidRating from true to false) and run it again:

A moment of reflection: why did we fake the first test run and made it fail? Because we always want to make sure we observe our new test failing. That way, we’ll know that in the future any successful passing of the test is not merely a false positive.

In praise of steady state

Software engineering is a balancing act between steady state and periods of unstable state. What do we mean by steady state?

If we have a system (a running application) that behaves the way we expect it to behave (i.e. it produces values we have specified as expected values), we declare that the system is in a steady state. It is running, and delivering some value.

That value delivery is still partial. In our case, the only value to the users this Tip Calculator delivers is its ability to recognize service rating “Terrible” as a legitimate rating. In addition, it is capable of informing us that service rating “Whatever” is not a legitimate rating.

That’s not much, but still is better than nothing. And good news – our running application is currently in a steady state. Now we want to look into how to add more valuable behaviour to our Tip Calculator. And the only way to add more value is by making some changes.

Any time we make a change to our application, we disturb its steady state. This disturbance is risky. It may mean our changes could break something that is already working. Because of that concern, we strive to make the duration of this unstable state as short as possible.

Remember how we compared TDD to riding a galloping horse? When the horse is in flight (i.e. not touching the ground) it is advancing toward our goal, but it’s not in the steady state. Only when the horse touches the ground does its state stabilize.

TDD encourages making small changes (in flight) and immediately grounding the system by verifying that it is back in the steady state. We value steady state despite the fact that we eagerly embrace changes. Without changes, we won’t be able to deliver value, but we must do it in a very deliberate, careful fashion.

When doing TDD, we treat changes to steady state like walking on eggshells. No matter how sure we may be in knowing what and how we’re doing software engineering, it is prudent to still let failing tests guide our decisions.

Check if correct tip percentage is associated with service rating

Let’s now introduce another change into our application – a test to verify if correct tip percentage is associated with service rating “Terrible”. Remember that we populated the instance of Hashtable ratingPercentages with the following values:

ratingPercentages.Add("Terrible", 0);

We have written a test that verifies that our Hashtable ratingPercentages does contain legitimate service rating “Terrible”. Now we need a test that verifies that service rating “Terrible” means that the tip percentage for that rating is 0.

[Fact]
public void CheckIfRatingTerribleHasZeroPercentTip() {    
  var expectedZeroPercentForTerribleRating = 0;    
  var actualZeroPercentForTerribleRating = 10;    
  Assert.Equal(expectedZeroPercentForTerribleRating, 
  actualZeroPercentForTerribleRating);
}

The new test CheckIfRatingTerribleHasZeroPercentTip should fail:

Again, we’re purposefully hard coding wrong actual values just so that we could observe our brand new test fail. Now we must replace hard coded value with the actual call to the Tip Calculator’s method that returns tip percentage for the service rating:

[Fact]
public void CheckIfRatingTerribleHasZeroPercentTip() {    
  var expectedZeroPercentForTerribleRating = 0;    
  var actualZeroPercentForTerribleRating =                 
  tipCalculator.GetPercentageTipForRating("Terrible");
  Assert.Equal(expectedZeroPercentForTerribleRating, 
  actualZeroPercentForTerribleRating);
}

As in the previous case, we have invented a new API for Tip Calculator. We call this new capability GetPercentageTipForRating("Terrible"). It takes the value of the service rating and returns the tip percentage for that rating.

Flip over to the app/TipCalculator.cs and add the hard coded skeleton of the new method:

public int GetPercentageTipForRating(string rating) {    
  return 10;
}

Running the test fails again, because we have hard coded the return value. Let’s replace it with actual processing:

public int GetPercentageTipForRating(string rating) {    
  int tipPercentage = Int32.Parse(ratingPercentages[rating].ToString());
  return tipPercentage;
}

Run the test again:

All three tests pass – we’re in green, we’re back to steady state!

What tip percentage do we expect for non-legitimate service ratings?

Many years of experience in the field taught me to be a bit pessimistic. Now that we have our application back in the steady state and delivering value (answering questions about legitimate service ratings and also giving us correct tip percentage for the “Terrible” rating), we need to see what happens when we run our application by giving it non-legitimate service rating value (for example, by giving it service rating “Whatever”).

Time for leaving the steady state yet again. We will write another test:

[Fact]
public void CheckIfRatingWhateverHasNegativeOnePercentTip() {    
  var expectedZeroPercentForWhateverRating = -1;    
  var actualZeroPercentForWhateverRating =       
  tipCalculator.GetPercentageTipForRating("Whatever");
  Assert.Equal(expectedZeroPercentForWhateverRating, 
  actualZeroPercentForWhateverRating);
}

We are describing our expectation when Tip Calculator is asked to return tip percentage for service rating “Whatever”. Because service rating “Whatever” is a non-legitimate rating, we are expecting Tip Calculator to return tip percentage of value -1.

This test now precipitates one improvement to our shipping code. We need to add some logic to first check whether the supplied service rating is legitimate or not. Only if it is legitimate do we ask Hashtable ratingPercentages to tell us what the associated value of the tip percentage is. If the supplied service rating is non-legitimate (for example, if it is “Whatever”) we bypass talking to Hashtable ratingPercentages and simply return -1.

public int GetPercentageTipForRating(string rating) {    
  int tipPercentage = -1;    
  if(CheckIfRatingIsValid(rating)) {        
    tipPercentage = Int32.Parse(ratingPercentages[rating].ToString());    
  }    
  return tipPercentage;
}

Run the tests, and all 4 tests pass:

We are back to the steady state. Another short excursion into the volatile area, and another swift victory and a safe return to steady, imperturbable state.

Populate other service rating tip percentages

Now is a good time to take a breather and make less risky changes, following the already established pattern. Leave the safety of the steady state and make short trips into the volatile territory by adding a new test to verify if service rating “Poor” is a valid, legitimate rating:

[Fact]
public void CheckIfRatingPoorIsValid() {    
  var expectedResponseForValidRating = true;    
  var actualResponseForValidRating =       
  tipCalculator.CheckIfRatingIsValid("Poor");
  Assert.Equal(expectedResponseForValidRating, 
  actualResponseForValidRating);
}

Running this test will fail:

Service rating “Poor” hasn’t been implemented yet. To make the test pass, implement service rating “Poor” by adding this line to the TipCalculator constructor:

ratingPercentages.Add("Poor", 5);

Run the tests, and we’re back to safety:

We’re enjoying steady state with 6 tests successfully passing.

Now that we have added service rating “Poor” associated with the 5% tip, let’s write a test that will describe that expectation:

[Fact]
public void CheckIfRatingPoorHasFivePercentTip() {    
  var expectedZeroPercentForPoorRating = 5;    
  var actualZeroPercentForPoorRating =       
  tipCalculator.GetPercentageTipForRating("Poor");
  Assert.Equal(expectedZeroPercentForPoorRating, 
  actualZeroPercentForPoorRating);
}

The tests run successfully, and we’re back to being safe in the steady state.
I will leave it to the reader to make the changes that will drive the implementation of the service ratings “Good”, “Great” and “Excellent”. At the end of the exercise you should have your system back in the steady state with 12 tests successfully passing:

Calculate grand total given the total and the service rating

We are now ready for the final step – given the total bill and the service rating, we expect Tip Calculator to calculate tip percentage and add it to the total, producing the grand total to be paid to the restaurant.

As we always do, first we describe the expectation:

[Fact]
public void CalculateTotalWithTip() {    
  var expectedTotalWithTip = 135.7;    
  var actualTotalWithTip = 200.0;    
  Assert.Equal(expectedTotalWithTip, actualTotalWithTip);
}

As usual, we first hard code some expectations that we know are going to fail. This is so that we observe our new test failing:

Time to implement processing logic that will calculate correct total with tip. Given the total of $118.0, and the service rating “Great” (15% tip), we’re expecting the total to be $135.7:

[Fact]
public void CalculateTotalWithTip() {    
  var rating = "Great";    
  var total = 118;    
  var expectedTotalWithTip = 135.7;    
  var actualTotalWithTip = tipCalculator.CalculateTotalWithTip(total, 
  rating);    
  Assert.Equal(expectedTotalWithTip, actualTotalWithTip);
}

We have designed a new API the Tip Calculator – a method called CalculateTotalWithTip(total, rating). It takes the total value and the service rating and returns the total with tip. The implementation of the method looks like this:

public double CalculateTotalWithTip(double total, string rating) {    
  double totalWithTip = -1;    
  if(CheckIfRatingIsValid(rating)) {        
    int percentage = GetPercentageTipForRating(rating);        
    totalWithTip = total + ((total/100) * percentage);    
  }    
  return totalWithTip;
}

Run the tests, and we’re back to steady state:

Are we done here?

No, not yet. Even when all tests are in green and we’re back to the steady state, there are still a couple of things we need to do.

To begin with, we need to add a pessimistic expectation for our Tip Calculator calculation of total with the tip based on the service rating:

[Fact]
public void CalculateTotalWithTipForNonlegitimateRating() {    
  var rating = "Meh";    
  var total = 118;    
  var expectedTotalWithTip = 135.7;    
  var actualTotalWithTip = tipCalculator.CalculateTotalWithTip(total, 
  rating);    
  Assert.Equal(expectedTotalWithTip, actualTotalWithTip);
}

Running the tests produces one failing test:

Our expectation for non-legitimate service rating (“Meh”) was incorrect. The actual total is -1, so we need to adjust our expectation by replacing 135.7 with -1. Run the tests again, and we’re back to the steady state!

We now have 14 tests, they all successfully pass, and our Tip Calculator works according to our expectations and satisfies the acceptance criteria.

We’re almost done. One more sanity check before we can confidently ship our shiny new Tip Calculator – we must run mutation testing.

Our mutation testing framework will mutate the shipping code, one line at a time, and will run all tests for each individual mutation.

If the tests complain about the mutated code, all is good, we have killed the mutant. If the tests don’t complain, we’re in trouble. We have a surviving mutant in our codebase, which means there are lines of code in our repo that are doing something for which we haven’t provided any expectations.

Let’s run mutation testing to see how solid our solution is. Good news – our solution has killed 100% of mutants!

Mutation testing has given our shipping application a clean bill of health. Our Tip Calculator seems to be in good shape.

Red-Green-Refactor-Reflect

Let’s review our Tip Calculator building exercise. We started the process by describing our expectations using the classical user story format. User story (as the name implies) is focused on describing scenarios that fulfill end user’s goals.

In this case, the simple goal is to calculate the tip amount from the supplied service rating and the restaurant bill total. The calculated tip amount is then automatically added to the total.

From there we proceeded to build our shipping application by following the TDD methodology. As we’ve demonstrated, the methodology consists of writing a failing test, observing it fail (the Red phase of TDD), then immediately making code changes that ensure the test passes (the Green phase of TDD). Once the test passes, we move into the Refactor phase (we restructure the code without affecting its behaviour). That way, we make sure our code is not expensive to change.

A proper TDD practice also mandates frequent retrospective – we call it reflection. We stop and think about the things we’ve accomplished thus far, to see if we could learn from our recent experiences. This reflection fortifies the process, as it relies on frequent and tight feedback provided by the failing, then succeeding tests.

I have already compared Test Driven Development to the experience of riding a galloping horse. While riding a horse, we’re alternating between flying through the air (i.e. speed achieved when the horse leaps from the ground) and steering the horse. It is impossible to steer the horse while we’re off the ground, up in the air. At that point, we gain speed, but we cannot make any changes of the direction. It is only once the horse touches the ground that we can make a change in direction.

In TDD, we strive to touch the ground as frequently as possible. The longer the leaps we make without touching the ground, the less chance we have for correcting the course.

I also compared software development practices that don’t follow TDD principles to the experience of flying a kite. When flying a kite, we never touch the ground. It is an exhilarating feeling of letting the wind pick the kite up and bounce it up in the air. We can achieve considerable speed that way. But we struggle in such situations to maintain desired course. And after we eventually land the kite, it usually does not land in the spot we originally wanted it to land.

Why is the emphasis of this article on “don’t write tests first”? Many software engineers who are not familiar with agile practices as implemented in TDD usually either claim that writing automated tests isn’t necessary, or claim that automated tests should be written after the code is complete.

Once they start learning about agile and TDD, they may reconsider their practices and decide that writing tests before writing implementation code may make more sense. Still, because of the ingrained waterfall mentality, some of those engineers make the mistake of writing all tests first, and only then move into writing the code.

That approach is completely wrong. It is equivalent to the traditional waterfall approach where we go through the development process by respecting gated phases.

First we write the requirements (in this case, requirements would be expectations written in the form of automated tests). Only once all the requirements (i.e. automated tests) have been written, signed off and frozen, do we move into the next gated phase – write the code for the shipping application.

TDD is the exact opposite of the “write tests first” approach. In TDD, we always write only one test. That test describes a desired behaviour. The desired behaviour does not exist yet (that’s why it is desired), and the test fails.

We then immediately move into making changes to the code in the attempt to create the desired behaviour. Once desired behaviour is created, it gets validated by the test, and if the expectations of the test are satisfied, we move into refactoring the code (to satisfy nonfunctional requirements, such as cost of change).

We practice a rigorous discipline to never succumb to the temptation to write more than one test at a time. That way, we ensure that we keep touching the ground as frequently as possible.

We prefer to remain ‘in flight’ for the shortest possible time. We are ‘in flight’ during that period when the desired behaviour described in the test has not materialized yet. The smaller the expected and desired behaviour is, the shorter will be our ‘in flight’ trajectory. That way, we keep touching the ground often, which gives us a chance to adjust the steering.

Conclusion

Building a simple Tip Calculator is a toy sized problem, and using that exercise to illustrate TDD methodology is not necessarily providing a convincing argument in favour of TDD. Still, within the constraints of a technical article, going over this hands-on exercise may provide valuable insights into the benefits of adopting TDD.

We would still argue that the real benefits of TDD only become apparent when dealing with much larger, more complex software engineering efforts. The ability to remain grounded while making potentially risky changes to a large, complex system is often a life saver.

In addition to that, building software using TDD methodology results in much less rework. TDD drives high degree of modularization, which results in high cohesiveness of the modules and low coupling between the modules.

All these characteristics produce a shipping application whose codebase is easy and inexpensive to change. And lowering the cost of change has proven to be the best way on the path to embracing changes and abandoning the concept known as ‘scope creep’.

Bottom line, TDD enables software engineering teams to deliver high degree of flexibility to the business.

Quality Assurance (commonly known as QA) is the means by which a product in development is checked to make sure it works as it’s supposed to. The actual methods used in QA processes vary hugely depending on the size and nature of the product.

For a personal project you will probably just test as you go, asking others to provide feedback at appropriate stages. By contrast, a banking application must have every aspect of every feature exhaustively checked and documented to ensure it is both functional and safe.

Regardless of how formal or detailed a QA process is, its aim is to identify bugs so they can be resolved before the product is released.

Methodologies

Agile

In an Agile approach to development, the aim is that each cycle of work (‘sprint’) produces working software that can be added to and improved upon iteratively. This makes the QA processes an intrinsic part of the development cycle.

By testing software components at each stage of their production you reduce the risk of bugs being present at release.

Terminology

Automation Testing

Testing done with pre-written scripts designed to control the execution of tests.

Black Box

These test do not look inside the system under test, but treat it as ‘closed’ in the same way that the end user will experience it.

Defect

Any deviation from an application’s specifications; often referred to as a “bug”.

Exploratory Testing

An unscripted approach to testing, which relies on the tester’s unique creativity in an effort to find unknown bugs and identify regressions.

Integration Testing

Testing individual components/modules together to ensure they connect and interact well with one another.

Negative Path Testing

A testing scenario designed to produce an error state in a feature/application and verify that the error is handled gracefully. An example of this is inputing a series of numbers in the email field in a user registration form and checking to make sure the registration is not accepted until an actual email address is provided.

Regression Testing

Testing done on a new build to ensure that new functionality has not unintentionally broken previously tested functionality.

Smoke Tests

A minimalistic approach to testing intended to ensure basic functionality is working before more in-depth testing takes place. Typically occurs at the beginning of the testing process.

Test Case

Specified preconditions, steps, and expected results referred to by a QA tester/engineer to determine whether or not a feature performs its task as expected.

White Box

Refers to tests performed at a structural level, within the codebase. Programmers checking that the inputs to and outputs from specific functions or components would be white box testing.

Also known as ‘Glass Box’, ‘Clear Box’, ‘Transparent Box’ because the tester can ‘see inside’ the system under test.

Main categories are

• Unit tests (individual units of code do what they should)
• Integration tests (units/components interact with each other properly)
• Regression tests (re-applying tests at later stages of development to ensure they still work)

There are three main techniques:

• Equivalence partitioning (the tested input values are representative of larger input datasets)
• Boundary Value Analysis (the system is tested with chosen inputs where behaviour and therefore output should change)
• Cause-Effect Graphing (tests are designed from a visualization of the input-output relations)

Other Resources

• How to write QA documentation that actually works
• Test Driven Development
• Unit tests

Isn’t Test Driven Development (TDD) twice the work? Should you do it anyway?

The short answer to the first question is NO. On the surface, it may seem like without TDD, time is only required to create the feature. With TDD, you need time to create the test AND create the feature, thus doubling the development time required.

What you’re not considering is the amount of time required for QA testing and debugging when the feature isn’t performing properly.

Case studies were conducted with three development teams at Microsoft and one at IBM that adopted TDD. The results of the case studies indicated that the pre-release defect density of the four products decreased between 40% and 90% relative to similar projects that did not use the TDD practice.

Subjectively, the teams experienced a 15–35% increase in initial development time after adopting TDD. (source)

That 40–90% decrease in pre-release defects means that QA teams and customers weren’t finding and reporting those issues. Engineering wasn’t trying to recreate bugs and develop patches, all of which have associated costs.

Number of Iterations per task

When discussing TDD, we consider a task to be a subset of a requirement that can be implemented in a few days or less. TDD software engineers develop production code through rapid iterations, as shown in the figure above.

What Is TDD?

Test-Driven Development is an approach to writing software in which the developer uses specifications to shape the way they implement a feature. For short, we describe it as the “red-green-refactor cycle”.

Before writing any code that adds new functionality to an application, the developer first writes an automated test describing how the new code should behave, and watches it turn red (fail to pass). They then write the code to the specification, and the test turns green (it passes). Finally, the developer takes a little time to make sure that the code just written is as clean as possible (refactoring).

Why you should care about TDD

Automated testing gives your software developers the confidence to make changes to the software and to know that no bugs were created as a byproduct.

Additionally, it allows more agility for developers who aren’t familiar with the details of the software to confidently modify the source code without introducing errors.

Let’s discuss some really cool advantages of TDD.

1. TDD Helps You Prevent Bugs

First, test suites ensure comprehensive test coverage of the codebase, so bugs are less likely to pop up unnoticed. Second, test suites allow developers to work out potential issues before the application is ready to go into production. Finally, because test suites are constantly maintained, they guarantee software quality.

2. Self Explanatory code (well-documented)

Because refactoring code is a built-in step in TDD, you end up with a much cleaner codebase as you go. Apps built with TDD tend to have less duplication, fewer edge cases that aren’t thought through, and a better overall architecture.

The test serves as a specification for what the code that will be written should do. As long as you’re writing good stories, your development team should be able to build exactly what you asked for. If your team agrees to use Acceptance Test-Driven Development, you can even write tests that describe how you want it to work in plain English!

3. Avoid the bugger debugger problem

Typically, when talking tech about software development, there are two main types of testing that can be integrated: functional and non-functional. These two types of testing practices are further divided into numerous types of testing techniques as you can see here:

And the list goes on

It becomes really important to strategize the testing plan before project commencement, since that helps in clearly defining the roles and responsibilities of developers with respect to testers when it comes to testing.

For example:

• Unit and Integration tests to be performed by developers before handing out builds to testers
• User acceptance testing to be performed by the testers
• Performance testing and UI testing should be done by both

A brief description of a few very important testing methodologies from the diagram above, that should be included in almost every test plan, are covered below.

Unit Testing involves testing individual units of source code to determine if they are fit for use. Intuitively, one can view a unit as the smallest testable part of an application. Faking, Mocking, and Stubbing are indispensable while writing unit tests for code which has API interactions.

Integration testing involves a combination of two or more “units” being tested. Integration tests verify that the components of the software all work together or “integrate” appropriately.

Performance testing is used to ensure that software applications will perform well under their expected workload. The features and functionality supported by a software system are not the only concerns. A software application’s performance, like its response time, reliability, resource usage, and scalability, matters. The goal of performance testing is not to find bugs, but to eliminate performance bottlenecks.

4. You can forecast troubles

A solid TDD approach alarms you about troubles upfront

The benefit of a comprehensive test suite is that it alerts you to changes early. For example, if your checkout flow stops charging users’ credit cards, you’ll know it right away, because the tests will fail. It also means that if someone makes a mistake and something doesn’t work the way it was supposed to, it will be obvious.

This is good, because it will give you a chance to fix it before it goes to production. If it becomes necessary down the road, you can even start a campaign of deep refactoring without fear, because you’ll have an ironclad test suite that will remain green.

5. Save Money

When code is complicated, it gets much harder to get anything done — one little change over here can result in a big problem over there. When following TDD, developers can make changes with confidence and your QA team will catch fewer regressions. In development speak, “time saved is equal to money earned.”

6. Invest the saved time in innovations and research

Spend the saved time in innovation

If we just adopted a TDD approach, much of this money (time saved as expressed in point 5) could be spent on new innovations instead.

7. TDD Helps You Avoid Scope Creep

The nightmare of any project manager is scope creep — any unexpected growth in the scope of work which leads to delays in project delivery.

Scope creep can happen for various reasons: poorly defined tasks, misinterpretation of project requirements, lack of documentation, and so on. There are many methods aimed at mitigating scope creep, and TDD is one of them.

Thanks for reading! Please share it if you found it useful :)