#LearnByDIY - How to create a JavaScript unit testing framework from scratch

freeCodeCamp — Mon, 09 Apr 2018 03:01:20 +0000

By Alcides Queiroz

I promise, this is gonna be fun. =)

Probably, automated tests are part of your daily routine (if not, please stop reading this article and start from the beginning, by learning from the father of TDD himself). You’ve been using testing frameworks such as Node-tap (or Tape), Jasmine, Mocha or QUnit for quite a while, just accepting that they do some magic stuff and not asking too many questions about them. Or, if you’re like me, maybe you’re always curious about how things work, including testing frameworks, of course.

This article will guide you through the process of creating a JavaScript testing framework from scratch, with a pretty decent DSL and a nicely detailed output. This is the first article in my #LearnByDIY series. The idea is to demystify certain kinds of software that we’re used to, by creating simpler versions of them.

Disclaimers

Before starting, some important notes:

The goal of this article is not to create a production-ready tool. Please, don’t use the framework that we’ll be creating to test production code. Its purpose is purely educational. =)
Naturally, our little framework won’t be full-featured. Things such as async tests, parallel executions, a richer set of matchers, a CLI (with options like --watch), pluggable reporters and DSLs, etc., won’t be present in our final version. However, I strongly recommend that you keep toying with this project and maybe try to implement some of these missing parts. Perhaps you can transform it into a serious open source project. I’d love to know that this toy project became an “actual” testing framework.

⚔️ Tyrion - A tiny testing framework

Our framework will be tiny, but “brave” for its size. So, there’s no better name than Tyrion (yeap, he’s my favorite GoT character, too).

Tyrion is small, but brave.

We’ll be using Node.js in this project, with good and old CommonJS modules. The minimum Node version you’ll need is v8.6.0. If you have an older version, please update it.

Oh, I almost forgot… I’m using Yarn throughout this article, for things like yarn init, yarn link and so on, but you can use “vanilla” NPM in a similar manner (npm init, npm link, …).

Creating the project folder structure

First, let’s create the following folder structure:

tyrion/||______ proj/|      ||      |______ src/||______ playground/       |       |______ src/       |______ tests/

In other “words”:

$ mkdir -p tyrion/proj/src tyrion/playground/src tyrion/playground/tests

We need two folders, each one to a separate project.

The proj folder will contain the Tyrion framework package.
The playground folder will contain a disposable Node project for playing with our framework. It will serve as a lab during our development process.

Initializing the Node projects

Go to the playground folder and run yarn init -y. This command generates a basic package.json file. Open it, remove the "main": "index.js", line, and add a “scripts” entry like the one in the example below:

{  "name": "playground",  "version": "1.0.0",  "scripts": {    "test": "node tests"  },  "license": "MIT"}

After creating this file, let’s do the same for the other project, the Tyrion package itself. In the proj folder, run yarn init. It will prompt you for some information to properly create the package.json file. Enter the following values (in bold):

question name (proj): tyrion question version (1.0.0): question description: question entry point (index.js): src/index.js question repository url: question author: question license (MIT): question private:

Now, we need to install Tyrion as a development dependency in our playground project. If it was a published package, we’d just need to install it directly, through npm i --dev or yarn add --dev. As we only have Tyrion locally, this is not possible. Luckily, both Yarn and NPM have a feature to help developers during this package “inception” phase, allowing us to simulate a link between two packages (one as a dependency of the other).

To create this dependency link, go to the proj folder and run:

$ yarn link

Then, in the playground folder, run:

$ yarn link tyrion

That’s all. Now Tyrion is a dependency of the playground project.

Creating some modules to be our “guinea pigs”

In the playground/src folder, let’s create two modules to be tested by Tyrion:

Writing some tests

Now is the time to use our imagination. How should Tyrion’s DSL look? Are you sick of expect, assert , and so on? Let’s make it different, just for the fun of it. I suggest guarantee as our assertion function. Do you like it?

Let’s write a few tests to see it more clearly. Of course, nothing will work, since we didn’t implement anything in our framework.

And a tests/index.js file, to import our tests in only one place.

Tyrion will borrow one of Node-tap’s principles:

Test files should be “normal” programs that can be run directly.

That means that it can’t require a special runner that puts magic functions into a global space. node test.js is a perfectly ok way to run a test, and it ought to function exactly the same as when it’s run by the fancy runner with reporting and such. JavaScript tests should be JavaScript programs; not english-language poems with weird punctuation.

https://www.node-tap.org/#tutti-i-gusti-sono-gusti.

As you might remember, in our playground’s package.json file, we have a test script which simply runs node tests. So, to execute it, just type npm test and hit enter. Yeap, do it. Let’s see it crashing:

This error is clear. We don’t have anything in our framework. No module is being exported at all. To fix it, in the proj folder, create a src/index.js file exporting an empty object, as you can see below:

module.exports = {};

Now, we’ll run npm test again:

Node is complaining because our guarantee function doesn’t exist. This is simple to fix, too:

const guarantee = () => {};

module.exports = { guarantee };

Run the test script again:

Voilà! No errors, but nothing happens, either. =(

The guarantee function

Our assertion function should execute flawlessly if the supplied value is truthy, but should throw an error if it’s falsy.

Let’s implement it:

And to test if it works, let’s append another assertion to the end of our number-utils.test.js file:

guarantee(123 === 321); // This should fail

Now run it once more:

A-ha! It works! It’s ugly, but it’s functional.

The check function

We need a way to wrap assertions into test units. Basically, all testing frameworks have this feature, like the it function in Jasmine or the test function in Node-tap.

In Tyrion, our test unit function will be called check. Its signature should be check(testDescription, callback). We also want it to give us a friendlier output, describing the passing and failing tests.

This is what it will look like:

Now, we can rewrite our tests to use the new check function:

And re-run our test suite:

Cool. But… what about some colors?? Wouldn’t it be a lot easier to distinguish between passing and failing tests?

Add the colors module as a dependency:

yarn add colors

So, import it at the top of the proj/src/index.js file:

const colors = require('colors');

And let’s put some colors in our output:

const check = (title, cb) => {  try{    cb();    console.log(`${' OK '.bgGreen.black} ${title.green}`);  } catch(e) {    console.log(`${' FAIL '.bgRed.black} ${title.red}`);    console.log(e.stack.red);  }};

That’s better. =)

The xcheck function

It would be nice to have an easy way to disable a specific test, like the xit function in Jasmine. This can be easily implemented by creating a no-op function which just outputs that a test is disabled (well, it’s not completely no-op, but almost):

const xcheck = (title, cb) => {  console.log(`${' DISABLED '.bgWhite.black} ${title.gray}`);};

module.exports = { guarantee, check, xcheck };

So, import the xcheck function in the number-utils.test.js file and disable one of our tests:

const { guarantee, check, xcheck } = require('tyrion');const numberUtils = require('../src/number-utils');

// method: isPrimexcheck('returns true for prime numbers', () => {  guarantee(numberUtils.isPrime(2));  guarantee(numberUtils.isPrime(3));  guarantee(numberUtils.isPrime(5));  guarantee(numberUtils.isPrime(7));  guarantee(numberUtils.isPrime(23));});

And here’s how it behaves:

Test summary and exit code

If we wanted to use Tyrion in a CI server, it would need to finish its process with different exit codes for error and success conditions.

Another desirable feature is a test summary. It would be nice to know how many tests passed, failed, or skipped (the disabled ones). For this, we could increment some counters in both check and xcheck functions.

We will create the end function, which prints the test summary and finishes with the appropriate exit code:

And don’t forget to call it in the playground/tests/index.js file:

const { end } = require('tyrion');

require('./string-utils.test');require('./number-utils.test');

end();

Or maybe:

const tyrion = require('tyrion');

require('./string-utils.test');require('./number-utils.test');

tyrion.end();

Now, let’s re-run npm test:

Great, it works.

The group function

Many test frameworks have some way of grouping related tests. In Jasmine, for example, there is the describe function. We will implement a group function for this purpose:

And update our tests to use this new function:

Here’s the new output:

Well, the good news is that it works. The bad news is that it’s getting hard to understand. We need a way to indent this output in order to make it more readable:

Run it again:

That’s way better!

So, how does it work?

The repeat function repeats a string n times.
The indent function repeats an indent (of four spaces) n times by using the repeat function.
The indentLines function indents a string with multiple lines by adding n indents to the beginning of each line. We’re using it to indent error stacks.
The indentLevel variable is incremented at the beginning of each group execution and decremented at its end. This way, nested groups can be correctly indented.

More matchers

The guarantee function is not flexible enough for a lot of scenarios. We need a richer set of matchers in order to make our tests more meaningful.

First, create the matchers folder:

$ mkdir proj/src/matchers

Now, we’ll create each matcher in a separate file:

The same matcher uses the strict equality operator (===) to test if two arguments are exactly the same object (for reference types) or equal (for primitive types). It behaves similarly to the toBe matcher in Jasmine and t.equal in node-tap.

Note: Node-tap also has a matcher called t.same, but it works differently (it won’t verify if two objects are exactly the same, but if both are deeply equivalent).

The identical matcher verifies that two arguments are equivalent. It uses the == operator for comparing values.

The deeplyIdentical matcher does a deep comparison of two objects. This kind of comparison can be considerably complex, or at least too complex for this article’s purpose. So, let’s install an existing module to handle deep equality and use it in our matcher:

$ yarn add deep-equal

Then:

This is how an error will look:

The falsy matcher will fail if the supplied value is truthy.

The truthy matcher works in a similar manner to our guarantee function. It passes when the supplied value is truthy and breaks if it’s falsy.

The throws matcher will pass if a function throws an error. It’s possible to specify the wanted error message, but this is not mandatory.

An index.js file to re-export all matchers:

And finally let’s glue them all together:

You can use our new matchers this way:

const { guarantee, check } = require('tyrion');

check('playing with our new matchers', () => {  // The original guarantee function still works  guarantee(123 === 123);

  guarantee.truthy('abc');  guarantee.falsy(null);

  const a = { whatever: 777 };  const b = a;  guarantee.same(a, b);  guarantee.identical(undefined, null);

  const c = { whatever: { foo: { bar: 'baz' } } };  const d = Object.assign({}, c);  guarantee.deeplyIdentical(c, d);

  function boom() { throw new Error('Some error...'); }  guarantee.throws(boom);  guarantee.throws(boom, 'Some error...');});

The beforeEach function

To implement a beforeEach function, we need to use a stack to accumulate all beforEach callbacks. This is done for each new scoped level created every time a group is declared:

How does it work?

Every time a group is declared, we’re pushing a new array to the beforeEachStack variable. This array will accumulate all beforeEach callbacks declared in that scope.
After a group execution is completed, we remove the array at the top of our callbacks stack.
The beforeEach function receives a callback and appends it to the array at the top of our callbacks stack.
At the beginning of each check function, we’re calling every beforeEach callback in all levels of our stack.

The beforeAll function

Our last addition will be the beforeAll function. For simplicity’s sake, we’re assuming that calls to the beforeAll function will always be put before all groups and tests (or, when scoped within a group, at its very top).

Otherwise, if we wanted to ensure that the beforeAll function works correctly even in the middle or at the end of a group, we should dramatically change our existing logic. Well, we’re not going to do that, since it isn’t a rational usage of this function.

Our version of beforeAll will just receive a callback and immediately execute it.

const beforeAll = cb => cb();

module.exports = {   group, check, xcheck, guarantee, beforeAll, end };

An example of usage:

const { guarantee, check, group, beforeAll } = require('tyrion');

let a;beforeAll(() => {  a = { something: 'example' };});

group('playing with the beforeAll function', () => {  let b;  beforeAll(() => {    b = { something: 'example' };  });

  check('some test', () => {    guarantee.deeplyIdentical(a, b);  });

  check('another test', () => {    guarantee.identical(11, 11);  });});

The final version of Tyrion

It has been a long journey, but Tyrion is finally complete. =)

I added a SILENT option which disables logging. It’s being used to make it easier to test Tyrion (yep, testing frameworks need to be tested too).

The complete project is available here.

Possible improvements

Tyrion lacks many features, like:

Support for async tests
Parallel execution of tests
afterEach and afterAll functions
A xgroup function, which disables an entire group
A function similar to Jasmine’s fit
Spies
Decoupling DSL from reporting logic.
Pluggable reporters
A terminal CLI (with a --watch option)
Yet more matchers
Friendlier error stacks

I encourage you to keep playing with this project. Feel free to use and expand it. Please let me know your thoughts, suggestions, and experiments by leaving a comment below. =)

DIY - freeCodeCamp.org