But they don't check whether the numbers on those elements are correct. A data-pipeline bug that shows Malaysia's population as 3.4 million instead of the real 34 million slips past every selector test in the suite.

The element still exists. A number still renders. The page still looks right. But the bug ships and sits there until a human notices.

I work as a full-stack engineer. Writing end-to-end (E2E) tests with Playwright and unit tests with Jest is part of my day job. I also use Playwright MCP, the bridge between AI assistants like Claude and a running browser, when I need to generate first-draft test code or debug a flow.

None of that tooling closes the maintenance tax on selector-based suites. Every E2E suite I keep alive at work accumulates data-testid selectors, waitForSelector calls, and tests that break because someone renamed a button.

Bug0's Breaking Apps Hackathon gave me a pretext to try something different. Over a weekend, I built an automated regression suite for Malaysia's three public open data portals, data.gov.my, OpenDOSM, and KKMNow, using Passmark, Bug0's open-source AI-driven Playwright library.

The tests are written in plain English. Two AI models verify each assertion. A third arbitrates disagreements.

What You'll Find Below:

• How to write an E2E test that checks whether a dashboard's numbers are correct, not only whether the page renders

• A specific assertion pattern (range-bounded KPIs) that catches an entire class of data-pipeline bug that selector tests miss, with working examples ready to copy

• A cross-field math assertion that takes one sentence in Passmark and around a hundred lines of code without it

• How Passmark's own failure explanations became my debugging loop (the single biggest shift in how I'll write E2E tests going forward)

• The real limits: a 14% cache-hit rate, a dependency on OpenRouter, and what two-model voting fails to catch

Table of Contents

• Why Malaysia's Open Data Portals?

• What Is Passmark?

• The Hero Spec: Range-Bounded Assertions

  • What Two-Model Voting Doesn't Catch

• Going Further: Cross-Field Math

• What I Found Across Three Runs

  • The Debugging Loop

  • The Two Specs That Still Fail Are the Most Interesting

• What It Cost, and Why Cache Rate Is Cost Rate

• The Pattern Worth Stealing

• Honest Verdict

• Resources

Why Malaysia's Open Data Portals?

The hackathon suggested targets like Vercel Commerce, Cal.com, and Hashnode. These all would've been solid picks.

But I wanted to test something local and closer to my day-to-day work instead. I also wanted a data-heavy site where the numbers on screen have to be accurate, as I work with numbers too on a daily basis.

Malaysia has three public open-data portals:

• data.gov.my, run by MAMPU, the government's digital transformation agency

• OpenDOSM, run by the Department of Statistics

• KKMNow, run by the Ministry of Health

They're public, no authentication required, with documented APIs. Seemed like a good fit for an automated test suite. The data on them is what Malaysians use every day, so accuracy isn't optional.

What Is Passmark?

Passmark is a Playwright library where the tests read like specs. Here's an example:

await runSteps({
  page,
  userFlow: "population dashboard smoke",
  steps: [
    { description: "Navigate to https://data.gov.my/dashboard/kawasanku" },
    {
      description: "Wait for the country-level Malaysia view to render",
      waitUntil: "A headline population number is visible",
    },
  ],
  assertions: [
    {
      assertion:
        "The page shows Malaysia's total population as a number greater than 20 million and less than 40 million",
    },
  ],
  test,
  expect,
});

There are no selectors, no data-testid, and no page.locator(). The assertion expresses what I care about, in the words I would use with a colleague.

On the first run, an AI agent drives the page and caches the resolved Playwright action to Redis. Every run after that replays at native Playwright speed with zero model calls.

When the UI changes and a cached action fails, the AI re-engages only for that step. Two assertion models (Claude and Gemini) vote. A third model arbitrates disagreements.

The Hero Spec: Range-Bounded Assertions

Range-bounded assertions were the first shape of test I wrote, and the one I came back to most across the suite.

The idea is straightforward: check that a number on the page falls inside a sensible range, not that a specific element exists.

The image below is the Playwright report from the population spec, with all four range-bounded assertions passing.

The range-bounded population test is the one that shows Passmark's real value.

Traditional Playwright asserts DOM structure. It confirms that an element with class kpi-total contains the text 34.2 million. That tells you the page rendered, not whether the number makes sense.

A bug that shows Malaysia's population as 3.42 million sails past any selector test. The DOM is correct. The number renders. Nothing breaks in the conventional sense.

Passmark reads the page, evaluates the claim, and fails because 3.42 million falls outside the sane range. Two models vote. A hallucination by one model alone produces no false pass.

What Two-Model Voting Doesn't Catch

Voting defends against one model misreading the page. It doesn't defend against both models misreading the page the same way. If Claude and Gemini both parse "32.4 million" as "3.24 million" because of the same unusual spacing in the DOM, they agree, they vote pass, and the bug ships.

The mitigation is assertion design. Write assertions that are hard to misread. A range check ("between 20 million and 40 million") is harder for a model to get wrong than a prose check ("roughly 34 million"). Numerical bounds leave less room for interpretation than adjectives. The more your assertion looks like a unit test written in English, the less room the models have to disagree.

Going Further: Cross-Field Math

Range-bounded assertions are a good first step. They catch "is this number in the right ballpark?" But they don't catch "do these numbers agree with each other?"

For that, you need cross-field math. If a dashboard shows a total population and a breakdown by gender, those two things are supposed to agree. Male plus female should equal total. Ethnicity breakdown percentages should sum to 100.

test("Cross-field math: sex breakdown sums to total population", async ({ page }) => {
  test.setTimeout(180_000);
  await runSteps({
    page,
    userFlow: "population sex breakdown consistency",
    steps: [
      { description: "Navigate to https://data.gov.my/dashboard/kawasanku" },
      {
        description: "Wait for the Malaysia country-level view with breakdown data",
        waitUntil:
          "A headline total population figure is visible and a breakdown by sex is shown on the page",
      },
    ],
    assertions: [
      {
        assertion:
          "The male and female population values shown on the page add up to approximately the headline total population, within a 5% margin",
      },
      {
        assertion:
          "Any percentage-based breakdowns visible on the page (by sex, age, or ethnicity) sum to approximately 100% within a 2 percentage-point margin",
      },
      {
        assertion: "No breakdown value is negative or greater than the headline total",
      },
    ],
    test,
    expect,
  });
});

Try writing that in vanilla Playwright. You need selectors for the headline number, selectors for the breakdown components, number parsing with a comma-aware regex, and a margin calculation. Seventy to a hundred lines of code to verify three invariants a primary school student would call obvious.

The Passmark version is one spec. I ran it against Kawasanku's live country view. All three assertions passed in 1.4 minutes. Passmark's annotation, verbatim:

"The headline total population figure 'Malaysia has a population of 32,447,385 people.' is visible, and 'Gender And Age Distribution' is shown, which implies a breakdown by sex (male, female) will be available."

Two models read the page, extract the numbers, do the arithmetic, and agree. When the dashboard changes layout in three months, the same assertion still works, because it never named a selector.

This is the class of test I want running against every dashboard product that I touch. Financial totals matching their line items. Percentages that sum to 100. Inventory counts equal to the sum of warehouse locations. This rarely gets checked today, because writing the check by hand outweighs the perceived value of running it.

What I Found Across Three Runs

Every passing spec after run 1 came from Passmark telling me, in plain English, why my assertion didn't match the page.

Here are three examples from run 1:

For dataset-detail.spec.ts, I asserted "an API usage snippet (curl or JS) is shown." knowingly that the page is using Python and I wanted to see what the result was. Passmark replied:

"The page contains API usage snippets, but they are specifically for Python using the requests library. There are no snippets provided in curl or JavaScript formats."

The page had snippets. I asked for the wrong languages. Fix: accept any language.

For dashboard-population.spec.ts, I asserted "a chart visualizing population by age or ethnicity is rendered." Passmark replied:

"The current page displays charts for vital statistics such as Live Births, Deaths, and Natural Increase over time, but there is no chart visualizing population specifically by age groups or ethnicity."

The charts are there. Not the slice I guessed. Fix: accept any chart about population.

For kkmnow/hospital-utilisation.spec.ts, I asserted a "headline bed-utilisation percentage." Passmark replied:

"While there are multiple bed-utilisation percentages listed in tables and rankings further down the page, there is no prominent, top-level headline KPI figure displaying the overall bed-utilisation percentage."

The numbers are there. I had asked for a layout the designers didn't build.

This is the killer feature: Passmark's failure messages aren't stack traces. They're explanations. The AI read the page, compared it against my words, and pointed me at the fix. Nothing like a selector-based test throwing TimeoutError: waiting for locator.

The Debugging Loop

Once I saw the pattern, the loop became my main technique. Here's the procedure:

1. Read the failure message word for word. Don't skim.

2. Trust it as a description of what is on the page. The AI has read the page. Your assertion has not.

3. Rewrite the assertion so it matches what's on the page. Broaden, narrow, or restate.

4. Run it again.

The discipline is to not argue with the tool. The page is what the page is. Your assertion is what is wrong. Every time I tried to "fix" the page (convinced my assertion was right and the site was broken), I lost some time. Every time I took the failure message at face value and rewrote, the test passed on the next run.

This is the one of the changes in how I'll write E2E tests going forward. The feedback loop is the tool. Every failed assertion is a draft of the correct one.

The Two Specs That Still Fail Are the Most Interesting

1. The two models disagreed and the arbiter call failed.

On catalogue-search.spec.ts, Claude voted fail (72% confidence) and Gemini voted pass (100% confidence) on the same assertion. I had written the assertion in a way that read two ways.

Passmark escalated to an arbiter model through OpenRouter. The call came back with a 504 from Cloudflare. The arbiter never ran. The suite failed the spec.

This is an honest limit, not a fluke. Any CI that runs Passmark depends on OpenRouter's availability. External gateway errors happen. My fix for the final run was a global retry wrapper around the OpenRouter call, and the 504 stopped being a problem in practice.

If you bring this to production CI, plan for retries and treat OpenRouter outages as a first-class failure mode in your runbook.

This failure taught me something about assertion design: my wording was ambiguous. Claude's reading was reasonable. Gemini's reading was reasonable. When you write tests in English, being precise about what you mean is part of writing a good test.

2. The wait condition fired too early.

On the KKMNow spec, I had waitUntil: "A utilisation metric is visible". The page showed the section label "Hospital Bed Utilisation (%)" before the numbers finished loading. The wait step saw the label, decided the condition was met, and moved on. By the time the numbers rendered, the test had run out of time. Once the page was fully loaded, the range assertions would have passed on content.

"The page displays multiple bed-utilisation percentages within the specified range (0% to 120%). For example, the ranked list shows Perlis at 93.1% and Melaka at 88.2%."

The lesson: your waitUntil wording needs the same care as your assertion wording. Both are read by AI. A vague wait is as bad as a vague assertion.

What It Cost, and Why Cache Rate Is Cost Rate

Each of the three runs took about 20 minutes on 13 specs with a single worker. The hackathon's pooled OpenRouter key covered the AI costs, so I have no personal dollar figure to report.

The more useful cost finding is what gets cached.

$ docker exec passmark-redis redis-cli DBSIZE
5

Five steps out of roughly 35 were cached across three runs. A 14% cache-hit rate. The Passmark README explains why:

Only steps that produced a single tool call get cached. Multi-step sequences are considered non-deterministic.

Most of my steps described multi-tool sequences. "Open the area selector and choose Selangor, then wait for navigation" becomes click, wait, verify. Those don't cache by design.

This matters for your budget. An 86% miss rate means 86% of your steps call a model on every run. The cost model is per-tool-call via OpenRouter.

To estimate your own bill: count non-atomic steps in your suite, multiply by your chosen model's per-call price at current OpenRouter rates, and the product is your recurring cost per run. Cache rate is cost rate.

The fix is authoring discipline. Split compound descriptions into atomic steps. Treat cache fill rate as a metric you track, not an implementation detail to ignore. A suite with 80% atomic steps costs a fifth of a suite with 14%.

The Pattern Worth Stealing

The idea here is bigger than Passmark.

Check that the numbers on your dashboards make sense. Most teams don't. They should.

A one-line assertion like "the headline number is between 20 million and 40 million" catches several classes of bug regular tests miss.

Here are four common ones:

• The data pipeline divided by the wrong thing, so the number on screen is ten times too small.

• A timezone bug made yesterday's total show up under tomorrow's date.

• The data never refreshed, so users are looking at last week's numbers.

• A locale flip swapped commas and decimals, so 1,234,567 is now reading as 1.234567.

Civic portals were my target. The pattern applies anywhere a dashboard shows numbers. Fintech reports, SaaS analytics, healthcare metrics, e-commerce admin panels. Any screen where a number is supposed to mean something.

Most of these numbers never get tested. Writing the check by hand is tedious. You need a selector to find the number, code to parse it, code to handle units, and a margin calculation. Fifty lines for one check. Nobody bothers.

You don't need Passmark to steal the idea. The same check works in plain Playwright with page.evaluate and number parsing. The Passmark version is just more efficient to write and readable by anyone on the team, not only engineers.

Honest Verdict

Passmark works. Across three runs I went from 4 of 13 passing to 12 of 13 without touching a selector, guided by the tool's own feedback.

Still, the caveats are real:

• On a cold cache, every step waits for a model. Budget more wall-clock time than a selector suite.

• In my suite only 14% of steps cached. The other 86% pays model cost on every run. Authoring discipline (atomic steps) is the difference between cents and dollars per run.

• Two-model voting doesn't protect against both models misreading the same way. Write assertions that are hard to misread.

• Every assertion depends on OpenRouter's availability. External gateway errors need a retry strategy before this runs in CI.

What stuck with me: Passmark didn't make me better at Playwright. It made me write tests I would have skipped otherwise.

What I imagine myself doing at work:

• Run a small nightly Passmark suite against the critical dashboards, focused on range and freshness checks.

• Keep traditional Playwright and Jest for everything that has to be fast and deterministic.

• Treat every Passmark failure message as a specification of the page, not an error to argue with.

Try this, even if you never touch Passmark. Pick a number on a dashboard you work with. Write a test that fails if the number is outside a sane range. See what breaks. That is the whole pattern and purpose of this article.

Resources

• Repo: github.com/LeeRenJie/passmark-hackathon

• Passmark: github.com/bug0inc/passmark

• Breaking Apps Hackathon: hashnode.com/hackathons/breaking-things

• Test targets: data.gov.my, OpenDOSM, KKMNow

Using a Node.js shopping cart example, I'll show why server-side validation and test-driven development beat "one-shot" AI outputs every time. Let's dive into how to make AI your most reliable collaborator.

Some Background

Last week I did something that felt amazing for about… five seconds. I opened an AI tool, typed one sentence, and it generated a whole shopping cart module for an e-commerce app. Lots of files, lots of code, even folders and patterns. It looked professional.

And then I realized something: the problem was not "how fast AI wrote code." The problem was "how do I know this code is correct?"

Here's the truth: a big pile of code that you didn't write is not a shortcut. For most developers, it's actually extra work. You have to read it, understand it, and still catch the hidden mistakes.

So today I'm not going to give you another "AI is coming" talk. Instead, I'll show you a simple loop that any developer can follow – beginner, mid-level, or senior – to get better results from AI, step by step, without getting trapped. And I'll show it with a real example you can run in one file.

Here’s What We’ll Cover:

• The 5-second high (and the real problem)

• The golden rule: never trust user prices

• The mindset shift: stop asking for the whole app

• The AI coding loop (the 7-step workflow)

• Apply the loop: a server-side cart total calculator

  • The prompt (small piece, strong constraints)

• One-file runnable example (with a wrong version on purpose)

  • What you should notice here

• How to use failing tests as a flashlight

• Copy-paste prompt template

• A calm hype check: why fundamentals matter more now

  • A simple exercise (do this once and you'll feel the skill)

• Recap

The 5-Second High (and the Real Problem)

A lot of people misunderstand AI coding. They think the main job is typing code. But the main job is thinking clearly. Typing is cheap now. Thinking is expensive.

When AI produces a "perfect-looking" module in one shot, the real work doesn't disappear. It moves downstream:

• You still need to understand what it generated

• You still need to verify it matches your rules

• You still need to catch the mistakes that hide inside "nice looking code"

If you can't verify it, you don't own it. And if you don't own it, you can't safely ship it.

Tip: Treat AI output like code from a stranger on the internet: useful, but untrusted until proven.

The Golden Rule: Never Trust User Prices

I started exactly like a beginner would start. I opened AI and wrote a vague prompt:

Design and develop an e-commerce shopping cart module for me.

AI replied with a big output. It looked clean. If you're new, you might think:

Wow, it solved it.

But then I asked myself:

What is the easiest way this can go wrong in real life?

And the answer is also simple: “money can be stolen”. Because a shopping cart has one golden rule: never trust prices coming from the user.

If the browser sends you: “T-shirt price is \(1" and you accept it, someone can pay \)1 for a $20 product. And when AI generates a big module quickly, that kind of mistake can easily hide inside "nice looking code."

Warning: Any system that accepts client-sent prices is basically inviting price tampering.

The Mindset Shift: Stop Asking for the Whole App

So instead of accepting the big AI output, I changed my approach. I said:

I'm not going to ask AI to build the whole app. I will break the big thing into small parts, and I will guide AI like a real engineer.

That is the first mindset shift. In the AI era, your value is not how fast you type. Your value is how well you can do three things:

• define the problem clearly

• break it into small pieces

• prove the result is correct

Big systems are built from small correct pieces. That's not "prompt engineering." That's engineering.

The AI Coding Loop (the 7-Step Workflow)

Here's the loop I use. It's simple English. You can copy it and use it for any project:

• Write the goal in one sentence

• Write the rules (what must be true)

• Write two examples (input → output)

• Write two bad situations (weird cases)

• Ask AI for a small piece, not the whole thing

• Ask for tests, then run them

• If something fails, improve the prompt and repeat

That's it. That's the loop. Here it is in visual form:

Tip: The loop is the skill. Tools will change. The loop will still work.

Apply the Loop: a Server-side Cart Total Calculator

Now let's apply it to the shopping cart example. Instead of "build me a cart module," I wrote a tiny requirement note:

We need a cart total calculator on the server. User sends productId and quantity. We must ignore any price from the user. We must use our own product list. We must handle unknown products and invalid quantity. We must calculate subtotal, discount, tax, and final total. We must round money correctly. We must have tests.

This is not a large or complex requirements specification - just a clear and concise note.

And then I asked AI for only one small piece:

• Not the UI

• Not the database

• Not the entire architecture

• Just one function, with tests

Because the fastest way to build something real is to prove one brick at a time. We have written down everything we discussed in the requirement note. It would be great to also create a visual representation of those ideas. Along with the requirement note, we can prepare a simple sketch or diagram for our own reference. This way, it can serve as a clean and well-documented requirement specification, which we can keep recorded in our project's GitHub README.md file.

In the diagram below, we can have a browser on the left and the server on the right. The browser/user is an untrusted input source. The user may send productId, qty, and even a fake price, but the server must treat only productId and qty as input and must ignore any client-sent price. The server then looks up the real price from its own trusted product catalog, validates the quantity, and calculates totals from server-side data. This is the trust boundary: prices come from the server, not from the client.

The prompt (small piece, strong constraints)

This is the shape of the prompt I used:

Create a single JavaScript file I can run with Node.

Goal:

Calculate shopping cart totals.

Rules:

• Input items have productId and qty.

• Do NOT trust price from user input.

• Use my product catalog.

• qty must be at least 1.

• discountPercent and taxPercent must not be negative.

• discount first, then tax.

• round money to 2 decimals.

Examples:

• 2 T-shirts (20 each) + 1 mug (12.50) => subtotal 52.50

• discount 10%, tax 8% => discount first, then tax

Deliver:

• one function

• simple tests using Node's built-in assert

• print one example output

One small change makes a massive difference: “rules + examples + tests”. AI still tries to help fast, but now it has guardrails. And if it still makes a mistake, you can catch it, because you asked for proof.

Here is a visual representation of the "Cart Totals Pipeline" that covers all the use cases involved in the cart totals calculation process.

In the diagram, the cart total calculation follows a fixed pipeline. First, validate inputs (known productId, valid qty, non-negative discount/tax). Next, compute subtotal from the trusted product catalog. Then apply the discount to get the discounted amount. After that, calculate tax on the discounted amount (not on the original subtotal). Finally, round values correctly and return the result (subtotal, discount, tax, and total). The key rule is the order: discount first, then tax.

One-File Runnable Example (with a Wrong Version on Purpose)

Now here's the one-file example you can run right now. No setup. Just Node. Create a file named cart.js, paste in the below code, and run node cart.js.

It includes two versions:

• a wrong version that trusts user price (this is the mistake we want to learn from)

• a correct version that uses a trusted catalog

// cart.js

// Run: node cart.js

const assert = require("node:assert/strict");

// Trusted product catalog (server-side truth)

const PRODUCTS = {
    tshirt: { name: "T-shirt", priceCents: 2000 }, // $20.00

    mug: { name: "Mug", priceCents: 1250 }, // $12.50

    book: { name: "Book", priceCents: 1599 }, // $15.99
};

function money(cents) {
    return (cents / 100).toFixed(2);
}

// WRONG: trusts user price

function cartTotal_WRONG(cartItems, discountPercent = 0, taxPercent = 0) {
    let subtotalCents = 0;

    for (const item of cartItems) {
        const priceCents = Math.round((item.price ?? 0) * 100); // user can cheat

        subtotalCents += priceCents * item.qty;
    }

    const discountCents = Math.round(subtotalCents * (discountPercent / 100));

    const afterDiscount = subtotalCents - discountCents;

    const taxCents = Math.round(afterDiscount * (taxPercent / 100));

    const totalCents = afterDiscount + taxCents;

    return totalCents;
}

// Correct: uses trusted catalog + checks

function cartTotal(cartItems, discountPercent = 0, taxPercent = 0) {
    if (!Array.isArray(cartItems))
        throw new Error("cartItems must be an array");

    if (typeof discountPercent !== "number" || discountPercent < 0)
        throw new Error("discountPercent must be non-negative");

    if (typeof taxPercent !== "number" || taxPercent < 0)
        throw new Error("taxPercent must be non-negative");

    let subtotalCents = 0;

    for (const item of cartItems) {
        const { productId, qty } = item || {};

        if (typeof productId !== "string" || !PRODUCTS[productId]) {
            throw new Error("Unknown productId: " + productId);
        }

        if (typeof qty !== "number" || qty < 1) {
            throw new Error("qty must be at least 1");
        }

        subtotalCents += PRODUCTS[productId].priceCents * qty;
    }

    const discountCents = Math.round(subtotalCents * (discountPercent / 100));

    let afterDiscountCents = subtotalCents - discountCents;

    if (afterDiscountCents < 0) afterDiscountCents = 0;

    const taxCents = Math.round(afterDiscountCents * (taxPercent / 100));

    const totalCents = afterDiscountCents + taxCents;

    return { subtotalCents, discountCents, taxCents, totalCents };
}

function runTests() {
    // Normal example

    const cart = [
        { productId: "tshirt", qty: 2 },

        { productId: "mug", qty: 1 },
    ];

    const r = cartTotal(cart, 10, 8);

    assert.equal(r.subtotalCents, 5250); // 52.50

    assert.equal(r.discountCents, 525); // 10% of 52.50

    assert.equal(r.taxCents, 378); // 8% of 47.25

    assert.equal(r.totalCents, 5103); // 51.03

    // Attack example: user tries to cheat with price = 1

    const attackerCart = [
        { productId: "tshirt", qty: 2, price: 1 },

        { productId: "mug", qty: 1, price: 1 },
    ];

    const wrong = cartTotal_WRONG(attackerCart, 0, 0);

    assert.equal(money(wrong), "3.00"); // totally wrong in real life

    const safe = cartTotal(attackerCart, 0, 0);

    assert.equal(money(safe.totalCents), "52.50"); // correct, ignores user price

    // Edge cases

    assert.throws(() => cartTotal([{ productId: "unknown", qty: 1 }], 0, 0));

    assert.throws(() => cartTotal([{ productId: "tshirt", qty: 0 }], 0, 0));

    assert.throws(() => cartTotal(cart, -1, 0));

    assert.throws(() => cartTotal(cart, 0, -1));
}

runTests();

console.log("All tests passed.");

const example = cartTotal(
    [
        { productId: "tshirt", qty: 1 },

        { productId: "book", qty: 2 },
    ],

    15,

    5,
);

console.log("Example subtotal:", money(example.subtotalCents));

console.log("Example discount:", money(example.discountCents));

console.log("Example tax:", money(example.taxCents));

console.log("Example total:", money(example.totalCents));

In this code, we didn't do a magic trick. We did some engineering:

• We took a big problem and broke it into a small piece

• We wrote rules so the AI doesn't guess

• We wrote examples so the AI understands

• We asked for tests so we can prove it

• We ran the tests so we can trust it

That is the loop you can reuse for any project.

How to Use Failing Tests as a Flashlight

This is the part many developers skip. They ask for code, but they don't ask for proof. When you run the tests, one of two things happens:

• Tests pass: great, you earned confidence

• Tests fail: even better, you earned clarity

A failing test is a flashlight. It shows you the exact place where your thinking (or your prompt) needs improvement. Instead of "AI is wrong," you get a real question:

Which rule was unclear, missing, or contradictory?

Then you adjust:

• add a stricter rule

• add an example that removes ambiguity

• add an edge case that forces the correct behavior

• regenerate only the small piece, not the whole codebase

Copy-Paste Prompt Template

Here is a copy-paste prompt template you can reuse from today (see below the image):

Build ONE small piece, not the full app.

Goal:

(One sentence)

Rules:

(3 to 7 bullets)

Examples:

(2 examples: input -> output)

Edge cases:

(2 cases that can break it)

Deliver:

- one runnable file

- include tests using Node assert

- print one example output

Then ask:

Before giving code, list the possible mistakes and confirm the rules.

That last line is powerful. It forces the AI to think about failure before writing code.

A Calm Hype Check: Why Fundamentals Matter More Now

A lot of content online makes it sound like: "AI codes now, so you don't need to learn coding." That idea is a trap. Because yes, AI can type code. But AI cannot replace your responsibilities as a developer and engineer.

If you ship a broken cart, you can lose money. If you ship insecure code, you can get hacked. If you ship unreliable software, users leave. And in real life, nobody will accept the excuse: "The AI wrote it."

In the AI era, learning coding isn't less important. It's more important, just in a different way. The goal isn't to become a fast typist. The goal is to become a strong thinker.

Fundamentals matter more than before:

• how data flows through a system

• how to break big problems into small parts

• how to write clear rules and requirements

• how to test and verify

• how to notice edge cases

• how to think about security

• how to understand the tools you use, not just copy answers

Average software will be everywhere. It will be cheap. It will be copied. It will be easy to make. So the only software that matters will be software that is truly valuable: safe, reliable, high quality, and built with real understanding.

That's good news for serious learners. Because the best engineers will become even more valuable, not less.

A Simple Exercise (do this once and you'll feel the skill)

Add one more rule to the cart, like:

• qty cannot be more than 10

• Write the test first. Then ask AI to update the function. Run the tests.

• That's how you train the real AI skill: not prompting, but guiding and verifying.

• Let AI type the code.

• You do the thinking.

• You do the breaking down.

• You do the proof.

Recap

• Don't ask AI to build the whole app

• Break the problem into one small piece

• Write rules, examples, and edge cases so AI doesn't guess

• Always ask for tests and run them

• Treat failing tests as a flashlight

• Repeat the loop until you can trust what you ship

That's the game now. And if you play it well, you're not behind, you're ahead.

Final Words

If you found the information here valuable, feel free to share it with others who might benefit from it.

I’d really appreciate your thoughts – mention me on X @sumit_analyzen or on Facebook @sumit.analyzen, watch my coding tutorials, or simply connect with me on LinkedIn.

You can also checkout my official website sumitsaha.me for details about me.

The success of using TDD, however, depends highly on how productive the developer can be while implementing code and application testing.

This is where automation tools come in.

Below is a list of 10 recommended testing automation frameworks/platforms and a short summary of their features.

Selenium

Selenium is, arguably, the most popular automated testing framework. It is a browser-based framework that works on different operating systems and browsers.

It features the Selenium IDE, which is Chrome and Firefox extension that allows the recording and playback of user interactions. For scaling tests (i.e, running on several machines) then Selenium Grid is the tool of choice.

The most popular tool in the framework is Selenium Webdriver, which is a collection of language specific bindings and allows for the creation of browser-based regression tests.

Perfecto.io

Perfecto is both a web and mobile app testing framework. It’s delivered as a SaaS tool and allows testing engineers to run their tests from anywhere.

The web testing tool provides for parallel testing of apps on different operating systems and browsers. It allows for a huge volume of daily tests to be run, and the tests are 50% faster than any other framework.

The mobile test tool allows engineers to test on both emulators and actual devices. Just like the web test tool, it allows for a high volume of test runs daily (10,000 executions).

Serenity

Serenity BDD’s slogan is “Automated Acceptance Testing with Style”. This is because Serenity’s unique angle is helping testers write world-class test reports and documentation.

Testing starts with the creation of user stories and acceptance criteria. Serenity then automates the acceptance criteria. Tests are automatically broken down into steps that make them more readable. Developers can easily run these tests against actual application executions.

Once tests are finished, Serenity creates detailed reports that include screenshots of the tests and all relevant information such as error messages and execution times.

Cypress

Cypress is a great tool to use for frontend or end-to-end automated testing.

Installing Cypress in a project is as easy as running a simple npm install cypress or yarn add cypress command. JavaScript developers who are used to unit testing tools such as Jest or Mocha will find writing Cypress test scripts a breeze.

Its dashboard makes testing more powerful and faster by allowing tests to be grouped by browser type, environment, package type, and so on. The parallelization feature allows developers to run more tests and test more features easily.

Lambda Test

LambdaTest is a leading test automation software for both desktop and web apps.

It allows for live and interactive testing of both publicly hosted and local machine hosted web applications and websites.

LambdaTest also allows developers to run Selenium test scripts with its Browser Testing Grid. It is a very powerful and versatile suite that also integrates with CI/CD tools such as Jenkins, Circle CI, and Travis CI.

Now the coolest part is that it allows for geo-testing, meaning web applications can be tested for how they perform depending on which location in the world they are being accessed from.

TestProject.io

TestProject’s community is probably one of the biggest among those in the testing community. It is an end-to-end and API cloud hosted test framework.

As an open-source project, users get easy access to updates and can even participate in its improvements. It is ready for use with Selenium with all of its pre-packaged dependencies.

With its myriad of integration tools, it allows for testing on different browsers, and even in Docker.

One of the best things about TestProject is it allows non-coders to perform testing through its Scriptless Test recorder.

Katalon Studio

Another open-source API, web, and mobile testing suite is Katalon Studio. It is one of the most complete testing suites in the entire list, with features like recording, auto-generating test scripts, and powerful integrations.

It is a testing framework that is easy to start (yes, even for non-coders) but is powerful enough for scaling.

Katalon Studio works well with existing CI/CD setups. For instance, it's easy to integrate it with Github or Gitlab’s continuous testing tools. This makes it great for Agile teams.

OpenTest

OpenTest is another open source tool that automates testing for API’s, web, mobile and desktop applications. It is mostly used for functional testing.

It still has lots of limitations as there are some test scenarios that are not yet supported. However, it is very good for beginners and non-coders alike since it uses plain English keywords for writing test actions.

Some of its most notable features include web testing with Selenium, mobile testing with Appium, keyword-driven testing, parallel testing, and data-driven testing.

AccelQ

AccelQ is a codeless test automation platform that is artificial-intelligence based. As such, it is best used for Agile development as it allows for rapid building and easy introduction of changes in the project.

The platform allows for API and end-to-end testing. The AI backbone provides predictive analytics that help in accelerating the test script generation. It has an intuitive and easy to use UI that allows even beginners to get started quickly.

Conclusion

The choice boils down to, not surprisingly, your particular use-case.

However, given the extensive capabilities of each of these tools, it is understandably difficult to select one. The beautiful thing about a lot of them being open source is that development teams can use one for a particular project, and another for a different one.

I'm always willing to learn, no matter how much I know. As a software engineer, my thirst for knowledge has increased a lot. I know that I have a lot of things to learn daily.

But before I could learn more, I wanted to master the fundamentals. To make myself a better developer, I wanted to understand more about how to create great product experiences.

This post is my attempt to illustrate a Proof of Concept (PoC) I built to try out some ideas.

I had some topics in mind for this project. It needed to:

• Use high-quality software
• Provide a great user experience

When I say high-quality software, this can mean so many different things. But I wanted to focus on three parts:

• Clean Code: Strive to write human-readable code that is easy to read and simple to maintain. Separate responsibility for functions and components.
• Good test coverage: It's actually not about coverage. It's about tests that cover important parts of components' behavior without knowing too much about implementation details.
• Consistent state management: I wanted to build with software that enables the app to have consistent data. Predictability is important.

User experience was the main focus of this PoC. The software and techniques would be the foundation that enabled a good experience for users.

To make the state consistent, I wanted a type system. So I chose TypeScript. This was my first time using Typescript with React. This project also allowed me to build custom hooks and test it properly.

Setting up the project

I came across this library called tsdx that sets up all the Typescript configuration for you. It's mainly used to build packages. Since this was a simple side project, I didn't mind giving it a try.

After installing it, I chose the React template and I was ready to code. But before the fun part, I wanted to set up the test configuration too. I used the React Testing Library as the main library together with jest-dom to provide some awesome custom methods (I really like the toBeInTheDocument matcher).

With all that installed, I overwrote the jest config by adding a new jest.config.js:

module.exports = {
  verbose: true,
  setupFilesAfterEnv: ["./setupTests.ts"],
};

And a setupTests.ts to import everything I needed.

import "@testing-library/jest-dom";

In this case, I just had the jest-dom library to import. That way, I didn't need to import this package in my test files. Now it worked out of the box.

To test this installation and configuration, I built a simple component:

export const Thing = () => <h1>I'm TK</h1>;

In my test, I wanted to render it and see if it was in the DOM.

import React from 'react';
import { render } from '@testing-library/react';
import { Thing } from '../index';

describe('Thing', () => {
  it('renders the correct text in the document', () => {
    const { getByText } = render(<Thing />);

    expect(getByText("I'm TK")).toBeInTheDocument();
  });
});

Now we are ready for the next step.

Configuring routes

Here I wanted to have only two routes for now. The home page and the search page - even though I'll do nothing about the home page.

For this project, I'm using the react-router-dom library to handle all things router-related. It's simple, easy, and fun to work with.

After installing it, I added the router components in the app.typescript.

import { BrowserRouter as Router, Switch, Route } from 'react-router-dom';

export const App = () => (
  <Router>
    <Switch>
      <Route path="/search">
        <h1>It's the search!</h1>
      </Route>
      <Route path="/">
        <h1>It's Home</h1>
      </Route>
    </Switch>
  </Router>
);

Now if we enter the localhost:1234, we see the title It's Home. Go to the localhost:1234/search, and we'll see the text It's the search!.

Before we continue to start implementing our search page, I wanted to build a simple menu to switch between home and search pages without manipulating the URL. For this project, I'm using Material UI to build the UI foundation.

For now, we are just installing the @material-ui/core.

To build the menu, we have the button to open the menu options. In this case they're the "home" and "search" options.

But to build a better component abstraction, I prefer to hide the content (link and label) for the menu items and make the Menu component receive this data as a prop. This way, the menu doesn't know about the items, it will just iterate through the items list and render them.

It looks like this:

import React, { Fragment, useState, MouseEvent } from 'react';
import { Link } from 'react-router-dom';
import Button from '@material-ui/core/Button';
import MuiMenu from '@material-ui/core/Menu';
import MuiMenuItem from '@material-ui/core/MenuItem';

import { MenuItem } from '../../types/MenuItem';

type MenuPropsType = { menuItems: MenuItem[] };

export const Menu = ({ menuItems }: MenuPropsType) => {
  const [anchorEl, setAnchorEl] = useState<null | HTMLElement>(null);

  const handleClick = (event: MouseEvent<HTMLButtonElement>): void => {
    setAnchorEl(event.currentTarget);
  };

  const handleClose = (): void => {
    setAnchorEl(null);
  };

  return (
    <Fragment>
      <Button aria-controls="menu" aria-haspopup="true" onClick={handleClick}>
        Open Menu
      </Button>
      <MuiMenu
        id="simple-menu"
        anchorEl={anchorEl}
        keepMounted
        open={Boolean(anchorEl)}
        onClose={handleClose}
      >
        {menuItems.map((item: MenuItem) => (
          <Link to={item.linkTo} onClick={handleClose} key={item.key}>
            <MuiMenuItem>{item.label}</MuiMenuItem>
          </Link>
        ))}
      </MuiMenu>
    </Fragment>
  );
};

export default Menu;

Don't panic! I know it is a huge block of code, but it is pretty simple. the Fragment wrap the Button and MuiMenu (Mui stands for Material UI. I needed to rename the component because the component I'm building is also called menu).

It receives the menuItems as a prop and maps through it to build the menu item wrapped by the Link component. Link is a component from react-router to link to a given URL.

The menu behavior is also simple: we bind the handleClick function to the button's onClick. That way, we can change anchorEl when the button is triggered (or clicked if you prefer). The anchorEl is just a component state that represents the Mui menu element to open the menu switch. So it will open the menu items to let the user chooses one of those.

Now, how do we use this component?

import { Menu } from './components/Menu';
import { MenuItem } from './types/MenuItem';

const menuItems: MenuItem[] = [
  {
    linkTo: '/',
    label: 'Home',
    key: 'link-to-home',
  },
  {
    linkTo: '/search',
    label: 'Search',
    key: 'link-to-search',
  },
];

<Menu menuItems={menuItems} />

The menuItems is a list of objects. The object has the correct contract expected by the Menu component. The type MenuItem ensures that the contract is correct. It is just a Typescript type:

export type MenuItem = {
  linkTo: string;
  label: string;
  key: string;
};

Search

Now we are ready to build the search page with all the products and a great experience. But before building the list of products, I wanted to create a fetch function to handle the request for products. As I don't have an API of products yet, I can just mock the fetch request.

At first, I just built the fetching with useEffect in the Search component. The idea would look like this:

import React, { useState, useEffect } from 'react';
import { getProducts } from 'api';

export const Search = () => {
  const [products, setProducts] = useState([]);
  const [isLoading, setIsLoading] = useState(false);
  const [hasError, setHasError] = useState(false);

  useEffect(() => {
    const fetchProducts = async () => {
      try {
        setIsLoading(true);

        const fetchedProducts = await getProducts();

        setIsLoading(false);
        setProducts(fetchedProducts);
      } catch (error) {
        setIsLoading(false);
        setHasError(true);
      }
    };

    fetchProducts();
  }, []);
};

I have:

• products initialized as an empty array
• isLoading initialized as false
• hasError initialized as false
• The fetchProducts is an async function that calls getProducts from the api module. As we don't have a proper API for products yet, this getProducts would return a mock data.
• When the fetchProducts is executed, we set the isLoading to true, fetch the products, and then set the isLoading to false, because the fetching finished, and the set the fetched products into products to be used in the component.
• If it gets any error in the fetching, we catch them, set the isLoading to false, and the hasError to true. In this context, the component will know that we had an error while fetching and can handle this case.
• Everything is encapsulated into a useEffect because we are doing a side effect here.

To handle all the state logic (when to update each part for the specific context), we can extract it to a simple reducer.

import { State, FetchActionType, FetchAction } from './types';

export const fetchReducer = (state: State, action: FetchAction): State => {
  switch (action.type) {
    case FetchActionType.FETCH_INIT:
      return {
        ...state,
        isLoading: true,
        hasError: false,
      };
    case FetchActionType.FETCH_SUCCESS:
      return {
        ...state,
        hasError: false,
        isLoading: false,
        data: action.payload,
      };
    case FetchActionType.FETCH_ERROR:
      return {
        ...state,
        hasError: true,
        isLoading: false,
      };
    default:
      return state;
  }
};

The idea here is to separate each action type and handle each state update. So the fetchReducer will receive the state and the action and it will return a new state. This part is interesting because it gets the current state and then returns a new state, but we keep the state contract by using the State type.

And for each action type, we will update the state the right way.

• FETCH_INIT: isLoading is true and hasError is false.
• FETCH_SUCCESS: hasError is false, isLoading is false, and the data (products) is updated.
• FETCH_ERROR: hasError is true and isLoading is false.

In case it doesn't match any action type, just return the current state.

The FetchActionType is a simple Typescript enum:

export enum FetchActionType {
  FETCH_INIT = 'FETCH_INIT',
  FETCH_SUCCESS = 'FETCH_SUCCESS',
  FETCH_ERROR = 'FETCH_ERROR',
}

And the State is just a simple type:

export type ProductType = {
  name: string;
  price: number;
  imageUrl: string;
  description: string;
  isShippingFree: boolean;
  discount: number;
};

export type Data = ProductType[];

export type State = {
  isLoading: boolean;
  hasError: boolean;
  data: Data;
};

With this new reducer, now we can useReducer in our fetch. We pass the new reducer and the initial state to it:

const initialState: State = {
  isLoading: false,
  hasError: false,
  data: fakeData,
};

const [state, dispatch] = useReducer(fetchReducer, initialState);

useEffect(() => {
  const fetchAPI = async () => {
    dispatch({ type: FetchActionType.FETCH_INIT });

    try {
      const payload = await fetchProducts();

      dispatch({
        type: FetchActionType.FETCH_SUCCESS,
        payload,
      });
    } catch (error) {
      dispatch({ type: FetchActionType.FETCH_ERROR });
    }
  };

  fetchAPI();
}, []);

The initialState has the same contract type. And we pass it to the useReducer together with the fetchReducer we just built. The useReducer provides the state and a function called dispatch to call actions to update our state.

• State fetching: dispatch FETCH_INIT
• Finished fetch: dispatch FETCH_SUCCESS with the products payload
• Get an error while fetching: dispatch FETCH_ERROR

This abstraction got very big and can be very verbose in our component. We could extract it as a separate hook called useProductFetchAPI.

export const useProductFetchAPI = (): State => {
  const initialState: State = {
    isLoading: false,
    hasError: false,
    data: fakeData,
  };

  const [state, dispatch] = useReducer(fetchReducer, initialState);

  useEffect(() => {
    const fetchAPI = async () => {
      dispatch({ type: FetchActionType.FETCH_INIT });

      try {
        const payload = await fetchProducts();

        dispatch({
          type: FetchActionType.FETCH_SUCCESS,
          payload,
        });
      } catch (error) {
        dispatch({ type: FetchActionType.FETCH_ERROR });
      }
    };

    fetchAPI();
  }, []);

  return state;
};

It is just a function that wraps our fetch operation. Now, in the Search component, we can import and call it.

export const Search = () => {
  const { isLoading, hasError, data }: State = useProductFetchAPI();
};

We have all the API: isLoading, hasError, and data to use in our component. With this API, we can render a loading spinner or a skeleton based on the isLoading data. We can render an error message based on the hasError value. Or just render the list of products using the data.

Before starting implementing our products list, I want to stop and add tests for our custom hook. We have two parts to test here: the reducer and the custom hook.

The reducer is easier as it is just a pure function. It receives value, process, and returns a new value. No side-effect. Everything deterministic.

To cover all the possibilities of this reducer, I created three contexts: FETCH_INIT, FETCH_SUCCESS, and FETCH_ERROR actions.

Before implementing anything, I set up the initial data to work with.

const initialData: Data = [];
const initialState: State = {
  isLoading: false,
  hasError: false,
  data: initialData,
};

Now I can pass this initial state for the reducer together with the specific action I want to cover. For this first test, I wanted to cover the FETCH_INIT action:

describe('when dispatch FETCH_INIT action', () => {
  it('returns the isLoading as true without any error', () => {
    const action: FetchAction = {
      type: FetchActionType.FETCH_INIT,
    };

    expect(fetchReducer(initialState, action)).toEqual({
      isLoading: true,
      hasError: false,
      data: initialData,
    });
  });
});

It's pretty simple. It receives the initial state and the action, and we expect the proper return value: the new state with the isLoading as true.

The FETCH_ERROR is pretty similar:

describe('when dispatch FETCH_ERROR action', () => {
  it('returns the isLoading as true without any error', () => {
    const action: FetchAction = {
      type: FetchActionType.FETCH_ERROR,
    };

    expect(fetchReducer(initialState, action)).toEqual({
      isLoading: false,
      hasError: true,
      data: [],
    });
  });
});

But we pass a different action and expect the hasError to be true.

The FETCH_SUCCESS is a bit complex as we just need to build a new state and add it to the payload attribute in the action.

describe('when dispatch FETCH_SUCCESS action', () => {
  it('returns the the API data', () => {
    const product: ProductType = {
      name: 'iPhone',
      price: 3500,
      imageUrl: 'image-url.png',
      description: 'Apple mobile phone',
      isShippingFree: true,
      discount: 0,
    };

    const action: FetchAction = {
      type: FetchActionType.FETCH_SUCCESS,
      payload: [product],
    };

    expect(fetchReducer(initialState, action)).toEqual({
      isLoading: false,
      hasError: false,
      data: [product],
    });
  });
});

But nothing too complex here. The new data is there. A list of products. In this case, just one, the iPhone product.

The second test will cover the custom hook we built. In these tests, I wrote three contexts: a time-out request, a failed network request, and a success request.

Here, as I'm using axios to fetch data (when I have an API to fetch the data, I will use it properly), I'm using axios-mock-adapter to mock each context for our tests.

The set up first: Initializing our data and set up an axios mock.

const mock: MockAdapter = new MockAdapter(axios);
const url: string = '/search';
const initialData: Data = [];

We start implementing a test for the timeout request:

it('handles error on timed-out api request', async () => {
  mock.onGet(url).timeout();

  const { result, waitForNextUpdate } = renderHook(() =>
    useProductFetchAPI(url, initialData)
  );

  await waitForNextUpdate();

  const { isLoading, hasError, data }: State = result.current;

  expect(isLoading).toEqual(false);
  expect(hasError).toEqual(true);
  expect(data).toEqual(initialData);
});

We set up the mock to return a timeout. The test calls the useProductFetchAPI, wait for an update, and then we can get the state. The isLoading is false, the data is still the same (an empty list), and the hasError is now true as expected.

The network request is pretty much the same behavior. The only difference is that the mock will have a network error instead of a timeout.

it('handles error on failed network api request', async () => {
  mock.onGet(url).networkError();

  const { result, waitForNextUpdate } = renderHook(() =>
    useFetchAPI(url, initialData)
  );

  await waitForNextUpdate();

  const { isLoading, hasError, data }: State = result.current;

  expect(isLoading).toEqual(false);
  expect(hasError).toEqual(true);
  expect(data).toEqual(initialData);
});

And for the success case, we need to create a product object to use it as a request-response data. We also expect the data to be a list of this product object. The hasError and the isLoading are false in this case.

it('gets and updates data from the api request', async () => {
  const product: ProductType = {
    name: 'iPhone',
    price: 3500,
    imageUrl: 'image-url.png',
    description: 'Apple mobile phone',
    isShippingFree: true,
    discount: 0,
  };

  const mockedResponseData: Data = [product];

  mock.onGet(url).reply(200, mockedResponseData);

  const { result, waitForNextUpdate } = renderHook(() =>
    useFetchAPI(url, initialData)
  );

  await waitForNextUpdate();

  const { isLoading, hasError, data }: State = result.current;

  expect(isLoading).toEqual(false);
  expect(hasError).toEqual(false);
  expect(data).toEqual([product]);
});

Great. We covered everything we needed for this custom hook and the reducer we created. Now we can focus on building the products list.

Products list

The idea of the products list is to list products that have some information: title, description, price, discount, and if it has free shipping. The final product card would look like this:

To build this card, I created the foundation for the product component:

const Product = () => (
  <Box>
    <Image />
    <TitleDescription/>
    <Price />
    <Tag />
  </Box>
);

To build the product, we will need to build each component that is inside it.

But before start building the product component, I want to show the JSON data that the fake API will return for us.

{
  imageUrl: 'a-url-for-tokyo-tower.png',
  name: 'Tokyo Tower',
  description: 'Some description here',
  price: 45,
  discount: 20,
  isShippingFree: true,
}

This data is passed from the Search component to the ProductList component:

export const Search = () => {
  const { isLoading, hasError, data }: State = useProductFetchAPI();

  if (hasError) {
    return <h2>Error</h2>;
  }

  return <ProductList products={data} isLoading={isLoading} />;
};

As I'm using Typescript, I can enforce the static types for the component props. In this case, I have the prop products and the isLoading.

I built a ProductListPropsType type to handle the product list props.

type ProductListPropsType = {
  products: ProductType[];
  isLoading: boolean;
};

And the ProductType is a simple type representing the product:

export type ProductType = {
  name: string;
  price: number;
  imageUrl: string;
  description: string;
  isShippingFree: boolean;
  discount: number;
};

To build the ProductList, I'll use the Grid component from Material UI. First, we have a grid container and then, for each product, we will render a grid item.

export const ProductList = ({ products, isLoading }: ProductListPropsType) => (
  <Grid container spacing={3}>
    {products.map(product => (
      <Grid
        item
        xs={6}
        md={3}
        key={`grid-${product.name}-${product.description}-${product.price}`}
      >
        <Product
          key={`product-${product.name}-${product.description}-${product.price}`}
          imageUrl={product.imageUrl}
          name={product.name}
          description={product.description}
          price={product.price}
          discount={product.discount}
          isShippingFree={product.isShippingFree}
          isLoading={isLoading}
        />
      </Grid>
    ))}
  </Grid>
);

The Grid item will display 2 items per row for mobile as we use the value 6 for each column. And for the desktop version, it will render 4 items per row.

We iterate through the products list and render the Product component passing all the data it will need.

Now we can focus on building the Product component.

Let's start with the easiest one: the Tag. We will pass three data to this component. label, isVisible, and isLoading. When it is not visible, we just return null to don't render it. If it is loading, we will render a Skeleton component from Material UI. But after loading it, we render the tag info with the Free Shipping label.

export const Tag = ({ label, isVisible, isLoading }: TagProps) => {
  if (!isVisible) return null;
  if (isLoading) {
    return (
      <Skeleton width="110px" height="40px" data-testid="tag-skeleton-loader" />
    );
  }

  return (
    <Box mt={1} data-testid="tag-label-wrapper">
      <span style={tabStyle}>{label}</span>
    </Box>
  );
};

The TagProps is a simple type:

type TagProps = {
  label: string;
  isVisible: boolean;
  isLoading: boolean;
};

I'm also using an object to style the span:

const tabStyle = {
  padding: '4px 8px',
  backgroundColor: '#f2f3fe',
  color: '#87a7ff',
  borderRadius: '4px',
};

I also wanted to build tests for this component trying to think of its behavior:

• when it's not visible: the tag will not be in the document.

describe('when is not visible', () => {
  it('does not render anything', () => {
    const { queryByTestId } = render(
      <Tag label="a label" isVisible={false} isLoading={false} />
    );

    expect(queryByTestId('tag-label-wrapper')).not.toBeInTheDocument();
  });
});

• when it's loading: the skeleton will be in the document.

describe('when is loading', () => {
  it('renders the tag label', () => {
    const { queryByTestId } = render(
      <Tag label="a label" isVisible isLoading />
    );

    expect(queryByTestId('tag-skeleton-loader')).toBeInTheDocument();
  });
});

• when it's ready to render: the tag will be in the document.

describe('when is visible and not loading', () => {
  it('renders the tag label', () => {
    render(<Tag label="a label" isVisible isLoading={false} />);

    expect(screen.getByText('a label')).toBeInTheDocument();
  });
});

• bonus point: accessibility. I also built an automated test to cover accessibility violations using jest-axe.

it('has no accessibility violations', async () => {
  const { container } = render(
    <Tag label="a label" isVisible isLoading={false} />
  );

  const results = await axe(container);

  expect(results).toHaveNoViolations();
});

We are ready to implement another component: the TitleDescription. It will work almost similar to the Tag component. It receives some props: name, description, and isLoading.

As we have the Product type with the type definition for the name and the description, I wanted to reuse it. I tried different things - and you can take a look here for more details - and I found the Pick type. With that, I could get the name and the description from the ProductType:

type TitleDescriptionType = Pick<ProductType, 'name' | 'description'>;

With this new type, I could create the TitleDescriptionPropsType for the component:

type TitleDescriptionPropsType = TitleDescriptionType & {
  isLoading: boolean;
};

Now working inside the component, If the isLoading is true, the component renders the proper skeleton component before it renders the actual title and description texts.

if (isLoading) {
  return (
    <Fragment>
      <Skeleton
        width="60%"
        height="24px"
        data-testid="name-skeleton-loader"
      />
      <Skeleton
        style={descriptionSkeletonStyle}
        height="20px"
        data-testid="description-skeleton-loader"
      />
    </Fragment>
  );
}

If the component is not loading anymore, we render the title and description texts. Here we use the Typography component.

return (
  <Fragment>
    <Typography data-testid="product-name">{name}</Typography>
    <Typography
      data-testid="product-description"
      color="textSecondary"
      variant="body2"
      style={descriptionStyle}
    >
      {description}
    </Typography>
  </Fragment>
);

For the tests, we want three things:

• when it is loading, the component renders the skeletons
• when it is not loading anymore, the component renders the texts
• make sure the component doesn't violate the accessibility

We will use the same idea we use for the Tag tests: see if it in the document or not based on the state.

When it is loading, we want to see if the skeleton is in the document, but the title and description texts are not.

describe('when is loading', () => {
  it('does not render anything', () => {
    const { queryByTestId } = render(
      <TitleDescription
        name={product.name}
        description={product.description}
        isLoading
      />
    );

    expect(queryByTestId('name-skeleton-loader')).toBeInTheDocument();
    expect(queryByTestId('description-skeleton-loader')).toBeInTheDocument();
    expect(queryByTestId('product-name')).not.toBeInTheDocument();
    expect(queryByTestId('product-description')).not.toBeInTheDocument();
  });
});

When it is not loading anymore, it renders the texts in the DOM:

describe('when finished loading', () => {
  it('renders the product name and description', () => {
    render(
      <TitleDescription
        name={product.name}
        description={product.description}
        isLoading={false}
      />
    );

    expect(screen.getByText(product.name)).toBeInTheDocument();
    expect(screen.getByText(product.description)).toBeInTheDocument();
  });
});

And a simple test to cover accessibility issues:

it('has no accessibility violations', async () => {
  const { container } = render(
    <TitleDescription
      name={product.name}
      description={product.description}
      isLoading={false}
    />
  );

  const results = await axe(container);

  expect(results).toHaveNoViolations();
});

The next component is the Price. In this component we will provide a skeleton when it is still loading as we did in the other component, and add three different components here:

• PriceWithDiscount: we apply the discount into the original price and render it
• OriginalPrice: it just renders the product price
• Discount: it renders the discount percentage when the product has a discount

But before I start implementing these components, I wanted to structure the data to be used. The price and the discount values are numbers. So let's build a function called getPriceInfo that receives the price and the discount and it will return this data:

{
  priceWithDiscount,
  originalPrice,
  discountOff,
  hasDiscount,
};

With this type contract:

type PriceInfoType = {
  priceWithDiscount: string;
  originalPrice: string;
  discountOff: string;
  hasDiscount: boolean;
};

In this function, it will get the discount and transform it into a boolean, then apply the discount to build the priceWithDiscount, use the hasDiscount to build the discount percentage, and build the originalPrice with the dollar sign:

export const applyDiscount = (price: number, discount: number): number =>
  price - (price * discount) / 100;

export const getPriceInfo = (
  price: number,
  discount: number
): PriceInfoType => {
  const hasDiscount: boolean = Boolean(discount);
  const priceWithDiscount: string = hasDiscount
    ? `$${applyDiscount(price, discount)}`
    : `$${price}`;

  const originalPrice: string = `$${price}`;
  const discountOff: string = hasDiscount ? `${discount}% OFF` : '';

  return {
    priceWithDiscount,
    originalPrice,
    discountOff,
    hasDiscount,
  };
};

Here I also built an applytDiscount function to extract the discount calculation.

I added some tests to cover these functions. As they are pure functions, we just need to pass some values and expect new data.

Test for the applyDiscount:

describe('applyDiscount', () => {
  it('applies 20% discount in the price', () => {
    expect(applyDiscount(100, 20)).toEqual(80);
  });

  it('applies 95% discount in the price', () => {
    expect(applyDiscount(100, 95)).toEqual(5);
  });
});

Test for the getPriceInfo:

describe('getPriceInfo', () => {
  describe('with discount', () => {
    it('returns the correct price info', () => {
      expect(getPriceInfo(100, 20)).toMatchObject({
        priceWithDiscount: '$80',
        originalPrice: '$100',
        discountOff: '20% OFF',
        hasDiscount: true,
      });
    });
  });

  describe('without discount', () => {
    it('returns the correct price info', () => {
      expect(getPriceInfo(100, 0)).toMatchObject({
        priceWithDiscount: '$100',
        originalPrice: '$100',
        discountOff: '',
        hasDiscount: false,
      });
    });
  });
});

Now we can use the getPriceInfo in the Price components to get this structure data and pass down for the other components like this:

export const Price = ({ price, discount, isLoading }: PricePropsType) => {
  if (isLoading) {
    return (
      <Skeleton width="80%" height="18px" data-testid="price-skeleton-loader" />
    );
  }

  const {
    priceWithDiscount,
    originalPrice,
    discountOff,
    hasDiscount,
  }: PriceInfoType = getPriceInfo(price, discount);

  return (
    <Fragment>
      <PriceWithDiscount price={priceWithDiscount} />
      <OriginalPrice hasDiscount={hasDiscount} price={originalPrice} />
      <Discount hasDiscount={hasDiscount} discountOff={discountOff} />
    </Fragment>
  );
};

As we talked earlier, when it is loading, we just render the Skeleton component. When it finishes the loading, it will build the structured data and render the price info. Let's build each component now!

Let's start with the OriginalPrice. We just need to pass the price as a prop and it renders with the Typography component.

type OriginalPricePropsType = {
  price: string;
};

export const OriginalPrice = ({ price }: OriginalPricePropsType) => (
  <Typography display="inline" style={originalPriceStyle} color="textSecondary">
    {price}
  </Typography>
);

Very simple! Let's add a test now.

Just pass a price and see it if was rendered in the DOM:

it('shows the price', () => {
  const price = '$200';
  render(<OriginalPrice price={price} />);
  expect(screen.getByText(price)).toBeInTheDocument();
});

I also added a test to cover accessibility issues:

it('has no accessibility violations', async () => {
  const { container } = render(<OriginalPrice price="$200" />);
  const results = await axe(container);

  expect(results).toHaveNoViolations();
});

The PriceWithDiscount component has a very similar implementation, but we pass the hasDiscount boolean to render this price or not. If it has a discount, render the price with the discount. Otherwise, it won't render anything.

type PricePropsType = {
  hasDiscount: boolean;
  price: string;
};

The props type has the hasDiscount and the price. And the component just renders things based on the hasDiscount value.

export const PriceWithDiscount = ({ price, hasDiscount }: PricePropsType) => {
  if (!hasDiscount) {
    return null;
  }

  return (
    <Typography display="inline" style={priceWithDiscountStyle}>
      {price}
    </Typography>
  );
};

The tests will cover this logic when it has or doesn't have the discount. If it hasn't the discount, the prices will not be rendered.

describe('when the product has no discount', () => {
  it('shows nothing', () => {
    const { queryByTestId } = render(
      <PriceWithDiscount hasDiscount={false} price="" />
    );

    expect(queryByTestId('discount-off-label')).not.toBeInTheDocument();
  });
});

If it has the discount, it will be the rendered in the DOM:

describe('when the product has a discount', () => {
  it('shows the price', () => {
    const price = '$200';
    render(<PriceWithDiscount hasDiscount price={price} />);
    expect(screen.getByText(price)).toBeInTheDocument();
  });
});

And as always, a test to cover accessibility violations:

it('has no accessibility violations', async () => {
  const { container } = render(
    <PriceWithDiscount hasDiscount price="$200" />
  );

  const results = await axe(container);

  expect(results).toHaveNoViolations();
});

The Discount component is pretty much the same as the PriceWithDiscount. Render the discount tag if the product has a discount:

type DiscountPropsType = {
  hasDiscount: boolean;
  discountOff: string;
};

export const Discount = ({ hasDiscount, discountOff }: DiscountPropsType) => {
  if (!hasDiscount) {
    return null;
  }

  return (
    <Typography
      display="inline"
      color="secondary"
      data-testid="discount-off-label"
    >
      {discountOff}
    </Typography>
  );
};

And all the tests we did for the other component, we do the same thing for the Discount component:

describe('Discount', () => {
  describe('when the product has a discount', () => {
    it('shows the discount label', () => {
      const discountOff = '20% OFF';
      render(<Discount hasDiscount discountOff={discountOff} />);
      expect(screen.getByText(discountOff)).toBeInTheDocument();
    });
  });

  describe('when the product has no discount', () => {
    it('shows nothing', () => {
      const { queryByTestId } = render(
        <Discount hasDiscount={false} discountOff="" />
      );

      expect(queryByTestId('discount-off-label')).not.toBeInTheDocument();
    });
  });

  it('has no accessibility violations', async () => {
    const { container } = render(
      <Discount hasDiscount discountOff="20% OFF" />
    );

    const results = await axe(container);

    expect(results).toHaveNoViolations();
  });
});

Now we will build an Image component. This component has the basic skeleton as any other component we've built. If it is loading, wait to render the image source and render the skeleton instead. When it finishes the loading, we will render the image, but only if the component is in the intersection of the browser window.

What does it mean? When you are on a website on your mobile device, you'll probably see the first 4 products. They will render the skeleton and then the image. But below these 4 products, as you're not seeing any of them, it doesn't matter if we are rendering them or not. And we can choose to not render them. Not for now. But on-demand. When you are scrolling, if the product's image is at the intersection of the browser window, we start rendering the image source.

That way we gain performance by speeding up the page load time and reduce the cost by requesting images on demand.

We will use the Intersection Observer API to download images on demand. But before writing any code about this technology, let's start building our component with the image and the skeleton view.

Image props will have this object:

{
  imageUrl,
  imageAlt,
  width,
  isLoading,
  imageWrapperStyle,
  imageStyle,
}

The imageUrl, imageAlt, and the isLoading props are passed by the product component. The width is an attribute for the skeleton and the image tag. The imageWrapperStyle and the imageStyle are props that have a default value in the image component. We'll talk about this later.

Let's add a type for this props:

type ImageUrlType = Pick<ProductType, 'imageUrl'>;
type ImageAttrType = { imageAlt: string; width: string };
type ImageStateType = { isLoading: boolean };
type ImageStyleType = {
  imageWrapperStyle: CSSProperties;
  imageStyle: CSSProperties;
};

export type ImagePropsType = ImageUrlType &
  ImageAttrType &
  ImageStateType &
  ImageStyleType;

The idea here is to give meaning for the types and then compose everything. We can get the imageUrl from the ProductType. The attribute type will have the imageAlt and the width. The image state has the isLoading state. And the image style has some CSSProperties.

At first, the component would like this:

export const Image = ({
  imageUrl,
  imageAlt,
  width,
  isLoading,
  imageWrapperStyle,
  imageStyle,
}: ImagePropsType) => {
  if (isLoading) {
    <Skeleton
      variant="rect"
      width={width}
      data-testid="image-skeleton-loader"
    />
  }

  return (
    <img
      src={imageUrl}
      alt={imageAlt}
      width={width}
      style={imageStyle}
    />
  );
};

Let's build the code to make the intersection observer works.

The idea of the intersection observer is to receive a target to be observed and a callback function that is executed whenever the observed target enters or exits the viewport. So the implementation would be very simple:

const observer: IntersectionObserver = new IntersectionObserver(
  onIntersect,
  options
);

observer.observe(target);

Instantiate the IntersectionObserver class by passing an options object and the callback function. The observer will observe the target element.

As it is an effect in the DOM, we can wrap this into a useEffect.

useEffect(() => {
  const observer: IntersectionObserver = new IntersectionObserver(
    onIntersect,
    options
  );

  observer.observe(target);

  return () => {
    observer.unobserve(target);
  };
}, [target]);

Using useEffect, we have two different things here: the dependency array and the returning function. We pass the target as the dependency function to make sure that we will re-run the effect if the target changes. And the returning function is a cleanup function. React performs the cleanup when the component unmounts, so it will clean up the effect before running another effect for every render.

In this cleanup function, we just stop observing the target element.

When the component starts rendering, the target reference is not set yet, so we need to have a guard to not observe an undefined target.

useEffect(() => {
  if (!target) {
    return;
  }

  const observer: IntersectionObserver = new IntersectionObserver(
    onIntersect,
    options
  );

  observer.observe(target);

  return () => {
    observer.unobserve(target);
  };
}, [target]);

Instead of using this effect in our component, we could build a custom hook to receive the target, some options to customize the configuration, and it would provide a boolean telling if the target is at the intersection of the viewport or not.

export type TargetType = Element | HTMLDivElement | undefined;
export type IntersectionStatus = {
  isIntersecting: boolean;
};

const defaultOptions: IntersectionObserverInit = {
  rootMargin: '0px',
  threshold: 0.1,
};

export const useIntersectionObserver = (
  target: TargetType,
  options: IntersectionObserverInit = defaultOptions
): IntersectionStatus => {
  const [isIntersecting, setIsIntersecting] = useState(false);

  useEffect(() => {
    if (!target) {
      return;
    }

    const onIntersect = ([entry]: IntersectionObserverEntry[]) => {
      setIsIntersecting(entry.isIntersecting);

            if (entry.isIntersecting) {
        observer.unobserve(target);
      }
    };

    const observer: IntersectionObserver = new IntersectionObserver(
      onIntersect,
      options
    );

    observer.observe(target);

    return () => {
      observer.unobserve(target);
    };
  }, [target]);

  return { isIntersecting };
};

In our callback function, we just set if the entry target is intersecting the viewport or not. The setIsIntersecting is a setter from the useState hook we define at the top of our custom hook.

It is initialized as false but will update to true if it is intersecting the viewport.

With this new information in the component, we can render the image or not. If it is intersecting, we can render the image. If not, just render a skeleton until the user gets to the viewport intersection of the product image.

How does it look in practice?

First we define the wrapper reference using useState:

const [wrapperRef, setWrapperRef] = useState<HTMLDivElement>();

It start as undefined. Then build a wrapper callback to set the element node:

const wrapperCallback = useCallback(node => {
  setWrapperRef(node);
}, []);

With that, we can use it to get the wrapper reference by using a ref prop in our div.

<div ref={wrapperCallback}>

After setting the wrapperRef, we can pass it as the target for our useIntersectionObserver and expect a isIntersecting status as a result:

const { isIntersecting }: IntersectionStatus = useIntersectionObserver(wrapperRef);

With this new value, we can build a boolean value to know if we render the skeleton or the product image.

const showImageSkeleton: boolean = isLoading || !isIntersecting;

So now we can render the appropriate node to the DOM.

<div ref={wrapperCallback} style={imageWrapperStyle}>
  {showImageSkeleton ? (
    <Skeleton
      variant="rect"
      width={width}
      height={imageWrapperStyle.height}
      style={skeletonStyle}
      data-testid="image-skeleton-loader"
    />
  ) : (
    <img
      src={imageUrl}
      alt={imageAlt}
      width={width}
    />
  )}
</div>

The full component looks like this:

export const Image = ({
  imageUrl,
  imageAlt,
  width,
  isLoading,
  imageWrapperStyle,
}: ImagePropsType) => {
  const [wrapperRef, setWrapperRef] = useState<HTMLDivElement>();
  const wrapperCallback = useCallback(node => {
    setWrapperRef(node);
  }, []);

  const { isIntersecting }: IntersectionStatus = useIntersectionObserver(wrapperRef);
  const showImageSkeleton: boolean = isLoading || !isIntersecting;

  return (
    <div ref={wrapperCallback} style={imageWrapperStyle}>
      {showImageSkeleton ? (
        <Skeleton
          variant="rect"
          width={width}
          height={imageWrapperStyle.height}
          style={skeletonStyle}
          data-testid="image-skeleton-loader"
        />
      ) : (
        <img
          src={imageUrl}
          alt={imageAlt}
          width={width}
        />
      )}
    </div>
  );
};

Great, now the loading on-demand works well. But I want to build a slightly better experience. The idea here is to have two different sizes of the same image. The low-quality image is requested and we make it visible, but blur while the high-quality image is requested in the background. When the high-quality image finally finishes loading, we transition from the low-quality to the high-quality image with an ease-in/ease-out transition to make it a smooth experience.

Let's build this logic. We could build this into the component, but we could also extract this logic into a custom hook.

export const useImageOnLoad = (): ImageOnLoadType => {
  const [isLoaded, setIsLoaded] = useState(false);
  const handleImageOnLoad = () => setIsLoaded(true);

  const imageVisibility: CSSProperties = {
    visibility: isLoaded ? 'hidden' : 'visible',
    filter: 'blur(10px)',
    transition: 'visibility 0ms ease-out 500ms',
  };

  const imageOpactity: CSSProperties = {
    opacity: isLoaded ? 1 : 0,
    transition: 'opacity 500ms ease-in 0ms',
  };

  return { handleImageOnLoad, imageVisibility, imageOpactity };
};

This hook just provides some data and behavior for the component. The handleImageOnLoad we talked earlier, the imageVisibility to make the low-quality image visible or not, and the imageOpactity to make the transition from transparent to opaque, that way we make it visible after loading it.

The isLoaded is a simple boolean to handle the visibility of the images. Another small detail is the filter: 'blur(10px)' to make the low-quality-image blur and then slowly focusing while transitioning from the low-quality image to the high-quality image.

With this new hook, we just import it, and call inside the component:

const {
  handleImageOnLoad,
  imageVisibility,
  imageOpactity,
}: ImageOnLoadType = useImageOnLoad();

And start using the data and behavior we built.

<Fragment>
  <img
    src={thumbUrl}
    alt={imageAlt}
    width={width}
    style={{ ...imageStyle, ...imageVisibility }}
  />
  <img
    onLoad={handleImageOnLoad}
    src={imageUrl}
    alt={imageAlt}
    width={width}
    style={{ ...imageStyle, ...imageOpactity }}
  />
</Fragment>

The first one has a low-quality image, the thumbUrl. The second has the original high-quality image, the imageUrl. When the high-quality image is loaded, it calls the handleImageOnLoad function. This function will make the transition between one image to the other.

Wrapping up

This is the first part of this project to learn more about user experience, native APIs, typed frontend, and tests.

For the next part of this series, we are going to think more in an architectural way to build the search with filters, but keeping the mindset to bring technical solutions to make the user experience as smooth as possible.

You can find other articles like this on TK's blog.

Resources

• Lazy Loading Images and Video
• Functional Uses for Intersection Observer
• Tips for rolling your own lazy loading
• Intersection Observer API - MDN
• React Typescript Cheatsheet

Test Driven Development (TDD) is sometimes described as “writing tests first”. The TDD mantra states that we should not write code before we have written automated tests that exercise that code. Writing code first is considered suboptimal.

And of course, writing code first is how we develop software following the so-called waterfall model. In that model, we divide software development activities into stages. For example, we have a ‘requirements gathering’ stage, we have an ‘application building’ stage, we have an ‘application testing’ stage, we have an ‘application deployment’ stage and so on.

But how is that different from the agile methodology? Don’t we have the exact same stages in agile?

Agile Methodology vs Waterfall Methodology

Of course we do. The crucial difference is that in agile, those stages are not gated.

In waterfall, we gate the stages and execute them in strict sequence. This means we won’t begin building the shipping application until such time as the requirements have been gathered, completed, signed off on and frozen. Once requirements are frozen (and controlled by our change management policies), we move into the next stage (or phase) – application building.

And similarly, we won’t move into the testing stage until the entire application has been built and we have reached the code complete milestone, at which point code changes have been frozen.

Once code gets frozen (and code freeze is then controlled by our change management policies), we hand it off to the testers. The testing phase begins, and only once all testing has completed (and provided that no significant defects have been detected), do we move into the deployment phase.

In agile, we do all the above activities in parallel. At the same time. We keep working on user stories (specs) while simultaneously building the shipping application. As we’re building the application we are also testing it. And as we are building and testing the application, we are also deploying it.

We learn from the shipping application deployed to production and use that validated learning as the feedback that will inform new user stories. That way, the loop gets closed, and we’re iterating, improving the value incrementally.

The only way to enable such iterative value stream delivery is by relying on automated tests. And as we’ve described, those tests are being written very early in the game. Actually, tests must be written before we write shipping code.

Why then is the title of this article “Don’t Write All Your Tests First, Just Write One”? It sounds a bit confusing. Let’s unpack the meaning of this title. But first, here's an overview of what tech we'll be using:

The technology stack used for this exercise

In the attempt to keep the exercise simple and easy to follow, I have chosen .NET Core platform, together with xUnit.net testing platform. To follow the coding examples, please install .NET Core and xUnit.net.

In order to be able to run the sample code, please open ./tests/tests.csproj file and add this line to the ItemGroup:

<ProjectReference Include="../app/app.csproj" />

You’re now all set for following the coding exercises.

A simple example

To understand the difference between writing all tests first and writing one test first, it may be better to show rather than just tell.

So let’s try to build a simple example – for this exercise I’ve chosen a trivial case of calculating a tip at a restaurant. Often times we find ourselves in a position where we want to tip the restaurant for the service, but it’s tough to calculate percentages in our head. So a nifty little Tip Calculator could come in handy.

Here are the expectations:

As a patron
I want to calculate the total bill (total plus the tip)
Because I want to compliment the restaurant for the service

Scenario #1: Patron calculates the total for terrible service

Given that the restaurant total is $100.00
And the service was terrible
When the tip calculator calculates the total charge
Then tip calculator shows $100.00 total charge

Scenario #2: Patron calculates the total for poor service

Given that the restaurant total is $100.00
And the service was poor
When the tip calculator calculates the total charge
Then tip calculator shows $105.00 total charge

Scenario #3: Patron calculates the total for good service

Given that the restaurant total is $100.00
And the service was good
When the tip calculator calculates the total charge
Then tip calculator shows $110.00 total charge

Scenario #4: Patron calculates the total for great service

Given that the restaurant total is $100.00
And the service was great
When the tip calculator calculates the total charge
Then tip calculator shows $115.00 total charge

Scenario #5: Patron calculates the total for excellent service

Given that the restaurant total is $100.00
And the service was excellent
When the tip calculator calculates the total charge
Then tip calculator shows $120.00 total charge

Let’s now implement the above user story.

We see that the story has 5 acceptance criteria (a.k.a. scenarios). Now we move into the analysis phase – think about what should be the first functionality that our Tip Calculator application should implement. But first, let’s open the command line terminal and create the new directory:

md TipCalculator
cd TipCalculator

and create app and tests directories inside the TipCalculator directory.

Now cd tests and run:

dotnet new xunit

Then cd .. and cd app, then run:

dotnet new classlib

We’re now ready to boogie!

Open your favourite text editor (mine is Visual Studio Code) and set your mind on the expectations. What behaviour are we expecting from the Tip Calculator?

To narrow the scope of our expectations, it usually helps to take one acceptance criteria (i.e. one scenario) and focus on it first. Let’s take scenario #1:

Scenario #1: Patron calculates the total for terrible service

Given that the restaurant total is $100.00
And the service was terrible
When the tip calculator calculates the total charge
Then tip calculator shows $100.00 total charge

In case the service was terrible, we are not adding any tips, and Tip Calculator is calculating a $0.00 tip. So how do we automate that scenario?

My first expectation would be that we need to somehow inform the Tip Calculator that the service was terrible. We either type the word ‘Terrible’ into the input field, or we select ‘Terrible’ from the list of available service ratings.

So the first thing to do here is to articulate some expectations regarding Tip Calculator’s ability to get notified that the service was terrible.

I like to always start with the expectation that what the user inputs is valid. So I’d first write a test that checks if the rating ‘Terrible’ is recognized by the Tip Calculator as a valid rating.

Go to the tests directory, rename the autogenerated UnitTest1.cs file to TipCalculatorTests.cs and add the following test:

[Fact]
public void CheckIfRatingTerribleIsValid(){    
  var expectedResponseForValidRating = true;    
  var actualResponseForValidRating = false;    
  Assert.Equal(expectedResponseForValidRating, 
  actualResponseForValidRating);
}

Now go to the command line, cd tests, and run:

dotnet test

Of course, the above trivial test will fail, because we have hardcoded the values. But it’s always a good practice to make sure we see our tests fail before we proceed. Not observing a test fail may give us false sense of safety later on, if no tests fail and we end up thinking that everything works as expected.

A few more observations about the above test:

• It helps if the test name is descriptive. I chose CheckIfRatingTerribleIsValid to communicate the fact that we must make sure our application is capable of recognizing our commands.
• It also helps if the expected and actual variable names are descriptive. I chose expectedResponseForValidRating and actualResponseForValidRating as fairly indicative of what our expectation in this test is, and also what actual value will the Tip Calculator produce.
• Test is a first-class source code and must be approached with equal care lavished upon the production code.

First design decision

At this point, we are forced to make a decision – how will our nascent Tip Calculator know if the service rating provided by the user is valid or not?

The design decision that comes to mind is that Tip Calculator must be able to store and retrieve some data. In this case, the data we’re interested in is the service rating.

If we go back to the user story and review the five acceptance criteria, we will see that the expectations are that Tip Calculator must be able to recognize five different service ratings:

1. Terrible
2. Poor
3. Good
4. Great
5. Excellent

So the simplest way to get Tip Calculator to store that information would be to endow it with an array, or a list.

But rather than rushing in to implement that list, we should examine the expectations again, to see if there’s anything else we may have missed. And there is – not only must Tip Calculator be able to recognize valid service ratings, it also must be able to associate each rating with a percentage value.

Our analysis shows the following associations:

1. Terrible => 0%
2. Poor => 5%
3. Good => 10%
4. Great => 15%
5. Excellent => 20%

In this case, a simple array or a simple list won’t be sufficient for holding the above associations. What’s the next simplest data structure that will allow us to implement these associations? After doing a little bit of research, we figure out that Hashtable is probably the most fitting data structure that can cover our needs with the least amount of ceremony.

We now navigate to the app directory and rename autogenerated Class1.cs file to TipCalculator.cs. We now want to add a Hashtable that will hold service ratings and the associated percentage values:

System.Collections.Hashtable ratingPercentages = new System.Collections.Hashtable();

Now is a good time to recall that TDD is focused on coupling the expectations to the application’s behaviour, not to the application’s structure. Knowing that, we need to modify our test to make Tip Calculator exhibit some behaviour. The test codifies some expectations with regards to how the application must behave, and the running application provides the evidence of the expected behaviour.

But what is the evidence of the application’s behaviour? There is no other way for us to assess and evaluate application’s behaviour other than through examining the values that the running application produces.

In this case, we are expecting the running application to produce values true or false (Boolean values) after we ask the application if certain value (i.e. service rating) is valid.

To teach the application how to behave in the expected fashion, we need to endow it with an API. In this case, we design the API as follows:

public bool CheckIfRatingIsValid(string rating)

In our test, we will modify the actual expected value to exercise the running application and collect the output value:

As you can see from the screenshot above, we have instantiated TipCalculator but when attempting to ask the instance to check if the supplied rating (“Terrible”) is valid, the editor is complaining that it cannot find that method.

Well of course, the method hasn’t been implemented yet. Now’s the time to go ahead and do it:

public bool CheckIfRatingIsValid(string rating) {    
  return false;
}

Now that the method is implemented, the test works; here is the complete listing:

using Xunit;
using app;

namespace tests {    
  public class TipCalculatorTests {        
    TipCalculator tipCalculator = new TipCalculator();        

    [Fact]        
    public void CheckIfRatingTerribleIsValid(){            
      var expectedResponseForValidRating = true;            
      var actualResponseForValidRating = 
      tipCalculator.CheckIfRatingIsValid("Terrible");                  
      Assert.Equal(expectedResponseForValidRating, 
      actualResponseForValidRating);        
    }    
  }
}

We see from the above example that we’re cheating again (we have hardcoded return false; in our newly minted method). What’s the point of beating around the bush and merely creating skeletons and scaffoldings instead of rolling up our sleeves and doing actual coding? Let’s discuss this important topic.

Discussion about our first design decision

We’re illustrating here how to do TDD step-by-step. The funny part is that this step-by-step illustration is actually the exact way how we do TDD: step-by-step. There is no other way to do TDD than by doing it step-by-step. One step at a time.

How’s that different from any other way of doing software development? Don’t we also do everything step-by-step even when not following TDD methodology? Well, not really. Let me explain:

TDD to me feels like riding a galloping horse. We’re moving swiftly toward our goal, but we’re frequently touching the ground (the galloping horse is every now and then hitting the ground in order to bounce off and run fast).

In comparison, when I’m doing software development without TDD, it feels to me like I’m flying a kite. I’m making swift moves with the kite, but I never touch the ground, not even once. By the time I land the kite, the landing place may not be where I intended the kite to go (it’s very hard to control the direction of a kite if it’s flying in a strong wind).

With TDD, any time we make a change to the code (both the test code and the shipping application code), we run the tests and so we touch the ground. We are galloping, but at the same time we need this frequent grounding. We need to see whether we’re going in the right direction and also whether we’ve broken anything during our galloping. Our tests are the Oracle who keeps telling us if everything works as expected or if something started misbehaving.

Making changes to the code is a risky business. TDD provides a nice harness that is both guiding our design decisions and ensuring we don’t mess up something that we’ve already confirmed works to our expectations.

Replace hardcoded value with actual processing logic

Let’s now replace the hardcoded value with actual running code. We first teach our Tip Calculator that there is a service rating called “Terrible” and that tip percentage associated with this rating is 0:

public bool CheckIfRatingIsValid(string rating) {      
  ratingPercentages.Add("Terrible", 0);    
  return false;
}

Our Tip Calculator is now knowledgeable about the fact that there is a service rating labeled “Terrible” and the tip percentage associated with terrible service is 0%. Great, but we’re still returning hardcoded value false. Time to replace it with actual calculation:

public bool CheckIfRatingIsValid(string rating) {    
  ratingPercentages.Add("Terrible", 0);    
  return ratingPercentages.ContainsKey(rating);
}

Run the test again:

Great, but the code still looks contrived. We are loading the “Terrible” value into the instance of Hashtable ratingPercentages and then immediately checking to see if that value exists in the Hashtable. Now that we have moved from the failing test (Red) to the passing test (Green), it’s time to perform the third step in the TDD loop – Refactor.

Refactoring is basically the activity of modifying the code structure without affecting the code behaviour. Our task here is simple: extract the code responsible for populating of the Hashtable ratingPercentages into a separate block of code.

The most natural place for this loading is in the block of code that is doing the initialization of the Tip Calculator – the constructor method. After refactoring, our shipping application source code looks like this:

using System.Collections;

namespace app {    
  public class TipCalculator {        
    private Hashtable ratingPercentages = new Hashtable();        
    public TipCalculator() {            
      ratingPercentages.Add("Terrible", 0);        
    }        

    public bool CheckIfRatingIsValid(string rating) {            
      return ratingPercentages.ContainsKey(rating);        
    }    
  }
}

Run the test again, and it passes (we’re in green). We have modified the structure of the code without modifying its behaviour! Good job.

Flip the coin

Any time we satisfy a positive expectation, it is a prudent practice to turn things on their head and describe the negative expectation.

At this point, since we’ve satisfied that a legitimate service rating value is found in the Tip Calculator, we want to ensure that non-legitimate values are not found in the Tip Calculator.

What do we mean by non-legitimate values? Any value other than “Terrible”, “Poor”, “Good”, “Great” and “Excellent”. Time to write the new expectation (i.e. test):

[Fact]
public void CheckIfRatingWhateverIsValid() {    
  var expectedResponseForValidRating = true;    
  var actualResponseForValidRating =                        
  tipCalculator.CheckIfRatingIsValid("Whatever");
  Assert.Equal(expectedResponseForValidRating, 
  actualResponseForValidRating);
}

Run the tests:

Fails. As expected. (We specified that our expectation when supplying the service rating as “Whatever” should be true. In reality, it is false, because our Tip Calculator does not contain value “Whatever”.)

Fix the test (change the expectedResponseForValidRating from true to false) and run it again:

A moment of reflection: why did we fake the first test run and made it fail? Because we always want to make sure we observe our new test failing. That way, we’ll know that in the future any successful passing of the test is not merely a false positive.

In praise of steady state

Software engineering is a balancing act between steady state and periods of unstable state. What do we mean by steady state?

If we have a system (a running application) that behaves the way we expect it to behave (i.e. it produces values we have specified as expected values), we declare that the system is in a steady state. It is running, and delivering some value.

That value delivery is still partial. In our case, the only value to the users this Tip Calculator delivers is its ability to recognize service rating “Terrible” as a legitimate rating. In addition, it is capable of informing us that service rating “Whatever” is not a legitimate rating.

That’s not much, but still is better than nothing. And good news – our running application is currently in a steady state. Now we want to look into how to add more valuable behaviour to our Tip Calculator. And the only way to add more value is by making some changes.

Any time we make a change to our application, we disturb its steady state. This disturbance is risky. It may mean our changes could break something that is already working. Because of that concern, we strive to make the duration of this unstable state as short as possible.

Remember how we compared TDD to riding a galloping horse? When the horse is in flight (i.e. not touching the ground) it is advancing toward our goal, but it’s not in the steady state. Only when the horse touches the ground does its state stabilize.

TDD encourages making small changes (in flight) and immediately grounding the system by verifying that it is back in the steady state. We value steady state despite the fact that we eagerly embrace changes. Without changes, we won’t be able to deliver value, but we must do it in a very deliberate, careful fashion.

When doing TDD, we treat changes to steady state like walking on eggshells. No matter how sure we may be in knowing what and how we’re doing software engineering, it is prudent to still let failing tests guide our decisions.

Check if correct tip percentage is associated with service rating

Let’s now introduce another change into our application – a test to verify if correct tip percentage is associated with service rating “Terrible”. Remember that we populated the instance of Hashtable ratingPercentages with the following values:

ratingPercentages.Add("Terrible", 0);

We have written a test that verifies that our Hashtable ratingPercentages does contain legitimate service rating “Terrible”. Now we need a test that verifies that service rating “Terrible” means that the tip percentage for that rating is 0.

[Fact]
public void CheckIfRatingTerribleHasZeroPercentTip() {    
  var expectedZeroPercentForTerribleRating = 0;    
  var actualZeroPercentForTerribleRating = 10;    
  Assert.Equal(expectedZeroPercentForTerribleRating, 
  actualZeroPercentForTerribleRating);
}

The new test CheckIfRatingTerribleHasZeroPercentTip should fail:

Again, we’re purposefully hard coding wrong actual values just so that we could observe our brand new test fail. Now we must replace hard coded value with the actual call to the Tip Calculator’s method that returns tip percentage for the service rating:

[Fact]
public void CheckIfRatingTerribleHasZeroPercentTip() {    
  var expectedZeroPercentForTerribleRating = 0;    
  var actualZeroPercentForTerribleRating =                 
  tipCalculator.GetPercentageTipForRating("Terrible");
  Assert.Equal(expectedZeroPercentForTerribleRating, 
  actualZeroPercentForTerribleRating);
}

As in the previous case, we have invented a new API for Tip Calculator. We call this new capability GetPercentageTipForRating("Terrible"). It takes the value of the service rating and returns the tip percentage for that rating.

Flip over to the app/TipCalculator.cs and add the hard coded skeleton of the new method:

public int GetPercentageTipForRating(string rating) {    
  return 10;
}

Running the test fails again, because we have hard coded the return value. Let’s replace it with actual processing:

public int GetPercentageTipForRating(string rating) {    
  int tipPercentage = Int32.Parse(ratingPercentages[rating].ToString());
  return tipPercentage;
}

Run the test again:

All three tests pass – we’re in green, we’re back to steady state!

What tip percentage do we expect for non-legitimate service ratings?

Many years of experience in the field taught me to be a bit pessimistic. Now that we have our application back in the steady state and delivering value (answering questions about legitimate service ratings and also giving us correct tip percentage for the “Terrible” rating), we need to see what happens when we run our application by giving it non-legitimate service rating value (for example, by giving it service rating “Whatever”).

Time for leaving the steady state yet again. We will write another test:

[Fact]
public void CheckIfRatingWhateverHasNegativeOnePercentTip() {    
  var expectedZeroPercentForWhateverRating = -1;    
  var actualZeroPercentForWhateverRating =       
  tipCalculator.GetPercentageTipForRating("Whatever");
  Assert.Equal(expectedZeroPercentForWhateverRating, 
  actualZeroPercentForWhateverRating);
}

We are describing our expectation when Tip Calculator is asked to return tip percentage for service rating “Whatever”. Because service rating “Whatever” is a non-legitimate rating, we are expecting Tip Calculator to return tip percentage of value -1.

This test now precipitates one improvement to our shipping code. We need to add some logic to first check whether the supplied service rating is legitimate or not. Only if it is legitimate do we ask Hashtable ratingPercentages to tell us what the associated value of the tip percentage is. If the supplied service rating is non-legitimate (for example, if it is “Whatever”) we bypass talking to Hashtable ratingPercentages and simply return -1.

public int GetPercentageTipForRating(string rating) {    
  int tipPercentage = -1;    
  if(CheckIfRatingIsValid(rating)) {        
    tipPercentage = Int32.Parse(ratingPercentages[rating].ToString());    
  }    
  return tipPercentage;
}

Run the tests, and all 4 tests pass:

We are back to the steady state. Another short excursion into the volatile area, and another swift victory and a safe return to steady, imperturbable state.

Populate other service rating tip percentages

Now is a good time to take a breather and make less risky changes, following the already established pattern. Leave the safety of the steady state and make short trips into the volatile territory by adding a new test to verify if service rating “Poor” is a valid, legitimate rating:

[Fact]
public void CheckIfRatingPoorIsValid() {    
  var expectedResponseForValidRating = true;    
  var actualResponseForValidRating =       
  tipCalculator.CheckIfRatingIsValid("Poor");
  Assert.Equal(expectedResponseForValidRating, 
  actualResponseForValidRating);
}

Running this test will fail:

Service rating “Poor” hasn’t been implemented yet. To make the test pass, implement service rating “Poor” by adding this line to the TipCalculator constructor:

ratingPercentages.Add("Poor", 5);

Run the tests, and we’re back to safety:

We’re enjoying steady state with 6 tests successfully passing.

Now that we have added service rating “Poor” associated with the 5% tip, let’s write a test that will describe that expectation:

[Fact]
public void CheckIfRatingPoorHasFivePercentTip() {    
  var expectedZeroPercentForPoorRating = 5;    
  var actualZeroPercentForPoorRating =       
  tipCalculator.GetPercentageTipForRating("Poor");
  Assert.Equal(expectedZeroPercentForPoorRating, 
  actualZeroPercentForPoorRating);
}

The tests run successfully, and we’re back to being safe in the steady state.
I will leave it to the reader to make the changes that will drive the implementation of the service ratings “Good”, “Great” and “Excellent”. At the end of the exercise you should have your system back in the steady state with 12 tests successfully passing:

Calculate grand total given the total and the service rating

We are now ready for the final step – given the total bill and the service rating, we expect Tip Calculator to calculate tip percentage and add it to the total, producing the grand total to be paid to the restaurant.

As we always do, first we describe the expectation:

[Fact]
public void CalculateTotalWithTip() {    
  var expectedTotalWithTip = 135.7;    
  var actualTotalWithTip = 200.0;    
  Assert.Equal(expectedTotalWithTip, actualTotalWithTip);
}

As usual, we first hard code some expectations that we know are going to fail. This is so that we observe our new test failing:

Time to implement processing logic that will calculate correct total with tip. Given the total of $118.0, and the service rating “Great” (15% tip), we’re expecting the total to be $135.7:

[Fact]
public void CalculateTotalWithTip() {    
  var rating = "Great";    
  var total = 118;    
  var expectedTotalWithTip = 135.7;    
  var actualTotalWithTip = tipCalculator.CalculateTotalWithTip(total, 
  rating);    
  Assert.Equal(expectedTotalWithTip, actualTotalWithTip);
}

We have designed a new API the Tip Calculator – a method called CalculateTotalWithTip(total, rating). It takes the total value and the service rating and returns the total with tip. The implementation of the method looks like this:

public double CalculateTotalWithTip(double total, string rating) {    
  double totalWithTip = -1;    
  if(CheckIfRatingIsValid(rating)) {        
    int percentage = GetPercentageTipForRating(rating);        
    totalWithTip = total + ((total/100) * percentage);    
  }    
  return totalWithTip;
}

Run the tests, and we’re back to steady state:

Are we done here?

No, not yet. Even when all tests are in green and we’re back to the steady state, there are still a couple of things we need to do.

To begin with, we need to add a pessimistic expectation for our Tip Calculator calculation of total with the tip based on the service rating:

[Fact]
public void CalculateTotalWithTipForNonlegitimateRating() {    
  var rating = "Meh";    
  var total = 118;    
  var expectedTotalWithTip = 135.7;    
  var actualTotalWithTip = tipCalculator.CalculateTotalWithTip(total, 
  rating);    
  Assert.Equal(expectedTotalWithTip, actualTotalWithTip);
}

Running the tests produces one failing test:

Our expectation for non-legitimate service rating (“Meh”) was incorrect. The actual total is -1, so we need to adjust our expectation by replacing 135.7 with -1. Run the tests again, and we’re back to the steady state!

We now have 14 tests, they all successfully pass, and our Tip Calculator works according to our expectations and satisfies the acceptance criteria.

We’re almost done. One more sanity check before we can confidently ship our shiny new Tip Calculator – we must run mutation testing.

Our mutation testing framework will mutate the shipping code, one line at a time, and will run all tests for each individual mutation.

If the tests complain about the mutated code, all is good, we have killed the mutant. If the tests don’t complain, we’re in trouble. We have a surviving mutant in our codebase, which means there are lines of code in our repo that are doing something for which we haven’t provided any expectations.

Let’s run mutation testing to see how solid our solution is. Good news – our solution has killed 100% of mutants!

Mutation testing has given our shipping application a clean bill of health. Our Tip Calculator seems to be in good shape.

Red-Green-Refactor-Reflect

Let’s review our Tip Calculator building exercise. We started the process by describing our expectations using the classical user story format. User story (as the name implies) is focused on describing scenarios that fulfill end user’s goals.

In this case, the simple goal is to calculate the tip amount from the supplied service rating and the restaurant bill total. The calculated tip amount is then automatically added to the total.

From there we proceeded to build our shipping application by following the TDD methodology. As we’ve demonstrated, the methodology consists of writing a failing test, observing it fail (the Red phase of TDD), then immediately making code changes that ensure the test passes (the Green phase of TDD). Once the test passes, we move into the Refactor phase (we restructure the code without affecting its behaviour). That way, we make sure our code is not expensive to change.

A proper TDD practice also mandates frequent retrospective – we call it reflection. We stop and think about the things we’ve accomplished thus far, to see if we could learn from our recent experiences. This reflection fortifies the process, as it relies on frequent and tight feedback provided by the failing, then succeeding tests.

I have already compared Test Driven Development to the experience of riding a galloping horse. While riding a horse, we’re alternating between flying through the air (i.e. speed achieved when the horse leaps from the ground) and steering the horse. It is impossible to steer the horse while we’re off the ground, up in the air. At that point, we gain speed, but we cannot make any changes of the direction. It is only once the horse touches the ground that we can make a change in direction.

In TDD, we strive to touch the ground as frequently as possible. The longer the leaps we make without touching the ground, the less chance we have for correcting the course.

I also compared software development practices that don’t follow TDD principles to the experience of flying a kite. When flying a kite, we never touch the ground. It is an exhilarating feeling of letting the wind pick the kite up and bounce it up in the air. We can achieve considerable speed that way. But we struggle in such situations to maintain desired course. And after we eventually land the kite, it usually does not land in the spot we originally wanted it to land.

Why is the emphasis of this article on “don’t write tests first”? Many software engineers who are not familiar with agile practices as implemented in TDD usually either claim that writing automated tests isn’t necessary, or claim that automated tests should be written after the code is complete.

Once they start learning about agile and TDD, they may reconsider their practices and decide that writing tests before writing implementation code may make more sense. Still, because of the ingrained waterfall mentality, some of those engineers make the mistake of writing all tests first, and only then move into writing the code.

That approach is completely wrong. It is equivalent to the traditional waterfall approach where we go through the development process by respecting gated phases.

First we write the requirements (in this case, requirements would be expectations written in the form of automated tests). Only once all the requirements (i.e. automated tests) have been written, signed off and frozen, do we move into the next gated phase – write the code for the shipping application.

TDD is the exact opposite of the “write tests first” approach. In TDD, we always write only one test. That test describes a desired behaviour. The desired behaviour does not exist yet (that’s why it is desired), and the test fails.

We then immediately move into making changes to the code in the attempt to create the desired behaviour. Once desired behaviour is created, it gets validated by the test, and if the expectations of the test are satisfied, we move into refactoring the code (to satisfy nonfunctional requirements, such as cost of change).

We practice a rigorous discipline to never succumb to the temptation to write more than one test at a time. That way, we ensure that we keep touching the ground as frequently as possible.

We prefer to remain ‘in flight’ for the shortest possible time. We are ‘in flight’ during that period when the desired behaviour described in the test has not materialized yet. The smaller the expected and desired behaviour is, the shorter will be our ‘in flight’ trajectory. That way, we keep touching the ground often, which gives us a chance to adjust the steering.

Conclusion

Building a simple Tip Calculator is a toy sized problem, and using that exercise to illustrate TDD methodology is not necessarily providing a convincing argument in favour of TDD. Still, within the constraints of a technical article, going over this hands-on exercise may provide valuable insights into the benefits of adopting TDD.

We would still argue that the real benefits of TDD only become apparent when dealing with much larger, more complex software engineering efforts. The ability to remain grounded while making potentially risky changes to a large, complex system is often a life saver.

In addition to that, building software using TDD methodology results in much less rework. TDD drives high degree of modularization, which results in high cohesiveness of the modules and low coupling between the modules.

All these characteristics produce a shipping application whose codebase is easy and inexpensive to change. And lowering the cost of change has proven to be the best way on the path to embracing changes and abandoning the concept known as ‘scope creep’.

Bottom line, TDD enables software engineering teams to deliver high degree of flexibility to the business.

But automation can be tough and can sometimes prove to be costly. How can Github Actions help harden our code and give us more time to work on features instead of bugs?

• What are Github Actions?
• What is CI/CD?
• What are we going to build?
• Part 0: Setting up a project
• Part 1: Automating tests
• Part 2: Post new pull requests to Slack

What are Github Actions?

Actions are a relatively new feature to Github that allow you to set up CI/CD workflows using a configuration file right in your Github repo.

Previously, if you wanted to set up any kind of automation with tests, builds, or deployments, you would have to look to services like Circle CI and Travis or write your own scripts. But with Actions, you have first class support to powerful tooling to automate your workflow.

What is CI/CD?

CD/CD stands for Continuous Integration and Continuous Deployment (or can be Continuous Delivery). They're both practices in software development that allow teams to build projects together quickly, efficiently, and ideally with less errors.

Continuous Integration is the idea that as different members of the team work on code on different git branches, the code is merged to a single working branch which is then built and tested with automated workflows. This helps to constantly make sure everyone's code is working properly together and is well-tested.

Continuous Deployment takes this a step further and takes this automation to the deployment level. Where with the CI process, you automate the testing and the building, Continuous Deployment will automate deploying the project to an environment.

The idea is that the code, once through any building and testing processes, is in a deployable state, so it should be able to be deployed.

What are we going to build?

We're going to tackle two different workflows.

The first will be to simply run some automated tests that will prevent a pull request from being merged if it is failing. We won't walk through building the tests, but we'll walk through running tests that already exist.

In the second part, we'll set up a workflow that sends a message to slack with a link to a pull request whenever a new one is created. This can be super helpful when working on open source projects with a team and you need a way to keep track of requests.

Part 0: Setting up a project

For this guide, you can really work through any node-based project as long as it has tests you can run for Part 1.

If you'd like to follow along with a simpler example that I'll be using, I've set up a new project that you can clone with a single function that has two tests that are able to run and pass.

A function with two tests

If you'd like to check out this code to get started, you can run:

git clone --single-branch --branch start git@github.com:colbyfayock/my-github-actions.git

Once you have that cloned locally and have installed the dependencies, you should be able to run the tests and see them pass!

Passing tests

It should also be noted that you'll be required to have this project added as a new repository on Github in order to follow along.

Follow along with the commit!

Part 1: Automating tests

Tests are an important part of any project that allow us to make sure we're not breaking existing code while we work. While they're important, they're also easy to forget about.

We can remove the human nature out of the equation and automate running our tests to make sure we can't proceed without fixing what we broke.

Step 1: Creating a new action

The good news, is Github actually makes it really easy to get this workflow started as it comes as one of their pre-baked options.

We'll start by navigating to the Actions tab on our repository page.

Github Actions starting page

Once there, we'll immediately see some starter workflows that Github provides for us to dive in with. Since we're using a node project, we can go ahead and click Set up this workflow under the Node.js workflow.

Setting up a Node.js Github Action workflow

After the page loads, Github will land you on a new file editor that already has a bunch of configuration options added.

We're actually going to leave this "as is" for our first step. Optionally, you can change the name of the file to tests.yml or something you'll remember.

Adding a new Github Action workflow file

You can go ahead and click Start commit then either commit it directory to the master branch or add the change to a new branch. For this walkthrough, I'll be committing straight to master.

To see our new action run, we can again click on the Actions tab which will navigate us back to our new Actions dashboard.

Viewing Github Action workflow events

From there, you can click on Node.js CI and select the commit that you just made above and you'll land on our new action dashboard. You can then click one of the node versions in the sidebar via build (#.x), click the Run npm test dropdown, and we'll be able to see the output of our tests being run (which if you're following along with me, should pass!).

Viewing logs of a Github Action workflow

Follow along with the commit!

Step 2: Configuring our new action

So what did we just do above? We'll walk through the configuration file and what we can customize.

Github Action Node.js workflow file

Starting from the top, we specify our name:

name: Node.js CI

This can really be whatever you want. Whatever you pick should help you remember what it is. I'm going to customize this to "Tests" so I know exactly what's going on.

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

The on key is how we specify what events trigger our action. This can be a variety of things like based on time with cron. But here, we're saying that we want this action to run any time someone pushes commits to master or someone creates a pull request targeting the master branch. We're not going to make a change here.

jobs:
  build:
    runs-on: ubuntu-latest

This next bit creates a new job called build. Here we're saying that we want to use the latest version of Ubuntu to run our tests on. Ubuntu is common, so you'll only want to customize this if you want to run it on a specific environment.

    strategy:
      matrix:
        node-version: [10.x, 12.x, 14.x]

Inside of our job we specify a strategy matrix. This allows us to run the same tests on a few different variations.

In this instance, we're running the tests on 3 different versions of node to make sure it works on all of them. This is definitely helpful to make sure your code is flexible and future proof, but if you're building and running your code on a specific node version, you're safe to change this to only that version.

    steps:
    - uses: actions/checkout@v2
    - name: Use Node.js ${{ matrix.node-version }}
      uses: actions/setup-node@v1
      with:
        node-version: ${{ matrix.node-version }}
    - run: npm ci
    - run: npm run build --if-present
    - run: npm test

Finally, we specify the steps we want our job to run. Breaking this down:

• uses: actions/checkout@v2: In order for us to run our code, we need to have it available. This checks out our code on our job environment so we can use it to run tests.
• uses: actions/setup-node@v1: Since we're using node with our project, we'll need it set up on our environment. We're using this action to do that setup for us for each version we've specified in the matrix we configured above.
• run: npm ci: If you're not familiar with npm ci, it's similar to running npm install but uses the package-lock.json file without performing any patch upgrades. So essentially, this installs our dependencies.
• run: npm run build --if-present: npm run build runs the build script in our project. The --if-present flag performs what it sounds like and only runs this command if the build script is present. It doesn't hurt anything to leave this in as it won't run without the script, but feel free to remove this as we're not building the project here.
• run: npm test: Finally, we run npm test to run our tests. This uses the test npm script set up in our package.json file.

And with that, we've made a few tweaks, but our tests should run after we've committed those changes and pass like before!

Logs of passing tests in Github Action workflow

Follow along with the commit!

Step 3: Testing that our tests fail and prevent merges

Now that our tests are set up to automatically run, let's try to break it to see it work.

At this point, you can really do whatever you want to intentionally break the tests, but here's what I did:

Code diff - https://github.com/colbyfayock/my-github-actions/pull/1

I'm intentionally returning different expected output so that my tests will fail. And they do!

Failing status checks on pull request

In my new pull request, my new branch breaks the tests, so it tells me my checks have failed. If you noticed though, it's still green to merge, so how can we prevent merges?

We can prevent pull requests from being merged by setting up a Protected Branch in our project settings.

First, navigate to Settings, then Branches, and click Add rule.

Github branch protection rules

We'll then want to set the branch name pattern to *, which means all branches, check the Require status checks to pass before merging option, then select all of our different status checks that we'd like to require to pass before merging.

Setting up a branch protection rule in Github

Finally, hit Create at the bottom of the page.

And once you navigate back to the pull request, you'll notice that the messaging is a bit different and states that we need our statuses to pass before we can merge.

Failing tests preventing merge in pull request

Note: as an administrator of a repository, you'll still be able to merge, so this technically only prevents non-administrators from merging. But will give you increased messaging if the tests fail.

And with that, we have a new Github Action that runs our tests and prevents pull requests from merging if they fail.

Follow along with the pull request!

Note: we won't be merging that pull request before continuing to Part 2.

Part 2: Post new pull requests to Slack

Now that we're preventing merge requests if they're failing, we want to post a message to our Slack workspace whenever a new pull request is opened up. This will help us keep tabs on our repos right in Slack.

For this part of the guide, you'll need a Slack workspace that you have permissions to create a new developer app with and the ability to create a new channel for the bot user that will be associated with that app.

Step 1: Setting up Slack

There are a few things we're going to walk through as we set up Slack for our workflow:

• Create a new app for our workspace
• Assign our bot permissions
• Install our bot to our workspace
• Invite our new bot to our channel

To get started, we'll create a new app. Head over to the Slack API Apps dashboard. If you already haven't, log in to your Slack account with the Workspace you'd like to set this up with.

Creating a new Slack app

Now, click Create New App where you'll be prompted to put in a name and select a workspace you want this app to be created for. I'm going to call my app "Gitbot" as the name, but you can choose whatever makes sense for you. Then click Create App.

Configuring a new Slack app

Once created, navigate to the App Home link in the left sidebar. In order to use our bot, we need to assign it OAuth scopes so it has permissions to work in our channel, so select Review Scopes to Add on that page.

Reviewing Slack app scopes

Scroll own and you'll see a Scopes section and under that a Bot Token section. Here, click Add an OAuth Scope. For our bot, we don't need a ton of permissions, so add the channels:join and chat:write scopes and we should be good to go.

Adding scopes for a Slack app Bot Token

Now that we have our scopes, let's add our bot to our workspace. Scroll up on that same page to the top and you'll see a button that says Install App to Workspace.

Installing Slack app to a workspace

Once you click this, you'll be redirected to an authorization page. Here, you can see the scopes we selected for our bot. Next, click Allow.

Allowing permission for Slack app to be installed to workspace

At this point, our Slack bot is ready to go. At the top of the OAuth & Permissions page, you'll see a Bot User OAuth Access Token. This is what we'll use when setting up our workflow, so either copy and save this token or remember this location so you know how to find it later.

Note: this token is private - don't give this out, show it in a screencast, or let anyone see it!

Copying OAuth Access Token for Slack bot user

Finally, we need to invite our Slack bot to our channel. If you open up your workspace, you can either use an existing channel or create a new channel for these notifications, but you'll want to enter the command /invite @[botname] which will invite our bot to our channel.

Inviting Slack bot user to channel

And once added, we're done with setting up Slack!

Slack bot was added to channel

Create a Github Action to notify Slack

Our next step will be somewhat similar to when we created our first Github Action. We'll create a workflow file which we'll configure to send our notifications.

While we can use our code editors to do this by creating a file in the .github directory, I'm going to use the Github UI.

First, let's navigate back to our Actions tab in our repository. Once there, select New workflow.

Setting up a new Github Action workflow

This time, we're going to start the workflow manually instead of using a pre-made Action. Select set up a workflow yourself at the top.

Setting up a Github Action workflow manually

Once the new page loads, you'll be dropped in to a new template where we can start working. Here's what our new workflow will look like:

name: Slack Notifications

on:
  pull_request:
    branches: [ master ]

jobs:
  notifySlack:

    runs-on: ubuntu-latest

    steps:
    - name: Notify slack
      env:
        SLACK_BOT_TOKEN: ${{ secrets.SLACK_BOT_TOKEN }}
      uses: abinoda/slack-action@master
      with:
        args: '{\"channel\":\"[Channel ID]\",\"blocks\":[{\"type\":\"section\",\"text\":{\"type\":\"mrkdwn\",\"text\":\"*Pull Request:* ${{ github.event.pull_request.title }}\"}},{\"type\":\"section\",\"text\":{\"type\":\"mrkdwn\",\"text\":\"*Who?:* ${{ github.event.pull_request.user.login }}\n*Request State:* ${{ github.event.pull_request.state }}\"}},{\"type\":\"section\",\"text\":{\"type\":\"mrkdwn\",\"text\":\"<${{ github.event.pull_request.html_url }}|View Pull Request>\"}}]}'

So what's happening in the above?

• name: we're setting a friendly name for our workflow
• on: we want our workflow to trigger when there's a pull request is created that targets our master branch
• jobs: we're creating a new job called notifySlack
• jobs.notifySlack.runs-on: we want our job to run on a basic setup of the latest Unbuntu
• jobs.notifySlack.steps: we really only have one step here - we're using a pre-existing Github Action called Slack Action and we're configuring it to publish a notification to our Slack

There are two points here we'll need to pay attention to, the env.SLACK_BOT_TOKEN and the with.args.

In order for Github to communicate with Slack, we'll need a token. This is what we're setting in env.SLACK_BOT_TOKEN. We generated this token in the first step. Now that we'll be using this in our workflow configuration, we'll need to add it as a Git Secret in our project.

_Github secrets including SLACK_BOTTOKEN

The with.args property is what we use to configure the payload to the Slack API that includes the channel ID (channel) and our actual message (blocks).

The payload in the arguments is stringified and escaped. For example, when expanded it looks like this:

{
  "channel": "[Channel ID]",
  "blocks": [{
    "type": "section",
    "text": {
      "type": "mrkdwn",
      "text": "*Pull Request:* ${{ github.event.pull_request.title }}"
    }
  }, {
    "type": "section",
    "text": {
      "type": "mrkdwn",
      "text": "*Who?:*n${{ github.event.pull_request.user.login }}n*State:*n${{ github.event.pull_request.state }}"
    }
  }, {
    "type": "section",
    "text": {
      "type": "mrkdwn",
      "text": "<${{ github.event.pull_request._links.html.href }}|View Pull Request>"
    }
  }]
}

Note: this is just to show what the content looks like, we need to use the original file with the stringified and escaped argument.

Back to our configuration file, the first thing we set is our channel ID. To find our channel ID, you'll need to use the Slack web interface. Once you open Slack in your browser, you want to find your channel ID in the URL:

https://app.slack.com/client/[workspace ID]/[channel ID]

Channel ID in Slack web app URL

With that channel ID, you can modify our workflow configuration and replace [Channel ID] with that ID:

with:
  args: '{\"channel\":\"C014RMKG6H2\",...

The rest of the arguments property is how we set up our message. It includes variables from the Github event that we use to customize our message.

We won't go into tweaking that here, as what we already have will send a basic pull request message, but you can test out and build your own payload with Slack's Block Kit Builder.

Follow along with the commit!

Test out our Slack workflow

So now we have our workflow configured with our Slack app, finally we're ready to use our bot!

For this part, all we need to do is create a new pull request with any change we want. To test this out, I simply created a new branch where I added a sentence to the README.md file.

Code diff - https://github.com/colbyfayock/my-github-actions/pull/2

Once you create that pull request, similar to our tests workflow, Github will run our Slack workflow! You can see this running in the Actions tab just like before.

As long as you set everything up correctly, once the workflow runs, you should now have a new message in Slack from your new bot.

Slack bot automated message about new pull request

Note: we won't be merging that pull request in.

What else can we do?

Customize your Slack notifications

The message I put together is simple. It tells us who created the pull request and gives us a link to it.

To customize the formatting and messaging, you can use the Github Block Kit Builder to create your own.

If you'd like to include additional details like the variables I used for the pull request, you can make use of Github's available contexts. This lets you pull information about the environment and the job to customize your message.

I couldn't seem to find any sample payloads, so here's an example of a sample github context payload you would expect in the event.

Sample github context

More Github actions

With our ability to create new custom workflows, that's not a lot we can't automate. Github even has a marketplace where you can browse around for one.

If you're feeling like taking it a step further, you can even create your own! This lets you set up scripts to configure a workflow to perform whatever tasks you need for your project.

Join in the conversation!

What do you use Github actions for?

Share with me on Twitter!

Audit website performance manually, programmatically, or automatically

Cityscape Birds Eye View

Lighthouse is an open-source project by Google that gives you a way to measure web page performance. It has configurable settings for reproducing various conditions. You can set network and device type to simulate, for example.

You give Lighthouse a URL to audit, it runs a series of audits against the page, and then it generates a report on how well the page did. From there, use the failing audits as indicators on how to improve the page. Each audit has a reference doc explaining why the audit is important, as well as how to fix it. Lighthouse

There are many reasons why you’d want to measure performance, but one of the most important is about the impact on SEO. I go into more detail about this and how to address certain metrics in this article.

Running Lighthouse with Chrome DevTools

You can run performance audits manually with the Chrome DevTools browser extension. Simply fire up the extension from the web page you’d like to test and select the “Audits” panel.

Chrome DevTools “Audits” Panel

Among a variety of audits, you can choose “performance”. You can also choose to simulate device type and network throttling. Some information specifically about throttling can be found in the Lighthouse project Github repo.

Click on “Run audits” next. Upon completion, Lighthouse provides a report within the extension UI.

Lighthouse Performance Report

This report is a general overview of important metrics, opportunities, and overall performance score. Thumbnails illustrate the lifecycle of page load. What does this all mean? Google provides a plethora of documentation describing each metric, how to address them and the overall performance score.

In the top left side of Chrome DevTools panel is a download icon that you can use to download the full report in JSON format. You can then use it to create a PDF report via Lighthouse Report Viewer.

Due to the high volume of factors playing into the lifecycle of page load, it’s important to compare results in batches. Taking an average of 5 runs, for example, will provide better insight.

Running Lighthouse Programmatically

For our standard “run of the mill” situations, the above should suffice. Another way to run Lighthouse involves installing the open-source package via NPM and following the instructions in the CLI documentation. This can be beneficial if you want to run audits programmatically in a build pipeline, for example.

Similar to the above, you can also run Lighthouse in code by following the documentation for using the Node module programmatically. You could create a full-fledged Node.js application with Lighthouse ?!

Running Lighthouse Automatically Over Time

So now that we’re pros — let’s take this to the next level. There are many integrations listed in the Lighthouse documentation, so let’s take a look at one of them.

Using “Foo” to Run Lighthouse and Compare Results Over Time

In an engineering setting where many developers are deploying application changes on a regular basis, it can be important to monitor website performance over time to associate change sets with performance degradation or improvement. Another example would be teams that have initiatives to improve performance for SEO ranking or other reasons. In these situations, it’s critical to monitor website performance over days, weeks, months, etc.

You can add URLs to track at www.foo.software and monitor performance change. Foo also provides email, Slack or PagerDuty notifications when performance has dropped below a threshold defined by the user, when it’s back to normal, and when improvements are identified automatically!

The best part about it is that you can create an account for free! Once registered and logged in, click the “Pages” link from the top navigation. This is where you can add URLs to monitor. Foo saves results and displays a timeline chart providing a visualization of important metrics. You can toggle through days, weeks, months and drill into detailed reports.

Amazon Example Foo Lighthouse Timeline Chart

Conclusion

Lighthouse is becoming an industry standard in website performance measurement. There are books worth documentation about Lighthouse that provides details of important metrics.

Motivation

Writing automated tests for Desktop applications is not an easy task. Especially when it uses Windows Presentation Foundation (WPF). This allows so many possibilities of nested control and complex grids and menus.

Here at Clemex, we use a tool to automate desktop tests that rely on the WPF [FrameworkElement.Name](https://msdn.microsoft.com/en-us/library/system.windows.frameworkelement(v=vs.110).aspx) property to interact with the application. Because we need to create dynamic controls based on collection data, they can end up with the same name for multiple UI elements.

For example, the following code generates a menu based on a collection of panels.

Inspecting the element tree, we would see that we now have multiple elements with the same name: “MenuBtn”.

To avoid this situation and have unique names for each button, we came up with four different approaches.

• Using the Code-Behind
• Using data binding
• Using attached properties
• Using collection indexes

Using the Code-Behind

Assuming that we have access to some unique ID on the elements data context. The easiest approach is to use the Loaded event of FrameworkElement to set a unique name using the code-behind model.

Now, when we check the element tree, we will see that we have unique names for each button.

Using data binding

Using data binding makes our code much cleaner, as well as easier to read and understand. If we try a similar approach using data binding we might end up with source code like the following:

Unfortunately, if we try to build this code, we will get a compilation error with the message:

MarkupExtensions are not allowed for Uid or Name property values, so ‘{Binding Panel.PanelType, StringFormat=’MenuBtn{0}’}’ is not valid.

This restriction prevents us from binding directly to the Name property.

Using attached properties

To overcome the limitation of the previous attempt we can define a new property that would set the name for us. To add new properties to existing controls we can use Attached Properties.

The OnValueChanged event triggers every time the value of our property changes. When that happens, we get the new value and set it to be the FrameworkElement name. We are giving our attached property the name Name. It could be anything we want, like CustomName or TestName.

To use the new property, we need to add a namespace to the XAML and attach the property to our button.

Our code will now compile without any problems, and we will have unique names for each element.

Using collection indexes

In the previous example, we created unique names by appending the property Id. There are other scenarios where we don’t have an ID on the item to create a unique element name. For that, we can instead use the collection index.

Let’s try to bind our button collection to a list of strings.

To achieve that, we can use the same AttachedProperty with a converter. It will look for the index of the element inside the collection.

In the XAML, we will now use MultiBinding because we need both the element and the collection.

Looking at the element tree we can see that our buttons are named MenuBtn00, MenuBtn01 and so on.

Summary

Generating unique names for dynamically created WPF controls can be done in an elegant way by using Attached Properties and using the multi-binding with a custom converter.