And honestly? It was a fair point. You'd write the contract — the interface, the abstract class, the protocol — and then write the implementation. Two files where one would do. That's more surface area, more indirection, and more to maintain.

The Ruby and Rails communities built an entire philosophy around this: convention over configuration, less ceremony, fewer keystrokes. If the framework could infer your intent, why spell it out?

Then AI happened.

I was recently chatting with a CEO about what current-generation software engineers get wrong, and he put it cleanly:

"Abstract interfaces were challenging a few months ago just because it required twice as much code. But with AI, lines of code are free. The reason we still need such constructs is because at some point a human still needs to look at the code. Interfaces reduce the cognitive load."

That framing stuck with me. The cost of writing code has collapsed. The cost of reading it hasn't moved. And that asymmetry changes everything about how you should think about abstraction.

Here's what I mean.

Table of Contents

• Your Brain Is the Bottleneck

• The Greats Already Knew This

• The Economics Have Flipped

• The Data Backs It Up

• The Contrarian Case (And Why It Actually Agrees)

• What This Means for You

• References

Your Brain Is the Bottleneck

This isn't a vibes argument. There's actual neuroscience behind why interfaces help.

In 1988, educational psychologist John Sweller introduced Cognitive Load Theory. A 2022 ACM review covers how it's been applied to computing education since.

The short version: your brain juggles three types of load when processing information. Intrinsic load is the inherent difficulty of the problem itself. Extraneous load is the noise — poorly organized information, unnecessary details, bad naming. Germane load is the good stuff — the mental effort you spend building useful mental models.

Here's the kicker: your working memory can only hold a handful of chunks of information at a time — cognitive scientists typically estimate somewhere between 2 and 6. Not 2 to 6 files, or 2 to 6 classes — 2 to 6 things.

Felienne Hermans explores this in The Programmer's Brain (2021), arguing that design patterns act as chunking aids. When you recognize a Strategy pattern, your brain collapses an entire class hierarchy into a single cognitive unit. The word "Strategy" replaces five classes and their relationships. That's not hand-waving about clean code — that's how human memory actually works.

And we can literally see it on brain scans. In 2021, a team led by Norman Peitek and Janet Siegmund published an fMRI study on program comprehension that won the ACM SIGSOFT Distinguished Paper Award at ICSE.

They put developers in brain scanners and watched what happened when they read code. The finding: semantic-level comprehension — understanding what code does — required measurably less neural activation than bottom-up syntactic parsing — tracing how it does it.

An interface lets you comprehend at the semantic level. UserRepository.findById(id) tells you everything you need to know without opening the implementation. Your brain doesn't need to hold the SQL query, the connection pool logic, the error handling, and the result mapping in working memory simultaneously. The interface compresses all of that into one chunk.

That's not elegance. That's neuroscience.

The Greats Already Knew This

The case for abstraction isn't new. The people who built the foundations of computer science were making this argument before most of us were born.

Dijkstra said it with precision:

"The purpose of abstracting is not to be vague, but to create a new semantic level in which one can be absolutely precise."

Abstraction isn't about hiding things from people who can't handle complexity. It's about creating a level of discourse where you can reason clearly.

David Parnas formalized information hiding in his 1972 ACM paper: "Every module is characterized by its knowledge of a design decision which it hides from all others." He proved that decomposing systems by design decisions (rather than processing steps) produced modules that were both more flexible and easier to understand. Comprehensibility wasn't a bonus — it was the design criterion.

Tony Hoare argued that abstraction is the most powerful tool available to the human intellect — a way to manage complexity by focusing on what matters and ignoring what doesn't. Martin Fowler brought it down to earth:

"Any fool can write code that a computer can understand. Good programmers write code that humans can understand."

And then there's John Ousterhout, whose book A Philosophy of Software Design (2018) makes the connection to cognitive load explicit. His central argument: more lines of code can actually be simpler if they reduce cognitive load.

His concept of deep modules — simple interfaces hiding complex implementations — is essentially the argument that interfaces are worth their weight in code. The Unix file system API (open, close, read, write, lseek) is five functions hiding an enormous amount of complexity. That's a deep module. That's the goal.

The Gang of Four put it first in their book for a reason. Page one: "Program to an interface, not an implementation."

None of this is controversial. But it's easy to forget when your AI tool just generated 200 lines of perfectly functional inline code in three seconds.

The Economics Have Flipped

Here's where the CEO's insight becomes an economic argument.

The historical case against interfaces was always about writing cost. Interfaces meant more code to write, more files to create, more boilerplate to maintain. The entire dynamic typing movement — Python, Ruby, JavaScript — was partly a reaction to the ceremony that languages like Java imposed. Convention over configuration. Don't Repeat Yourself. Less is more.

But ask yourself: what exactly is the cost of writing boilerplate now?

GitHub's 2022 controlled study found that developers using Copilot completed tasks 55% faster. The boilerplate that used to justify skipping interfaces — the extra file, the type definitions, the method signatures — takes seconds to generate. The writing cost of an interface has effectively collapsed to zero.

But again, the reading cost hasn't budged.

Robert C. Martin argued in Clean Code (2008) that developers spend far more time reading code than writing it — an observation he framed as a ratio of 10 to 1.

You can quibble with the exact number (it's anecdotal), but the direction is consistent across studies. A large-scale field study tracking 78 professional developers across 3,148 working hours found they spend roughly 58% of their time on program comprehension alone. New developer onboarding averages six weeks — most of which is spent understanding existing systems, not producing new ones.

Addy Osmani named this asymmetry perfectly. In a March 2026 piece, he described comprehension debt:

"When a developer on your team writes code, the human review process has always been a bottleneck — but a productive and educational one. Reading their PR forces comprehension. AI-generated code breaks that feedback loop. The volume is too high."

The output looks clean, passes linting, follows conventions — precisely the signals that historically triggered merge confidence. But comprehension debt is distinct from technical debt because it accumulates invisibly — your velocity metrics, your DORA scores, your PR counts all look fine while your team's actual understanding of the codebase quietly erodes.

So here's the math: AI reduced the cost of writing abstractions to near zero. The cost of not having them — in human reading time, onboarding friction, and comprehension debt — hasn't changed at all. The break-even point for "is this interface worth it?" just shifted massively in favor of "yes."

The Data Backs It Up

This isn't theoretical. We have data on what happens when AI generates code without good abstractions.

GitClear analyzed 211 million changed lines of code between 2020 and 2024. Their findings: code churn — lines reverted or updated within two weeks — doubled compared to the pre-AI baseline. Copy-pasted code blocks rose from 8.3% to 12.3%. And refactoring-associated changes dropped from 25% to under 10%.

AI-generated code, as they put it, "resembles an itinerant contributor, prone to violate the DRY-ness of the repos visited."

The METR study (2025) found something even more striking. Experienced open-source developers predicted AI would make them 24% faster. They perceived being 20% faster while using it. They were actually 19% slower. The perception gap is the story — you feel productive while generating code that creates more work downstream.

And then there's a study from Anthropic (yes, the company that makes Claude — full disclosure). They observed 52 software engineers learning a new library. The AI-assisted group completed tasks at the same speed, but scored 17% lower on comprehension quizzes afterward — 50% versus 67%. The biggest declines were in debugging ability. You can ship code you don't understand. You can't debug code you don't understand.

Kent Beck put it bluntly: "The value of 90% of my skills just dropped to $0. The leverage for the remaining 10% went up 1000x." What that remaining 10% is, he leaves deliberately open — but it's hard to read that and not think about system design.

The Contrarian Case (And Why It Actually Agrees)

I'd be dishonest if I didn't address the people who argue against abstraction. And some of them are very smart.

Casey Muratori's "Clean Code, Horrible Performance" demonstrated that polymorphism and virtual dispatch can make code 10 to 15 times slower than straightforward procedural alternatives.

His benchmark is real. If you're writing a game engine or a high-frequency trading system, abstract interfaces on your hot path will cost you.

Dan Abramov wrote "Goodbye, Clean Code" after watching a premature abstraction make his codebase harder to modify:

"My code traded the ability to change requirements for reduced duplication, and it was not a good trade."

Sandi Metz put it more sharply: "Duplication is far cheaper than the wrong abstraction."

And Rich Hickey, in his talk "Simple Made Easy", draws the critical distinction: simple (not intertwined) is not the same as easy (familiar). Wrong abstractions complect — they braid concerns together rather than separating them.

Here's the thing: none of these are arguments against abstraction. They're arguments against bad abstraction.

Muratori's performance argument applies to hot paths in performance-critical systems — not to your REST API's service layer. Abramov and Metz argue against premature abstraction — pulling patterns out before you understand the domain. And Hickey's entire talk is a case for the right abstractions, the ones that genuinely decompose rather than complect.

The irony is that in an AI-assisted world, these arguments are easier to address. You can generate the explicit, unabstracted version first. Let it stabilize. Watch the patterns emerge. Then extract the abstraction — with AI handling the mechanical refactoring. The cost of the "duplicate first, abstract later" approach just dropped to near zero.

What This Means for You

If you're writing code with AI tools — and at this point, most of us are — the temptation is to let the AI produce whatever it produces and move on. It works. It passes the tests. Ship it.

But "it works" is table stakes. The harder question is: can the next person who opens this code understand it in under five minutes? Can you understand it in six months?

Interfaces aren't about making code prettier or satisfying some abstract (pun intended) design principle. They're compression algorithms for human cognition. They let your brain operate at the semantic level instead of the syntactic level. And now that AI has eliminated the only real cost of creating them — the boilerplate — there's no economic argument left for skipping them.

The rules haven't changed. The excuse has just expired.

References

Academic Papers

• Duran, R., Zavgorodniaia, A., & Sorva, J. (2022). "Cognitive Load Theory in Computing Education Research: A Review." ACM Transactions on Computing Education, 22(4), Article 40.

• Parnas, D.L. (1972). "On the Criteria To Be Used in Decomposing Systems into Modules." Communications of the ACM, 15(12), 1053–1058.

• Peitek, N., Apel, S., Parnin, C., Brechmann, A., & Siegmund, J. (2021). "Program Comprehension and Code Complexity Metrics: An fMRI Study." ICSE 2021. ACM SIGSOFT Distinguished Paper Award.

• Peng, S., Kalliamvakou, E., Cihon, P., & Demirer, M. (2023). "The Impact of AI on Developer Productivity: Evidence from GitHub Copilot." arXiv:2302.06590.

• Shen, J.H. & Tamkin, A. (2026). "How AI Impacts Skill Formation." arXiv:2601.20245.

• Xia, X., Bao, L., Lo, D., Xing, Z., Hassan, A.E., & Li, S. (2018). "Measuring Program Comprehension: A Large-Scale Field Study with Professionals." IEEE Transactions on Software Engineering, 44(10), 951–976.

• METR. (2025). "Measuring the Impact of Early 2025 AI on Experienced Open Source Developer Productivity." metr.org.

Talks and Blog Posts

• Hickey, R. (2011). "Simple Made Easy." Strange Loop Conference.

• Beck, K. (2023). "90% of My Skills Are Now Worth $0." Tidy First? Substack.

• Osmani, A. (2026). "Comprehension Debt: The Hidden Cost of AI-Generated Code." addyosmani.com.

• Muratori, C. (2023). "Clean Code, Horrible Performance." Computer Enhance.

• Abramov, D. (2020). "Goodbye, Clean Code." overreacted.io.

• Metz, S. (2016). "The Wrong Abstraction." sandimetz.com.

• GitClear. (2025). "AI Assistant Code Quality in 2025." gitclear.com.

This is the fundamental problem with single-pass AI code generation: the same context that created the code is the one evaluating it. There's no adversarial pressure. No second opinion. No fresh eyes.

What if you could structure the work so that separate agents generate and critique each other in iterative loops, the way a generator and discriminator improve each other in a GAN? The code that reaches you has already survived an argument between agents who disagreed about whether it was good enough.

This article walks through why that pattern works, how to build it, and when it is (and is not) worth the extra tokens. The concrete example is an open source project called Claude Forge, but the ideas are framework-agnostic. Anything that supports subagent spawning with fresh context windows can implement this pattern.

Table of Contents

• The Single-Pass Problem

• What the Ecosystem Is Solving

• The GAN Pattern Applied to Code

• Why Rhetorical Questions Outperform Direct Instructions

• Feedback as Filesystem

• The Zero-Context Engineer

• Phase-0: Immutable Conventions

• Convergence Design: Knowing When to Stop

• Ground Truth Documents and the Pipeline

• What the Adversarial Loop Actually Catches

• Honest Trade-offs

• When to Use This (And When Not To)

• Getting Started

Prerequisites

• Familiarity with Claude Code or a similar AI coding agent

• A working installation of Claude Code (for the hands-on sections)

• Basic understanding of how LLM context windows work

• Git installed and configured

No machine learning background is required. The GAN concepts are explained from first principles where they appear.

The Single-Pass Problem

The AI generates code in one pass. If it hallucinates a file path, misunderstands the architecture, or writes tests that don't actually test anything, you catch it during review. Or worse, you don't.

This isn't a hypothetical. Anyone who has used AI coding agents at scale has seen placeholder tests like expect(true).toBe(true), phantom dependencies where Phase 2 assumes a model that Phase 1 never creates, and instructions so ambiguous that two valid interpretations exist. These aren't rare edge cases. They're the predictable failure mode of single-pass generation.

The problem compounds with task complexity. A simple utility function generates fine in one pass. An auth middleware with token refresh, error handling, rate limiting, and logging across multiple files? The agent starts cutting corners, because the entire generation happened inside one context window that is simultaneously tracking the plan, the code, the tests, and the growing weight of its own prior reasoning.

What the Ecosystem Is Solving

There is a growing ecosystem of frameworks tackling different aspects of this problem. They each bring real contributions worth understanding.

Superpowers focuses on development methodology. It uses subagent-driven development, TDD enforcement, and multi-stage review. The framework generates a design spec, then an implementation plan, then dispatches subagents to execute. Review subagents check the output, and if they find issues, the implementer revises and gets re-reviewed until approved.

Get Shit Done (GSD) focuses on context engineering. Its key insight is fighting context window degradation through fresh 200k subagent contexts, parallel wave execution, and XML-structured plans. A JavaScript CLI handles the deterministic work (tracking progress, dependency ordering, context budgets) so the LLM never wastes tokens on bookkeeping it would do unreliably anyway.

Both frameworks share a crucial design decision: fresh context windows. When an agent has been reasoning for 100k tokens, its attention degrades. By spawning subagents with clean 200k contexts, these frameworks sidestep the "context rot" problem that plagues long-running agent sessions.

Where these frameworks diverge is in how they handle quality assurance. GSD relies on mechanical verification: lint, test, type-check, and auto-fix retries if the checks fail. There is no agent reading another agent's code to assess whether it matches the spec's intent. The "review" is whether npm run test passes.

Superpowers does have agent-to-agent review with iterative loops. But the review is enforced by in-context instructions, which means the agent can (and frequently does) rationalize skipping the review step to save tokens.

This is a known issue in the project. When review enforcement lives inside the same prompt that the model is also using to make efficiency decisions, the model sometimes decides that review is not worth the cost.

The adversarial GAN pattern addresses this differently. Instead of asking an agent to review its own work or trusting in-context instructions to enforce review, it structures the pipeline so that review is architecturally mandatory. The reviewer is a separate agent that cannot be skipped, because the orchestrator will not advance the pipeline without the reviewer's signal. The reviewer cannot modify source code, only feedback.md. The generator cannot approve its own output. Role separation is enforced by the system, not suggested by the prompt.

The GAN Pattern Applied to Code

In machine learning, GANs pit two networks against each other: a generator creates content, a discriminator evaluates it, and the feedback loop between them drives both to improve. The generator gets better at producing realistic output. The discriminator gets better at finding flaws. The adversarial tension is what produces quality.

Applied to software development, this creates two stacked feedback loops:

Each role runs as a separate agent with its own fresh context window. The Plan Reviewer has never seen the Planner's reasoning process. It only sees the output. The Code Reviewer has never seen the Implementer's struggles. It only sees the code.

This separation fundamentally changes what the reviewer can catch. When a reviewer shares context with the generator, it inherits the generator's blind spots. When a reviewer starts fresh, it reads the plan the way an actual engineer would: with no assumptions about what the author "meant" versus what they wrote.

The adversarial Plan Reviewer doesn't just verify structure. It actively tries to break the plan:

• Deadlock search: Is there a task ordering that would deadlock the implementer? (Task 3 needs the output of Task 5.)

• False positive verification: Could any verification checklist pass even with a wrong implementation?

• Ambiguity search: Are there instructions that could be interpreted two valid ways?

• Missing context: Could the implementer get stuck because a task assumes knowledge not provided?

This is where the GAN analogy is most literal. The discriminator isn't checking if the plan looks good. It's trying to find failure modes.

Why Rhetorical Questions Outperform Direct Instructions

When a reviewer finds an issue, there are two ways to communicate it.

Direct instruction:

Fix line 45: the error handler returns 500 instead of 401 for invalid tokens.

Rhetorical question:

Consider: The test test_invalid_token_rejection expects a 401 status code.
Are you returning the correct HTTP status in your error handling?

Think about: In src/auth/middleware.js:45, what happens when the token is
invalid? Is the error properly caught?

Reflect: Look at how other middleware handles auth errors. Are you following
the same pattern?

The direct instruction produces a mechanical edit. The agent changes line 45 and moves on. The rhetorical question produces a deeper investigation. The agent re-examines the surrounding code, considers the pattern used elsewhere, and is more likely to find the root cause rather than just patching the symptom.

This maps to how the underlying models work. When given an explicit instruction, the model follows it literally. When guided to reason about a problem, it activates a broader search through its understanding of the codebase. The fix addresses related issues that a mechanical edit would miss.

Reviewer prompts structured around "Consider," "Think about," and "Reflect" prefixes consistently produce better fixes than "Fix" or "Change" directives. The implementer agent receives these as feedback in feedback.md and addresses them in the next iteration of the GAN loop.

Feedback as Filesystem

Most agent orchestration systems rely on some form of message passing: API calls, databases, queue systems, in-memory state. These all work, but they introduce infrastructure dependencies and make the agent conversation opaque after the fact.

An alternative: use the filesystem as the message bus and git as the orchestration layer.

All agent communication flows through feedback.md, a structured markdown file with two sections:

## Active Feedback (OPEN)

### FB-001: Auth middleware missing rate limiting
- **Status:** OPEN
- **Source:** Plan Reviewer
- **Phase:** 1
- **Detail:** The plan specifies JWT validation but does not address rate
  limiting for failed auth attempts. Consider: what happens if an attacker
  brute-forces tokens?

## Resolved Feedback

### FB-000: Missing error codes in API spec
- **Status:** RESOLVED
- **Resolution:** Added error code table to Phase-0 conventions

This design has several properties that matter in practice:

Full audit trail: Every piece of feedback, every resolution, every signal is committed to git alongside the code it produced. When you want to understand why the auth middleware was designed a certain way, the conversation that shaped it is right there in the commit history.

State recovery: If a pipeline gets interrupted (token limits, network issues, you need to step away), resuming is trivial. The orchestrator re-reads feedback.md and git log, determines what stage the pipeline reached, and picks up where it left off. No cloud infrastructure, no database, no queue. Just files.

Transparency: You can read the agent conversation in your editor. You can see exactly what the reviewer flagged, exactly how the implementer responded, and whether the resolution actually addressed the concern.

Agents communicate through structured signals routed by the orchestrator:

• PLAN_COMPLETE / REVISION_REQUIRED / PLAN_APPROVED (plan GAN loop)

• IMPLEMENTATION_COMPLETE / CHANGES_REQUESTED / PHASE_APPROVED (code GAN loop)

• GO / NO-GO (final gate)

• VERIFIED / UNVERIFIED (post-remediation verification)

Each signal marks a state transition. The orchestrator reads the signal, determines the next agent to invoke, and passes it the relevant context. The orchestrator itself is a Claude Code session, but the agents it spawns are fresh subagents with clean context windows.

The Zero-Context Engineer

One of the most effective constraints in the system is the "zero-context engineer" framing. The Planner writes every plan as if it will be executed by an engineer who:

• Is skilled but has zero context on the codebase

• Is unfamiliar with the toolset and problem domain

• Will follow instructions precisely

• Will not infer missing details. If it's not in the plan, it won't happen.

This constraint forces explicit instructions. No "add the usual auth middleware." Instead: which library, which pattern, which error codes, which files to create, which existing files to modify, and how to verify the result.

The Plan Reviewer then simulates this zero-context experience: "If I knew nothing about this codebase, could I follow these instructions and produce a working result?"

This framing catches a class of failures that are invisible to someone with context. The author of the plan knows what they meant. The zero-context reviewer only knows what is written. The gap between intention and specification is where bugs live.

Phase-0: Immutable Conventions

Every pipeline run starts with a Phase-0 document that defines immutable rules: tech stack, testing strategy, deployment approach, shared patterns, commit format. Every subsequent phase inherits from Phase-0. Every reviewer checks against it.

This solves a common multi-agent problem: drift. Without a shared source of truth, Agent A might decide to use Jest while Agent B sets up Vitest. Agent C might use a different error handling pattern than Agent D. Phase-0 prevents this by establishing conventions before any code is written.

The conventions aren't suggestions. They're constraints that every agent in the pipeline must respect, and every reviewer must verify against.

Convergence Design: Knowing When to Stop

An adversarial loop without exit conditions is just two agents arguing forever. The convergence design has three mechanisms:

Iteration caps: Each GAN loop (plan review, code review) runs a maximum of 3 iterations. If the planner and reviewer cannot converge in 3 rounds, the issue requires human judgment, not more machine cycles.

Signal protocol: The structured signals (PLAN_APPROVED, GO, NO-GO) are explicit state transitions, not suggestions. When the final reviewer issues NO-GO, the pipeline rolls back the phase. There is no "let's try one more time." The rollback is automatic.

Token budget: Each phase targets roughly 50k tokens with a 75k hard ceiling. This prevents any single phase from consuming the entire context budget and ensures the orchestrator retains enough headroom to manage the pipeline.

These caps exist because adversarial loops have a cost curve. The first iteration catches major issues. The second iteration catches subtle issues. The third iteration catches edge cases. A fourth iteration almost never catches anything the previous three missed, but it costs just as many tokens. Three iterations hit the sweet spot between thoroughness and efficiency.

Ground Truth Documents and the Pipeline

The adversarial pipeline doesn't start from a vague prompt. Every workflow begins with an intake skill that produces a structured ground truth document. The pipeline then runs from that document, not from the original user request.

Brainstorm: Turning Ideas into Specs

The /brainstorm skill is the feature creation workflow. Given a feature idea, it first explores the codebase to understand the existing architecture, tech stack, and patterns. Then it asks 5-15 clarifying questions designed to front-load high-impact decisions:

The codebase uses DynamoDB for storage. For this feature's data, should we:

A) Add tables to the existing DynamoDB setup
B) Use a different storage approach (e.g., S3 for documents)
C) Both - DynamoDB for metadata, S3 for content

These aren't generic questions. They're grounded in what the skill found during codebase exploration. The skill identifies the real decision points for this specific project and surfaces them before any planning or code generation begins.

The output is brainstorm.md, a structured design spec. Not a conversation transcript, but a distilled set of decisions that the Planner agent can consume cold. This document becomes the single source of truth for the entire pipeline run.

Repository Evaluation, Health, and Documentation Audits

The same ground-truth-document pattern applies to the audit workflows:

• /repo-eval spawns three evaluator agents in parallel (the Pragmatist, the Oncall Engineer, the Team Lead), each scoring the codebase from a different lens across 12 pillars. The output is eval.md.

• /repo-health runs a technical debt auditor across four vectors (architectural, structural, operational, hygiene). The output is health-audit.md.

• /doc-health runs six detection phases comparing documentation against actual code. The output is doc-audit.md.

• /audit runs any combination of the above. It asks scoping questions once, then spawns up to 5 agents in parallel (3 evaluators + health auditor + doc auditor). All intake documents land in one directory.

Each of these intake skills produces a read-only assessment. The agents doing the evaluation never modify the codebase. They only write their findings into the intake document.

The Pipeline Runs from Ground Truth

The /pipeline skill reads whatever intake documents exist and runs the adversarial GAN loop from them. For a feature, it reads brainstorm.md. For an audit, it reads whichever combination of eval.md, health-audit.md, and doc-audit.md are present.

When multiple intake documents exist (from a combined audit), the Planner reads all findings together and consolidates overlapping concerns into a single unified plan. Phases are tagged by implementer type and ordered:

1. [HYGIENIST] phases first, subtractive cleanup (deleting dead code, simplifying over-abstractions)

2. [IMPLEMENTER] phases next, structural fixes on clean code

3. [FORTIFIER] phases next, locking in the clean state (linting, CI checks, git hooks)

4. [DOC-ENGINEER] phases last, documentation reflecting final code

The ordering matters. You don't want the implementer building on top of dead code that the hygienist would have removed. You don't want the doc-engineer documenting an API that the fortifier is about to add validation to.

This separation between intake and pipeline is deliberate. The intake skills are exploratory and interactive. They ask questions, explore the codebase, and produce a document. The pipeline is autonomous. It reads the document and runs through the adversarial loops with minimal human intervention, stopping only at explicit decision points.

What the Adversarial Loop Actually Catches

In practice, the adversarial loops catch issues that single-pass generation consistently misses.

Plan Review catches:

• Hallucinated file paths (the Planner says "modify" a file that doesn't exist)

• Phantom dependencies (Phase 2 assumes a model that Phase 1 never creates)

• Test strategies that require live cloud resources instead of mocks

• Ambiguous instructions that a zero-context engineer could misinterpret

• Deadlocks in task ordering (Task 3 needs the output of Task 5)

Code Review catches:

• Placeholder tests (expect(true).toBe(true))

• Deviations from Phase-0 architecture conventions

• Missing error path coverage (only happy paths tested)

• Hardcoded secrets and input validation gaps

Verification catches:

• Remediation targets that weren't actually addressed

• Regressions introduced during fixes

• Partial fixes where the symptom changed but the root cause remains

An earlier design re-ran the full evaluator or auditor agents after remediation, 3-5 agents re-scanning the entire codebase. This was token-expensive and redundant since the per-phase reviewers had already verified each fix. The current design uses a single verification agent with a targeted scope: read the original intake document findings and check each specific file:line location. One agent, targeted scope, a fraction of the tokens. Evaluator and auditor agents run exactly once (during intake) and never again.

Honest Trade-offs

This pipeline is not free. There are some trade-offs you'll want to consider and be aware of:

Token Cost

Multiple agents reviewing each other's work uses significantly more tokens than a single-pass approach. The adversarial loops can triple the total token usage for a feature. On a subscription plan, this means hitting session limits faster. On API billing, this means real money.

Time

A feature that takes one agent 10 minutes might take the pipeline 30-45 minutes with review loops. Multi-agent frameworks in general are slower than single-pass. The adversarial loops add time on top of the orchestration overhead that any multi-agent system carries.

Orchestrator Context Pressure

The orchestrator accumulates agent result summaries across phases. Long pipelines with many phases may hit context compression, which degrades the orchestrator's ability to route effectively.

Not Fire-and-Forget

Despite the automation, complex features benefit from human checkpoints. The pipeline stops and asks for judgment at key moments. If you skip those checkpoints, you may end up with technically correct code that misses the actual requirement.

Diminishing Returns on Simple Tasks

For a quick script, a utility function, or a prototype, the adversarial overhead is pure waste. Single-pass generation is faster, cheaper, and sufficient.

The trade-off is worth it for features where correctness matters more than speed: anything touching auth, payments, data integrity, or infrastructure. When the cost of a bug in production exceeds the cost of the extra tokens to prevent it, the math works. For everything else, single-pass is fine.

When to Use This (And When Not To)

Use adversarial multi-agent patterns when:

• The feature touches authentication, authorization, or session management

• The code handles payments or financial transactions

• Data integrity is critical (migrations, schema changes, ETL pipelines)

• Infrastructure changes could affect production (IaC, CI/CD modifications)

• The codebase is unfamiliar to the agents (large legacy systems)

Use single-pass generation when:

• Prototyping or exploring an idea

• Writing utility scripts or one-off tools

• Making small, well-scoped changes to familiar code

• Speed matters more than thoroughness

• You will review the output carefully yourself anyway

Getting Started

Claude Forge is built entirely from Claude Code custom skills. No external tooling, no CI integration required. Install by copying the skills directory into your project:

git clone https://github.com/hatmanstack/claude-forge.git
cp -r claude-forge/.claude/skills/ /path/to/your-project/.claude/skills/

Then in your project:

# Feature development
/brainstorm I want to add webhook support for payment events
/pipeline 2026-03-12-payment-webhooks

# Full audit (health + eval + docs), one command
/audit all
/pipeline 2026-03-16-audit-remediation

# Individual audits
/repo-eval
/repo-health
/doc-health

The pipeline handles the orchestration. You'll see progress reports between stages, and it will stop and ask when something needs human judgment.

Wrapping Up

The adversarial pattern (separate generator and discriminator with isolated context windows, structured feedback as the communication channel, iteration caps for convergence) can be implemented in any agent system that supports subagent spawning with fresh contexts. The specific implementation uses Claude Code skills, but the pattern is the contribution, not the tooling.

Sometimes the best code comes from the argument, not the agreement.

In this tutorial, we will cover:

1. What is SonarQube?

2. How SonarQube Improves Code Quality

3. Step-by-step Installation and Configuration

4. How to Run Your First Code Analysis

What is SonarQube?

SonarQube is an open-source tool that checks for code quality continuously. It analyzes code to find issues like duplication, bad practices, test coverage gaps, bugs, and vulnerabilities, giving detailed reports. It works with many programming languages like Java, C#, JavaScript, Python, TypeScript, and Kotlin.

You can add SonarQube to your CI/CD pipelines, IDEs, and version control systems like GitHub, GitLab, or Bitbucket. It provides detailed dashboards that show metrics, trends, and issues in your code.

You can use custom rules to enforce coding standards and reduce technical debt. SonarQube also supports code coverage analysis to help teams improve their tests. With the Quality Gate feature, teams can ensure only clean, maintainable code goes into production.

SonarQube offers both free and paid versions to suit any team size. Overall, it helps improve software quality and encourages good coding practices.

How Does SonarQube Improve Code Quality?

Here’s how SonarQube helps improve code quality:

1. Early bug detection: Identifies bugs before they reach production

2. Improved maintainability: Highlights code and design issues

3. Security insights: Identifies vulnerabilities and security risks

4. Code coverage: Integration with testing tools to monitor unit test coverage

5. Customizable rules: Allows teams to set coding standards and policies

6. Team collaboration: Ensures consistent code quality across development teams

Step-by-Step Installation and Configuration

Prerequisites:

Here are the prerequisites that you will need before installing SonarQube

1. Java Runtime Environment(JRE): Java 11 or above installed in your system.

2. System Requirements: 2GB RAM minimum (Recommended: 4GB+).

3. MacOS: You can use HomeBrew, which is the package manager for MacOS that simplifies the installation of software.

Below are the steps to install SonarQube in your local machine:

Download SonarQube

Download the software from sonarsource downloads and choose the Community Edition for open-source projects.

Extract and Configure

To install SonarQube, you need to run the below command to unzip the file:

unzip sonarqube-<version>.zip
cd sonarqube-<version>/bin/<your-OS-folder>

Start SonarQube

On Linux/Mac, you need to run the below command:

./sonar.sh start

On Windows, you need to run this one:

StartSonar.bat

Access SonarQube

To access SonarQube, you need to open browser and go to: http://localhost:9000

Enter the default credentials:

• Username: admin

• Password: admin (you’ll be prompted to change it)

The page will look similar to below:

Set Up SonarQube in Your Project

To set up SonarQube in your project, start by opening the Java project on your machine. In the project root, create a sonar-project.properties file.

Add the below key value pairs in the file:

sonar.projectKey=spring-myproject
sonar.projectName=My Project
sonar.projectVersion=1.0
sonar.sources=.
sonar.host.url=http://localhost:9000

How to Run Your First Code Analysis

Configure and Run SonarScanner

SonarScanner is the tool that actually sends your code to SonarQube for analysis. Below are the detailed steps to follow to use it:

Install SonarScanner:

On Windows/Linux, download the software from SonarSource and unzip it:

unzip sonar-scanner-cli-<version>.zip

On MacOS, run the below command:

>brew install sonar-scanner

For both Windows/Linux and MacOS, verify the install by running the below command:

>sonar-scanner -v

Configure SonarScanner

After installing SonarScanner, you’ll need to configure it by setting the SonarQube server URL and authentication token. Then go to your SonarQube profile (top-right corner > My Account > Security) and generate a token.

Provide a name for the token and click ‘Generate’:

In the sonar-project.properties file in your project, add ‘sonar.login’ property and save.

sonar.projectKey=test-project
sonar.projectName=Test Project
sonar.host.url=http://localhost:9000
sonar.login=<YOUR_TOKEN_HERE>

Run the Analysis

Once the SonarScanner is configured, you can start scanning your project.

In a terminal or command prompt, go to the root of your project (where sonar-project.properties is located).

Run the following command:

>sonar-scanner

SonarScanner will analyze your code and push the results to your local SonarQube server. Visit http://localhost:9000, and you’ll see your project listed on the dashboard.

To view the analysis report, go to http://localhost:9000/dashboard?id=java-sonar-demo:

If you go to the ‘Issues’ tab at top left corner, you can view different categories of Software Quality, Severity of the Issues, and various other attributes in your code.

Conclusion

Now you have installed and configured SonarQube and learned how to scan your code using SonarScanner. You can easily configure it in your projects for continuous code quality analysis.

This is a fantastic tool for keeping your code base clean and maintainable. As the next steps, you can consider adding test coverage reports, enforcing quality gates in your pipeline, and exploring SonarCloud for cloud-based analysis.

A buffer overflow happens when a program writes more data into a buffer than it was allocated, leading to memory corruption, crashes, or even security vulnerabilities. A buffer corruption occurs when unintended modifications overwrite unread data or modify memory in unexpected ways.

In safety-critical systems like cars, medical devices, and spacecraft, buffer overflows can cause life-threatening failures. Unlike simple software bugs, buffer overflows are unpredictable and depend on the state of the system, making them difficult to diagnose and debug.

To prevent these issues, it's important to understand how buffer overflows and corruptions occur, and how to detect and fix them.

Article Scope

In this article, you will learn:

1. What buffers, buffer overflows, and corruptions are. I’ll give you a beginner-friendly explanation with real-world examples.

2. How to debug buffer overflows. You’ll learn how to use tools like GDB, LLDB, and memory maps to find memory corruption.

3. How to prevent buffer overflows. We’ll cover some best practices like input validation, safe memory handling, and defensive programming.

I’ll also show you some hands-on code examples – simple C programs that demonstrate buffer overflow issues and how to fix them.

What this article doesn’t cover:

1. Security exploits and hacking techniques. We’ll focus on preventing accidental overflows, not hacking-related buffer overflows.

2. Operating system-specific issues. This guide is for embedded systems, not general-purpose computers or servers.

3. Advanced RTOS memory management. While we discuss interrupt-driven overflows, we won’t dive deep into real-time operating system (RTOS) concepts.

Now that you know what this article covers (and what it doesn’t), let’s go over the skills that will help you get the most out of it.

Prerequisites

This article is designed for developers who have some experience with C programming and want to understand how to debug and prevent buffer overflows in embedded systems. Still, beginners can follow along, as I’ll explain key concepts in a clear and structured way.

Before reading, it helps if you know:

1. Basic C programming.

2. How memory works – the difference between stack, heap, and global variables.

3. Basic debugging concepts – if you’ve used a debugger like GDB or LLDB, that’s a plus, but not required.

4. What embedded systems are – a basic idea of how microcontrollers store and manage memory.

Even if you’re not familiar with these topics, this guide will walk you through them in an easy-to-understand way.

Before you dive into buffer overflows, debugging, and prevention, let’s take a step back and understand what a buffer is and why it’s important in embedded systems. Buffers play a crucial role in managing data flow between hardware and software but when handled incorrectly, they can lead to serious software failures.

Table of Contents

• What is a Buffer, and How Does it Work?

• What is a Buffer Overflow?

• Common Causes of Buffer Overflows and Corruption

• Consequences of Buffer Overflows

• How to Debug Buffer Overflows

• How to Prevent Buffer Overflows

• Conclusion

What is a Buffer, and How Does it Work?

A buffer is a contiguous block of memory used to temporarily store data before it is processed. Buffers are commonly used in two scenarios:

1. Data accumulation: When the system needs to collect a certain amount of data before processing.

2. Rate matching: When the data producer generates data faster than the data consumer can process it.

Buffers are typically implemented as arrays in C, where elements are indexed from 0 to N-1 (where N is the buffer size).

Let’s look at an example of a buffer in a sensor system.

Consider a system with a sensor task that generates data at 400 Hz (400 samples per second or 1 sample every 2.5 ms). But the data processor (consumer) operates at only 100 Hz (100 samples per second or 1 sample every 10 ms). Since the consumer task is slower than the producer, we need a buffer to store incoming data until it is processed.

To determine the buffer size, we calculate:

Buffer Size = Time to consume 1 sample / Time to generate 1 sample = 10 ms/ 2.5 ms = 4

This means the buffer must hold at least 4 samples at a time to avoid data loss.

Once the buffer reaches capacity, there are several strategies to decide which data gets passed to the consumer task:

1. Max/min sampling: Use the maximum or minimum value in the buffer.

2. Averaging: Compute the average of all values in the buffer.

3. Random access: Pick a sample from a specific location (for example, the most recent or the first).

In real-world applications, it’s beneficial to use circular buffers or double buffering to prevent data corruption.

• Circular buffer approach: A circular buffer (also called a ring buffer) continuously wraps around when it reaches the end, ensuring old data is overwritten safely without exceeding memory boundaries. The buffer size should be multiplied by 2 (4 × 2 = 8) to hold 8 samples. This allows the consumer task to process 4 samples while the next 4 samples are being filled, preventing data overwrites.

• Double buffer approach: Double buffering is useful when data loss is unacceptable. It allows continuous data capture while the processor is busy handling previous data. A second buffer of the same size is added. When the first buffer is full, the write pointer switches to the second buffer, allowing the consumer task to process data from the first buffer while the second buffer is being filled. This prevents data overwrites and ensures a continuous data flow.

Buffers help manage data efficiently, but what happens when they are mismanaged? This is where buffer overflows and corruptions come into play.

What is a Buffer Overflow?

A buffer overflow occurs when a program writes more data into a buffer than it was allocated, causing unintended memory corruption. This can lead to unpredictable behavior, ranging from minor bugs to critical system failures.

To understand buffer overflow, let's use a simple analogy. Imagine a jug with a tap near the bottom. The jug represents a buffer, while the tap controls how much liquid (data) is consumed.

The jug is designed to hold a fixed amount of liquid. As long as water flows into the jug at the same rate or slower than it flows out, everything works fine. But if water flows in faster than it flows out, the jug will eventually overflow.

Similarly, in software, if data enters a buffer faster than it is processed, it exceeds the allocated memory space, causing a buffer overflow. In the case of a circular buffer, this can cause the write pointer to wrap around and overwrite unread data, leading to buffer corruption.

Buffer Overflows in Software

Unlike the jug, where water simply spills over, a buffer overflow in software overwrites adjacent memory locations. This can cause a variety of hard-to-diagnose issues, including:

1. Corrupting other data stored nearby.

2. Altering program execution, leading to crashes.

3. Security vulnerabilities, where attackers exploit overflows to inject malicious code.

When a buffer overflow occurs, data can overwrite variables, function pointers, or even return addresses, depending on where the buffer is allocated.

Buffer overflows can occur in different memory regions:

1. Buffer overflows in global/static memory (.bss / .data sections)

   • These occur when global or static variables exceed their allocated size.

   • The overflow can corrupt adjacent variables, leading to unexpected behavior in other modules.

   • Debugging is easier because memory addresses are fixed at compile time unless the compiler optimizes them. Map files provide a memory layout of variables during the compilation and linking.

2. Stack-based buffer overflow (more predictable, easier to debug):

   • Happens when a buffer is allocated in the stack (for example, local variables inside functions).

   • Overflowing the stack can affect adjacent local variables or return addresses, potentially crashing the program.

   • In embedded systems with small stack sizes, this often leads to a crash or execution of unintended code.

3. Heap-based buffer overflow (harder to debug):

   • Happens when a buffer is dynamically allocated in the heap (for example, using malloc() in C).

   • Overflowing a heap buffer can corrupt adjacent dynamically allocated objects or heap management structures.

   • Debugging is harder because heap memory is allocated dynamically at runtime, causing memory locations to vary.

Buffer Overflow vs Buffer Corruption

Buffer overflow and buffer corruption are of course related, but refer to different situations.

A buffer overflow happens when data is written beyond the allocated buffer size, leading to memory corruption, unpredictable behavior, or system crashes.

A buffer corruption happens when unintended data modifications result in unexpected software failures, even if the write remains within buffer boundaries.

Both issues typically result from poor write pointer management, lack of boundary checks, and unexpected system behavior.

Now that we've covered what a buffer overflow is and how it can overwrite memory, let’s take a closer look at how these issues affect embedded systems.

In the next section, we’ll explore how buffer overflows and corruption happen in real-world embedded systems and break down common causes, including pointer mismanagement and boundary violations.

Common Causes of Buffer Overflows and Corruption

Embedded systems use buffers to store data from sensors, communication interfaces (like UART (Universal Asynchronous Receiver-Transmitter), SPI (Serial Peripheral Interface), I2C (Inter-integrated Circuit), and real-time tasks. These buffers are often statically allocated to avoid memory fragmentation, and many implementations use circular (ring) buffers to efficiently handle continuous data streams.

Here are three common scenarios where buffer overflows or corruptions occur in embedded systems:

Writing Data Larger Than the Available Space

Issue: The software writes incoming data to the buffer without checking if there is enough space.

Example: Imagine a 100-byte buffer to store sensor data. The buffer receives variable-sized packets. If an incoming packet is larger than the remaining space, it will overwrite adjacent memory, leading to corruption.

So why does this happen?

• Some embedded designs increment the write pointer after copying data, making it too late to prevent overflow.

• Many low-level memory functions (memcpy, strcpy, etc.) do not check buffer boundaries, leading to unintended writes.

• Without proper bound checking, a large write can exceed the buffer size and corrupt nearby memory.

Here’s a code sample to demonstrate buffer overflow in a .bss / .data section:

  #include <stdint.h>
  #include <stdio.h>
  #include <string.h>

  #define BUFFER_SIZE 300

  static uint16_t sample_count = 0;
  static uint8_t buffer[BUFFER_SIZE] = {0};

  // Function to simulate a buffer overflow scenario
  void updateBufferWithData(uint8_t *data, uint16_t size)
  {
      // Simulating a buffer overflow: No boundary check!
      printf("Attempting to write %d bytes at position %d...\n", size, sample_count);

      // Deliberate buffer overflow for demonstration
      if (sample_count + size > BUFFER_SIZE)
      {
          printf("WARNING: Buffer Overflow Occurred! Writing beyond allocated memory!\n");
      }

      // Copy data (unsafe, can cause overflow)
      memcpy(&buffer[sample_count], data, size);

      // Increment sample count (incorrectly, leading to wraparound issues)
      sample_count += size;
  }

  int main()
  {   
      // Save 1 byte to buffer
      uint8_t data_to_buffer = 10;
      updateBufferWithData(&data_to_buffer, 1);

      // Save an array of 20 bytes to buffer
      uint8_t data_to_buffer_1[20] = {5};
      updateBufferWithData(data_to_buffer_1, sizeof(data_to_buffer_1));

      // Intentional buffer overflow: Save an array of 50 x 8 bytes (400 bytes)
      uint64_t data_to_buffer_2[50] = {7};
      updateBufferWithData((uint8_t*)data_to_buffer_2, sizeof(data_to_buffer_2));

      return 0;
  }

Interrupt-Driven Overflows (Real-time Systems)

Issue: The interrupt service routine (ISR) may write data faster than the main task can process, leading to buffer corruption or buffer overflow if the write pointer is not properly managed.

Example: Imagine a sensor ISR that writes incoming data into a buffer every time a new reading arrives. Meanwhile, a low-priority processing task reads and processes the data.

What can go wrong?

• If the ISR triggers too frequently (due to a misbehaving sensor or high interrupt priority), the buffer may fill up faster than the processing task can keep up.

• This can result in one of two failures:

  1. Buffer Corruption: The ISR overwrites unread data, leading to loss of information.

  2. Buffer Overflow: The ISR exceeds buffer boundaries, causing memory corruption or system crashes.

So why does this happen?

• In real-time embedded systems, ISR execution preempts lower-priority tasks.

• If the processing task doesn't not get enough CPU time, the buffer may become overwritten or overflow beyond its allocated scope.

System State Changes & Buffer Corruption

Issue: The system may unexpectedly reset, enter low-power mode, or changes operating state, leaving the buffer write pointers in an inconsistent state. This can result in buffer corruption (stale or incorrect data) or buffer overflow (writing past the buffer’s limits.

Example Scenarios:

1. Low-power wake-up issue (Buffer Overflow risk): Some embedded systems enter deep sleep to conserve energy. Upon waking up, if the buffer write pointer is not correctly reinitialized, it may point outside buffer boundaries, leading to buffer overflow and unintended memory corruption.

2. Unexpected mode transitions: If a sensor task is writing data and the system suddenly switches modes, the buffer states and pointers may not be cleaned up. The next time the sensor task runs, it may continue writing without clearing previous data. This can cause undefined behavior due to presence of stale data.

Now that you understand how buffer overflows and corruptions happen, let’s examine their consequences in embedded systems ranging from incorrect sensor readings to complete system failures, making debugging and prevention critical.

Consequences of Buffer Overflows

Buffer overflows can be catastrophic in embedded systems, leading to system crashes, data corruption, and unpredictable behavior. Unlike general-purpose computers, many embedded devices lack memory protection, making them particularly vulnerable to buffer overflows.

A buffer overflow can corrupt two critical types of memory:

1. Data Variables Corruption

A buffer overflow can overwrite data variables, corrupting the inputs for other software modules. This can cause unexpected behavior or even system crashes if critical parameters are modified.

For example, a buffer overflow could accidentally overwrite a sensor calibration value stored in memory. As a result, the system would start using incorrect sensor readings, leading to faulty operation and potentially unsafe conditions.

2. Function Pointer Corruption

In embedded systems, function pointers are often used for interrupt handlers, callback functions, and RTOS task scheduling. If a buffer overflow corrupts a function pointer, the system may execute unintended instructions, leading to a crash or unexpected behavior.

As an example, a function pointer controlling motor speed regulation could be overwritten. Instead of executing the correct function, the system would jump to a random memory address, causing a system fault or erratic motor behavior.

Buffer overflows are among the hardest bugs to identify and fix because their effects depend on which data is corrupted and the values it contains. A buffer overflow can affect memory in different ways:

• If a buffer overflow corrupts unused memory, the system may seem fine during testing, making the issue harder to detect.

• if a buffer overflow alters critical data variables, it can cause hidden logic errors that cause unpredictable behavior.

• If a buffer overflow corrupts function pointers, it may crash immediately, making the problem easier to identify.

During development, if tests focus only on detecting crashes, they may overlook silent memory corruption caused by a buffer overflow. In real-world deployments, new use cases not covered in testing can trigger previously undetected buffer overflow issues, leading to unpredictable failures.

Buffer overflows can cause a chain reaction, where one overflow leads to another overflow or buffer corruption, resulting in widespread system failures. So how does this happen?

1. A buffer overflow corrupts a critical variable (for example, a timer interval).

2. The corrupted variable disrupts another module (for example, triggers the timer interrupt too frequently, causing it to push more data into a buffer than intended.).

3. This increased interrupt frequency forces a sensor task to write data faster than intended, eventually causing another buffer overflow or corruption by overwriting unread data.

This chain reaction can spread across multiple software modules, making debugging nearly impossible. In real-word applications, buffer overflows in embedded systems can be life-threatening:

• In cars: A buffer overflow in an ECU (Electronic Control Unit) could cause brake failure or unintended acceleration.

• In a spacecraft: A memory corruption issue could disable navigation systems, leading to mission failure.

Now that we’ve seen how buffer overflows can corrupt memory, disrupt system behavior, and even cause critical failures, the next step is understanding how to detect and fix them before they lead to serious issues.

How to Debug Buffer Overflows

Debugging buffer overflows in embedded systems can be complex, as their effects range from immediate crashes to silent data corruption, making them difficult to trace. A buffer overflow can cause either:

1. A system crash, which is easier to detect since it halts execution or forces a system reboot.

2. Unexpected behavior, which is much harder to debug as it requires tracing how corrupted data affects different modules.

This section focuses on embedded system debugging techniques using memory map files, debuggers (GDB/LLDB), and a structured debugging approach. Let’s look into the debuggers and memory map files.

Memory Map File (.map file)

A memory map file is generated during the linking process. It provides a memory layout of global/static variables, function addresses, and heap/stack locations. It provides a memory layout of Flash and RAM, including:

• Text section (.text): Stores executable code.

• Read-only section (.rodata): Stores constants and string literals.

• BSS section (.bss): Stores uninitialized global and static variables.

• Data section (.data): Stores initialized global and static variables.

• Heap and stack locations, depending on the linker script.

If a buffer overflow corrupts a global variable, the .map file can identify nearby variables that may also be affected, provided the compiler has not optimized the memory allocation. Similarly, if a function pointer is corrupted, the .map file can reveal where it was stored in memory.

Debuggers (GDB & LLDB)

Debugging tools like GDB (GNU Debugger) and LLDB (LLVM Debugger) allow:

• Controlling execution (breakpoints, stepping through code).

• Inspecting variable values and memory addresses.

• Getting backtraces (viewing function calls before a crash).

• Extracting core dumps from microcontrollers for post-mortem analysis.

If the system halts on a crash, a backtrace (bt command in GDB) can reveal which function was executing before failure. If the overflow affects a heap-allocated variable, GDB can inspect heap memory usage to detect corruption.

The Debugging Process

Now, let’s go through a step-by-step debugging process to identify and fix buffer overflows. Once a crash or unexpected behavior occurs, follow these techniques to trace the root cause:

Step 1: Identify the misbehaving module

If the system crashes, use GDB or LLDB backtrace (bt command) to locate the last executed function. If the system behaves unexpectedly, determine which software module controls the affected functionality.

Step 2: Analyze inputs and outputs of the module

Every function or module has inputs and outputs. Create a truth table listing expected outputs for all possible inputs. Check if the unexpected behavior matches any undefined input combination, which may indicate corruption.

Step 3: Locate memory corruption using address analysis

If a variable shows incorrect values, determine its physical memory location. Depending on where the variable is stored:

1. Global/static variables (.bss / .data): Look up the memory map file for nearby buffers.

2. Heap variables: Snapshot heap allocations using GDB.

   Here’s an example of using GDB to find corrupted variables:

    (gdb) print &my_variable  # Get memory address of the variable
    $1 = (int *) 0x20001000
    (gdb) x/10x 0x20001000   # Examine memory near this address, Display 10 memory words in hexadecimal format starting from 0x20001000
   

Step 4: Identify the overflowing buffer

If a buffer is located just before the corrupted variable, inspect its usage in the code. Review all possible code paths that write to the buffer. Check if any design limitations could cause an overflow under a specific use cases.

Step 5: Fix the root cause

If the buffer overflow happened due to missing bounds checks, add proper input validation to prevent it. Buffer design should enforce strict memory limits. The module should implement strict boundary checks for all inputs and maintain a consistent state.

In addition to GDB/LLDB, you can also use techniques like hardware tracing and fault injection to simulate buffer overflows and observe system behavior in real-time.

While debugging helps identify and fix buffer overflows, prevention is always the best approach. Let’s explore techniques that can help avoid buffer overflows altogether.

How to Prevent Buffer Overflows

You can often prevent buffer overflows through good software design, defensive programming, hardware protections, and rigorous testing. Embedded systems, unlike general-purpose computers, often lack memory protection mechanisms, which means that buffer overflow prevention critical for system reliability and security.

Here are some key techniques to help prevent buffer overflows:

Defensive Programming

Defensive programming helps minimize buffer overflow risks by ensuring all inputs are validated and unexpected conditions are handled safely.

First, it’s crucial to validate input size before writing to a buffer. Always check the write index by adding the size of data to be written prior to writing data to make sure more data is not written than the available buffer space.

Then you’ll want to make sure you have proper error handling and fail-safe mechanisms in place. If an input is invalid, halt execution, log the error, or switch to a safe state. Also, functions should indicate success/failure with helpful error codes to prevent misuse.

Sample Code:

   #include <stdint.h>
   #include <string.h>
   #include <stdbool.h>
   #include <stdio.h>

   #define BUFFER_SIZE 300

   static uint16_t sample_count = 0;
   static uint8_t buffer[BUFFER_SIZE] = {0};

   typedef enum
   {
       SUCCESS = 0,
       NOT_ENOUGH_SPACE = 1,
       DATA_IS_INVALID = 2,
   } buffer_err_code_e;


   buffer_err_code_e updateBufferWithData(uint8_t *data, uint16_t size)
   {
       if (data == NULL || size == 0 || size > BUFFER_SIZE)  
       {
           return DATA_IS_INVALID; // Invalid input size
       }

       uint16_t available_space = BUFFER_SIZE - sample_count;
       bool can_write = (available_space >= size) ? true : false;

       if (!can_write)  
       {
           return NOT_ENOUGH_SPACE;
       }

       // Copy data safely
       memcpy(&buffer[sample_count], data, size);
       sample_count += size;

       return SUCCESS;
   }

   int main()
   {   
       buffer_err_code_e ret;

       // Save 1 byte to buffer
       uint8_t data_to_buffer = 10;
       ret = updateBufferWithData(&data_to_buffer, sizeof(data_to_buffer));
       if (ret)  
       {
           printf("Buffer update didn't succeed, Err:%d\n", ret);
       }

       // Save an array of 20 bytes to buffer
       uint8_t data_to_buffer_1[20] = {5};
       ret = updateBufferWithData(data_to_buffer_1, sizeof(data_to_buffer_1));
       if (ret)  
       {
           printf("Buffer update didn't succeed, Err:%d\n", ret);
       }

       // Save an array of 50 x 8 bytes, Intentional buffer overflow
       uint64_t data_to_buffer_2[50] = {7};
       ret = updateBufferWithData((uint8_t*)data_to_buffer_2, sizeof(data_to_buffer_2));  
       if (ret)  
       {
           printf("Buffer update didn't succeed, Err:%d\n", ret);
       }

       return 0;
   }

Choosing the Right Buffer Design And Size

Some buffer designs handle overflow better than others. Choosing the correct buffer type and size for the application reduces the risk of corruption.

• Circular Buffers (Ring Buffers) prevent out-of-bounds writes by wrapping around. They overwrite the oldest data instead of corrupting memory. These are useful for real-time streaming data (for example, UART, sensor readings). This approach is ideal for applications where data loss is unacceptable.

• Ping-Pong Buffers (Double Buffers) use two buffers. One buffer fills up with data. Then, once it’s full, it switches to the second buffer while the first one is processed. This approach is beneficial for application that have strict requirements on no data loss. The buffer design should be based on the speed of write and read tasks.

Hardware Protection

Memory Protection Unit (MPU)

An MPU (Memory Protection Unit) helps detect unauthorized memory accesses, including buffer overflows, by restricting which regions of memory can be written to. It prevents buffer overflows from modifying critical memory regions and triggers a MemManage Fault if a process attemps to write outside an allowed region.

But keep in mind that, an MPU does not prevent buffer overflows – it only detects and stops execution when they occur. Not all microcontrollers have an MPU, and some low-end MCUs lack hardware protection, making software-based safeguards even more critical.

Modern C compilers provide several flags to identify memory errors at compile-time:

1. -Wall -Wextra: Enables useful warnings

2. -Warray-bounds: Detects out-of-bounds array access when the array size is known at compile-time

3. -Wstringop-overflow: Warns about possible overflows in string functions like memcpy and strcpy.

Testing and Validation

Testing helps detect buffer overflows before deployment, reducing the risk of field failures. Unit testing each function independently with valid inputs, boundary cases, and invalid inputs helps detect buffer-related issues early. Automated testing involves feeding random and invalid inputs into the system to uncover crashes and unexpected behavior. Static Analysis Tools like Coverity, Clang Static Analyzer help detect buffer overflows before runtime. Run real-world inputs on embedded hardware to detect issues.

Now that we've explored how to identify, debug, and prevent buffer overflows, it’s clear that these vulnerabilities pose a significant threat to embedded systems. From silent data corruption to catastrophic system failures, the consequences can be severe.

But with the right debugging tools, systematic analysis, and preventive techniques, you can effectively either prevent or mitigate buffer overflows in your systems.

Conclusion

Buffer overflows and corruption are major challenges in embedded systems, leading to crashes, unpredictable behavior, and security risks. Debugging these issues is difficult because their symptoms vary based on system state, requiring systematic analysis using memory map files, GDB/LLDB, and structured debugging approaches.

In this article, we explored:

• The causes and consequences of buffer overflows and corruptions

• How to debug buffer overflows using memory analysis and debugging tools

• Best practices for prevention

Buffer overflow prevention requires a multi-layered approach:

1. Follow a structured software design process to identify risks early.

2. Apply defensive programming principles to validate inputs and handle errors gracefully.

3. Use hardware-based protections like MPUs where available.

4. Enable compiler flags that help identify memory errors.

5. Test extensively, unit testing, automated testing, and code reviews help catch vulnerabilities early.

By implementing these best practices, you can minimize the risk of buffer overflows in embedded systems, improving reliability and security.

In embedded systems, where reliability and safety are critical, preventing buffer overflows is not just a best practice, it is a necessity. A single buffer overflow can compromise an entire system. Defensive programming, rigorous testing, and hardware protections are essential for building secure and robust embedded applications.

ReactJS is one of the most popular JavaScript libraries for building scalable and performant applications.

When you're working on ReactJS projects, whether they're large or small, you'll need to focus on code quality, readability, maintainable, and scalability.

Writing good code will also help you reduce PR comments from your teammates (and let's be honest – who doesn't like comments like LGTM :) ).

In this tutorial, you will learn how to improve your React code. I'll share my favorite tips along with code examples to show you how everything works. These will help you write maintainable, scalable, and readable code.

You should have basic familiarity with React to get the most out of this guide.

1. Use Constants

In JavaScript we can declare constants by using the const keyword. This helps us avoid re-declaring the same value. So constants are a great choice for storing API keys and other values like that.

Constants improve the scalability, readability, and internationalization of any React codebase.

In every ReactJS project, you should avoid hard-coding strings (content) in your components.

This will make your project more maintainable, scalable, and readable, as it isolates the UI, data, and content layers from each other.

Constants include:

• API keys
• URLs
• Content

Many websites support multiple languages, like English, French, Spanish, and so on. This is known as internationalization (or i18n for short).

If you're enabling i18n features on your site, you should create separate constant files for your content – for example en.js and fr.js. Even if you don't have multiple language support or no i18n, it's still a good idea to keep your content outside your code in a constant file.

You can either name your constant file[LANGUAGE_INITIAL].js, or constants.js. These are the most common file names developers use for this purpose.

How to create your constant file:

Constants are simply JavaScript objects with key/values. We start with declaring the object with a name which reflects the content it’s holding. As these are strings, we use quotes to wrap them. Before exporting the messages, do an Object.freeze() – this will avoid any accidental value change from outside any key.

To use the constants, we need to import the file into the component file. Once imported, we can use the dot operator to access the keys:

// constants.js or en.js
const MESSAGES = { 
    'HEADING': 'welcome to the website",
    'ENTER_YOUR_NAME': 'Enter user name',
    'HOME': [{
        'HEADING': 'welcome to the home'
     }]
}

Object.freeze(MESSAGES);

export default MESSAGES;

// Using constants.js in component
import MESSAGES from '../constants/constants

const Home = () => {
    return(
        <p>{MESSAGES.HEADING}</h1>
    )
}

export default Home;

2. Use Helpers / Utils

While working on a ReactJS codebase it's crucial to identify the parts in the code which can be independent utils or helpers, instead of tightly coupling the components.

Helpers or utils are responsible for performing a task that can be used in multiple places and by multiple devs. Examples include a Date format, string formation, API call code, and DOM manipulation, to name a few.

Why Use Helpers / Utils?

Every component should be responsible for only one job, which is something known as the “Single responsibility principle”.

We should identify reusable functions and move them to utils for the following reasons:

1. It results in cleaner components and cleaner code
2. No tight coupling
3. Easily scalable functionality
4. Easy to maintain and debug
5. Better reusability
6. Components are now responsible only for the UI

// dateUtils.js : Moved the formatDate to a seprate util file to have reusability

export function formatDate(date) { const options = { year: 'numeric', month: 'long', day: 'numeric' }; return new Date(date).toLocaleDateString(undefined, options); }

```react
// Updated Blog.jsx component after util
import React, { Component } from 'react';
import { formatDate } from './dateUtils'; 

const Blog = ({title, content, date}) => {
    return (
        <div>
            <h2>{title}</h2>
            <p>{content}</p>
            <p>Published on: {formatDate(date)}</p>
        </div>
    );
}
}

For example, in the above code, you can see that we have a formatDate function inside the component Blog. Here we can move the formatDate to utils. Why?

• formatDate is responsible for formatting the date, and not for publishing the date
• formatDate can be used by another component
• formatDate can have different formats based on the business requirements. For example, now we are passing a second argument based on this component requirement. If any new requirement comes up, the developer needs to re-write the component.

3. Learn How to Use Props

To communicate between components in ReactJS, we use props. But there are different ways to do that.

It is crucial to choose only one style to consume props in the component in your codebase. This will make the codebase consistent. There's more than one way to destructure props which is why you should pick only one for consistency and readability of your code.

Let's talk about the ways you can work with props in React.

How to Use Props

In this approach we have to repeat props every time we are using props.

This approach is not a great way of consuming props because we are repeating props whenever we want to use them. Besides being repetitive, when it comes to creating nested props, it will require too much typing.

Your time as a developer is important, and we want to optimize the code wherever possible and not repeat things unless absolutely necessary.

In the below code example, you can see why this approach isn't the best. We have the Input component with props such as type, placeholder, name, and changeHandler. In the return section, we are repeating props with every attribute such as props.type.

const Input = (props) => {
    return <input 
    type={props.type} 
    placeholder={props.placeholder} 
    name={props.name} 
    className="block p-2 my-2 text-black" 
    onChange={props.changeHandler}/>
}

export default Input;

How to destructure props

In the second way of working with props, we use the JavaScript destructuring assignment.

This is an improvement over the first approach, as in this one we won't repeat props whenever we're using props.

Here is an example of destructuring props. In the first code snippet, we are getting type, placeholder, name, and changeHandler from the props the first thing in the component.

const { type, placeholder, name, changeHandler } = props

In the below code example, we can see the improvement in the code. We have Input component with props but instead of repeating props.name we are destructing the props. It is a huge improvement in readability, and developer experience.

const Input = (props) => {
    const { type, placeholder, name, changeHandler } = props;
    return <input 
    type={type} 
    placeholder={placeholder} 
    name={name} 
    className="block p-2 my-2 text-black" 
    onChange={changeHandler}/>
}

How to destructure props in component arguments

This is my favourite method of destructuring props. Developers can see at the start of the component which props would get used in the component. We also don't have any repetition of the props keywords.

Compared with the last approach, it is:

• DRY (Don't Repeat Yourself): we are not repeating the props
• Readable: At the component's first line (definition), we know which props it is expecting. This improves the readability and clarity of the component.

In the below code, we can see that we are destructing props in the definition of the component. This makes readability way better. As now developers can look at the first line and understand how many and what all props are expected in this component.

const Input = ({ type, placeholder, name, changeHandler }) => {
    return <input 
    type={type} 
    placeholder={placeholder} 
    name={name} 
    className="block p-2 my-2 text-black" 
    onChange={changeHandler}/>
}

4. Have One File for Each Component

In ReactJS, it is important to have one file per component. This helps make your code cleaner and more maintainable.

It also follows the single responsibility principle I mentioned earlier.

It is tempting to have one file and write all the code inside it for isolation – but we should break it into smaller components.

In the below example, we have one file Input.jsx which has two components Input and Icon. We are using Icon in the return section of Input.

It looks like Input and Icon are related and its logical to group them into one file. But we should not do this, as it is not a scalable solution (and isn't reusable, either).

// Don't do this:
import React from 'react';

// File name: Input.jsx
// This example shows how we are exporting 2 components from one file
// We should NOT do this
const Input = ({ type, placeholder, name, changeHandler }) => {
    return <>
    <input 
    type={type} 
    placeholder={placeholder} 
    name={name} 
    className="block p-2 my-2 text-black" 
    onChange={changeHandler}/>

    <Icon type="warning"/>
}
        <>

const Icon = ({ type, url}) => {
    return <img src={url} data-type={type} />
}

export {Input, Icon};

Instead, we should be making two separate components for Input and Icon, as shown below. This will help you reuse both components and scale them individually.

// Do this instead:
// Input.jsx: create 2 separate files for Input and InputIcon
import React from 'react';

const Input = ({ type, placeholder, name, changeHandler }) => {
    return <input 
    type={type} 
    placeholder={placeholder} 
    name={name} 
    className="block p-2 my-2 text-black" 
    onChange={changeHandler}/>
}

export default Input;

5. Don't Use Inline Functions

It is common to write inline functions in JavaScript, but it is better to avoid adding them when possible.

You should keep your JSX separate from your logical code. Inline functions are not reusable, don't help with code abstraction, and are hard to test.

This is why you should always avoid inline functions.

In the below code snippet we have the handleIncrement function as an inline function on button. It is not reusable, and it's tightly coupled with the component:

// Don't do this:
import React, { useState } from 'react';

function CounterInline() {
  const [count, setCount] = useState(0);

  const handleIncrement = () => {
    setCount(count + 1);
  };

  return (
    <div>
      <p>Count: {count}</p>
      <button onClick={handleIncrement}>Increment</button> 
    </div>
  );
}

export default CounterInline;

Then how we can move away from inline functions? Let's look at how you can refactor of the above code.

In the below code, we can have incrementCount used on button and it is expecting two arguments. We have made the function here reusable:

// Do this instead:
import React, { useState } from 'react';

// Standalone function for incrementing
function incrementCount(currentCount, setCount) {
  setCount(currentCount + 1);
}

const CounterStandalone = () => {
  const [count, setCount] = useState(0);

  return (
    <div>
      <p>Count: {count}</p>
      <button onClick={() => incrementCount(count, setCount)}>Increment</button>
    </div>
  );
}

export default CounterStandalone;

6. Implement a 404 Component and Route

While implementing routing in ReactJS we should add a 404 component.

When the user tries to access a page which doesn't exist, the status code from the server will be 404. As a front-end developer, it’s a good idea to show the user an error message to give them some context here.

React-router provides an easy way to display the error when the server returns a 404. You'll need to create a component that should be rendered when the 404 status code returns from the server.

Whenever a user types in or reaches a route which is not found, the 404 page will show the error to the user, which is better user experience (rather than just seeing an unexplained "404").

Tip: In the component, add a link to the homepage of your website. This will help the user redirect to the homepage of your website.

    <route path="*" component={<Error404/>} />

7. Fetch Data Progressively

In React applications, you'll often fetch data through APIs.

Instead of fetching and creating the UI in one go, we should fetch the data on-demand – for example, on scrolling into view, on clicking of the pagination, and so on.

This will improve the performance of the application as well as the User Experience.

There are a few packages that can help you implement lazy-loading. Lazy loading is a technique you can use to load the data on-demand or progressively as needed. Instead of showing whole API data on the screen at one time, it'll only show the data on-demand.

Here's how you can install the react-lazyload package:

// install react-lazyload package
npm install react-lazyload

And here's the code:

// Create a component - ItemList.jsx
import React, { useState, useEffect } from 'react';
import axios from 'axios';

const ItemList = () => {
  const [items, setItems] = useState([]);

  useEffect(() => {
    // Fetch data from your API here
    axios.get('https://example.com/api/items')
      .then(response => {
        setItems(response.data);
      })
      .catch(error => {
        console.error('Error fetching data:', error);
      });
  }, []);

  return (
    <div>
      <h2>Item List</h2>
      <ul>
        {items.map(item => (
          <li key={item.id}>
            {item.name}
          </li>
        ))}
      </ul>
    </div>
  );
};

export default ItemList;

// App.js
import React, { lazy, Suspense } from 'react';
import LazyLoad from 'react-lazyload';

// Import the lazy-loaded component
const ItemList = lazy(() => import('./ItemList'));

function App() {
  return (
    <div>
      <h1>React LazyLoad with API Data Example</h1>

      {/* Use React LazyLoad to lazy-load the component */}
      <LazyLoad height={200}>
        <Suspense fallback={<div>Loading...</div>}>
          <ItemList />
        </Suspense>
      </LazyLoad>
    </div>
  );
}

export default App;

In the above example of code, have created a component which will make the API requests and render the API data ItemList.jsx. While using the ItemList in App, instead of rendering whole data in one go, we will use the LazLoad to load the component.

As a result, component is loaded lazily when it's close to being visible in the viewport, and the data from the API will be displayed.

Some of the packages you can use for lazy loading are react-lazyload, react-infinite-scroll-component , and react-paginate.

8. Use Unique Values for Key Attributes

One of the reasons React is popular is because of its "virtual DOM".

The virtual DOM (VDOM) helps you optimize the process of updating the UI.

React will only update nodes that get changed and not the whole DOM unless it is required. This is one of the secrets of the most performant apps.

React needs a key attribute to identify to which node the change happened. This is why we should always use a unique value for key.

A good example of how to do this is to add the id of every item.

Note: use index as a key only when your data is static, not re-ordering or filtered.

// List.jsx
import React from 'react';

const List = ({ items }) => (

  return(<ul>
    {items.map((item, index) => (
      <li key={index}>{item}</li>
    ))}
  </ul>)
);

export default List;

// App.jsx
import List from './List';

const App = () => {
  const items = ['Item 1', 'Item 2', 'Item 3', 'Item 4', 'Item 5'];

  return (
    <div>
      <h1>Simple List Example</h1>
      <List items={items} />
    </div>
  );
};

export default App;

In the above code example, we are creating a list by mapping over the data. In the List component we are assigning the index to the key attribute. This will be used by ReactJS under the hood to optimize the performance whenever any li will be updated or changed.

9. Use Types

Using tools with static type checking built in (like TypeScript) can help you avoid unnecessary bugs in your code.

It will also support your code with quality and type-checking.

If you are someone who is just starting out learning about type checking, then you can start with proptypes and later learn TypeScript.

JavaScript is not strictly typed, which means there's a higher possibility that you'll have unexpected bugs or type errors.

For example, when we expect a prop to be a number, it could end up being a string which would cause an error.

So, strict types and TypeScript can help you avoid such unexpected bugs during development.

import React from 'react';
import PropTypes from 'prop-types';

const UserCard = ({ name, age, email }) => {
  return (
    <div>
      <h2>User Card</h2>
      <p>Name: {name}</p>
      <p>Age: {age}</p>
      <p>Email: {email}</p>
    </div>
  );
};

// Define the prop types for the UserCard component
UserCard.propTypes = {
  name: PropTypes.string.isRequired, // A required string prop
  age: PropTypes.number.isRequired,  // A required number prop
  email: PropTypes.string,           // An optional string prop
};

import UserCard from 'userCard';

const App = () => {
  return (
    <div>
      <h1>PropTypes Example</h1>
      <UserCard name="John Doe" age={30} email="john@example.com" />
    </div>
  );
};

export default App;

In the above code example, we have created a component UserCard. This component expects 3 props: name, age, and email. By using proptypes , we will declare two things for props – datatype (what would be the datatype of the props, for example string, number, and so on), and whether it's required or optional.

While using the UserCard component, if anyone passes the wrong datatype or misses a required prop, then the code will throw an error and warn them to fix it.

10. Use the lazy() and Suspense() Functions

ReactJS uses the Webpack bundler (if you are using creat-react-app).

Webpack takes care of bundling of the code, and performs functions like tree shaking.

But the way React works is it downloads the whole code on the client side even if we don't need it. This is an expensive task. If your bundle size is big then it will impact the performance of your apps.

A good way to avoid this is to lazy load the code on demand by using lazy(), which will let the routes get loaded when they're needed.

const LazyComponent = lazy(() => import('./LazyComponent'));

While using lazy() we should also use Suspense(), as lazy() is an async way of loading the components.

We don't want to show the user a blank screen until our route is done loading. Suspense() helps by showing a message while the component is loading.

 <Suspense fallback={<div>Loading...</div>}>
        {/* The LazyComponent will only be loaded when needed */}
        <LazyComponent />
 </Suspense>

Wrapping Up

Phew, we have reached the end. These tips are not just limited to large codebases but to projects of any size.

At high level we learned about the following concepts:

1. Following the DRY principle
2. Following the Single Responsibility Principle
3. Creating a good user experience by loading the data progressively
4. Improving readability
5. Improving the developer experience
6. Avoiding bugs at the development time
7. Improving performance

Happy Learning!

Don't be shy. Come and say hi! You can find me on Twitter, LinkedIn, and YouTube.

Want to see some calligraphy? Check my art on instagram.com/calligraphyzen.

As a React developer, maintaining code quality is essential for building robust and maintainable applications. Fortunately, there's a powerful tool called ESLint that can significantly improve the quality of your React projects.

In this article, we will explore how ESLint works and create a sign-up button component to demonstrate its benefits.

What is ESLint?

ESLint is a popular open-source JavaScript linting utility. It analyzes your code for potential errors, and enforces coding standards while improving code quality.

It can also help you as a developer to identify and fix common mistakes, use best practices, and maintain consistency across your codebase.

You can integrate ESLint seamlessly into your React projects, providing real-time feedback and improving your overall code quality.

How to Set Up Your Project

Let's start by setting up a new React project and installing ESLint. To demonstrate this, we will create a signup app in React.

Imagine that we want to store our project on the desktop, so first we will set up our file structure. Let's start by creating a root directory for our project on the desktop (eslintExample in this case).

mkdir eslintExample 
cd eslintExample

Install the React app

Now we'll use create-react-app to set up our React app.

npx create-react-app signup-app

Install ESLint

And you can use the following command to set up ESLint:

npm install eslint eslint-plugin-react eslint-plugin-react-hooks --save-dev

Your directory structure should look like this:

eslintExample/
  └── signup-app/
      └── node_modules/
      └── public/
      └── src/
          └── App.css
          └── App.js
          └── App.test.js
          └── SignupButton.js
          └── index.css
          └── logo.svg
          └── reportWebVitals.js
          └── setupTests.js
      └── .eslintrc.json
      └── .gitignore
      └── package-lock.json
      └── package.json
      └── README.md

How to Configure ESLint in a React Project

To work with ESLint in our React project, we need to configure it. To do this, the first step is to create a .eslintrc.json file in the project's root directory and add the following code:

{
    "env": {
      "browser": true,
      "es2021": true
    },
    "extends": ["eslint:recommended", "plugin:react/recommended", "plugin:react-hooks/recommended"],
    "parserOptions": {
      "ecmaVersion": 12,
      "sourceType": "module",
      "ecmaFeatures": {
        "jsx": true
      }
    },
    "plugins": ["react", "react-hooks"],
    "rules": {
      //add customize rules here as per your project's needs
    }
}

The above configuration sets up ESLint to work with React and React Hooks using the recommended configurations. You can add or customize rules according to your project's specific requirements.

Created the.eslintrc.json file and added the configuration that allows ESLint to work with React.

How to Create the Signup Button Component

Now, let's create a simple Signup Button component (SignupButton.js) inside the "src" folder.

Inside the src file, create the SignupButton.js file. It should look like this: src/SignupButton.js. Inside your SignupButton.js, paste the following code:

import React from "react";

const SignupButton = () => {
  const handleSignup = () => {
    alert("Sign up successful!");
  };

  return (
    <button onClick={handleSignup} className="signup-button">
      Sign Up
    </button>
  );
};

The above component is a basic button that triggers an alert when clicked, simulating the signup process. Now you can run this command:

npm start

This will start our React app at the root of the project. Then you should see this error below:

We started the local server, and we are seeing an error coming from the app.js file. React needs to be imported in app.js file. We need to export our button component.

This error might confuse you, because you may not know where it's coming from. We are seeing this error because we set up ESLint in our project, and it scans through our project to tell us that React needs to be imported in the app.js file.

But let’s run ESLint to see where the exact error is coming from.

How to Run ESLint

With ESLint configured, let's run it to analyze our SignupButton component for potential issues. Open your terminal and run the following command at the root of your project:

npx eslint src/SignupButton.js

ESLint will analyze the SignupButton.js file and display any problems it finds below.

We run the eslint command to show us where we are having error: in the signup button and app.js

How to Fix the ESLint Issues

From the above, you can see that ESLint identifies where we are having errors. We did not import React in our App.js file, and we did not export our signup button component. Let’s fix that.

We imported React in the app.js file and exported SignupButton component.

As you can see, our code was now built successfully.

We were able to export our SignupButton component and imported React in App.js component. This solves our problem.

If ESLint identifies any further issues in our SignupButton component, it will display them along with suggestions on how to fix them.

For example, ESLint might detect missing semicolons, unused variables, or violations of React best practices like the import React from “react” in the app.js file.

By addressing the issues highlighted by ESLint, we can ensure that our code adheres to best practices, is easier to read, and has fewer potential bugs.

Conclusion

ESLint is an indispensable tool for React developers to maintain code quality and improve productivity. By integrating ESLint into your React projects, you can catch errors early, follow coding standards, and foster collaboration within your team.

In this article, I have explained how to set up ESLint in a React project and demonstrate its benefits with a simple Signup Button component. By utilizing ESLint effectively, you can write cleaner, more consistent code and deliver higher-quality React applications.

So, why wait? Start using ESLint in your React projects and watch your code quality soar. Happy coding! 🚀

Your feedback is highly appreciated.
you can follow me on twitter and linkedIn

Greetings, fellow coding enthusiasts!

We're going to dive deep into the realm of Django code quality assessment. In this comprehensive guide, I'll walk you through an in-depth approach to measuring the code quality of your Django-based application.

By the end of this tutorial, you will be able to:

1. Build CRUD APIs using Django and DRF (Django REST Framework)
2. Write automated tests for the APIs using Pytest
3. Measure code test coverage using Coverage
4. Utilize SonarQube to assess code quality, identify code smells, security vulnerabilities, and more

Prerequisites to follow along in this tutorial include:

1. Python 3 installation on your chosen Operating System (OS). We'll use Python 3.10 in this tutorial.
2. Basic knowledge of Python and Django
3. Any code editor of your choice

Without any further delay, let's jump right in and get started.

How to Get the APIs Up and Running

To begin, open your Terminal or bash. Create a directory or folder for your project using the command:

mkdir django-quality && cd django-quality

In my case, the folder name is "django-quality".

To isolate the project dependencies, we need to utilize a Python virtual environment.

To create a virtual environment, use the following command in your Terminal or bash:

python3 -m venv venv

Activate the virtualenv by running this command:

source venv/bin/activate

If everything works fine, you should see the virtual environment indicator enclosed in brackets, similar to the image shown below:

Python Virtualenv activated successfully

At the root directory of your project, create a folder called "requirements" that will house the external packages required for various development stages, such as dev (development) and staging.

Inside the "requirements" folder, create two files: "base.txt" and "dev.txt". The "base.txt" file will include generic packages required by the application, while the "dev.txt" file will contain dependencies specific to development mode.

By now, the contents in your project folder should have the following structure

- requirements
    ├── base.txt
    └── dev.txt
- venv

Here are the updated contents for the "base.txt" and "dev.txt" files:

base.txt

Django==4.0.6
djangorestframework==3.13.1
drf-spectacular==0.22.1

dev.txt

-r base.txt
pytest-django==4.5.2
pytest-factoryboy==2.5.0
pytest-cov==4.1.0

• djangorestframework: Used for API development.
• drf-spectacular : Used for automated documentation of the APIs.
• pytest-cov: Utilized for measuring code coverage during testing.
• pytest-factoryboy: Used for creating test data using factory patterns.

Make sure your virtual environment is activated, then run the following command at the root directory to install the dependencies specified in "dev.txt":

pip install -r requirements/dev.txt

To create a new Django project, you can run the following command:

django-admin startproject core .

The name of the project is 'core'. You can decide to use any suitable name that fits your use case.

By now, you should see a couple of files and folders automatically created after running the command.

Here is the current project structure:

├── core
│   ├── asgi.py
│   ├── __init__.py
│   ├── settings.py
│   ├── urls.py
│   └── wsgi.py
├── manage.py
├── requirements
│   ├── base.txt
│   └── dev.txt
└── venv

Current Folder Structure in VSCode

The APIs we will create will be a basic blog API with CRUD functionality. Let's create a new app within the project to host all the files related to the blog features.

Run this command to create a new app called 'blog':

python manage.py startapp blog

By now, a new folder named 'blog' has been auto-created by the command.

Here is the folder structure:

├── blog
│   ├── admin.py
│   ├── apps.py
│   ├── __init__.py
│   ├── migrations
│   ├── models.py
│   ├── tests.py
│   └── views.py
├── core
├── manage.py
├── requirements
└── venv

Update the models.py file in the blog folder. The Blog class defines the database schema for the blog.

blog/models.py

from django.db import models

class Blog(models.Model):
    title = models.CharField(max_length=50)
    body = models.TextField()
    published = models.BooleanField(default=True)
    created_at = models.DateTimeField(auto_now_add=True)

Create a new file named 'serializers.py' inside the 'blog' folder and update its content as shown below:

blog/serializers.py

from rest_framework import serializers

from .models import Blog

class BlogSerializer(serializers.ModelSerializer):
    class Meta:
        model = Blog
        fields = '__all__'

    extra_kwargs = {
            "created_at": {"read_only": True},
        }

The BlogSerializer class is utilized for validating incoming blog data sent by the client (such as from the frontend or mobile app) to ensure it adheres to the expected format.

Additionally, the serializer class is used for both serialization (converting Python objects to a transmittable format like JSON) and deserialization (converting a transmittable format like JSON back to Python objects).

Let's create the view to handle CRUD functionality, leveraging the DRF ModelViewSet to effortlessly create APIs with just a few lines of code.

blog/views.py

from rest_framework import filters, viewsets

from .models import Blog
from .serializers import BlogSerializer


class BlogViewSet(viewsets.ModelViewSet):
    queryset = Blog.objects.all()
    http_method_names = ["get", "post", "delete", "patch","put"]
    serializer_class = BlogSerializer
    filter_backends = [
        filters.SearchFilter,
        filters.OrderingFilter,
    ]
    filterset_fields = ["published"]
    search_fields = ["title", "body"]
    ordering_fields = [
        "created_at",
    ]

Create a new file named 'blog.urls' in the 'blog' folder.

By utilizing the DRF router for URL configuration, the URLs are automatically generated based on the allowed methods defined in the BlogViewSet.

blog/urls.py

from django.urls import include, path

from rest_framework.routers import DefaultRouter

from .views import BlogViewSet

app_name = "blog"

router = DefaultRouter()
router.register("", BlogViewSet)

urlpatterns = [
    path("", include(router.urls)),
]

The next step is to register the urls.py file defined in the 'blog' app within the main project's urls.py file. To do this, you should locate the project's urls.py file, which serves as the starting point for URL routing.

core/urls.py

from django.contrib import admin
from django.urls import path, include
from drf_spectacular.views import (
    SpectacularAPIView,
    SpectacularRedocView,
    SpectacularSwaggerView,
)

urlpatterns = [
    path('api/schema/', SpectacularAPIView.as_view(), name='schema'),
    path('api/v1/doc/', SpectacularSwaggerView.as_view(url_name='schema'), name='swagger-ui'),
    path('api/v1/redoc/', SpectacularRedocView.as_view(url_name='schema'), name='redoc'),
    path('admin/', admin.site.urls),
    path('api/v1/blogs/', include('blog.urls')),
]

The api/v1/blogs/ URL is mapped to the URLs defined in blog.urls. Additionally, other URLs are utilized for automated API documentation.

Update the settings.py file located inside the core folder. This file contains configurations for the Django application.

In the INSTALLED_APPS section, register the newly created 'blog' app, along with any desired third-party apps. Note that for brevity, the default Django apps are not included in the following list:

settings.py

INSTALLED_APPS = [


    #Third-party Apps
    'drf_spectacular',

    #Local Apps
    'blog',
]

Update the settings.py file to include configurations related to Django REST Framework (DRF) and documentation.

settings.py

REST_FRAMEWORK = {
    "DEFAULT_SCHEMA_CLASS": "drf_spectacular.openapi.AutoSchema",
    "TEST_REQUEST_DEFAULT_FORMAT": "json",
}


SPECTACULAR_SETTINGS = {
    'SCHEMA_PATH_PREFIX': r'/api/v1',
    'DEFAULT_GENERATOR_CLASS': 'drf_spectacular.generators.SchemaGenerator',
    'SERVE_PERMISSIONS': ['rest_framework.permissions.AllowAny'],
    'COMPONENT_SPLIT_PATCH': True,
    'COMPONENT_SPLIT_REQUEST': True,
    "SWAGGER_UI_SETTINGS": {
        "deepLinking": True,
        "persistAuthorization": True,
        "displayOperationId": True,
        "displayRequestDuration": True
    },
    'UPLOADED_FILES_USE_URL': True,
    'TITLE': 'Django-Pytest-Sonarqube - Blog API',
    'DESCRIPTION': 'A simple API setup with Django, Pytest & Sonarqube',
    'VERSION': '1.0.0',
    'LICENCE': {'name': 'BSD License'},
    'CONTACT': {'name': 'Ridwan Ray', 'email': 'ridwanray.com'},
    #OAUTH2 SPEC
    'OAUTH2_FLOWS': [],
    'OAUTH2_AUTHORIZATION_URL': None,
    'OAUTH2_TOKEN_URL': None,
    'OAUTH2_REFRESH_URL': None,
    'OAUTH2_SCOPES': None,
}

With all the necessary configurations in place, let's run the migrations command to ensure that the models in the application are synchronized with the database schema.

Execute the following commands in the root directory to synchronize the models with the database schema:

python manage.py makemigrations
python manage.py migrate

To start the development server, run the following command:

python manage.py runserver

Starting local development server with runserver command

The application is now running at http://127.0.0.1:8000/.
To access the documentation, visit http://127.0.0.1:8000/api/v1/doc/.

Automated Blog API documentation using drf-spectacular

How to Write Automated Tests with Pytest

Pytest, the testing tool we're using for writing automated tests, is included as part of the dependencies declared in the requirement folder. Now, let's write some tests and explore its functionality.

In the blog folder, a file named "tests.py" is automatically generated when starting the blog app. To organize the tests, create a new folder called "tests" within the blog directory.

Move the initial "tests.py" file into the newly created "tests" folder. To make the "tests" folder a module, create an empty file named "init.py".

Create a new file named 'conftest.py' inside the 'tests' folder. This file will store any pytest fixtures (that is, reusable components) required during the test writing process.

Test folder structure:

├── tests
│   ├── conftest.py
│   ├── factories.py
│   ├── __init__.py
│   ├── __pycache__
│   └── tests.py

tests/conftests.py

import pytest
from rest_framework.test import APIClient

@pytest.fixture
def api_client():
    return APIClient()

The api_client() is a Pytest fixture utilized for making actual API calls.

Create a new file named 'factories.py'. This file will include the factories used during test writing. Factories provide a convenient way to create objects (that is, model instances) without the need to specify all attributes each time.

tests/factories.py

import factory
from faker import Faker
from blog.models import Blog

fake = Faker()

class BlogFactory(factory.django.DjangoModelFactory):
    class Meta:
        model = Blog

    title = fake.name()
    body = fake.text()
    published = True

tests/tests.py

import pytest
from django.urls import reverse
from .factories import BlogFactory

pytestmark = pytest.mark.django_db


class TestBlogCRUD:
    blog_list_url = reverse('blog:blog-list')

    def test_create_blog(self, api_client):
        data = {
            "title": "Good news",
            "body": "Something good starts small",
            "published": True
            }

        response = api_client.post(self.blog_list_url, data)
        assert response.status_code == 201
        returned_json = response.json()
        assert 'id' in returned_json
        assert returned_json['title'] == data['title']
        assert returned_json['body'] == data['body']
        assert returned_json['published'] == data['published']

    def test_retrieve_blogs(self, api_client):
        BlogFactory.create_batch(5)
        response = api_client.get(self.blog_list_url)
        assert response.status_code == 200
        assert len(response.json()) == 5

    def test_delete_blog(self, api_client):
        blog = BlogFactory()
        url = reverse("blog:blog-detail",
                      kwargs={"pk": blog.id})
        response = api_client.delete(url)
        assert response.status_code == 204

    def test_update_blog(self, api_client):
        blog = BlogFactory(published= True)
        data = {
            "title": "New title",
            "body": "New body",
            "published": False,
        }
        url = reverse("blog:blog-detail",
                      kwargs={"pk": blog.id})

        response = api_client.patch(url, data)
        assert response.status_code == 200
        returned_json = response.json()
        assert returned_json['title'] == data['title']
        assert returned_json['body'] == data['body']
        assert returned_json['published'] == data['published']

The TestBlogCRUD class tests the CRUD functionalities of the application. The class defines four methods, each testing a specific CRUD functionality.

Create a Pytest configuration file named pytest.ini in the root directory. This file will contain settings that instruct Pytest on how to locate the test files.

pytest.ini

[pytest]
DJANGO_SETTINGS_MODULE = core.settings
python_files = tests.py test_*.py *_tests.py
addopts = -p no:warnings --no-migrations --reuse-db

To run the tests, execute the pytest command in the root directory as shown below:

pytest

Pytest testcases result

The test results indicate that all four test cases have passed successfully.

As of the time of writing, two popular tools used in the Python community for reporting test coverage in a codebase are Coverage and pytest-cov.

In our case, we'll be using pytest-cov for its flexibility when it comes to reporting test coverage.

Create a new file named 'setup.cfg' in the root directory. This file serves as the configuration file for coverage.

setup.cfg

[coverage:run]
source = .
branch = True
[coverage:report]
show_missing = True
skip_covered = True

The source value in the [coverage:run] section specifies the root directory location from which test coverage will be measured.

In addition to statement coverage in the test report, branch coverage identifies uncovered branches when using conditional statements (for example if, else, case).

Note: It is possible to specify folders to omit from test coverage, such as migration folders, in the setup.cfg file. We will configure these settings in SonarQube.

Let's rerun the test cases using the following command:

pytest --cov --cov-report=xml

The --cov-report option specifies the format of the coverage report. Various formats like HTML, XML, JSON, and so on are supported. In this case, we specify xml because it is supported by SonarQube.

Pytest coverage report in XML format

For HTML format, a folder named 'htmlcov' will be generated in the root directory. This folder contains the 'index.html' file, which allows you to visualize the coverage results and areas that are not covered.

How to Setup SonarQube

SonarQube is a tool used for static code analysis. It helps in identifying code quality issues, bugs, vulnerabilities, and code smells in software projects.

To simplify the process, we can run a Docker container based on the SonarQube image.

Execute the following command in the command line:

docker run -d -p 9000:9000 -p 9092:9092 sonarqube

After a few moments, depending on your internet speed, visit http://0.0.0.0:9000/.

You can use the following login credentials to access the application: Username: admin Password: admin

Next, you need to download Sonar Scanner. Visit this link and select the option that is compatible with your operating system (OS).

SonarScanner download on the Sonarsource.com website

Unzip the sonar-scanner and move it from the 'Downloads' folder to a secure directory .

unzip sonar-scanner-cli-4.8.0.2856-linux.zip

mv sonar-scanner-4.2.0.1873-linux /opt/sonar-scanner

Add the following lines to the content of the sonar-scanner.properties file located at /opt/sonar-scanner/conf/sonar-scanner.properties:

vim  /opt/sonar-scanner/conf/sonar-scanner.properties

Add these two lines and save the file:

sonar.host.url=http://localhost:9000
sonar.sourceEncoding=UTF-8

Add /opt/sonar-scanner/bin to the system's PATH environment variable by executing this command:

export PATH="$PATH:/opt/sonar-scanner/bin

Update the content of .bashrc:

vim ~/.bashrc

Add this line to the .bashrc file and save it:

export PATH="$PATH:/opt/sonar-scanner/bin

Run the following command to apply the changes to your current terminal session:

source ~/.bashrc

To ensure that everything is functioning properly, execute the following command:

sonar-scanner -v

Checking sonarqube version on the terminal

Navigate to the 'Projects' tab on the SonarQube dashboard and proceed to manually create a new project.

Creating a new project on the sonarqube dashboard

Provide a suitable name for the project, then select the option "Use the global setting" before proceeding to create the project.

Choosing appropriate name as the name of the new preoject

Configuring new project to use global settings

After creating the project, you will be prompted to select the analysis method for your project. Choose the 'Locally' option.

Run analysis on the project locally

After selecting the 'Locally' option, you will be required to generate a token. Click on 'Continue' to proceed. Next, select the programming language of your project and the operating system (OS) it will be running on.

Choose the programming language of project and OS

Copy the command displayed, as we'll use it to execute the analysis for the project.

Code needed to run analyis

Here is the content of the command:

sonar-scanner \
  -Dsonar.projectKey=newretailer \
  -Dsonar.sources=. \
  -Dsonar.host.url=http://0.0.0.0:9000 \
  -Dsonar.token=sqp_7b6aada8ce53e97ebb7b2bf5e9b64d53b8938a6f \
  -Dsonar.python.version=3

Note: We have added an additional line to the command to specify the Python version as -Dsonar.python.version=3.

Before executing the analysis command, follow these steps:

1. Click on "Project Settings" and then select "General Settings".
2. Next, navigate to the "Analysis Scope" tab.

Source files that should be ignored by the analysis

Source File Exclusions are used to specify files or folders that SonarQube should not analyze as part of the codebase. These may include files or directories that are not directly part of the code but are still present in the project directory.

Some common examples of such files or folders are:

• venv (virtualenv)
• htmlcov (coverage HTML format)
• node_modules (Node.js modules directory)

Code Coverage Exclusions are used to specify files or folders that should be excluded when calculating the coverage percentage.

Here are the patterns for the files and folders ignored:
/tests/, /migrations/, /admin.py, /apps.py, core/asgi.py, core/wsgi.py, manage.py

Patterns used to exclude some files from coverage report and coverage percent calculation

On the "Languages" tab, select "Python" as the programming language for the project. Then update the path to the coverage report as "coverage.xml".

Programming language selection and XML coverage report location

Execute the previously provided command at the root directory:

sonar-scanner   -Dsonar.projectKey=DjangoSonar   -Dsonar.sources=.   -Dsonar.host.url=http://0.0.0.0:9000   -Dsonar.token=sqp_bb1dc2534249bf567c681f4acc440c2e278cb43f   -Dsonar.python.coverage.reportPaths=coverage.xml -Dsonar.python.version=3

If everything is functioning properly, you should see a successful result.

Running sonarqube analysis on the project with command given on the dashboard

If you encounter errors related to unauthorized access or permission issues when trying to analyze a project locally, follow these steps:

1. Visit the SonarQube Administrator interface.
2. Navigate to the 'Security' section.
3. Look for the option labeled 'Force user authentication' and disable it.
4. Save the changes and rerun the analysis using the previous command.

Debugging authentication error during project analysis

Another way to troubleshoot any errors is to visit the warning notifications and check for any errors encountered during the project analysis.

Warning messages for analysis

Click on "Overall Code" to access the overall code analysis section:

SonarQube analysis result for the project on the dashboard

Wrapping Up

The complete source code for this project is available on Github.

Remember to create a .gitignore file in the root directory of your GitHub repository to specify files and directories that should be ignored and not committed.

This article has explored the process of measuring Django code quality using powerful tools such as SonarQube, Pytest, and Coverage. By integrating these tools, you can gain insights into code health, write effective tests, and ensure adequate code coverage.

Applying these practices enhances code quality, resulting in efficient development processes and high-quality software.

If you enjoyed this article, you can check out my video collection on YouTube to find more fun stuff to learn. And follow me on LinkedIn

References:

• https://github.com/amirajoodani/sonarqube
• https://docs.sonarsource.com/sonarqube/latest/analyzing-source-code/scanners/sonarscanner/
• https://coverage.readthedocs.io/en/7.2.7/
• https://docs.pytest.org/en/7.1.x/getting-started.html

So instead, I want to talk about the experiences I've had building frontends from scratch at DelightChat and at my previous companies.

These projects require a deeper understanding of React and extended usage in a production setting.

If you want to watch a video version of this tutorial to supplement your reading, you can do so here.

Introduction

In a nutshell, a complex React project should be structured like this. Although I use NextJS in production, this file structure should be quite useful in any React setting.

src
|---adapters
|---contexts
|---components
|---styles
|---pages

Note: In the above file structure, the assets or static files should be placed in whatever the variant of public * folder for your framework is.

For each of the above folders, let’s discuss them in order of precedence.

1. Adapters

Adapters are the connectors of your application with the outside world. Any form of API call or websocket interaction which needs to happen, to share data with an external service or client, should happen within the adapter itself.

There are cases where some data is always shared between all the adapters – for example, sharing of cookies, base URL and headers across your AJAX (XHR) adapters. These can be initialized in the xhr folder, and then imported inside of your other adapters to be used further.

This structure will look like this:

adapters
|---xhr
|---page1Adapter
|---page2Adapter

In the case of axios, you can use axios.create to create a base adapter, and either export this initialized instance, or create different functions for get, post, patch and delete to abstract it further. This would look like this:

// adapters/xhr/index.tsx

import Axios from "axios";

function returnAxiosInstance() {
  return Axios.create(initializers);
}

export function get(url){
  const axios = returnAxiosInstance();
  return axios.get(url);
}

export function post(url, requestData){
  const axios = returnAxiosInstance();
  return axios.post(url, requestData);
}

... and so on ...

After you have your base file (or files) ready, create a separate adapter file for each page, or each set of functionalities, depending on how complex your app is. A well-named function makes it very easy to understand what each API call does and what it should accomplish.

// adapters/page1Adapter/index.tsx

import { get, post } from "adapters/xhr";
import socket from "socketio";

// well-named functions
export function getData(){
  return get(someUrl);
}

export function setData(requestData){
  return post(someUrl, requestData);
}

... and so on ...

But how will these adapters be of any use? Let’s find out in the next section.

2. Components

Although in this section we should talk about contexts, I want to talk about components first. This is to understand why context is required (and needed) in complex applications.

Components are the life-blood of your application. They will hold the UI for your application, and can sometimes hold the Business Logic and also any State which has to be maintained.

In case a component becomes too complex to express Business Logic with your UI, it is good to be able to split it into a separate bl.tsx file, with your root index.tsx importing all of the functions and handlers from it.

This structure would look like this:

components
|---page1Components
        |--Component1
        |--Component2
|---page2Component
        |--Component1
               |---index.tsx
               |---bl.tsx

In this structure, each page gets its own folder inside of components, so that it’s easy to figure out which component affects what.

It’s also important to limit the scope of a component. Hence, a component should only use adapters for data-fetching, have a separate file for complex Business Logic, and only focus on the UI part.

// components/page1Components/Component1/index.tsx

import businessLogic from "./bl.tsx";

export default function Component2() {

  const { state and functions } = businessLogic();

  return {
    // JSX
  }
}

While the BL file only imports data and returns it:

// components/page1Components/Component1/bl.tsx

import React, {useState, useEffect} from "react";
import { adapters } from "adapters/path_to_adapter";

export default function Component1Bl(){
  const [state, setState] = useState(initialState);

  useEffect(() => {
    fetchDataFromAdapter().then(updateState);
  }, [])
}

However, there’s a problem which is common across all complex apps. State Management, and how to share state across distant components. For example, consider the following file structure:

components
|---page1Components
        |--Component1
               |---ComponentA
|---page2Component
        |--ComponentB

If some state has to be shared across ComponentA and B in the above example, it will have to be passed through all the intermediate components, and also to any other components who want to interact with the state.

To solve this, their are several solutions which can be used like Redux, Easy-Peasy, and React Context, each of them having their own pros and cons. Generally, React Context should be “good enough” to solve this problem. We store all of the files related to context in contexts.

3. Contexts

The contexts folder is a bare minimum folder only containing the state which has to be shared across these components. Each page can have several nested contexts, with each context only passing the data forward in a downward direction. But to avoid complexity, it is best to only have a single context file. This structure will look like this:

contexts
|---page1Context
        |---index.tsx (Exports consumers, providers, ...)
        |---Context1.tsx (Contains part of the state)
        |---Context2.tsx (Contains part of the state)
|---page2Context
        |---index.tsx (Simple enough to also have state)

In the above case, since page1 may be a bit more complex, we allow some nested context by passing the child context as a child to the parent. However, generally a single index.tsx file containing state and exporting relevant files should be enough.

I won’t go into the implementation part of React state management libraries since each of them are their own beasts and have their own upsides and downsides. So, I recommend going through the tutorial of whatever you decide to use to learn their best practises.

The context is allowed to import from adapters to fetch and react to external effects. In case of React Context, the providers are imported inside pages to share state across all components, and something like useContext is used inside these components to be able to utilize this data.

Moving on to the final major puzzle-piece, pages.

4. Pages

I want to avoid being biased to a framework for this piece, but in general, having a specific folder for route-level components to be placed is a good practise.

Gatsby & NextJS enforce having all routes in a folder named pages. This is quite a readable way of defining route-level components, and mimicking this in your CRA-generated application would also result in better code readability.

A centralized location for routes also helps you utilize the “Go To File” functionality of most IDEs by jumping to a file by using (Cmd or Ctrl) + Click on an import.

This helps you move through the code quickly and with clarity of what belongs where. It also sets a clear hierarchy of differentiation between pages and components, where a page can import a component to display it and do nothing else, not even Business Logic.

However, it’s possible to import Context Providers inside of your page so the child components can consume it. Or, in the case of NextJS, write some server-side code which can pass data to your components using getServerSideProps or getStaticProps.

5. Styles

Finally, we come to styles. Although my go-to way is to just embed styles inside of the UI by using a CSS-in-JS solution like Styled-Components, it’s sometimes helpful to have a global set of styles in a CSS file.

A plain old CSS file is more shareable across projects, and can also affect the CSS of components which styled-components can’t reach (for example, third-party components).

So, you can store all of these CSS files inside of the styles folder, and import or link to them freely from wherever you wish.

Those were my thoughts. Feel free to email me in case you want to discuss something or have any more inputs on how this can be improved!

For further updates or discussions, you can follow me on Twitter here.

My last article on freeCodeCamp was written on how you can get started with Deno by building a URL shortener, which you can read here.

Environment variables are one of the foundational concepts for application developers. And they're something we use on a daily basis.

Environment variables have even claimed a part in the de-facto twelve-factor app. They have a long list of benefits that includes application configurability and security, which are covered in many resources like this one, or even this one from StackOverflow.

Environment variables are great and I'm completely behind the idea. However, everything comes at a cost – and environment variables, if wielded carelessly, can have harmful effects on our codebases and applications.

The curse of environment variables

How could environment variables be a bad thing if they help us write more secure code and more easily configure our applications for different environments?

Funny enough, environment variables' shortcomings actually derive from their very nature that makes them great: they're global and external, through which application developers are able to inject configurations and manage these secrets somewhere that's more difficult to compromise.

As developers, we all know how bad global states are for our applications. And please don't take my word for it, these evils have been discussed in a lot of places like here, here, and here.

For this article, I will be focusing on the 2 main flaws that I mostly encounter when dealing with environment variables:

• Inflexibility / Poor testability
• Code comprehension / readability

How to use environment variables properly

Similarly to how I deal with global variables or global patterns (like singleton) that are applied in bad locations, my favourite weapon is dependency injection.

It's not going to be the exact same thing that we do to code dependencies, but the principles are the same. Instead of using environment variables (dependencies) directly, we inject them at callsites (that is, the place they are actually used). This inverts the relationship from "callsites depending" to "callsites requiring".

Dependency injection solves these issues by:

• allowing developers to inject configurations more easily at test time
• reducing the mental scope for code readers to the package only, eliminating all externalities

So how do we apply these principles?

I will be using a Node.js example to demonstrate how we can refactor a codebase and eliminate scattered environment variables.

Hypothetical situation

Let's say we have a simple app with a single endpoint that will query for all TODOs in a PostGres database. Here is our database module with environment variables scattered into it:

const { Client } = require("pg");

function Postgres() {
  const c = new Client({
    connectionString: process.env.POSTGRES_CONNECTION_URL,
  });
  this.client = c;
  return this;
}

Postgres.prototype.init = async function () {
  await c.connect();
  return this;
};

Postgres.prototype.getTodos = async function () {
  return this.client.query("SELECT * FROM todos");
};

module.exports = Postgres;

and this module will be injected into our HTTP controller through the application's entrypoint:

const express = require("express");
const TodoController = require("./controller/todo");
const Postgres = require("./pg");

const app = express();

(async function () {
  const db = new Postgres();
  await db.init();
  const controller = new TodoController(db);
  controller.install(app);

  app.listen(process.env.PORT, (err) => {
    if (err) console.error(err);
    console.log(`UP AND RUNNING @ ${process.env.PORT}`);
  });
})();

Glancing at the above entry point file, we have no way of telling what the app's requirements for environment variables (or environment config in general) are (minus point for code glanceability 👎 ).

Refactoring the code

The first step to improve the previously laid-out code is to identify all locations that environment variables are being used directly.

For our specific case above, it's pretty straightforward as the codebase is small. But for larger codebases, you can use linting tools like eslint to scan for all locations that use environment variables directly. Just set up a rule, for example, forbidding environment variables (like node/no-process-env from eslint-plugin-node).

Now it's time to remove direct uses of environment variables from our app's modules and include these configurations as part of the module's requirements:

...
function Postgres(opts) {
  const { connectionString } = opts;
  const c = new Client({
    connectionString,
  });
  this.client = c;
  return this;
}
...

These configurations will then be supplied only from our application's entrypoint:

...
const db = new Postgres({
  connectionString: process.env.POSTGRES_CONNECTION_URL,
});
...

It's much clearer what the environment requirements for our application are now, looking at the entrypoint. This prevents potential problems with forgotten-to-be-added environment variables.

The full source code for the above demo can be found here.

Bonus: Frequently-asked-questions

These are some of the questions that I think might be asked by those reading this post. Maybe they're not actual frequently asked questionsk, but hey what's the harm in addressing possible alternative opinions?

Why not use a central config file/module?

I have seen quite a few attempts to solve this problem using a central location for drawing out these values (like a config.js file/module for Node projects).

But this approach is no better than actually using the environment variables provided by the application's runtime (like process.env) because everything is still consolidated in a somewhat global state (a single config object used throughout the app).

In fact, it could be even worse, as now we're introducing another location for code to rot.

What if I want a zero-config setup for my module?

Yes, who doesn't love zero-config, ready-to-roll modules. Again, I would like to re-iterate that building software is all about making trade-offs, and this comes at the cost of readability as this whole post has been discussing.

If you would still like a possible zero-config setup, I would suggest having config objects (that is, the opts constructor argument in the prior code example) and direct environment variable usage only as a fallback, something like this:

function Postgres(opts) {
  const connectionString =
    opts.connectionString || process.env.POSTGRES_CONNECTION_URL;
  const c = new Client({
    connectionString,
  });
  this.client = c;
  return this;
}

This way, readers of our code will still be able to recognise (although with less glanceability, as it's been traded for zero-configurability) the module's requirements.

Thank you for reading!

Last but not least, if you like my writings, please head over to my blog for similar commentaries and follow me on Twitter. 🎉

"Any fool can write code that a computer can understand. Good programmers write code that humans can understand." – Martin Fowler

Writing clean, understandable, and maintainable code is a skill that is crucial for every developer to master.

In this post, we will look at the most important principles to improve code quality and I will give you code examples for each of them.

Most examples are taken from Robert J. Martin's Clean Code. It is a programming classic and I suggest you read the whole text when you have time.

How to Name Variables (and other things)

"There are only two hard things in Computer Science: cache invalidation and naming things." – Phil Karlton

There is a reason why we do not use memory addresses and have names instead: names are much easier to recall. And, more importantly, they can give you more information about the variable, so someone else can understand its significance.

It can take some time to find a good name but it will save you and your team even more time in the future. And I am sure most readers have faced the situation where you visit your code only a few months later and have a hard time understanding what you did before.

How to Create Meaningful Names

Do not use comments to explain why a variable is used. If a name requires a comment, then you should take your time to rename that variable instead of writing a comment.

"A name should tell you why it exists, what it does, and how it is used. If a name requires a comment, then the name does not reveal its intent." – Clean Code

Bad:

var d; // elapsed time in days

I have seen this type of code so many times. It is a common misconception that you should hide your mess with comments. Do not use letters like x, y, a, or b as variable names unless there is a good reason (loop variables are an exception to this).

Good:

var elapsedTimeInDays;
var daysSinceCreation;
var daysSinceModification;

These names are so much better. They tell you what is being measured and the unit of that measurement.

Avoid Disinformation

Be careful about words that mean something specific. Do not refer to a grouping of accounts as accountList unless its type is actually a List. The word has a specific meaning and it may lead to false conclusions.

Even if the type is a list, accounts is a simpler and better name.

Bad:

var accountList = [];

Good:

var accounts = []

Avoid Noise Words

Noise words are the words that do not offer any additional information about the variable. They are redundant and should be removed.

Some popular noise words are:

• The (prefix)
• Info
• Data
• Variable
• Object
• Manager

If your class is named UserInfo, you can just remove the Info and make it User. Using BookData instead of Book as class name is just a no-brainer, as a class stores Data anyways.

You can also read Jeff Atwood's blog post about SomethingManager naming here.

Use Pronounceable Names

If you can't pronounce a name, you can't discuss it without sounding silly.

Bad:

const yyyymmdstr = moment().format("YYYY/MM/DD");

Good:

const currentDate = moment().format("YYYY/MM/DD");

Use Searchable Names

Avoid using magic numbers in your code. Opt for searchable, named constants. Do not use single-letter names for constants since they can appear in many places and therefore are not easily searchable.

Bad:

if (student.classes.length < 7) {
   // Do something
}

Good:

if (student.classes.length < MAX_CLASSES_PER_STUDENT) {
    // Do something
}

This is much better because MAX_CLASSES_PER_STUDENT can be used in many places in code. If we need to change it to 6 in the future, we can just change the constant.

The bad example creates question marks in the reader's mind, like what is the importance of 7?

You should also make use of your language's constant naming and declaration conventions such as private static final in Java or const in JavaScript.

Be Consistent

Follow the one word for each concept rule. Do not use fetch, retrieve, and get for the same operation in different classes. Choose one of them and use it all over the project so people who maintain the codebase or the clients of your API can easily find the methods they are looking for.

How to Write Functions

Keep them Small

Functions should be small, really small. They should rarely be 20 lines long. The longer a function gets, it is more likely it is to do multiple things and have side effects.

Make Sure They Just Do One Thing

Functions should do one thing. They should do it well. They should do it only. – Clean Code

Your functions should do only one thing. If you follow this rule, it is guaranteed that they will be small. The only thing that function does should be stated in its name.

Sometimes it is hard to look at the function and see if it is doing multiple things or not. One good way to check is to try to extract another function with a different name. If you can find it, that means it should be a different function.

This is probably the most important concept in this article, and it will take some time to get used to. But once you get the hang of it, your code will look much more mature, and it will be more easily refactorable, understandable, and testable for sure.

Encapsulate Conditionals in Functions

Refactoring the condition and putting it into a named function is a good way to make your conditionals more readable.

Here is a piece of code from a school project of mine. This code is responsible for inserting a chip on the board of the Connect4 game.

The isValidInsertion method takes care of checking the validity of the column number and allows us the focus on the logic for inserting the chip instead.

public void insertChipAt(int column) throws Exception {
        if (isValidInsertion(column)) {
            insertChip(column);
            boardConfiguration += column;
            currentPlayer = currentPlayer == Chip.RED ? Chip.YELLOW : Chip.RED;
        } else {
            if (!columnExistsAt(column))
                throw new IllegalArgumentException();
            else if (isColumnFull(column - 1) || getWinner() != Chip.NONE)
                throw new RuntimeException();
        }
    }

Here is the code for isValidInsertion, if you are interested.

    private boolean isValidInsertion(int column) {
        boolean columnIsAvailable = column <= NUM_COLUMNS && column >= 1 && numberOfItemsInColumn[column - 1] < NUM_ROWS;
        boolean gameIsOver = getWinner() != Chip.NONE;
        return columnIsAvailable && !gameIsOver;
    }

Without the method, if condition would look like this:

if (column <= NUM_COLUMNS
 && column >= 1
 && numberOfItemsInColumn[column - 1] < NUM_ROWS 
 && getWinner() != Chip.NONE)

Gross, right? I agree.

Fewer Arguments

Functions should have two or fewer arguments, the fewer the better. Avoid three or more arguments where possible.

Arguments make it harder to read and understand the function. They are even harder from a testing point of view, since they create the need to write test cases for every combination of arguments.

Do not use Flag Arguments

A flag argument is a boolean argument that is passed to a function. Two different actions are taken depending on the value of this argument.

For example, say there is a function that is responsible for booking tickets to a concert and there are 2 types of users: Premium and Regular. You can have code like this:

    public Booking book (Customer aCustomer, boolean isPremium) {
      if(isPremium) 
       // logic for premium book
      else
       // logic for regular booking
    }

Flag arguments naturally contradict the principle of single responsibility. When you see them, you should consider dividing the function into two.

Do Not Have Side Effects

Side effects are unintended consequences of your code. They may be changing the passed parameters, in case of passing by reference, or maybe changing a global variable.

The key point is, they promised to do another thing and you need to read the code carefully to notice the side-effect. They can result in some nasty bugs.

Here is an example from the book:

public class UserValidator {
      private Cryptographer cryptographer;
      public boolean checkPassword(String userName, String password) { 
        User user = UserGateway.findByName(userName);
        if (user != User.NULL) {
          String codedPhrase = user.getPhraseEncodedByPassword();
          String phrase = cryptographer.decrypt(codedPhrase, password);
          if ("Valid Password".equals(phrase)) {
            Session.initialize();
            return true; 
          }
        }
        return false; 
      }
}

Can you see the side-effect of this function?

It is checking the password, but when the password is valid, it is also initializing the session which is a side-effect.

You can change the name of the function to something like checkPasswordAndInitializeSession to make this effect explicit. But when you do that, you should notice that your function is actually doing two things and you should not initialize the session here.

Don't Repeat Yourself

Code repetition may be the root of all evil in software. Duplicate code means you need to change things in multiple places when there is a change in logic and it is very error prone.

Use your IDE's refactoring features and extract a method whenever you come across a repeated code segment.

IntelliJ Extract Method

Bonus

Do not leave code in comments

Please, do not. This one is serious because others who see the code will be afraid to delete it because they do not know if it is there for a reason. That commented out code will stay there for a long time. Then when variable names or method names change, it gets irrelevant but still nobody deletes it.

Just delete it. Even if it was important, there is version control for that. You can always find it.

Know your language's conventions

You should know your language's conventions in terms of spacing, comments, and naming things. There are style guides available for many languages.

For example, you should use camelCase in Java but snake_case in Python. You put opening braces on a new line in C# but you put them on the same line in Java and JavaScript.

These things change from language to language and there is no universal standard.

Here are some useful links for you:

• Python Style Guide
• Google's Javascript Style Guide
• Google Java Style Guide

Conclusion

Clean coding is not a skill that can be acquired overnight. It is a habit that needs to be developed by keeping these principles in mind and applying them whenever you write code.

Thank you for taking your time to read and I hope it was helpful.

If you are interested in reading more articles like this, you can subscribe to my blog.

Developers are usually concerned with the quality of their code. There are different kinds of tests that help us avoid breaking code when a new feature is added in a project. But what can we do to ensure that components don't look different over time?

In this post, you will learn how to use Cypress to capture parts of pages of a website. After that, you will integrate the testing tool in CI to ensure that in the future no one will make unwanted changes to your project.

My motivation for creating this testing strategy came from work. At Thinkific we have an internal Design System and we added Cypress to avoid surprises when working in CSS/JS files.

By the end of this post we will have PRs with Cypress tests:

Before we start

I created a sample website to mimic a Component Library. It is a very simple website created with TailwindCSS and hosted in Vercel. It documents 2 components: badge and button.

You can check out the source code in GitHub. The website is static and it is inside the public folder. You can see the website locally by running npm run serve and checking in the browser http://localhost:8000.

Adding Cypress and Cypress Image Snapshot

Start by cloning the example repository. Next, create a new branch and install Cypress Image Snapshot, the package responsible for capturing/comparing screenshots.

git checkout -b add-cypress
npm install -D cypress cypress-image-snapshot

After adding the packages, a few extra steps are needed to add Cypress Image Snapshot in Cypress.

Create a cypress/plugins/index.js file with the following content:

const { addMatchImageSnapshotPlugin } = require('cypress-image-snapshot/plugin');

module.exports = (on, config) => {
  addMatchImageSnapshotPlugin(on, config);
};

Next, create a cypress/support/index.js file containing:

import { addMatchImageSnapshotCommand } from 'cypress-image-snapshot/command';

addMatchImageSnapshotCommand();

Creating the screenshot test

Time to create the screenshot test. Here is the plan:

1. Cypress will visit each page (badge and button) of the project.
2. Cypress will take a screenshot of each example in the page. The Badge page has 2 examples (Default and Pill), while the Button page has 3 examples (Default, Pill and Outline). All these examples are inside a <div> element with a cypress-wrapper. This class was added with the only intention being to identify what needs to be tested.

The first step is creating Cypress configuration file (cypress.json):

{
  "baseUrl": "http://localhost:8000/",
  "video": false
}

The baseUrl is the website running locally. As I mentioned before, npm run serve will serve the content of the public folder. The second option, video disables Cypress video recording, which we won't use in this project.

Time to create the test. In cypress/integration/screenshot.spec.js, add:

const routes = ['badge.html', 'button.html'];

describe('Component screenshot', () => {
  routes.forEach((route) => {
    const componentName = route.replace('.html', '');
    const testName = `${componentName} should match previous screenshot`;

    it(testName, () => {
      cy.visit(route);

      cy.get('.cypress-wrapper').each((element, index) => {
        const name = `${componentName}-${index}`;

        cy.wrap(element).matchImageSnapshot(name);
      });
    });
  });
});

In the code above, I am dynamically creating tests based in the routes array. The test will create one image per .cypress-wrapper element that the page has.

Last, inside the package.json let's create the command to trigger the tests:

{
  "test": "cypress"
}

From here, there are 2 options: run Cypress in headless mode with npm run cypress run or use the Cypress Test Runner with npm run cypress open.

Headless option

Using npm run cypress run, the output should be similar to the next image:

The tests will pass and 5 images will be created under the /snapshots/screenshot.spec.js folder.

Test Runner option

Using npm run cypress open, Cypress Test Runner will be opened and you can follow the tests step by step.

Our first milestone is done, so let's merge this branch to master. If you want to see the work done so far, jump in my Pull Request.

Using Cypress inside Docker

If you run the test above alternating between headless and Test Runner, you may notice that the screenshot will vary.

Using the Test Runner with a retina display computer, you may get retina images (2x), while the headless mode doesn't give you high-quality screenshots.

Also, it is important to say that the screenshots may vary according to your Operating System.

Linux and Windows, for instance, have apps with visible scrollbars, while macOS hides the scrollbar.

If the content captured in the screenshot doesn't fit a component, you may or may not have a scrollbar. If your project relies on OS default fonts, screenshots will also be different according to the environment.

In order to avoid these inconsistencies, tests will run inside Docker so the developer's computer won't affect the screenshot captures.

Let's start by creating a new branch:

git checkout -b add-docker

Cypress offers different Docker images - you can check out the details in their documentation and their blog.

For this example, I will use the cypress/included image, which includes Electron and is ready to be used.

We need to make two changes: change the baseUrl in the cypress.json file:

{
  "baseUrl": "http://host.docker.internal:8000/",
}

and the test command in the package.json file:

{
  "test": "docker run -it -e CYPRESS_updateSnapshots=$CYPRESS_updateSnapshots --ipc=host -v $PWD:/e2e -w /e2e cypress/included:4.11.0"
}

Running npm run test will bring us a problem:

The images are slightly different but why? Let's see what is inside the __diff_output__ folder:

As I mentioned earlier, typography inconsistencies! The Button component uses the OS default font. Since Docker is running inside Linux, the rendered font won't be the same that I have installed on macOS.

Since now we moved to Docker, these screenshots are outdated. Time to update the snapshots:

CYPRESS_updateSnapshots=true npm run test

Please notice that I am prefixing the test command with the environment variable CYPRESS_updateSnapshots.

The second milestone is done. In case you need help, check out my pull request.

Let's merge this branch and move forward.

Adding CI

Our next step is adding the tests in CI. There are different CI solutions in the market but for this tutorial, I will use Semaphore. I am not affiliated with them and I use their product at work, so it was for me a natural choice.

The configuration is straightforward and it can be adapted to other solutions like CircleCI or Github Actions.

Before we create our Semaphore configuration file, let's prepare our project to run in CI.

The first step is installing start-server-and-test. As the package name says, it will start a server, wait for the URL, and then run a test command:

npm install -D start-server-and-test

Second, edit the package.json file:

{
  "test": "docker run -it -e CYPRESS_baseUrl=$CYPRESS_baseUrl -e CYPRESS_updateSnapshots=$CYPRESS_updateSnapshots --ipc=host -v $PWD:/e2e -w /e2e cypress/included:4.11.0",
  "test:ci": "start-server-and-test serve http://localhost:8000 test"
}

In the test script, we are adding the CYPRESS_baseUrl environment variable. This will allow us to change the base URL used by Cypress dynamically. Also, we are adding the test:ci script, which will run the package we just installed.

We are ready for Semaphore. Create the .semaphore/semaphore.yml file with the following content:

 1 version: v1.0
 2 name: Cypress example
 3 agent:
 4   machine:
 5     type: e1-standard-2
 6     os_image: ubuntu1804
 7 blocks:
 8   - name: Build Dependencies
 9     dependencies: []
10     task:
11       jobs:
12         - name: NPM
13           commands:
14             - sem-version node 12
15             - checkout
16             - npm install
17   - name: Tests
18     dependencies: ['Build Dependencies']
19     task:
20       prologue:
21         commands:
22           - sem-version node 12
23           - checkout
24       jobs:
25         - name: Cypress
26           commands:
27             - export CYPRESS_baseUrl="http://$(ip route | grep -E '(default|docker0)' | grep -Eo '([0-9]+\.){3}[0-9]+' | tail -1):8000"
28             - npm run test:ci

Breaking the configuration down in detail:

• Lines 1-6 define which kind of instance we will use in their environment
• Lines 8 and 16 create 2 blocks: the first block, "Build Dependencies" will run npm install, downloading the dependencies we need. The second block, "Tests" will run Cypress, with a few differences.
• In line 27, we are dynamically setting the CYPRESS_baseUrl environment variable based on the IP Docker is using at the moment. This will replace http://host.docker.internal:8000/, which may not work in all environments.
• In line 28, we finally run the test using start-server-and-test: once the server is ready for connections, Cypress will run the test suite.

Another milestone is done, time to merge our branch! You can check out the Pull request that contains all the files from this section and check the build inside Semaphore.

Recording the tests in cypress.io

Reading the output of tests in CI is not very friendly. In this step, we will integrate our project with cypress.io.

The following steps are based on the Cypress documentation.

Let's start by getting a project ID and a record key. In the terminal, create a new branch and run:

git checkout -b add-cypress-recording
CYPRESS_baseUrl=http://localhost:8000 ./node_modules/.bin/cypress open

Earlier I mentioned that we would be using Cypress inside Docker. But here we are opening Cypress locally since this is the only way to integrate with the website dashboard.

Inside Cypress, let's go to the Runs tab, click in "Set up project to record", and choose a name and visibility. We will get a projectId that is automatically added in the cypress.json file and a private record key. Here is a video of the steps:

In Semaphore, I added the record key as an environment variable called CYPRESS_recordKey. Next let's update our test script to use the variable:

{
  "test:ci": "start-server-and-test 'serve' 8000 'npm run test -- run --record --key $CYPRESS_recordKey'"
}

That is pretty much all that needs to be done. In the Pull request we can see the cypress.io integration in the comments. There is even a deep link that takes us to their dashboard and shows all the screenshots. Check out the video below:

Time to merge our work, and that is the end of our integration.

Testing in real life

Imagine we are working on a change that affects the padding of the buttons: time to test if Cypress will capture the difference.

In the example website, let's double the horizontal padding from 16px to 32px. This change is quite simple since we are using Tailwind CSS: px-4 gets replaced by px-8. Here is that Pull request.

As we might expect, Cypress captured that the button doesn't match the screenshots. Visiting the page, we can check the screenshot of the broken test:

The diff file shows the original screenshot on the left, the current result on the right, and they are combined in the middle. We also have the option to download the image so we can see the issue better:

If this is not an issue, update the screenshots:

CYPRESS_updateSnapshots=true npm run test

The end

That's it for today. I hope you have learned how Cypress can be useful to ensure no one is adding unexpected changes to a project.

Also posted on my blog. If you like this content, follow me on Twitter and GitHub.

In this article we'll explore what kinds of things you can do to make it a little easier on your future self.

Revisiting "prior art"

We've all been there. Six months into a project, you're trying to squash a bug, and what you find is shocking. You might be asking yourself, "who would write this kind of code?"

So you dig through your git commit history using git log -p filename.js showing changes for a specific file, trying to see who would come up with something like that. And then your heart drops – you're the one who wrote it!

Tituss Burgess shocked

This is a common scenario for any developer, experienced or new. If you haven't hit that scenario, I promise if you stick with coding long enough, you will.

Becoming more conscious about our coding habits

That six month reflection point is inevitable. That's a lot of time which you've most likely been using to work on other parts of the project or another project completely. Chances are, you've leveled up which has changed the way you write code.

On the other hand, sometimes it takes stepping outside of the code to see the bigger picture and get a better look at how all the pieces fit together. We can naturally dig ourselves too deep into a solution and can become a little narrowly focused as we work to solve those challenges.

But either way, while part of the code journey will simply be gaining more experience and learning more about your craft, there are other small habits you can get used to early on that will help you down the road.

So let's jump in.

Improving the readability of your code

What is the challenge?

Part of the fun about our craft is there are a ton of ways you can do the same thing. Think an if statement is too many lines? Well we can write it ternary style!

// Example ternary statement
const isFreezing = temperature <= 32 ? true : false;

But sometimes this takes a toll on the readability of your code. While it might seem like it looks nice and super clean on one line, imagine that as that ternary gets more complex, someone will have to spend more time understanding what it means.

const minutes = 30;
const cookie = {
  color: 'black'
};

const cookieStatus = minutes > 20 ? cookie.color === 'black' ? 'burned' : 'done' : 'not done';

What can we do better?

Now I would imagine most of us can figure out what cookieStatus is in this example (spoilers: burned). But think about the time you spent figuring it out. Whether an extra 1s or 2s, it forces you to spend additional cognitive energy to read through the code.

On the other hand:

const minutes = 30;
const cookie = {
  color: 'black'
};
let cookieStatus;

if ( minutes <= 20 ) {
  cookieStatus = 'not done';
} else if ( cookie.color === 'black' ) {
  cookieStatus = 'burned';
} else {
  cookieStatus = 'done';
}

No, it may not be as clean or clever as the 1-line ternary statement, but the next time you visit it, the less you'll have to think about what the answer is.

It's also going to make it that much easier for bugs to creep up and get past your code reviewers when all of your code changes are in a 1-line git diff.

And yes, this is a simple example. But imagine this in a real world scenario with important business logic where you could run into these situations frequently.

Say you need to add another condition – that ternary is just going to keep getting more complex! You're just making it more difficult to debug or extend, where the if statements can continue on in an easily readable way.

For what it's worth, ternaries and other shortcuts can be simple and effective in code, but don't abuse that effectiveness and end up making things more difficult.

Keeping things consistent

What is the challenge?

We all have our favorite way to code. Though I'd argue not including a semicolon at the end of your JavaScript is just completely wrong, you might prefer to write your code the wrong way without them.

// Jim's code style

function MyComponent() {
  function handleOnClick() {
    alert('Click!')
  }
  return (
    <button onClick={handleOnClick}>My Button</button>
  )
}

// Creed's code style

const MyComponent = () => <button onClick={() => alert('Click!')}>My Button</button>;

But it's not always about what you prefer. When working with a team, chances are, everyone's opinion on how code should look is slightly different. You might agree about the semi-colon, but disagree about whitespace.

And no one is wrong (except for the non-semi-coloners)! There are valid arguments to most code styles, whether for or against, but the solution isn't for everyone to write their code their own way.

What can we do better?

Keeping code consistent is important to maintain code health. A typical goal is to "make the codebase look like one person wrote it."

The point isn't that one person gets their way, it's that the team came to a conclusion about a set of rules they would use that everyone would follow. Having this consistency provides less cognitive overhead as people work through the code. It gives everyone the ability to know what to expect when reading the code.

Errors from linting code

And achieving this doesn't need to be hard. There are tools that can simply check for these inconsistencies like Eslint for Javascript. And even better, there is another level of tools like Prettier that will fix it for you!

Commenting your code

What is the challenge?

Keeping up with commenting your code is an important way to put context to complex logic. As much as we all want our code to be self-documenting, that's rarely the case.

Too often we find ourselves dealing with a block of code that just doesn't make sense. And even if it makes sense on its own, we might not be able to figure out how it fits in to the rest of the application.

What can we do better?

By providing a good set of comments, you're setting up the next person who touches that code to have a better understanding of what the code is doing before they make a change.

// DONT CHANGE - WILL STOP MAKING MONEY

const shouldMakeMoney = true;

function makeMoney() {
  if ( shouldMakeMoney ) {
    return noMoney;
  }
  return moreMoney;
}

While this is a silly example, it brings up a real world case. Businesses are increasingly dependent on being able to maintain a reliable website to make money. Whether this is as an ecommerce business or an ad giant, these websites rely on business logic that determine things like cost, taxes, discounts, and other math related things we tend to not want to think about, but could make or break a business on the internet.

But it's not all about the company you work for. Touching old code can be scary. It's even scarier when no one on your team was around when it was written, so no one knows what it does!

Patton Oswalt hands over mouth

While you might not be the next person who touches that code, try to help out your future friend who's tackling the next ticket involving that code. Because there's also a good chance you will be that person and you'll wish you remembered how it worked.

Documenting your solutions

What is the challenge?

Documentation is similar to commenting your code, but from a different perspective. Documentation and commenting are both about finding ways to describe a solution in a human-readable way that will ultimately give more context. But documentation is more about the overall solution rather than the implementation details.

Having a high performing application is everyone's goal. But how did we get there? There's a realistic chance that someone will have to be working on the same project you are like onboarding a new team member. How will they be able to maintain that high performance if they don't know how it works?

What can we do better?

Whether it's introducing that new team member to the project or trying to share knowledge with another project team, documentation is an important part of maintaining a project. It helps keep everyone on the same page so we all confidently know what we're working towards.

For example, if we're still working with our ecommerce project with our business logic, there will be rules that the code needs to implement. While commenting might give details inline about how the rules were implemented, the documentation would define those rules.

/**
 * DOCUMENTATION
 * Order Total >= 25: Discount %10
 * Order Total >= 50: Discount %15
 * Order Total >= 100: Discount %20
 * Order Total >= 75: Free Shipping
 */

const orderSubTotal = 84.00;
let orderTotal = orderSubTotal;

// If the order total is under 75, apply shipping discount

if ( orderTotal < 75 ) {
  orderTotal = addShipping(orderTotal);
}

// If the order total reaches a threshold, apply given discount

if ( orderTotal >= 100) {
  orderTotal = applyDiscount(orderTotal, .2);
} else if ( orderTotal >= 50 ) {
  orderTotal = applyDiscount(orderTotal, .15);
} else if ( orderTotal >= 25 ) {
  orderTotal = applyDiscount(orderTotal, .1);
}

This is a minimal example, but we can see the difference between the rules at the top and how we apply them. The documentation should clearly explain what the rules are, but it shouldn't care about how those rules were implemented.

On the other hand, the comments might not care about what the rules are, but need to implement them in an efficient and logical way. We should be able to update the code with the business rules, such as changing the top level discount tier from $100 to $80, without having to rework the code.

But documentation is much more than business rules – it's providing a way for anyone to understand your work from a higher level. This could include anything from architectural diagrams to the theory behind your core algorithm.

While maybe code isn't the best place for details like this to live, it's really important information that can help instill confidence in your project and give others an opportunity to understand more about the work.

Creating effective Pull Requests

What is the challenge?

Pull requests (or merge requests) are a core part of any development team's project lifecycle. It provides a way to package and present your code in a consumable way for your peers to review and understand your work.

There's a lot that can go into a pull request from a single commit to the entirety of the next version of your website. That's a lot of context to expect someone to understand by reading through the commits alone.

What can we do better?

Pull requests don't need to be an art. There should be one primary goal of the preparation you put into it – providing context into your changes. At a minimum, it should answer the questions of "what" and "why".

We can even use tools like pull request templates to push us in the right direction. Define an outline of what you want explained and chances are, people will follow that outline. This helps avoid the 1-line "closes [ticket]" description or even worse, an empty description.

With my projects, I hope to have a few questions answered before I dive into a code review:

• What is the change?
• What does it impact?
• How do I reproduce or test the change?

Just a few details around the change set can provide much needed context for those reviewing your code. It's easy to look at code, but it's harder to understand it without knowing how it fits into the bigger picture.

Hardening your code with tests

What is the challenge?

Tests are a way to ensure your code runs the same way each time. Being able to prove that the same input will always have the same output will help provide you and your team with a higher level of confidence that your application won't come crashing down with the next small change.

Without them, we're left with human error, where no matter how good your QA Engineer is (shoutout to my testers out there), something will always slip through. And that's not to say your tests will always catch every problem, but we can use the tools available to help prevent it.

What can we do better?

Where comments are a way of providing the context of how something works, test are a way to ensure they work. Providing test cases that are repeatable helps enforce that.

function applyDiscount(value, discount) {
  const discountAmount = value * discount;
  return value - discountAmount;
}

expect(applyDiscount(10, .1)).toEqual(.9);
expect(applyDiscount(532151235, .1054)).toEqual(476062494.831);

If I fudge the math on our applyDiscount function above, there's a high probability that the test would fail (never say never).

But testing doesn't need to be hard. There are many tools out there that help from different perspectives. For example, you might use Jest to run your unit tests or add Enzyme on top to test your React components. But you can also bring in Cypress as an integration test solution that will work like a robot clicking through your application to make sure all the components actually work together.

There's also different methodologies of testing. While you probably see most teams write their tests after they have a working solution, some people swear by test-driven development. They might write their tests first where the code has to pass the tests rather than the other way around. This is a great way to define the requirements of the code before diving right in.

Whatever the method, capture the points that are most likely to break or the functions that add the most business value. You'll be helping to prevent a potential business loss or even simpler, a headache.

What can we learn from this?

That might be a lot to digest, but they're important points to consider as you grow as a developer. Starting these habits early in your career will help you naturally build these skills and work that way by default.

And if you're late in your career, it's never too late to start. We should all want to strive to be the best developer we can be and do our best to help make our teammate's lives easier, as we're all in this together.

Looking for more to learn?

• Put Down the Javascript - Learn HTML & CSS
• How to Become a Full Stack Web Developer in 2020
• What is the JAMstack and how do I get started?
• What is linting and how can it save you time?
• Why you should write merge requests like you’re posting to Instagram

What’s your advice for growing as a developer?

Share with me on Twitter!

I was at a software craftsmanship meetup recently, where we formed pairs to solve a simplified Berlin Clock Kata. A Berlin Clock displays the time using rows of flashing lights, which you can see below (although in the kata we just output a text representation, and the lights in a row are all the same colour).

Initial Test Driven solution

Most pairs used inside out TDD, and there were a lot of solutions that looked something like this (complete code available on GitHub).

def berlin_clock_time(julian_time):
    hours, minutes, seconds = list(map(int, julian_time.split(":")))

    return [
        seconds_row_lights(seconds % 2)
        , five_hours_row_lights(hours)
        , single_hours_row_lights(hours % 5)
        , five_minutes_row_lights(minutes)
        , single_minutes_row_lights(minutes % 5)
    ]

def five_hours_row_lights(hours):
    lights_on = hours // 5
    lights_in_row = 4
    return lights_for_row("R", lights_on, lights_in_row)

# ...

This type of solution drops out naturally from applying inside out TDD to the problem. You write some tests for the seconds row, then some tests for the five hours row, and so on, and then you put it all together and do some refactoring. This solution does expose some of the domain concepts at a glance:

• There are 5 rows
• There is one second row, 2 hour rows and 2 minute rows

Some more concepts are available after a bit of digging, but aren't immediately obvious. The rows are made up of lights that can be on (or presumably off), and that the number of lights on is an indication of the time.

However there are some big parts of the problem that are not exposed. And since I haven't yet explained it, you probably don't know exactly how the Berlin Clock works yet.

Elevate the concepts

To improve this we can bring some of the details that are buried in the helper functions (such as get_five_hours) closer to the top of the file. This brings you to something like the following (complete code available on GitHub), although the downside is that it breaks nearly all of the tests. Solutions like this are rarer on GitHub, but do exist.

def berlin_clock_time(julian_time):
    hours, minutes, seconds = list(map(int, julian_time.split(":")))

    single_seconds = seconds_row_lights(seconds % 2)
    five_hours = row_lights(
        light_colour="R",
        lights_on=hours // 5,
        lights_in_row=4)
    single_hours = row_lights(
        light_colour="R",
        lights_on=hours % 5,
        lights_in_row=4)
    five_minutes = row_lights(
        light_colour="Y",
        lights_on=minutes // 5,
        lights_in_row=11)
    single_minutes = row_lights(
        light_colour="Y",
        lights_on=minutes % 5,
        lights_in_row=4)

    return [
        single_seconds,
        five_hours,
        single_hours,
        five_minutes,
        single_minutes
    ]

# ...

This improves the concepts that are now exposed at a glance:

• There are 5 rows
• The seconds row is a special case
• There are 2 hour rows and 2 minute rows
• The rows use different colour lights
• The rows have a different number of lights

This is pretty good, and is already better that most of the solutions out there. However, it's still a bit mysterious how the rows are related to each other (there are 2 rows to display the hours and the minutes, so presumably these are linked). It's also not obvious what amount of time each light represents.

Name implicit concepts

At the moment some of the concepts (such as the amount of time each light represents) are implicit in the code. Making these explicit, and naming them, forces us to understand them and to embed that understanding in the code.

In order to make the amount of time each light represents explicit, it seems like it would be sensible to pass a time_per_light value to row_lights. This means we have to push the calculation of lights_on down into row_lights.

This in turn makes it obvious that there are two kinds of rows: one related to the quotient (\\) of the time value, and one related to the remainder / modulus (%). If we look at the quotient case, we see that the 2nd parameter to the operation is the time_per_light, which is 5 in both cases (5 hours in one case and 5 minutes in the other).

This allows us to write these rows like this:

five_hour_row = row_lights(
    time_per_light=5,
    value=hours, 
    light_colour="R",
    lights_in_row=4)

If we now turn our attention to the remainder case, we realise that time_per_light is always singular (one hour or one minute), as it is filling in the gaps in the quotient case.

For example, the five hours row can represent 0, 5, 10, 15, or 20 hours, but nothing in between. In order to represent any hour, there must be another row to represent +1, +2, +3 and +4. This means that this row must have exactly 4 lights, and that each light must represent 1 hour.

This implies that the remainder case is dependent on the quotient one, which most people would describe as a parent / child relationship.

With this knowledge in hand, we can now create a function for the child remainder rows, and the solution now looks like this (complete code on GitHub):

def berlin_clock_time(julian_time):
    hours, minutes, seconds = list(map(int, julian_time.split(":")))

    return [
        seconds_row_lights(
            seconds % 2),
        parent_row_lights(
            time_per_light=5,
            value=hours, 
            light_colour="R",
            lights_in_row=4),
        child_remainder_row_lights(
            parent_time_per_light=5,
            value=hours,
            light_colour="R"),
        parent_row_lights(
            time_per_light=5,
            value=minutes, 
            light_colour="Y",
            lights_in_row=11),
        child_remainder_row_lights(
            parent_time_per_light=5,
            light_colour="Y",
            value=minutes)
    ]

# ...

A quick glance at this code now reveals nearly all the domain concepts

• The first row represents the seconds and is a special case
• On the second row each "R" light represents 5 hours
• The third row shows the remainder from the second
• On the fourth row each "Y" light represents 5 hours
• The fifth row shows the remainder from the fourth

This took something thinking about, which will have cost us some time / money. But we increased our understanding of the problem while we did it, and most importantly we embedded that knowledge in to the code. This means that the next person to read the code will not have to do this, which will save some time / money. Since we spend about 10 times longer reading code than we do writing it, this is probably a worthwhile endeavour.

Embedding this understanding has also made it harder for future programmers to make mistakes. For example, the concept of parent / child rows didn't exist in earlier examples, and it would be easy to mismatch them. Now the concept is plain to see, and the values are mostly worked out for you. It is also easier to refactor to support new clock variants, for example where lights in the first hours row represent 6 hours.

How far should you take it?

There are things we can do to take this further. For example the parent_time_per_light of a child row must match the time_per_light of its parent, and there is nothing enforcing this. There is also a relationship between time_per_light and lights_in_row for the parent rows, and again it is not enforced.

However, at the moment we are only required to support one clock variant, so these probably aren't worth doing. When a change is required for the code, we should refactor so that the change is easy (which might be hard) and then make the easy change.

Conclusions

Embedding domain concepts in code requires thought and skill, and TDD won't necessarily do it for you. It takes longer than a naive solution, but makes the code easier to understand, and will very likely save time in the medium term. Time is money, and finding the right balance of spending time now versus saving time later is also an important skill for a professional programmer to have.

But first, let me give you some context. I was writing a bunch of code involving third party APIs (Stripe's recurring billing and subscription APIs, to be specific), and had written a wrapper class and some server route-handlers to respond to requests from the front-end web app. The entire application is React +TypeScript + Node, with a Koa server.

As part of this, I was trying to handle the following errors:

1. Errors thrown by Stripe's API

2. Errors thrown by my wrapper class, especially when fetching user data from the database

3. Errors in route-handlers that arise from a combination of the above.

During development, my most common errors were incomplete data in the server requests and incorrect data passed to Stripe.

To help you visualize the flow of data, let me give you some background on the server-side code. Typically this is what the function call chain looked like:

Route-Handler -> Stripe Wrapper -> Stripe API

The first function being called would be in the Route-Handler, then in the Stripe Wrapper class, inside which the Stripe API method would be called. So the call stack has Route-Handler at the bottom (first called function) and the Stripe API method on the top (last called function).

The problem was that I did not understand where to put my error handling. If I did not put an error handler in the server code, then node would crash (literally, exit execution!) and the front end would receive an error HTTP response (typically a HTTP 5xx err0r). So I put a few try-catch handlers inside the various methods being called, and added logging statements inside the catch block. That way I could debug the error by tracking the logs.

An example of the calling logic:

 function stripeAPI(arg){
    console.log('this is the first function')
    if(!arg) throw new Error('no arg!')
    // else
    saveToDb()
}

function stripeWrapper(){
    console.log('this is the second function, about to call the first function')
    try{
        stripeAPI()
    } catch(err) {
//         console.log(' this error will not bubble up to the first function that triggered the function calls!')
    }
}

function routeHandler(){
    console.log('this is the third  function, about to call the second function')
    stripeWrapper()
}


function callAll(){
    try{
       routeHandler() 
       return 'done'
    } catch (err){
       console.log('error in callAll():', err)
       return ' not done '
    }

}


callAll()

The problems?

1. If I didn't log the error, I lost the error! In the above snippet, note that even though I've called first() without the required arguments, the error defined in the definition of first did not get thrown! Also, there is no saveToDb() method defined... and yet this was not caught! If you run this code above, you will see it returns 'done' - and you've got no idea that your database wasn't updated and something had gone wrong! ☠️☠️☠️

2. My console had way too many logs, repeating the same error. It also meant that in production, there was excessive logging... ?

3. The code looked ugly. Almost as ugly as my console.

4. Others who worked with code found it confusing and a debugging nightmare. ?

None of these are good outcomes, and all are avoidable.

The concepts

So, let's get some basics out of the way. I'm sure you know them but some people may not, and let's not leave them behind!

Some basic terminology:

Error - also known as an 'exception', is when something goes wrong in the node code, and the program exits immediately. Errors, if not handled, will cause the program to come to a screeching halt, and ugly messages are spewed into the console, with a long and generally hideous error stack trace message.

Throw - the throw operator is how the language handles an error. By using throw you generate an exception using the value you put after the operator. Note that the code after throw does not get executed - in that sense it is like a return statement.

Error - there is a JavaScript object called Error. An error gets 'thrown' in order to help the programmer know something needs to be handled. Think of it as a little ticking bomb ? that gets thrown from one function to another inside a chain of function calls. Technically, you can throw any data, including JavaScript primitives as an error, but it's generally a good idea to throw an Error object.

You typically construct the Error object by passing in a message string like so: new Error('This is an error'). But simply creating a new Error? object is unhelpful as that's only half the job. You've got to throw it so it can be caught. That's how it becomes useful.

Languages generally come with a standard set of errors, but you can create a custom error message with the new Error('this is my error message') constructor, and your error message should help you work out what's going on. More on Node errors.

Catch - this is what you do when someone throws something at you, right? You'd probably do it reflexively even if someone threw you one of these... ?!

The catch statement in JavaScript lets you handle an error ? that gets thrown. If you don't catch the error, then the error 'bubbles up' (or down, depending on how you view the call stack) until it reaches the first called function and there it will crash the program.

In my example an error thrown by the Stripe API will bubble up all the way to my Route-Handler function, unless I catch it somewhere along the way and deal with it. If I don't handle the error, Node will throw an uncaughtException error and then terminate the program.

Let's return to my example:

Call stack

Route-Handler -> Stripe Wrapper -> Stripe API

Error path

Stripe API (? _thrown here) -> API Wrapper (� �_not caught) -> _Route-Handler (_� �still not caught) -> ccrraashh ???

We want to avoid app crashes as it can cause your data to corrupt, your state to be inconsistent, and your user to think your app sucks. So handling errors thoughtfully requires many levels of analysis.

There are some detailed guides to error handling in JavaScript and one of my favourites is here, but I will summarize my key leanings for you here.

Try-Catch statement

Use these to gracefully handle errors, but be careful about where and when. When errors are caught and not handled properly they are lost. That 'bubbling up' process happens only up until the error encounters a catch statement. If there is a catch statement in the call chain that intercepts the error then the error won't crash the app, but not handling the error will hide it! Then it gets passed as an argument to catch and it requires you to handle it there.

try{
// code logic
} catch (error) {
// handle the error appropriately
}

So it's very important to catch and handle the error at a point where it makes the most logical sense for you when you have to debug it. It's tempting to think that you must catch it at the very first place it comes up (the last function called that sits right on the top of the call stack), but that isn't true!

Route-Handler -> Stripe Wrapper (don't catch here!) -> Stripe API

If I put my try-catch in the Stripe Wrapper which directly invokes Stripe's API, then I don't have information on where my Stripe Wrapper function was called. Maybe it was the handler, maybe it was another method inside my wrapper, maybe it was in another file altogether! In this simple example it's obviously called by Route-Handler, but in a real world app, it could be called in multiple places.

Instead, it makes sense for me to put the try-catch in the Route-Handler, which is the very first place where the function calls begin that resulted in the error. That way you can trace the call stack (also called unwinding the call stack) and drill down into the error. If I send bad data to Stripe it will throw an error, and that error will pass through my code until I catch it.

But when I catch it I need to handle it properly, or I could inadvertently conceal this error. Handling errors usually means deciding whether I need my front end user to know something has gone wrong (their payment didn't work, for example), or is it just an internal server error (for example, Stripe could not find the product ID I passed) that I need to handle gracefully without tripping up my front end users and crashing the Node code. If I added things to the database that are not correct, then I should clean up those false writes now.

When handling the error, it is a good idea to log it so I can monitor the app for bugs and failures in production and debug efficiently. So at the very, very least, handling would include logging the error in the catch statement. But...

 function stripeAPI(arg){
    console.log('this is the first function')
    if(!arg) throw new Error('no arg!')
    // else
    saveToDb()
}

function stripeWrapper(){
    console.log('this is the second function, about to call the first function')
    try {
        stripeAPI()
    } catch(err) {
        console.log('Oops!  err will not bubble up to the first function that triggered the function calls!')
    }
}

function routeHandler(){
    console.log('this is the third  function, about to call the second function')
    stripeWrapper()
}


function callAll(){
    try {
       routeHandler() 
       return 'done'
    } catch (err){  
       console.log('error in callAll():', err)
       return ' not done '
    }

}


callAll()

...as you can see above, if I catch it and log it in the middle level (my Stripe Wrapper class), it won't reach routeHandler or callAll, and my app will not know something went wrong. callAll still returns done and the only evidence something went wrong was in the log statement: 'Oops! err will not bubble up to to first function that triggered the function calls!'. Had we not put a log statement there the error would have vanished without a trace.

This is 'error hiding' and it makes debugging a pain. If I add a try-catch but don't do anything in the catch statement, I will prevent my program from crashing. But I also end up 'hiding' the problem! It usually leads to inconsistent state - parts of my server code thinks everything is OK, and tells my front end that. But another part of my server code had indicated something was wrong!

In this simple example, it's easy to unravel, but think of deeply nested called across your entire application - what a nightmare!

If you absolutely need to handle the error in the middle of your call stack, then be sure to re-throw the error appropriately. That means ending your catch statement with another throw error operation. That way the error will get thrown again and continue to 'bubble up' towards the first function (bottom of the call stack) that triggered the call chain where it can be properly handled again.

Here's what it looks like, adding just one small re-throw in the stripeWrapper() function. Run the code and see the difference in outcome because callAll() now gets passed the error!

function stripeWrapper(){
    console.log('this is the second function, about to call the first function')
    try{
        stripeAPI()
    } catch(err) {
        console.log('Oops!  err will not bubble up to to first function that triggered the function calls!')

        throw err  // add this to re-throw!

    }
}

function callAll(){
    try{
       routeHandler() 
       return 'done'
    } catch (err){  // catches the re-thrown error and prints it to console!
       console.log('error in callAll():', err)
       return ' not done '
    }

}

Since you threw the error in the middle stage, it went to the outer boundary, and got caught there. The code returns not done and you can investigate why the error says 'no arg'. You can also then see that it never executed saveToDb(), as the error threw before that code could be executed! That could be a good thing in cases where you're saving things to the database assuming that there were no errors until that point. Imagine saving things to the database that should never have been saved - that's dirty data in the database now! ???

So, don't do what I did in my early days of programming and simply log the error at every step in the call stack and re-throw it. It just means you will get multiple logs for each error as it passes through the call stack! Only intercept the error at a place where you can most efficiently and usefully handle it, ideally once in a given chain of calls.

In general, it really helps if you place your try catch statement at the outermost (first calling) function that lies at the bottom of the call stack. You can identify this as the place the error will bubble up to just before throwing an uncaughtException error. That's a good place to catch, log, and handle it.

To see the difference in handling when you don't use the try-catch simply modify callAll() to look like this:

function callAll(){
    routeHandler()  

    // this won't run!
    console.log('This function is not contained inside a try-catch, so will crash the node program.')
}

callAll()

You'll note that the console.log statement never runs here because the program crashes when routeHandler() finishes executing.

Rules of Thumb ???

So let's summarize some quick rules that will cover 90+% of your needs:

1. Do not litter your code with try-catch statements

2. Try as much as possible to catch only once in a given chain of function calls

3. Try and place that catch at the outermost boundary - the first function that starts the chain of function calls (bottom of the call stack)

4. Do not leave your catch statement empty as a way to stop your program from crashing! If you don't handle it, chances are it will lead to inconsistent state between your front end and back end. This can be dangerous and lead to a horrible user experience ?!

5. Do not use a catch statement only in the middle of the call stack, and not at the outer boundary. This will cause the error to get 'hidden' in the middle of your code where it isn't going to help you debug or manage data properly. Others who work with your code will find where you live and cut off your internet connection.

6. Catch it where you need to know, and where you can meaningfully do all the things necessary to clean things up.

Stripe API (? thrown here) -> API Wrapper (? passing through) -> Route-Handler (? caught, handled, logged) -> ???

Thanks for reading!

If you would like to learn more about my journey into code, check out episode 53 of the freeCodeCamp podcast, where Quincy (founder of freeCodeCamp) and I share our experiences as career changers that may help you on your journey. You can also access the podcast on iTunes, Stitcher, and Spotify.

I will also hold a few AMAs and webinars in the coming months. If this is of interest to you please let me know by going here. And of course, you can also Tweet me at @ZubinPratap.

(Banner photo by Thomas Smith on Unsplash)

Previously I wrote about linting, what it is, and how it makes your life easier. At the end, I actually included a way that you could automatically fix your code. So why am I writing this?

What do you mean fix it?

Before we roll into it, let’s hit this quick. Linters are powerful and provide an easy way to scan your code for syntax errors that could lead to bugs. Or they can simply help keep a codebase clean, healthy, and consistent. When run, it will show all the issues and let you go through each one individually to fix them.

Taking that to the next level, some linters will actually allow you to pass in an argument to the command running the linter that allows it to fix it for you automagically. This means you don’t have to manually go through and make all of those little whitespace and semicolon (add them! ?) tweaks yourself!

Ron Swanson happy

So what more can I do to fix things?

If you already use the fix option, thats a good start. But there are tools out there that have been developed specifically to tackle this problem beyond just a flag into your command. The one I’m going to cover is Prettier.

What is Prettier?

Prettier pegs itself as “an opinionated code formatter." It takes an input of your code and outputs it in a consistent format stripping any of the original code style. It actually converts your code to a syntax tree, then rewrites it using the styles and rules you and Prettier provide together via your ESLint config and Prettier’s default rules.

You can easily use Prettier alone just to format your code, which works just fine. But if you combine this with an underlying ESLint process, you get both a powerful linter and a powerful fixer. I’m going to show you how to make those work together.

Voltron

Getting started with Prettier

For this walkthrough, I’m going to assume that you have ESLint set up and configured in an application. Particularly, I’m going to pick up where I left off in my previous walkthrough where we installed ESLint to a React application.

Additionally of note, Prettier tells us right from the start that it's an opinionated code formatter. You should expect that it will format your code in a consistent way, but maybe a different way than you currently have it configured. But don’t fret! You can tweak this configuration.

So what are we starting off with? We already:

• Have installed ESLint
• Have added Babel as our parser
• Have added a plugin that includes React configurations

Next, let’s get started by installing a few packages:

yarn add prettier prettier-eslint prettier-eslint-cli -D

Note: the command above is similar to using npm. If your project doesn't use yarn, swap out to npm as appropriate.

Above, we’re installing:

• prettier: core Prettier package and engine
• prettier-lint: passes the Prettier result to ESLint to fix using your ESLint config
• prettier-eslint-cli: helps Prettier and ESLint work together on various files across your project

And we’re installing them as a dev dependency, as we don’t need it outside development.

Configuring your new formatter

Now that our packages are installed, we can set up yarn to run this command for us.

Previously, we set up a lint script to look like this in our package.json:

"scripts": {
  ...
  "lint": "eslint . --ext .js"
  ...
}

We’re going to leave that as it is, but we’ll do something similar and create a new script right next to it called format for our formatter Prettier:

"scripts": {
  ...
  "format": "prettier-eslint --eslint-config-path ./.eslintrc.js --write '**/*.js'",
  "lint": "eslint . --ext .js"
  ...
}

So what’s going on there? We’re:

• Adding a new script called format, that we can run as yarn format
• We’re utilizing our prettier-eslint-cli package to run the formatting for us
• We’re passing in our ESLint config located next to our package.json in the root of the project (change this if it’s in a different location)
• And finally, we’re telling prettier to write all files matching **/*.js, or any JS files it finds recursively through our project

The beauty here is that we're passing in our ESLint config to Prettier. This means we only have to maintain 1 config for both tools, but we still leverage the linting power of ESLint along with the formatting power of Prettier.

Run your formatter!

Now that we’re all set up, let’s run it! Run this following:

yarn format

and immediately, we see that it works:

Successfully running Prettier

Hey, my code looks different!

Angry mob with pitchforks

As I mentioned earlier, Prettier tells us straight up, it’s an opinionated formatter. It ships with its own rules, sort of like its own ESLint config, so it will go through and make those changes as well.

Don’t abandon your code! Instead, you can review the changes, see if maybe it makes sense to keep it that way (it will be very consistent) or you can update your ESLint config (.eslintrc.js) to overwrite the rules you don’t like. This is also a good way to maybe learn some new things that you might not have expected to get caught before.

So where does this leave us?

If you’ve followed along so far, we now have two commands:

• lint: which will check your code for you and tell you what's wrong
• format: will automatically try to fix the problems for you

When using these in practice, your best bet is to always run format first to let it try to automatically fix anything it can. Then immediately run lint to catch anything Prettier wasn’t able to fix automatically.

What’s next?

Now that we can format our code automatically, we should be able to fix our code automatically!

Eddie from Fresh Off the Boat's mind blown

Next time we’ll take this a step further and set up a git hook that will allow this to run before you commit. This means you won't ever have to worry about forgetting to run this again!

Originally published at https://www.colbyfayock.com/2019/11/dont-just-lint-your-code-fix-it-with-prettier/