Recommendation systems, fraud detection engines, personalization platforms, and enterprise search solutions all rely on integrating data from multiple systems while preserving context and relationships.

Enterprise Knowledge Graphs (EKGs) have emerged as a foundational architecture for addressing this challenge. By modeling enterprise data as entities and relationships, EKGs enable richer semantics, improved data discoverability, and more intelligent downstream decision making.

While the conceptual benefits of knowledge graphs are well understood, scaling them to production grade digital platforms remains complex. Graph systems that perform well at small or medium scale often struggle under high ingestion rates, complex traversal queries, and strict latency requirements.

This article outlines some practical, field tested strategies for optimizing enterprise knowledge graphs for real world scalability. Rather than presenting purely theoretical models, we'll focus on architectural patterns, operational lessons, and performance insights from large scale enterprise deployments.

What We'll Cover:

• Prerequisites

• Why Scalability Becomes the Core Challenge

• Moving Beyond a Single Graph Store: Hybrid Architectures

• Partitioning for Scale: Reducing Distributed Traversal Costs

• Managing Semantic Inference Without Sacrificing Performance

• Improving Query Performance with Smarter Planning

• Observability as a First Class Requirement

• Impact on Digital Product Platforms

• Conclusion

Prerequisites

This is an architectural guide intended for data engineers, platform architects, and developers managing production-grade graph systems. To get the most out of this article, you should have the following:

Conceptual Knowledge

• A solid understanding of Enterprise Knowledge Graphs (EKGs) and the fundamental differences between RDF triple stores and Labeled Property Graphs (LPGs).

• Familiarity with distributed systems concepts, including data partitioning, semantic inference, and event-driven architectures.

Technical Background

• Experience working with real-time data integration pipelines (such as CDC, Kafka, or Pulsar).

• Familiarity with database observability, query execution planning, and general performance optimization techniques at scale.

Understanding the Enterprise Knowledge Graph (EKG)

Before exploring how to scale these systems, it's helpful to understand exactly what a knowledge graph is and how it organizes information.

At its core, a knowledge graph is a data model that represents real-world entities and the complex relationships between them. Unlike traditional relational databases that lock data into rigid, disconnected tables, knowledge graphs store data as a flexible, interconnected network.

A knowledge graph is built on three fundamental components:

• Nodes (Entities): The distinct objects, concepts, or people in your data ecosystem (for example a Customer, a Product, a Location).

• Edges (Relationships): The lines connecting the nodes that define how they interact (for example "PURCHASED," "LOCATED_IN," "MANUFACTURED_BY").

Properties: The descriptive metadata attached to nodes or edges (for example, a customer's signup date, or the price of a product).

Our Running Example: The Global Electronics Supply Chain Graph

To ground these concepts, we'll use a unified example throughout this article: an enterprise graph for a global electronics manufacturer managing product data, suppliers, and manufacturing compliance.

• Nodes (Entities): Customer (Alice), Product (NeoPhone 15), Component (MX-200 Chip), Supplier (MaxSemi), and Region (EU).

• Edges (Relationships): PURCHASED, PART_OF, SUPPLIES, and LOCATED_IN.

• Properties: The NeoPhone 15 node has properties like price: 999 and sku: "NP15-01". The PURCHASED edge has a property of timestamp: 2026-06-03.

Imagine you're building the data foundation for a retail recommendation engine. To build the graph, you move through a few distinct phases:

1. Establish ontology: First, you define the blueprint – the rules dictating what kinds of entities exist and how they are allowed to interact.

2. Define the nodes: You integrate data to generate specific entity nodes, such as a Customer node for "Alice," a Product node for "Noise-Canceling Headphones," and a Brand node for "TechAudio."

3. Map the edges: You connect these nodes based on user actions and inventory data. Alice VIEWED the Headphones. The Headphones are MANUFACTURED_BY TechAudio.

Why does this matter? Because the data is natively structured as a relationship network, the system can rapidly execute context-rich queries.

If you want to know what else Alice might buy, you don't need to write a heavy, expensive SQL query that joins millions of rows across five different tables. Instead, the graph simply "walks" the pathways you've already built. It traverses from Alice, across the VIEWED edge to the Headphones, across the MANUFACTURED_BY edge to TechAudio, and can instantly return other products connected to that same brand.

By prioritizing the relationships between data points as much as the data points themselves, EKGs provide the contextual intelligence required for modern digital products.

Why Scalability Becomes the Core Challenge

Most enterprise knowledge graph initiatives begin with a limited scope, integrating a small number of datasets, enabling semantic search, or improving reporting accuracy. Early-stage deployments often succeed using a single graph database or RDF store.

Scalability challenges emerge when EKGs become production critical infrastructure, particularly when supporting customer facing or latency-sensitive applications. At this stage, multiple pressures converge:

1. Rapid data growth as more systems and entities are integrated

2. Continuous ingestion from streaming pipelines and transactional systems

3. Increasing query complexity, including multi hop traversals

4. Strict response time requirements, often under tens of milliseconds

5. Inference overhead introduced by ontologies and reasoning engines

Simply adding hardware or scaling nodes horizontally rarely resolves these issues. Performance degradation often results from architectural mismatches between graph workloads and system design.

Moving Beyond a Single Graph Store: Hybrid Architectures

The Limits of Monolithic Graph Deployments

RDF triple stores offer strong semantic expressiveness and standards compliance but may struggle with high volume transactional updates or deep real time traversals. Conversely, labeled property graph (LPG) databases often provide efficient traversal performance but lack native semantic reasoning capabilities.

Attempting to consolidate semantic modeling, inference, operational queries, and analytics into a single system frequently results in trade offs that affect performance, cost, or maintainability.

A Pragmatic Hybrid Model

A hybrid or polyglot architecture distributes responsibilities across systems optimized for specific workloads:

1. Semantic layer (RDF / OWL): Ontology management, schema governance, reasoning workflows.

2. Operational graph layer (LPG): Real time traversals, recommendation engines, application queries.

3. Analytical stores: Aggregations, reporting, and historical analysis.

To maintain consistency between the semantic layer (RDF/OWL) and the operational graph layer (LPG), many teams implement synchronization strategies like Change Data Capture (CDC) and event driven pipelines.

In this approach, updates in one layer are captured as events and propagated to the other layer in near real time using streaming platforms such as Kafka or Pulsar. For example, updates in the operational graph can trigger semantic updates, ensuring that ontologies and relationships remain aligned.

Some systems also use dual write patterns or scheduled reconciliation jobs to detect and resolve inconsistencies. In practice, event-driven synchronization combined with periodic validation provides a balance between real time accuracy and system reliability.

This separation isolates performance critical paths while preserving semantic richness where it adds value.

In production environments, hybrid architectures consistently demonstrate improved query latency and operational flexibility compared to monolithic graph deployments, particularly for traversal-heavy workloads. Some teams have also reported latency reductions of 30–60% when separating traversal-heavy workloads into LPG layers, compared to monolithic graph deployments.

This improvement is primarily due to reduced query complexity and optimized storage for specific access patterns.

In Practice: Splitting the Supply Chain Graph

In a production-grade digital platform, a single database engine struggles to handle both semantic governance and high-speed operational queries on this data simultaneously.

Here is how the hybrid model divides the labor:

• The Semantic layer (RDF/OWL): Manages strict ontological classification and compliance rules. For example, it defines the rule: “If a Component is supplied by an entity in a country under a trade embargo, the final Product inherits a 'High Risk' compliance flag.”

• The Operational Layer (LPG): Optimized for fast, multi-hop traversals required by customer-facing apps. When Alice views the NeoPhone 15 on a mobile app, the system queries a Labeled Property Graph (like Neo4j) using a language like Cypher to instantly traverse from the product to its components for a real-time availability check:

MATCH (p:Product {id: 'NeoPhone15'})-[:HAS_COMPONENT]->(c:Component)
RETURN c.name, c.stock_level

Partitioning for Scale: Reducing Distributed Traversal Costs

As enterprise knowledge graphs outgrow single node capacity, distributed execution becomes necessary. Partitioning strategy then becomes a critical performance factor.

Why Default Partitioning Often Fails

Many graph systems use hash-based or random partitioning to distribute data evenly across nodes. While this approach balances storage, it often fragments highly connected subgraphs. Even moderately complex traversals may then require excessive cross-node communication, increasing latency and reducing throughput.

Topology-Aware Partitioning

Topology-aware partitioning colocates frequently connected entities to minimize network hops during traversal. Common approaches include:

1. Partitioning by business domain (for example, customers, products, organizations).

2. Community detection based clustering.

3. Partitioning informed by observed query patterns.

In practice, teams can achieve topology-aware partitioning by first analyzing query patterns and identifying frequently traversed relationships. Based on this analysis, related entities are co-located within the same partition to minimize cross-partition queries.

Graph processing frameworks and database tools often provide built-in algorithms for community detection, which help group highly connected nodes. Teams can also monitor query performance over time and iteratively refine partitioning strategies to align with evolving workloads.

By combining domain driven design with continuous performance monitoring, teams can incrementally optimize graph layouts without requiring major architectural changes.

In production-inspired environments, topology-aware strategies significantly reduce traversal fan out and improve both median and tail latency under concurrent load.

Though repartitioning introduces operational complexity, the performance gains justify the effort once the knowledge graph becomes central to digital product delivery.

In Practice: Partitioning by Product Domain

Let’s look at what happens when our supply chain graph scales across multiple database nodes.

If we use Default Hash Partitioning, the graph is split randomly by node IDs. Alice might end up on Machine 1, the NeoPhone 15 on Machine 2, and the MX-200 Chip on Machine 3. A query tracking whether a component shortage affects Alice's order requires a slow, expensive network hop across three separate physical servers.

Using Topology-Aware Partitioning, we can configure the cluster to use the Region or Product_Line as a partitioning key.

• Partition A (Europe Hub): Co-locates Region: EU, Product: NeoPhone 15, its internal MX-200 Chip, and local customer orders.

Result: A multi-hop traversal checking component supply chains for European customers happens entirely within local memory on a single machine, reducing query latency.

Managing Semantic Inference Without Sacrificing Performance

Semantic inference is a defining strength of EKGs but also a frequent source of scalability challenges.

The Inference Cost Problem

Applying full ontology reasoning at query time can dramatically increase computational overhead. In some systems, inference effectively multiplies graph size, increasing memory and CPU consumption. Not all inferred relationships are equally valuable for every workload.

Strategies for Selective Inference and Materialization

Scalable EKG platforms typically adopt a selective strategy:

1. Precompute and materialize frequently accessed inferences

2. Offload complex reasoning to batch or asynchronous pipelines

3. Disable low value inference paths in latency-sensitive workloads

Hierarchical classifications and role-based relationships are often materialized ahead of time, while complex rule based reasoning is reserved for offline processing. This approach stabilizes query latency and reduces peak CPU utilization in enterprise deployments.

In Practice: Materializing the Compliance Path

Recall our semantic rule: If a component has a supply risk, the final product inherits that risk.

• The Scalability Bottleneck (Query-Time Inference): Every time an enterprise dashboard loads a product catalog of 10,000 items, the engine must recursively calculate: Product -> Has Component -> Supplied By -> Supplier Country -> Embargo List. Under high concurrent load, this calculation crashes performance.

• The Optimization (Materialization): We run an asynchronous batch job or Kafka consumer that listens for supplier updates. When a supplier's status changes, it computes the inference once and writes a direct property is_high_risk: true directly onto the Product node in the operational LPG.

Now, the customer-facing application reads a simple, static property without running an expensive multi-hop recursive inference query during runtime.

Improving Query Performance with Smarter Planning

As query complexity increases, query planning becomes a decisive performance lever.

Limitations of Static Planning

Traditional graph engines often rely on static heuristics or limited statistics for execution planning. In dynamic enterprise environments where data distributions evolve, these heuristics frequently produce suboptimal execution plans, leading to unpredictable performance.

ML-Assisted Query Optimization

Machine learning techniques are increasingly being applied to query optimization, particularly for cardinality estimation. By learning from historical query execution data, ML models can predict plan costs more accurately than rule-based systems.

In controlled experiments and production pilots, ML-assisted planning has demonstrated substantial reductions in execution time for complex traversals, as well as improved consistency in response times.

While implementation requires operational maturity, this represents a promising direction for large scale graph optimization.

In Practice: Optimizing Traversal Direction

Consider this query on our data: "Find all customers who purchased a product containing the MX-200 Chip."

There are two ways the graph execution planner can execute this:

1. Plan A: Start at Component: MX-200, find the products it belongs to, and then find the customers who bought those products.

2. Plan B: Scan all Customer nodes in the database, look at their purchases, and filter for the ones containing the chip.

If the MX-200 is a rare chip used in only one niche product, Plan A is incredibly fast. If it is a generic resistor used in millions of products, Plan B or a modified hybrid plan might be more efficient.

An ML-assisted query planner analyzes the real-time cardinality (the actual count) of the PART_OF and PURCHASED relationships in your specific database instance. It prevents the graph engine from choosing a disastrously slow traversal path when data distributions shift unexpectedly.

Observability as a First Class Requirement

Scalability can't be managed without deep observability.

Beyond Infrastructure Metrics

Monitoring CPU and memory alone provides limited insight into graph-specific performance issues. Effective EKG observability includes:

1. Query level latency metrics

2. Traversal depth and fan-out tracking

3. Inference cost monitoring

4. Partition imbalance detection

Closing the Optimization Loop

By continuously analyzing these signals, teams can iteratively refine partitioning strategies, caching policies, and materialization decisions. This feedback loop improves predictability and reduces production incidents.

In practice, strong observability often distinguishes proactive optimization from reactive firefighting.

Impact on Digital Product Platforms

When applied collectively, these optimization strategies materially enhance scalability and reliability. Across enterprise deployments, teams commonly observe:

1. Reduced latency in real time workloads

2. Improved ingestion throughput under sustained load

3. Linear or near linear scaling as datasets grow

4. Greater stability during traffic spikes

These technical improvements translate directly into business outcomes: faster recommendations, more relevant search results, and increased confidence in deploying EKGs as mission critical infrastructure.

Conclusion

Enterprise knowledge graphs are no longer experimental. They're becoming the backbone of intelligent, data driven systems. As teams move toward AI-powered decision making, the role of knowledge graphs is expanding beyond storage into enabling context-aware reasoning and automation.

An optimized EKG isn't just a database – it acts as the connective tissue between data, models, and real world applications. It provides the structured context that modern AI systems, including agentic workflows and autonomous decision engines, rely on to operate effectively.

By adopting hybrid architectures, topology-aware partitioning, and intelligent query strategies, teams can build scalable and resilient graph systems that support both operational and analytical workloads.

Ultimately, organizations that invest in well-designed knowledge graph infrastructure will be better positioned to power the next generation of AI systems where retrieval, reasoning, and action are seamlessly integrated.

We just published a course on the freeCodeCamp.org YouTube channel that progresses from foundational concepts to production-ready systems, covering databases, scaling, and load balancing. You will learn practical techniques for building and securing APIs, including RESTful and GraphQL. This course was developed by Hayk Simonyan.

Here are the sections in this course:

• Introduction

• Single Server Setup

• Databases: SQL, NoSQL, Graph

• Vertical vs Horizontal Scaling

• Load Balancing

• Health Checks

• Single Point of Failure (SPOF)

• API Design

• API Protocols

• Transport Layer: TCP, UDP

• RESTful APIs

• GraphQL

• Authentication

• Authorization

• Security

Watch the full course on the freeCodeCamp.org YouTube channel (2-hour watch).

Errors like that could be fine for little tasks, but it's a disaster for production systems. The gap between "this worked in my ChatGPT conversation" and "this runs reliably in production" is massive. It's not closed by better prompts. It's closed by engineering.

This article is about that engineering. You'll learn the architecture patterns, failure modes, and implementation strategies that separate AI experiments from AI products.

What You'll Learn

In this tutorial, you'll learn how to:

• Understand why AI systems fail differently from traditional software

• Identify and prevent the three critical failure modes in production AI

• Implement the validator sandwich pattern for consistent outputs

• Build observable pipelines with proper monitoring and alerting

• Control costs at scale with rate limiting and circuit breakers

• Design a complete production-ready AI architecture

Prerequisites

To get the most from this tutorial, you should have:

• Basic understanding of any programming language

• Familiarity with REST APIs and asynchronous programming

• Experience with at least one LLM API (OpenAI, Anthropic, or similar)

• Node.js installed locally (optional, for running code examples)

You don't need to be an expert in any of these. Intermediate knowledge is sufficient.

Table of Contents

1. What Makes AI Systems Fundamentally Different

2. Failure Mode #1: Inconsistent Outputs

3. Failure Mode #2: Silent Failures

4. Failure Mode #3: Uncontrolled Costs

5. How to Build a Complete Production Architecture

6. Conclusion

What Makes AI Systems Fundamentally Different

Traditional software is deterministic. You write if (urgency > 8) { return 'high' } and it does exactly that, every single time. Same input, same output. Forever. You can write unit tests that cover every path. You can predict every failure mode.

AI systems, on the other hand, are probabilistic. You ask an large language model (LLM) to classify urgency and sometimes it says "high," sometimes "urgent," sometimes it gives you a 1–10 score, sometimes it writes a paragraph explaining its reasoning. Same input, different outputs, depending on temperature settings, model version, context window, and factors you can't fully control.

Here's what that looks like in practice:

The engineering challenge is: how do you build reliability on top of inherent unpredictability?

The answer is not "use a better model." The model is maybe 20% of the solution. The remaining 80% is the system you build around it.

Failure Mode #1: Inconsistent Outputs

The Problem

You ask the AI to extract a customer email from a support ticket. Sometimes you get the email back. Sometimes you get just the name. Sometimes you get a phone number. The format changes every time. Same prompt, different outputs.

Prompt: "Extract the customer email from this support ticket"

Output on Monday:    "john@example.com"
Output on Tuesday:   "Customer email: john@example.com (verified)"
Output on Wednesday:   "John Doe"
Output on Thursday: {
                       "customer_info": {
                         "email": "john@example.com"
                       }
                     }

All three outputs contain correct information, but you can't parse them programmatically. You can't route tickets, trigger workflow systems, or integrate with other code because your response data lacks consistency.

The Solution: The Validator Sandwich Pattern

The validator sandwich pattern (also called the guardrails pattern) ensures the AI system doesn't generate or process the wrong data by sandwiching your AI between two layers of deterministic code.

Essentially, you have three layers:

1. The top bun: Input guardrails (deterministic)

2. The meat: The LLM (probabilistic)

3. The bottom bun: Output guardrails (deterministic)

Let's break down each layer.

The Top Bun: Input Guardrails

Before anything touches the AI, validate it. Reject garbage immediately, fail fast and cheaply. Here's a basic example with deterministic code that checks the data being received:

function validateTicketInput(raw): TicketInput {
  // Type checks
  if (!raw.email || typeof raw.email !== "string") {
    throw new ValidationError("Missing or invalid email");
  }

  // Format checks
  if (!isValidEmail(raw.email)) {
    throw new ValidationError(`Invalid email format: ${raw.email}`);
  }

  // Range checks
  if (!raw.body || raw.body.length < 10) {
    throw new ValidationError("Ticket body too short to classify");
  }

  if (raw.body.length > 10000) {
    throw new ValidationError("Ticket body exceeds max length");
  }

  // Return typed, validated input
  return {
    email: raw.email.toLowerCase().trim(),
    subject: raw.subject?.trim() || "No subject",
    body: raw.body.trim(),
    timestamp: new Date(raw.timestamp),
  };
}

This runs before the LLM is ever called. It's fast, cheap, and deterministic. It catches easy failures immediately.

The Meat: Structured Outputs from the LLM

Stop asking the AI for free text. Force it into a schema. Most modern APIs support this directly.

So what does "free text" mean? When you prompt an LLM without constraints, it returns unstructured natural language. The model decides the format. Sometimes it's a sentence, sometimes a paragraph, sometimes it adds extra context you didn't ask for. This makes programmatic parsing nearly impossible.

Forcing it into a schema, on the other hand, means that you explicitly tell the model: "Respond only with JSON matching this exact structure", for example. Modern LLM APIs have built-in features to enforce this. Instead of hoping the AI formats its response correctly, you make it structurally impossible for it to return anything else.

Here's the difference in practice:

Without schema enforcement (free text):

const response = await openai.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{
    role: "user",
    content: "Classify this support ticket as bug, billing, or feature request: " + ticketText
  }]
});

// Response could be:
// "This appears to be a billing issue"
// "billing"
// "Category: Billing (confidence: high)"
// { "type": "billing" }  <- if you're lucky

With schema enforcement:

const response = await openai.chat.completions.create({
  model: "gpt-4o-mini",
  messages: [{
    role: "user",
    content: "Classify this support ticket: " + ticketText
  }],
  response_format: {
    type: "json_schema",
    json_schema: {
      name: "ticket_classification",
      strict: true,
      schema: {
        type: "object",
        properties: {
          category: {
            type: "string",
            enum: ["bug", "billing", "feature", "other"]
          },
          confidence: {
            type: "number",
            minimum: 0,
            maximum: 1
          },
          priority: {
            type: "integer",
            minimum: 1,
            maximum: 5
          }
        },
        required: ["category", "confidence", "priority"],
        additionalProperties: false
      }
    }
  }
});

// Response is GUARANTEED to be:
// { "category": "billing", "confidence": 0.89, "priority": 2 }

The response_format parameter forces the model to output valid JSON matching your schema. If it can't, the API will retry internally until it does. You get predictable, parseable data every single time.

The key difference: you're making the AI conform to your format instead of hoping it does the right thing.

The Bottom Bun: Output Guardrails

This is the most critical layer. LLMs will hallucinate. This layer catches those hallucinations before they break your database or confuse your users.

Guardrails are validation checks that run after the LLM responds. Think of them as safety barriers on a highway: they don't prevent the car from moving, but they can stop it from going off the road.

In AI systems, guardrails verify that:

1. The output matches your expected schema

2. The data types are correct

3. The values fall within acceptable ranges

4. The business logic makes sense

Alright, now you have a structured response. Now you'll want to validate it aggressively before you use it:

function validateClassification(raw): Classification {
  const required = ["category", "confidence", "priority", "reasoning"];
  for (const field of required) {
    if (raw[field] === undefined || raw[field] === null) {
      throw new ValidationError(`Missing required field: ${field}`);
    }
  }

  if (!["bug", "billing", "feature", "other"].includes(raw.category)) {
    throw new ValidationError(`Invalid category: ${raw.category}`);
  }

  if (typeof raw.confidence !== "number" || 
      raw.confidence < 0 || raw.confidence > 1) {
    throw new ValidationError(`Invalid confidence: ${raw.confidence}`);
  }

  if (!Number.isInteger(raw.priority) || 
      raw.priority < 1 || raw.priority > 5) {
    throw new ValidationError(`Invalid priority: ${raw.priority}`);
  }

  if (raw.category === "billing" && raw.priority > 3) {
    logger.warn("Suspicious: billing classified as low priority", raw);
  }

  return raw as Classification;
}

Validating aggressively means checking everything, not just schema compliance. You're validating:

• Schema compliance: Does the JSON have the right fields?

• Type safety: Is "confidence" actually a number, not a string?

• Range validity: Is confidence between 0 and 1, not -5 or 999?

• Business logic: Does the combination of fields make sense for your domain?

• Confidence thresholds: Is the AI actually confident in this answer?

If any validation fails, you don't silently accept bad data. You have three options:

1. Retry with a clearer prompt: Ask the model to try again with stricter instructions

2. Escalate to human review: Log the failure and route to a review queue

3. Use a fallback: Return a safe default value that requires human attention

The Deterministic Rule

Here's a rule to follow religiously:

If it can be solved with an if-statement, don't use AI.

Email format validation? Use regex. Date parsing? Use a date library. Checking if a string contains a keyword? Use a string method. Math? Use actual math.

AI is expensive and probabilistic. Traditional code is free, instant, and deterministic. Use AI for genuinely ambiguous tasks, extracting meaning from unstructured text, generating content, and reasoning about complex inputs. Let deterministic code handle everything else.

Failure Mode #2: Silent Failures

The Problem

Model hallucinations are quite common in AI workflows, ranging from degraded accuracy to outdated training data to misclassification issues. This is the scariest failure mode because you don't know it's happening.

Consider accuracy drift. You trained your model on 2024 data. It's now mid-2026. Your vendors changed their invoice formats. Your classification accuracy has drifted from 95% down to 71%. You won't know until you do a quarterly audit. And by then, thousands of records have been processed incorrectly.

The principle is simple: you cannot fix what you cannot see.

The Solution: Observable Pipelines

Every production AI system needs observability baked in from day one. Here's how this plays out in a production system:

In the diagram above:

1. Input arrives: A user request comes in (support ticket, document, query). You log: request ID, timestamp, user ID, input hash (for deduplication).

2. LLM Processing: The request goes to your AI model. You log which model was called, how long it took (latency), how many tokens used, what it cost, and critically, the confidence score.

3. Confidence Gate: This is where you make a routing decision:

   • High confidence (>0.8): Auto-process and execute the action

   • Medium confidence (0.6-0.8): Send to human review queue

   • Low confidence (<0.6): Immediate escalation + alert

4. Monitoring Dashboard: All this data flows into your observability tools, where you track trends over time.

With monitoring, you can detect issues in your system and address them as soon as possible. Monitoring doesn't just catch problems. It gives you data to diagnose and fix them in hours instead of months.

What you're measuring and why:

The goal is not to remove humans from the loop, it's to only involve humans when the system is genuinely uncertain.

Failure Mode #3: Uncontrolled Costs

The Problem

You test your workflow with 10 tickets. It works great and costs 50 cents. You deploy to production. 1,000 requests hit your API. Your bill: $500 for the day.

Or you write a retry loop incorrectly. It creates infinite API calls. Your bill: $5,000 for the day.

Or you're using the most expensive model for everything, including simple tasks that a cheaper model could handle.

The reality: "works for 10 requests" ≠ "works for 10,000 requests." Scale changes everything.

The Solution: Gated Pipelines with Circuit Breakers

To move from a fragile prototype to a robust production system, you must abandon the naive approach of directly connecting user inputs to LLM APIs. Instead, implement a gated pipeline.

Think of this architecture as a series of blast doors. A request must successfully pass through each gate before it earns the right to cost you money. If any gate closes, the request is rejected cheaply and quickly, protecting your budget and your upstream dependencies.

From the diagram above, these gates are:

1. The rate limiter

2. The cache check

3. The request queue

4. The circuit breaker

Let's examine each one.

Gate 1: Rate limiting

The first line of defence stops abuse before it enters your system. In standard web development, rate limiting is about protecting the server CPU. In AI development, it's about protecting your wallet.

Gate 2: Cache check

The cheapest LLM API call is the one you never have to make. Many AI requests are repeated or highly similar. Cache aggressively.

Gate 3: Request queue

LLM APIs are not like standard REST APIs; requests often take 10–30 seconds to complete. If 500 users hit "submit" simultaneously, your server cannot open 500 simultaneous connections without crashing or hitting provider concurrency limits. A request queue solves this by batching requests and processing them at a controlled rate.

Gate 4: Circuit breaker

Retry logic is necessary for transient network blips, but it is destructive during a real outage. If an LLM provider is experiencing downtime and returning 500 errors, a naive retry loop will frantically hammer their API, wasting your money on failed requests.

How to implement a gated pipeline

Here's an example implementation showing all four gates working together:

Step 1: Rate Limiter (using Redis)

import { RateLimiterRedis } from "rate-limiter-flexible";
import Redis from "ioredis";

const redis = new Redis({
  host: process.env.REDIS_HOST,
  port: 6379
});

// Rate limiting per user
const userLimiter = new RateLimiterRedis({
  storeClient: redis,
  keyPrefix: "rl:user",
  points: 100,        
  duration: 3600,     
  blockDuration: 60   
});

// Rate limiting globally 
const globalLimiter = new RateLimiterRedis({
  storeClient: redis,
  keyPrefix: "rl:global",
  points: 1000,       
  duration: 3600      
});

Step 2: Cache Layer

import { createHash } from "crypto";

class AICache {
  private redis: Redis;
  private ttl: number = 3600; 

  hashInput(input: string): string {
    return createHash("sha256").update(input).digest("hex");
  }

  async get(input: string): Promise {
    const key = `ai:cache:${this.hashInput(input)}`;
    const cached = await this.redis.get(key);
    
    if (cached) {
      // Cache hit - free!
      await metrics.increment("ai.cache.hits");
      return JSON.parse(cached);
    }
    
    await metrics.increment("ai.cache.misses");
    return null;
  }

  async set(input: string, result: T): Promise {
    const key = `ai:cache:${this.hashInput(input)}`;
    await this.redis.setex(key, this.ttl, JSON.stringify(result));
  }
}

Step 3: Request Queue

import Queue from "bull";

const aiQueue = new Queue("ai-requests", {
  redis: {
    host: process.env.REDIS_HOST,
    port: 6379
  }
});

aiQueue.process(5, async (job) => {
  // Only 5 simultaneous LLM calls max
  const { ticket } = job.data;
  return await callLLM(ticket);
});

async function enqueueRequest(ticket: Ticket) {
  const job = await aiQueue.add(
    { ticket },
    {
      attempts: 3,
      backoff: {
        type: "exponential",
        delay: 2000
      }
    }
  );
  
  return job.finished(); 
}

Step 4: Circuit Breaker

enum CircuitState {
  CLOSED,   
  OPEN,     
  HALF_OPEN 
}

class CircuitBreaker {
  private state = CircuitState.CLOSED;
  private failures = 0;
  private lastFailureTime?: Date;
  private successesInHalfOpen = 0;

  private readonly failureThreshold = 3;
  private readonly openDurationMs = 5 * 60 * 1000; 
  private readonly halfOpenSuccesses = 2;

  async execute(
    fn: () => Promise,
    fallback?: () => T
  ): Promise {
    if (this.state === CircuitState.OPEN) {
      const elapsed = Date.now() - (this.lastFailureTime?.getTime() || 0);
      
      if (elapsed < this.openDurationMs) {
        // Still in open state - use fallback or throw
        if (fallback) {
          logger.warn("Circuit OPEN - using fallback");
          return fallback();
        }
        throw new Error("Circuit breaker OPEN - service unavailable");
      }
      
      // Transition to half-open
      this.state = CircuitState.HALF_OPEN;
      logger.info("Circuit transitioning to HALF_OPEN");
    }

    try {
      const result = await fn();
      this.onSuccess();
      return result;
    } catch (error) {
      this.onFailure();
      throw error;
    }
  }

  private onSuccess() {
    if (this.state === CircuitState.HALF_OPEN) {
      this.successesInHalfOpen++;
      
      if (this.successesInHalfOpen >= this.halfOpenSuccesses) {
        // Service recovered - close circuit
        this.state = CircuitState.CLOSED;
        this.failures = 0;
        this.successesInHalfOpen = 0;
        logger.info("Circuit CLOSED - service recovered");
      }
    } else {
      this.failures = 0;
    }
  }

  private onFailure() {
    this.failures++;
    this.lastFailureTime = new Date();

    if (this.state === CircuitState.HALF_OPEN) {
      // Failed during test - back to open
      this.state = CircuitState.OPEN;
      this.successesInHalfOpen = 0;
      logger.error("Circuit reopened during HALF_OPEN test");
    } else if (this.failures >= this.failureThreshold) {
      // Too many failures - open circuit
      this.state = CircuitState.OPEN;
      logger.error(`Circuit OPEN after ${this.failures} failures`);
    }
  }
}

Step 5: Putting it all together

const cache = new AICache();
const circuitBreaker = new CircuitBreaker();

async function processWithGatedPipeline(ticket: Ticket) {
  try {
    await userLimiter.consume(ticket.userId);
    await globalLimiter.consume("global");
  } catch (error) {
    throw new Error("Rate limit exceeded. Please try again later.");
  }

  const cacheKey = ticket.body;
  const cached = await cache.get(cacheKey);
  if (cached) {
    logger.info("Cache hit - returning cached result");
    return cached;
  }

  const queuedResult = await enqueueRequest(ticket);

  const result = await circuitBreaker.execute(
    async () => {
      const classification = await callLLM(ticket);
      await cache.set(cacheKey, classification);
      return classification;
    },
    () => ({
      category: "other",
      confidence: 0,
      requiresHumanReview: true,
      reason: "service_unavailable"
    })
  );

  return result;
}

What this achieves:

• Rate limiting: Prevents abuse and runaway costs

• Caching: 30-40% cost reduction on repeated queries

• Queueing: Prevents server overload during traffic spikes

• Circuit breaker: Fails fast during outages instead of wasting money on retries

Each gate is cheap to operate. Together, they protect your system from the most common production failures.

How to Build a Complete Production Architecture

When you combine all three failure mode solutions-consistent outputs, observability, and cost control, you get a complete production architecture.

When you solve for all three major failure modes, inconsistent outputs, silent failures, and uncontrolled costs. You graduate from a simple script to a true enterprise-grade system. This architecture doesn't just generate text; it actively protects itself, manages resources, and learns from its mistakes.

The Complete Workflow Implementation

Here's how all the pieces we've covered fit together in a single workflow. This brings together the validation functions from Failure Mode #1, the observability from Failure Mode #2, and the gated pipeline from Failure Mode #3:

class TicketWorkflow {
  async processTicket(rawInput: unknown): Promise<TicketResult> {
    const requestId = generateId();
    const startTime = Date.now();

    try {
      // LAYER 1: Input validation + rate limiting + cache
      const ticket = validateTicketInput(rawInput);
      await rateLimiter.consume(ticket.userId);
      
      const cached = await cache.get(ticket.body);
      if (cached) return { ...cached, source: "cache" };

      // LAYER 2: AI processing with circuit breaker protection
      const classification = await circuitBreaker.execute(() => 
        classifyTicket(ticket)
      );

      // LAYER 3: Output validation + confidence routing
      const validated = validateClassification(classification);
      
      let action: string;
      if (validated.confidence >= 0.8) {
        await sendToAgent(ticket, validated);
        action = "auto_assigned";
      } else {
        await sendToReviewQueue(ticket, validated);
        action = "needs_review";
      }

      // LAYER 4: Log everything for observability
      await logger.log({
        requestId,
        userId: ticket.userId,
        confidence: validated.confidence,
        action,
        latencyMs: Date.now() - startTime,
        cost: calculateCost(classification.tokensUsed)
      });

      await cache.set(ticket.body, validated);
      return { classification: validated, action };

    } catch (error) {
      await logger.logError(requestId, error);
      throw error;
    }
  }
}

What each layer does:

Layer 1 (Input) protects your system from bad data and abuse:

• Validates the ticket has required fields (email, subject, body)

• Checks rate limits (prevents one user from overwhelming the system)

• Returns cached results if we've seen this exact ticket before

Layer 2 (Orchestration) is where the AI does its work:

• Calls the LLM with structured output requirements

• Wrapped in a circuit breaker (fails fast if the API is down)

• Uses the cheapest model that works (Haiku for classification)

Layer 3 (Validation) ensures the output is safe to use:

• Validates the response matches our schema

• Routes based on confidence (high confidence → auto-assign, low → human review)

• Never blindly trusts AI output

Layer 4 (Observability) tracks everything:

• Logs every request with latency, cost, and confidence scores

• Sends metrics to your monitoring dashboard

• Alerts on anomalies (confidence dropping, costs spiking)

This architecture takes you from "it worked in my ChatGPT demo" to "it runs reliably at 10,000 tickets per day." The code is more complex than a simple API call, but the complexity is intentional. It's what makes the system production-ready.

Conclusion: Engineering Over Prompting

The teams winning with AI right now aren't winning because they have better models. They're winning because they've built better systems around imperfect models.

Any company can call the OpenAI API. The ones that pull ahead are the ones who wrap that API call in validation, observability, cost controls, and thoughtful architecture — the ones who treat AI as a component in an assembly line, not a creative partner in a conversation.

The three things every production AI system needs:

1. Structure: Validators, schemas, deterministic layers that enforce consistency and eliminate unpredictability at the edges.

2. Visibility: Logging, monitoring, and alerting so you catch problems in hours, not months. Observable pipelines that let you see exactly what the system is doing and why.

3. Control: Rate limits, caching, circuit breakers, and cost gates so scale doesn't turn your experiment into a budget emergency.

Reliable AI workflows aren't about better prompts. They're about better architecture around unreliable components.

If you found this helpful, you can connect with me on LinkedIn or subscribe to my newsletter. You can also visit my website.

Personalization is one of the most requested features in AI-powered applications. Users expect an agent to remember their preferences, adapt to their style, and improve over time.

In practice, personalization is unfortunately also one of the fastest ways to break an otherwise working AI agent.

Many agents start with a simple idea: keep adding more conversation history to the prompt. This approach works for demos, but it quickly fails in real applications. Context windows grow too large. Irrelevant information leaks into decisions. Costs increase. Debugging becomes nearly impossible.

If you want a personalized agent that survives production, you need more than a large language model. You need a way to connect the agent to tools, manage multi-step workflows, and store user preferences safely over time – without turning your system into a tangled mess of prompts and callbacks.

In this tutorial, you’ll learn how to design a personalized AI agent using three core building blocks:

• Agent Development Kit (ADK) to orchestrate agent reasoning and execution

• Model Context Protocol (MCP) to connect tools with clear boundaries

• Long-term memory to store preferences without polluting context

Rather than focusing on setup commands or vendor-specific walkthroughs, we'll focus on the architectural patterns that make personalized agents reliable, debuggable, and maintainable.

Figure 1 — Personalization influences agent responses

Table of Contents

• Prerequisites

• What “Personalized” Means in a Real AI Agent

• How the Agent Architecture Fits Together

• How to Design the Agent Core with ADK

• How to Connect Tools Safely with MCP

• How to Add Long-Term Memory Without Polluting Context

  • Privacy, Consent, and Lifecycle Controls (Production Checklist)

• How the End-to-End Agent Flow Works

• Common Pitfalls You’ll Hit (and How to Avoid Them)

• What You Learned and Where to Go Next

Prerequisites

To follow along with this tutorial, you should have:

• Basic familiarity with Python

• A general understanding of how large language models work

• Optional: a Google Cloud account if you want to run an end-to-end demo. Otherwise, you can follow the architecture and code patterns locally with stubs. We’ll avoid deep infrastructure setup and focus on design patterns rather than deployment mechanics.

You don’t need prior experience with ADK or MCP. I’ll introduce each concept as it appears.

What “Personalized” Means in a Real AI Agent

Figure 2 — Keep preferences out of the prompt: agent ↔ tools across a protocol boundary

Before writing any code, it’s important to define what personalization means in an AI agent.

Personalization is not the same as “remembering everything.” In practice, agent state usually falls into three categories:

1. Short-term context: Information needed to complete the current task. This belongs in the prompt.

2. Session state: Temporary decisions or selections made during a workflow. This should be structured and scoped to a session.

3. Long-term memory: Durable user preferences or facts that should persist across sessions.

Figure 3 — Three kinds of agent state: context (now), session (today), memory (always)

Most problems happen when these categories are mixed together.

If you store long-term preferences directly in the prompt, the agent’s behavior becomes unpredictable. If you store everything permanently, memory grows without bounds. If you don’t scope memory at all, unrelated sessions start influencing each other.

A well-designed, personalized agent treats memory as a first-class system component, not as extra text added to a prompt.

In the next section, we'll look at how to structure the agent so these concerns stay separated.

By the end of this tutorial, you’ll understand how to design a personalized AI agent that uses long-term memory safely, connects to tools through clear boundaries, and remains debuggable as it grows.

How the Agent Architecture Fits Together

Figure 4 — Reference architecture: agent core + tools + memory service + orchestration runtime

The above diagram shows a high-level, personalized AI agent architecture. In it, an agent core handles reasoning and planning while interacting with a tool interface layer, a long-term memory service, and an orchestration runtime.

Let’s now understand the moving parts of a personalized agent and how they interact.

At a high level, the system has four responsibilities:

1. Reasoning – deciding what to do next

2. Execution – calling tools and services

3. Memory – storing and retrieving long-term preferences

4. Boundaries – controlling what the agent is allowed to do

A common mistake you’ll see is to blur these responsibilities together. For example, letting the model decide when to write memory, or allowing tools to execute actions without clear constraints.

Instead, you'll design the system so each responsibility has a clear owner. The core components look like this:

• Agent core: Handles reasoning and planning

• Tools: Perform external actions (read or write)

• MCP layer: Defines how tools are exposed and invoked

• Memory services: Store long-term user data safely

ADK sits at the center, orchestrating how requests flow between these components. The model never directly talks to databases or services. It reasons about actions, and ADK coordinates execution.

This separation makes the system easier to reason about, debug, and extend.

How to Design the Agent Core with ADK

Before we dive in, a quick note on what ADK is.

Agent Development Kit (ADK) is an agent orchestration framework – the glue code between a large language model and your application. Instead of treating the model as a black box that directly “does things”, ADK helps you structure the agent as a system:

• The model focuses on reasoning (turning user intent, context, and memory into a structured plan)

• Your runtime stays in control of execution (deciding which tools can run, how they run, and what gets logged or persisted)

In other words, ADK is what lets you take tool calling and multi-step workflows out of a giant prompt and turn them into a maintainable and testable architecture. In this tutorial, we’ll use ADK to refer to that orchestration layer. The same patterns apply if you use a different agent framework.

Note: The following code snippets are simplified reference examples intended to illustrate architectural patterns. They’re not production-ready drop-ins.

Once you understand the architecture, you can start designing the agent core. The agent core is responsible for reasoning, not execution.

A helpful mental model is to think of the agent as a planner, not a doer. Its role is to interpret the user’s goal, consider available context and memory, and produce a structured plan that can later be executed in a controlled way.

To make this concrete, the following example shows how an agent can translate user input and memory into an explicit plan. In practice, ADK orchestrates this using a large language model, but the important idea is that the output is structured intent, not side effects.

# Reference example for illustration.

from dataclasses import dataclass
from typing import List, Dict, Any

@dataclass
class Step:
    tool: str
    args: Dict[str, Any]

@dataclass
class Plan:
    goal: str
    steps: List[Step]

def build_plan(user_text: str, memory: Dict[str, Any]) -> Plan:
    # In practice, the LLM produces this structure via ADK orchestration.
    goal = f"Help user: {user_text}"
    steps = []
    if memory.get("prefers_short_answers"):
        steps.append(Step(tool="set_style", args={"verbosity": "low"}))
    steps.append(Step(tool="search_docs", args={"query": user_text}))
    steps.append(Step(tool="summarize", args={"max_bullets": 5}))
    return Plan(goal=goal, steps=steps)

This example illustrates an important constraint: the agent produces a plan, but it doesn’t execute anything directly.

The agent decides what should happen and in what order, while ADK controls when and how each step runs. This separation lets you inspect, test, and reason about decisions before they result in real-world actions.

When personalization is involved, this distinction becomes critical. Preferences may influence planning, but execution should remain tightly controlled by the runtime.

Again, we can consider the agent to be a planner, not a doer.

It should not:

• Perform side effects directly

• Write to databases

• Call external APIs without supervision

In ADK, this separation is natural. The agent produces intents and tool calls, while the runtime controls how and when those calls are executed.

This design has two major benefits:

1. Safety – you can restrict which tools the agent can access

2. Debuggability – you can inspect decisions before execution

When personalization is involved, this becomes even more important. Preferences influence reasoning, but execution should remain tightly controlled.

How to Connect Tools Safely with MCP

Figure 5 — Tool calls with guardrails: request → validate → execute → respond

Tools are how agents interact with the real world. They fetch data, generate artifacts, and sometimes perform actions with side effects.

Without clear boundaries, tool usage quickly becomes a source of fragility. Hardcoded API calls leak into prompts, tools evolve independently, and agents gain more authority than intended.

To avoid these problems, tools should be explicitly registered and invoked through a narrow interface. The following example shows a simple tool registry pattern that mirrors how MCP exposes tools to an agent without tightly coupling it to implementations.

# Reference example (pseudocode for illustration)

from typing import Callable, Dict, Any

ToolFn = Callable[[Dict[str, Any]], Dict[str, Any]]

TOOLS: Dict[str, ToolFn] = {}

def register_tool(name: str):
    def decorator(fn: ToolFn):
        TOOLS[name] = fn
        return fn
    return decorator

@register_tool("search_docs")
def search_docs(args: Dict[str, Any]) -> Dict[str, Any]:
    query = args["query"]
    # Replace with your MCP client call (or local tool implementation).
    return {"results": [f"doc://example?q={query}"]}

def invoke_tool(name: str, args: Dict[str, Any]) -> Dict[str, Any]:
    if name not in TOOLS:
        raise ValueError(f"Tool not allowed: {name}")
    return TOOLS[name](args)

The Model Context Protocol (MCP) provides a clean way to formalize this pattern. You can think of MCP the same way operating systems treat system calls.

An application does not directly manipulate hardware. Instead, it requests operations through well-defined system calls. The kernel decides whether the operation is allowed and how it executes.

In the same way, the agent knows what capabilities exist, MCP defines how those capabilities are invoked, and the runtime controls when and whether they execute.

This separation prevents several common problems, including hardcoded API details in prompts, unexpected breakage when tools change, and agents performing unrestricted side effects.

When designing tools, it helps to classify them by risk: read tools for safe queries, generate tools for planning or synthesis, and commit tools for irreversible actions. In a personalized agent, commit tools should be rare and tightly guarded.

Figure 6 — Observability around tool calls: logs, traces, timing, decision points

How to Add Long-Term Memory Without Polluting Context

Figure 7 — Memory admission pipeline: extract → filter/validate → persist asynchronously

Memory is where personalization either succeeds or fails.

You can start by storing everything the user says and feed it back into the prompt. This works briefly, then collapses under its own weight as context grows, costs rise, and behavior becomes unpredictable.

A better approach is to treat memory as structured, curated data so you can control what the agent remembers and why with clear admission rules. Before persisting anything, the system should explicitly decide whether the information is worth remembering. The following function demonstrates a simple memory admission policy.

# Simplified Reference Only
from typing import Optional, Dict, Any

def memory_candidate(user_text: str) -> Optional[Dict[str, Any]]:
    text = user_text.lower()

    # Durable
    if "for this session" in text or "ignore after" in text:
        return None

    # Reusable
    if "my preferred language is" in text:
        return {"type": "preference", "key": "language", "value": user_text.split()[-1]}

    # Safe (basic example; add PII checks for your use case)
    if "password" in text or "ssn" in text:
        return None

    return None  # default: don’t store

This policy encodes three questions every memory candidate must answer:

• Is it durable? Will it still matter in the future?

• Is it reusable? Will it influence future decisions meaningfully?

• Is it safe to persist? Does it avoid sensitive or session-specific data?

Only information that passes all three checks should become long-term memory. In practice, this usually includes stable preferences and long-lived constraints, not temporary instructions or intermediate reasoning.

Privacy, Consent, and Lifecycle Controls (Production Checklist)

Even if your admission rules are solid, long-term memory introduces governance requirements:

• User control: allow users to view, export, and delete stored preferences at any time.

• Sensitive data handling: never store secrets/PII. Run PII detection on every memory candidate (and consider redaction).

• Retention + consent: use explicit consent for persistent memory and apply retention windows (TTL) so memory expires unless it’s still useful.

• Security + auditability: encrypt at rest, restrict access by service identity, and keep an audit log of memory writes/updates.

Memory writes should also be asynchronous. The agent should never block while persisting memory, which keeps interactions responsive and avoids coupling reasoning to storage latency.

How the End-to-End Agent Flow Works

Figure 8 — End-to-end request lifecycle: user input → plan → tools → memory updates

At this point, you can trace exactly how memory and tools interact during a single request. With the individual components in place, it’s helpful to see how they work together during a single request. The following example walks through the full lifecycle of a personalized interaction, from user input to response.

# Reference example (pseudocode for illustration)

def handle_request(user_id: str, user_text: str) -> str:
    memory = memory_store.get(user_id)  # e.g., {"prefers_short_answers": True}
    plan = build_plan(user_text, memory)

    tool_outputs = []
    for step in plan.steps:
        out = invoke_tool(step.tool, step.args)
        tool_outputs.append({step.tool: out})

    response = render_response(goal=plan.goal, tool_outputs=tool_outputs, memory=memory)

    cand = memory_candidate(user_text)
    if cand:
        # Never block the user on storage.
        memory_store.write_async(user_id, cand)
    return response

At a high level, the flow looks like this:

1. The user sends a message.

2. Relevant long-term memory is retrieved.

3. The agent reasons about the request and produces a plan.

4. ADK invokes tools through MCP as needed.

5. Results flow back to the agent.

6. The agent decides whether new information should be persisted.

7. Memory is written asynchronously.

8. The final response is returned to the user.

Notice what does not happen: the model does not directly write memory, tools do not execute without coordination, and context does not grow without bounds. This structure keeps personalization controlled and predictable.

Common Pitfalls You’ll Hit (and How to Avoid Them)

Even with a solid architecture, there are a few failure modes that show up repeatedly in real systems. Many of them stem from allowing agents to perform irreversible actions without explicit checks.

The following example shows a simple guardrail for commit-style tools that require approval before execution.

# Reference example (pseudocode for illustration)

def invoke_commit_tool(name: str, args: Dict[str, Any], approved: bool) -> Dict[str, Any]:
    if not approved:
        # Require explicit confirmation or policy approval before side effects.
        return {"status": "blocked", "reason": "commit tools require approval"}

    # For example: create_ticket, send_email, submit_order, update_record
    return invoke_tool(name, args)

This pattern forces a clear decision point before side effects occur. It also creates an audit trail that explains why an action was allowed or blocked.

Other common pitfalls include over-personalization, leaky memory that persists session-specific data, uncontrolled tool growth, and debugging blind spots caused by unclear boundaries. If you see these symptoms, it usually means responsibilities are not clearly separated.

What You Learned and Where to Go Next

Personalized AI agents are powerful, but they require discipline. The key insight is that personalization is a systems problem, not a prompt problem.

By separating reasoning from execution, structuring memory carefully, and using protocols like MCP to enforce boundaries, you can build agents that scale beyond demos and remain maintainable in production.

As you extend this system, resist the urge to add “just one more prompt tweak.” Instead, ask whether the change belongs in memory, tools, or orchestration.

That mindset will save you time as your agent grows in complexity.

If you’d like to continue the conversation, you can find me on LinkedIn.

*All diagrams in this article were created by the author for educational purposes.

You go in with the calm confidence of a developer who just wants to understand how things work. You open BluetoothAdapter.java and think, “Ah, this looks clean.” Then you click through a few methods. Suddenly, you’re in AdapterService.java, then StateMachine.java, and before you realize it, you’re staring at a JNI bridge leading straight into native C++ code that talks to daemons with names like bluetoothd.

Somewhere between the Binder calls, message queues, and “Unexpected state” logs, your curiosity quietly turns into existential dread.

That, my friend, is the Android Bluetooth experience.

But here’s the twist: it’s not chaos. It’s choreography. Every message, callback, and native call exists for a reason. Android Bluetooth has been built, rebuilt, and evolved over more than a decade to support everything from old-school car kits to cutting-edge LE Audio.

Underneath that ever-expanding complexity lies a remarkably disciplined foundation built on system design patterns. These patterns are the reason Bluetooth can still work across thousands of devices, dozens of chip vendors, and millions of random user interactions that happen every second.

What’s fascinating is how the Bluetooth stack mirrors Android’s entire design philosophy: isolate complexity, define clear roles, and let components communicate through predictable contracts.

The app layer talks to managers. The managers talk to services. The services talk to native daemons. And the daemons finally talk to the hardware. Each layer speaks its own language but follows a shared rhythm –like musicians who have never met but somehow stay in tune.

Without these patterns, the system would collapse under its own ambition. Imagine writing logic for pairing, bonding, discovery, connection, streaming, and low-energy data transfer without structure. Every change would be a minefield.

Design patterns bring sanity to this chaos.

• The Manager-Service split ensures clear boundaries.

• The State Machine keeps connection lifecycles predictable.

• The Handler-Looper mechanism turns concurrency into an orderly queue.

• The Facade hides native messiness behind friendly APIs.

• And the Observer pattern lets everyone stay updated without tripping over each other.

This article is about peeling back those layers and seeing the design ideas that quietly keep Android Bluetooth alive. We won’t just list patterns like a textbook. Instead, we’ll explore how each one appears in real AOSP code, why it exists, and how you can apply the same ideas to your own projects.

If you’ve ever wondered how something as temperamental as Bluetooth manages to stay mostly reliable, this is your backstage pass.

So grab your debugger, open a terminal window, and get ready to look at Bluetooth not as a mysterious black box, but as one of Android’s most elegant examples of long-term system design done right.

Table of Contents

1. The Manager–Service Pattern: Divide and Delegate

2. The Facade Pattern: Making Complexity Look Simple

3. The State Machine Pattern: Keeping Bluetooth Sane

4. The Handler–Looper Pattern: Message-Driven Concurrency

5. The Observer Pattern: When Bluetooth Talks Back

6. The Builder Pattern: Making GATT Bearable

7. The Strategy Pattern: Adapting to Different Devices

8. The Template Method Pattern: Common Flows, Custom Details

9. The Service Locator Pattern: Finding the Right Profile at Runtime

10. The Layered Architecture Pattern: From App to Radio Without Losing the Plot

11. Putting It All Together: Designing Bluetooth-Style Systems

The Manager–Service Pattern: Divide and Delegate

When you start exploring Android’s Bluetooth codebase, one of the first things you’ll notice is how often you come across the words “Manager” and “Service.” There is BluetoothManagerService, AdapterService, GattService, A2dpService, and many more.

At first, it seems repetitive and unnecessarily complicated. Why do we need so many layers just to connect to a pair of earbuds? Wouldn’t one class that says “connect” be enough? The short answer is no. The longer answer involves one of Android’s most reliable architectural habits: the separation of responsibility.

Think of a restaurant. The customers talk to the waiter. The waiter talks to the kitchen. The kitchen talks to suppliers. Everyone has a job. The waiter doesn’t need to know how to cook, and the chef doesn’t need to explain menu prices to customers. That separation is what keeps the whole operation smooth and manageable.

Android’s Bluetooth system works in exactly the same way. The Manager is like the waiter, the public face that interacts with apps, while the Service is like the kitchen, where the actual work happens out of sight.

When you write an app that uses Bluetooth, you might call something like BluetoothAdapter.enable() or BluetoothDevice.connectGatt(). These methods live inside Manager classes in the Android framework. They are deliberately simple, because their only job is to talk to the Bluetooth Service behind the scenes. That Service runs in another process entirely, one that has the necessary system permissions and the ability to interact with the native Bluetooth stack and hardware.

A small example from the Android source code shows this relationship very clearly:

public class BluetoothManagerService extends IBluetoothManager.Stub {
    private AdapterService mAdapterService;

    public boolean enable() {
        if (mAdapterService != null) {
            return mAdapterService.enable();
        }
        return false;
    }
}

At first glance, this looks trivial, but it demonstrates one of the most important ideas in the system. The BluetoothManagerService does not handle radio operations itself. Instead, it delegates to another internal class called AdapterService, which communicates with lower layers. That service will eventually pass instructions down to native C++ code, which then communicates with the Bluetooth controller chip through the Host Controller Interface.

This relay-style design has several advantages. The first is reliability. If the lower-level service crashes, the Manager layer can detect it and restart it, keeping the system stable. Because the Manager and the Service live in separate processes, your app will not crash when the service does. You might see Bluetooth temporarily toggle off and on again, but that recovery is intentional and automatic.

The second advantage is security. Every Bluetooth action goes through permission checks in the Manager layer before it reaches the Service. If an app without proper privileges tries to perform a restricted operation, the Manager stops it immediately. This prevents unsafe or malicious behavior and ensures that only trusted system components can access the hardware.

The third is flexibility. The Service layer can evolve without affecting the public API. That means Google and device manufacturers can modify or replace internal Bluetooth logic say, to support a new chipset or feature, without breaking existing apps. The Manager acts as a contract that remains stable even if the internal wiring changes.

If you trace what happens when you tap the Bluetooth toggle on your phone, you can see this pattern in action. Your tap calls BluetoothAdapter.enable() in the app layer. That call travels to BluetoothManagerService in the system server process. The manager checks permissions, then calls AdapterService.enable(). Inside the service, a JNI bridge triggers a native C++ function called enableNative(), which finally sends a command to the hardware abstraction layer. From there, it reaches the Bluetooth chip itself. Each layer knows its exact role.

This organization also makes debugging easier. If something goes wrong, you can tell whether it’s the Manager that didn’t send a message, the Service that failed to respond, or the native stack that stopped working. Each part logs its own activity in logcat, so you can follow the chain of events without guessing where the problem began.

At its core, the Manager–Service pattern is Android’s way of keeping large systems under control. It divides authority, enforces security, and lets the entire Bluetooth subsystem recover gracefully from errors. It may look complicated at first, but it is this design that makes Bluetooth remarkably resilient. Every time your phone connects to your car or your earbuds, it happens through this carefully choreographed handoff between the Manager and the Service. It’s a quiet partnership that keeps billions of connections running smoothly every single day.

The Facade Pattern: Making Complexity Look Simple

If the Manager–Service pattern is about dividing responsibility, the Facade pattern is about hiding chaos behind elegance. In many ways, this is the reason most Android developers can use Bluetooth without needing to understand what happens inside the stack.

The Facade pattern provides a friendly public face that masks a labyrinth of underlying operations, creating an illusion of simplicity while managing a tremendous amount of behind-the-scenes work.

To understand this, think about the front desk of a large hotel. When you check in, you talk to one receptionist. That person gives you your key, answers questions, and takes requests. You never meet the maintenance crew fixing the air conditioning or the kitchen staff preparing food or the team handling room cleaning schedules. Yet all those systems quietly operate through that one friendly front desk.

That front desk is the Facade. It provides a simple interface to a complex system, ensuring guests never have to deal with the hotel’s internal machinery.

Android’s Bluetooth framework works in the same way. Developers interact with high-level classes such as BluetoothAdapter, BluetoothDevice, and BluetoothGatt. These classes are the front desks of the Bluetooth system. They provide clean, easy-to-use APIs like enable(), getBondedDevices(), and connectGatt().

When a developer calls one of these methods, it looks straightforward. But beneath the surface, that call passes through multiple layers of services, IPC mechanisms, and native components before reaching the Bluetooth controller hardware.

Here is a simplified example to illustrate how this works in practice:

BluetoothGatt gatt = device.connectGatt(context, false, callback);

This single line looks simple. But in reality, it triggers an entire orchestra of operations. The call goes through the BluetoothDevice class, which forwards the request to BluetoothGatt. The BluetoothGatt instance then communicates with the system’s Bluetooth service through Binder IPC. That service eventually invokes native code that sets up an L2CAP channel, negotiates attributes, configures encryption, and starts the Generic Attribute Profile (GATT) procedure. None of that complexity is visible to the developer who wrote the original line.

This is what makes the Facade pattern so powerful. It provides abstraction without removing capability. The Android team knows that very few app developers want to worry about connection intervals, PHY configurations, or attribute protocol responses. They just want to connect to a device and get data. By exposing a Facade, Android lets developers stay productive while the internal layers handle the technical details.

If you look at the Android source tree, you can see this pattern clearly in how Bluetooth is organized. The classes in the android.bluetooth package are intentionally designed to be simple and self-contained. They never reveal how the system service works.

For example, BluetoothAdapter doesn’t know how to send HCI commands, and BluetoothGatt doesn’t know how to open a socket. Instead, they act as representatives, forwarding user requests to the Bluetooth Manager or the corresponding Service, which then interacts with the native stack.

This pattern is what makes the Bluetooth API approachable to beginners. Imagine if Android exposed every detail of the underlying protocols to developers. You would have to manually construct attribute requests, negotiate connection intervals, and handle packet fragmentation. The result would be technically accurate but completely unusable for most app developers. The Facade prevents that by serving as a translation layer between human expectations and machine complexity.

There is also a deeper design reason behind this approach. A Facade protects stability. Because developers only see the outermost layer, Android engineers can modify the internals without breaking existing apps. This allows the system to evolve freely, improving performance and adding new features while keeping the public API consistent.

The Bluetooth internals have changed countless times since the early days of Android, but BluetoothAdapter.startDiscovery() still works the same way it did a decade ago. That consistency is a direct benefit of the Facade pattern.

In a sense, the Facade pattern is about empathy. It respects the developer’s time by not forcing them to learn every Bluetooth nuance. It makes working with a complicated protocol feel human. Whether you are scanning for nearby devices, connecting to a smartwatch, or transferring data, you only need to call a few readable methods and handle a handful of callbacks. Behind those calls, a world of threads, sockets, and packet exchanges whirs silently to life, all hidden behind a calm, minimal interface.

So the next time you call BluetoothAdapter.enable() and your phone’s Bluetooth magically comes to life, remember that you are not flipping a simple switch. You are sending a message through a carefully designed Facade that talks to multiple services, native layers, and hardware interfaces. It is like pressing a single button on a spaceship console while a thousand mechanical parts start moving in perfect synchronization. You don’t see the complexity, and that is precisely the point.

The State Machine Pattern: Keeping Bluetooth Sane

If you have ever debugged Bluetooth connections, you have probably experienced moments of pure confusion. One minute the device says “Connecting,” then suddenly it jumps to “Connected,” then “Disconnected,” then “Connecting” again, and before you know it, you have no idea what the current state actually is.

Bluetooth is, by nature, an unpredictable environment. Devices move in and out of range, radio interference causes delays, and remote devices can behave differently depending on their chipsets. To make sense of all this unpredictability, Android relies on one of the most battle-tested concepts in computer science: the State Machine pattern.

A state machine is like a rulebook that defines how a system behaves depending on its current situation. Instead of reacting randomly to every event, the system maintains a clear notion of “state.”

For Bluetooth, these states might include Disconnected, Connecting, Connected, or Disconnecting. Each state knows exactly what actions are allowed and what transitions are possible.

For example, you can only go from Disconnected to Connecting when a connection attempt starts, and you can only go from Connecting to Connected if the handshake succeeds. If something happens that does not make sense for the current state, the system simply ignores it. This structure prevents chaos.

In Android’s Bluetooth implementation, almost every major profile uses a state machine. You can find them in classes like A2dpStateMachine.java and HeadsetStateMachine.java. Each one extends a generic StateMachine framework that Android provides. The structure is surprisingly elegant. You define individual classes for each state, implement their behaviors, and let the system handle the transitions. Conceptually, it looks like this:

class A2dpStateMachine extends StateMachine {
    private final State mDisconnected = new Disconnected();
    private final State mConnecting = new Connecting();
    private final State mConnected = new Connected();

    A2dpStateMachine() {
        addState(mDisconnected);
        addState(mConnecting);
        addState(mConnected);
        setInitialState(mDisconnected);
    }
}

Although the code may look technical, the idea is simple. Each “State” represents a specific mode of operation, and each one defines how to react to incoming events.

The system starts in Disconnected. When a “connect” command arrives, it moves to Connecting. When the connection completes, it moves to Connected. If the user turns off Bluetooth or the remote device disappears, it transitions back to Disconnected. Every action follows a logical, well-defined path.

This pattern is what keeps Bluetooth stable despite the messy nature of wireless communication. Without it, you would constantly end up with half-open connections, dangling callbacks, and undefined behaviors. Imagine a phone that still thinks it’s connected to your headphones long after you have turned them off. The state machine eliminates that by keeping a single source of truth for connection status.

Beyond correctness, the state machine pattern also improves readability and maintenance. Each state is self-contained, so developers can easily locate the logic that handles a particular situation. If you need to change how Bluetooth behaves when connecting, you only modify the Connecting class, not the entire codebase. This modularity makes the Bluetooth stack easier to evolve as new profiles and features appear.

There is also a subtle psychological benefit to using state machines. When debugging, engineers can trace log messages that indicate transitions, such as “A2dpStateMachine: Transitioning from CONNECTING to CONNECTED.” These logs act like a map of the system’s thought process. Instead of guessing what happened, you can follow a clear narrative of cause and effect. That is invaluable in a system as complex as Bluetooth, where timing issues can hide bugs that are otherwise impossible to reproduce.

State machines also ensure graceful recovery. Suppose a connection fails halfway through. Without structured states, the system might leave resources allocated or callbacks registered. But with a state machine, the Connecting state knows how to clean up before returning to Disconnected. This reduces leaks, power drain, and inconsistent user experiences.

Even at higher levels of Android, you can see the influence of this pattern. For example, when you toggle Bluetooth on or off, the adapter itself transitions through a sequence of states internally: Turning On, On, Turning Off, Off. This ensures that all dependent services, such as GATT and A2DP, are brought up or down in the right order. The pattern guarantees that nothing jumps ahead or lags behind during these transitions.

In everyday terms, the state machine pattern is like traffic lights for Bluetooth. It prevents every component from driving through the intersection at the same time. Each action has a green, yellow, or red light depending on the current situation. This orderliness is what keeps Bluetooth from descending into radio chaos every time multiple devices try to connect or disconnect at once.

So, the next time your phone automatically reconnects to your headphones after a short disconnection, remember that it is not luck. It is a carefully choreographed set of state transitions keeping track of where everything stands. Behind every smooth Bluetooth experience lies a quiet but dependable state machine making sure each event happens exactly when it should and never when it shouldn’t.

The Handler–Looper Pattern: Message-Driven Concurrency

If Bluetooth had a personality, it would be that friend who cannot sit still. It’s constantly juggling tasks: scanning for devices, maintaining connections, handling GATT operations, streaming audio, and sending data to the controller, all at once. Underneath that hustle is one of Android’s most reliable design foundations: the Handler–Looper pattern. This pattern is what keeps Bluetooth responsive, synchronized, and stable even when a dozen things happen at the same time.

To understand why it exists, imagine running a busy coffee shop with only one employee who tries to handle every customer request immediately. One person takes an order, makes the drink, cleans the counter, and washes the cups all in real time. Within minutes, chaos erupts. Customers start yelling, the counter gets sticky, and no one knows who’s being served.

Now, imagine a more organized system: every order goes into a queue, and the barista processes them one by one. That’s essentially how the Handler–Looper system works.

In Android, almost everything that involves background work happens through message queues. The Looper represents a thread that waits for messages, and the Handler is the entity that posts those messages into the queue.

Instead of letting different threads modify shared Bluetooth state directly, which could easily lead to race conditions, Android forces all Bluetooth operations to happen on specific threads managed by loopers. Messages arrive, get handled in order, and the system never loses track of what happened first or last.

Inside the Bluetooth system, this pattern appears everywhere. Each service, such as AdapterService, GattService, or A2dpService, has its own Handler running on a dedicated thread. When a Bluetooth event occurs, like “Device Connected” or “Start Discovery,” the event is wrapped in a Message object and sent to the appropriate Handler. That Handler then decides what to do next. The pattern turns what could have been a tangle of multithreaded chaos into a clear, sequential pipeline.

Here’s a simplified example inspired by Android’s real Bluetooth code:

private class AdapterServiceHandler extends Handler {
    @Override
    public void handleMessage(Message msg) {
        switch (msg.what) {
            case MSG_START_DISCOVERY:
                startDiscoveryNative();
                break;
            case MSG_STOP_DISCOVERY:
                stopDiscoveryNative();
                break;
        }
    }
}

This code might look plain, but it’s quietly doing something brilliant. Instead of running startDiscoveryNative() directly, the system posts a message saying, “Hey, when you get a chance, start discovery.” The Looper thread eventually picks up that message and executes it in the correct order. No two threads ever collide, and the main thread stays free to handle user interactions.

The beauty of this approach lies in its predictability. Bluetooth events often happen in unpredictable sequences: a connection attempt might fail while a scan is still in progress, or a new device might appear while another is being paired. Without strict message ordering, these overlaps could lead to deadlocks or inconsistent states. By channeling every operation through a single message queue, Android ensures that Bluetooth behaves deterministically, no matter how chaotic the radio environment becomes.

It also helps with thread safety. Instead of sprinkling locks everywhere in the code, Android simply guarantees that all critical Bluetooth work happens on the same thread. This means developers can focus on logic instead of worrying about synchronization bugs. It’s one of those design choices that looks simple but saves thousands of hours of debugging across devices and vendors.

There’s another hidden benefit too: graceful recovery. If something goes wrong inside a message handler, say a native call fails or a timeout occurs, the system can isolate that failure to a single message. The rest of the queue continues processing normally. This containment prevents one bad operation from crashing the entire Bluetooth stack.

When you watch logcat during a Bluetooth session, you can often see the Handler–Looper pattern in action. You’ll find lines like “MSG_START_DISCOVERY received” followed by “Starting discovery” and “MSG_STOP_DISCOVERY received.” Those logs are more than just printouts – they are breadcrumbs showing the system’s thought process as it moves through the queue.

In simpler terms, the Handler–Looper pattern is how Android Bluetooth keeps its cool. It takes a storm of asynchronous events, pairing requests, advertisements, data packets, disconnections, and lines them up in a single, calm queue. It ensures that everything happens in order, every time.

So, the next time your phone seamlessly switches from one Bluetooth speaker to another while still streaming music and scanning for your watch in the background, remember what’s quietly at work beneath it all. There’s a dedicated thread looping patiently, reading messages, and keeping order in a world of wireless chaos. It’s the unsung hero of concurrency, one message at a time.

The Observer Pattern: When Bluetooth Talks Back

Bluetooth is a chatterbox. It never works alone, and is always reacting to something. A device connects, another disconnects, a new advertisement appears, a bond is created, or a characteristic changes its value. The system needs to keep dozens of components informed about these changes in real time.

This is where the Observer pattern comes in. This pattern is all about communication, letting different parts of the system stay updated without constantly asking what’s going on.

The basic idea is simple. You have one source of truth that broadcasts updates, and you have multiple listeners that care about those updates. Whenever the source changes, it notifies everyone who subscribed. It’s like a news channel that sends breaking alerts to subscribers instead of waiting for each viewer to call in and ask, “Anything new today?”

In Android Bluetooth, this is how almost all notifications and callbacks are delivered. When your phone connects to a Bluetooth device, the Bluetooth system service sends out an event. The app doesn’t have to keep checking the connection status every second. Instead, it simply registers a listener that reacts whenever the connection state changes. That listener could be a BroadcastReceiver in the app or a callback interface provided by the framework.

For example, when a device connects, Android sends out a broadcast intent like this:

sendBroadcast(new Intent(BluetoothDevice.ACTION_ACL_CONNECTED));

Apps that have registered for this intent receive it automatically. They can then update their user interface, show a notification, or start another operation based on the new state. The same mechanism works for disconnections, bonding events, and discovery results. It’s an elegant way of keeping apps informed without them wasting energy by constantly polling the system.

At the GATT level, the Observer pattern takes a slightly different form. When you connect to a Bluetooth Low Energy device and subscribe to a characteristic, you provide a callback called BluetoothGattCallback. This callback has methods such as onConnectionStateChange() and onCharacteristicChanged(). Whenever the device sends new data, the system automatically invokes the appropriate callback on your behalf. You don’t need to ask for updates repeatedly – you simply react when they arrive.

The real beauty of this pattern is how decoupled it makes the system. The Bluetooth framework can notify multiple apps and services simultaneously without knowing anything about how they use the information. It just broadcasts an event and moves on. Each listener independently decides what to do with it.

This design is crucial for a multitasking operating system like Android, where Bluetooth events may be relevant to different components at the same time. For example, the system settings might need to update the connection icon, the media framework might need to route audio, and an app might need to sync data — all triggered by the same connection event.

The Observer pattern also helps with efficiency. Because updates are sent only when something changes, there is no unnecessary processing or battery drain from constant status checks. This design allows the Bluetooth stack to stay responsive while minimizing overhead, which is especially important for mobile devices that need to preserve both power and performance.

In practical terms, this pattern is what makes Bluetooth feel alive. When you open your Bluetooth settings and instantly see your device name appear or disappear, that’s the result of observers doing their job. They are always listening for broadcasts and updating the interface the moment something changes. Without this mechanism, your Bluetooth menu would lag or require manual refreshing just to stay current.

There is also a subtle reliability benefit. Observers can join or leave at any time without breaking the system. If one app crashes or unregisters its listener, others still receive updates normally. This flexibility ensures that the Bluetooth service remains stable even if individual apps behave unpredictably.

So, the next time your phone pops up a notification that your earbuds have connected or your smartwatch silently syncs in the background, remember that it is not magic. It’s the Observer pattern at work: a polite messaging system that lets Bluetooth quietly talk to everyone who is listening, all without raising its voice.

The Builder Pattern: Making GATT Bearable

If you have ever worked with Bluetooth Low Energy, you already know that the GATT layer can be a maze. The Generic Attribute Profile, or GATT, is how devices expose data to one another. It defines services, characteristics, and descriptors that describe everything from a heart rate monitor’s readings to a light bulb’s brightness. On paper, it’s beautifully organized. In practice, setting it up manually can feel like assembling furniture without instructions, using only an Allen key and pure faith.

When Android engineers designed the Bluetooth GATT APIs, they realized that developers would need a way to build these services and characteristics without losing their minds. That is where the Builder pattern comes in. This pattern is all about constructing complex objects step by step, instead of trying to do everything in one chaotic go.

Think of it like building a sandwich. You start with a base, then add layers: bread, sauce, lettuce, tomato, cheese, and so on. You can add or skip ingredients as needed, and by the end, you have a complete meal that makes sense.

The Builder pattern works the same way. It lets you create a GATT service one piece at a time, adding characteristics and descriptors in a readable, modular fashion.

In Android, a GATT service is represented by the BluetoothGattService class, and each piece of data it exposes is represented by a BluetoothGattCharacteristic. Instead of requiring you to manually wire all of these together in one long, confusing block, Android allows you to build them step by step, like this:

BluetoothGattService service = new BluetoothGattService(SERVICE_UUID,
        BluetoothGattService.SERVICE_TYPE_PRIMARY);

BluetoothGattCharacteristic characteristic =
        new BluetoothGattCharacteristic(CHAR_UUID,
                BluetoothGattCharacteristic.PROPERTY_READ | BluetoothGattCharacteristic.PROPERTY_WRITE,
                BluetoothGattCharacteristic.PERMISSION_READ | BluetoothGattCharacteristic.PERMISSION_WRITE);

service.addCharacteristic(characteristic);

Even though this looks simple, it reflects a powerful design philosophy. Each method call adds a new layer of configuration without breaking readability. You can look at the code and instantly understand what kind of service you’re creating, what characteristics it contains, and what permissions each one has. There are no massive constructors, no messy parameter lists, and no confusion about what goes where.

This pattern does more than make code pretty. It also prevents errors. GATT structures are very sensitive to incorrect configurations, for example if a characteristic lacks the right permission or if a descriptor is missing. By breaking the setup into small, incremental steps, the Builder pattern helps developers validate each part as they go. It’s much easier to debug a missing characteristic when each one is clearly defined, rather than buried inside a giant, monolithic block of code.

The same idea applies internally within the Android Bluetooth stack. When the system builds its own GATT tables or processes client requests, it follows the same step-by-step assembly model. Each stage of the process adds more detail to the overall structure. The result is not only easier to read but also more robust in handling changes.

There is also a psychological benefit to this approach. Developers can focus on one small piece at a time instead of feeling overwhelmed by the entire setup. It feels like progress, and it reduces the cognitive load that often comes with working on protocols like GATT, where small mistakes can cause big headaches.

In a broader sense, the Builder pattern in Android Bluetooth is a lesson in humility. It acknowledges that complex systems are built incrementally, not in one heroic line of code. It invites you to slow down, define what you need clearly, and construct it carefully. Whether you are setting up a health monitor or designing a custom BLE sensor, the Builder pattern ensures that your code remains clear and maintainable as your project grows.

So the next time you define a Bluetooth service in your app and everything just works, take a moment to appreciate the quiet genius of the Builder pattern. It’s the reason you can build an entire wireless data model with a few readable lines instead of a spaghetti of function calls. It turns the intimidating world of GATT into something almost enjoyable, a reminder that even in low-level systems programming, design elegance still matters.

The Strategy Pattern: Adapting to Different Devices

Bluetooth, as anyone who has worked with it knows, is not one single, predictable standard in practice. It’s more like a family reunion where every cousin claims to follow the same rules but each one interprets them differently. One device might handle extended advertising perfectly, another insists on using legacy commands, and yet another behaves strangely when it comes to pairing.

In this unpredictable world, Android cannot rely on one fixed set of behaviors. It needs a system that can adapt depending on what kind of device or chipset it is dealing with. This is where the Strategy pattern quietly saves the day.

The Strategy pattern is all about flexibility. It allows a system to choose between multiple approaches at runtime depending on the situation. Instead of writing huge if-else blocks to handle every possible scenario, developers define a common interface that represents a behavior, and then create different implementations of that behavior. The system can then pick the right strategy dynamically.

Imagine you are a chef who must cook for guests with different dietary preferences. You don’t rewrite the entire recipe each time someone says they are vegan or gluten-free. Instead, you have multiple cooking strategies, one for each diet, and you simply pick the right one when the order comes in. Android does the same thing with Bluetooth.

Inside the Bluetooth stack, different devices and chipsets support different capabilities. Some controllers can handle multiple advertising sets, some cannot. Some prefer extended packet formats, while others only understand the older legacy commands. To manage this diversity without making the code unreadable, Android uses interchangeable strategies.

For example, when the system needs to start Bluetooth advertising, it doesn’t hard-code every possible hardware path. Instead, it defines an abstract interface, something like:

interface AdvertisingStrategy {
    void startAdvertising();
    void stopAdvertising();
}

Then it provides specific implementations for each scenario, such as a LegacyAdvertisingStrategy and an ExtendedAdvertisingStrategy. Depending on the chipset capabilities, the system decides which strategy to use at runtime:

AdvertisingStrategy strategy = controller.supportsExtendedAdvertising()
        ? new ExtendedAdvertisingStrategy()
        : new LegacyAdvertisingStrategy();
strategy.startAdvertising();

This design keeps the code clean and extensible. If a new Bluetooth version introduces a new advertising method, developers can simply implement another strategy class without touching the existing ones. The same approach appears in connection handling, power management, and even encryption policies.

The Strategy pattern also allows for graceful fallback. Suppose a modern device supports extended advertising but something goes wrong, maybe the controller firmware has a bug. Instead of crashing, the system can quietly switch back to the legacy strategy. Users never notice the change, and Bluetooth continues working.

Beyond hardware adaptability, this pattern also simplifies testing. Developers can easily substitute one strategy with another in unit tests to simulate different hardware configurations. It encourages modularity, which is crucial for a system that runs across hundreds of Android devices made by dozens of manufacturers.

You can also see the philosophical elegance in how this pattern aligns with Bluetooth itself. The Bluetooth protocol is inherently designed for negotiation. Devices exchange capabilities, choose compatible settings, and then proceed. Android’s software architecture mirrors that philosophy at the code level. By using strategies, it lets the system negotiate internally too, not between devices, but between code paths.

From a practical standpoint, the Strategy pattern gives Android the superpower of evolution. As new Bluetooth versions emerge with new features like LE Audio, Isochronous Channels, or Periodic Advertising, Android can keep up simply by introducing new strategy classes. There is no need to overhaul the entire system or rewrite large chunks of legacy logic.

So when your phone seamlessly connects to both a five-year-old Bluetooth speaker and a brand-new pair of earbuds using LE Audio, it’s not luck. It is design. Underneath the surface, Android is quietly picking the right strategy for each device, making the whole experience look effortless. It’s one of those cases where smart architecture turns what could have been a compatibility nightmare into a smooth, invisible handshake between hardware generations.

The Template Method Pattern: Common Flows, Custom Details

In large systems like Android Bluetooth, not every part of the code can be entirely unique. Some operations follow the same general flow every time, but with small variations in the details. For example, connecting to a device, discovering services, or streaming audio all share similar high-level steps.

The pattern that allows Android to reuse these general flows while still letting each Bluetooth profile define its own personality is the Template Method pattern.

The essence of this pattern is simple: define the overall process once, but let subclasses decide how specific parts should behave. It’s like giving every chef in a restaurant the same recipe outline – prepare ingredients, cook, and plate – but letting each of them choose their own spices and techniques for flavor. The structure remains constant, but the details can vary.

Bluetooth needs this because different profiles, such as A2DP for audio or GATT for data exchange, often perform similar actions in slightly different ways. They all start connections, maintain states, and handle disconnections, but the way they handle timing, acknowledgments, or retries can differ. The Template Method pattern keeps these flows consistent while allowing room for customization.

Inside Android’s Bluetooth stack, you can see this pattern in how connection management is implemented. The process of connecting to a Bluetooth device typically follows the same structure: initialize the stack, attempt a connection, verify success, and then notify other components. Each profile, however, defines its own way of handling the lower-level details.

In conceptual form, it looks something like this:

abstract class BluetoothProfileConnection {
    public final void connect() {
        prepareConnection();
        performConnection();
        finalizeConnection();
    }

    protected abstract void prepareConnection();
    protected abstract void performConnection();
    protected abstract void finalizeConnection();
}

A class such as A2dpService or GattService would then implement the abstract methods in its own way. One might set up audio channels, while another negotiates attribute protocols. The overall template (prepare, perform, finalize) never changes. This is what keeps the Bluetooth system organized even when dozens of profiles coexist and evolve over time.

This pattern is particularly useful in a codebase as large as Android’s because it enforces discipline without killing flexibility. It ensures that every Bluetooth operation follows the same skeleton, which makes debugging and extending the system far easier. When an engineer wants to add a new feature or fix a connection bug, they already know where to look and which parts are shared or unique.

Another advantage of the Template Method pattern is that it reduces duplication. Without it, each profile might write its own version of “connect,” “disconnect,” and “reconnect,” each slightly different but doing almost the same thing. That would make the code hard to maintain and error-prone. With a template, the core logic lives in one place, and only the necessary variations appear in subclasses.

There is also an important design insight here: Bluetooth, like many communication protocols, is inherently procedural. You must do things in the correct order, initialize before connecting, connect before discovering, and discover before reading data. The Template Method pattern encodes this order directly into the architecture. It prevents accidental mistakes, such as skipping a required step or performing actions out of sequence.

From a broader perspective, this pattern teaches an important engineering lesson about balance. Too much abstraction, and systems become rigid and bureaucratic. Too little structure, and they turn into chaos. The Template Method pattern sits comfortably in the middle. It provides consistency while still leaving space for creativity and variation.

So the next time your phone connects to your car, switches to the right Bluetooth profile, and starts playing music without skipping a beat, you’ll know that there is a quiet choreography happening inside. Each profile follows the same dance steps – prepare, perform, and finalize – but each does it in its own rhythm. That harmony between structure and flexibility is what makes Bluetooth both powerful and adaptable.

The Service Locator Pattern: Finding the Right Profile at Runtime

At this point, we have seen how Android Bluetooth manages complexity through delegation, structure, and controlled flexibility. But there is still a practical question to answer: with so many Bluetooth services and profiles running in the system (like A2DP, GATT, HFP, MAP, HID, and more), how does the framework know which one to talk to at any given moment? When you stream audio, it needs A2DP. When you sync contacts, it needs PBAP. When you connect a keyboard, it needs HID. Android’s answer to this problem is the Service Locator pattern.

In the simplest terms, the Service Locator is a central registry that helps different parts of a system find the service or component they need without having to know where it lives. It’s like the information desk at a large airport. You don’t need to memorize the location of every gate or airline office – you just ask the information desk, and they point you to the right place.

Inside the Android Bluetooth system, this pattern appears everywhere, especially within the AdapterService and BluetoothManagerService classes. These services manage a variety of Bluetooth profiles, and each profile is responsible for its own behavior. Instead of hard-coding every possible profile into every part of the stack, Android maintains a registry where each service can be looked up dynamically.

Here is a simplified version of what this looks like conceptually:

public class AdapterService {
    private Map<Integer, ProfileService> mProfileServices = new HashMap<>();

    public void registerProfile(int profileId, ProfileService service) {
        mProfileServices.put(profileId, service);
    }

    public ProfileService getProfileService(int profileId) {
        return mProfileServices.get(profileId);
    }
}

When a Bluetooth operation occurs, such as starting audio streaming or initiating a data transfer, the system asks the AdapterService for the correct profile implementation. The Service Locator then returns the matching service instance, such as the A2DP service for audio or the GATT service for BLE data. Each profile operates independently, but the Service Locator acts as the phonebook that ties them all together.

This pattern solves several key problems. First, it removes the need for every part of the system to know about every other part. Without it, each class would have to keep track of dozens of others, creating a tangled web of dependencies. With a Service Locator, everything becomes more modular. Each component can register itself once and be discovered whenever needed.

Second, it makes the system flexible. Android devices can enable or disable certain Bluetooth profiles depending on hardware support or user configuration. For example, a smartwatch might only need GATT, while a car infotainment system needs A2DP, HFP, and MAP. The Service Locator allows Android to load only the relevant profiles at runtime instead of baking them all in permanently.

Third, it helps with scalability. As new Bluetooth profiles are introduced, such as LE Audio or Broadcast Audio, they can be added without rewriting existing code. The Service Locator acts as the central meeting point that stays the same even as new services join the system. It’s like a well-organized switchboard that never needs rewiring, no matter how many new phones, watches, or speakers show up.

From a debugging standpoint, this design also makes life easier. Developers can trace which service is currently active or verify that a profile is registered correctly simply by inspecting the registry. It provides a single source of truth that reflects the system’s state at any moment.

On a philosophical level, the Service Locator pattern represents Android’s pragmatic approach to complexity. Instead of trying to make every module aware of the entire Bluetooth world, it centralizes coordination in a controlled, predictable way. It acknowledges that Bluetooth is not a single, monolithic feature but an ecosystem of cooperating components that need a shared directory to find each other efficiently.

So when your phone automatically switches from streaming audio over A2DP to transferring a file over OBEX or syncing notifications with your smartwatch, it happens seamlessly because the system always knows exactly which profile to use. That knowledge comes from the quiet work of the Service Locator pattern, acting like a backstage coordinator ensuring that the right performer walks on stage at the right time.

The Layered Architecture Pattern: From App to Radio Without Losing the Plot

If there is one pattern that truly defines Android’s Bluetooth design philosophy, it is Layered Architecture. This is the invisible backbone that keeps the entire system structured, predictable, and scalable. In a world where Bluetooth involves everything from mobile apps to kernel drivers, layering is not just a matter of organization, but one of survival.

At first glance, Bluetooth might seem like a single feature. You turn it on, pair a device, and it works. But in reality, it’s a long, intricate journey that starts at the app layer, where you press “Connect”, and travels all the way down to the radio hardware, which emits electromagnetic signals into the air. Between those two points lies an entire vertical stack of software layers, each playing a distinct role, each isolated from the others by well-defined interfaces.

Think of it as a city with multiple levels. The top layer is where people live and work: that’s your app. Below that are roads and traffic systems, which are your Android framework services. Beneath that, you have subways and utilities, the native daemons written in C and C++ that handle protocol specifics. At the very bottom is the foundation, the hardware abstraction layer and the Bluetooth controller chip itself. Every level has a clear boundary. You can remodel one floor without collapsing the whole building.

Here is how those layers roughly line up in Android’s Bluetooth stack.

At the top layer, app developers interact with classes such as BluetoothAdapter, BluetoothDevice, and BluetoothGatt. These are part of the Android framework, written in Java or Kotlin, and serve as the public interface. They provide clean, stable methods like startDiscovery() and connectGatt(), hiding the technical chaos below.

The next layer down is the system service layer. This includes classes such as BluetoothManagerService and AdapterService. These are responsible for managing Bluetooth as a system feature, enforcing permissions, and coordinating multiple profiles. They act as the brain of the operation, processing commands, routing messages, and maintaining global state.

Below that is the JNI and native layer, written primarily in C and C++. This is where the logic gets closer to the metal. JNI (Java Native Interface) acts as a translator between the Java world and the native code. When a Java method like enable() is called, JNI forwards it to the native daemon that actually speaks Bluetooth protocol commands. This bridge keeps performance high while maintaining safety through strict boundaries.

Finally, we reach the hardware abstraction layer (HAL) and the Bluetooth controller. The HAL defines how the operating system interacts with the underlying hardware. It sends and receives HCI (Host Controller Interface) packets, the low-level binary messages that control the Bluetooth chip. From there, the controller takes over, turning digital instructions into radio signals that travel invisibly through the air to another device.

The brilliance of this design is in how each layer only needs to know about the one directly below it. The app layer never worries about the hardware, and the hardware never needs to know about the app. This clear separation makes it possible for Android to run across thousands of devices built by different manufacturers using different chipsets. It is a pattern that enforces order through boundaries.

There are practical benefits, too. The layered architecture makes the system modular. For instance, when new Bluetooth features arrive, like LE Audio or Bluetooth 5.4, Android engineers can modify only the relevant layers. The app APIs at the top can remain stable while the lower layers evolve to support the new specifications. This is how Android manages to maintain backward compatibility while still introducing new capabilities with every release.

The layering also helps with debugging and reliability. When something breaks, engineers can trace the issue by moving down through the layers like a detective. If an app crashes, the problem is likely near the top. If packets are missing, the issue may be in the native layer or HAL. Each layer leaves its own signature in the logs, helping developers pinpoint where things went wrong.

This pattern also teaches a timeless software design lesson: complexity becomes manageable only when divided. The layered architecture prevents the Bluetooth stack from turning into a tangled mess of cross-dependencies. It lets Android evolve gracefully rather than collapse under the weight of its own history.

So when you tap “Pair new device” on your phone and watch your earbuds connect, remember that your request travels down a carefully organized highway of software, from the app you see, through the framework, into native code, across the hardware abstraction, and finally out into the air as a radio signal. Every piece knows its role, every layer does its part, and together they make Bluetooth feel effortless. The magic of wireless connection is not just in the radio waves, but in the architecture that makes those waves behave.

Putting It All Together: Designing Bluetooth-Style Systems

By now, it’s easy to see that Android’s Bluetooth stack is not just a pile of random services and classes. It’s a carefully choreographed system built on timeless design principles that keep it reliable, flexible, and surprisingly elegant despite its complexity.

Each pattern – the Manager–Service split, the Facade, the State Machine, the Handler–Looper, the Observer, the Builder, the Strategy, the Template Method, the Service Locator, and the Layered Architecture – exists for a reason. Together, they form the invisible scaffolding that allows Bluetooth to connect billions of devices every day without falling apart.

The magic of these patterns is not that they make Bluetooth simple. Bluetooth will never be simple, as it’s an enormous specification with quirks, edge cases, and competing priorities. What these patterns do instead is make the system manageable. They turn unpredictability into structure, they replace chaos with order, and they make it possible for teams of engineers around the world to work on the same stack without tripping over each other.

If you step back, you’ll notice that every pattern in the Bluetooth system reflects a deeper philosophy:

• The Manager–Service pattern teaches the value of separation.

• The Facade reminds us that good design hides unnecessary complexity.

• The State Machine shows the power of predictability.

• The Handler–Looper demonstrates the beauty of serialized concurrency.

• The Observer proves that communication doesn’t require coupling.

• The Builder celebrates incremental construction.

• The Strategy encourages adaptability.

• The Template Method enforces discipline without rigidity.

• The Service Locator maintains organization in a crowded ecosystem.

• And the Layered Architecture ties it all together, ensuring that every piece fits logically into the whole.

These same ideas extend far beyond Bluetooth. You can apply them to almost any software system, a web service, a game engine, or even a simple mobile app. The principles remain the same: divide responsibilities, enforce clear boundaries, keep your interfaces stable, and design for change rather than permanence.

Systems that last are not the ones that are perfect on day one. They are the ones that can grow without collapsing under their own weight.

Android Bluetooth has been evolving for more than a decade. It has absorbed new technologies like LE Audio, Fast Pair, and broadcast audio. It has adapted to new hardware, new chipsets, and new use cases. Yet, at its core, the same patterns continue to guide it. That consistency is the reason Bluetooth on Android, despite its quirks, works as well as it does. It’s not just a story of wireless communication, it’s a story of good architecture.

So the next time you tap “Connect” on your phone and your earbuds instantly respond, pause for a moment. Beneath that single tap lies an orchestra of design patterns working in perfect harmony: managers delegating to services, handlers processing messages, observers reacting to broadcasts, and strategies choosing the right behavior for your hardware. It’s a quiet miracle of software design, a reminder that even the most invisible features on your device are built with care, patience, and an eye for long-term evolution.

And if you ever find yourself building a complex system that seems impossible to manage, take a cue from Android Bluetooth. Start small, define your layers, choose the right patterns, and let structure do the heavy lifting. The real magic in engineering isn’t in writing clever code. It’s in designing systems that stay calm, even when the world around them isn’t.

For me, a system means a set of rules, processes, and triggers that guide the entire workflow. This helps you conserve energy and not have to figure things out in the moment. It keeps things productive, efficient, and consistent.

Now that the chaos has settled, I’ve been thinking a lot about systems – not just in life, but in tech. I wish I had applied the same principles of system design earlier.

In this article, we’re going to explore real-world system design examples from domains like gaming and job platforms. These industries don’t just scale massively – they also demand high availability, low latency, and seamless customer experiences. Understanding how they’re built is a powerful way to level up your thinking as a developer or architect.

What We’ll Cover

1. Introduction: What is System Design and Why Scale Matters

2. Approaches to System Design

   • 1. Bottom-Up Approach

   • 2. Top-Down Approach

   • 3. Hybrid Design

3. Important Concepts in System Design

   • Full Stack Web Application Components

   • How Computers Talk to Each Other (The Internet)

   • The Problem of Growth

   • Speeding Up Your Website with Caching and CDNs

   • Building Your Application: Monolith vs. Microservices

   • The APIs

   • Handling Real-Time Data

   • Databases

   • Understanding the CAP Theorem

   • Rate Limiting and Monitoring

4. Case Studies: Scaling in the Real World

   • Case Study 1: Scaling a Job Search Application

   • Case Study 2: Scaling an Online Gaming Application

5. Q&A

6. Final Notes

7. Conclusion

Introduction: What is System Design and Why Scale Matters

System design is the process of defining the architecture, modules, interfaces, and data of a system.

In other words, system design means explaining the different parts of a system, like its structure, building blocks (modules), and components.

It’s a process used to define, develop, and design a system in a way that meets the specific needs of a business or organisation.

The main goal of system design is to give enough information and details about the system, and to properly implement its parts using models and views. Let’s now talk about the different parts of a system.

Elements of a System:

• Architecture: This is a basic structure or model that shows how the system works, looks, and behaves.
  We often use flowcharts to explain and represent this architecture.

• Modules: These are smaller parts or sections of the system. Each module handles a specific task. When all modules are combined, they make the complete system.

• Components: These provide a specific function or a group of related functions. Components are usually made from one or more modules.

• Interfaces: This is the connection point where different parts (components) of the system exchange information with each other.

• Data: This refers to managing information and how it flows through the system.

Why System Design Matters

System design is important for a number of practical reasons. First, it can help companies and teams solve complex business problems and make sure they thoroughly analyse all requirements before building. It also reduces the chance that errors will be introduced into processes while making design phases more efficient and structured. Finally, it helps you efficiently gather and present your data in a useful format and improves the overall quality of the system.

Approaches to System Design

There are several methods you can use to approach system design. The main ones are:

1. Bottom-Up Approach

In this method, the design starts from the lowest-level components or subsystems. These small parts are gradually combined to form higher-level components. This process continues until the entire system is built as one complete structure.

The more abstraction we use, the higher the level of the design becomes.

Advantages:

• Components can be reused in other systems.

• It’s easier to identify risks early.

• It helps in hiding low-level technical details and can be combined with the top-down approach.

Disadvantages:

• It’s not very focused on the overall structure of the problem.

• Building high-quality bottom-up solutions is hard and time-consuming.

2. Top-Down Approach

Here, the design starts from the entire system, and you break it down into smaller subsystems and components as you go. Each of these subsystems then gets broken down further, step by step, creating a hierarchical structure.

In simple terms, you start with the big picture and keep dividing it until you reach the smallest parts of the system.

To sum up, design starts with defining the whole system, then continues by defining its subsystems and components. When all definitions are ready and fit together, the system is complete.

Advantages:

• The focus is on understanding the requirements first, which leads to a responsive and purpose-driven design.

• It’s easier to handle errors in interfaces between components.

Disadvantages:

• Components can’t be reused easily in other systems.

• The resulting architecture is often less flexible or not very clean.

3. Hybrid Design

The hybrid design approach is a mix of top-down and bottom-up methods. Instead of committing to just one way, it takes the strengths of both. You start by looking at the overall system (like in top-down) so that you don’t lose sight of the big picture. At the same time, you also focus on building solid, reusable modules or components (like in bottom-up).

In simple terms, you first plan the big picture, then create smaller components that can work independently, and finally combine everything into a cohesive system.

For instance, in our sports team site, we’d use top-down to define the whole fan journey (homepage → match details → live scores). But bottom-up, we’d build modular components like authentication or stats tracking, which can later be reused in new features like ticket booking or merchandise sales.

Advantages:

• You get the clarity of a top-down plan while still building reusable modules.

• It strikes a balance between high-level design and detailed implementation.

• Risks are easier to manage since you’re considering both structure and components.

Disadvantages:

• It can be complex to manage since you’re juggling two approaches.

• Requires more coordination between teams working on different levels.

• It might take more time compared to using a single approach.

Important Concepts in System Design

Before exploring core components, I want you to first understand two key concepts:

• Full stack web application components

• How computers talk to each other (via the internet)

Full Stack Web Application Components

A full-stack web application is a software application that combines both the frontend (what users see and interact with) and the backend (the server, database, and logic that power the app) into one complete system.

Generally, simple websites don’t require much system design – and in some cases, no system design at all. But when it comes to viral applications or platforms offering complex services, system design becomes essential. Most modern applications are full-stack applications, meaning they involve multiple interconnected layers working together.

Here’s a simplified overview of a typical full-stack application:

Before diving deep into each of these components, let me first give you a quick, high-level overview of what they are and how they fit into the bigger picture (starting from the bottom of the image above).

• Frontend – The user interface where people interact with your application.

• Backend – The logic and brain of the application that processes requests.

• APIs – The bridge that allows communication between frontend, backend, and external services.

• Database – The storage system where all your structured information lives.

• Server – The infrastructure that hosts, runs, and delivers your application.

Now, we need to understand how computers talk to each other.

How Computers Talk to Each Other (The Internet)

When you type a website's URL into the browser – and this site could be a simple portfolio site or a full stack app – how does your computer know where to send the request? It uses the Domain Name System (DNS). The DNS is like a phonebook for the internet – it translates a human-readable website name, like "example.com," into a unique numeric IP address that computers can understand.

Once your computer has the IP address, it uses communication protocols to send and receive data. One of the most important protocols is TCP. It breaks data into small, numbered packets. If a packet gets lost or arrives out of order, TCP ensures it's resent and reassembled correctly, making it a very reliable way to send data.

On top of TCP, we use higher-level protocols like HTTP. This is an application-level protocol that's easier for developers to use. It's the language your browser speaks to the server.

HTTPS is the same, but it adds an extra layer of encryption for security.

Now that we understand the basics of the Internet, remember that it serves billions of people worldwide.

Let’s break this down with a real-life example. Imagine you own a restaurant with a seating capacity of 50 people. One day, 10 extra guests arrive – and with a bit of adjustment, you still manage. But suddenly, a thousand more people show up at your door. What would you do then? It’s not just about adding more chairs and tables anymore – you’ll need extra food supplies, more staff, and a bigger setup to handle such massive traffic.

This simple example reflects the real challenge of growth and scalability. And that’s exactly what I’ll be diving into in the next chapter of this tutorial.

The Problem of Growth

Imagine you've built a simple website for a local sports team. Initially, it's just you and a few friends using it, so a single server is sufficient. This server holds all the website's logic and connects to a single database where player stats are stored.

As the team becomes more popular, though, more people visit your site, and it suddenly becomes slow. This is a scaling issue. Your system can't handle all the new traffic.

Scaling Your System: Two Main Ways

There are two ways to solve this. The first is vertical scaling. This is like giving your one server a bigger engine and more memory. You'd upgrade the CPU (the brain) or add more RAM (temporary memory). You could also use a faster disk storage like an SSD.

The problem is, you can only upgrade so much before you hit the limits of what's available. Plus, if that single server fails, your entire website goes down.

A better approach is horizontal scaling. This means adding more servers instead of just upgrading one. Now you have a team of servers, and each can handle a portion of the incoming user requests.

This approach allows for almost unlimited growth. It also creates redundancy and fault tolerance, because if one server breaks, the others can pick up the slack, and your site stays online.

Directing Traffic with a Load Balancer

With multiple servers, you need a way to make sure no single server gets overwhelmed. This is where a load balancer comes in. It acts like a traffic cop, sitting in front of your servers and directing each new request to the best-suited server. It uses different algorithms to decide where to send the traffic.

For example, the Round Robin method sends requests to servers one by one, in a cycle. Another method is Least Connection, which sends the request to the server that has the fewest active connections.

Speeding Up Your Website with Caching and CDNs

Imagine your website is now used by people all over the world. A user in another country might experience slow loading times because their request has to travel all the way to your servers.

To fix this, you can use a Content Delivery Network (CDN). A CDN is a network of servers around the world that store copies of your website's static files – like images, videos, and text files. When a user requests one of these files, the CDN serves it from the closest server, making the website load much faster for them.

This process is a form of caching. Caching is the general idea of making copies of data and storing them in a faster-to-access location. You can cache data on your server so it doesn't need to fetch the same player stats from the database every time. This reduces the load on your database and speeds up the entire application.

You can read more about the difference between CDNs and caching here.

Building Your Application: Monolith vs. Microservices

As your website grows, its code can become a tangled mess. You might start with a monolith, where all the features (like player stats and live scores, in our example) are built into a single, large program. A monolith is easier to start with, but it can be hard to manage and update.

A better approach for a large-scale application is to use a microservice architecture. This means breaking your application into smaller, independent services, each with a specific job. For example, one service could handle player stats and another could handle live scores. This makes your code more organised and easier to update, because a change in one service won't affect the others.

With microservices, you need an API Gateway. This acts as a single entry point for all user requests, directing them to the correct microservice behind the scenes. It also handles security and other common tasks.

The APIs

Think of APIs (Application Programming Interfaces) as the “middlemen” that let different pieces of software talk to each other.

In simple terms, an API is like a waiter in a restaurant. You (the user) tell the waiter what you want, the waiter takes your order to the kitchen (the system), and then brings the food (data or result) back to you.

Without APIs, your app, website, or software wouldn’t know how to ask another system for information or services.

For example, on our sports team website:

• The front-end (what fans see) uses an API to fetch player stats from the database.

• When someone buys match tickets, the API talks to the payment system to confirm the transaction.

• If fans want live score updates, the API makes sure the real-time data flows smoothly from the server to their screen.

So, APIs are important for system design because they shape how efficiently different systems connect, share data, and stay reliable under real-world use.

Your front-end and back-end services can communicate in several ways. The most common is a REST API. It's a standardised set of rules that uses HTTP to create a consistent way for a client and server to talk to each other. For example, it defines a standard way to signal a successful request ("OK") or a server error ("Internal Server Error").

When to use REST

• Best when: you need simplicity, broad adoption, and easy integration with browsers, mobile apps, or third-party services.

• Example: CRUD apps (blogging platforms, e-commerce sites, user management).

• Strength: Human-readable JSON, stateless, widely supported.

• Weakness: Over-fetching (getting more data than needed) or under-fetching (not enough data).

Another style is GraphQL. Instead of getting all the data a REST API provides, GraphQL lets the client ask for only the specific data it needs, which can make things faster and more efficient.

When to use GraphQL

• Best when: clients (like mobile apps) need fine-grained control over exactly what data they fetch.

• Example: social media feeds, dashboards with lots of widgets, mobile apps with limited bandwidth.

• Strength: Flexible queries, reduces over-fetching, strong typing system.

• Weakness: More complex server setup, which can cause performance issues if queries aren’t optimised.

For server-to-server communication, gRPC is often used. It's known for being very fast because it uses a more efficient data format called Protocol Buffers instead of JSON.

When to use gRPC

• Best when: services talk to each other in microservice architectures, and speed/efficiency is critical.

• Example: real-time systems (streaming, payments, IoT, machine learning inference).

• Strength: Super fast (binary Protocol Buffers), built-in support for streaming, strong contracts.

• Weakness: Not browser-native (needs extra tooling for web), harder debugging compared to REST

So to summarize based on my observations of what I have worked on so far:

• If you’re building something public-facing and widely consumed → go for REST.

• If your app has complex, dynamic queries from clients → go for GraphQL.

• If you’re dealing with high-performance internal service-to-service calls → go for gRPC.

In system design, choosing the right API style directly affects performance, scalability, and user experience. If you pick REST for its simplicity, GraphQL for its flexibility, or gRPC for its speed, you’re shaping how well your system can grow and adapt as real-world demands change.

Handling Real-Time Data

Real-time data handling is challenging because it requires maintaining an active connection to continuously transmit and receive data simultaneously. Traditional servers follow a request–response model, where data is only sent when explicitly requested.

That's where WebSockets come in. Unlike HTTP, which is a one-and-done request-and-response model, a WebSocket creates a continuous, two-way connection between the client and server. This allows the server to send updates to the user as soon as they happen, creating a real-time experience.

When microservices need to communicate without being directly connected, they can use message queues. A service sends a message to the queue, and another service picks it up when it's ready. This helps to decouple the services, so they don't have to worry about the other service being available at that exact moment.

On our sports site, WebSockets allow fans to see live scores instantly without refreshing the page – just like in chat apps, but here it keeps the excitement of the game alive in real time

Databases

Databases are a critical part of any full-stack application because they serve as the permanent home for user data. Once you’ve decided how to scale your servers and manage communication, you also need to consider the database layer. If everything else scales but the database does not, it can quickly become a bottleneck – leading to crashes, inconsistent records, or even data loss.

Many applications rely on relational databases (SQL), which store data in structured tables with rows and columns and are great for handling structured information. But for applications requiring high flexibility or handling massive unstructured datasets, NoSQL databases (like MongoDB or Cassandra) are often chosen. These databases don't follow the strict rules of SQL and are better for handling massive amounts of data.

They follow ACID properties:

• Atomicity: A transaction is all or nothing.

• Consistency: The data always remains in a valid state.

• Isolation: Multiple transactions don't interfere with each other.

• Durability: Once a transaction is complete, the data is permanently saved.

Just like with servers, you might need to scale your database. You can use sharding, which divides your data across multiple databases, or replication, which creates copies of your database to handle more read requests.

Understanding the CAP Theorem

When you're dealing with a distributed system and multiple databases, you inevitably face trade-offs. The CAP Theorem states that you can only guarantee two out of the following three properties at the same time:

• Consistency – Every user sees the same, most up-to-date data.

• Availability – The system is always available to respond to requests.

• Partition Tolerance – The system continues to operate even if a part of the network fails.

Now, from a system design perspective, this theorem forces us to make conscious architectural choices. For example, in financial applications (like banking), consistency often takes priority over availability because even a small inconsistency in balance data can cause chaos.

On the other hand, in social media feeds, availability and partition tolerance are often prioritised – it's okay if you see a slightly outdated post, but the system should never be down.

In the flow we’ve been discussing, whenever we introduce a new component or scale out across multiple regions, we need to reassess which two guarantees matter most for our business case. That decision directly drives what database technology we pick, how we design failover strategies, and what trade-offs we accept in user experience.

In short, the CAP theorem isn’t just a theory – it’s a practical compass. It guides us to balance user expectations, business priorities, and technical feasibility without breaking existing functionality, while still leaving room for future growth.

Rate Limiting and Monitoring

When designing a system, it’s not just about making it work – it’s about making it resilient. Two core guardrails here are rate limiting and monitoring.

What is Rate Limiting?

Rate limiting is the practice of controlling how many requests a user, client, or service can make to your system within a given timeframe. For example, you might cap an API at 100 calls per user per hour. This prevents abuse, safeguards against denial-of-service attempts, and ensures fair usage across all consumers.

Rate limiting comes into play any time your service is exposed publicly or internally to multiple clients.

To incorporate it, you can implement limits at the API gateway, reverse proxy (like NGINX), or within your service logic itself. Many cloud providers (AWS API Gateway, GCP Endpoints) also have built-in support.

What is Monitoring?

Monitoring is the practice of collecting metrics, logs, and traces from your system to understand its health in real time. Typical signals include:

• Error rates (for example, how often requests fail)

• Latency (how long requests take)

• Traffic volume (load across the system)

• Resource utilisation (CPU, memory, disk, and so on)

Monitoring is important from day one – it’s your feedback loop. Without it, you’re essentially flying blind.

To work it into your system, you can use observability stacks like Prometheus + Grafana, or managed solutions like Datadog, New Relic, or CloudWatch. You can also set alerts for threshold breaches (for example, 5% error rate spike).

In practice, rate limiting and monitoring work hand-in-hand. Rate limiting proactively guards against overload, while monitoring gives you visibility into whether the limits are working, whether scaling is needed, or whether a new type of failure is emerging.

For example, if you’ve designed a booking system (like in our earlier flow), rate limiting would ensure a single user can’t spam seat reservations, while monitoring would flag anomalies such as unusual spikes in request volume or sudden latency increases – helping you act before the system collapses.

Why Does This Matter for System Design?

These topics matter for good system design because they form the foundational building blocks of how modern applications actually operate in the real world. The way systems communicate, the type of APIs we adopt, and how we manage real-time interactions directly influence whether a product feels fast, reliable, and seamless – or slow and frustrating. In short, they determine how well the overall experience holds up when real users put it to the test.

When we develop a deeper understanding of how computers communicate, we begin to see the inner mechanics of client–server architecture – how APIs fetch data from databases through backend system calls. From this baseline, we can pivot into higher-level concerns:

• Scalability and resilience: Using load balancers to protect against server overload.

• Security: Introducing rate limiting to mitigate potential cyberattacks.

• Efficiency: Choosing the right type of API calls and leveraging caching/CDNs for speed and reduced overhead.

• Reliability: Implementing logging and monitoring to detect issues early and debug faster.

Together, these practices elevate a system from simply working to being robust, performant, and future-ready.

We’ve discussed the basics of all the most important concepts you’ll need to understand before building an end-to-end system. Now it’s time to deep dive into the case studies, where I’ll show you how different types of applications use system design to scale and serve billions of users.

I have picked services that are complex to build and handle multiple different types of components at a time, like gaming, education, and job search platforms.

Now let’s decode each of them together, and I’ll explain how I would scale the application if I were the developer building it.

Case Studies: Scaling in the Real World

System design is best understood when you see it in action. To show how principles like scaling, caching, load balancing, and real-time data management come together, let’s walk through two very different types of applications:

• A job search platform (focused on structured data and reliability).

• An online gaming platform (focused on real-time speed and responsiveness).

Looking at both will show you that, while the tools and concepts may be similar, the way we apply them depends on the type of system we’re building.

Both are high-traffic platforms, but with totally different needs. The job portal is about accuracy, reliability, and data-driven workflows, while the gaming platform is about instant responsiveness, fairness, and global reach.

In a job portal, a 1-second delay just means waiting. In a gaming app, a 1-second delay could mean losing the match. Both are failures – but for completely different reasons, and with different consequences.

Together, they show how the same building blocks of system design (scaling, caching, APIs, monitoring) are applied differently depending on context.

Case Study 1: Scaling a Job Search Application

A job search platform is one of the most used applications nowadays, as there are always people looking for a job. And there are many different job portals out there that handle the complete process, from finding jobs to user onboarding.

We’ll look at an example site called Upstaff. It’s a platform that focuses on hiring AI engineers as its core service (although it services other job profiles as well). At its core, it handles structured information – things like user profiles, job postings, and applications.

Day one, you have a few hundred users. On day one hundred, you may have tens of thousands. And in a year? Possibly millions. That growth means you have to think about scale, speed, and data integrity from the start.

🔹 The Core Components

• User Management: registration, login, and role-based access (job seeker vs employer).

• User Profiles: résumés, skills, preferences, stored in structured databases.

• Job Posting and Listings: employers create jobs, seekers browse/search/filter.

• Application Tracking: Every job seeker’s application status needs to be accurate and up to date.

• Recommendation Engine: jobs matched to users based on history and profile.

• Notifications: alerts for new job matches, recruiter replies, deadlines.

Every one of these features depends on the system’s ability to handle large amounts of structured data – and handle it reliably.

Step 1: Starting Small

At the beginning, everything can run on one server with a single database. This setup is enough for a few thousand users.

Step 2: Growth and Traffic Spikes

As more users join, the single server starts to slow down. To fix this, we add a load balancer and scale horizontally – adding multiple servers that share the traffic.

Step 3: Database Challenges

Soon, the database becomes the bottleneck. Searching across thousands of jobs slows things down. To fix this, we:

• Use sharding (split the database by user IDs or job IDs).

• Add a cache (like Redis) to store frequent queries such as “Software Engineer in New York.”

• Use a CDN to deliver logos, profile pictures, and other static files faster.

Step 4: Heavy Features

New features like a résumé parser or recommendation engine require extra computing power. Instead of overloading the main app, we move these into separate microservices.

Step 5: Security and Reliability

Finally, as traffic grows, we add:

• Rate limiting to stop any one user from spamming APIs.

• Monitoring to track errors, latency, and user activity in real time.

• API Gateway to ensure all requests are secure and validated. Here is an overview of the entire system scaling in an image :

This example shows how careful planning makes growth smooth. By scaling horizontally, caching smartly, and splitting heavy features into microservices, a job portal like Upstaff can handle millions of users without breaking.

Case Study 2: Scaling an Online Gaming Application

Now let’s flip the script. In a gaming platform like this site, speed and responsiveness matter more than anything. A 1-second delay in a job search is annoying. But in gaming, a 1-second delay can make players quit forever. Unlike job portals, the biggest challenge here is real-time responsiveness. A tiny delay can ruin the user experience.

🔹 The Core Components

• User Management Service: accounts, profiles, and login.

• Game Lobby and Matchmaking: pair players by skill, region, and latency.

• Game Server Manager: spin up and manage live matches.

• Real-Time Communication: powered by WebSockets or UDP for low latency.

• Game State Store (Redis): fast sync of health, scores, and positions.

• Leaderboard & Stats Engine: global rankings, achievements, and progress.

• In-Game Economy: coins, tokens, inventory.

• Payment Gateway: subscriptions and purchases.

• Anti-Cheat Security Layer: fairness across all players.

• Monitoring and Logging: server uptime, latency, and crash reports.

Unlike a job portal, every millisecond counts.

Step 1: Starting Small

At first, one powerful server is enough to run both the game logic and user accounts. With just a few players, things run smoothly.

Step 2: More Players, More Problems

As millions of players log in, the single server crashes. To fix this, we:

• Add a Game Server Manager that spins up separate servers for each match.

• Introduce a load balancer that assigns players to available servers.

Step 3: Real-Time Data Handling

In gaming, speed is everything. Instead of slow HTTP, we switch to WebSockets or UDP for instant communication. To keep everyone’s game view in sync:

• Use in-memory databases like Redis for positions, scores, and health.

• Update leaderboards in near real time.

Step 4: Scaling Features

Other services run in parallel:

• Matchmaking service pairs players by skill, location, and latency.

• Economy service manages coins, rewards, and in-game items.

• Payment gateway handles subscriptions and purchases securely.

• Notification system sends updates like “new event starting.”

Step 5: Global Expansion and Security

When the game expands worldwide:

• Use a CDN to deliver maps and skins quickly to all regions.

• Add an Anti-Cheat layer to detect and block unfair play.

• Build an Admin and Monitoring panel to track system health and user behavior.

In gaming, system design focuses less on structured data and more on low latency, real-time communication, and fairness. Scaling here means keeping gameplay smooth and secure, even when millions of players join at once. Here is the image representation of the complete game platform system design

Why Both Case Studies Matter

You might wonder – why show two different systems instead of just one? The answer is that system design isn’t “one-size-fits-all.”

• The job portal teaches us how to scale structured, data-heavy applications where reliability and accuracy matter most.

• The gaming platform shows us how to design for speed, real-time communication, and fairness under extreme load.

Together, these examples prove that the same system design principles of scaling, caching, monitoring, and microservices apply everywhere. What changes is how you use them to solve the unique challenges of your platform.

Q&A

How to get into system design if you don’t understand anything (yet)?

I get this question all the time – and the first thing you need to know is that system design isn’t some separate, elite domain. It’s an additional skill that complements your development journey.

If you're a full-stack developer (or aiming to be one), learning system design gives you a huge edge. After all, building an app isn't just about making it work – it’s about making it work well at scale.

So if you’re just starting and don’t even know how to become a full-stack developer yet, start there. Learn to build applications first, and then system design will start making a lot more sense. Read this guide How to Become a Full-Stack Developer in 2025 (and Get a Job) – A Handbook for Beginners to learn how to become a full-stack developer.

How do you understand system design concepts?

The short answer: with time and consistent practice.

Think of it like this: if you know how to use a pencil, it’s up to you whether you use it to sketch or to write. The pencil is just a tool. Similarly, in system design, once you understand the core concepts, it’s about knowing when and where to apply them. The rest – frameworks, tools, and technologies – are just means to an end.

It’s not about memorising patterns, it’s about developing the instinct to use the right building blocks at the right time.

What tools should you know before diving into system design?

The truth is, the list keeps growing. New tools and platforms are constantly emerging. But in my experience, having a solid foundation in the following areas makes a huge difference:

• Full Stack Development – so you understand how both frontend and backend systems interact.

• Cloud Platforms (like AWS, GCP, or Azure) – because most modern systems are cloud-native.

• CI/CD Pipelines – for automating testing, integration, and deployment.

• Deployment Strategies – to know how to roll out new changes with minimal risk.

Mastering these gives you the technical muscle to design systems that are scalable, reliable, and production-ready. I am a frontend developer, why should I know the system design

What resources should I study to learn system design?

In my last article, I shared all the resources that helped me learn system design.

System design is crucial for building reliable, high-performance applications. I explored the following resources:

• Free GitHub system design repository

• Another free GitHub system design repository

• Free System Design Handbook

• Free Hands-on System Design Learning Platform

• Free Intro to System Design Blog

Case studies and real-world architectures can also help you understand large-scale systems. You can follow any big tech engineering blog (Uber has a great one).

For high-level concepts, I went through the Grokking System Design course. It’s a paid resource, and I used it to deepen my understanding of system design. It’s not mandatory, but it helped me think about architecture at scale.

Note: there are other sites and courses out there of course, but I only share what I have personally experienced and used, and I focus on FREE material first.

Where to practice system design

This is where real learning begins. Start by picking any existing application from the internet, just like I did. Google something specific, like “job application portal,” but avoid the results on the first page. Those apps are usually well-optimised and already follow best practices in system design.

Instead, dig deeper and explore results from the second or third page. Look for an app that seems to be in its early stages.

Once you find one, try to understand how the entire application works. Break it down into its core components and then imagine what would happen if that app started receiving 1 million users a day. You’ll naturally begin to see what system design elements are needed to handle that kind of load.

Final Notes

Learning system design becomes much easier when you’ve already built something. Let’s say you’ve created an app and now you're thinking about how to scale it – that’s where real learning begins. The moment you start writing down your requirements (like how your app should behave when it starts getting more traffic), you naturally begin to develop system-level thinking. It’s this process of planning and anticipating real-world usage that turns theory into a practical skill.

Conclusion

Full Stack + System Design = The Ultimate Developer Stack 🔥

By mastering these skills, you can turn any idea into a real-world product, secure high-paying jobs, and even start your tech venture.

Now it's your turn – what are you building next? Let me know!

That’s all from my side. If you found this article helpful, feel free to share it and connect with me. I’m always open to new opportunities:

• Follow me on X: Prankur's Twitter

• Connect with me on LinkedIn: Prankur's LinkedIn

• Follow me on GitHub: Prankur’s Github

• View my Portfolio: Prankur's Portfolio

As projects grew in complexity and teams worked in parallel across different stacks, it became clear that monolithic approaches couldn’t keep up. I needed practical tools that allowed easy cross-app interaction, independent deployability, better team autonomy, framework-agnosticism, and more. Some solutions worked elegantly in theory but struggled in real-world conditions. Others made things messier and more painful than helpful.

After diving deep into different paradigms—from iframes to Web Components, single-spa, Module Federation, Piral, Luigi, and hybrid setups—I even distilled my proven experience into a full-fledged online course on Udemy.

And today, in this comprehensive hands-on tutorial, I want to share my expertise and tell you more about micro-frontend architecture—method by method—with code, tradeoffs, visuals, and real-world insights.

Table of Contents

• What are Micro Frontends For?

• Method #1: Iframes & Cross-Window Messaging

• Method #2: Web Components (Custom Elements + Shadow DOM)

• Method #3: Single-SPA — The Meta-Framework Approach

• Method #4: Module Federation - Sharing Code at Runtime

• Other Tools & Ecosystem Additions

• Final Thoughts

What are Micro Frontends For?

In traditional frontend development, we often build single, monolithic apps—one codebase, one repo, one deployment pipeline, one team. It works great for small to medium projects, sometimes even for larger ones.

But challenges arise when:

• Your frontend codebase expands beyond 50+ components.

• Multiple development teams need autonomy over different parts and tech stacks.

• Different sections require varying deployment frequencies (weekly or monthly).

• You need to integrate diverse frameworks, like combining React features with an Angular-based CMS.

This is where micro frontends step in.

Micro frontends extend the principles of microservices to the frontend world. Instead of one big frontend app, you build independent frontend modules, each owned by a team, using its own tech stack, deployed separately, and integrated at runtime.

Think of it like Lego blocks:

• Each block is similar to a self-contained micro frontend.

• They plug into a shared layout or shell.

• Each can evolve, update, or be replaced without affecting the others.

For example, imagine that you’re building a modern e-commerce site, and here’s what your business side expects from you:

Each team wants autonomy, and with micro frontends, each of these sections becomes a separate app, loaded dynamically into a shell at runtime.

Why It’s Getting Popular?

Here are a few things everyone considers:

1. Independent deployments – A little or no effort to coordinate every release.

2. Team autonomy – Teams choose their own stack and tools on the project.

3. Incremental upgrades – Migrate legacy apps piece by piece incrementally without the need to rewrite the whole app at once.

4. Technical agnosticism – Vue, React, Angular? Doesn’t matter. They can all work together seamlessly at the same time in a single app.

5. Better scalability – Parallelize work across teams to enable efficiency of delivery and scale at ease.

Now let’s discover how we can bring this idea to life in our projects.

Nowadays, there are different ways to achieve that, but not all solutions are equal. The implementation method you choose will drastically affect:

• Developer experience

• Bundle sizes and performance

• SEO and accessibility

• Runtime stability

• Interoperability across stacks

So let’s begin by exploring the oldest, but still surprisingly viable method.

Method #1: Iframes & Cross-Window Messaging

You may ask, “Aren’t iframes bad?” They’re often misunderstood. While yes, iframes can feel clunky and isolated, they’re also the most secure and decoupled way to host micro frontends—especially when you don’t trust the team on the other side.

What Is an IFRAME?

An iframe (inline frame) is an HTML element that allows you to embed another HTML page within your current webpage. The whole communication between apps is strictly based on events and delivered by means of the Post Message API.

If you need to send data to another app, you simply call the postMessage() method on that element. On the other side, to receive a message, you just have to subscribe to the message event. That’s it.

Real-World Example

Let’s see a simple example of two apps communicating with each other using iframes on two apps:

• The Main Web App

• A Search App.

Every iframe must be hosted somewhere to serve static content from it. It can be AWS Amplify, Digital Ocean, Heroku, GitHub Pages, or alike.

To help you out here, here’s an official GitHub guideline explaining how to host a website on their platform.

Let’s say you deployed a Search App on Github Pages and you were given this URL to host your app: https://example.github.io. Now let’s write some content for it.

Assuming that you want to post messages from the Search App to the Main Web App, and to subscribe to the incoming messages from it there. You can do it in this way:

console.log('Initializing Search App...');

// Subscribe to messages from outside the iframe (like Main Web App)
window.addEventListener('message', (event) => {
  if (event.data?.type === 'init') {
    console.log('Main Web App passed userId:', event.data.userId);
  }
});

// Simulate sending Search results back to Main Web App
window.parent.postMessage({
  type: 'searchResult',
  payload: ['Item A', 'Item B']
}, '*');

Here, you initialize the search app and set up two-way communication with a parent application (such as a main web app) using the Post Message API. You listen for incoming messages using the built-in message event. Once received, that message becomes available in the event.data object. Finally, you simulate sending data back to the parent by posting a searchResult message containing a list of items. This setup enables isolated iframe-based apps to communicate safely with the main shell application.

Then, in the DOM of the main web app, you need to include the iframe that will render the search app, specifying the URL to the hosted search app in this way:

<iframe
  id="search-mfe"
  src="https://example.github.io"
  style="width: 100%; height: 200px; border: none;"
></iframe>

Styles were added here to ensure that the iframe displays seamlessly within the layout for a cleaner UI integration.

And now you can pass some content from the main web app down to the search app and get some messages from it. You can accomplish it in the main web app’s JavaScript code in this way:

console.log('Initializing Main Web App...');

const iframe = document.getElementById('search-mfe');
iframe.onload = () => {
  // Send message to child iframe (inputs)
  iframe.contentWindow.postMessage({ type: 'init', userId: 42 }, '*');
};

window.addEventListener('message', (event) => {
  // Receive data from the Search App (outputs)
  if (event.data?.type === 'searchResult') {
    console.log('Received result from Search App: ', event.data.payload);
  }
});

As you see, when the iframe loads, the init event is sent to the search app (the type can be anything you want, just ensure it matches the one that another app expects from you). And then, in the message event handler as before, you can receive the incoming messages from the search app, and do something with them.

Here are a few pros and cons to consider, along with popular use cases:

✅ Pros:

• Strong sandboxing: No shared memory, no shared styles.

• Zero dependency clashes: One iframe is equivalent to one environment.

• Perfect for legacy: Easy to wrap old apps in an iframe.

• Practical for micro-apps in PHP, Java, Razor (ASP.NET)

❌ Cons:

• Slow rendering

• Difficult shared navigation

• Inconsistent/complicated styling

• Complex communication

• Must be hosted somewhere

👨🏻‍💻 Popular Use Cases

• Embedding legacy dashboards (for example, old AngularJS or Java apps)

• Secure cross-domain apps (for example, payments, 3rd party analytics)

• Highly untrusted integrations

• Embedded Ads

But if you want a more fluid UX, shared components, and a smoother dev experience, you’ll want something better. That brings us to Web Components.

Method #2: Web Components (Custom Elements + Shadow DOM)

“What if you could ship a self-contained natively understood widget that works in any framework — React, Vue, Angular, or plain HTML?”

That’s exactly what Web Components make possible. They’re natively built into the browser as an API, you don’t need a framework or extra dependency. They allow you to create reusable, scalable, encapsulated UI elements that work just like native HTML tags.

Moreover, you can easily use them as wrappers around any elements from other UI frameworks (React, Angular, Svelte, etc) and use your framework-based components as regular native DOM elements in any web application.

They are, in many ways, the ideal foundation for micro frontends.

A web component is made of:

• Custom Element - defines your own HTML tag (<user-profile>) and behavior

• Shadow DOM – provides scoped, encapsulated styles and DOM structure

• HTML Template – brings reusable HTML blocks/fragments

• Slots – acts as placeholder areas for host content (used in content projection)

In web components, you have to sync the data (input/output) via:

• Attributes (inputs):

  • In Javascript: element.setAttribute(), element.getAttribute(), and so on.

  • In HTML: <element attr1=”value1” attr2=”value2”></element>

• Properties (inputs) – element.someProp = value (only Javascript)

• Custom Events (outputs) - new CustomEvent('name', data)

First, let me show you a basic implementation of a web component, and then you’ll learn how to leverage it for micro-frontends.

Assuming that you’re building a reusable product-tile component that must:

• Accept one input parameter – “title”

• Send an output event "add-to-cart" with this “title” to the outside world, when the component is mounted to the DOM.

Here’s how this web component could look:

// product-tile.js
class ProductTile extends HTMLElement {
  // Specify which attributes (inputs) to observe for changes
  static get observedAttributes() { return ['title']; }

  constructor() {
      super(); // Call base HTMLElement constructor (obligatory)
      // Create a Shadow DOM for style and DOM encapsulation
      const shadow = this.attachShadow({ mode: 'open' });
      // Populate Shadow DOM with a DIV container where React will render the player
      shadow.innerHTML = `<div id="title"></div>`;
  }

  // Built-in Lifecycle Reaction.
  // Called when the custom element ProductTile is added to the DOM
  connectedCallback() {
      // When added to the DOM, read and render the title attribute
      const title = this.getAttribute('title') ?? 'Unnamed Product';
      this.updateTitle(title);

      // Dispatch a custom event with the current title
      const event = new CustomEvent('add-to-cart', {
          detail: { title },
          bubbles: true,
          composed: true,
      });

      this.dispatchEvent(event);
  }

  // Built-in Lifecycle Reaction.
  // Called whenever observed attributes change.
  // In our case it's "title" only
  attributeChangedCallback(name, oldValue, newValue) {
      if (name === 'title' && oldValue !== newValue) {
          this.updateTitle(newValue);
      }
  }

  // Internal method to safely update the title content
  updateTitle(title) {
      const titleElem = this.shadowRoot.querySelector('#title');
      titleElem.textContent = title;
  }
}

customElements.define('product-tile', ProductTile);

Now, let me explain what’s happening here:

• First, you create a custom element class that extends from HTMLElement or its children. This gives you access to web component lifecycle hooks and DOM integration capabilities.

• If you want to react to changes in input parameters (attributes), you have to define a static observedAttributes() getter that returns a list of attribute names to watch. In our case, we observe “title”.

• Then, in the constructor:

  • Call super() to properly inherit from HTMLElement.

  • Create a shadow DOM using attachShadow({ mode: 'open' }). This encapsulates your component’s internal DOM and styles. You can even use a closed mode here to add a higher level of isolation to the shadow DOM.

  • Then, populate the shadow DOM with minimal inner HTML—in this case, a <div> element that will later display the product title.

• When the component is added to the DOM, the built-in connectedCallback() lifecycle reaction runs:

  • It reads the current value of the "title" attribute.

  • Updates the UI with an initial value in the "title" attribute.

  • Then it dispatches a custom event named "add-to-cart", passing the "title" as detail down to it. The events are bubbles: true and composed: true, so that parent elements or host apps outside the shadow DOM can subscribe to it and catch it.

• When the title attribute changes at runtime, another built-in lifecycle reaction named attributeChangedCallback() runs automatically:

  • It checks the new value and updates the "title" display accordingly.

  • This enables reactive behavior in the component—similar to input bindings in UI frameworks.

• Finally, you register the component globally using customElements.define() method (it’s available in the global window object), giving it:

  • A tag name of <product-tile> that can be used anywhere in HTML.

  • A reference to the custom element you previously created to associate one with another.

Ultimately, here’s how you can use this component in your apps, which will work in vanilla JS, React, Angular, Svelte, Vue, whatever UI framework you choose:

<product-tile title="Coffee Mug"></product-tile>

And then you can listen to the "add-to-cart" event from inside ProductTile component like so:

const elem = document.querySelector('product-tile');
elem.addEventListener('add-to-cart', e => {
  console.log('Add to cart!', e.detail);
});

As you see, no ReactDOM.render, no NgModule, no extra glue. Everything is entirely native, pure JavaScript code that browsers understand.

And now, due to the Shadow DOM and other Web Components’ features, you can easily wrap and embed any web app written in a different framework into the Shadow Tree that will isolate your app entirely and won’t allow its layout or styles to leak out.

Alternatively, if you decide to publish it as a separate npm package (for example, @webcomp/product-tile), you can even dynamically import and mount the Web Component like so:

import('@webcomp/product-tile').then(() => {
  // Now <product-tile> is defined — you can create and use it
  const elem = document.createElement('product-tile');
  elem.setAttribute('title', 'Wireless Mouse');
  document.body.appendChild(elem);
});

Or load from CDN or any hosting provider:

<script type="module" src="https://example.github.io/product-tile.js"></script>

It’s simple, clean, and independent.

But you’re not here just for that, right? :) Now, let’s learn the real power of Web Components in a micro-frontends world!

Micro-Frontends with Web Components

Imagine that you’ve built a Video Player in React—or perhaps want to reuse one from another team. Now the question is: How can you make this React-based player usable in any other frontend application, regardless of its underlying framework, using Web Components?

Let’s figure it out!

Let’s say, this video player:

• Accepts src and controls as inputs

• Emits events: play and pause as outputs

• Can be used in any app via <magic-player> in this way:

    <magic-player
      src="https://cdn.example.com/video.mp4"
      controls="true"
    ></magic-player>
  

Now let’s get to implementation!

🔹 Step #1: Include your React player in the project

Here, you can play around with any React component of your choice, to be honest, or you can just use a simple React Video Player like the one below:

// ReactVideoPlayer.jsx

import React from 'react';

export function ReactVideoPlayer({ src, controls, onPlay, onPause }) {
  return (
      // HTML5 video element with full width and controls enabled
    <video
      width="100%"
      controls={controls}  {/* Enable / Disable controls */}
      onPlay={onPlay}      {/* Callback for play event */}
      onPause={onPause}    {/* Callback for pause event */}
    >
      <source src={src} type="video/mp4" />
      Your browser does not support the video tag.
    </video>
  );
}

🔹 Step #2: Create the Web Component Wrapper

Now, you need to create a Web Component wrapper around this React player app by mounting it into the shadow DOM of a custom element in this way:

// magic-player.element.js

// Define a new custom element class
class MagicPlayerElement extends HTMLElement {
  constructor() {
    super(); // Call base HTMLElement constructor (obligatory)

    // Create a Shadow DOM for style and DOM encapsulation
    const shadowRoot = this.attachShadow({ mode: 'open' });
    // Populate Shadow DOM with a DIV container where React will render the player
    shadowRoot.innerHTML = `
        <div id="react-video-player"></div>
    `;
  }
}

customElements.define('magic-player', MagicPlayerElement);

Then you need to add inputs and outputs like so:

// magic-player.element.js

// Define a new custom element class
class MagicPlayerElement extends HTMLElement {
  // Specify which attributes (inputs) to observe for changes
  static get observedAttributes() { return ['src', 'controls']; }

  constructor() {
    super(); // Call base HTMLElement constructor (obligatory)

    // Create a Shadow DOM for style and DOM encapsulation
    const shadowRoot = this.attachShadow({ mode: 'open' });
    // Populate Shadow DOM with a DIV container where React will render the player
    shadowRoot.innerHTML = `
        <div id="react-video-player"></div>
    `;
  }

  // Helper-like method to dispatch native-like events (our outputs)
  // In our case, it will be triggered for "onPlay" and "onPause" events
  dispatch(eventName, detail = {}) {
      const event = new CustomEvent(eventName, {
      detail,            // Pass custom data ("onPlay" or "onPause")
      bubbles: true,     // Allow event to bubble up
      composed: true     // Allow it to cross the Shadow DOM boundary
    });
    this.dispatchEvent(event);
  }
}

customElements.define('magic-player', MagicPlayerElement);

And lastly, add two built-in lifecycle reactions to render a React video player app when the page loads and every time the inputs change:

// magic-player.element.jsx

// Define a new custom element class
class MagicPlayerElement extends HTMLElement {
  // Specify which attributes (inputs) to observe for changes
  static get observedAttributes() { return ['src', 'controls']; }

  constructor() {
    super(); // Call base HTMLElement constructor (obligatory)

    // Create a Shadow DOM for style and DOM encapsulation
    const shadow = this.attachShadow({ mode: 'open' });
    // Populate Shadow DOM with a DIV container where React will render the player
    shadow.innerHTML = `
        <div id="react-video-player"></div>
    `;
  }

  // Helper-like method to dispatch native-like events (our outputs)
  // In our case, it will be triggered for "onPlay" and "onPause" events
  dispatch(eventName, detail = {}) {
      const event = new CustomEvent(eventName, {
      detail,            // Pass custom data ("onPlay" or "onPause")
      bubbles: true,     // Allow event to bubble up
      composed: true     // Allow it to cross the Shadow DOM boundary
    });
    this.dispatchEvent(event);
  }

  // Built-in Lifecycle Reaction.
  // Called when the custom element <magic-player> is added to the DOM
  connectedCallback() {
    this.render();
  }

  // Built-in Lifecycle Reaction.
  // Called whenever observed attributes change.
  // In our case it's "src" and "controls"
  attributeChangedCallback() {
    this.render();
  }

  // Render the React player inside the container
  render() {
    const src = this.getAttribute('src');
    const controls = this.getAttribute('controls') === 'true';
    const mount = this.shadowRoot.querySelector('#react-video-player');

    ReactDOM.createRoot(mount).render(
      <ReactVideoPlayer
        src={src}
        controls={controls}
        onPlay={() => this.dispatch('play')}
        onPause={() => this.dispatch('pause')}
      />
    );
  }
}

customElements.define('magic-player', MagicPlayerElement);

🔹 Step #3: Connect your React-Player to any UI framework:

Then, in the main web app (whatever UI framework you’re using there). We put our newly created React video player wrapper in any place in the DOM, passing down initial attributes (inputs) to it:

<!-- Use your new React-based player anywhere! -->
<magic-player
  src="https://cdn.example.com/movie.mp4"
  controls="true"
></magic-player>

And then you can easily subscribe to the custom events (outputs) from inside the React app:

// Listen to native-style events from the custom element
const magicPlayer = document.querySelector('magic-player');
magicPlayer.addEventListener('play', () => {
  console.log('Video has started playing!');
});

magicPlayer.addEventListener('pause', () => {
  console.log('Video has been paused.');
});

That’s it! Now, try to accomplish the same with a different UI framework!

✅ Pros

• Framework-agnostic: Works in React, Angular, Vue, Svelte, or even plain HTML — no rewrites needed

• Natively supported by browsers: No need for external libraries or frameworks — just HTML, JS, and CSS.

• No extra configuration or hosting needed as in iframes. But still, components can be published to npm/CDNs and reused across multiple apps.

• Intuitive & easy communication: Expose native DOM attributes as inputs and native custom events as outputs.

• SSR-friendly with hydration: It supports serialization, declarative shadow DOM, and can be server-rendered and hydrated, especially using modern tools.

• Supports Accessibility (ARIA attributes and roles).

❌ Cons

• Integration Difficulties: If you want to bridge two apps in different technical stacks, you need to properly manage their communication in a custom element wrapper and its shadow DOM.

• Limited Support for old Browsers: If you need compatibility with legacy browsers like Internet Explorer 10, Web Components need a polyfill. But here’s a popular repository with all polyfills for Web Components: https://github.com/webcomponents/polyfills

• Global State Isolation: There’s no built-in way to share state across components. You’ll need to implement your own global bus or event bridge using CustomEvents or alike.

👨🏻‍💻 Popular Use Cases

• Reusable Design systems & UI libraries

• Micro frontends inside framework apps

• Legacy integration to modern stack and vice versa

• Cross-team component delivery

• CDN-based plug-and-play UIs

The Web Components API has many more possibilities and power. So, if you want, you can go deeper and advance your knowledge by passing any available free course on freeCodeCamp or passing the one I’ve built myself around this technique on Udemy.

Now let’s move on!

Method #3: Single-SPA — The Meta-Framework Approach

“What if instead of embedding micro frontends as Web Components or iframes, we had a system that orchestrated multiple SPAs together in one layout?”

That’s what single-spa is all about. It’s not a rendering library, it’s a runtime JavaScript router and orchestrator for micro frontends.

Source: https://single-spa.js.org

What Is single-spa?

single-spa (Single Page Application) lets you build and run multiple independent SPAs (React, Vue, Angular, and so on) inside one webpage. Each SPA is responsible for part of the UI and is loaded dynamically depending on the current route.

In short, it’s a framework that:

• Loads your micro frontends when needed

• Mounts/unmounts them cleanly

• Coordinates routing and lifecycles

• Supports different frameworks in the same app.

Real-Life Example

Let’s say you have this route breakdown:

Each one is a fully independent SPA, and single-spa loads them as needed.

🔹 Step #1: single-spa installation

First, you need to install the single-spa as a dependency for your project:

# Create a new project (if it's not yet)
npm init

# Install Single SPA
npm install single-spa systemjs

Notice that we also installed the systemjs package. This package is responsible for the dynamic runtime module loading that makes Single-SPA work seamlessly. It uses SystemJS as a module loader to allow micro frontends to be:

1. Loaded at runtime

2. Independently deployed

3. Framework-agnostic

4. Lazy-loaded only when needed

Now you need to implement each micro-app. For instance, let’s see how the @shop/products app written in React could be managed.

🔹 Step #2: Project Structure

The project structure for each micro app can look like this:

shop/products/
├── src/
│   ├── root.component.jsx
│   └── index.single-spa.js
├── public/
│   └── index.html
├── package.json
└── webpack.config.js

🔹 Step #3: Root Micro App Component

The root.component.jsx file represents the root of the React app that will be mounted to the main DOM using single-spa. Here’s a simple example:

// src/root.component.jsx
import React from 'react';

export default function Root() {
  return (
    <div style={{ padding: '1rem', border: '1px solid #ccc' }}>
      <h2>🛍 Product Micro App</h2>
      <p>This is a micro frontend powered by React + Single-SPA!</p>
    </div>
  );
}

🔹 Step #4: Set Up Lifecycle Hooks

Also, each Micro App in single-spa requires an entry point with at least three core functions/lifecycle hooks. For that purpose, you will need a separate file, which you can name as index.single-spa.js and it will provide the implementation of those hooks, like:

• bootstrap() - Called when the micro app is launched by the main app (Shell) before mounting to the DOM

• mount() - Called when the app is attached to the host in the DOM

• unmount() - Called when the app is removed/detached from the DOM

And here’s an example of what they could look like:

// src/index.single-spa.js

import React from 'react';
import ReactDOM from 'react-dom/client';
import Root from './root.component.jsx';

// Hold the React root instance for reuse
let root = null;

// Called once when the micro frontend is first initialized
export function bootstrap() {
  return Promise.resolve();
}

// Called every time the route matches and the app should appear
export function mount(props) {
  return Promise.resolve().then(() => {
    const container = document.getElementById('product-container') || createContainer();
    root = ReactDOM.createRoot(container);
    root.render(<Root />);
  });
}

// Called when the route no longer matches (cleanup)
export function unmount() {
  return Promise.resolve().then(() => {
    if (root) {
      root.unmount();
    }
  });
}

// Create a container div if it doesn't exist
function createContainer() {
  const div = document.createElement('div');
  div.id = 'product-container';
  document.body.appendChild(div);
  return div;
}

As you see, you have to resolve a Promise in all lifecycle hooks and ensure the React app is mounted and unmounted properly based on the React best practices.

🔹 Step #5: Configuring Webpack for SystemJS

Also, each micro-app in single-spa needs a separate configuration. For that, you will include a webpack.config.js file, specifying how to build the app (output), where to host it (publicPath), and so on.

Since single-spa uses the SystemJS package, the libraryTarget will be system for all micro apps.

// webpack.config.js
module.exports = {
  externals: {
    react: 'React',
    'react-dom': 'ReactDOM',
  },
  output: {
    filename: 'products.js',
    libraryTarget: 'system', // SystemJS-compatible format
    publicPath: 'http://localhost:8500/', // Host location of this micro app
  },
};

This app will be hosted on the localhost:8500. For production, you will have to use any suitable hosting provider (like the ones described in the iframes section).

🔹 Step #6: Registering the Micro App in Root-Config

Next, it’s time to register a new micro-app in the Singla-SPA root config. Here’s how you can do it:

Create a root-config.js file in the root of the project and fill it with this content:

// root-config.js (host shell)
import { registerApplication, start } from 'single-spa';

registerApplication({
  name: '@shop/products',
  app: () => System.import('@shop/products'),
  activeWhen: ['/products'],
});

start(); // Initializes routing and micro app lifecycles

First, you have to register the application, and then you start it to enable routing and the micro app lifecycle. The registration for other micro apps will look the same.

Note: System.import() is part of SystemJS, used by default in single-spa for loading remote apps.

Also, single-spa comes with so-called "Parcels" – a lower-level construct in comparison to applications. They’re essentially self-contained pieces of UI that you can dynamically mount anywhere. Think of them like “mini microfrontends” or reusable widgets that don’t control routing:

// Example
mountParcel(SomeParcelComponent, { domElement: document.getElementById('micro-app') });

You’d use them when:

• You don’t want the parcel to own a route.

• You need to inject a micro frontend dynamically inside another one.

• You want encapsulated logic (like a widget) embedded within a larger app.

In all other cases, prefer the usage of a registerApplication(...) function.

🔹 Step #7: Adding Micro App to SystemJS Import Map

The last step is to register the micro app in SystemJS. For that, in your root index.html file, you need to add the following two scripts:

<!-- public/index.html -->

<!DOCTYPE html>
<html lang="en">
<head> <title>Micro Frontend Shell</title> </head>
<body>
  <nav>
    <a href="/products">Products</a> |
    <a href="/checkout">Checkout</a>
  </nav>

  <!-- Import maps handled by bundler or injected at runtime -->
  <script type="systemjs-importmap">
    {
      "imports": {
        "@shop/root-config": "http://localhost:9000/root-config.js",
        "@shop/products": "http://localhost:8500/products.js",
        // other micro apps
      }
    }
  </script>

  <!-- Start the root-config application -->
  <script>
    System.import('@shop/root-config');
  </script>
</body>
</html>

First, you have to add a script with an import map declaration. As you see, it represents a JSON where:

• Each key is the micro app name and

• Each value is the URL where the main JS file (from the bundle) actually lives

Note that we’ve added the @shop/root-config here to the import map to tell SystemJS where to fetch the main JavaScript file for the main/shell app so it knows how to resolve and execute System.import('@shop/root-config') properly.

Secondly, you include another script to start the main / shell application. It executes the JS file you just mapped in the import map above. Treat it as the real “boot” of your shell app:

<script>
  System.import('@shop/root-config');
</script>

That’s it! Now go ahead and try doing the same with other micro-apps in Vue (Checkout App) and Angular (Account Dashboard).

Here’s a simple diagram illustrating this connection:

Now that you’ve registered and integrated your first micro app, you might be wondering if this approach right for you. Let’s quickly look at the benefits and limitations of using single-spa in production.

✅ Pros

• Built-in Routing & Lifecycles - No need to reinvent navigation or mounting logic

• Cross-framework support - React, Vue, Angular can all co-exist

• Fine-grained loading - Only load the active app (lazy and efficient)

• Flexible project structure - can be monorepo or polyrepo

• Good CLI tooling - create and link MFEs with create-single-spa & helpers

❌ Cons

• Complex learning curve - Lifecycle APIs and SystemJS can be intimidating

• Configurations can get verbose – Managing multiple registries, import maps, deployment URLs, and lifecycle wrappers across apps adds setup overhead

• Shared state is manual - You must implement custom global state solutions

• Hard to SSR - Designed for full client-side rendering

• More boilerplate - Each app needs wrappers for lifecycles, routing, and so on.

• Global styles leak - No default encapsulation like Shadow DOM

And a few popular use cases for it:

👨🏻‍💻 Popular Use Cases

You can use single-spa when:

• You want a central router managing all micro frontends

• Teams are using different frameworks

• You prefer full SPA experiences over isolated widgets

• You don’t mind some boilerplate for orchestration

• You’re okay with a purely client-side setup

Let’s move on!

Method #4: Module Federation - Sharing Code at Runtime

“What if your micro frontends could load each other’s components, modules, or libraries at runtime — without iframes, without import maps, and without repackaging?”

That’s exactly what Module Federation, introduced in Webpack 5, makes possible. It’s fairly new and it allows multiple, separately built and deployed applications to share modules in real-time, via the browser.

Source: https://module-federation.io/

With Module Federation, you can:

• Import components across independent builds

• Share React, Vue, or any dependency

• Version-control exposed modules

• Ship independently, yet consume each other

Module Federation is what makes micro frontends in a single cohesive layout truly feel like one app.

Now let’s see it in action!

Real-Life Example

Let’s assume that you have to build two self-contained apps:

• Main / Host app (shell) — loads components from others (let’s say it’s in React)

• Remote app (product-app) — exposes components written also in React to others

Module Federation allows you to export these components without publishing them to NPM or wrapping them as a Web Component. Instead, the host app will load the component directly at runtime from the compiled JavaScript bundle.

Here’s how the project structure could look:

Product App:

product-app/                ← Remote Micro Frontend
├── public/
│   └── index.html          ← Mount point for optional local test render
├── src/
│   ├── ProductTile.jsx     ← Component to expose
│   └── index.js            ← Optional: local entry point
├── webpack.config.js       ← Exposes Product App
├── package.json
└── .babelrc / .gitignore / etc

Note, that webpack.config.js must be at the root level, same as package.json, so Webpack can locate it automatically.

Main / Host App (shell):

host-app/                     
├── public/
│   └── index.html        ← Mount point
├── src/
│   ├── App.jsx           ← Mounts ProductTile from remote
│   └── bootstrap.js      ← App entry point
├── webpack.config.js     ← Loads remotes via Module Federation
└── package.json

You can keep them both in a monorepo or host them in entirely different repos.

🔹 Step #0: Initiate projects (Host + Product Apps)

If you know how to do it, you can set up two separate React applications yourself for the Host App and one for the Remote (Product App), or initialize them in this way:

npm init
npm install react react-dom

🔹 Step #1: Install Webpack 5 + dependencies (Host + Product Apps)

Before you do anything federation-related, both the host and remote apps must be set up with Webpack 5 and its plugins. Go ahead and run this in both projects:

npm install webpack webpack-cli webpack-dev-server html-webpack-plugin --save-dev

A few notes about these packages:

• webpack + webpack-cli — Core bundler and CLI

• webpack-dev-server — Local server for hot reload + module exposure

• html-webpack-plugin — Automatically injects your bundles into HTML

• Optional but common: You can add Babel, React preset, loaders, and so on, for JSX/TSX support later.

This setup gives you a foundation. From here, you can add module federation to connect apps together.

🔹 Step #2: Create the Remote App (Product App)

Let’s start with the remote app, the one exposing a React component to be consumed by others.

Here’s a simple ProductTile React component (of course, you can implement yours):

// product-app/src/ProductTile.jsx

import React from 'react';

export default function ProductTile({ title }) {
  return (
    <div style={{ border: '1px solid #aaa', padding: '1rem' }}>
      <h3>🛍 {title}</h3>
    </div>
  );
}

A ProductTile component supplies a prop – “title” – and renders it.

Now let’s expose this component to other apps, not just render it locally.

🔹 Step #3: Configure Webpack in the Remote App (Product App)

This will be done utilizing module federation, which you must enable in webpack.config.js file. Here’s how it can be done. At the very top of the file, you will need to import these packages:

// product-app/webpack.config.js

const HtmlWebpackPlugin = require('html-webpack-plugin');
const ModuleFederationPlugin = require('webpack').container.ModuleFederationPlugin;
const path = require('path');

• HtmlWebpackPlugin – Handles HTML generation and script injection.

• ModuleFederationPlugin – The core Webpack plugin that lets you expose and consume modules at runtime

Then, define the actual config in module.exports:

// product-app/webpack.config.js

const HtmlWebpackPlugin = require('html-webpack-plugin');
const ModuleFederationPlugin = require('webpack').container.ModuleFederationPlugin;
const path = require('path');

module.exports = {
  entry: './src/index.js',                         // Entry file to the product app
  mode: 'development',                             // Must be production if you go live
  devServer: {
    port: 3001                                     // Product app runs on this port
  },
  output: {
    publicPath: 'auto',                            // Required for dynamic federation
  },
  plugins: [
    new ModuleFederationPlugin({
      name: 'productApp',                         // Internal name of the remote app
      filename: 'remoteEntry.js',                 // Entry file others will load
      exposes: {
        './ProductTile': './src/ProductTile.jsx', // Expose this module
      },
      shared: {                                   // Shared packages if needed
        react: { singleton: true },
        'react-dom': { singleton: true },
      },
    }),
    new HtmlWebpackPlugin({
      template: './public/index.html',
    }),
  ],
};

Now it’s time to use the product app in the main/host app:

// host-app/src/App.jsx

import React, { Suspense } from 'react';

// Dynamically import ProductTile from the remote
const RemoteProductTile = React.lazy(() => import('productApp/ProductTile'));

export default function App() {
  return (
    <div style={{ padding: '2rem' }}>
      <h1>📦 Host App</h1>
      <Suspense fallback={<div>Loading product tile...</div>}>
        <RemoteProductTile title="Bluetooth Speaker" />
      </Suspense>
    </div>
  );
}

In React, you can use the React.lazy() function to dynamically import the federated module. It returns a promise that React renders as soon as it’s ready.

That’s it. There’s nothing related to the module federation in the bootstrap.js and index.html files, but regular setup, so you can put whatever you want there:

// host-app/src/bootstrap.js

import React from 'react';
import { createRoot } from 'react-dom/client';
import App from './App';

const root = createRoot(document.getElementById('root'));
root.render(<App />);

<!-- host-app/public/index.html -->

<!DOCTYPE html>
<html>
  <head>
    <title>Host App</title>
  </head>
  <body>
    <div id="root"></div>
  </body>
</html>

And lastly, you can launch the host app:

npx webpack serve

That’s it!

Here are a few advantages and limitations of Module Federation, along with popular use cases.

✅ Pros

• Runtime Integration – Import remote components after both apps are built

• Independent Deployment – Teams can ship apps on separate pipelines

• Code Sharing – Share common libraries (React, lodash) to reduce duplication

• No iframes or wrappers – Native component integration, not isolated like Web Components

• No import maps needed – Webpack handles all the resolution logic

• Works across frameworks – Can be used in React, Angular, Vue, even Web Components

❌ Cons

• Tied to Webpack – Federation is Webpack-specific (Vite/Rollup alternatives exist but are not native)

• Initial setup is complicated – Requires per-app Webpack configuration and shared dependency coordination

• Runtime failures are possible – If the remote is down, the host may break unless you handle fallbacks

• Version mismatch risks – Shared libs (like React) must be tightly versioned and aligned

• No automatic SSR – Requires custom hydration logic for federated components

👨🏻‍💻 Popular Use Cases

Use Module Federation when:

• You want to build a platform composed of independently deployed apps

• You need runtime module loading (not just widgets)

• You want to share design systems or UI libraries across apps

• Your team is federating complex app sections, not just components

• You want to avoid loading dependencies multiple times across apps

Other Tools & Ecosystem Additions

While iframes, Web Components, single-spa, and Module Federation are the major players in the micro-frontend arena, there’s a growing ecosystem of alternative tools and strategies. They don’t always serve as full micro-frontend methods, but still solve important pieces of the puzzle. Let’s walk through some of the less prominent, yet practical solutions that are worth your attention.

Import Maps + Native ES Modules

Import Maps allow you to define where modules are loaded from, directly in the browser. Combined with native ES module support, they enable zero-build micro frontend setups.

<script type="importmap">
{
  "imports": {
    "ui-library/": "https://cdn.example.com/ui/v1.2.3/",
    "square": "./modules/shapes/square.js"
  }
}
</script>

You might’ve noticed that it looks similar to what single-spa + SystemJS does.

Use it when:

• You want to dynamically load shared libraries (like design systems)

• You’re building federated apps without bundlers

• You’re targeting modern browsers only

Piral: Micro Frontends as Pluggable Portals

Piral is a specialized framework for building portal-based micro frontends. It provides a structured environment where micro apps (called pilets) can be plugged into a central shell (the Piral instance).

Source: https://piral.io/

This framework comes with built-in:

• Routing

• Layout orchestration

• Shared state

• Module loading

• Authentication hooks

Great for:

• Enterprise-scale portals

• Apps with lots of features teams

• Admin dashboards or CMS-heavy UIs

Luigi: Micro Frontends + SAP-style Shells

Luigi is a microfrontend framework built by SAP to enable consistent layout shells with side navigation, top bars, permissions, and more.

Source: https://luigi-project.io/

This framework comes with built-in:

• Config-driven app registration

• Automatic route activation

• Role-based access control (RBAC)

• Seamless iframe integration with a shell

Great for:

• Intranet tools

• Cloud admin panels

• Productized dashboards

Open Components

OpenComponents is a framework-agnostic way to build self-contained microservices with UI logic, registered to a central registry.

Source: https://github.com/opencomponents/oc

This framework comes with built-in:

• Server-rendered or client-rendered

• REST-like model for UI consumption

• Great CDN + registry story

Great for:

• Used when your company treats UI as deployable microservices, just like APIs.

Bit: Meet a composable architecture

Bit isn’t a micro frontend framework per se, but a component-driven development and distribution platform. It organizes source code into composable components, empowering to build reliable, scalable applications in the era of AI.

Source: https://bit.dev

Use it alongside Web Components or Module Federation to supercharge reuse. If you want to practice, they have an Official Guide on how to master Micro-Frontends with Module Federation.

It’s a great addition when:

• You want to publish reusable components across teams

• You need to manage versions, ownership, and discovery

• You’re aiming for component-first delivery, not app-first

Final Thoughts

Micro frontends offer immense power, but that power comes with architectural responsibility.

Each method we explored solves a different kind of problem:

• IFrames are secure, but come with complex communication and high isolation.

• Web Components are native, framework-agnostic, dependency-free, and perfect for reusable UI Kits

• single-spa shines when you need orchestration and multiple SPAs under one shell.

• Module Federation is the go-to for runtime code sharing and independent deployment.

• And tools like Import Maps, Piral, Luigi, and others fill in the gaps, each in their own way.

There’s no one-size-fits-all solution here, but with the right match for your team structure and product strategy, you can build apps that scale across teams, tech stacks, and time.

If you liked this guide, feel free to repost and share it with your friends, colleagues, and social network.

If you want to take your micro-frontend skills to a new level, especially around Web Components, I invite you to check out my best-selling Udemy course called “Web Components: The Ultimate Guide from Zero to Hero“.

And of course, if you have questions, feedback, or need help with your micro frontend setup, feel free to reach out to me on my social media such as LinkedIn / X / Telegram. I’m always happy to chat, connect, and help other devs build amazing things! 💚

Let’s build the IT future we could be proud of! 💪🏼 Thanks for reading — and happy decoupling! 🚀

From healthcare to finance, robust software systems power our daily operations, making good software design essential to avoid inefficiencies and bottlenecks. This involves not just writing code but also designing systems that are easy to scale, maintain, and debug, while allowing others to contribute effectively.

Inefficient or ineffective software design can lead to significant issues, like scope creep, miscommunication within teams, project delays, resource misallocation, and complex systems that are difficult to maintain or understand. Without a strong design, teams often accumulate technical debt, which hinders long-term progress and increases maintenance costs.

This article will introduce you to key software design elements that will help you and your team address these challenges and guide you in building efficient, scalable systems. By understanding and applying these elements correctly, you can set up a project for both short-term and long-term success.

Prerequisites

I’ll explain these concepts through examples, but a basic understanding of programming in any language is required for this article (knowledge of Python will be especially beneficial).

Scope

The article will introduce key software design elements and explain them using an example. While I won’t provide a full software design for the example problem, I will include enough details to effectively illustrate each design element.

Table of Contents

• Overview of Key Software Design Elements

• A Walkthrough of the Software Design Process

  • Problem Statement

  • Use Cases

  • Requirements

  • High Level System Architecture

  • Detailed Software Design and Component Breakdown

• Conclusion: The Value of Thoughtful Software Design

Overview of Key Software Design Elements

To fully understand the benefits of the software design process, you’ll need to understand some key elements and their scope.

Once you have a good grasp of these, the next step is to define them for the specific problem at hand. Accurately defining these elements reduces risks and simplifies the implementation phase.

Doing this groundwork before implementation helps prevent late discoveries, minimizes the need for rewriting, and makes sure that the design can handle constraints and corner cases.

Now let’s briefly go over the key elements of the software design process:

1. Creating a problem statement: This step involves creating a clear and concise description of the problem that needs to be solved, along with its scope. The scope is essential because it focuses on the exact problem to be addressed and includes assumptions that must be considered during design.

2. Identifying use cases: This step outlines all possible user interactions with the software to achieve the desired outcome. It is a critical input to the architecture, as it helps create a design that addresses both general and edge-case use cases.

3. Stating requirements: This step defines the expectations of the software, such as its limitations, behaviors, and capabilities for different use cases.

4. Designing the architecture: This step provides a high-level structure of the software design, focusing on how to meet the requirements. The architecture typically includes components, how they interact, and how data flows through the system.

5. Drafting a detailed design: This step refines the high-level architecture into detailed, component-specific designs, ready for implementation.

In addition to these core elements, there are two important factors you need to consider throughout the design phase.

First, you’ll need to identify and state any assumptions you have. Assumptions can be present at any stage in the design process. Making correct assumptions increases the likelihood of success, improves focus, and reduces complexity in the design.

Second, you’ll need to create good documentation. Documentation is one of the most important elements in the software design process. It’s essential to document each stage as you go along. Documentation serves as the only formal record of the software design and is invaluable for presentations to management, for onboarding new team members, and for anyone returning to the project after a break. It saves valuable time and ensures continuity, as we often overestimate our own memory.

The figure below provides a visual summary of the key software design elements discussed in this section.

Next, we’ll apply these key software design elements to a practical example, demonstrating how each element contributes to building a robust and scalable system.

A Walkthrough of the Software Design Process

In any well-structured software project, clearly defining the problem is the first crucial step before diving into design and implementation. A well-defined problem ensures that the software meets user needs, remains maintainable, and scales effectively over time.

For this walkthrough, we will focus on designing a financial expense categorization system that processes and analyzes transaction data. This system is a part of a larger financial management solution and needs to be easy to debug, maintain, and scale.

Problem Statement

The problem statement provides a high-level goal for the software that we’ll design.

For this example, here’s our statement: Design a software solution that categorizes monthly expenses and generates a report from a list of transactions.

Define the scope

Defining the scope clarifies the smaller tasks that must be accomplished to meet the high-level goal. It outlines the focus of the software design and includes some assumptions.

Includes:

1. Implementing a parser to process a list of transactions provided as input.

2. Filtering transactions for a given month.

3. Analyzing, categorizing, and generating a report for each expense category.

Excludes:

Performance and memory optimization (excluded due to the limited scope of this article). While performance and memory optimizations are not the primary focus here, it’s important to keep future scalability in mind. Small design choices made now, such as selecting data structures, can help avoid significant refactoring later when the system grows.

Assumptions:

1. The list of transactions will be provided as a CSV file in the following format:
   Columns: "Date, Description, Amount, Type, Category Label".

2. Expense categories will be provided as input through a JSON file.

3. The software will run in a shell environment, and inputs will be taken as command-line arguments.

Now that the scope is clear, let’s examine how users will interact with the system through various use cases.

Use Cases

Use cases define how users will interact with the system to accomplish specific goals. Identifying accurate and valid use cases is critical to creating comprehensive requirements. Failing to capture enough use cases can lead to a design that is incomplete and lacks robustness. This may result in the need for redesigns, which increases time and resource consumption.

On the other hand, identifying too many use cases without considering their feasibility can lead to overly complex designs that are difficult to maintain and implement in the short term.

For our specific problem, the user will need to provide the following inputs while running the software in a shell:

1. A CSV file containing a list of transactions.

2. A month number.

3. A JSON file containing expense categories.

We need to consider all possible ways the user can interact with the script to achieve the desired outcome. For each of the three inputs, there are two possibilities: valid input or invalid input. This gives us 8 potential use cases (2 possibilities per input: valid and invalid). It's important to define what constitutes valid and invalid inputs for this problem:

• CSV File: Valid if it is in the format described in Assumption 1 (columns: "Date, Description, Amount, Type, Category Label").

• Month Number: Valid if the value is between 1 and 12.

• JSON File: Valid if it contains expense categories in the correct JSON format.

An input is invalid if it doesn't meet these definitions or if the input is absent.

It’s also crucial to consider the correlation between inputs when evaluating the feasibility of certain use cases, as they may interact with each other in unforeseen ways. Based on these use cases, we can now define the specific requirements that the system must meet.

Requirements

Now, let’s define the expected behaviors, limitations, and capabilities for each use case. Requirements serve as the foundation for architecture, specifications, and implementation. Based on our problem statement, the software will need to accomplish the following tasks:

1. The script shall take three inputs: a CSV file of transactions, a month number, and a JSON file of expense categories.

2. The script shall verify all inputs.

3. The script shall throw an error and exit if the CSV file cannot be opened or if it does not match the format in Assumption 1.

4. The script shall throw an error and exit if the JSON file cannot be opened.

5. The script shall throw an error if the month number is not between 1 and 12.

6. The script shall parse each transaction and load it into a data structure.

7. The script shall filter transactions by the specified month.

8. The script shall load the expense categories from the JSON file into a data structure.

9. The script shall categorize transactions based on the category label provided in the CSV file.

10. The script shall throw an exception if a category label in the CSV file is not present in the expense categories.

11. The script shall use a categorizing function to assign transactions to categories from the JSON file.

12. A class shall encapsulate categorized transactions, providing APIs to modify or access them.

13. The script shall support statistics calculation and report generation for categorized transactions.

With the requirements in place, we can now design a high-level architecture to meet those needs.

High Level System Architecture

In this stage, we will design the system at a high level, much like creating a master plan. Architecture involves organizing the software's functions into distinct components, illustrating how they interact, and mapping the flow of control and data through the system. While designing the architecture in this tutorial, we’ll incorporate good design principles.

For this example, the high-level requirements include:

1. Loading inputs and verifying them.

2. Applying time-based filtering.

3. Categorizing transactions based on category labels and descriptions.

4. Managing categorized transactions in a finance registry.

5. Generating reports from the categorized data.

One important component of software architecture is telemetry. Telemetry gathers data on the software's behavior, which is invaluable for debugging and performance assessment in real-world environments.

For smaller systems, simpler logging mechanisms may be sufficient to track basic errors and monitor performance. The decision to implement telemetry should depend on the complexity of the system and operational requirements.

Since telemetry provides such a helpful feedback loop for improving the design in future iterations, we’ll add it to the list of components here.

We’ll build our system architecture around a Test-Driven Development (TDD) approach. We’ll design each component with testing in mind to ensure it meets our requirements.

Just keep in mind that while TDD is a strong practice for ensuring code quality, it may not be the best fit for all projects. In scenarios where you need rapid prototyping or exploratory development, testing might be prioritized after initial iterations. Balancing between TDD and other methodologies depends on the project context and team preferences.

Our architecture will follow a modular structure, meaning the system will be divided into self-contained components. Each component will be responsible for specific functionality, making the system easier to test, maintain, and scale.

To achieve this, the architecture will emphasize loose coupling between components. Each component will interact with others through well-defined interfaces or APIs, ensuring minimal dependencies. We’ll abstract and encapsulate internal implementation details, exposing only the necessary information for interaction. Also, each component will handle its own errors and exceptions to ensure robustness and fault isolation.

But it is also important to consider a centralized error-handling strategy in some cases. Centralizing error handling can reduce redundancy, improve consistency, and make maintenance easier. The choice between local and centralized error handling should depend on the system's complexity and how components interact. This will contribute to the overall scalability and maintainability of the system.

Below is a summary of each component's functionality in this architecture:

• Load and verify input: This component will take the CSV file, JSON file, and month number as input, verify their validity, and load the data into structures.

• Time-based filter: This component will filter transactions based on the input month and store the filtered transactions in a data structure.

• Label-based categorization: This component will categorize transactions based on the category label in the CSV file.

• Description-based categorization: This component will categorize transactions using an algorithm based on the transaction description.

• Finance registry: This component will store all categorized transactions for further processing. It isolates the post-processing of categorized transactions from the categorization process and provides methods for updating or retrieving datasets.

• Report generation: This component will generate expense reports from the categorized transaction data.

• Telemetry: This component will monitor the performance of other components. It will track the flow of transactions, ensuring that all transactions are categorized either by label or description. Additional parameters can be added as needed to monitor specific functionalities.

The diagram below demonstrates the flow of data through these components:

Detailed Software Design and Component Breakdown

While we won't cover the full system design, this section will highlight key components and their specifications. For this example, I will assume the role of both the designer and implementer of the software.

Software design and specifications depend on several factors, including the designer's knowledge, skill set, available time, and resources. We’ll define some of the design details for the system, starting with the choice of the implementation language.

Choosing the right language is based on several important factors:

1. The language must meet the software requirements.

2. It should be stable, and have strong support from an active developer community.

3. Additional considerations include performance (speed and memory), scalability (ability to grow with future requirements), and platform support (ability to run on all major operating systems).

If you’re the one implementing this design, you’ll need to be familiar with and confident using that programming language. For this project, I chose Python because it meets all the project requirements, has a robust developer community for support, it’s stable, and I’m confident in using it to complete the implementation successfully.

Data Structures

Now, let’s look at the fundamental data structures that we’ll use in the design. We need to load the contents of the CSV file into a data structure for further analysis and processing. In Python, the Pandas DataFrame from the Pandas library is ideal for analyzing and processing tables, so we will use it to store the transactions.

For generating report, we will encapsulate categorized transactions along with relevant statistics, such as the total number of transactions, mean amount, and maximum amount, within a dedicated dataset class. This approach ensures a clear separation of concerns, where the dataset class manages data processing, while the reporting component focuses on presentation.

By structuring the system this way, we enhance reusability, maintainability, and scalability, making it easier to extend and modify in the future.

This dataset class will include:

• Member variables: category name, category description, a Pandas DataFrame for transactions, total number of transactions, mean amount, and max amount of transactions.

• Member functions: set/get DataFrame, save dataset to CSV (useful for debugging).

Here’s an example of a Dataset class in Python for structured data management and processing:

import pandas as pd  # Import Pandas for data handling

class Dataset:
    """
    A class representing a structured dataset with a name, predefined keys, 
    and a Pandas DataFrame.
    """

    def __init__(self, name, keys):
        """
        Initializes the Dataset object.

        Parameters:
        name (str): The name of the dataset.
        keys (list): A list of expected column names for the dataset.

        Attributes:
        self.name (str): Stores the dataset name as a string.
        self.keys (list): Stores the expected column names for data organization.
        self.mean_amt (float): Tracks the mean (average) transaction amount.
        self.max_amt (float): Tracks the maximum transaction amount.
        self.count (int): Stores the total number of transactions in the dataset.
        self.dataframe (pd.DataFrame): A Pandas DataFrame initialized with the specified column names.
        """
        self.name = str(name)  # Convert and store dataset name as a string
        self.keys = keys  # Store expected column names for consistency
        self.mean_amt = 0  # Initialize mean transaction amount to zero
        self.max_amt = 0  # Initialize max transaction amount to zero
        self.count = 0  # Initialize transaction count to zero
        self.dataframe = pd.DataFrame(columns=keys)  # Initialize empty DataFrame with predefined columns

    def getName(self):
        """
        Returns the name of the dataset.

        Returns:
        str: The name of the dataset.
        """
        return self.name  # Fixed: Removed incorrect parentheses

    def getValue(self, key):
        """
        Retrieves a specific column from the DataFrame.

        Parameters:
        key (str): The column name to retrieve.

        Returns:
        pandas.Series or None: The column data if the key exists, otherwise None.
        """
        if key in self.dataframe.columns:
            return self.dataframe[key]
        else:
            print(f"Warning: Key '{key}' not found in DataFrame.")
            return None  # Prevents KeyError

    def getKeys(self):
        """
        Returns the list of expected keys (column names) of the dataset.

        Returns:
        list: The keys defining the dataset.
        """
        return self.keys

    def setDataFrame(self, dataframe):
        """
        Sets the dataset's DataFrame while ensuring it contains only expected keys.

        Parameters:
        dataframe (pandas.DataFrame): The DataFrame to assign to the dataset.
        """
        if not isinstance(dataframe, pd.DataFrame):
            raise TypeError("Provided data is not a valid pandas DataFrame.")

        # Ensure only the expected columns are included
        self.dataframe = dataframe[self.keys].copy() if set(self.keys).issubset(dataframe.columns) else dataframe.copy()

    def getDataFrame(self):
        """
        Returns the DataFrame associated with the dataset.

        Returns:
        pandas.DataFrame: The dataset's DataFrame.
        """
        return self.dataframe

    def save_to_csv(self, file_name):
        """
        Saves the dataset's DataFrame to a CSV file.

        Parameters:
        file_name (str): The name of the CSV file to save.
        """
        self.dataframe.to_csv(file_name, mode='w', index=False)  # Save the DataFrame to CSV

In the previous section, we outlined the high-level system architecture, detailing the core components and their interactions. Now, let’s dive into the detailed design of some of the individual components, specifying how we’ll implement each one and how it’ll function within the system. We’ll also break down the components to explain how they work together to process the input and generate the report.

Below, you can see the flow diagram for the software, illustrating the interaction between the core components and the flow of data through the system.

Category Label-Based Filtering Component

The Category Label-Based Filtering Component classifies transactions by matching their "Category Label" with predefined expense categories from a JSON file. Transactions with valid category labels are stored in the finance registry, while unmatched ones remain for further processing.

• Input: DataFrame of time-filtered transactions, expense categories from JSON.

• Libraries used: Pandas DataFrame.

• Software design: Filters transactions based on the "Category Label" column and assigns them to corresponding categories. Transactions that cannot be categorized remain for further processing.

• Output: DataFrame of remaining transactions with empty values in the "Category Label" field.

• Component tests: Validate handling of valid, invalid, and missing category labels.

Finance Registry Component

The Finance Registry Component manages categorized transactions by storing them as datasets for each expense category. It maintains a structured collection of DataFrames, each containing transactions and summary statistics such as total count, max amount, and mean amount.

• Input: Expense categories from JSON.

• Libraries used: Pandas DataFrame.

• Software design: Implements a class that organizes datasets for all expense categories, providing methods to set and retrieve DataFrames.

• Component tests: Validate dataset creation, ensuring correct storage and retrieval of categorized transactions.

Here’s a simple and efficient Finance Registry implementation in Python for managing categorized financial datasets:

from Dataset import Dataset
import pandas as pd  # Ensure Pandas is imported if used elsewhere

# Define column structure for datasets
KEYS = ("Date", "Description", "Amount", "Transaction Type", "Category", "Account Name", "Labels", "Notes")

# Define dataset names for different financial categories
EXAMPLE_DATASET_NAMES = ("Investment", "Expense", "Savings")

class FinanceRegistry:
    """
    A class to manage categorized financial datasets, including investment, expense, and savings datasets.
    This registry allows structured access to transaction data and maintains aggregated financial metrics.
    """

    def __init__(self):
        """
        Initializes the FinanceRegistry object.

        Attributes:
        self.example_dataset (dict): A dictionary storing Dataset objects for financial datasets.
        """
        self.example_dataset = {name: Dataset(name, KEYS) for name in EXAMPLE_DATASET_NAMES}  # Create datasets for categories

    def setExampleDatasetToRegistry(self, name, dataframe):
        """
        Merges a new dataframe into the existing dataset for a given financial category.

        Parameters:
        name (str): The category name (e.g., "Investment", "Expense", or "Savings").
        dataframe (pd.DataFrame): The new data to be added.

        If the dataset already contains data, it concatenates the new dataframe to the existing one.

        Raises:
        ValueError: If the provided name is not a valid dataset category.
        """
        if name not in self.example_dataset:
            raise ValueError(f"Invalid dataset name: '{name}'. Expected one of {EXAMPLE_DATASET_NAMES}")

        df = self.example_dataset[name].getDataFrame()  # Get existing dataset

        if not dataframe.empty:  # Ensure the new dataframe is not empty
            dataframe = pd.concat([df, dataframe], axis=0, ignore_index=True)  # Append new data

        self.example_dataset[name].setDataFrame(dataframe)  # Update dataset in registry

    def getExampleDatasetFromRegistry(self, name):
        """
        Retrieves the dataset for a given financial category.

        Parameters:
        name (str): The category name (e.g., "Investment", "Expense", or "Savings").

        Returns:
        Dataset: The dataset corresponding to the given name.

        Raises:
        ValueError: If the provided name is not a valid dataset category.
        """
        if name not in self.example_dataset:
            raise ValueError(f"Invalid dataset name: '{name}'. Expected one of {EXAMPLE_DATASET_NAMES}")

        return self.example_dataset[name]

The diagram below illustrates how the Finance Registry organizes these datasets for further processing in the Report Generation component.

Report Generation Component

The Report Generation Component processes categorized transaction datasets from the finance registry and generates summary statistics. It calculates key financial metrics such as maximum amount, mean amount, and total transaction count. It also provides functionality to display categorized transactions in a structured format within the shell.

• Input: Datasets of categorized transactions from the finance registry.

• Libraries used: Numpy for calculations, Tabulate for formatted shell output (if needed).

• Software design: Implements a class with methods to compute financial statistics and display transaction summaries per expense category.

• Component tests: Validate correct calculation of mean, max, and total transactions, and ensure accurate display of categorized datasets in the shell.

Here’s a function to compute transaction statistics, including mean, max, and count, from a dataset in the report generation component:

from Dataset import Dataset
import numpy as np

def calculateStats(dataset):
    """
    Computes statistical metrics for a given dataset.

    Parameters:
    dataset: The dataset containing transaction data.

    Updates:
    - dataset.mean: Mean transaction amount.
    - dataset.max: Maximum transaction amount.
    - dataset.count: Number of transactions.
    """

    # Return early if the dataset has no transactions
    if dataset.dataframe.empty:
        return

    # Extract transaction amounts as a list
    tx_amount_list = dataset.dataframe['Amount'].astype(float).round(2).tolist()

    # Adjust transaction amounts based on "Transaction Type"
    for i, tx_type in enumerate(dataset.dataframe['Transaction Type']):
        if tx_type == 'debit':
            tx_amount_list[i] *= -1  # Convert debit transactions to negative values

    # Compute statistical metrics
    dataset.mean = round(np.mean(tx_amount_list), 2)
    dataset.max = max(tx_amount_list)
    dataset.count = len(tx_amount_list)

This concludes the design section, where we explored key software design elements with a practical example. The next step, implementation, is beyond the scope of this article. But it's crucial to recognize that new challenges often emerge during development, requiring updates to requirements, architecture, and specifications.

The purpose of this article is not to provide a full implementation, but to teach you some basic software design principles through an example. The focus is on understanding how to structure software, define clear requirements, and create scalable architectures, all before writing code.

By following a structured design process, you can shift complex problem-solving from implementation to the architecture phase, where you can explore solutions more effectively using flowcharts, block diagrams, and documentation. This makes the development process more organized, efficient, and maintainable, a crucial skill for real-world software engineering.

If you're learning to code, remember that good design is just as important as writing code itself!

Conclusion: The Value of Thoughtful Software Design

With well-defined problem statements, scope, requirements, specifications, and design, even complex problems can be solved and maintained in a sustainable way.

The steps we went through in this article can help you break down any problem, regardless of its complexity, into smaller, actionable tasks that you and your team can efficiently tackle.

Without proper planning, projects are often plagued by scope creep, wasted time and resources, miscommunication between teams, overly complicated designs, technical debt, and frequent redesigns.
Good design is often simple design, but achieving simplicity is difficult without thorough planning.

Approaching each problem with the mindset of defining a Problem Statement, Scope, Use Cases, Requirements, Architecture, and Specifications helps cultivate a strong software design mindset. This mindset is crucial for developing software that is scalable, maintainable, and high quality.

Control theory is the study of how to make systems behave in a desired way using inputs.

Planes, cars, trains, circuits, rockets and many more systems need to have a brain or an architecture inside them.

Control theory is the study of the control architectures of these complex systems.

In this article, we will explore how to apply control theory to create a rocket control system using Python.

This is a simple guide to how the architecture of complex systems is created. In this case, it's based on a rocket.

In this article, you will learn about:

• Rocket Systems and Cake Baking: A Fun Comparison
• Rocket Control Made Simple: Understanding PID Controllers
• Code example: Designing a simple PID controller
• Conclusion: Non-linear control systems

Note: We'll assume the rocket is time-invariant, meaning its behavior doesn't change over time. Addressing time-varying dynamics would complicate this tutorial more than I'd like.

Rocket Systems and Cake Baking: A Fun Comparison

Photo by Brent Keane on Pexels

What is a Rocket Control System?

Imagine you are backing a cake. Your recipe provides the steps and ingredients needed to bake the cake.

In this analogy:

• The cake is the rocket
• The recipe is the rocket flight plan
• The baker's actions are the rocket control system

Just as you change the oven temperature or mixing time to get the best cake, a control system changes rocket's parameters to ensure it stays on its course and remains stable.

Why are control systems important in programming?

By understanding control systems, you'll become better at algorithmic design and systems thinking.

It also enables you to figure out how to adjust processes in feedback loops. This is very important in many areas of programming.

You'll mainly use control theory and control systems when creating software for:

• Robotics and Automation: Control systems enable precise movement and adaptability in robots using feedback loops based on sensor input.
• Signal Processing and Communication: They optimize data transmission, error correction, and filtering for reliable communication.
• Embedded Systems and IoT: Control systems manage device interactions with environments, processing sensor inputs and adjusting outputs efficiently.

How to Create a Rocket Control System

In terms of our cake baking analogy:

1. Choose the Cake and Recipe: Select a simple control strategy, like choosing a basic cake recipe. A common choice is a PID controller because it's simple and effective.
2. Understanding the Ingredients: Derive a mathematical model of the characteristics and trajectory of the rocket. Like studying the recipe and ingredients. This way, we get a clear understanding of the system.
3. Gathering Initial Ingredients: Set initial parameters, similar to gathering your basic ingredients.
4. Mixing and Baking: Adjust and test the system, much like mixing ingredients and baking. This involves using various graphs to check stability and performance.
5. Adding Final Touches: Fine-tune the parameters, just like adding final decorations to your cake, to optimize the control system for efficiency.
6. Following the Recipe: Convert your design into a practical form, like carefully following a cake recipe.

Rocket Control Made Simple: Understanding PID Controllers

A simple control system: The PID controller

Example of control system diagram (source)

Every control system has a controller that runs it. One of the most used controllers is the PID controller.

In the code example here, we will use the PID controller. This is because it's simple and effective for simple control systems.

In a rocket control system, the rocket's PID controller constantly adjusts the rocket's path (processing block) by comparing its current position to where it should be (feedback block).

This way, the rocket stays on course and reaches its final destination.

The PID controller has three key parts that work in the processing and feedback part of the system: proportional gain (Kp), integral gain (Ki), and derivative gain (Kd).

• The proportional gain (Kp): Reacts immediately to any error, making the system respond quickly but sometimes causing it to overshoot the target.
• The integral gain (Ki): Fixes past errors by adding them up over time, getting rid of any leftover errors, but it can make the system unstable if set too high.
• The derivative gain (Kd): Predicts future errors to help prevent overshooting and smooth out the system's response.

This is why it's called a PID (Proportional-Integral-Derivative) controller.

These three parts work together to create a control signal that changes the rocket's setting. This ensures that it's stable, accurate and effective.

With the PID controller, we can control how the inputs like thrust and altitude change the position and speed to ensure the rocket is stable and on its intended path.

Analyzing Stability

Photo by Shiva Smyth on Pexels

To design a PID controller means to design a stable control system.

The process of designing a stable control system is called stability analysis.

There are many methods, but in the code example we will use:

• Root locus: Shows system stability and response
• Bode plot: Displays system gain and phase margins
• Nyquist plot: Illustrates stability and potential oscillations

In this case, the gain and phase margins simply mean that the control system can tolerate changes.

The gain margin tells us how much the system gain can increase without losing stability. Gain means how much to amplify the input signal to make the output signal.

The phase margin tells us how much delay is tolerable without losing stability. Delay in control theory means how much time it takes for the output to respond to the input.

This tells us how to change the Kp, Ki, and Kd so that the PID controller can control the rocket in an effective manner.

The Need for Transfer Functions: Controlling the Rocket and Determining Component Values

To implement any control system, we need two transfer functions: one theoretical and one physical.

Transfer functions tell us how inputs convert to outputs in a mathematical way.

The theoretical function is, in this case, the PID controller.

The physical system transfer function represents real-world dynamics and behavior of the physical components in the system.

By combining both, we can understand the behavior of materials and component values such as:

• Capacitor values for energy storage
• Sensor calibration values for accurate data measurement and feedback
• Spring constants for shock absorption systems
• Pressure ratings for fuel and oxidizer tanks

This way, the PID controller is not only the brain of the rocket but also can tell us the values of the components needed so that the rocket can fly its path.

How do engineers find the physical transfer function equation?

First, we need to understand what the rocket is for.

Will it send a LEO (Low Earth Orbit) or MEO (medium Earth orbit) satellite to space or a rocket to the moon?

After knowing its use case, we can, with math and physics, find the physical equation of the transfer function.

There is actually an entire field of engineering called system identification dedicated to this.

Now let's see how to find, for any control system, its physical transfer function.

Code example: Designing a simple PID controller

Photo by Pixabay

Now with this code example, we will create a simple control system for a rocket.

Before we dive into the code, let's talk about decibels.

Decibels use a logarithmic scale to measure sound. In control theory, they measure gain in a way that's easier to visualize on graphs.

This way, we can see many more large and small values in a manageable range.

In other words, by seeing the gain in a logarithmic scale, we are seeing how much the input is amplified to be the output in a manageable range of values.

I'll also explain how root locus, Bode plot, and Nyquist plots assist engineers in stability analysis.

Let's see the code – and then we'll analyze it block by block:

# Step 1: Import libraries
import matplotlib.pyplot as plt
import control as ctrl

# Step 2: Define a new rocket transfer function with poles closer to the imaginary axis
num = [10] 
den = [2, 2, 1] 
G = ctrl.TransferFunction(num, den)

# Step 3: Design a PID controller with new parameters
Kp = 5
Ki = 2
Kd = 1
C = ctrl.TransferFunction([Kd, Kp, Ki], [1, 0])

# Step 4: Applying the PID controller to the rocket transfer function
CL = ctrl.feedback(C * G, 1)

# Step 5: Plot Root Locus for Closed-Loop System
plt.figure(figsize=(10, 6))
ctrl.root_locus(C * G, grid=True)
plt.title("Root Locus Plot (Closed-Loop)")

# Step 6: Plot Bode Plot for Closed-Loop System
plt.figure(figsize=(10, 6))
ctrl.bode_plot(CL, dB=True, Hz=False, deg=True)
plt.suptitle("Bode Plot (Closed-Loop)", fontsize=16)

# Step 7: Plot Nyquist Plot for Closed-Loop System
plt.figure(figsize=(10, 6))
ctrl.nyquist_plot(CL)
plt.title("Nyquist Plot (Closed-Loop)")

plt.show()

Full Code

Step 1: Import libraries

import matplotlib.pyplot as plt
import control as ctrl

Importing libraries

Here we import two libraries:

• matplotlib: A plotting library for creating various types of visualizations
• Control: A library for analyzing and designing control systems

Step 2: Define the Transfer Function of the Rocket System

num = [10] 
den = [2, 2, 1] 
G = ctrl.TransferFunction(num, den)

Define the Transfer Function of the Rocket System

In this code we define the transfer function of the physical system

• num=[10]: Sets the system gain to 10.
• den=[2,2,1]: Defines the denominator.
• G = ctrl.transferFunction(num, cen): Constructs the transfer function.

This is the transfer function we are going to control with PID:

<!DOCTYPE html>

In this code example, the transfer function rocket equation is very simple. But in real life, rocket transfer functions are not time-invariant linear systems. Usually, they are very complex non-linear systems.

Step 3: Design a PID controller with new parameters

Kp = 5
Ki = 2
Kd = 1
C = ctrl.TransferFunction([Kd, Kp, Ki], [1, 0])

Design a PID controller with new parameters

This code sets up a PID controller with specific gains and creates a transfer function:

• Kp = 5: Sets the proportional gain to 5.
• Ki = 2: Sets the integral gain to 2.
• Kd = 1: Sets the derivative gain to 1.
• C = ctrl.TransferFunction([Kd, Kp, Ki], [1, 0]): Creates a transfer function of the PID controller

Step 4: Applying the PID controller to the rocket transfer function

CL = ctrl.feedback(C * G, 1)

Applying the PID controller to the rocket transfer function

• C * G: Multiplies the PID controller C with the system G (the rocket) to form the open-loop transfer function, which models the system's behavior without feedback and relies on predefined settings.
• ctrl.feedback(C * G, 1): Computes the closed-loop transfer function by applying feedback and representing the system's behavior with feedback. This allows it to adjust inputs and automatically correct errors.
• CL: Stores the resulting closed-loop system, integrating the controller with the rocket to maintain desired performance through feedback, and is used for further analysis or simulation.

Step 5: Root Locus for gain analysis

In this code:

plt.figure(figsize=(10, 6))
ctrl.root_locus(C * G, grid=True)
plt.title("Root Locus Plot (Closed-Loop)")

Create the Root Locus Graph

We generate this plot:

Simple Root Locus Graph

This is a root locus graph. It was invented to help engineers study the stability of control systems.

The crosses on the graph, called poles, are very important.

If they are on the left side of the graph, the system is stable. If they are on the right side, the system is unstable.

The more to the left they are, the quicker the system will return to normal after a disturbance, and thus, the more stable it will be.

But moving more to the left can cause too many oscillations, depending on their specific locations.

The key point is:

• By changing the Kp, Ki, and Kd, this moves the poles to be as far left as possible without causing oscillations.

However, the root locus graph is not enough to ensure stability. We need to use the Bode and Nyquist plots as well. Only with them can we get the best PID controller values for the rocket control system.

Step 6: Bode Plot for Stability Analysis

In this code:

plt.figure(figsize=(10, 6))
ctrl.bode_plot(CL, dB=True, Hz=False, deg=True)
plt.suptitle("Bode Plot (Closed-Loop)", fontsize=16)

Create the Bode Plot Graph

We generate this plot:

Simple Bode Plot

The Bode plot was invented to help engineers understand how a system responds to changes and how stable it will be under different conditions.

The Bode plot also shows the system's stability and safety margins.

Let's understand how it works:

Bode Plot in detail

The graph on top is called the Magnitude Plot and the one below it is called the Phase Plot.

The magnitude plot measures the gain of a system across different frequencies. Higher gain means quicker and stronger reactions, which is good for precise control.

The phase plot measures the phase shift introduced by the system across different frequencies. The phase shift is seen when the gain is 0.

In this case, we can see with the green line when the gain is zero and what phase shift is associated with that in the red line. It is approximately 63 degrees.

An ideal range is a phase shift of 30 to 60 degrees, which balances stability and response speed.

Over 60 degrees, the system is very stable, but might slow down the system response to changes.

So after analyzing the plot, we can conclude this PID controller is stable.

Step 7: Nyquist Plot for Stability Analysis

In this code:

plt.figure(figsize=(10, 6))
ctrl.nyquist_plot(CL)
plt.title("Nyquist Plot (Closed-Loop)")

Create the Nyquist Plot Graph

We generate this plot:

Nyquist Plot Graph

The Nyquist Plot is a tool to help engineers quickly check if a control system is stable or not.

It is very simple:

• If there is no circle around the red cross at point (-1 0), the system is stable.
• If there are circles around the red cross, namely clockwise circles, at point (-1 0), the system is unstable.

Since there aren't circles around the red cross, this control system is stable.

Last step after designing the rocket control system

After completing the design of this PID control system, we can use tools like Simulink to find the necessary values for many components.

In other words, after getting the best PID controller variables, it's time to find the physical component values of the rocket.

Some of these values are:

• Resistor values for controlling current flow
• Capacitor values for energy storage
• Inductor values for managing electromagnetic interference
• Sensor calibration values for accurate data measurement and feedback
• Strength and durability of materials for the rocket's body and fins
• Torque and speed requirements for servo motors
• Spring constants for shock absorption systems
• Pressure ratings for fuel and oxidizer tanks

Thanks to Simulink, we can get all these values needed to design a rocket according to its mission.

With a stable control system, based on a PID controller to control the physical transfer function of a rocket, we can get all the values needed for each component.

Conclusion: Non-linear control systems

Photo by Peter de Vink: https://www.pexels.com/photo/photo-of-full-moon-975012/

There are many methods available to us to optimize a Linear Time-Invariant (LTI) control system:

1. Root Locus Method: Adjust system poles to reduce oscillations.
2. Bode Plot Analysis: Maintain phase margin and stability.
3. Nyquist Plot: Confirm overall system stability.

With these tools, it's possible to create a control system.

However, in this process, it is good practice to use methods like the Ziegler-Nichols method to more quickly find the best PID controller variables.

In our exploration, we worked with a very simple rocket system.

In real life, only non-linear tools are used because all rocket systems are non-linear systems.

One example is adaptive control, where the control system adjusts itself in real-time to handle changing conditions

Another one is Lyapunov's method. In this case, it is used for stability analysis instead of these three plots.

Still, the process of making these control systems is always the same. This article explained how this process works and how it is applied in a time-invariant system.

We just published a course on the freeCodeCamp.org YouTube channel that will teach you all about system design. Created by Hayk Simonyan, this comprehensive tutorial teaches the core aspects of system design, providing clear explanations, real-world examples, and practical strategies. This course covers essential topics like scalability, reliability, data handling, and high-level architecture, making it an invaluable resource for mastering system design.

Course Breakdown

Introduction

The course kicks off with an introduction to system design, setting the stage for the detailed topics that follow. This section provides an overview of why system design is important and what you can expect to learn.

Computer Architecture

In this section, you will explore the fundamentals of computer architecture, including disk storage, RAM, cache, and CPU. Understanding these components is essential for designing systems that are both efficient and scalable.

Production App Architecture

Here, the course teaches the architecture of production applications, covering Continuous Integration/Continuous Deployment (CI/CD), load balancers, and logging & monitoring. These concepts are important for maintaining and scaling applications in a real-world environment.

Design Requirements

This section focuses on the critical design requirements of modern systems. Topics include the CAP Theorem, throughput, latency, and Service Level Objectives (SLOs) and Service Level Agreements (SLAs). These principles help ensure that systems meet their performance and reliability goals.

Networking

A deep dive into networking covers TCP, UDP, DNS, IP addresses, and IP headers. Networking is the backbone of any distributed system, and understanding these protocols is important for designing robust architectures.

Application Layer Protocols

The course also covers various application layer protocols such as HTTP, WebSockets, WebRTC, and MQTT. These protocols are important for building interactive and real-time applications.

API Design

Effective API design is crucial for creating scalable and maintainable systems. This section provides guidelines and best practices for designing APIs that are easy to use and efficient.

Caching and CDNs

Learn about caching mechanisms and Content Delivery Networks (CDNs) to optimize performance and reduce latency. These techniques are essential for handling high traffic loads and ensuring fast response times.

Proxy Servers

The course explains the roles of forward and reverse proxy servers in system design. Proxies can enhance security, load balancing, and caching, making them a vital part of modern architectures.

Load Balancers

Explore different types of load balancers and their importance in distributing traffic across multiple servers. Load balancers help maintain system reliability and availability.

Databases

Finally, the course covers database design, including sharding, replication, ACID properties, and vertical and horizontal scaling. These concepts are important for managing large datasets and ensuring data integrity and availability.

Hayk Simonyan's course is packed with detailed explanations and practical examples to help you master system design. Watch the full course on the freeCodeCamp.org YouTube channel (1-hour watch).

We just published a course on the freeCodeCamp.org YouTube channel about high-level system design. This course offers a unique hands-on approach to understanding high-level system design (HLD) concepts by building a fully functional YouTube-like platform. Keerti Purswani developed this courese.

What is High-Level System Design?

High-Level System Design involves creating a blueprint for complex systems, focusing on architecture, component interactions, and scalability. It addresses how different parts of a system communicate, manage data, and handle user requests efficiently.

Course Overview

In this course, you will start with a basic system flow and gradually incorporate three key services: upload, watch, and transcoder. Each service is important to building a scalable and robust video platform. Here’s a detailed look at what you will learn:

1. Upload Service: Learn how to handle video uploads effectively, including chunking and managing large file transfers.

2. Transcoder Service: Dive into transcoding with FFmpeg, a powerful tool for converting video formats and optimizing videos for different devices.

3. Watch Service: Implement Adaptive Bitrate Streaming using HLS (HTTP Live Streaming) to ensure smooth playback across various network conditions and devices.

Technologies Covered

This course leverages a range of modern technologies to build the YouTube clone:

• Front-end: JavaScript and React for creating dynamic user interfaces.

• Back-end: Node.js and Express for building scalable server-side applications.

• Database: Prisma as an ORM (Object-Relational Mapping) tool to interact with databases.

• Frameworks: Next.js for server-side rendering and improved performance.

• Other Tools: Docker for containerization and Redis for caching to enhance performance and scalability.

By the end of this course, you will have a deep understanding of high-level system design principles and practical experience in building a complex application. Watch the full course on the freeCodeCamp.org YouTube channel (2-hour watch).

We just published a course on the freeCodeCamp.org YouTube channel that will teach you how to design software systems. This course is designed to provide a comprehensive understanding of the various concepts and techniques involved in designing and building software systems.

Gaurav Sen developed this course. He is an experienced software engineer and he also has a popular YouTube channel with almost half-a-million subscribers.

You will learn about basic engineering design patterns that are used to build large-scale distributed systems. In the second part of the course you will learn how to use the principles from the fist part to design and code a live streaming video app.

We will begin by discussing the basics of system design, including what it is and why it is important. We will then delve into specific design patterns that are commonly used in software development, such as live streaming system design, fault tolerance, extensibility, and testing.

The course uses video streaming service as an example for demonstrating system design principles. You will learn about different diagramming approaches, API design, database design, and network protocols. We will also discuss the importance of choosing the right datastore for your system and the process of uploading raw video footage.

We will also cover advanced topics such as Map Reduce for video transformation and the pros and cons of different streaming protocols such as WebRTC, MPEG DASH, and HLS. Additionally, we will discuss the role of Content Delivery Networks in system design and provide a high-level summary of the key concepts covered in the course.

In addition to the high-level concepts, we will also delve into the low-level design of a video player, including engineering requirements, use case UML diagrams, class UML diagrams, and sequence UML diagrams. We will also cover the process of coding the server and provide resources for further learning and development in the field of system design.

Overall, this course will provide a comprehensive understanding of the various concepts and techniques involved in designing and building software systems. Whether you are a beginner or an experienced software developer, this course will provide valuable insights and knowledge that you can apply to your own projects.

Watch the full course on the freeCodeCamp.org YouTube channel (1.5 hour watch).

System Design is an important topic to understand if you want to advance further in your career as a software engineer. Even if you are just beginning your coding journey, it's a good idea to get a head start on learning about system design.

Early in your career you will mostly just be tested on your coding ability. In higher level interviews, however, there will often be a greater focus on testing your ability and experience at designing applications.

The biggest struggle engineers have with system design interviews is that they are more open-ended and there isn't any single correct answer. This lack of structure can be intimidating, so my goal with this article is to give you a roadmap for navigating these types of interviews with confidence.

What this article will cover:

• What is a system design interview and why they are used
• The main stages of a system design interview
• Example interview problem – Design YouTube

Video Tutorial

You can also watch this tutorial on YouTube if you like:

And I've created a playlist of videos on specific topics related to system design and web architecture:

System Design Interview Overview

At first glance it seems silly to ask somebody to design a huge app like Twitter or YouTube in 45-60 minutes. These apps were designed over a period of years by hundreds of engineers working together, so it's clearly an impossible task to do in a short interview.

There are two main reasons why companies use these types of interviews. The first is, of course, to test your knowledge about the technologies being discussed. They want you to go deep enough to make sure you aren't just throwing buzzwords around without understanding how things actually work.

The second reason might be more important, though. The system design interview is a way to simulate a realistic scenario where you are working together with the interviewer to determine the best design decision.

Getting the perfect answer isn't necessarily the most important thing here – it's some of the other things you can show, like:

• How do you handle being challenged? Do you get defensive or take feedback with a positive attitude? Are you stubborn or narrow-minded?
• Do you show knowledge of the various tradeoffs certain design decisions involve? There's a big difference between blindly making a decision and not realizing the consequences, and knowing the pros/cons and accepting the tradeoffs.
• Are you able to effectively communicate and if necessary explain complex technical concepts in an easy to understand way?
• Are you candidate somebody the interviewer would want to work with long term? Even if somebody is a genius, if they are miserable to work with they might not be a good hire.

Stages of a System Design Interview

In this section you'll learn a general framework for structuring how to handle a problem during a system design interview.

Clarify the problem and establish design scope

The first thing you'll want to do after your interviewer gives you the problem is to take a few minutes to ask some clarifying questions and figure out what exactly they are looking for.

The worst thing you could do here is just start off in the completely wrong direction because you didn't take the time to ask a few questions. You have a limited amount of time during the interview, so you want to make sure you focus on what's important.

Here are some examples of questions you might ask:

What are the use cases / features of the app?

In this article we will be using YouTube as an example. There are hundreds of different features you could design like ad delivery, authentication, recommendation algorithms, comments, video upload, video processing, and many others.

During an interview you only have time to cover a few of those, so make sure to ask the interviewer questions to figure out what they want you to focus on designing.

How many users are expected / what is the likely traffic volume?

The complexity of the system will depend on the amount of traffic it needs to handle, so make sure to gather this information.

You don't want to over-engineer things if the traffic is relatively low and you also don't want to get stuck with an app that can't scale because you didn't design it properly.

Ask questions like how many users the app will have, the average amount of data per request, how long data needs to be stored, and how reliable and available does the system need to be?

This step is going to help you beyond just getting more information to work with. You're also showing the interviewer that you understand how to gather information about a vague problem.

Determine Rough Capacity Estimates

Using the information you gathered during the first step, you can begin to make some rough estimates and generalizations for things like storage and bandwidth requirements.

This process will involve some basic math like multiplying the number of users by the average request size and the amount of requests each user is expected to make daily.

Create a High Level Design

Here you want to create a rough architecture for the system. Draw out things like load balancers, web servers, app servers, task queues, database, caching, file storage, and so on. You should include all the core components you need to create the system.

Make sure to communicate with the interviewer during this stage and check to ensure that you aren't missing anything. While they probably won't tell you directly, they will give you a nudge in the right direction if you forgot about some crucial feature.

API Design

This part is almost cheating because you are using the structure of the interview to your advantage to confirm that you are on the right path.

The interviewer is never going to deliberately lead you down the wrong path, so once you've created your high level design you can start sketching out some rough API endpoints for each component.

For the YouTube example they might look something like this, depending on which features you are building:

• uploadVideo (userID, video, description, title)
• comment (userID, videoID, comment)
• viewVideo (videoID)
• videoSearch (query)

In some cases you might not need to drill down to this level. If the interview question is very high level like "design Youtube", you can probably skip this part. On the other hand if you get a more focused question like "design YouTube's comment system", it would make sense to go more in depth.

Create a Data Schema

At this point you should have a good idea of all the requirements and data needed for the application to work, so now you can plan out how your data is structured.

Depending on what you are building and the requirements, you'll need to weigh the costs and benefits of things like using a relational vs non-relational database. When modeling your data you'll also want to account for things like potential data partitioning and replication.

Take a Detailed Look at the Components

What happens during this section will mainly depend on the feedback of the interviewer. They will probably pick out a few specific components to focus on and ask why you made certain decisions.

The most important part here isn't necessarily being 100% right. Instead, it's to show that you didn't just blindly make decisions and understand exactly what tradeoffs you were making.

You should be able to propose alternate design decisions that could have been used and explain why you didn't use them.

How to Design YouTube

Now that you have a general idea of how a system design interview works and a framework for handling a system design problem, I'm going to show you how to put it all into practice using YouTube as an example.

Step 1 – Define Problem Scope and Requirements

This will be a high level problem where we implement a few of YouTube's major features without diving too in-depth on any of them. The features to focus on will be:

• Users can upload videos
• Users can view videos
• Users can comment on videos

Step 2 – Determine Capacity estimates

The two biggest capacity factors in an app handling large amounts of video like YouTube will be storing all that content and bandwidth requirements to deliver the content to users. In this section you'll learn how to make rough estimates for capacity requirements.

The main focus here is not on being highly accurate, but showing a logical thought process for calculating these numbers based on the information available to you.

In an interview you would be given the data, but in this case I'm using two key pieces of data that YouTube has made public:

• YouTube creators upload 500 hours of video every minute
• YouTube users watch 1 billion hours of video per day

You can use these numbers to calculate storage and bandwidth requirements with a few assumptions.

Bandwidth Calculation

Daily bandwidth calculation

To calculate an estimate for bandwidth, we start with the amount of video watched daily. The key assumption here is how much bandwidth is used per hour watched, as this would depend on the quality of video most users choose to watch.

The 3 Gigabyte estimate is based on a rough percentage of users watching in standard definition and others choosing HD or 4K, which consume much more bandwidth per hour watched.

The math here is fairly simple: multiply 1 billion hours by the average bandwidth of an hour of video, then divide that by 1000 to convert to terabytes, then divide by 1000 again to get to Petabytes. The final bandwidth estimate is 3,000 PB used daily.

Storage Calculation

Step by step calculations for storage

Based on a few assumptions we can calculate that YouTube will need to store around 2.16 Petabytes of new video every day. Here's how we get that number:

• Convert 500 hours to 30,000 minutes of video uploaded per minute
• Each minute of HD video is roughly 50 Megabytes due to having copies of each video in multiple formats. We multiply that by 30,000 minutes and then divide by 1000 to convert to Gigabytes.
• We then take the 1,500GB uploaded per minute and multiply by 60 then 24 to calculate the daily amount of video uploaded. We divide by 1000 again to convert Gigabytes to Terabytes
• Our final total is 2,160 Terabytes uploaded daily or 2.16 Petabytes

Step 3 – Database Design

For our database we will use a standard relational database like MySQL. The schema will look something like this:

This design is very simple but has the essentials that you'd need for a basic implementation. It would be a good idea to do some research into the differences between relational and non-relational databases so you understand what kind of situations each excel at and when to use them.

For certain apps with different requirements a NoSQL database might make sense. Often large systems will have many different services that use different types of databases depending on their needs.

Step 4 – High Level Design

That's a pretty complex diagram, so let me break down what's happening:

• Client – This could be a user on a mobile app or their computer trying to upload a video, make a comment, or watch a video
• CDN – A content distribution network is used to reduce latency and improve reliability when it comes to delivering static content like videos or images. A CDN works by storing content in data centers all around the world so that the content is closer to users. This results in reduced latency because requests travel a shorter distance. There's also an added benefit of content being stored in multiple locations so even if one location can't serve traffic for some reason, another location can.
• Load Balancers – A load balancer accepts requests and routes them to servers depending on a number of factors. At YouTube's scale, a single server can't handle all the traffic and you want replication to prevent a single point of failure. The load balancer can check the status of servers and verify they can handle traffic or choose another server that can handle the request.
• Services – You can think of this as the app layer of the system. Instead of using a single monolith to handle traffic, we'll use several microservices to handle specific tasks. The second box for each of these services in the diagram represents multiple servers running for each of them to increase reliability. If one replica of the service goes down, there's always another to step in and handle traffic.
• Data Stores – When using microservices it is generally best practice for each microservice to own its own data. If one service needs data from another they can access it through an API.
• Video Upload Process – Handling the video uploads will involve multiple steps, as trying to handle it synchronously with the app server would be fragile and reduce performance. I'll cover this more in depth in the next section

I don't want to go too in-depth on these individual components because I could write entire articles on any of them if I wanted to explain them fully.

If you are interested in a more detailed explanation you can check out the system design playlist I linked to above which has videos covering most of these concepts.

Step 5 – Go Over Specific Components and Details

At this stage you have a working design. Now let's look at some of the specific components in detail.

Video Upload

Video content is the lifeblood of YouTube, and it doesn't exist without it. This means that making it quick and easy for users to upload videos is probably the most important feature.

Imagine uploading a multi-gigabyte video to YouTube and then seeing the upload fail after 30 minutes when it's 95% done. To prevent this you'll want to support the ability for resuming uploads if the client's connection is lost temporarily. The uploaded video can then be stored with a distributed file system like HDFS.

Once the upload is complete there's still a lot more to do before the video is ready for users to access. The video needs to be encoded into multiple different quality formats, you need to generate thumbnails, and push copies of the video to the global CDN.

Again, at any stage one of these processes could fail. To prevent this you'll have a task queue to manage this process and retry the processing attempt if it fails at any stage.

Database Scaling

The database is often the bottleneck of an application. You will probably be tested on whether you understand some of the fundamental concepts around database scaling. This could include caching to handle read requests, sharding, and replication.

Conclusion

Hopefully this article has given you a better understanding of what to expect during a system design interview.

This article mainly focused on the structure of the interview itself rather than the concepts you need to understand to answer the questions given during the interview.

Two great resources for beginning to learn about that are:

A great article posted here on Free Code Camp News: https://www.freecodecamp.org/news/systems-design-for-interviews/

The system design primer repo on GitHub: https://github.com/donnemartin/system-design-primer

Both cover just about every major concept you need to know for your system design interview and should put you in a great position for success.