Desktop and server processors hide memory latency behind multi-level caches. Many embedded processors, especially ARM Cortex-M and Cortex-R based chips, take a different approach. They give you direct control over multiple memory regions, each with very different performance characteristics.

This handbook covers what ITCM, DTCM, and DDR memory are, how they differ, how to place code and data in the right region, and how to profile and monitor firmware memory usage over time.

Table of Contents

• Prerequisites

• Why Embedded Memory Architecture Matters

• What is ITCM (Instruction Tightly-Coupled Memory)?

• What is DTCM (Data Tightly-Coupled Memory)?

• What is DDR (Double Data Rate) Memory?

• How They Compare: A Side-by-Side Overview

• How to Decide Where to Place Code and Data

• How the Linker Script Controls Memory Placement

• Common Mistakes to Avoid

• Performance Comparison With Real Numbers

• How TCM Affects Power Consumption

• How to Profile Memory Usage

• Summary

Prerequisites

To get the most from this guide, you should have a basic understanding of C programming, including pointers, structs, and the difference between static and local variables.

Some familiarity with embedded development concepts like compiling, linking, and flashing firmware to a target board will also help.

Finally, a general sense of how a CPU fetches and executes instructions will make the performance discussions easier to follow.

You don't need to be an expert in any of these. The article explains each concept as it comes up.

Why Embedded Memory Architecture Matters

A modern embedded processor might be clocked at 400 MHz or higher. It can execute an instruction every few nanoseconds.

But when it needs to fetch that instruction from memory, or read a variable, the memory might not keep up. The processor ends up stalling, waiting for the memory subsystem to deliver the data it asked for. Those stall cycles add up fast.

On a desktop computer, hardware caches (L1, L2, L3) sit between the CPU and main memory, automatically keeping recently-used data nearby. The cache hardware decides what to keep and what to evict, and it does this transparently. The programmer rarely needs to think about it, and performance is generally good enough without manual intervention.

On many embedded processors, the situation is different. Instead of hardware caches, you get three distinct memory regions, each attached to the CPU in a different way.

The table above shows the three memory types you'll encounter on a typical ARM Cortex-M or Cortex-R-based embedded system. ITCM and DTCM are fast but small. DDR is slow but large.

The "deterministic" label on TCM means that the access time is always the same, every single time, regardless of what accessed that memory before or what else is happening on the chip. The "variable" label on DDR means the access time can change depending on the internal state of the DDR chip and its controller.

You, the developer, control which region each piece of your firmware lives in. The compiler and linker don't make these decisions automatically. You specify them through section attributes in your source code and placement rules in your linker script. Getting this right is often the difference between firmware that meets its real-time deadlines and firmware that misses them.

What is ITCM (Instruction Tightly-Coupled Memory)?

ITCM stands for Instruction Tightly-Coupled Memory.

The "Instruction" part means this memory is used for storing executable machine code, the compiled instructions your CPU fetches and runs.

The "Tightly-Coupled" part means the memory is physically located on the same silicon die as the CPU core, connected through a dedicated bus with no arbitration or contention. There's no shared bus to compete with. There's no cache hierarchy to traverse. The CPU asks for an instruction, and ITCM delivers it directly, through a private path that nothing else on the chip can interfere with.

The CPU can fetch an instruction from ITCM in a single clock cycle, every time. This access time is both fast and deterministic. It doesn't vary based on access patterns, recent history, or what else is happening on the bus.

This determinism is just as important as the raw speed, because it makes worst-case execution time analysis possible. In safety-critical systems, you need to be able to prove that a function will always complete within a certain number of cycles. ITCM makes that proof much simpler.

Why Single-Cycle Fetch Matters

Every line of C code compiles down to one or more machine instructions. Each of those instructions must be fetched from memory before the CPU can decode and execute it. This fetch step happens for every single instruction, so even small per-instruction delays compound rapidly in loops and frequently-called functions.

Consider a loop that runs 1,000,000 iterations, where each iteration involves 10 instruction fetches. That's 10 million fetches total.

ITCM:  10,000,000 fetches x 1 cycle  = 10,000,000 cycles
DDR:   10,000,000 fetches x 8 cycles = 80,000,000 cycles

Difference: 70,000,000 cycles
At 400 MHz: 70,000,000 / 400,000,000 = 0.175 seconds = 175 ms

This calculation compares the total cycle count when the same loop runs from ITCM versus DDR. With ITCM, each fetch takes 1 cycle, so 10 million fetches cost 10 million cycles.

With DDR, each fetch takes 8 cycles (a conservative average), so the same 10 million fetches cost 80 million cycles. The difference is 70 million cycles, which at 400 MHz translates to 175 milliseconds.

In a real-time system running a control loop at 1 kHz (one iteration every 1 ms), 175 ms of extra latency spread across your processing isn't a minor inconvenience. It can cause the system to miss deadlines, drop sensor readings, or produce incorrect outputs. In motor control applications, a missed deadline can mean physical damage to the hardware. In audio processing, it means audible glitches. The cost of slow instruction fetch isn't abstract.

What Should Go in ITCM?

Because ITCM is small (typically 512 KB to 2 MB), you can't fit your entire firmware in it. You need to be selective about what earns a spot.

Interrupt Service Routines (ISRs) are the highest-priority candidates. ISRs run in response to hardware events like a timer tick, an ADC conversion completing, or a communication peripheral receiving data. They need to execute and return as quickly as possible.

A slow ISR delays all lower-priority interrupts and can cause missed events. If your ISR fetches its instructions from DDR, each fetch takes multiple cycles, and the total ISR execution time increases by a factor that could push it past its deadline.

Placing ISRs in ITCM ensures they run at maximum speed with completely predictable timing.

Real-time processing functions are the next priority. These include signal processing routines, motor control loops, audio processing pipelines, and any function that runs at a fixed rate and must complete within a strict time budget.

If your audio codec callback needs to process a buffer of samples every 5 ms, every instruction fetch cycle counts. Placing these functions in ITCM gives you the maximum amount of CPU time for actual computation rather than waiting on memory.

Inner loops of your main processing pipeline also benefit significantly from ITCM placement. If your firmware spends 80% of its time in a handful of functions, those functions should be in ITCM. Profiling tools and the linker map file (covered later in this article) can help you identify which functions are the hottest.

Functions that require deterministic timing belong in ITCM even if they aren't the fastest path. ITCM access time doesn't vary, which makes timing analysis predictable. This matters for safety-critical systems (automotive, medical, aerospace) where you need to prove worst-case execution times to a certification authority.

How to Place a Function in ITCM

You use a GCC section attribute to tell the compiler that a function belongs in a specific memory section. Then, in your linker script, you map that section to the ITCM memory region.

__attribute__((section(".itcm_text")))
void my_critical_isr(void) {
    volatile uint32_t *sensor_reg = (volatile uint32_t *)0x40001000;
    uint32_t reading = *sensor_reg;
    process_sample(reading);
}

In this code, the __attribute__((section(".itcm_text"))) directive tells the compiler to emit this function's compiled machine code into a section called .itcm_text instead of the default .text section. The function itself reads a sensor register at the memory-mapped address 0x40001000, stores the result in a local variable, and passes it to process_sample() for further processing. The volatile keyword tells the compiler that this memory address can change at any time (because it is a hardware register), so the compiler must not optimize away the read.

On its own, the section attribute doesn't determine where the function ends up in physical memory. It just tells the compiler to label the function's code with a specific section name.

The actual memory placement is the linker script's job, which maps .itcm_text to the ITCM address range. We'll cover the linker script in detail in a later section.

How Much ITCM is Typical?

A real-world memory profile from an embedded project, to give you a sense of scale:

Memory region         Used Size  Region Size  %age Used
            ITCM:      570936 B         2 MB     27.22%
            DTCM:      727240 B    1572608 B     46.24%
             DDR:      622915 B         4 MB     14.85%

This output comes from the linker map file's summary section. It shows three memory regions and how much of each one is used by the compiled firmware.

ITCM has 2 MB available and the firmware is using about 557 KB (27.22%). DTCM has about 1.5 MB available and is using 727 KB (46.24%). DDR has 4 MB available and is using about 609 KB (14.85%).

This project uses about 557 KB of the available 2 MB of ITCM, roughly 27%. That leaves good headroom for growth.

In practice, you want to keep ITCM utilization below 80-85% to leave room for future features and library updates. If utilization climbs above 90%, you're one feature addition away from a build failure, and you should proactively move less-critical code to DDR.

What is DTCM (Data Tightly-Coupled Memory)?

DTCM stands for Data Tightly-Coupled Memory. It works on the same principle as ITCM (physically close to the CPU core, connected via a dedicated bus, single-cycle access) but it stores data instead of instructions.

If ITCM is where your code lives, DTCM is where your code works. It's the fast scratch space that the CPU reads from and writes to while executing your performance-critical functions. Every variable read, every array access, every stack push and pop in your hot code paths goes through data memory. Making that data memory as fast as possible eliminates one of the biggest sources of stall cycles.

What Kind of Data Belongs in DTCM?

Stack frames are the most important thing in DTCM. Every function call pushes a stack frame containing local variables, the return address, and saved registers. Every function return pops that frame. I

f your stack is in DTCM, the memory-access portion of function calls and returns happens in a single cycle. If your stack were in DDR, every function call and return would incur multiple cycles of memory latency just for the stack operations alone, before the function even begins doing useful work.

On most Cortex-M and Cortex-R configurations, the startup code initializes the stack pointer to point into DTCM by default, so you get this benefit without any extra configuration.

Frequently accessed global variables are another strong candidate. State machine variables, control flags, sensor readings that are updated and read in every loop iteration, counters that are incremented in ISRs and read in the main loop: all of these benefit from single-cycle access.

If a variable is read or written thousands of times per second, the cumulative latency difference between DTCM and DDR adds up.

Small lookup tables used in hot paths belong in DTCM when they're small enough to fit. Sine/cosine tables for motor control, filter coefficients for audio processing, and CRC tables for communication protocols are common examples.

These tables are typically a few hundred bytes to a few kilobytes, and they get accessed on every iteration of a processing loop. The key word is "small." A 512-byte sine table is a good fit for DTCM. A 64 KB calibration table is not, and should go in DDR instead.

DMA buffers can sometimes go in DTCM, but this depends on your chip's bus architecture. On some chips, the DMA controller has a direct path to DTCM through the bus matrix. On others, the DMA controller can only reach DDR and possibly other SRAM regions. If you place a DMA buffer in DTCM on a chip where the DMA controller can't reach it, the transfer will silently fail or write to a completely wrong address.

Always check your chip's bus matrix diagram in the reference manual before putting DMA buffers in DTCM.

How to Place Data in DTCM

Placing data in DTCM uses the same section attribute mechanism as ITCM, but with a section name that your linker script maps to the DTCM address range.

__attribute__((section(".dtcm_data")))
static int16_t audio_buffer[256];

__attribute__((section(".dtcm_data")))
static volatile uint32_t sensor_state = 0;

In this code, audio_buffer is an array of 256 signed 16-bit integers (512 bytes total) that will be placed in DTCM. This could be a buffer for audio samples that gets filled by a DMA transfer and processed by an ISR. The static keyword means the buffer has file scope and persists for the lifetime of the program (it's not allocated on the stack).

The sensor_state variable is a 32-bit unsigned integer marked as volatile, meaning the compiler must read it from memory every time it's accessed rather than caching it in a register.

This is important for variables that are written in an ISR and read in the main loop, since the compiler needs to know the value can change at any time. Placing it in DTCM ensures that both the ISR write and the main loop read happen in a single cycle.

DTCM Fills Up Faster Than ITCM

Looking at the memory profile again:

            DTCM:      727240 B    1572608 B     46.24%

This single line from the linker map file summary shows that DTCM has 1,572,608 bytes (about 1.5 MB) available, and the firmware is using 727,240 bytes (about 710 KB), which is 46.24% of the total capacity.

DTCM fills up faster than ITCM because many things compete for it: your stack, your heap (if you have one), your global variables, and data sections from every library you link against. Every C library function that uses static data, every RTOS data structure, every middleware component brings its own data footprint. This creates a constant sizing exercise.

For every data structure, you need to ask: does this really need single-cycle access, or can it work from DDR?

A Concrete Example of the Performance Impact

Say your processor runs at 400 MHz. DTCM gives you 1-cycle access. DDR gives you 8-cycle access. You have a lookup table that gets accessed 100,000 times per second.

DTCM: 100,000 accesses x 1 cycle  = 100,000 cycles/sec
DDR:  100,000 accesses x 8 cycles = 800,000 cycles/sec

Difference: 700,000 cycles/sec
At 400 MHz: 700,000 / 400,000,000 = 0.00175 seconds = 1.75 ms

This calculation shows the cycle cost of 100,000 memory accesses per second in both memory types. In DTCM, each access is 1 cycle, totaling 100,000 cycles. In DDR, each access is 8 cycles, totaling 800,000 cycles. The difference of 700,000 cycles per second, at a 400 MHz clock rate, translates to 1.75 milliseconds of additional CPU time spent waiting on memory.

If you're running a real-time control loop at 1 kHz (1 ms period), 1.75 ms of additional memory latency per second means that some individual iterations are running longer than their 1 ms budget. Whether this causes actual deadline misses depends on how the accesses are distributed across iterations and how much slack you have in your time budget, but it shows why memory placement decisions have real consequences in embedded systems.

What is DDR (Double Data Rate) Memory?

DDR is external memory. It sits on the circuit board outside the processor die, connected through a memory controller. It's much larger than TCM (typically 4 MB to several GB), but significantly slower to access.

The name "Double Data Rate" refers to how data is transferred between the DDR chip and the memory controller: data is sent on both the rising edge and the falling edge of the clock signal, effectively doubling the transfer rate compared to a single-data-rate design. But this doesn't eliminate the latency of activating rows and columns inside the DDR chip, which is where the slowness comes from.

How DDR Access Works

When your CPU reads from DDR, a multi-step process occurs inside the memory controller and DDR chip.

First, the CPU sends an address request to the memory controller. The memory controller is a hardware block inside the processor that translates CPU addresses into the specific row and column addresses that the DDR chip understands.

Second, the memory controller activates the correct row inside the DDR chip. This step is called the RAS (Row Address Strobe) phase. The DDR chip is organized as a grid of tiny capacitors, and "activating a row" means reading all the capacitors in that row into a row buffer inside the DDR chip. This takes several clock cycles.

Third, the memory controller selects the correct column within the activated row. This is called the CAS (Column Address Strobe) phase. The DDR chip uses the column address to pick the right bits out of the row buffer. This also takes several clock cycles.

Fourth, the data is transferred back to the memory controller, and from there to the CPU. The data transfer happens on both clock edges (the "double data rate" part), which helps with throughput but doesn't reduce the initial latency of the RAS and CAS phases.

The total latency depends on what state the memory is in when the request arrives. If the correct row is already activated from a previous access (a "row hit"), the RAS phase can be skipped, and the access is faster. If a different row is active and needs to be closed (precharged) before the new row can be opened (a "row miss"), the access takes longer. If the DDR chip happens to be performing a refresh cycle at that moment, the access is delayed further.

In practice, DDR access latency ranges from about 5 to 20+ CPU clock cycles, depending on the access pattern and timing.

Why DDR is Necessary

Because firmware often doesn't fit in TCM alone. Real embedded projects include protocol stacks, connectivity libraries, file system drivers, debug interfaces, and more. TCM is typically 2 to 3.5 MB total (ITCM + DTCM combined), and a full-featured firmware image can easily exceed that.

A real example showing memory usage before and after adding a wireless connectivity stack:

Without connectivity stack:
    ITCM:      506,996 B     (24.18%)
    DTCM:      628,408 B     (39.96%)
    DDR:       558,779 B     (13.32%)

With connectivity stack:
    ITCM:      570,936 B     (27.22%)
    DTCM:      727,240 B     (46.24%)
    DDR:       622,915 B     (14.85%)

Delta:
    ITCM: +63,940 B   (~62 KB of additional code)
    DTCM: +98,832 B   (~96 KB of additional data)
    DDR:  +64,136 B   (~62 KB of additional data/code)

This comparison shows memory usage from the same project built with and without a wireless connectivity stack.

The "Without" rows show the baseline. The "With" rows show the usage after adding the connectivity feature. The "Delta" rows show the difference.

Adding this single feature consumed an extra ~220 KB across all three memory regions. The time-critical parts of the stack (interrupt handlers, buffer management) went into ITCM and DTCM. The rest (packet parsers, connection management, configuration logic) went into DDR where it doesn't need single-cycle performance.

What Belongs in DDR?

Initialization and configuration code is the easiest category. Functions that run once at boot, like parsing a configuration file, initializing peripherals, or setting up data structures, don't need fast execution. They run once, take a few extra milliseconds because of DDR latency, and then never run again. Nobody notices. Put them in DDR and save TCM space for the code that runs a million times per second.

Large buffers must go in DDR because they simply can't fit in TCM. An image framebuffer for a 320x240 display at 16 bits per pixel is 150 KB. A network packet pool might be 32 KB or more. A file system cache might be 64 KB. These buffers would consume a significant fraction of DTCM's total capacity, leaving no room for the stack and variables that actually need single-cycle access.

Infrequently accessed data belongs in DDR as well. Calibration tables that are loaded once at boot and then read occasionally during operation, string tables for debug messages that are only printed during development or error conditions, and error description tables are all fine in DDR. The extra latency per access is irrelevant when the access count is low.

Non-time-critical code rounds out the DDR category. Protocol stacks (Bluetooth, Wi-Fi, TCP/IP), file system drivers, OTA update handlers, and shell/debug command interpreters all do important work, but none of them need to execute in a single clock cycle per instruction. They can tolerate the higher latency of DDR without affecting system behavior.

How to Place Code and Data in DDR

__attribute__((section(".ddr_text")))
void parse_config_file(const char *path) {
    // Runs from DDR, slower instruction fetch,
    // but config parsing happens once at boot,
    // so the latency does not affect runtime performance.
}

__attribute__((section(".ddr_bss")))
static uint8_t network_packet_pool[32768];

__attribute__((section(".ddr_bss")))
static uint8_t framebuffer[320 * 240 * 2];  // 150 KB, far too large for TCM

In this code, parse_config_file is placed in the .ddr_text section, which the linker script maps to DDR. Every instruction in this function will be fetched from DDR at multi-cycle latency, but since config parsing happens once at boot, the extra time is negligible.

The network_packet_pool is a 32 KB buffer placed in .ddr_bss. The .bss suffix is a convention indicating that this is zero-initialized data (the linker will ensure the memory is zeroed at startup rather than storing 32 KB of zeros in the firmware image). This buffer is used for network packet storage, which is not time-critical enough to justify DTCM space.

The framebuffer is a 150 KB buffer (320 pixels wide, 240 pixels tall, 2 bytes per pixel) also placed in .ddr_bss. At 150 KB, this single buffer would consume about 10% of DTCM's total capacity, which is far too expensive when the display update isn't a hard real-time operation.

How They Compare: A Side-by-Side Overview

This table summarizes the key differences between the three memory types. The most important columns are "Access latency" and "Typical size," because they represent the fundamental tradeoff: TCM is fast but small, DDR is slow but large.

The "Technology" column explains why: TCM uses SRAM (static RAM), which stores each bit using a flip-flop circuit that holds its state as long as power is applied. DDR uses DRAM (dynamic RAM), which stores each bit as charge in a tiny capacitor. Because capacitors leak charge, DRAM must be periodically refreshed, which adds power consumption and introduces occasional access delays when a refresh cycle coincides with a read request.

The Memory Map

Address Space:
  +------------------------------+  0x00000000
  |                              |
  |         ITCM (2 MB)          |  Single-cycle Inst Fetch
  |    ISRs, real-time loops,    |
  |    DSP, critical code        |
  |                              |
  +------------------------------+  0x00200000
  |       (reserved/gap)         |
  +------------------------------+  0x20000000
  |                              |
  |       DTCM (~1.5 MB)         |  Single-cycle Data Access
  |    Stack, hot variables,     |
  |    lookup tables, DMA bufs   |
  |                              |
  +------------------------------+  0x20180000
  |       (reserved/gap)         |
  +------------------------------+  0x80000000
  |                              |
  |         DDR (4 MB)           |  Multi-cycle Access
  |    Large buffers, init code, |
  |    protocol stacks, config   |
  |                              |
  +------------------------------+  0x80400000

This diagram shows the CPU's address space laid out from low addresses at the top to high addresses at the bottom. ITCM occupies the lowest 2 MB starting at address 0x00000000. After a gap of reserved/unused address space, DTCM sits at 0x20000000 and spans about 1.5 MB. Another gap of reserved space follows, and then DDR starts at 0x80000000 with 4 MB of space.

The gaps between regions are important. They're reserved address ranges that don't map to any physical memory. If your code accidentally reads from or writes to an address in one of these gaps, the result depends on the chip's bus fault configuration: it might trigger a HardFault exception, or it might silently return garbage data.

These addresses are illustrative. Every chip has its own memory map, documented in its Technical Reference Manual (TRM). Always consult your chip's TRM for the exact addresses and sizes.

How to Decide Where to Place Code and Data

Is it code or data?
|
+-- CODE (instructions):
|   +-- Called from an ISR or runs in a real-time loop?
|   |   +-- YES -> ITCM (deterministic timing is critical)
|   +-- Called frequently in the main processing pipeline?
|   |   +-- YES -> ITCM (if space is available)
|   +-- Called rarely (init, config, debug)?
|       +-- DDR (save ITCM space for critical code)
|
+-- DATA (variables, buffers, tables):
    +-- Accessed in an ISR or real-time context?
    |   +-- YES -> DTCM (single-cycle, deterministic)
    +-- Small and frequently accessed?
    |   +-- YES -> DTCM (if space is available)
    +-- Large buffer (>16 KB)?
    |   +-- Probably DDR (DTCM cannot afford the space)
    +-- Accessed only once at boot or very rarely?
        +-- DDR (do not use DTCM for this)

This decision tree captures the thought process for placing each piece of firmware into the right memory region.

Start by asking whether you're placing code (instructions) or data (variables, buffers, tables). For code, the primary question is how often it runs and whether it has timing constraints. ISR code and real-time loop code goes in ITCM. Everything else goes in DDR. For data, the primary question is how often it's accessed and how large it is. Small, frequently accessed data goes in DTCM. Large buffers and rarely-accessed data go in DDR.

The general principle: put the hottest code and data in TCM, and everything else in DDR. "Hot" means frequently accessed, latency-sensitive, or requiring deterministic timing. When in doubt, start with DDR placement and move things to TCM only when profiling shows it's necessary. It's much easier to promote a function from DDR to ITCM after discovering it's a bottleneck than to cram everything into ITCM from the start and run out of space.

How the Linker Script Controls Memory Placement

Everything we've discussed so far (section attributes, memory placement, address assignments) comes together in the linker script. This is a file (usually with a .ld extension) that tells the linker exactly which sections go into which memory regions. The linker script is the single source of truth for your firmware's memory layout.

MEMORY
{
    ITCM    (rx)  : ORIGIN = 0x00000000, LENGTH = 2M
    DTCM    (rw)  : ORIGIN = 0x20000000, LENGTH = 1536K
    DDR     (rwx) : ORIGIN = 0x80000000, LENGTH = 4M
}

SECTIONS
{
    /* === ITCM: Critical code === */
    .itcm_text :
    {
        KEEP(*(.isr_vector))          /* Interrupt vector table */
        *(.itcm_text)                 /* Functions with __attribute__((section(".itcm_text"))) */
        *audio_processing.o(.text)    /* All code from audio_processing.c */
        *motor_control.o(.text)       /* All code from motor_control.c */
    } > ITCM

    /* === DDR: Non-critical code === */
    .ddr_text :
    {
        *(.text)                      /* Default catch-all for remaining code */
        *(.text*)
        *(.rodata)                    /* Read-only data (string literals, constants) */
        *(.rodata*)
    } > DDR

    /* === DTCM: Critical data === */
    .dtcm_data :
    {
        *(.dtcm_data)                 /* Data with __attribute__((section(".dtcm_data"))) */
        *audio_processing.o(.data)    /* All initialized data from audio_processing.c */
        *audio_processing.o(.bss)     /* All zero-initialized data from audio_processing.c */
    } > DTCM

    /* === DTCM: Stack === */
    .stack (NOLOAD) :
    {
        . = ALIGN(8);
        __stack_start = .;
        . = . + 8K;                  /* 8 KB stack */
        __stack_end = .;
    } > DTCM

    /* === DDR: Everything else === */
    .ddr_data :
    {
        *(.data)                      /* Default catch-all for remaining initialized data */
        *(.bss)                       /* Default catch-all for remaining zero-initialized data */
        *(COMMON)
    } > DDR
}

This linker script has two main blocks: MEMORY and SECTIONS.

The MEMORY block defines the physical memory regions available on the chip. Each line declares a region name, its permissions (rx for read-execute, rw for read-write, rwx for read-write-execute), its starting address (ORIGIN), and its size (LENGTH). These values must match your chip's actual memory map as documented in its reference manual.

The SECTIONS block defines how the linker should distribute compiled code and data across those memory regions. Each section rule consists of a section name (like .itcm_text), a list of input patterns that specify which object file sections to include, and a > REGION directive that tells the linker which memory region to place the output section in.

The .itcm_text section collects the interrupt vector table (KEEP(*(.isr_vector))), any functions explicitly marked with __attribute__((section(".itcm_text"))), and all code from audio_processing.o and motor_control.o. The KEEP directive prevents the linker from discarding the interrupt vector table during garbage collection, even if no code appears to reference it directly. All of this goes into ITCM.

The .ddr_text section uses catch-all patterns *(.text) and *(.text*) to collect all remaining code that wasn't claimed by the ITCM section above. It also collects read-only data (.rodata), which includes string literals and const variables. All of this goes into DDR.

The .dtcm_data section collects explicitly-placed data and all data from audio_processing.o. The .stack section reserves 8 KB for the stack with 8-byte alignment, and exports the __stack_start and __stack_end symbols that your startup code and stack profiling code can reference. Both go into DTCM.

The .ddr_data section collects all remaining data with catch-all patterns, and goes into DDR.

How Section Matching Works

The linker processes sections from top to bottom. When it encounters a wildcard pattern like *(.text), it matches all .text sections that haven't already been claimed by a more specific rule earlier in the script.

So in the example above, *audio_processing.o(.text) in the ITCM section claims all code from audio_processing.c first. Then, when the linker reaches *(.text) in the DDR section, audio_processing.o's .text section has already been placed, so it's skipped. Only unclaimed .text sections from other object files match the DDR catch-all.

This means the order of sections in your linker script matters. Place your specific rules (individual object files, named sections) before the generic catch-all rules. If you put the *(.text) catch-all before the *audio_processing.o(.text) rule, the catch-all would claim everything first, and the specific rule would match nothing.

Common Mistakes to Avoid

1. Stack Overflow in DTCM

Your stack lives in DTCM. DTCM is small. If you declare a large local array inside a function, it goes on the stack:

void problematic_function(void) {
    uint8_t huge_local_buffer[65536];  // 64 KB allocated on the stack
    // This consumes 64 KB of DTCM immediately
}

This code declares a 64 KB local array. Because it's a local variable (not static), it is allocated on the stack when the function is called. If your total stack size is 8 KB (as in the linker script example above), this single declaration overflows the stack by 56 KB, writing into whatever memory is adjacent to the stack in DTCM.

On a desktop OS, a stack overflow triggers a segmentation fault because the OS uses virtual memory and guard pages to detect it.

In an embedded system without memory protection, the stack silently grows into adjacent memory regions, corrupting whatever data is stored there. The resulting bugs are extremely difficult to diagnose because the symptoms (corrupted variables, erratic behavior, intermittent crashes) appear unrelated to the actual cause. You might spend days debugging a seemingly random data corruption issue before realizing the root cause is a stack overflow from a function three call levels deep.

The fix: Use static allocation or heap allocation for large buffers, and place them in DDR:

void fixed_function(void) {
    __attribute__((section(".ddr_bss")))
    static uint8_t huge_buffer[65536];  // In DDR, not on the stack

    // Stack is safe, DTCM is not wasted
}

By making the buffer static, it's no longer allocated on the stack. Instead, the linker allocates it once in the .ddr_bss section, which maps to DDR. The buffer persists for the entire lifetime of the program (like a global variable), but its name is scoped to this function. The stack only holds a pointer to the buffer, which is a few bytes instead of 64 KB.

2. Overfilling ITCM

If you exceed ITCM's capacity, the linker will produce an error along the lines of "region ITCM overflowed by N bytes." But if you're close to the limit, you're one library update or feature addition away from a build failure. A minor version bump of your RTOS or connectivity stack could add enough code to push ITCM over the edge.

Keep headroom. The 27% utilization shown earlier is healthy. If you're above 85%, you should actively work on moving less-critical code to DDR. If you're above 95%, you have no room for growth and need to make immediate changes. Setting up automated memory budget checks in your CI pipeline (covered later in this article) prevents surprises.

3. Ignoring Alignment Requirements

TCM memories often have alignment requirements. On Cortex-M processors with strict alignment enforcement, accessing a 32-bit value at an unaligned address causes a HardFault exception.

/* Problematic: packed struct can create unaligned fields */
__attribute__((section(".dtcm_data"), packed))
struct badly_aligned {
    uint8_t  flag;
    uint32_t counter;  // May be at byte offset 1, unaligned
};

/* Correct: natural alignment, with minor padding */
__attribute__((section(".dtcm_data")))
struct properly_aligned {
    uint32_t counter;  // At offset 0, 4-byte aligned
    uint8_t  flag;     // At offset 4
    // 3 bytes of padding follow, a small cost for correctness
};

In the first struct, the packed attribute tells the compiler to use no padding between fields. This means counter starts at byte offset 1 (right after the 1-byte flag), which isn't a multiple of 4. When the CPU tries to read a 32-bit value from a non-4-byte-aligned address in TCM, it triggers a HardFault on processors with strict alignment (which includes most Cortex-M cores).

In the second struct, the fields are ordered so that counter (4 bytes) comes first at offset 0, which is naturally 4-byte aligned. The flag (1 byte) follows at offset 4. The compiler inserts 3 bytes of padding after flag to bring the struct size to 8 bytes (a multiple of 4), but this is a small price for correct, crash-free operation.

4. DMA Transfers to TCM on Incompatible Bus Architectures

Some DMA controllers can't access TCM memory. Whether DMA can reach TCM depends entirely on your chip's internal bus architecture (the bus matrix).

If you set up a DMA transfer from a peripheral to a DTCM buffer, but the DMA controller doesn't have a bus path to DTCM, the transfer will either silently fail or write to an incorrect address.

Neither produces an obvious error. The DMA controller thinks it completed successfully, your code reads the buffer expecting fresh data, and you get stale or garbage values instead. This is one of the most confusing bugs in embedded development because everything looks correct in the code.

Always check your chip's bus matrix diagram in the reference manual before using DMA with TCM buffers. The bus matrix diagram shows which masters (CPU, DMA, USB, and so on) can access which slaves (ITCM, DTCM, SRAM, DDR, peripherals). Look for whether the DMA controller's master port has a connection line to the TCM slave port. If it doesn't, your DMA transfers to TCM will not work.

Performance Comparison With Real Numbers

The following table compares access latencies across memory types, assuming a Cortex-R class processor at 400 MHz:

+---------------------+----------+----------+----------+
| Operation           | ITCM/    |   DDR    | Slowdown |
|                     | DTCM     |          | Factor   |
+---------------------+----------+----------+----------+
| Instruction fetch   | 1 cycle  | 5-20 cyc |   5-20x  |
| Data read (32-bit)  | 1 cycle  | 5-20 cyc |   5-20x  |
| Data write (32-bit) | 1 cycle  | 5-20 cyc |   5-20x  |
| Sequential burst    | 1 cyc/wd | 2-4 cy/wd|    2-4x  |
| Random access       | 1 cycle  | 10-20 cyc|  10-20x  |
+---------------------+----------+----------+----------+

This table shows the latency for five different types of memory operations. The first three rows (instruction fetch, data read, data write) show that individual accesses to TCM are always 1 cycle, while individual accesses to DDR range from 5 to 20 cycles depending on the memory's internal state. The slowdown factor is the ratio between the two.

The "Sequential burst" row shows what happens when you read or write consecutive addresses. DDR performs much better in burst mode (2-4 cycles per word instead of 5-20) because once a row is activated, subsequent reads from the same row skip the RAS phase. TCM is still 1 cycle per word because it doesn't have the row/column structure of DDR.

The "Random access" row shows the worst case for DDR. When each access hits a different row, the memory controller must precharge the old row and activate the new one every time. This is the 10-20 cycle range, and it's common in workloads that jump around in memory (traversing linked lists, hash table lookups, and indirect function calls through function pointer arrays).

The practical takeaway: if your code accesses DDR data, try to access it sequentially. Iterating through an array in order is much faster than jumping to random positions. Your memory controller and the DDR chip's internal prefetch logic work in your favor during sequential access patterns.

How TCM Affects Power Consumption

Memory placement has a direct impact on power consumption, something that becomes critical for battery-powered products.

DDR requires constant refresh cycles. DRAM stores each bit as charge in a tiny capacitor, and that charge leaks over time.

To prevent data loss, the memory controller must read and rewrite every row in the DDR chip approximately every 64 ms. This refresh process consumes power even when the processor is sleeping and no code is running. On some systems, DDR refresh can account for a significant portion of the total sleep-mode power budget.

TCM is SRAM-based and doesn't require refresh. SRAM stores data using flip-flop circuits that hold their state as long as power is applied. There is some leakage current (no transistor is perfect), but it is orders of magnitude lower than DDR refresh power.

For battery-powered devices (wearables, IoT sensors, medical devices), this means you should keep data that must survive sleep modes in DTCM when possible.

If your hardware supports it, power-gate the DDR chip during deep sleep to eliminate its refresh power entirely. The less DDR your firmware uses at runtime, the more aggressively you can manage DDR power states, which directly extends battery life.

How to Profile Memory Usage

After placing code and data into ITCM, DTCM, and DDR, you need to verify that everything fits, monitor usage over time, and catch regressions before they become build failures. There are several techniques for this, ranging from simple command-line tools to automated CI checks.

Method 1: The Linker Map File

Every time you build your firmware, the linker can produce a map file, a detailed text file that records where every symbol (function, variable, constant) ended up and how large it is. This is the most useful single artifact in embedded development for understanding memory usage.

To generate one, add -Wl,-Map=output.map to your linker flags:

arm-none-eabi-gcc \
    -T linker_script.ld \
    -Wl,-Map=firmware.map \
    -o firmware.elf \
    main.o audio.o bluetooth.o

This command invokes the ARM GCC toolchain to link three object files (main.o, audio.o, bluetooth.o) using the linker script linker_script.ld. The -Wl,-Map=firmware.map flag tells GCC to pass the -Map=firmware.map option to the linker, which causes it to write a detailed map file alongside the output ELF binary. The map file can be thousands of lines long, but the most useful part is the summary at the end.

The summary at the end of the map file shows overall utilization per memory region:

Memory region         Used Size  Region Size  %age Used
            ITCM:      570936 B         2 MB     27.22%
            DTCM:      727240 B    1572608 B     46.24%
             DDR:      622915 B         4 MB     14.85%

This summary shows three columns: how many bytes are used, the total size of the region, and the percentage used. It gives you the health of your firmware at a glance. As a rule of thumb, below 80% is healthy with room for growth. Between 80% and 90% is getting tight, and you should plan for how you will accommodate the next feature. Above 90% requires action: start moving things to a cheaper memory region or optimizing existing placement.

Method 2: Parsing the Map File for Per-Module Breakdown

The summary tells you how much memory is used, but not who is using it. The map file contains per-symbol details, but they're difficult to read manually because the file can be thousands of lines long with a format that isn't designed for human consumption.

The following Python script parses the map file and produces a per-module report showing which object files are consuming memory in which regions.

#!/usr/bin/env python3
"""Parse a linker map file and report memory usage per object file."""

import re
import sys
from collections import defaultdict

def parse_map_file(map_path):
    """Extract symbol placements from a GCC linker map file."""
    usage = defaultdict(lambda: defaultdict(int))

    regions = {
        'ITCM': (0x00000000, 0x00200000),
        'DTCM': (0x20000000, 0x20180000),
        'DDR':  (0x80000000, 0x80400000),
    }

    def addr_to_region(addr):
        for name, (start, end) in regions.items():
            if start <= addr < end:
                return name
        return 'UNKNOWN'

    symbol_re = re.compile(
        r'^\s+\S+\s+(0x[0-9a-fA-F]+)\s+(0x[0-9a-fA-F]+)\s+(\S+\.o)'
    )

    with open(map_path) as f:
        for line in f:
            m = symbol_re.match(line)
            if m:
                addr = int(m.group(1), 16)
                size = int(m.group(2), 16)
                obj = m.group(3).split('/')[-1]
                region = addr_to_region(addr)
                usage[obj][region] += size

    return usage

def print_report(usage):
    """Print a sorted memory usage report."""
    print(f"{'Object File':<35} {'ITCM':>10} {'DTCM':>10} {'DDR':>10} {'Total':>10}")
    print("-" * 80)

    totals = defaultdict(int)
    rows = []

    for obj, regions in usage.items():
        total = sum(regions.values())
        rows.append((obj, regions, total))
        for r, s in regions.items():
            totals[r] += s

    rows.sort(key=lambda x: x[2], reverse=True)

    for obj, regions, total in rows[:20]:
        print(f"{obj:<35} "
              f"{regions.get('ITCM', 0):>10,} "
              f"{regions.get('DTCM', 0):>10,} "
              f"{regions.get('DDR', 0):>10,} "
              f"{total:>10,}")

    print("-" * 80)
    grand = sum(totals.values())
    print(f"{'TOTAL':<35} "
          f"{totals.get('ITCM', 0):>10,} "
          f"{totals.get('DTCM', 0):>10,} "
          f"{totals.get('DDR', 0):>10,} "
          f"{grand:>10,}")

if __name__ == '__main__':
    usage = parse_map_file(sys.argv[1])
    print_report(usage)

This script does three things. First, parse_map_file reads the map file line by line, looking for lines that match the format of a symbol placement entry (a section name, an address, a size, and an object file name). For each match, it converts the hex address to an integer, determines which memory region it falls in using the addr_to_region helper, and accumulates the size into a nested dictionary keyed by object file and region.

Second, print_report sorts the object files by total memory usage (largest first), prints the top 20, and shows how much each one uses in each region.

Third, the if __name__ == '__main__' block makes the script runnable from the command line.

You'll need to adjust the address ranges in the regions dictionary to match your chip's memory map.

Run it with:

python3 parse_map.py firmware.map

Sample output:

Object File                              ITCM       DTCM        DDR      Total
--------------------------------------------------------------------------------
bluetooth_stack.o                      42,380     65,200     38,400    146,080
audio_processing.o                     89,200     32,000          0    121,200
wifi_driver.o                          21,560     33,632     25,736     80,928
sensor_hub.o                           45,000     18,400          0     63,400
libc.a(memcpy.o)                       12,340          0          0     12,340
...
--------------------------------------------------------------------------------
TOTAL                                 570,936    727,240    622,915  1,921,091

This output shows the top memory consumers in the firmware, sorted by total usage. Each row shows an object file and how many bytes it contributes to each memory region.

The bluetooth_stack.o file is the largest consumer at 146 KB total, spread across all three regions. The audio_processing.o file uses 121 KB, all in ITCM and DTCM (0 bytes in DDR), which makes sense because audio processing is time-critical and was placed entirely in TCM. The libc.a(memcpy.o) entry shows a C library function that was placed in ITCM, likely because it is called from performance-critical code paths.

Method 3: The size Command

For a quick check without parsing the map file, use arm-none-eabi-size:

arm-none-eabi-size -A firmware.elf

Output:

firmware.elf  :
section               size        addr
.itcm_text          570936           0
.dtcm_data          530240   536870912
.dtcm_bss           196000   537401152
.stack                8192   537600000
.ddr_text           422915  2147483648
.ddr_data           120000  2147906563
.ddr_bss             80000  2148026563
Total              1928283

This output lists every section in the ELF binary, its size in bytes, and its starting address (shown in decimal).

You can map sections to memory regions by looking at the address: addresses near 0 are ITCM, addresses near 536 million (0x20000000) are DTCM, and addresses near 2.1 billion (0x80000000) are DDR.

Alternatively, the section names themselves indicate the region (.itcm_text is in ITCM, .dtcm_data and .dtcm_bss are in DTCM, .ddr_text and .ddr_data and .ddr_bss are in DDR).

The -A flag gives per-section sizes instead of the default BSD-format output. It's less detailed than the map file approach, but it runs instantly and gives you the big picture.

Method 4: Runtime Stack Profiling

Static analysis (map files, size output) tells you about compile-time placement. But some memory usage is dynamic, particularly the stack, which grows and shrinks at runtime based on call depth and local variable sizes. A function that allocates a 2 KB local buffer only uses that stack space while it is executing, so static analysis can't tell you the peak stack usage.

A common technique is stack watermarking: fill the entire stack region with a known pattern at boot, then periodically check how much of the pattern has been overwritten.

#define STACK_FILL_PATTERN 0xDEADBEEF

void stack_watermark_init(void) {
    extern uint32_t __stack_start;
    extern uint32_t __stack_end;
    uint32_t *p = &__stack_start;

    register uint32_t sp asm("sp");
    while (p < (uint32_t *)(sp - 64)) {
        *p++ = STACK_FILL_PATTERN;
    }
}

uint32_t stack_usage_bytes(void) {
    extern uint32_t __stack_start;
    extern uint32_t __stack_end;
    uint32_t *p = &__stack_start;

    while (p < &__stack_end && *p == STACK_FILL_PATTERN) {
        p++;
    }

    return (uint32_t)(&__stack_end) - (uint32_t)p;
}

void check_stack_health(void) {
    uint32_t used = stack_usage_bytes();
    uint32_t total = 8192;
    uint32_t percent = (used * 100) / total;

    if (percent > 80) {
        log_warning("Stack usage: %lu / %lu bytes (%lu%%)",
                    used, total, percent);
    }
}

The stack_watermark_init function fills the stack memory (from __stack_start to just below the current stack pointer) with the pattern 0xDEADBEEF. The extern declarations reference the linker symbols defined in the linker script's .stack section. The register uint32_t sp asm("sp") line reads the current stack pointer value so the function knows where to stop filling (you do not want to overwrite your own stack frame). The 64-byte safety margin ensures the fill loop doesn't get too close to the active stack.

The stack_usage_bytes function scans from the bottom of the stack upward, counting how many words still contain the fill pattern. The first word that does not match the pattern indicates the deepest point the stack has reached (the high-water mark). The function returns the number of bytes from that point to the top of the stack.

The check_stack_health function computes the percentage of stack used and logs a warning if it exceeds 80%. Call this function periodically during normal operation to monitor stack usage.

Call stack_watermark_init() as early as possible in your startup code (before main() if you can), then call check_stack_health() periodically during normal operation. This tells you the high-water mark, the maximum stack depth your firmware has reached so far.

Method 5: Tracking Memory Across Builds

Every time you add a feature or merge a change, run the memory profile before and after:

arm-none-eabi-size -A firmware_before.elf > mem_before.txt
arm-none-eabi-size -A firmware_after.elf > mem_after.txt
diff mem_before.txt mem_after.txt

These three commands capture the section sizes of two firmware builds (before and after a change) into text files, then diff them to see what changed. This is useful but the raw diff output can be hard to read. The following script provides a cleaner view by computing the delta per memory region:

#!/bin/bash
# memory_diff.sh - Compare memory usage between two builds

echo "Memory Impact of Change:"
echo "========================"

parse_size() {
    arm-none-eabi-size -A "$1" | awk '
    /\.itcm/  { itcm += $2 }
    /\.dtcm/  { dtcm += $2 }
    /\.ddr/   { ddr += $2 }
    /\.stack/ { dtcm += $2 }
    END { printf "%d %d %d", itcm, dtcm, ddr }
    '
}

read itcm_before dtcm_before ddr_before <<< \((parse_size "\)1")
read itcm_after  dtcm_after  ddr_after  <<< \((parse_size "\)2")

printf "ITCM: %+d bytes (%d -> %d)\n" \
    \(((itcm_after - itcm_before)) \)itcm_before $itcm_after
printf "DTCM: %+d bytes (%d -> %d)\n" \
    \(((dtcm_after - dtcm_before)) \)dtcm_before $dtcm_after
printf "DDR:  %+d bytes (%d -> %d)\n" \
    \(((ddr_after - ddr_before)) \)ddr_before $ddr_after

This script takes two ELF files as arguments (the "before" and "after" builds). The parse_size function runs arm-none-eabi-size -A on the given ELF file and uses awk to sum up section sizes by memory region. Sections whose names contain .itcm are counted toward ITCM, sections containing .dtcm or .stack toward DTCM, and sections containing .ddr toward DDR. The main body reads the before and after values, then prints the delta for each region with a + or - sign.

Usage and output:

$ ./memory_diff.sh firmware_without_bt.elf firmware_with_bt.elf

Memory Impact of Change:
========================
ITCM: +63940 bytes (506996 -> 570936)
DTCM: +98832 bytes (628408 -> 727240)
DDR:  +64136 bytes (558779 -> 622915)

This output shows that adding the Bluetooth feature increased ITCM by about 62 KB, DTCM by about 96 KB, and DDR by about 62 KB. You can put this in your CI/CD pipeline so that every pull request shows exactly how much memory it costs.

Method 6: Automated Memory Budget Checks in CI

You can integrate memory profiling into your CI/CD pipeline to catch overflows before they land in your main branch.

#!/bin/bash
# memory_check.sh - Fail CI if memory usage exceeds thresholds

ITCM_LIMIT=85   # percent
DTCM_LIMIT=80
DDR_LIMIT=90

check_region() {
    local name=\(1 used=\)2 total=\(3 limit=\)4
    local percent=$((used * 100 / total))

    if [ \(percent -ge \)limit ]; then
        echo "FAIL: \(name usage is \){percent}% (limit: ${limit}%)"
        echo "      Used: \(used / \)total bytes"
        return 1
    else
        echo "OK:   \(name usage is \){percent}% (limit: ${limit}%)"
        return 0
    fi
}

ITCM_USED=\((grep "ITCM:" firmware.map | awk '{print \)2}')
ITCM_TOTAL=$((2 * 1024 * 1024))

DTCM_USED=\((grep "DTCM:" firmware.map | awk '{print \)2}')
DTCM_TOTAL=1572608

DDR_USED=\((grep "DDR:" firmware.map | awk '{print \)2}')
DDR_TOTAL=$((4 * 1024 * 1024))

FAILED=0
check_region "ITCM" \(ITCM_USED \)ITCM_TOTAL $ITCM_LIMIT || FAILED=1
check_region "DTCM" \(DTCM_USED \)DTCM_TOTAL $DTCM_LIMIT || FAILED=1
check_region "DDR"  \(DDR_USED  \)DDR_TOTAL  $DDR_LIMIT  || FAILED=1

exit $FAILED

This script reads memory usage numbers from the linker map file and compares them against configurable percentage thresholds. The check_region function takes a region name, the number of bytes used, the total bytes available, and the percentage limit. It computes the actual percentage and prints either "OK" or "FAIL" along with the numbers. If any region exceeds its limit, the script exits with a non-zero status, which causes the CI build to fail.

The thresholds at the top (85% for ITCM, 80% for DTCM, 90% for DDR) should be adjusted based on your project's growth rate and how much headroom you want to maintain. DTCM has a lower limit because it fills up faster and is harder to free up.

Add this script to your build pipeline so every pull request shows its memory cost. If a change pushes any region past its threshold, the build fails and the developer knows immediately.

Method 7: Heap Tracking at Runtime

If your embedded project uses dynamic memory allocation (malloc/free), you can wrap the allocator to track usage.

static size_t heap_used = 0;
static size_t heap_peak = 0;

void *tracked_malloc(size_t size) {
    size_t *block = (size_t *)malloc(size + sizeof(size_t));
    if (!block) return NULL;

    *block = size;
    heap_used += size;
    if (heap_used > heap_peak) {
        heap_peak = heap_used;
    }

    return (void *)(block + 1);
}

void tracked_free(void *ptr) {
    if (!ptr) return;
    size_t *block = ((size_t *)ptr) - 1;
    heap_used -= *block;
    free(block);
}

void print_heap_stats(void) {
    printf("Heap: current=%zu bytes, peak=%zu bytes\n",
           heap_used, heap_peak);
}

This code wraps malloc and free with tracking logic. The tracked_malloc function allocates slightly more memory than requested (an extra sizeof(size_t) bytes) and stores the requested size in the first word of the allocation. It then updates the heap_used counter and, if the new total exceeds the previous peak, updates heap_peak. It returns a pointer that's offset past the size header, so the caller sees a normal pointer to their data.

The tracked_free function reverses the process: it subtracts one size_t from the pointer to find the hidden size header, subtracts that size from heap_used, and calls the real free on the original block.

The print_heap_stats function prints the current and peak heap usage. Call it periodically or on demand through a debug interface (UART console, debug CLI) to monitor how much heap your firmware is using.

This approach has a small overhead (one extra word per allocation), but it gives you visibility into dynamic memory usage that's otherwise completely invisible. It's especially useful for tracking down memory leaks: if heap_used keeps growing over time without ever decreasing, something is allocating without freeing.

Summary

Embedded processors based on ARM Cortex-M and Cortex-R architectures give you direct control over three memory regions with very different performance characteristics.

ITCM (Instruction Tightly-Coupled Memory) stores your most performance-critical code. It provides single-cycle, deterministic instruction fetch. It's small (typically 512 KB to 2 MB), so reserve it for ISRs, real-time processing functions, and hot loops.

DTCM (Data Tightly-Coupled Memory) stores your most performance-critical data. It also provides single-cycle, deterministic access. Your stack lives here by default. It's even smaller than ITCM and fills up quickly, so be deliberate about what you place in it.

DDR (Double Data Rate) memory stores everything else. It's much larger but slower (5 to 20+ cycles per access, with variable latency). Use it for initialization code, large buffers, protocol stacks, and anything that doesn't need deterministic timing.

You control placement through __attribute__((section(...))) in your C code and section-to-region mappings in your linker script. You verify placement through map files, the size command, and runtime profiling techniques like stack watermarking. The core skill is knowing which region each piece of your firmware belongs in, and having the tooling to catch mistakes early.

Then you try to connect one more thing. Maybe a fitness band, a smart lock, or that tiny temperature sensor you ordered online because it was on sale. That is when the charm fades and reality walks in. Suddenly the connection drops, your phone cannot find the device anymore, and the once-friendly Bluetooth logo on your screen starts to feel like a taunt. You restart, you unpair, you try again, and somehow it only gets worse. What was once effortless turns into a puzzle with no clear solution.

Here is the secret that few people know: Bluetooth was never meant to handle the chaos we put it through today. When engineers designed it in the late 1990s, they imagined a world of simple one-to-one connections. A laptop talking to a mouse. A phone connecting to a headset. That was the whole idea. Fast-forward to the present and we are using the same technology to run entire networks of wearables, sensors, and smart appliances. We ask it to connect not just one or two devices but sometimes dozens of them at the same time, each running on different hardware and software. It is a miracle that it works at all.

To make things even more interesting, these devices live in very different worlds. Android devices are like an open playground where every manufacturer adds its own slide and swing set. iPhones live inside Apple’s carefully fenced garden where everything is polished but also tightly controlled. Embedded devices, like the ones built on tiny chips inside sensors or IoT boards, are the quiet introverts of the group. They have little memory, tiny batteries, and a strong preference for naps to save power. Getting all three to cooperate is a bit like trying to organize a band where one member only plays jazz, another insists on classical, and the third speaks in Morse code.

That is what engineers mean when they talk about scaling Bluetooth. It is not just about adding more devices. It is about making sure completely different systems can talk to each other reliably and continuously without draining their batteries or losing their minds. It requires design decisions that consider timing, power management, data formats, and even how the operating system schedules background tasks.

This article will guide you through that strange world. We will peel back the layers of how Bluetooth actually works and what happens when Android, iOS, and embedded devices try to share the same airwaves. We will explore why each one behaves the way it does and what you can do to build systems that stay connected instead of collapsing under their own complexity.

By the end, you will see that Bluetooth is not really broken. It is simply overworked. It is a polite translator trying to keep three very different languages in sync. Once you learn how to manage its quirks and give it the structure it needs, Bluetooth becomes not a source of frustration but a quiet, invisible network that holds the modern world together.

Table of Contents

• Bluetooth Has Two Personalities — Meet Classic and BLE

• Android, iOS, and Embedded Devices — The Odd Trio

• Architecting for Scale — Herding Cats, but Wirelessly

• Connection, Discovery, and Data Flow — The Bluetooth Dating Game

• Platform Quirks — And How to Stay Sane

• Security and Privacy at Scale

• Power and Performance Tuning

• Provisioning and Firmware Updates — Welcome to Device Kindergarten

• Debugging, Monitoring, and Testing Across Platforms

• Real-World Architecture Example — When Bluetooth Finally Behaves

• Checklist — Building a Truly Scalable Bluetooth System

• Wrap-Up — Lessons from the Field

Bluetooth Has Two Personalities — Meet Classic and BLE

Before we can talk about scaling Bluetooth, we have to understand that Bluetooth itself has a bit of an identity crisis. It actually comes in two flavors: Classic Bluetooth and Bluetooth Low Energy, also called BLE. They share the same name and sometimes even live on the same chip, but under the hood they behave very differently. Think of them as twins who went to completely different schools and now have opposite personalities.

Classic Bluetooth is the older sibling. It was designed for steady, high-speed data streams. This is the version your headphones, speakers, and car systems use. It is reliable for sending large amounts of data like audio, but it is also chatty and power-hungry. It likes to stay connected all the time, constantly keeping the line open so it can send sound packets smoothly. You could say Classic Bluetooth is like that one friend who calls instead of texting and keeps the conversation going even when there is nothing left to say.

Then there is Bluetooth Low Energy, the younger, more introverted sibling. BLE was designed for devices that need to last for weeks or months on tiny batteries. It does not keep a constant connection open. Instead, it wakes up, sends or receives a little bit of data, and then goes back to sleep. It is the protocol behind fitness trackers, heart rate monitors, smart locks, and most modern IoT devices. If Classic Bluetooth is a full-time conversation, BLE is more like sending quick text messages throughout the day, short, efficient, and battery-friendly.

The funny thing is that even though they share the same wireless spectrum and sometimes even the same antenna, these two modes do not talk to each other directly. A BLE device cannot communicate with a Classic Bluetooth-only device. This is why your wireless headphones can pair with your phone, but your BLE heart rate monitor cannot talk to your old Bluetooth speaker. They live in the same neighborhood but never attend the same parties.

Most of the world’s scaling problems come from BLE, not Classic Bluetooth. Classic has been around long enough that its use cases are stable and well understood. BLE, on the other hand, is used in thousands of different kinds of devices, each with different timing requirements, power limits, and operating systems. When you try to make Android, iOS, and embedded systems all use BLE together, you are juggling three slightly different interpretations of the same rulebook.

To make things trickier, each platform implements BLE its own way. Android exposes it through flexible but sometimes unpredictable APIs. iOS keeps it tidy under Apple’s strict Core Bluetooth framework. Embedded devices rely on lightweight vendor stacks that can vary from chip to chip. Every one of these stacks follows the same Bluetooth specification, but like recipes written by different chefs, the results can taste a little different.

Understanding this dual nature is key to building anything that scales. You must know when to use Classic Bluetooth for high-speed continuous data, when to use BLE for low-power bursts, and how to design your system so that the right devices use the right mode. It is the first step in turning Bluetooth from a confusing mystery into a reliable network you can actually control.

Android, iOS, and Embedded Devices — The Odd Trio

Now that we know Bluetooth has two personalities, let’s meet the three characters that make scaling it so complicated: Android, iOS, and embedded devices. They all speak Bluetooth, but in their own unique accents. Sometimes they understand each other perfectly, and other times it feels like they’re arguing in three different languages while pretending they’re on the same page.

Let’s start with Android. Android is the enthusiastic extrovert of the group. It gives you tons of control and freedom. You can scan, connect, advertise, read, write, and basically poke around every corner of the Bluetooth stack. But that freedom comes with chaos. Because Android runs on phones made by dozens of manufacturers, each one tweaks the Bluetooth implementation a little differently. On one phone, everything works flawlessly. On another, the same code randomly drops connections or refuses to scan in the background. Even Android engineers joke that if your Bluetooth works the same on every device, you’ve probably entered a parallel universe.

Android is powerful but unpredictable. It’s like a sports car that can win a race on a good day but sometimes refuses to start if it doesn’t like the weather. The trick is to write code that expects weird behavior, to build your own connection queues, add retries, and prepare for the occasional glitch. Developers who survive Android Bluetooth bugs don’t just gain experience, they gain humility.

Then there’s iOS, Apple’s polished and opinionated perfectionist. Unlike Android, iOS is consistent. The same code usually behaves the same way across every iPhone and iPad. Apple’s Bluetooth framework, called Core Bluetooth, is beautifully organized and well-documented. But Apple also has strict rules about what you can and can’t do. Background scanning? Only in very specific cases. Advertising? Only for certain UUIDs. Access to lower-level Bluetooth layers? Absolutely not. Apple’s approach is like a luxury hotel: everything looks gorgeous, but you’re not allowed in the kitchen.

Working with iOS feels calm at first. Your connections are stable, your APIs are clear, and your devices behave predictably. But the moment you need to do something slightly unconventional, like connecting to multiple peripherals at once or keeping the app alive in the background, iOS politely says, “No, that’s not how we do things here.” Developers often end up performing delicate dances with background modes, notifications, and clever reconnection tricks just to make things feel seamless for users.

And then we have the third member of the trio: embedded devices. These are the quiet, uncomplaining ones that actually do most of the work. They live inside your smart sensors, wearables, and IoT nodes. They’re usually built around tiny chips with limited memory and low-power processors. They don’t have fancy operating systems or flashy UI frameworks. All they know is how to advertise, connect, send data, and then go back to sleep to save battery.

Embedded devices are loyal but easily overwhelmed. They can’t handle constant large data transfers, and they get cranky if you make them maintain too many simultaneous connections. Imagine trying to run a marathon after eating one grape, that’s what it’s like for a small BLE chip to handle too much traffic. Yet, these little devices are the backbone of every scalable Bluetooth network. They measure your heart rate, control your smart lights, and track your environmental sensors, all while running quietly in the background.

The real challenge begins when you try to make these three cooperate. Android wants freedom, iOS wants structure, and embedded devices just want a nap. Getting them all to work together is like managing a group project where one person writes essays at midnight, another color-codes everything, and the third forgets to charge their laptop. But when you finally get it right, when Android, iOS, and your embedded nodes connect seamlessly, it feels like magic again.

In the next section, we’ll explore how to actually make that happen. You’ll see how to design a Bluetooth architecture that scales gracefully across these platforms instead of collapsing into a pile of logs and retries. It’s part engineering, part patience, and part diplomacy.

Architecting for Scale — Herding Cats, but Wirelessly

If there’s one secret to scaling Bluetooth, it’s this: treat it like herding cats. You’ll never truly control it, but with enough structure, patience, and a bit of catnip (or clever engineering), you can convince all the cats to move in roughly the same direction.

Building a Bluetooth system that spans Android, iOS, and embedded devices isn’t just about writing code that connects things. It’s about designing relationships, the rules and boundaries that keep those connections healthy. The key idea here is architecture, which is a fancy word for “deciding who does what, when, and how.” Without a solid architecture, your Bluetooth project quickly turns into a tangle of callbacks, disconnections, and unanswered packets.

The first principle of Bluetooth architecture is abstraction. Every platform has its own Bluetooth API, but the basic idea is always the same: scan for devices, connect, exchange data, and disconnect. So instead of writing separate logic for each platform, you create one unified interface, a sort of translator layer, that hides all the messy differences underneath. In practice, this means you can write something like connect(device) in your app, and whether you’re on Android, iOS, or even a Raspberry Pi, the underlying code figures out how to make it happen.

This abstraction layer is your peacekeeper. It prevents the rest of your app from needing to know whether it’s talking to a Nordic chip on a wristband, a smart bulb using an ESP32, or an iPhone pretending to be a peripheral. When you have hundreds or thousands of devices, abstraction isn’t just convenient, it’s survival.

Next comes connection management. BLE connections are like toddlers: they demand constant attention and can vanish the moment you look away. A scalable Bluetooth system can’t afford to panic every time a device disconnects. Instead, you design it to expect chaos. You add automatic retries, reconnection strategies, and timeouts that gracefully handle failures instead of freezing your app. Good systems don’t assume the network will always behave, they assume it won’t.

Then there’s data orchestration, deciding who talks first, how much data gets sent, and how you keep multiple connections from tripping over each other. Imagine you’re a conductor in an orchestra where half the instruments fall asleep randomly to save power. You need a plan that lets each device play its part in harmony without draining its battery. That’s what managing Bluetooth data flow feels like.

And finally, there’s power strategy. Embedded devices live on tight energy budgets. Every scan, advertisement, and data exchange eats into their lifespan. So, your architecture must schedule communication intelligently, let devices wake up briefly, share data, and return to sleep before they burn out. The best Bluetooth systems look lazy on the surface but are actually brilliant planners underneath.

When you put all of this together, abstraction, connection management, orchestration, and power control, you get something that scales. It doesn’t matter if you’re managing three wearables or three thousand sensors. The system behaves predictably, logs issues instead of panicking, and recovers from disconnections automatically.

Think of it like a well-run airport. Planes (your devices) take off and land constantly. The control tower (your app’s Bluetooth manager) keeps track of who’s in the air, who’s landing next, and who needs maintenance. No single pilot needs to know everything, they just follow the protocol.

Scaling Bluetooth isn’t about being clever with one device. It’s about designing systems that keep working even when dozens of devices act unpredictably. You don’t tame Bluetooth by force; you do it by creating a world where even chaos feels organized.

In the next section, we’ll dig deeper into how these connections actually behave in real time, how devices discover each other, exchange data, and, sometimes, break up without warning.

Connection, Discovery, and Data Flow — The Bluetooth Dating Game

Every Bluetooth connection starts like a modern love story. One device sends out signals into the air, announcing that it’s available. Another device scans the surroundings, hoping to find something compatible. When they finally spot each other, they exchange a few polite packets, decide they’re a good match, and try to make it official with a connection. It’s wireless romance, until one of them walks away without saying goodbye.

This is the heart of how Bluetooth works: advertising, discovery, and connection. An embedded sensor or wearable device usually plays the role of the advertiser. It broadcasts tiny packets called advertisements that contain just enough information to say, “Hey, I’m here, and I can measure temperature or heart rate or unlock your door.” These packets are intentionally small because transmitting data takes energy, and low-power devices have to conserve every drop of battery life.

Meanwhile, your phone or tablet acts as the scanner, it listens to the radio waves around it, searching for those signals. When it finds one that matches what it’s looking for, it sends a request to connect. If the peripheral accepts, they move into a new relationship phase: the GATT connection. GATT stands for Generic Attribute Profile, which is basically the language they use to talk. Once connected, your phone can ask the device for specific data, like reading a heart rate measurement or writing a configuration setting.

Now, if all of this sounds peaceful and predictable, that’s because we haven’t talked about what happens in the real world. In reality, devices move around, signals weaken, and phones go into power-saving modes that forget they were even connected. Connections drop. Pairing sometimes fails. And when you have ten or more devices talking at once, managing all those tiny wireless conversations becomes a circus act.

Scaling Bluetooth is all about keeping this circus under control. You can’t force every device to stay connected forever, that would drain batteries and jam the radio channels. Instead, you design a rhythm. Devices connect only when needed, exchange data quickly, and then disconnect to rest. This constant dance of connecting and disconnecting keeps the system efficient and stable.

Think of it like a well-run coffee shop. Customers (phones) walk in, place their order (data request), get their coffee (response), and leave. The barista (the embedded device) doesn’t serve one person all day, it serves everyone in quick cycles. The trick is to make sure no one gets stuck waiting for their latte forever.

Timing is everything in this dance. If a device advertises too infrequently, the phone might not discover it in time. If it advertises too often, it wastes power. If the phone sends too many requests at once, the device might crash or slow down. Bluetooth connections live in this delicate balance between performance and efficiency.

When you scale, you also have to think about coordination. Imagine one phone trying to talk to ten sensors at once. You can’t have it flood them all with requests simultaneously, it needs a queue, a polite way of saying “you first, then me.” This is called connection orchestration, and it’s one of the hardest parts of scaling BLE systems.

And then there’s the breakup. Devices disconnect all the time, sometimes intentionally, sometimes accidentally. The best Bluetooth systems treat disconnections not as failures but as normal events. The app automatically retries, reconnects, and syncs data without asking the user to “try again.” To users, it feels seamless. Underneath, there’s a lot of quiet heroism happening, background threads, timers, and reconnection logic all working together to patch up relationships on the fly.

So, at its core, Bluetooth is less like a stable marriage and more like speed dating with excellent scheduling. Everyone meets briefly, exchanges information, and moves on. When done right, this model scales effortlessly. When done wrong, it’s chaos.

In the next section, we’ll explore the quirks that make Android, iOS, and embedded devices behave differently in this dating game, and how to keep the peace when one of them inevitably ghosts the others.

Platform Quirks — And How to Stay Sane

Once you start scaling Bluetooth, you’ll notice something odd. The same code that works perfectly on one device suddenly refuses to behave on another. It’s like watching identical twins argue about who gets the last slice of pizza, they may look the same, but their personalities couldn’t be more different.

Let’s start with Android, the unpredictable one. Android gives developers more power than any other mobile platform. You can scan however you like, filter by services, read and write any characteristic, and even customize connection intervals. But that power comes at a price. Every phone manufacturer modifies the Bluetooth stack slightly. Samsung, Pixel, OnePlus, Xiaomi, each adds its own flavor of “enhancement,” which sometimes translates to “surprise, nothing works the same.”

One Android phone might handle ten connections at once without blinking. Another might drop all of them the moment the screen turns off. Some versions ignore Bluetooth permissions until you grant location access. Others claim they’re scanning when they actually stopped five minutes ago. Android developers eventually stop asking why and simply build more logging instead. The rule of thumb with Android Bluetooth is simple: test everything, assume nothing, and expect the unexpected.

Then there’s iOS, which at first feels like a breath of fresh air. Apple’s Core Bluetooth framework is clean, consistent, and almost elegant. You get predictable callbacks, smooth reconnections, and well-behaved devices. But if you step outside Apple’s boundaries, you’ll quickly find invisible fences. iOS doesn’t let apps scan in the background freely. It limits how often you can advertise. And if your app tries to keep too many simultaneous connections alive, iOS politely steps in and shuts them down.

Apple’s philosophy is control. It wants Bluetooth connections to behave in ways that don’t drain the battery or clutter the radio. That’s great for users, but for developers it can feel like being handed the keys to a Ferrari and told you can only drive in the parking lot. It works beautifully, as long as you color inside the lines.

And then we have embedded devices, which are in a category of their own. These are the little chips sitting inside your wearables, sensors, or IoT gadgets. They don’t have operating systems or background processes. They just run tiny loops of firmware that listen, respond, and sleep. Their quirks are more about physics than software. If the antenna isn’t tuned properly, signals drop. If the power supply fluctuates, the radio turns off. Sometimes they disconnect simply because a human walked between two devices and absorbed the signal.

Embedded Bluetooth stacks also differ by manufacturer. Nordic, Espressif, Silicon Labs, Texas Instruments, each has its own libraries, quirks, and limitations. Even small changes like increasing the packet size or adjusting the advertising interval can make or break communication. It’s a careful dance between efficiency and reliability.

Now imagine you’re trying to get all three of these worlds to cooperate. Android wants freedom, iOS enforces discipline, and embedded devices want long naps. Building a Bluetooth system that works across all of them is like running a daycare with overachievers, rule-followers, and kids who fall asleep mid-activity. You can’t treat them all the same, but you can design a routine that keeps everyone content.

The secret is resilience. Instead of expecting perfect behavior, build your system around imperfections. Add retries when connections fail. Cache data so you don’t lose progress during disconnections. Keep your embedded devices simple, your mobile apps forgiving, and your logs brutally honest.

If you design with these quirks in mind, your Bluetooth system will feel almost magical, even though, behind the scenes, it’s a web of error handling, reconnections, and polite compromise.

In the next section, we’ll take a look at another side of scaling: keeping everything secure and private while all these devices whisper secrets over the air.

Security and Privacy at Scale

Once your Bluetooth system starts working reliably, there’s another challenge waiting in the wings: keeping it secure. It’s one thing to get devices talking to each other, it’s another to make sure no one else is eavesdropping on the conversation. Bluetooth security can sound intimidating, but at its core, it’s about making sure your devices trust each other and that strangers can’t sneak into the chat.

Let’s start with pairing. Pairing is Bluetooth’s version of saying, “Hey, can I trust you?” It’s a handshake where two devices exchange keys that let them communicate securely in the future. There are a few ways this handshake can happen. The simplest is called Just Works, which basically means, “We’ll trust each other without asking too many questions.” It’s convenient but about as safe as leaving your front door unlocked because you live in a nice neighborhood. For harmless gadgets like wireless speakers, that’s fine. But for medical devices or smart locks, “Just Works” can turn into “Just Got Hacked.”

A safer approach is Passkey Entry, where one device shows a code and the other types it in, proving they’re physically near each other. Even better is Out-of-Band (OOB) pairing, where the devices exchange security information through another method, maybe a QR code, NFC tap, or even an optical blink, before connecting over Bluetooth. OOB pairing is like verifying someone’s identity face-to-face before continuing a conversation online.

Once paired, devices use encryption to scramble their communication. Anyone listening nearby will hear only gibberish. The strength of that encryption depends on the version of Bluetooth being used. Modern devices using Bluetooth 4.2 or later support something called LE Secure Connections, which is based on advanced cryptography. Older devices use weaker methods that are easier to crack. So, if you’re building something new, never rely on outdated pairing modes.

But security isn’t just about encryption. It’s also about privacy. Every Bluetooth device has an address, kind of like its phone number, that it uses when broadcasting. If that address stays the same, someone could track you by following your device’s broadcasts. That’s why newer standards support random address rotation, where devices periodically change their Bluetooth address. Your phone and smartwatch still recognize each other, but strangers can’t follow your signal around the city.

When you scale Bluetooth systems, these little details become critical. A single insecure device in your network can become the weak link that compromises everything. It’s like locking every door in your house but leaving one window open. Attackers don’t need to break the whole system, they just need to find the lazy one.

Building security into a large Bluetooth deployment means standardizing your pairing process, using strong encryption everywhere, and handling key storage carefully. On embedded devices, that can be tricky because they have limited memory and no secure element by default. Still, even small steps help, like regenerating keys periodically and disabling “Just Works” mode for devices that control anything important.

On mobile platforms, the rules are slightly different. Android and iOS handle much of the heavy lifting for you, but you still have to design your app logic carefully. Always confirm which device you’re connecting to before exchanging sensitive data. Always check bonding state before sending configuration commands. In short, treat Bluetooth communication with the same seriousness you’d give to a login session or an online payment.

At scale, security isn’t something you bolt on later. It’s part of the system’s DNA. You can’t fix a weak handshake by adding a stronger password later. You have to start from the first pairing and make sure every connection trusts the right partner.

The reward is worth it. When done right, your Bluetooth network becomes invisible but secure, a quiet, encrypted web of trust that just works. No drama, no leaks, and no nearby strangers hijacking your sensors.

In the next section, we’ll talk about another invisible problem that decides whether your Bluetooth network lives for days or months: power. Because what good is a secure device if its battery dies halfway through the handshake?

Power and Performance Tuning

If you’ve ever wondered why your Bluetooth gadget dies right when you need it most, you’ve just met the oldest enemy in wireless communication: power consumption. Bluetooth may be clever, flexible, and everywhere, but it also has a bit of a caffeine problem. It loves to talk, and talking burns energy. Keeping your devices alive longer, especially when you scale, means learning the quiet art of power management.

At first, it’s easy to assume that Bluetooth is low power by default. After all, it’s called Bluetooth Low Energy, right? But BLE’s efficiency only shines when it’s used correctly. A poorly tuned BLE system can drain a battery faster than streaming music over Classic Bluetooth. The magic lies in controlling when devices talk, how long they talk, and how much they say each time.

Let’s start with the advertising interval. This is how often a device shouts, “I’m here!” into the air. If you set it to broadcast every 20 milliseconds, you’ll discover devices quickly, but you’ll also burn through the battery like it’s running a marathon. Increase the interval to once every second, and your device will last much longer, but phones may take a moment to find it. It’s a tradeoff between speed and stamina. Every system has to find its sweet spot.

Next comes the connection interval, how often two connected devices exchange data. This is like deciding how frequently you check your messages. If you check every second, you stay perfectly up to date but never get anything else done. If you check once every minute, you save time but risk missing something important. In Bluetooth terms, a shorter connection interval means faster communication but higher power usage. Longer intervals conserve battery but add delay. Smart systems adjust these intervals dynamically depending on what the device is doing.

Then there’s the MTU, or Maximum Transmission Unit, the size of each Bluetooth data packet. Bigger packets mean fewer total transmissions for large chunks of data, which can improve efficiency. But some devices, especially older ones, can’t handle large MTUs, so finding the right balance is important.

Power management is not just about numbers, it’s about habits. A well-designed embedded device spends most of its life asleep. It wakes up only to advertise or exchange data, then returns to rest as quickly as possible. Imagine a hummingbird darting out for a sip of nectar and then zipping back to rest before anyone notices. That’s how efficient Bluetooth devices survive on coin-cell batteries for months or even years.

On the phone side, energy management is just as critical, especially when your app needs to handle multiple connections. Constant scanning, reconnecting, or keeping GATT channels open drains your user’s battery, and patience. Android and iOS both have built-in mechanisms that throttle background Bluetooth activity to save power. Developers have to work with these rules, not against them. The best apps schedule scans intelligently, reconnect only when necessary, and avoid holding connections open when no data needs to be sent.

Scaling Bluetooth systems makes these power decisions even more important. When you have one device, wasting a bit of energy doesn’t matter. When you have hundreds of devices, each one burning just a few extra milliwatts, the total waste adds up quickly. Power efficiency becomes the difference between a network that runs for months and one that collapses after a week.

The golden rule of power tuning is simple: talk less, talk smarter. A Bluetooth device that knows when to speak and when to stay quiet can scale beautifully, even in large networks. It’s not about being fast all the time, it’s about being clever with timing.

In the next section, we’ll look at how these devices join your network in the first place and what happens when you need to update their software later. Because once your system scales, you’re not just connecting devices, you’re managing an entire population.

Provisioning and Firmware Updates — Welcome to Device Kindergarten

Imagine setting up one Bluetooth device. It’s easy: you pair it, give it a name, and maybe tweak a few settings. Now imagine doing that a hundred times. Or a thousand. Suddenly, what felt like a simple task starts to look like a factory assembly line powered by frustration. That’s where provisioning comes in, the process of onboarding new devices into your Bluetooth network so they can start working right away, without manual babysitting.

Provisioning is like a first day at school for your devices. Each new student needs to be identified, assigned to a class, and given a name tag. In the Bluetooth world, a newly manufactured device begins life in an “unprovisioned” state. It doesn’t belong to any network yet, so it advertises with a special signal that says, “Hey, I’m new here.” When your mobile app or gateway spots that advertisement, it can connect, authenticate the device, and hand over the credentials it needs to join the system.

The app usually performs a few key steps during provisioning. It verifies that the device is genuine, assigns it a unique identifier, and exchanges security keys so future connections can happen securely. It might also store metadata like which room the sensor belongs to or what type of data it will report. After provisioning, the device switches to its normal operation mode, where it advertises with its new identity and starts behaving like a member of the family.

When you have just one or two devices, you can do all this manually. But when you scale up to hundreds or thousands, manual setup becomes impossible. That’s when you start thinking about automation, QR codes on packaging, NFC tags for instant pairing, or out-of-band provisioning where a separate channel (like Wi-Fi or a wired link) handles secure onboarding. The goal is to make provisioning quick, repeatable, and error-free, even when your factory or users are adding new devices by the dozens.

Once your devices are out in the world, the next challenge appears: firmware updates. Every system eventually needs to fix bugs, patch security holes, or add new features. For Bluetooth devices, this means pushing new firmware over the same wireless link, a process known as FOTA, or firmware-over-the-air updates.

Updating firmware over Bluetooth can be nerve-wracking. The connection is relatively slow, and interruptions can leave a device half-updated and confused about who it is. Good update systems handle this carefully. They divide the firmware into chunks, verify each piece with checksums, and only switch to the new version once the whole update has been safely received and validated. If anything fails midway, the device rolls back to the old firmware instead of bricking itself.

Scaling makes this even more complex. Updating ten devices is fine. Updating a thousand can overwhelm your network if you try to do them all at once. Smart systems stagger the updates in waves, track which devices have finished, and retry the ones that didn’t. Some even let devices report their status back to a central dashboard, so you can see which ones are ready and which ones are still stuck halfway through.

Provisioning and firmware updates might not sound glamorous, but they’re the backbone of every scalable Bluetooth system. Without smooth onboarding and reliable updates, your network slowly falls apart as devices drift out of sync or miss critical fixes.

Think of it this way: provisioning is how devices join the family, and firmware updates are how they grow up. Both are essential if you want your Bluetooth ecosystem to stay healthy and dependable over time.

In the next section, we’ll talk about what happens when something inevitably goes wrong, how to debug and monitor a network full of devices without losing your mind.

Debugging, Monitoring, and Testing Across Platforms

At some point, every Bluetooth developer faces the same moment of quiet despair. The logs look fine, the devices are paired, the code hasn’t changed, and yet… nothing works. Connections fail, packets vanish, and everything that worked yesterday now refuses to cooperate. Welcome to the wonderful, mysterious world of Bluetooth debugging, a place where logic takes a vacation and patience becomes your most valuable skill.

Debugging Bluetooth is tricky because so much of it happens invisibly. The data is flying through the air, hopping between frequencies dozens of times per second, and all you can see is whether the connection succeeds or fails. It’s like trying to diagnose a conversation between two people whispering in another room. You can tell they’re talking, but not what they’re saying.

The first rule of Bluetooth debugging is simple: log everything. Log when you start scanning, when you find a device, when you connect, and when you disconnect. Log the signal strength, the UUIDs you discover, the number of bytes you read, and the time it took. Bluetooth problems rarely announce themselves loudly, they hide in tiny details. A small delay in a callback or a missing acknowledgment can reveal exactly why your system seems haunted.

Different platforms give you different kinds of help. Android, for example, offers detailed Bluetooth logs through developer options or tools like adb. You can capture the raw Bluetooth HCI logs and analyze them later to see what really happened under the hood. iOS, on the other hand, gives you less direct visibility. Apple handles most of the Bluetooth stack internally, so your only clues come from Core Bluetooth callbacks. Embedded devices often let you log directly from the firmware, showing connection events, error codes, and sometimes even packet-level information if the stack supports it.

Testing across platforms is just as important as debugging. You can’t assume that if it works on one phone, it will work on another. Android devices, especially, have a habit of interpreting Bluetooth timing slightly differently. A system that’s rock-solid on a Pixel may stutter on a Samsung or freeze on a low-cost tablet. The only cure is diversity, test on multiple brands, OS versions, and firmware builds until you’re confident the system behaves everywhere.

For embedded devices, testing is a different challenge. Because they often run continuously, you need long-term endurance tests to catch issues that only appear after hours or days of operation. You might discover that a connection fails only after 300 reconnections, or that a memory leak appears after a week of normal use. Building test rigs that automate these scenarios: connecting, disconnecting, and verifying data repeatedly, is a huge time saver.

Monitoring is what happens after you’ve deployed your devices into the real world. It’s like keeping a health tracker on your entire Bluetooth network. Your mobile apps or gateways can collect statistics such as signal strength, connection failures, uptime, and battery levels. That data tells you which devices are performing well and which ones might be drifting toward trouble.

Adding this kind of visibility pays off enormously at scale. When you’re managing hundreds of devices, it’s impossible to check each one manually. Instead, you rely on trends, for example, if one location shows consistently weak signal strength, maybe there’s interference nearby. If multiple devices drop connections at the same time, maybe the central device needs a firmware update. Monitoring transforms guesswork into insight.

The truth is, debugging and monitoring never really end. Even after your system is stable, new versions of Android and iOS will appear with small Bluetooth changes that break something you didn’t know could break. Treat Bluetooth maintenance like car maintenance: routine, ongoing, and essential.

Once you learn to capture good logs, read them calmly, and build systems that report their own health, debugging stops being a nightmare and becomes a science. Bluetooth may always be a little mysterious, but with the right tools and attitude, you can keep the ghosts out of your connection list.

In the next section, we’ll put everything together with a real-world example of what scaling Bluetooth actually looks like when all the pieces: mobile apps, embedded devices, and architecture, finally work in harmony.

Real-World Architecture Example — When Bluetooth Finally Behaves

Let’s take everything we’ve talked about and bring it to life with a real-world scenario. Imagine you’re building a smart factory system with hundreds of Bluetooth sensors scattered across the floor. Each sensor measures temperature, vibration, or humidity. Some are attached to machines, others hang on walls, and a few are hidden in places even the janitor doesn’t know about. Your goal is simple on paper: collect data from all these sensors, send it to a central dashboard, and keep everything running smoothly.

The reality, of course, is much more complicated. Each sensor is an embedded device powered by a coin-cell battery that has to last for months. They advertise periodically to announce they’re alive. Your Android or iOS tablets, placed around the factory as gateways, act as Bluetooth centrals. Their job is to scan, connect to nearby sensors, read data, and upload it to the cloud. It sounds straightforward, but you’re juggling dozens of invisible connections at once, and they all have different moods.

The architecture begins with careful planning. Each gateway tablet knows which part of the factory it’s responsible for. That way, you avoid overcrowding the airwaves with multiple devices trying to connect to the same sensors. The sensors use slightly staggered advertising intervals so they don’t all shout at the same time. The gateways maintain a queue, connecting to a few sensors at a time, reading data, and then disconnecting before moving on to the next group. This rotation keeps everything balanced and prevents Bluetooth traffic jams.

Power management is built into every step. Each sensor wakes up, advertises briefly, sends its data when connected, and goes right back to sleep. The connection interval and MTU size are tuned for efficiency, large enough for smooth data transfer, but not so large that slower devices choke. Every byte is treated like gold because every transmission costs energy.

The gateways handle the messy parts: reconnections, retries, and data aggregation. They buffer readings in case the Wi-Fi link to the cloud goes down and sync later when it’s back. They also monitor each sensor’s signal strength, battery level, and uptime. If a sensor hasn’t reported in a while, the system flags it automatically so a technician can check on it.

Now imagine scaling this setup to multiple factory buildings. Suddenly, you’re managing thousands of sensors, dozens of gateways, and countless wireless interactions. At this scale, the design choices you made early, abstracted Bluetooth logic, retry mechanisms, power optimization, and logging, are the difference between a quiet, self-running network and a system that collapses into constant reconnections.

When everything works as intended, something beautiful happens. The sensors collect data silently. The gateways synchronize automatically. The dashboards stay green. Nobody has to restart anything, and Bluetooth quietly fades into the background where it belongs. It’s the rare moment when technology stops demanding attention and simply does its job.

This kind of architecture isn’t science fiction. Companies use it in factories, hospitals, and warehouses every day. From smart lighting systems to patient monitors, Bluetooth at scale can be astonishingly reliable, but only if you treat it like a distributed system, not a single gadget. Each device is a citizen of a larger ecosystem, and your job as the architect is to keep that ecosystem healthy.

The biggest takeaway is that success doesn’t come from fancy algorithms or expensive hardware. It comes from the small, deliberate decisions that make your system resilient: how you handle disconnections, how you schedule connections, how you monitor performance. Scaling Bluetooth is not about avoiding problems, it’s about designing a system that recovers gracefully when problems happen.

In the next section, we’ll wrap up everything we’ve learned into a practical checklist, a simple guide you can use whenever you’re designing a Bluetooth system that has to survive in the wild.

Checklist — Building a Truly Scalable Bluetooth System

By now, you’ve seen Bluetooth in all its moods, charming, confusing, unpredictable, and surprisingly capable when handled with care. So how do you actually put everything together? What makes a Bluetooth system scalable instead of just “working on my desk”? The answer isn’t a single trick or secret API. It’s a mindset, a way of designing your system to expect chaos and still function gracefully when it happens.

The first part of that mindset is consistency. Every Bluetooth system should have one clear and stable way of communicating. Keep your data formats simple, your GATT profiles predictable, and your naming conventions sensible. If you have ten devices made by ten different vendors, make them all speak the same language. The moment one device starts improvising, the whole orchestra sounds off.

Next comes patience, and in Bluetooth, patience means retries. Connections drop. Devices go out of range. A phone might go to sleep or decide that scanning is no longer fashionable. Instead of treating every disconnection as a crisis, treat it as part of the process. A good Bluetooth app quietly retries in the background, restores the connection, and carries on as if nothing happened. To the user, it feels seamless. Underneath, it’s a flurry of logic keeping the experience smooth.

Then there’s the question of power. Remember that every advertisement and connection eats into battery life. A scalable Bluetooth system doesn’t talk all the time, it talks smart. It plans when to wake up, when to exchange data, and when to stay silent. Devices that last longer need fewer replacements, fewer updates, and far less human attention. Power efficiency is the hidden currency of scalability.

Monitoring is another essential habit. If you can’t see what’s happening inside your system, you’re flying blind. Log your connections, track your signal strengths, record how often devices drop out, and visualize it somewhere. A simple dashboard that shows which devices are healthy and which ones are struggling can save you countless hours later. When you scale, visibility turns guesswork into control.

Security, too, can’t be an afterthought. Use secure pairing, proper encryption, and rotating addresses. The bigger your system gets, the more interesting it becomes to people who might want to peek at it. Make sure they can’t. A secure Bluetooth network doesn’t just protect users, it protects your reputation.

Finally, build for change. Bluetooth isn’t static, Android and iOS update their stacks every year, chip vendors release new firmware, and new security standards appear. A scalable system doesn’t break when something changes, it adapts. That’s why abstraction layers, modular code, and updatable firmware matter so much. They keep your system flexible long after the first version ships.

If you do all of this, keep it consistent, patient, efficient, observable, secure, and adaptable, something magical happens. Your Bluetooth system starts to feel less like a fragile web of devices and more like a living network. It keeps running, keeps healing, and quietly gets the job done without constant supervision. That’s when you know you’ve built something that scales.

In the final section, we’ll step back and reflect on the bigger picture, what scaling Bluetooth really teaches us about building technology that has to work not just once, but over and over again in the messy, beautiful real world.

Wrap-Up — Lessons from the Field

If you’ve made it this far, you’ve probably realized that scaling Bluetooth isn’t really about Bluetooth at all. It’s about learning how complex systems behave when they leave the comfort of your desk and enter the real world. It’s about understanding that wireless connections are not just electrical signals, they’re relationships between unpredictable, battery-powered, opinionated little machines.

Bluetooth gets a bad reputation because people expect it to be simple. They imagine it’s like Wi-Fi or USB, plug and play, pair and forget. But in truth, Bluetooth is more like a polite conversation at a crowded party. Everyone is talking at the same time, the music is loud, and you have to keep repeating yourself until the other person hears you correctly. When you think of it that way, it’s a miracle that it works as well as it does.

Scaling Bluetooth across Android, iOS, and embedded devices teaches you humility. You stop assuming things will always behave, and instead you start building systems that recover when they don’t. You learn that error handling is not an afterthought, it’s the main event. You discover that batteries are precious, timing is everything, and the smallest design decisions can ripple through an entire ecosystem of devices.

You also start to appreciate the quiet beauty of resilience. There’s something deeply satisfying about watching dozens of sensors, gateways, and phones connect, share data, and disconnect, all without human intervention. When it works, it feels effortless. You forget about the retries, the power cycles, the reconnections, and the debugging sessions that made it possible. All you see is a smooth network humming quietly in the background, doing exactly what it was meant to do.

And that’s the real magic of Bluetooth, not the flashy tech demos or the pairing animations, but the invisible collaboration that happens beneath the surface. It’s the heartbeat of every wearable, every sensor, every tiny device that quietly makes our lives a little easier. Scaling it isn’t just an engineering challenge; it’s a lesson in patience, design, and empathy for systems that can’t always speak for themselves.

So, the next time your Bluetooth device disconnects, take a breath. Somewhere in the chaos, it’s just trying to reconnect, to find its partner again and pick up where it left off. Because deep down, that’s what Bluetooth really is: a network built on trust, persistence, and tiny packets of hope flying through the air.

With the release of Swift 6 and its new Embedded Swift compilation mode, developers now have access to a modern, memory-safe, and performant alternative that’s tailored specifically for resource-constrained systems.

While languages like Rust have also emerged to address these issues, Embedded Swift brings the clarity and safety of Swift to microcontroller environments, without giving up on determinism, binary size, or hardware access.

This article introduces Embedded Swift and explores how it compares to traditional C/C++ development. We’ll cover its key features, programming and memory models, how to set up the toolchain for STM32 microcontrollers, and how to link Swift with existing C drivers.

Along the way, we’ll examine performance trade-offs, growing ecosystem support, and the broader industry movement toward memory-safe languages. As I hope you’ll see, Swift is a serious contender in the future of embedded development.

Prerequisites

To get the most out of this article, you should have a basic understanding of programming in Swift and C. Familiarity with embedded hardware platforms and firmware development concepts will also be helpful.

If you're new to embedded systems, consider reviewing this introductory guide to embedded firmware to build foundational knowledge before diving into Embedded Swift.

Scope

This article is intended as a practical introduction to Embedded Swift. It covers:

• An overview of Embedded Swift and its key language features

• Swift’s programming and memory model in an embedded context

• Setting up the Embedded Swift toolchain on macOS for STM32 microcontrollers

• Interoperability with C code and linking to existing low-level drivers

• A look at memory and instruction-level performance

• Future directions and use cases for Embedded Swift

Note that this article does not provide a full tutorial on the Swift language itself. While the primary focus is on STM32, similar principles apply to other supported platforms such as ESP32, Raspberry Pi Pico, and nRF52.

Table of Contents:

• What is Swift? What is Embedded Swift?

• Swift Programming Model

• Swift Memory Management

• Memory and Instruction Cycle Comparison

• How to Setup Embedded Swift

• C-Swift Linkages:

• Future Work

• Conclusion

What is Swift? What is Embedded Swift?

Swift is a modern programming language developed by Apple that combines the performance of compiled languages with the expressiveness and safety of modern language design. While Swift was originally created for iOS and macOS development, it has evolved into a powerful general-purpose language used in server-side development, systems programming, and increasingly, embedded systems.

Embedded Swift is a special compilation mode introduced in Swift 6 that brings the benefits of Swift to resource-constrained platforms like microcontrollers. It lets developers use a safe, high-level language while still producing compact, deterministic, and performant binaries suitable for embedded applications.

Key Features of Swift

Embedded Swift retains many of the powerful language features that make Swift an attractive alternative to C/C++ in embedded development:

Type Safety: Swift uses a strong static type system, which prevents many programming errors at compile time. Unlike C, where type mismatches can result in undefined behavior, Swift ensures all types are used correctly before code even runs.

Strict Type Checking: Swift doesn't allow implicit type conversions that could lose data or cause unexpected behavior. For example:

// This won't compile in Swift
let integer: Int = 42
let decimal: Double = 3.14
let result = integer + decimal  // Error: Cannot convert value of type 'Int' to expected argument type 'Double'

// You must be explicit about conversions
let result = Double(integer) + decimal  // Correct

Non-nullable Types by Default: In C, pointers can be null by default, which introduces risk. In Swift, variables cannot be nil unless explicitly marked as optionals:

var name: String = "John"
name = nil  // Compile error - String cannot be nil

var optionalName: String? = "John"
optionalName = nil  // This is allowed

Memory Safety via ARC (Covered in detail later):

Swift manages memory automatically using Automatic Reference Counting (ARC). Unlike manual memory management in C/C++, ARC handles object lifecycles efficiently without unpredictable garbage collection pauses. We'll cover ARC and its impact in embedded contexts in a dedicated section later.

Modern Syntax:
Swift's syntax is clean, consistent, and designed for readability. It supports modern paradigms including:

• Functional programming (map, filter, reduce)

• Generics (type-safe abstractions)

• Protocol-Oriented Programming (discussed in the next section)

These features allow you to write more expressive and maintainable code compared to procedural C or inheritance-heavy C++.

Performance:
Swift is designed to perform on par with C++ in many scenarios. Optimizations such as inlining, dead code elimination, and static dispatch help ensure that high-level abstractions don’t compromise performance. In embedded mode, Swift disables features like runtime reflection and dynamic dispatch to further reduce overhead.

To fully leverage Swift for embedded development, it's important to understand its programming model. Unlike C’s procedural approach or C++’s class-heavy design, Swift promotes protocol-oriented programming and composition, which offers both flexibility and safety in embedded system design.

Swift Programming Model

Swift embraces a multi-paradigm programming model that blends object-oriented, functional, and protocol-oriented programming, all underpinned by strong type safety and memory safety.

For embedded developers coming from C or C++, this model may feel different at first. But it provides a more modular and testable way to build complex systems, something especially valuable in embedded applications where hardware abstraction and strict reliability are critical.

Protocol-Oriented Programming (POP)

Swift emphasizes protocols over inheritance, encouraging developers to define behaviors through protocols and implement them using value types like struct and enum, rather than relying heavily on classes.

This philosophy favors composition over inheritance, allowing you to build complex functionality by combining smaller, well-defined components.

Key Concepts:

• protocol defines required behavior.

• Protocol extensions provide default behavior.

• Prefer value semantics using struct.

Example:

protocol Speakable {
    func speak()
}

extension Speakable {
    func speak() {
        print("Default sound")
    }
}

struct Dog: Speakable {
    func speak() {
        print("Woof!")
    }
}

Embedded Swift uses protocols with static dispatch. With static dispatch, the compiler knows the exact memory address of the function to call and can generate a direct jump instruction. There's no runtime lookup, no indirection, and no uncertainty.

Why POP Matters for Embedded Systems

First, you get flexible hardware extraction. Protocols make it easy to define interfaces for hardware components, allowing for mock implementations during testing or platform-specific variations.

Second, you have nice low overhead. Embedded Swift uses static dispatch for protocols, meaning there’s no runtime lookup, and calls are resolved at compile time for maximum performance.

Also, struct and enum types avoid heap allocations, making code more efficient and predictable in low-memory environments.

Now that we’ve explored how Swift’s programming model enables safer and more modular embedded code, let’s turn to another critical piece of the puzzle: memory management. Swift’s use of Automatic Reference Counting (ARC) replaces manual memory handling and offers important benefits, and tradeoffs, for embedded systems.

Swift Memory Management

One of Swift’s most impactful features, especially in the context of embedded systems, is its use of Automatic Reference Counting (ARC) for memory management. Unlike C/C++, where memory must be manually allocated and freed using malloc and free, Swift automates this process while maintaining deterministic performance.

This automation significantly reduces the risk of common memory-related bugs like leaks, dangling pointers, or use-after-free errors, all of which are notorious in low-level C code.

How ARC works

Swift supports ARC not only for the Cocoa Touch API's but for all APIs, providing a streamlined approach to memory management. Unlike garbage collection systems that can cause unpredictable pauses, ARC works deterministically at compile time and runtime to manage memory.

ARC automatically tracks and manages the lifetime of objects in memory based on how many references point to them.

• Reference Counting: Every object has a counter that tracks how many strong references point to it.

• Retain / Release: The compiler inserts retain and release calls automatically during assignment and deinitialization.

• Immediate Deallocation: When the reference count reaches zero, the object is deallocated immediately.

• Deterministic: Unlike garbage collectors, ARC doesn’t introduce unpredictable pauses or runtime scanning.

Swift offers multiple reference types to give you precise control over memory behavior and prevent cycles:

Strong References (default)

• Keeps the referenced object alive.

• Used in most cases.

class MotorController {
    var sensor: SensorData?  // Strong reference

    func updateReading(newData: SensorData) {
        self.sensor = newData  // Previous sensor data automatically deallocated
    }
}

Weak References

• Used to break reference cycles (especially in two-way object relationships).

• Automatically becomes nil when the referenced object is deallocated.

class Device {
    var controller: MotorController?

    deinit {
        print("Device deallocated")
    }
}

class MotorController {
    weak var device: Device?  // ← Weak reference breaks the cycle

    deinit {
        print("MotorController deallocated")
    }
}

func breakCycle() {
    let device = Device()
    let controller = MotorController()

    device.controller = controller
    controller.device = device  // ← This is now a weak reference

    // When this function ends, both objects are properly deallocated
}

breakCycle()
// Output:
// Device deallocated
// MotorController deallocated

Unowned References

• Non-optional version of weak.

• Assumes the object will never be deallocated while still in use.

• More lightweight than weak, but unsafe if misused.

class SensorSystem {
    unowned let controller: MotorController  // unowned reference

    init(controller: MotorController) {
        self.controller = controller
    }
}

class MotorController {
    var sensorSystem: SensorSystem?

    func setupSensors() {
        sensorSystem = SensorSystem(controller: self)
    }

    deinit {
        print("MotorController deallocated")
    }
}

func testUnowned() {
    let controller = MotorController()
    controller.setupSensors()
    // sensorSystem deallocates before controller ends
}

testUnowned()
// Output: MotorController deallocated

ARC Overhead in Embedded Systems

While ARC provides safety benefits, it does introduce some overhead compared to manual memory management:

Memory Overhead:

ARC-managed class instances in Swift typically include an additional 4 or 8 bytes to store reference count metadata, depending on the system architecture, 4 bytes on 32-bit systems and 8 bytes on 64-bit systems. This metadata allows the runtime to track how many active references exist to a given object and deallocate it when no references remain. When developers use weak or unowned references, the memory footprint increases further. These references require additional data structures, such as side tables or tracking mechanisms, to manage object liveness and cleanup. In the case of weak references specifically, Swift maintains zeroing weak reference tables that automatically null out pointers once the referenced object is deallocated, ensuring memory safety.

CPU Overhead:

ARC introduces some runtime overhead due to retain and release operations, which are inserted automatically during reference assignments. These operations involve incrementing or decrementing the reference count and are especially common in code that passes objects between functions or stores them in collections. To ensure thread safety, these updates are typically implemented using atomic operations, which add further instruction cycles. In complex object graphs, ARC may also engage in cycle detection and cleanup through the use of weak references to prevent memory leaks caused by strong reference cycles. While Swift's ARC provides deterministic and efficient memory management, it does so with both memory and CPU costs that developers should consider carefully, especially in performance-critical embedded systems.

Type Safety and Error Prevention

Swift's type system prevents many common errors that plague C/C++ programs:

• Buffer Overflows: Swift arrays are bounds-checked, preventing buffer overflow vulnerabilities that are common in C.

• Null Pointer Dereferences: Swift's optional types make null pointer dereferences impossible at compile time.

• Use After Free: Swift's ownership model prevents use-after-free errors that can cause crashes or security vulnerabilities.

Now that we’ve covered Swift's memory model and ARC behavior, let’s explore how it compares to C in terms of memory usage and instruction cycles, a crucial aspect when evaluating Embedded Swift for real-world deployment.

Memory and Instruction Cycle Comparison

Understanding the performance characteristics of Swift versus C is essential for embedded systems, where every instruction cycle and byte of memory matters. While Swift brings advantages like safety and expressiveness, these benefits come with certain trade-offs in terms of memory usage and runtime behavior that embedded developers must evaluate carefully.

Memory Management:

Swift uses Automatic Reference Counting (ARC) to manage memory. ARC tracks the number of references to each object and deallocates it when no references remain. This eliminates the need for explicit free() calls but introduces overhead.

C, in contrast, uses manual memory management. Developers allocate memory using malloc and release it using free, or rely on the stack for most short-lived data.

The table below provides the memory management comparison between Swift and C:

Swift ensures safety through deterministic cleanup and predictable memory usage. But this comes at the cost of added memory and CPU overhead.

C’s approach offers complete control over memory layout and minimal runtime cost, but increases the risk of memory leaks and fragmentation without disciplined practices.

Instruction Cycle Analysis

The safety features in Swift, such as bounds checking, optional unwrapping, and ARC updates, translate into additional CPU instructions. While this can impact performance, the Swift compiler is aggressive about optimization in release builds. For example, inlining and ARC elision can remove much of the overhead in performance-critical paths.

C has no built-in safety checks, allowing it to generate highly efficient, predictable code. Developers can even use inline assembly for tight control over performance.

The table below provides the instruction cycle comparison between Swift and C:

Although Swift inserts extra instructions for safety, much of this cost can be mitigated through compiler optimization.

C has no such features by default, making it ideal for applications where performance must be tightly controlled and the developer is willing to take full responsibility for safety.

Instruction Count Comparison: Swift vs C Loop Performance

When evaluating Swift and C for embedded use, it's helpful to analyze instruction-level performance on basic operations, such as a loop that processes an array of floating-point numbers. This gives us a concrete sense of the computational cost of each language's safety and abstraction features.

Let’s consider a simple example: summing an array of Float values and returning the average. In Swift, the code uses a high-level for-in loop over an array:

Simple loop performance:

// Swift loop with safety checks
func processData(_ data: [Float]) -> Float {
    var sum: Float = 0.0
    for value in data {  // Iterator with bounds checking
        sum += value     // Safe arithmetic
    }
    return sum / Float(data.count)  // Safe division
}
// Estimated: ~8-10 instructions per iteration

Although elegant and safe, this loop includes several safety mechanisms:

1. Bounds checking on every array access

2. Reference counting if data is passed as a reference type

3. Overflow protection in debug mode

4. Optional handling or runtime checks if data might be empty

These checks introduce runtime overhead, resulting in an estimated 8–10 instructions per iteration on most platforms (depending on optimization level and target architecture). In release builds, Swift aggressively inlines and strips redundant checks, but some level of abstraction cost remains, especially compared to raw memory access in C.

Now, compare that to its equivalent in C:

// C loop without safety checks
float process_data(float* data, int count) {
    float sum = 0.0f;
    for (int i = 0; i < count; i++) {  // Direct pointer arithmetic
        sum += data[i];                // Direct memory access
    }
    return sum / count;  // Direct division (no safety check)
}
// Estimated: ~4-5 instructions per iteration

This version performs direct memory access with pointer arithmetic, no bounds checks, and no type safety. The C code is lower-level, with fewer runtime checks, and compiles down to just 4–5 instructions per iteration, depending on the target CPU and compiler flags. It is lean and fast, ideal for cycles-per-instruction-critical scenarios.

The table below shows the comparison of single loop performance between Swift and C:

Now that we’ve compared Swift and C in terms of memory and cycle costs, let’s move into the practical side: how to set up Embedded Swift on an STM32 platform and get started with real-world development.

How to Setup Embedded Swift

In this section, we'll walk through how to configure and use Embedded Swift for development on STM32 microcontrollers. STM32 is a popular family of ARM Cortex-M–based microcontrollers, commonly used in industrial, consumer, and IoT applications.

Prerequisites

Required Software:

• Swift Development Snapshot (includes the Embedded Swift toolchain)

• Swiftly - Easiest way to manage and install swift toolchains

• Swiftc - Swift Compiler command-line tool

• Python3 - Required to run scripts to convert Mach-O to binary files

• Git (to clone sample repositories) like https://github.com/swiftlang/swift-embedded-examples

• A Unix-like development environment (macOS is currently best supported)

Target Hardware: This guide focuses on STM32 microcontrollers, which are widely used in embedded applications and have excellent community support.

This guide walks you through the full setup process, from installing the required Swift toolchain to flashing the final binary onto your board. We’ll begin by installing the Swift Development Snapshot using Swiftly, a simple command-line utility for managing Swift toolchains. From there, we’ll configure the build system, set up the correct board variant, customize the build script, and compile the Swift and C source code into a binary. Finally, we’ll flash the firmware onto the STM32 using standard tools

Install Swift Development Snapshot

The easiest way to install and manage Embedded Swift toolchains is by using the swiftly tool, which simplifies downloading and using Swift snapshots.

macOS Installation:

The below steps will help install the Swift embedded toolchain:

# Using Swiftly (Recommended)
curl -O https://download.swift.org/swiftly/darwin/swiftly.pkg
installer -pkg swiftly.pkg -target CurrentUserHomeDirectory
~/.swiftly/bin/swiftly init --quiet-shell-followup
source "${SWIFTLY_HOME_DIR:-$HOME/.swiftly}/env.sh"

# Install and use development snapshot
swiftly install main-snapshot
swiftly use main-snapshot

# Verify installation
swift --version

You can clone this Github example repository:

git clone https://github.com/swiftlang/swift-embedded-examples.git 
cd swift-embedded-examples/projects/stm32-blink

The stm32-blink contains:

• Swift code that toggles GPIOs

• A C startup file with vector table

• A build.sh script that uses swiftc, clang, and a custom linker setup

Setup the STM32 Board

Tell the build script which STM32 board is being used:

export STM_BOARD=STM32F746G_DISCOVERY

You can add your own board variant by defining the appropriate memory map and compiler flags in the script.

Modify build.sh (Optional)

Ensure the script correctly locates the following:

• swiftc: should point to the toolchain you installed with Swiftly

• clang: can be macOS’s default Clang

• libBuiltin.a, crt0.s, and macho2bin.py: used to provide minimal runtime support and convert output to flashable binaries

If needed, update these paths:

SWIFT_EXEC=${SWIFT_EXEC:-$(swiftly which swiftc)}
CLANG_EXEC=${CLANG_EXEC:-$(xcrun -f clang)}
PYTHON_EXEC=${PYTHON_EXEC:-$(which python3)}

Ensure the linker flags match your target’s flash and RAM sizes.

Build and Flash the Project:

Run:

./build.sh

This compiles Swift and C code, links them, and produces a blink.bin file.

If successful, you’ll see:

.build/blink.bin  # ready to flash Step 6: Flash the Firmware to STM32

Use ST-Link tools or openocd to flash your board. Example using st-flash:

brew install stlink
st-flash write .build/blink.bin 0x8000000

You should now see an LED blinking.

Here’s a more detailed step by step approach to writing a bare metal code on STM32. For comprehensive installation guides covering other platforms (Raspberry Pi Pico, ESP32, nRF52), detailed IDE configuration, troubleshooting, and advanced examples, you can check out the official documentation:

• Complete Setup Guide: Install Embedded Swift

• Platform Examples: Swift Embedded Examples Repository

• Getting Started Tutorial: Embedded Swift on Microcontrollers

Now that we’ve set up Embedded Swift and explored how to build and run an example project, let’s look at a critical real-world scenario: interfacing Swift with low-level C drivers.

C-Swift Linkages

In many embedded projects, low-level hardware drivers are written in C because of its close-to-metal control and widespread ecosystem support. Embedded Swift supports seamless interoperability with C, which lets you reuse existing C libraries and drivers, write hardware control logic in C, and implement higher-level application logic in Swift.

This hybrid model lets you combine Swift’s safety and productivity with C’s hardware-level control, with no runtime overhead or object translation.

Let’s walk through an example where a low-level sensor driver is implemented in C and the application logic is written in Swift.

C Header File (sensor_driver.h):

This C header file defines the public interface for a low-level sensor driver. It includes standard fixed-width integer types and declares four functions:

• sensor_init(): Initializes the hardware sensor

• sensor_read_temperature() and sensor_read_humidity(): Read raw sensor values

• sensor_delay_ms(): Delays execution for a given number of milliseconds

This interface acts as a bridge between Swift and C. Swift will link to these functions by name, no wrappers or bindings required.

#ifndef SENSOR_DRIVER_H
#define SENSOR_DRIVER_H

#include <stdint.h>

// Low-level sensor driver functions
void sensor_init(void);
uint32_t sensor_read_temperature(void);
uint32_t sensor_read_humidity(void);
void sensor_delay_ms(uint32_t milliseconds);

#endif

C Implementation (sensor_driver.c):

This implementation assumes the sensor is memory-mapped at a fixed address (0x40001000). Each register, temperature, humidity, and control, is accessed by offset from that base address.

The sensor_init() function writes 0x01 to the control register, presumably enabling or starting the sensor hardware.

The sensor_read_temperature() method and sensor_read_humidity() method reads from memory-mapped registers and return the raw ADC values from the sensor.

The sensor_delay_ms() method performs a simple busy-wait loop using nop (no-operation) instructions to approximate a delay. This is suitable for short, coarse-grained delays in bare-metal contexts.

#include "sensor_driver.h"

// Hardware register addresses
#define SENSOR_BASE_ADDR    0x40001000
#define TEMP_REG_OFFSET     0x00
#define HUMIDITY_REG_OFFSET 0x04
#define CONTROL_REG_OFFSET  0x08

void sensor_init(void) {
    // Initialize sensor hardware
    volatile uint32_t* control_reg = (volatile uint32_t*)(SENSOR_BASE_ADDR + CONTROL_REG_OFFSET);
    *control_reg = 0x01; // Enable sensor
}

uint32_t sensor_read_temperature(void) {
    volatile uint32_t* temp_reg = (volatile uint32_t*)(SENSOR_BASE_ADDR + TEMP_REG_OFFSET);
    return *temp_reg;
}

uint32_t sensor_read_humidity(void) {
    volatile uint32_t* humidity_reg = (volatile uint32_t*)(SENSOR_BASE_ADDR + HUMIDITY_REG_OFFSET);
    return *humidity_reg;
}

void sensor_delay_ms(uint32_t milliseconds) {
    // Simple delay implementation
    for (uint32_t i = 0; i < milliseconds * 1000; i++) {
        __asm__("nop");
    }
}

Swift Code Using C Driver:

To use these C functions from Swift, you declare them using @_silgen_name, which tells the Swift compiler to link directly to these symbol names at runtime.

The SensorController class encapsulates sensor-related logic. In its init() method, it calls the sensor_init() function defined in C to initialize the sensor hardware.

The readSensors() method reads the raw values from the C driver, converts them into human-readable units using helper functions, stores them internally, and returns the processed values.

The convertTemperature() and convertHumidity() conversion methods apply a basic linear formula to turn raw ADC values into temperature in Celsius and humidity in percentage, respectively. These formulas would be based on the specific sensor’s datasheet.

The checkThresholds() method applies simple threshold logic, a good example of where Swift’s readability and type safety shine. You could easily expand this logic to include error bounds, state machines, or alerts.

// Import C driver functions

/*
These declarations match the C function signatures exactly. 
They allow Swift to invoke the C functions as if they were native Swift functions 
— with zero overhead.
*/
@_silgen_name("sensor_init")
func sensor_init()

@_silgen_name("sensor_read_temperature")
func sensor_read_temperature() -> UInt32

@_silgen_name("sensor_read_humidity")
func sensor_read_humidity() -> UInt32

@_silgen_name("sensor_delay_ms")
func sensor_delay_ms(_ ms: UInt32)

// Swift sensor controller using C driver
class SensorController {
    private var lastTemperature: Float = 0.0
    private var lastHumidity: Float = 0.0

    init() {
        // Initialize the C driver
        sensor_init()
    }

    func readSensors() -> (temperature: Float, humidity: Float) {
        // Read raw values from C driver
        let rawTemp = sensor_read_temperature()
        let rawHumidity = sensor_read_humidity()

        // Convert raw values to meaningful units in Swift
        let temperature = convertTemperature(rawValue: rawTemp)
        let humidity = convertHumidity(rawValue: rawHumidity)

        // Store for comparison
        lastTemperature = temperature
        lastHumidity = humidity

        return (temperature: temperature, humidity: humidity)
    }

    private func convertTemperature(rawValue: UInt32) -> Float {
        // Convert raw ADC value to Celsius
        return (Float(rawValue) * 3.3 / 4095.0 - 0.5) * 100.0
    }

    private func convertHumidity(rawValue: UInt32) -> Float {
        // Convert raw ADC value to percentage
        return Float(rawValue) * 100.0 / 4095.0
    }

    func checkThresholds() -> Bool {
        // Swift logic for threshold checking
        let tempThreshold: Float = 25.0
        let humidityThreshold: Float = 60.0

        return lastTemperature > tempThreshold || lastHumidity > humidityThreshold
    }
}

// Main application loop
func main() -> Never {
    let sensorController = SensorController()

    while true {
        // Read sensors using Swift controller with C driver
        let readings = sensorController.readSensors()

        // Process data with Swift's type safety and expressiveness
        if sensorController.checkThresholds() {
            print("Warning: Temperature: \(readings.temperature)°C, Humidity: \(readings.humidity)%")
        } else {
            print("Normal: Temperature: \(readings.temperature)°C, Humidity: \(readings.humidity)%")
        }

        // Delay using C driver function
        sensor_delay_ms(1000) // 1 second delay
    }
}

The func main() is the main event loop standard for embedded systems. It creates the sensor controller, reads sensor data in a loop, checks thresholds, and prints results accordingly. The loop includes a delay (via the C driver) to avoid hammering the sensor continuously.

In an actual embedded context, instead of using print(), you might blink an LED, send UART messages, or log data to memory.

With Embedded Swift and C now working together, let’s explore what lies ahead. The next section outlines ongoing improvements, emerging use cases, and research directions that are shaping the future of Embedded Swift.

Future Work

Embedded Swift is still a young but rapidly evolving technology. Its modern language features, type safety, and performance make it an attractive option for embedded development, and ongoing work is expanding its capabilities, reach, and ecosystem.

Ongoing Improvements

Compiler Optimizations: The Swift compiler team is actively improving code generation for embedded targets, including:

• Reducing binary size

• Minimizing ARC overhead

• Improving static dispatch performance

Hardware Support: Embedded Swift can target a wide variety of ARM and RISC-V microcontrollers, which are popular for building industrial applications. Support for additional architectures is being developed.

Tooling Enhancements: Tooling support for Embedded Swift is still evolving, but several community-driven and open-source efforts are making development more accessible:

• Build Systems: The Swift Embedded Working Group provides example projects that adapt Swift Package Manager (SwiftPM) for cross-compilation. Custom linker scripts and build helpers are available for platforms like STM32 and nRF52.

• Debugging Support: Developers can debug Embedded Swift programs using existing tools like GDB or OpenOCD, provided the build includes appropriate debug symbols. While not yet officially streamlined, this approach enables step-through debugging on real hardware.

• IDE Integration: There is no official IDE support yet, but some developers use VSCode with Swift syntax highlighting and external build tasks. These setups are still manual but serve as early prototypes for embedded workflows.

Emerging Use Cases

There are a number of emerging use cases for embedded Swift. For example, Swift’s memory safety, type guarantees, and protocol-oriented design make it ideal for secure and scalable IoT devices, especially where firmware bugs could affect user safety or privacy.

The automotive sector is also exploring Swift for infotainment systems, driver assistance features, and safety-critical logic (where deterministic execution and safety matter).

Swift’s expressive syntax and compile-time safety make it suitable for industrial automation – think real-time control loops, sensor fusion systems, and edge devices in smart manufacturing.

It’s also useful for medical devices, as it aligns well with strict medical regulations around memory safety, type guarantees, and predictable resource usage.

Community and Ecosystem

Open Source Projects

The Swift Embedded working group maintains example repositories showcasing how to use Embedded Swift on microcontrollers such as STM32, nRF52, and ESP32. Early-stage libraries for UART, GPIO, and basic peripherals are emerging, though the ecosystem is still young compared to C or Rust.

Learning Resources

While Embedded Swift is not yet widely taught in formal curricula, community tutorials and exploratory projects (for example, Swift for Arduino) are lowering the barrier for hobbyists and independent learners. As tooling matures, educational adoption is likely to follow.

Industry Interest

Embedded Swift is beginning to draw attention from developers and companies looking for safer, more maintainable alternatives to C. Although large-scale adoption remains limited, use cases like rapid prototyping, IoT development, and internal experimentation are gaining traction.

Conclusion

Embedded Swift represents a major step forward in embedded programming. By combining the power and safety of Swift with the low-level control needed for microcontrollers, it offers an exciting alternative to traditional C and C++ development.

While C will remain essential for hardware-level programming and performance-critical paths, Swift brings compelling advantages to many embedded scenarios:

• Memory safety: Swift eliminates entire categories of bugs such as buffer overflows, use-after-free, and null pointer dereferencing.

• Type safety: Many logic errors are caught at compile time, long before they can cause runtime failures.

• Modern language features: Developers can use functional paradigms, generics, and protocol-oriented design even in embedded code.

• C interoperability: Swift works seamlessly with existing C libraries, allowing gradual adoption without rewriting low-level drivers.

• Developer productivity: Clear syntax, automatic memory management, and strong tooling lead to faster development and easier maintenance.

Government and regulatory bodies are increasingly encouraging or mandating the use of memory-safe programming languages to reduce vulnerabilities in critical software systems. For example:

• In 2022, the U.S. National Security Agency (NSA) recommended moving away from unsafe languages like C/C++ for new software projects, promoting memory-safe alternatives.

• In June 2025, the NSA and CISA released a joint Cybersecurity Information Sheet titled “Memory Safe Languages: Reducing Vulnerabilities in Modern Software Development”, which emphasized that memory safety flaws remain a persistent risk, and organizations should develop strategies to adopt memory-safe programming languages in new systems.

• The U.S. Cybersecurity and Infrastructure Security Agency (CISA) and NIST have echoed similar guidance in the context of national cybersecurity.

While these documents do not mention Swift explicitly, Swift's strong type system, ARC-based memory model, and compile-time safety guarantees align closely with the goals outlined in these recommendations. As such, it offers a practical, developer-friendly path toward safer embedded development.

Swift may not be the right fit for every embedded system. In applications where every byte of memory or instruction cycle is critical, real-time guarantees are hard requirements, or toolchain maturity is essential (for example, RTOS integration, static analyzers), C or Rust may still be preferred.

But in many modern embedded applications, especially those involving rapid prototyping, fast product iteration, safety-critical or maintainable firmware, and interoperability with existing C codebases, Swift offers a highly productive and safe development experience.

Embedded Swift is still maturing, but its momentum is undeniable. With ongoing compiler work, community-driven examples, and growing interest from developers, it’s poised to play a major role in the future of embedded systems.

Whether you're building an IoT device, a piece of industrial equipment, or a proof-of-concept wearable, Swift can help you write safer, more expressive firmware, without giving up performance or control.

Swift can be especially powerful during the prototyping phase, when the primary goal is to validate functionality quickly and safely. And with its increasing support for multiple hardware platforms, it offers a strong foundation for bringing modern software development practices to the embedded world.

In this tutorial, you will learn how to interface a resistive soil moisture sensor with an Arduino UNO microcontroller. You will learn about the parts and components of the sensor, how to calibrate the sensor for your soil type, and how to read both analog and digital output data from the sensor.

You will implement two practical examples in this tutorial:

1. The first example illustrates how you can read the analog output data from the sensor and convert the analog reading to a percentage value.

2. The second example illustrates how you can use the digital output from the sensor to determine if the soil is wet or dry, and indicate the result using a red and green LED.

At the end of the tutorial, you will have a solid understanding of how a resistive soil moisture sensor works and how to integrate the sensor into your microcontroller-based projects.

Prerequisites

To effectively follow along with this tutorial, you should have the following components:

• Arduino UNO

• Resistive Soil Moisture Sensor

• Breadboard

• 5 LEDs (any color for the analog example)

• 1 red LED and 1 green LED (for the digital example)

• 220-ohm resistors (one per LED)

• Jumper wires

Table of Contents

• What is a Soil Moisture Sensor?

• Types of Soil Moisture Sensors

• Parts of a Resistive Soil Moisture Sensor

• How to Calibrate the Sensor for Your Soil

• Example 1: How to Determine Soil Moisture Level in Percentage from Analog Output

• Example 2: How to Determine Soil Moisture State from Digital Output

• Conclusion

What is a Soil Moisture Sensor?

A soil moisture sensor is a device that estimates the moisture content in the soil. Soil moisture sensors typically operate by measuring the soil's electrical properties, such as dielectric and resistance. Some soil moisture sensors also use time domain methods to determine the propagation speed of electromagnetic waves through the soil.

Soil moisture sensors have various applications in different fields. These applications include, but are not limited to:

• Climate and environmental research

• Automated/smart irrigation system

• Greenhouse monitoring systems

• Urban planning

Types of Soil Moisture Sensors

Soil moisture sensors are categorized based on the property of the soil that they measure as an indicator of the moisture content. The most common soil moisture sensors for small-scale projects are:

• Resistive soil moisture sensors

• Capacitive soil moisture sensors

Resistive Soil Moisture Sensor

A resistive soil moisture sensor estimates the moisture content based on the relationship between water content and soil resistivity. The electrical resistivity of the soil reduces exponentially as the water content increases.

The resistive soil moisture sensor has two probes inserted into the soil. It measures the electrical resistance of the soil between the two probes.

Capacitive Soil Moisture Sensor

A capacitive soil moisture sensor determines the moisture content of the soil based on the relationship between water content and the dielectric properties of the soil. The dielectric constant of a soil increases as the water content increases.

A capacitive soil moisture sensor typically has a positive plate and a negative plate with space between them. When inserted into the soil, the soil becomes the dielectric medium between both plates. The sensor measures the change in the soil's dielectric property.

How to Choose Between Resistive and Capacitive Sensors

The choice between a resistive and capacitive soil moisture sensor depends on a couple of factors:

• Cost: A resistive soil moisture sensor is typically less expensive than a capacitive sensor.

• Accuracy: Capacitive soil moisture sensors are more accurate than their resistive counterparts. Factors like soil type and application of fertilizers have a lesser effect on the sensitivity of the capacitive soil moisture sensor.

• Long-term use: Resistive soil moisture sensors are prone to corrosion with frequent use. This happens because the current flowing through the probes in contact with the soil causes electrolysis of the metal electrodes in the probes. On the other hand, capacitive soil moisture sensors are resistant to corrosion. This is because the contact plates are embedded in corrosion-resistant materials and don't need to be in direct contact with the soil.

Parts of a Resistive Soil Moisture Sensor

The resistor soil moisture sensor module is usually made of two parts: the sensor probes and the voltage comparator module.

The Sensor Probes

The probes are the part of the sensor that is placed in the soil. The probes are used to detect the electrical resistance across two points in the soil.

As shown in the diagram above, the sensor probes have the following components:

• Electrodes: The metal electrodes conduct current through the soil. When the sensor is powered, current flows from one electrode through the soil to the other electrode and back to the comparator module. The sensor then measures the soil's resistance to the electrical signal flowing through the soil across the electrodes to determine the moisture level.

• Non-polarized connector pins: The two connector pins are used to connect the probes to the voltage comparator module. The pins have no polarity and can be connected in any order to the respective connector pins on the voltage comparator module. The probe is connected to the comparator module using female-to-female jumper wires.

The Voltage Comparator Module

The voltage comparator module interprets the electrical signal from the probes, processes the signals, and provides both analog and digital outputs that can be read by the microcontroller.

The voltage comparator module has the following components:

• LM393 Comparator Chip: The LM393 comparator is a dual comparator that compares the electrical signal from the probes to a reference set by the potentiometer and produces a digital output. The reference signal corresponds to a certain soil moisture level (threshold) and is set using the potentiometer.

  • The comparator outputs a HIGH when the analog signal read from the probes is above the reference, this means the soil has less moisture than the threshold moisture level.

  • The comparator outputs a LOW when the analog signal read from the probes is less than the reference; this means the soil has more moisture than the threshold moisture level.

• Potentiometer: The potentiometer is used to set the reference electrical signal that is used by the LM393 comparator chip. The potentiometer raises or lowers the threshold moisture level. It consists of a knob that can be turned either clockwise or counterclockwise.

• Power Indicator (PWR-LED): The power indicator is an LED that turns on when the module is powered on.

• Digital Output Indicator (DO-LED): The digital output indicator is an LED that turns on when the sensor detects wet soil. That is, the current moisture level read from the sensor is above the threshold, and the comparator outputs a LOW.

• Power Supply Pin (VCC): This pin is used to supply power to the sensor. The sensor module can be powered by a 5V or 3.3V voltage source. You should note that changing the voltage source also changes the analog output from the sensor. In this tutorial, you will be using one of the digital pins to power the sensor. The digital pins of the Arduino output 5V. The sensor typically has an operating current of 15mA, and the Arduino digital output pin can provide a maximum current of 40mA, so it can safely power the sensor.

• Ground Pin (GND): This pin is used to provide a ground reference for the sensor. It is usually connected to any ground pin in your microcontroller.

• Digital Output Pin (DO): The digital output pin outputs a HIGH or a LOW based on the value obtained from the LM393 comparator. This pin is usually used by a microcontroller to read the digital output of the LM393 comparator.

• Analog Output Pin (AO): The analog output pin provides a 10-bit analog voltage value. The values range from 0 to 1023, and they indicate the moisture level of the soil. Typically, in most sensors, higher analog values indicate drier soil and lower analog values indicate wetter soil.

• Sensor Probe Connector Pins: These two pins are used to connect the sensor probes to the voltage comparator module.

How to Calibrate the Sensor for Your Soil

As explained earlier in the article, the resistive soil moisture sensor is sensitive to soil type. This means that it’s important that you calibrate the sensor to the soil type you intend to use it on. You do this to improve the accuracy of your readings on a particular soil.

One way of calibrating the sensor is to determine the possible range of values for the soil type. That means you measure the sensor's output when the soil is totally dry and when the soil is totally wet. You can then use this range of values to map the sensor's reading to a new scale, like a percentage. The following steps describe how you can calibrate the sensor:

Step 1: Connect the Sensor to the Arduino

Using the image above as a reference, connect the sensor to the Arduino microcontroller as follows:

1. Connect the VCC or Power pin of the sensor's module to digital pin 7 of the Arduino. This allows you to control the supply voltage to the sensor in the Arduino sketch, ensuring that the sensor is only powered when you want to take a reading. Doing this can help improve the durability of the sensor.

2. Connect the GND pin of the sensor's module to a ground pin in the microcontroller.

3. Connect the analog output pin AO of the sensor's module to the analog pin A0 of the Arduino. This is the pin where you will read the analog data from the sensor.

4. Connect the two pins on the probes to the two connector pins on the sensor's module. The connector pins have no polarity.

Step 2: Upload Calibration Sketch to Microcontroller

Upload the following sketch into the Arduino:

const int sensorPin = A0; // Analog input pin for sensor
const int powerPin = 7; // Digital pin to power the sensor

int getAverageReading(int analogPin, int powerPin, int samples = 10) {
  long total = 0;

  digitalWrite(powerPin, HIGH); // Power ON sensor
  delay(500); // Wait for sensor to stabilize

  for (int i = 0; i < samples; i++) {                    
 total += analogRead(analogPin);
    delay(10); // Short gap between cycles
 }

  digitalWrite(powerPin, LOW); // Power OFF sensor
  return total / samples;
}

void setup() {
  Serial.begin(9600);

  pinMode(powerPin, OUTPUT);
  digitalWrite(powerPin, LOW); // Ensure sensor is off at start

  Serial.println("Calibration mode: Insert into DRY or WET soil and observe values.");
}

void loop() {
  int avgReading = getAverageReading(sensorPin, powerPin, 20); // Take 20 averaged samples
  Serial.print("Average analog reading: ");
  Serial.println(avgReading);
  delay(2000); // Update every 2 seconds
}

In the sketch, you begin by defining the pins for powering the sensor and reading the analog data from the sensor. The digital pin 7 powers the sensor, and the analog pin A0 reads data from the sensor.

The getAverageReading function powers on the sensor, takes multiple readings from the sensor, powers off the sensor, and returns the average of the readings taken. The function has three parameters:

• analogPin – The analog input pin, which reads data from the sensor to the microcontroller.

• powerPin – The pin used to power the sensor. The sensor is powered only when you want to take a reading.

• samples – The number of readings taken from the sensor. This defaults to 10. You take multiple readings as a way of filtering out noise in the sensor data.

In the setup function, you begin by setting the baud rate, which is 9600 for Arduino UNO and differs across different microcontrollers. Then you set the digital pin 7 used to power the sensor as an output pin, and write a LOW to the sensor to ensure it is off at the start.

Lastly, in the loop function, you get the average analog reading from the getAverageReading function and print it to the Serial Monitor. You should note that in the sketch, the readings are taken every 2 seconds – this delay should be longer in a practical application in order to improve the durability of the sensor.

Step 3: Record the Value for Dry Soil

Insert the sensor's probe into a completely dry soil sample, and record the average analog reading. You can dry the soil by baking it in an oven to remove moisture. Note that you must allow the soil to cool before inserting the sensor's probe. Hot soil may damage the sensor. The analog readings from the sensor differ according to soil type but are typically very high for dry soil. This is what I obtained for a dry soil sample:

Step 4: Record the Value for Wet Soil

Insert the sensor's probe into a completely wet soil sample, and record the average analog reading. The value is typically low when the soil is saturated. This is what I obtained for a wet soil sample:

You will use these recorded analog values for wet and dry soil in the subsequent section as reference points to map future readings onto a percent scale. In the example provided later in this tutorial, the moisture level is expressed as a percentage, between 0%(dry) and 100%(wet). Using the calibration values ensures that the percentage output accurately reflects the conditions of your soil type.

Example 1 – How to Determine Soil Moisture Level in Percentage from Analog Output

In this example, you will learn how to read the analog data from the soil moisture sensor, convert the analog reading to a percentage value, and visually represent the moisture level using five LEDs.

The analog reading from the sensor is inversely proportional to the soil moisture level. This means that higher analog readings indicate drier soil and lower readings indicate wetter soil.

The LEDs act as a visual indicator of the moisture percentage. The LEDs are lit based on a range of soil moisture levels:

• 1 LED: 0-20% (very dry)

• 2 LEDs: 20-40%

• 3 LEDs: 40-60%

• 4 LEDs: 60-80%

• 5 LEDs: 80-100% (very wet)

Circuit Diagram

Using the schematic above as a reference, design the circuit as follows:

1. Connect the sensor to the Arduino in the same way as described in the calibration section. That is, connect the sensor's power pin (VCC) to digital pin 7, analog output pin AO to analog pin A0, and ground pin to any ground pin of the microcontroller.

2. Connect each of the LEDs in series with a 220-ohm resistor.

3. Connect the anode of the five LEDs to digital pins 3, 4, 5, 6, and 8 of the microcontroller.

This is how my setup looked:

Arduino Code

Upload the following sketch to your Arduino:

const int sensorPin = A0; // Analog input from soil moisture sensor
const int powerPin = 7; // Digital pin to power the sensor

// LED bar pins (from lowest to highest)
const int ledPins[5] = {3, 4, 5, 6, 8};

// Calibrated analog values - replace with your values
const int dryValue = 1005;
const int wetValue = 254;

// Global to hold the last analog value
int lastAnalogReading = 0;

// Read and calculate soil moisture percentage
int getMoisturePercent(int analogPin, int powerPin, int samples = 10) {
  unsigned long total = 0;

  digitalWrite(powerPin, HIGH);
  delay(10); // Allow sensor to stabilize

  for (int i = 0; i < samples; i++) {
 total += analogRead(analogPin);
    delay(10);
 }

  digitalWrite(powerPin, LOW);

  int avgReading = total / samples;
 lastAnalogReading = avgReading;

  int percent = map(avgReading, dryValue, wetValue, 0, 100);
  return constrain(percent, 0, 100);
}

// Update LED states using boolean array
void updateLEDBar(int percent) {
  bool ledStates[5] = {0, 0, 0, 0, 0}; // Default all OFF

  if (percent <= 20) {
    ledStates[0] = true;
 } else if (percent <= 40) {
    ledStates[0] = ledStates[1] = true;
 } else if (percent <= 60) {
    ledStates[0] = ledStates[1] = ledStates[2] = true;
 } else if (percent <= 80) {
    ledStates[0] = ledStates[1] = ledStates[2] = ledStates[3] = true;
 } else {
    for (int i = 0; i < 5; i++) ledStates[i] = true;
 }

 // Write LED states to pins
  for (int i = 0; i < 5; i++) {
    digitalWrite(ledPins[i], ledStates[i] ? HIGH : LOW);
 }
}

void setup() {
  Serial.begin(9600);
  pinMode(sensorPin, INPUT);
  pinMode(powerPin, OUTPUT);
  digitalWrite(powerPin, LOW); // Start with sensor powered off

  for (int i = 0; i < 5; i++) {
    pinMode(ledPins[i], OUTPUT);
    digitalWrite(ledPins[i], LOW); // Ensure all LEDs start OFF
 }

  Serial.println("Soil Moisture Monitor with 5-LED Bar (Boolean Array)");
}

void loop() {
  int moisturePercent = getMoisturePercent(sensorPin, powerPin, 20);

 // Print both analog reading and percent
  Serial.print("Analog Reading: ");
  Serial.print(lastAnalogReading);
  Serial.print("  |  Moisture: ");
  Serial.print(moisturePercent);
  Serial.println(" %");

  updateLEDBar(moisturePercent);

  delay(2000); // Update every 2 seconds
}

In the sketch, you start by defining the analog pin for reading data from the sensor and the digital pin for powering the sensor. You also define the following variables, which will be used in the code:

• ledPins[5] – An array that stores the digital pins used to power each LED. The pins are arranged from the first LED to the last one. That is the visual display order from left to right.

• dryValue – This variable stores the analog value recorded for dry soil during the calibration section.

• wetValue – This variable stores the analog value recorded for wet soil during the calibration section.

• lastAnalogReading – This variable stores the last reading taken by the sensor. You use this variable to log the actual analog reading to the Serial Monitor.

The getMoisturePercent function powers on the sensor, takes multiple readings, powers off the sensor, calculates the average analog reading, represents the analog reading in percent, and returns the percent value. The function also saves the average analog reading to the lastAnalogReading variable. You can print it directly here, but this sketch saves it in a separate variable so that you can print it later in the loop function for readability.

You can express the average analog reading in percentage with the map(avgReading, dryValue, wetValue, 0, 100) function. The function remaps the average reading stored in avgReading from the range of your calibration values dryValue and wetValue to a new range between 0 and 100 (where 0 is the driest and 100 is the wettest). You then use the constrain function to keep values within the 0 and 100 range.

The updateLEDBar function displays the percent value using the LEDs. The ledStates array in the function stores the logic state of each LED. You begin by setting all LEDs off – that is, having a state 0. The next bit of logic is a simple if statement where you turn on the required LEDs corresponding to a particular percent range by setting the elements in the array to true (equivalent to 1). You end the function by writing the states in the ledStates to the pins in ledPins.

The setup function is pretty routine: you set the baud rate for serial communication, define input and output digital pins, and write a LOW to the digital output pins to ensure they are all turned off at the start.

In the loop function, you call the getMoisturePercent function to get the percent moisture value. You then print the percent value and the average analog reading to the Serial Monitor for clarity. Lastly, you call the updateLEDBar function with the percent value as a parameter to turn on the respective LEDs to indicate the moisture level.

Test the System

You can proceed to test the example with different moisture levels. For example:

• At around a 37% moisture level, two LEDs are lit.

• At about a 76% moisture level, four LEDs are lit.

You can also simulate the example on Tinkercad here: https://www.tinkercad.com/embed/6s47ZvIrNOP.

Example 2 – How to Determine Soil Moisture State from Digital Output

In the previous project, you learned how to read the analog data from the sensor and convert the value into a percentage. If your application requires a binary output, you can use the digital output pin. In this section, you'll learn how to use the digital output pin of the sensor.

The digital output pin has only two states:

• LOW – This state corresponds to 0V and is the output when the soil is wet, that is, the moisture level is above the set threshold.

• HIGH – This state corresponds to 5V and is the output when the soil is dry, that is, the moisture level is below the threshold moisture level.

Circuit Diagram

The example built in this section is very simple. It consists of a red and green LED. When the sensor outputs a LOW from the digital output pin, which means the soil is wet, the green LED turns on. When the sensor outputs a HIGH from the digital output pin, which means the soil is dry, the red LED turns on.

Using the circuit diagram above as a reference, design the circuit as follows:

1. Place the sensor module on the breadboard.

2. As in the previous project, connect the power (VCC) pin of the sensor module to digital pin 7 of the microcontroller, and the ground (GND) pin to a ground pin on the microcontroller.

3. Connect the digital output (DO) pin of the sensor to digital pin 2 of the microcontroller. This pin is where you will read the data from the sensor.

4. Connect the red and green LEDs each to a 220-ohm resistor in series.

5. Connect the anode of the red and green LEDs to digital pins 12 and 13 of the microcontroller, respectively.

This is how my physical connection looked:

How to Set the Threshold Moisture Level

Before using the digital output pin of the soil sensor module, you first have to set a threshold moisture level.

The sensor's module has a built-in potentiometer that allows you to adjust the moisture level, which will be used as a threshold. The sensor also has a built-in LM393 comparator, which continuously compares the actual reading from the sensor to the threshold set by the potentiometer and outputs the appropriate state.

You can set the threshold moisture level by rotating the knob on the potentiometer. Rotating the potentiometer clockwise lowers the threshold. Rotating the potentiometer counterclockwise raises the threshold. The sensor module also has a built-in LED labeled DO-LED that turns on when the sensor output is LOW. Set the threshold as follows:

1. Place the sensor's probe in soil just dry enough that it requires irrigation. That is the soil whose moisture level you want to use as a threshold. Note that the DO-LED should be off.

2. Use a small screwdriver to rotate the potentiometer clockwise until the DO-LED comes on. Turning clockwise reduces the threshold level until it is slightly lower than the current moisture level, thereby triggering the digital output indicator (DO-LED) to turn on.

3. Turn the screw slightly counterclockwise just enough to turn the built-in digital LED off. Given that the current moisture level of the soil is dry enough that it needs to be watered, you still want the sensor to read this level as dry. So turning counterclockwise reduces the threshold to a level just above the current moisture level of the soil, which triggers the DO-LED to turn off.

Arduino Code

Upload the following sketch to the Arduino:

const int sensorPower = 7; // Digital pin to power the sensor
const int sensorPin = 2; // Digital input from the sensor
const int greenLED = 13; // Green LED pin
const int redLED = 12; // Red LED pin

int getSensorReading(int digitalPin, int powerPin) {  
  digitalWrite(powerPin, HIGH); // Power ON sensor
  delay(500); // Wait for sensor to stabilize

  int reading = digitalRead(digitalPin);

  digitalWrite(powerPin, LOW); // Power OFF sensor
  return reading;
}

void setup() {
  Serial.begin(9600);

  pinMode(sensorPower, OUTPUT);
  pinMode(sensorPin, INPUT);
  pinMode(greenLED, OUTPUT);
  pinMode(redLED, OUTPUT);

 // Ensure everything starts OFF
  digitalWrite(sensorPower, LOW);
  digitalWrite(greenLED, LOW);
  digitalWrite(redLED, LOW);
}

void loop() {
  int sensorReading = getSensorReading(sensorPin, sensorPower);

  Serial.println("===================================");
  Serial.print("Digital Reading: ");
  Serial.println(sensorReading);

  if (sensorReading == HIGH) {
    Serial.println("Status: Soil moisture is LOW (dry)");
    Serial.println("Action: Water the soil");
    digitalWrite(redLED, HIGH);
    digitalWrite(greenLED, LOW);
 } else {
    Serial.println("Status: Soil moisture is GOOD (wet)");
    Serial.println("Action: No watering needed");
    digitalWrite(greenLED, HIGH);
    digitalWrite(redLED, LOW);
 }

  Serial.println("===================================");
  Serial.println(); 
  delay(2000); // Update every 2 seconds
}

In the sketch, you define the digital pin used to read data from the sensor, the digital pins to power the sensor, and the green and red LEDs.

The getSensorReading function powers on the sensor, takes the digital reading from the sensor, powers off the sensor, and returns the digital reading taken from the sensor.

The setup function is routine: you set the baud rate, define the digital pins as input or output, and write a LOW to the output pins to ensure they are off at the start.

In the loop function, you call the getSensorReading to get the digital reading from the sensor. If the sensor outputs a HIGH, you turn on the red LED and print a message that the soil is dry. If the sensor outputs a LOW, you turn on the green LED and print a message that the soil is wet.

Test the System

You can proceed to test the project using different moisture levels. For example:

• For wet soil, that is, if the moisture level is above the threshold, you should have the following output:

• For dry soil, that is, if the moisture level is below the threshold, you should have the following output:

You can also simulate the example on Tinkercad here: https://www.tinkercad.com/embed/2wHwfKherNz.

Conclusion

In this tutorial, you learned what a resistive soil moisture sensor is, how to calibrate the sensor for your soil type, and how to use the sensor's analog and digital data to determine the moisture level of the soil.

The examples provided a solid foundation for working with the sensor. You can extend these examples into more complex projects, such as an automated irrigation system or remote monitoring of soil moisture.

For projects that require greater accuracy and frequent or sustained sensor operation, you should explore the capacitive sensor.

You can access all the code on GitHub.

An embedded system typically goes through a simple but powerful cycle:

1. Sense – Gather information from the environment using sensors.

2. Process – Use software logic to decide what to do with the data.

3. Act – Trigger a response, like turning on a motor or lighting an LED.

Each project begins with a use case – a specific goal like brewing coffee or controlling a car’s fuel injection. From that, engineers define system requirements, which are split into:

• Hardware (for example, microcontrollers, sensors, actuators)

• Software (what we call embedded software)

This handbook focuses on the software side of embedded systems: how we write code to make embedded systems intelligent. Embedded software runs on resource-constrained devices like microcontrollers, which may have just a few kilobytes of memory. The software might need to be highly efficient, reliable, and often capable of working in real-time.

But embedded software isn't just about writing code – it’s also about understanding:

• How hardware works

• How to manage memory and power

• How to handle timing and communication

• How to build robust, fail-safe systems

While embedded systems development isn’t typically research-focused in most industry roles, it demands a broad skill set, from low-level programming to system-level design. What makes this field especially exciting is how it brings together diverse domains like machine learning, digital signal processing (DSP), and control systems, all of which can be applied directly in real-world devices.

In this article, I’ll give you:

• A high-level overview of what embedded software involves

• Key concepts every developer should know

• A tour of commonly used tools and frameworks

• Resources to help you learn and understand basics.

Whether you're just curious or planning a career in embedded systems, this guide is your launchpad.

Table of Contents

• HW Layer: Microcontroller

• Firmware Design and Tools

• Tools and Concepts for Embedded Development

• Bare Metal, RTOS, and Embedded Operating Systems

• Designing Drivers for Embedded Systems

• Security in Embedded Systems

• Debugging and Forensics in Embedded Systems

• Automation and Testing in Embedded Systems

• Where to Go from Here

This article offers a broad overview of embedded firmware development, but it doesn’t cover every aspect, particularly advanced software architecture frameworks or comprehensive lists of open source software and tools. Where appropriate, I have included external resources that were valuable in expanding my own understanding.

Prerequisites

You don’t need to be an expert to follow this guide, but some prior knowledge will help you get the most out of it:

• Basic C or C++ programming**:** Familiarity with functions, pointers, and memory concepts is helpful.

• Computer architecture fundamentals**:** Understanding what a CPU does, how memory works, and basic instruction execution will make embedded concepts clearer.

• Electronics basics (optional): Knowing how sensors, resistors, or microcontrollers interact at a circuit level is useful but not mandatory.

• Comfort with the command line**:** Especially for working with build systems, compilers, and flashing tools.

This guide is ideal for students, engineers, or hobbyists looking to deepen their understanding of how software interacts with hardware in real-world systems.

With that, let’s start from the ground up, hardware. Throughout this guide, most examples will reference ARM Cortex-M microcontrollers, as they are among the most commonly used in the embedded world.

HW Layer: Microcontroller

One of the most important knowledge blocks in embedded firmware development is understanding how a microcontroller (MCU) works and how it connects to sensors, actuators, and other microcontrollers.

If you’re familiar with basic computer architecture (like instruction sets and memory organization), that knowledge translates well to embedded systems. In fact, Computer System Organization, often taught in computer science and electrical engineering programs, is a great foundation for understanding microcontrollers.

What is a Microcontroller?

A microcontroller is a compact computing unit that includes:

• A CPU (Central Processing Unit or Microprocessor)

• Memory (Flash and RAM)

• Peripherals (for I/O, timers, communication, and so on)

In essence, it's a tiny computer-on-a-chip, optimized for specific control tasks like reading sensors or driving motors.

By contrast, a microprocessor is just the CPU. It requires external memory and peripherals to function. Microcontrollers are self-contained and better suited for embedded applications.

For example, this reference manual for the STM32F4 series (from STMicroelectronics) provides detailed documentation on not just the CPU but each peripheral’s functionality and the register map.

Instruction Set Architecture (ISA)

A microprocessor executes a series of instructions defined by its Instruction Set Architecture (ISA). ISA as defined by ARM is a part of the abstract model of a computer that defines how the CPU is controlled by the software. The ISA acts as an interface between the hardware and the software, specifying both what the processor is capable of doing as well as how it gets done.

For example:

• ARMv7 – used in ARM Cortex-M3.

• ARMv7E – used in Cortex-M4 and M7.

Many vendors (for example, STMicroelectronics, NXP, TI) manufacture MCUs that support ARM ISAs but include their own peripheral sets. Understanding the ISA is essential for low-level coding and interpreting assembly instructions.

This ARMv7-M architecture reference manual provides more details on v7 Architecture.

Memory in Microcontrollers

Most microcontrollers typically feature two types of memory:

• Flash – Stores your code and read-only data.

• RAM – Used during program execution to hold:

  • The heap (for dynamic memory)

  • The stack

  • The .data and .bss sections (initialized/uninitialized global/static variables)

Later sections have resources that go deeper into memory mapping and how these regions interact during runtime.

Clock and Power Management

Microcontrollers are digital logic devices built from:

• Combinatorial logic – Logic gates that evaluate outputs instantly

• Sequential logic – Relies on clocks to move through states

The clock tree distributes timing signals across the CPU and peripherals. MCUs often support multiple clock sources (internal RC, external crystal, PLL), and use prescalers to drive components at different frequencies.

For power-sensitive applications, MCUs offer multiple low-power modes:

• Sleep – CPU off, timers and peripherals are mostly active, memory is retained

• Deep Sleep – CPU off, most clocks off, memory is retained, wake-up is slower than sleep, power consumption is lower than Sleep

• Standby – CPU off, few interrupts are active, everything else is powered down, memory is not retained. Lowest power mode.

These modes reduce power consumption by turning off clocks and disabling unused peripherals. Designing the system to switch in and out of low-power states effectively is a core skill in embedded software development.

This article talks about Clock Trees and Oscillators for the ARM Cortex microcontrollers.

Interrupts

Interrupts let MCUs react to asynchronous events, like button presses or sensor signals.

An interrupt temporarily pauses normal code execution to run a dedicated handler. After it’s serviced, the CPU resumes its previous task. They are vital for:

• Fast event response

• Reduced polling

• Efficient power use (for example, waking from sleep)

Timers

Timers are built-in peripherals used to track time or generate events.

Common uses are:

• Implementing software delays

• Creating precise software timers

• Waking up from low-power modes

Mastering timers helps with real-time behavior and precise event scheduling.

Communication Protocols

Microcontrollers often need to talk to other devices via built-in communication peripherals:

• UART (Universal Asynchronous Receiver/Transmitter): Serial communication between two devices, great for logs and debugging.

• I²C (Inter-Integrated Circuit): Two wire protocol for talking to sensors and EEPROMs.

• SPI (Serial Peripheral Interface): High Speed, full-duplex protocol for devices like Flash or displays.

• USB (Universal Serial Bus): Complex but widely used for PCs, data acquisition and HID devices.

Here’s a figure showing multiple peripherals connected to a MCU:

DMA or Direct Memory Access is an important peripheral which can be used to transfer data to/from memory without CPU involvement. It improves performance and allows the CPU to perform other tasks or enter low power mode to reduce power consumption.

This article provides a good overview of the communication protocols I2C, UART and SPI.

We’ve now covered the essential building blocks of microcontroller hardware – from memory and clocks to interrupts and communication buses.

Next, we’ll explore the software principles and tools that bring these microcontrollers to life, including compilers, debuggers, and embedded development frameworks.

Firmware Design and Tools

Designing Embedded Software

Even though embedded systems operate under unique hardware constraints, software design principles are still crucial. Applying them thoughtfully becomes even more important when memory, CPU cycles, and responsiveness are limited.

Most Embedded firmware projects begin with a structured design approach:

1. Understand the problem statement

2. List assumptions

3. Define use cases

4. Define system and software requirements

5. Create high-level architecture

6. Drill down to detailed design and implementation

If you’re new to software design, check out my article on design principles.

Here’s a figure showing the five blocks of software design:

Using Design Patterns

Once you're designing individual components, design patterns help you write scalable and maintainable code. Here are some common patterns in embedded systems:

• Publisher-Subscriber (Observer) – Useful for decoupling event producers and consumers (for example, sensor data being broadcast to multiple modules).

• Singleton – Ensures only one instance of a module or resource manager exists (for example, for drivers or HAL layers).

• Adapter – Translates between incompatible interfaces (for example, wrapping platform-specific code into a portable application layer).

• State Machine – Represents system behavior as transitions between states (for example, Bluetooth states: IDLE → SCANNING → CONNECTING → CONNECTED → DISCONNECTED).

Design patterns often need to be adapted for memory and timing constraints, but the core concepts remain highly relevant.

There are lot of great resources on design patterns – here are a few that helped me:

1. Book: Head-first Design patterns - A great book to get understand the concept of design patterns

2. Book: Design Patterns: Elements of Reusable Object-Oriented Software

3. Course: Object-Oriented Programming and Design Patterns in C#

4. Article on HSM: Hierarchical State Machine Overview (Barr Group)

Programming Languages for Embedded Systems

While any language can theoretically be used if it compiles to machine code, in practice, three dominate the embedded world:

• C – The industry standard. Provides deterministic behavior and low-level access, making it ideal for memory and timing-sensitive code.

• C++ – Adds object-oriented features while maintaining control. Once considered risky in embedded due to synthesized code and overhead, it’s now widely adopted where systems benefit from abstraction and modularity.

• Rust – A memory-safe alternative gaining traction in safety-critical and open-source embedded development.

Languages like Python (via MicroPython or CircuitPython) are used in educational or prototyping contexts but are not suitable for production due to performance and memory overhead.

Some resources on programming languages that might be helpful to understand concepts:

1. The Embedded Rust Book

2. C Programming Language by K&R

3. Inside the C++ Object model – There are a lot of books and lectures on C++, but for embedded, understanding the object model benefits a lot.

Data Structures Matter

Embedded systems require careful data handling due to strict memory and timing constraints. Mastering core data structures is essential:

• Arrays – fixed-size data.

• Linked Lists – Common in software timers, queues.

• Stacks and Queues – Task scheduling, event management and data storage.

• Bitfields/Flags – Memory efficient state representation.

• Binary Trees – Used in routing tables or decision logic.

You'll often build event queues, circular buffers, or timer lists, all of which rely on these foundational structures.

There are a lot of resources for understanding data structures, but I have found this one to be helpful for learning and practicing: GeeksForGeeks DSA Tutorial. And here’s a full course on DSA if you want to dive deeper.

Bit Manipulation: A Core Embedded Skill

Unlike general-purpose software, embedded systems often require low-level access to registers and require precise bit control:

• Setting and clearing individual bits

• Using bitwise operators like AND (&), OR (|), XOR (^)

• Bit masking and shifting (<<, >>)

Mastering bit hacks is essential for writing hardware drivers or manipulating control registers.

This resource provides a good number of examples for bit manipulation: Stanford Bit Hacks.

Tools and Concepts for Embedded Development

Cross Compilation

Embedded code is compiled on a host (like your PC) for a target architecture using cross-compilers.

To do this, you need:

• A compiler (for example, arm-none-eabi-gcc for ARM Cortex-M) that compiles high level language code into Assembly language instructions.

• A linker to layout and combine object files.

• A Makefile or build system to organize and automate compilation, linking and binary creation.

Here’s an example to compile a main.c to create a main.elf that can be flashed on the device:

arm-none-eabi-gcc main.c -o main.elf

A Makefile is a script used by the make build automation tool to compile and link programs to create a binary. It defines how to build your program from source files, manages compilation order based on dependencies and defines commands to complete the build.

For example, lets write a Makefile for building a project for an ARM Cortex-M4 target that has three source files: a main.c, utils.c, and sensor.c

CC = arm-none-eabi-gcc
CFLAGS = -c -mcpu=cortex-m4 -mthumb -Wall -O2
LDFLAGS = -mcpu=cortex-m4 -mthumb
TARGET = main.elf
OBJS = main.o utils.o sensor.o
SRC = main.c utils.c sensor.c

$(TARGET): $(OBJS)
    $(CC) $(OBJS) -o $(TARGET)

main.o: main.c
    $(CC) $(CFLAGS) main.c

utils.o: utils.c
    $(CC) $(CFLAGS) utils.c

sensor.o: sensor.c
    $(CC) $(CFLAGS) sensor.c

clean:
    rm -f *.o *.elf

In the above makefile, here’s a description of the flags:

• -mcpu=cortex-m4: Targets the ARM Cortex-M4 processor.

• -mthumb: Enables Thumb instruction set, which is used by ARM Cortex-M series.

• -Wall: Enables all common warnings.

• -O2: Optimization level 2 for balance between performance and code size.

Makefiles can seem intimidating, but they’re just scripts that define how to build your program from source. Once you understand the basics, they’re a huge productivity booster.

A linker script tells the linker (ld) how to organize the program in memory where to place code, data, stack, heap, and so on. It's crucial for embedded systems because you're working with limited memory and specific memory-mapped hardware.

Here’s an example of a simple linker script for a STM32F4 microcontroller:

/* STM32F4 Cortex‑M4 Simple Linker Script */

ENTRY(Reset_Handler)

/* Define memory regions based on STM32F4 datasheet */
MEMORY
{
  FLASH (rx) : ORIGIN = 0x08000000, LENGTH = 1024K
  RAM   (rwx): ORIGIN = 0x20000000, LENGTH = 128K
}

/* Section layout */
SECTIONS
{
  /* Interrupt vectors and code go into Flash */
  .isr_vector :
  {
    KEEP(*(.isr_vector))    /* Keep vector table (reset, etc.) */
  } > FLASH

  .text :
  {
    *(.text*)               /* All code */
    *(.rodata*)             /* Read-only data */
    . = ALIGN(4)
    _etext = .             /* End of code (used for data init) */
  } > FLASH

  /* Initialized data: load from Flash, run in RAM */
  .data : AT(_etext)
  {
    _sdata = .            /* Start of .data in RAM */
    *(.data*)
    . = ALIGN(4)
    _edata = .            /* End of .data */
  } > RAM

  /* Uninitialized data (zero-filled) */
  .bss :
  {
    _sbss = .
    *(.bss*)
    *(COMMON)
    . = ALIGN(4)
    _ebss = .
  } > RAM

  /* Define stack end (top of RAM) */
  _estack = ORIGIN(RAM) + LENGTH(RAM);
}

Descriptions of the above file:

• MEMORY: Defines your microcontroller’s memory layout – 1 MB Flash and 128 KB SRAM.

• ENTRY(Reset_Handler): Sets the reset handler as the program entry point.

• .isr_vector and **.**text: Code sections placed in Flash. .isr_vector must use KEEP() so it's not removed during linking.

• .data : AT(_etext): Loads initialized variables from Flash but places them in RAM.

• **.**bss: Zero-initialized data, allocated in RAM

• _estack: Defines the initial stack pointer using the end of RAM.

Here are some sources to understand Makefiles, cross-compilation, and Linkers. And just note that using Makefile in a project is the best way to learn and master Makefiles:

1. Makefiles:

   • GNU Make Manual

   • Makefile Tutorial

   • In Pyjama Makefile Article

2. Linker Scripts:

   • Interrupt Blog on Linker Scripts

   • Intro to Linker Files – Medium

Flashing the Binary

Once you’ve compiled your code into a binary file, the next step is to flash it into the target microcontroller’s non-volatile memory via SWD (Serial Wire Debug) or JTAG. Flashing tools like OpenOCD, ST-Link, J-Link, or vendor-specific utilities manage this process.

What Is Flashing?

Flashing is the process of writing a compiled firmware image (typically a .bin or .hex file) into the microcontroller’s Flash memory. This enables the embedded system to retain and run your code even after power is removed.

The flashing tool communicates with the microcontroller over SWD or JTAG to:

• Halt the MCU (if needed)

• Access the internal flash controller

• Erase the relevant flash sectors

• Write the binary data to specific memory addresses

• Verify that the data was written correctly

OpenOCD (Open On-Chip Debugger) is a powerful, open-source utility that facilitates debugging and flashing of ARM-based microcontrollers. It supports a wide variety of hardware interfaces and microcontroller families, including STM32.

OpenOCD provides:

• Flashing capabilities for .elf, .bin, and .hex files

• Debugging via GDB (GNU’s open source debugger) integration

• Support for multiple debug probes (J-Link, ST-Link, CMSIS-DAP)

• Scripting via configuration files for board-specific and target-specific setups

A simple command to flash a binary using OpenOCD might look like this:

bashCopyEditopenocd -f interface/stlink.cfg -f target/stm32f4x.cfg -c "program main.elf verify reset exit"

This tells OpenOCD to:

• Use the ST-Link interface

• Load the STM32F4 target configuration

• Program main.elf into flash

• Verify it was written correctly

• Reset the MCU

• Exit the session

For a detailed walkthrough, check out: OpenOCD Deep Dive – Kickstart Embedded

Bare Metal, RTOS, and Embedded Operating Systems

When writing embedded software, you can approach the problem in three main ways, each with its own trade-offs:

1. Bare-Metal Programming

2. Real-Time Operating Systems (RTOS) (like FreeRTOS, Zephyr)

3. Embedded Operating Systems (like Embedded Linux)

The best choice depends on your use case, application’s complexity, hardware constraints, and real-time needs.

Most Modern 32-bit microcontrollers (for example, STM32, NXP, Renesas) come with vendor-provided development tools that include:

• HAL (Hardware Abstraction Layer) libraries

• Startup code and linker scripts

• Peripheral drivers

• Sometimes even middleware like USB, BLE, or file system stacks

These tools (like STM32Cube Config Tools) simplify setup and peripheral configuration, helping you get started quickly, without needing to write low-level code manually.

Benefits of HALs:

• Rapid prototyping and development

• Clean, reusable APIs for peripherals

• Great for onboarding and small teams

Drawbacks:

• Code bloat – HALs support many edge cases and configurations, which can inflate your binary size

• Extra latency – HAL often inserts unnecessary layers that reduce performance.

For performance-critical systems, developers often replace HAL drivers with custom, low-level implementations.

Bare-Metal Programming

Bare-metal programming is the most direct and lightweight approach. There’s no OS, and your code runs directly on the hardware with full control.

Typical setup includes:

• Include the correct header files, especially MCU and peripheral-specific headers provided by the vendor’s HAL (Hardware Abstraction Layer).

• Implement a main() function with an infinite loop (while(1))

• Perform all hardware initialization before entering the loop

• Use Interrupts to handle asynchronous events.

• Continuously check and control inputs/outputs inside the loop

This assumes your toolchain provides startup code and memory setup from the vendor.

#include "MCU_Header.h"

int main(void) {
    /* Initialize the MCU and the peripherals */
    init_clock();
    init_peripherals();

    /* runs in a loop forever */
    while (1) {
        // Task 1 : Read sensor data
        read_sensor(); 
        // Task 2 : Update the actuator based on the sensor data
        update_actuator(); 
    }
}

How does it run?

When the device powers on or resets, the startup code provided by the vendor is executed first. This code:

• Initializes the reset vector

• Copies initialized data from Flash to RAM

• Zeros out the .bss section (for uninitialized global/static variables)

• Calls your main() function

After calling main(), the system enters an infinite loop where your logic runs. The only other context switch occurs when an interrupt is triggered, briefly diverting control to an Interrupt Service Routine (ISR), after which it returns to the main loop.

When to use it:

• Simpler applications (for example, blinking LEDs, reading sensors)

• Ultra-low-power or ultra-low-latency needs

• When every byte of Flash and RAM matters

Pros:

• Minimal memory usage

• Maximum control

• Great for learning

Cons:

• No built-in task management or scheduling

• Can become hard to maintain for complex systems

This resource provides good details and example on Bare Metal Programming. For more details, this book is great as well: ARM Baremetal Ebook.

Real-Time Operating Systems (RTOS)

A Real-Time Operating System (like FreeRTOS, Zephyr) adds lightweight multitasking capabilities to your embedded application. It allows you to split your software into independent tasks that run concurrently and communicate through queues, semaphores, or message passing.

RTOS kernels often support different scheduling strategies like:

• Rate Monotonic Scheduling (RMS) – Tasks with shorter periods get higher priority

• Earliest Deadline First (EDF) – Tasks are prioritized based on impending deadlines

Example use cases:

• A drone where sensor data, motor control, and telemetry need to run in parallel

• A medical device where timing is critical for safety

• Rockets

Typical RTOS features:

• Task scheduling

• Timers

• Inter-task communication

• Interrupt handling integration

• Power management

Pros:

• Modular code structure with tasks

• Easier to scale as complexity grows

• Deterministic execution (when configured correctly)

Cons:

• Slightly higher memory footprint than bare-metal

• Learning curve for scheduling and priority tuning

RTOS Scheduling techniques are interesting – this part of the docs talks about Zephyr scheduling.

Embedded Operating Systems

Sometimes an embedded system is powerful enough to run a full-fledged OS like Embedded Linux, Android Things, or Windows IoT Core. This is common on devices with a display, networking stack, or file system.

It’s best used when the system requires multitasking, user interfaces, file systems, or network stacks, and when there’s plenty of processing power (for example, ARM Cortex-A).

Think of:

• Smart home hubs

• Automotive infotainment

• Industrial gateways

This table provides a high level methodology for choosing the right type of OS based on your application:

To understand OS fundamentals, this is a great book: Operating System Concepts and this is a great course: UC Berkeley: CS162.

So far, we’ve looked at how embedded applications are structured, whether using bare-metal loops, RTOS multitasking, or full operating systems. But regardless of which execution model you choose, your software ultimately needs to interact with the hardware.

This is where driver development comes in. Drivers form the crucial link between your code and the peripherals it controls, whether it's reading temperature, blinking an LED, or transmitting data over SPI. Let’s take a closer look at how to design robust, portable drivers for embedded systems.

Designing Drivers for Embedded Systems

When working with embedded software, one of the most practical and common tasks you’ll encounter is driver development.

A driver is a piece of software that enables the microcontroller (MCU) to interface with a hardware peripheral. This could be a temperature sensor, a motor controller, a display, or even a wireless module.

Drivers act as a bridge between your hardware and the application logic. They abstract away the raw register-level programming so that higher-level code can use clear function calls like read_temperature() or start_motor().

What Goes Into a Driver?

A typical embedded driver will include:

• Configuration – Setting up the peripheral with initial parameters (for example, baud rate for UART)

• Initialization – Preparing the peripheral for use, including enabling clocks and interrupts

• Calibration (if needed) – Adjusting the peripheral based on specific environment or use case

• Register Access – Reading from and writing to hardware registers (if applicable)

• Power Management – Enabling/disabling the peripheral to save power or putting the peripheral into a low power mode

• Interrupt Management – Handling asynchronous events triggered by the peripheral

Here’s a simplified view of a sensor driver API:

void sensor_init(void);
void sensor_calibrate(void);
float sensor_read_temperature(void);
void sensor_sleep(void);
void sensor_write(uint8_t reg, uint8_t value); // Assumption : 8 bit register address and 8 bit data value

The actual implementation might involve:

• Register definitions from the peripheral’s datasheet

• Bit manipulations for control and status registers

• Interrupt Service Routines (ISRs)

• Timing and delay management

Platform Abstraction: Why It Matters

One of the most important principles in driver design is decoupling the application from the platform. This makes your code easier to:

• Port to different MCUs

• Adapt for similar hardware (for example, different sensor models)

• Test across simulated or real environments

Platform-Agnostic Design Example (in C++) :

Let’s say you're writing a driver for a temperature sensor:

// Abstracts the HW platform on which the sensor driver is being written
class TemperatureSensorPlatform {
public:
    void i2cInit(void);
    void i2cWrite(uint8_t reg, uint8_t value);
    uint8_t i2cRead(uint8_t reg);
};

// Creates a generic Temperature sensor driver interface
class TemperatureSensor {
public:
    virtual void init() = 0;
    virtual float read() = 0;
    virtual void sleep() = 0;
};

You can implement this interface differently for a specific type of temperature sensor and also add the platform support for the HW platform you are writing the driver on for example STM32.

class TempSensorTMP117 : public TemperatureSensor {
public:

    TempSensorTMP117(TemperatureSensorPlatform platform) : 
    _platform(platform)
    TemperatureSensor()
    {}

    void init() override {
        // TMP117-specific register configuration
    }

    float read() override {
        // Read ADC value and convert
        return 25.4f;
    }

    void sleep() override {
        // Put sensor in low-power mode
    }
private:
    TemperatureSensorPlatform _platform; // Implements the I2C driver for STM32
};

Your application code now depends on the TemperatureSensor interface and Temperature Sensor Platform passed in the constructor making it portable and testable across temperature sensors and HW platforms.

One of my previous articles provides details on how to interface a sensor and how to design a driver for it.

Designing robust and modular drivers helps your firmware interact seamlessly with hardware, but in today’s connected world, that’s only part of the challenge. As embedded devices increasingly communicate with other systems, security becomes just as critical as functionality.

Now that we’ve covered how to interface with hardware, let’s explore how to protect those systems from unauthorized access, tampering, and data breaches.

Security in Embedded Systems

Security is often overlooked in embedded development but it shouldn’t be. Embedded systems are increasingly connected to networks, cloud services, or other devices, which makes them vulnerable to attacks like unauthorized access, firmware tampering, or data leaks.

Even simple devices like smart plugs or fitness trackers can be exploited if their firmware is insecure.

Key Security Practices

• Secure Boot: Ensure the firmware is cryptographically signed and verified before execution. This prevents unauthorized firmware from running.

• Firmware Update Integrity: Use encrypted or signed updates, especially for Over-the-Air (OTA) upgrades. Unprotected updates can be a major attack vector.

• Lock Debug Interfaces: After flashing the final firmware, disable or lock access to JTAG, SWD, or UART debug ports to prevent reverse engineering.

• Minimal Exposure: Disable unused peripherals (for example, Bluetooth, USB, network interfaces) and avoid exposing debug info (like UART prints) in production.

• Watchdog Timers: While not security features per se, watchdogs help ensure system recovery in the event of unexpected software behavior – which could result from attacks or bugs.

Security should be layered, as no single mechanism is sufficient on its own. Build security into every stage of the development process, from boot to communication to update handling.

Whether you're designing a consumer product or an industrial controller, proactive security practices are essential for protecting user data, system reliability, and device reputation.

This resource provides a good understanding of Embedded Systems Security: BlackBerry QNX: Embedded System Security Guide

Debugging and Forensics in Embedded Systems

Debugging embedded systems is one of the most challenging and fascinating aspects of development. Unlike in desktop or web applications, bugs in embedded systems often manifest as unexpected hardware behavior rather than error messages.

For example, suppose your code is supposed to blink an LED once per second:

• If the LED stays on, your delay code might be broken.

• If it blinks erratically, you might have a timing bug.

• If it doesn’t blink at all, you might never be reaching that part of your code or the hardware might not be configured correctly.

Why Debugging is Critical

Embedded systems directly control real-world hardware, often in critical or safety-sensitive environments. A small bug can lead to large consequences.

Historical Note: During the Apollo 11 moon landing, the onboard computer started throwing alarms due to a task overflow. The system restarted and was able to recover itself and allowing the mission to continue safely.

Debugging and post-mortem analysis (forensics) are essential skills for embedded developers.

Common Debugging Tools and Techniques

1. Print Statements (UART Logging)

The simplest and most common method. They send debug messages over a serial connection (UART).

You can use printf() or similar to track variable values, function entries/exits, and system state

• Pros: Easy to implement

• Cons: Can affect timing – not usable if UART is unavailable or disabled

2. Trace Variables

In systems without output peripherals (like UART), you can use trace flags, setting bits in a global variable to indicate code progress.

uint32_t trace_flags = 0;

void init_sensor() 
{
    trace_flags |= (1 << 0); // Bit 0: sensor init started
    // ...
    trace_flags |= (1 << 1); // Bit 1: sensor init complete
}

You can then examine trace_flags in memory to track execution flow, even post-mortem. The trace flags can be printed out or dumped via lldb or gdb.

3. Hardware Debugging: JTAG, SWD, and Debuggers

Modern microcontrollers (like ARM Cortex-Ms) support hardware debugging interfaces such as:

• JTAG (Joint Test Action Group)

• SWD (Serial Wire Debug)

These allow a debugger to:

• Pause execution

• Set breakpoints

• Inspect and modify memory

• Single-step through code

ARM CoreSight is a debug and trace architecture developed by ARM for its processor cores (like Cortex-M, Cortex-A, Cortex-R). It provides a set of hardware modules built into ARM-based chips that allow developers to:

• Debug the system while it's running (non-intrusively)

• Trace code execution, memory accesses, and peripheral activity

• Analyze system performance and find hard-to-catch bugs

In short: CoreSight lets you look inside your embedded system while it's alive and working, without halting it unnecessarily.

Why CoreSight Exists

Traditional debugging tools (like breakpoints or single-stepping with JTAG) are often intrusive (they pause the system), limited (can't capture what happened right before a crash), or not suitable for real-time systems.

CoreSight solves these by enabling real-time tracing and non-intrusive observation of what's happening inside the chip.

Popular Debug Tools:

• ST-Link – HW from STMicrocontrollers

• J-Link – Universal debugger supporting a wide range of MCUs

• OpenOCD – Open-source interface for hardware debugging

• GDB / LLDB – Command-line debuggers used alongside the above

Single-stepping is most effective when compiler optimizations are off. With optimization, code might be reordered, inlined, or even eliminated.

4. Using Map and Disassembly Files

When debugging complex issues, especially crashes or memory overflows, you'll need to go deeper.

Map Files show the layout of functions and variables in memory (Flash and RAM). They help you locate:

• Stack overflows

• Unexpected memory usage

• Function addresses

Disassembly Files let you see the machine code generated from your source. This is critical when:

• Code is heavily optimized

• You’re diagnosing instruction-level failures

• You’re working without source code (e.g., binary-only drivers)

This resource provides a good overview on Map files, linkers and ELF format: Tenouk’s ELF/Map/Linker Guide

Common Bug: Buffer Overflows

Buffer overflows are one of the most frequent (and dangerous) issues in embedded systems. They happen when data is written past the end of an allocated array, overwriting nearby memory and causing unpredictable behavior.

Symptoms:

• Code crashes mysteriously

• Data appears to “corrupt itself”

• Variables change value without explanation

You can learn more in my article on Debugging Buffer Overflows, which walks through ways to debug a buffer overflow and build robust buffer code.

Embedded Forensics

Sometimes, a device fails in the field, where you can’t attach a debugger. That’s where forensics comes in:

• Use watchdog timers to reset the system and log failure info

• Save crash signatures to non-volatile memory (for example, EEPROM, Flash)

• Implement assert handlers that log file names, line numbers, or fault types

These techniques help you reconstruct what went wrong after the device has rebooted or been recovered.

You can learn more here: Debugging Techniques for Embedded Systems – Medium.

Debugging and forensics are invaluable when something goes wrong – but a robust system should aim to catch issues before they reach deployment.

That’s where automated testing becomes essential. With embedded software increasingly powering critical applications, the ability to run consistent, repeatable tests across hardware configurations saves time, improves reliability, and enables faster development cycles.

Next, let’s explore how embedded testing works, the challenges unique to hardware, and how automation frameworks help streamline validation.

Automation and Testing in Embedded Systems

Like all other areas of software engineering, testing is essential in embedded systems. But testing embedded software comes with its own set of challenges, mainly because it interacts with hardware.

Manual testing can be time-consuming and resource-intensive, especially when tests need to be repeated for multiple firmware versions or configurations. That’s where automated testing becomes invaluable.

Why Automated Testing?

Automated testing helps:

• Catch regressions early

• Test edge cases consistently

• Reduce human error

• Scale testing across versions and hardware setups

But automating tests for embedded systems isn’t just writing test cases – it’s about setting up an infrastructure that connects your code to the physical hardware under test.

Test Architecture: Host + DUT

Most embedded test setups involve two components:

• Host: Your development PC or CI test controller, which sends test commands and receives data.

• DUT (Device Under Test): The microcontroller board or embedded system running the firmware.

These two communicate over a physical link, commonly USB, UART, or FTDI, which carries commands and test data between them.

Diagram (suggested structure)

You could visualize this as:

Key Components of Embedded Test Automation

1. File Management

Many automated tests rely on CSV or JSON files to define:

• Input configurations

• Expected outputs

• Test parameters

Python makes it easy to:

• Read input vectors from CSVs

• Write logs or pass/fail results

• Parse structured data

2. Data Communication

Maintaining a stable and reliable link between the Host and DUT is critical. This includes:

• Opening and managing UART or USB connections (for example, with pyserial)

• Framing test commands using opcodes or simple protocols

• Handling timeouts, retries, and error recovery

Example (Python with PySerial):

import serial

ser = serial.Serial('/dev/ttyUSB0', 115200) #set Baud rate
ser.write(b'\x01')  # Send opcode for "start test"
response = ser.read(64)  # Read 64 bytes of response

3. Automation Manager (DUT-side)

A lightweight software agent runs on the embedded device. Its responsibilities:

• Parse incoming commands

• Trigger specific test routines

• Send response data back to the host

This is often implemented using a switch-case structure in C or C++:

void automation_manager(uint8_t opcode) {
    switch(opcode) {
        case 0x01: run_sensor_test(); break;
        case 0x02: run_motor_test(); break;
        default: break;
    }
}

4. Automation Manager (Host-side)

This is the control center of your test workflow:

• Sends test commands and parameters to the DUT

• Waits for and logs results

• Compares responses to expected output

• Handles communication retries or failures

Often written in Python using:

• pyserial for communication

• pandas for file/data processing

• unittest or pytest for test structure

Tips for Effective Automation

• Use unique opcodes for each test command to avoid ambiguity

• Implement timeout handling to avoid hanging scripts

• Log everything, responses, errors, test timestamps

• Use versioned test input files to track changes over time

• Include self-tests on the DUT to validate hardware state before running full tests

Automated testing in embedded systems is not just about running scripts, it's about building a bridge between your host PC and your device, managing the flow of commands and data, and ensuring tests are consistent, repeatable, and reliable.

While this requires effort to set up, the payoff is huge: confidence in your firmware, faster development cycles, and reduced risk of bugs making it into production.

Where to Go from Here

Building your Embedded Project

After exploring the theory and tooling of embedded systems, it's time to apply what you've learned. This section walks you through the steps to create your own embedded system – from concept to code and deployment.

Use the checklist below to guide your first project, whether you're prototyping a sensor device or automating a simple process.

Project Setup Checklist:

1. Define the Goal

   • What task does the system perform?

   • Identify inputs (for example, temperature sensor) and outputs (for example, relay or LED).

2. Requirements Gathering

   • Functional: What features must it support?

   • Non-functional: Memory limits, real-time behavior, power constraints.

   • Any security or safety-critical elements?

3. Choose Your Hardware

   • Microcontroller (for example, STM32F4)

   • Sensors and actuators

   • Communication interfaces (UART, I2C, SPI, and so on)

4. Software Architecture

   • Bare-metal, RTOS, or embedded OS?

   • Driver abstraction: will you use HAL or custom low-level code?

   • Organize code into layers: application logic, drivers, hardware init.

5. Toolchain Setup

   • Install GCC toolchain (for example, arm-none-eabi-gcc)

   • Configure Makefile and linker script

   • Set up debugger and flashing tools (for example, OpenOCD, ST-Link)

6. Firmware Implementation

   • Initialize peripherals

   • Implement control logic inside main() or tasks

   • Use interrupts or timers for responsiveness

7. Flashing and Initial Tests

   • Use OpenOCD or ST-Link to flash the binary

   • Test peripheral behavior and debug with UART or GDB

8. Debug and Profile

   • Use JTAG/SWD, CoreSight, and trace logs

   • Check memory layout with map/disassembly files

   • Identify bottlenecks and edge cases

9. Security Hardening

   • Disable debug interfaces post-flash

   • Add firmware signing and secure boot

   • Minimize surface area: disable unused features

10. Testing and Automation

• Connect Host to DUT via UART/USB

• Use Python + PySerial to send test vectors

• Log, compare, and report test outcomes

Embedded firmware development is a deep and rewarding field where software meets the hardware. Whether you're controlling an LED, reading from a sensor, or orchestrating multiple tasks in real time, the embedded stack teaches you how hardware, software, timing, and efficiency all come together.

Summary:

In this guide, we walked through the essential building blocks at a high level:

• What embedded systems are, and how they sense → process → act

• How microcontrollers work, from memory layout to interrupts and protocols

• How to design robust, scalable embedded software with clean architecture

• When to choose bare-metal, RTOS, or full OS solutions

• How to build drivers, write modular code, and interface with peripherals

• Tools for debugging, tracing, and analyzing system behavior

• Strategies for automating embedded testing using Python and host-device communication

• And finally, why security matters, especially in a connected world

Whether you're preparing for embedded job interviews, building your own IoT projects, or just exploring how software drives real-world systems, this article gives you a launchpad for deeper learning.

A buffer overflow happens when a program writes more data into a buffer than it was allocated, leading to memory corruption, crashes, or even security vulnerabilities. A buffer corruption occurs when unintended modifications overwrite unread data or modify memory in unexpected ways.

In safety-critical systems like cars, medical devices, and spacecraft, buffer overflows can cause life-threatening failures. Unlike simple software bugs, buffer overflows are unpredictable and depend on the state of the system, making them difficult to diagnose and debug.

To prevent these issues, it's important to understand how buffer overflows and corruptions occur, and how to detect and fix them.

Article Scope

In this article, you will learn:

1. What buffers, buffer overflows, and corruptions are. I’ll give you a beginner-friendly explanation with real-world examples.

2. How to debug buffer overflows. You’ll learn how to use tools like GDB, LLDB, and memory maps to find memory corruption.

3. How to prevent buffer overflows. We’ll cover some best practices like input validation, safe memory handling, and defensive programming.

I’ll also show you some hands-on code examples – simple C programs that demonstrate buffer overflow issues and how to fix them.

What this article doesn’t cover:

1. Security exploits and hacking techniques. We’ll focus on preventing accidental overflows, not hacking-related buffer overflows.

2. Operating system-specific issues. This guide is for embedded systems, not general-purpose computers or servers.

3. Advanced RTOS memory management. While we discuss interrupt-driven overflows, we won’t dive deep into real-time operating system (RTOS) concepts.

Now that you know what this article covers (and what it doesn’t), let’s go over the skills that will help you get the most out of it.

Prerequisites

This article is designed for developers who have some experience with C programming and want to understand how to debug and prevent buffer overflows in embedded systems. Still, beginners can follow along, as I’ll explain key concepts in a clear and structured way.

Before reading, it helps if you know:

1. Basic C programming.

2. How memory works – the difference between stack, heap, and global variables.

3. Basic debugging concepts – if you’ve used a debugger like GDB or LLDB, that’s a plus, but not required.

4. What embedded systems are – a basic idea of how microcontrollers store and manage memory.

Even if you’re not familiar with these topics, this guide will walk you through them in an easy-to-understand way.

Before you dive into buffer overflows, debugging, and prevention, let’s take a step back and understand what a buffer is and why it’s important in embedded systems. Buffers play a crucial role in managing data flow between hardware and software but when handled incorrectly, they can lead to serious software failures.

Table of Contents

• What is a Buffer, and How Does it Work?

• What is a Buffer Overflow?

• Common Causes of Buffer Overflows and Corruption

• Consequences of Buffer Overflows

• How to Debug Buffer Overflows

• How to Prevent Buffer Overflows

• Conclusion

What is a Buffer, and How Does it Work?

A buffer is a contiguous block of memory used to temporarily store data before it is processed. Buffers are commonly used in two scenarios:

1. Data accumulation: When the system needs to collect a certain amount of data before processing.

2. Rate matching: When the data producer generates data faster than the data consumer can process it.

Buffers are typically implemented as arrays in C, where elements are indexed from 0 to N-1 (where N is the buffer size).

Let’s look at an example of a buffer in a sensor system.

Consider a system with a sensor task that generates data at 400 Hz (400 samples per second or 1 sample every 2.5 ms). But the data processor (consumer) operates at only 100 Hz (100 samples per second or 1 sample every 10 ms). Since the consumer task is slower than the producer, we need a buffer to store incoming data until it is processed.

To determine the buffer size, we calculate:

Buffer Size = Time to consume 1 sample / Time to generate 1 sample = 10 ms/ 2.5 ms = 4

This means the buffer must hold at least 4 samples at a time to avoid data loss.

Once the buffer reaches capacity, there are several strategies to decide which data gets passed to the consumer task:

1. Max/min sampling: Use the maximum or minimum value in the buffer.

2. Averaging: Compute the average of all values in the buffer.

3. Random access: Pick a sample from a specific location (for example, the most recent or the first).

In real-world applications, it’s beneficial to use circular buffers or double buffering to prevent data corruption.

• Circular buffer approach: A circular buffer (also called a ring buffer) continuously wraps around when it reaches the end, ensuring old data is overwritten safely without exceeding memory boundaries. The buffer size should be multiplied by 2 (4 × 2 = 8) to hold 8 samples. This allows the consumer task to process 4 samples while the next 4 samples are being filled, preventing data overwrites.

• Double buffer approach: Double buffering is useful when data loss is unacceptable. It allows continuous data capture while the processor is busy handling previous data. A second buffer of the same size is added. When the first buffer is full, the write pointer switches to the second buffer, allowing the consumer task to process data from the first buffer while the second buffer is being filled. This prevents data overwrites and ensures a continuous data flow.

Buffers help manage data efficiently, but what happens when they are mismanaged? This is where buffer overflows and corruptions come into play.

What is a Buffer Overflow?

A buffer overflow occurs when a program writes more data into a buffer than it was allocated, causing unintended memory corruption. This can lead to unpredictable behavior, ranging from minor bugs to critical system failures.

To understand buffer overflow, let's use a simple analogy. Imagine a jug with a tap near the bottom. The jug represents a buffer, while the tap controls how much liquid (data) is consumed.

The jug is designed to hold a fixed amount of liquid. As long as water flows into the jug at the same rate or slower than it flows out, everything works fine. But if water flows in faster than it flows out, the jug will eventually overflow.

Similarly, in software, if data enters a buffer faster than it is processed, it exceeds the allocated memory space, causing a buffer overflow. In the case of a circular buffer, this can cause the write pointer to wrap around and overwrite unread data, leading to buffer corruption.

Buffer Overflows in Software

Unlike the jug, where water simply spills over, a buffer overflow in software overwrites adjacent memory locations. This can cause a variety of hard-to-diagnose issues, including:

1. Corrupting other data stored nearby.

2. Altering program execution, leading to crashes.

3. Security vulnerabilities, where attackers exploit overflows to inject malicious code.

When a buffer overflow occurs, data can overwrite variables, function pointers, or even return addresses, depending on where the buffer is allocated.

Buffer overflows can occur in different memory regions:

1. Buffer overflows in global/static memory (.bss / .data sections)

   • These occur when global or static variables exceed their allocated size.

   • The overflow can corrupt adjacent variables, leading to unexpected behavior in other modules.

   • Debugging is easier because memory addresses are fixed at compile time unless the compiler optimizes them. Map files provide a memory layout of variables during the compilation and linking.

2. Stack-based buffer overflow (more predictable, easier to debug):

   • Happens when a buffer is allocated in the stack (for example, local variables inside functions).

   • Overflowing the stack can affect adjacent local variables or return addresses, potentially crashing the program.

   • In embedded systems with small stack sizes, this often leads to a crash or execution of unintended code.

3. Heap-based buffer overflow (harder to debug):

   • Happens when a buffer is dynamically allocated in the heap (for example, using malloc() in C).

   • Overflowing a heap buffer can corrupt adjacent dynamically allocated objects or heap management structures.

   • Debugging is harder because heap memory is allocated dynamically at runtime, causing memory locations to vary.

Buffer Overflow vs Buffer Corruption

Buffer overflow and buffer corruption are of course related, but refer to different situations.

A buffer overflow happens when data is written beyond the allocated buffer size, leading to memory corruption, unpredictable behavior, or system crashes.

A buffer corruption happens when unintended data modifications result in unexpected software failures, even if the write remains within buffer boundaries.

Both issues typically result from poor write pointer management, lack of boundary checks, and unexpected system behavior.

Now that we've covered what a buffer overflow is and how it can overwrite memory, let’s take a closer look at how these issues affect embedded systems.

In the next section, we’ll explore how buffer overflows and corruption happen in real-world embedded systems and break down common causes, including pointer mismanagement and boundary violations.

Common Causes of Buffer Overflows and Corruption

Embedded systems use buffers to store data from sensors, communication interfaces (like UART (Universal Asynchronous Receiver-Transmitter), SPI (Serial Peripheral Interface), I2C (Inter-integrated Circuit), and real-time tasks. These buffers are often statically allocated to avoid memory fragmentation, and many implementations use circular (ring) buffers to efficiently handle continuous data streams.

Here are three common scenarios where buffer overflows or corruptions occur in embedded systems:

Writing Data Larger Than the Available Space

Issue: The software writes incoming data to the buffer without checking if there is enough space.

Example: Imagine a 100-byte buffer to store sensor data. The buffer receives variable-sized packets. If an incoming packet is larger than the remaining space, it will overwrite adjacent memory, leading to corruption.

So why does this happen?

• Some embedded designs increment the write pointer after copying data, making it too late to prevent overflow.

• Many low-level memory functions (memcpy, strcpy, etc.) do not check buffer boundaries, leading to unintended writes.

• Without proper bound checking, a large write can exceed the buffer size and corrupt nearby memory.

Here’s a code sample to demonstrate buffer overflow in a .bss / .data section:

  #include <stdint.h>
  #include <stdio.h>
  #include <string.h>

  #define BUFFER_SIZE 300

  static uint16_t sample_count = 0;
  static uint8_t buffer[BUFFER_SIZE] = {0};

  // Function to simulate a buffer overflow scenario
  void updateBufferWithData(uint8_t *data, uint16_t size)
  {
      // Simulating a buffer overflow: No boundary check!
      printf("Attempting to write %d bytes at position %d...\n", size, sample_count);

      // Deliberate buffer overflow for demonstration
      if (sample_count + size > BUFFER_SIZE)
      {
          printf("WARNING: Buffer Overflow Occurred! Writing beyond allocated memory!\n");
      }

      // Copy data (unsafe, can cause overflow)
      memcpy(&buffer[sample_count], data, size);

      // Increment sample count (incorrectly, leading to wraparound issues)
      sample_count += size;
  }

  int main()
  {   
      // Save 1 byte to buffer
      uint8_t data_to_buffer = 10;
      updateBufferWithData(&data_to_buffer, 1);

      // Save an array of 20 bytes to buffer
      uint8_t data_to_buffer_1[20] = {5};
      updateBufferWithData(data_to_buffer_1, sizeof(data_to_buffer_1));

      // Intentional buffer overflow: Save an array of 50 x 8 bytes (400 bytes)
      uint64_t data_to_buffer_2[50] = {7};
      updateBufferWithData((uint8_t*)data_to_buffer_2, sizeof(data_to_buffer_2));

      return 0;
  }

Interrupt-Driven Overflows (Real-time Systems)

Issue: The interrupt service routine (ISR) may write data faster than the main task can process, leading to buffer corruption or buffer overflow if the write pointer is not properly managed.

Example: Imagine a sensor ISR that writes incoming data into a buffer every time a new reading arrives. Meanwhile, a low-priority processing task reads and processes the data.

What can go wrong?

• If the ISR triggers too frequently (due to a misbehaving sensor or high interrupt priority), the buffer may fill up faster than the processing task can keep up.

• This can result in one of two failures:

  1. Buffer Corruption: The ISR overwrites unread data, leading to loss of information.

  2. Buffer Overflow: The ISR exceeds buffer boundaries, causing memory corruption or system crashes.

So why does this happen?

• In real-time embedded systems, ISR execution preempts lower-priority tasks.

• If the processing task doesn't not get enough CPU time, the buffer may become overwritten or overflow beyond its allocated scope.

System State Changes & Buffer Corruption

Issue: The system may unexpectedly reset, enter low-power mode, or changes operating state, leaving the buffer write pointers in an inconsistent state. This can result in buffer corruption (stale or incorrect data) or buffer overflow (writing past the buffer’s limits.

Example Scenarios:

1. Low-power wake-up issue (Buffer Overflow risk): Some embedded systems enter deep sleep to conserve energy. Upon waking up, if the buffer write pointer is not correctly reinitialized, it may point outside buffer boundaries, leading to buffer overflow and unintended memory corruption.

2. Unexpected mode transitions: If a sensor task is writing data and the system suddenly switches modes, the buffer states and pointers may not be cleaned up. The next time the sensor task runs, it may continue writing without clearing previous data. This can cause undefined behavior due to presence of stale data.

Now that you understand how buffer overflows and corruptions happen, let’s examine their consequences in embedded systems ranging from incorrect sensor readings to complete system failures, making debugging and prevention critical.

Consequences of Buffer Overflows

Buffer overflows can be catastrophic in embedded systems, leading to system crashes, data corruption, and unpredictable behavior. Unlike general-purpose computers, many embedded devices lack memory protection, making them particularly vulnerable to buffer overflows.

A buffer overflow can corrupt two critical types of memory:

1. Data Variables Corruption

A buffer overflow can overwrite data variables, corrupting the inputs for other software modules. This can cause unexpected behavior or even system crashes if critical parameters are modified.

For example, a buffer overflow could accidentally overwrite a sensor calibration value stored in memory. As a result, the system would start using incorrect sensor readings, leading to faulty operation and potentially unsafe conditions.

2. Function Pointer Corruption

In embedded systems, function pointers are often used for interrupt handlers, callback functions, and RTOS task scheduling. If a buffer overflow corrupts a function pointer, the system may execute unintended instructions, leading to a crash or unexpected behavior.

As an example, a function pointer controlling motor speed regulation could be overwritten. Instead of executing the correct function, the system would jump to a random memory address, causing a system fault or erratic motor behavior.

Buffer overflows are among the hardest bugs to identify and fix because their effects depend on which data is corrupted and the values it contains. A buffer overflow can affect memory in different ways:

• If a buffer overflow corrupts unused memory, the system may seem fine during testing, making the issue harder to detect.

• if a buffer overflow alters critical data variables, it can cause hidden logic errors that cause unpredictable behavior.

• If a buffer overflow corrupts function pointers, it may crash immediately, making the problem easier to identify.

During development, if tests focus only on detecting crashes, they may overlook silent memory corruption caused by a buffer overflow. In real-world deployments, new use cases not covered in testing can trigger previously undetected buffer overflow issues, leading to unpredictable failures.

Buffer overflows can cause a chain reaction, where one overflow leads to another overflow or buffer corruption, resulting in widespread system failures. So how does this happen?

1. A buffer overflow corrupts a critical variable (for example, a timer interval).

2. The corrupted variable disrupts another module (for example, triggers the timer interrupt too frequently, causing it to push more data into a buffer than intended.).

3. This increased interrupt frequency forces a sensor task to write data faster than intended, eventually causing another buffer overflow or corruption by overwriting unread data.

This chain reaction can spread across multiple software modules, making debugging nearly impossible. In real-word applications, buffer overflows in embedded systems can be life-threatening:

• In cars: A buffer overflow in an ECU (Electronic Control Unit) could cause brake failure or unintended acceleration.

• In a spacecraft: A memory corruption issue could disable navigation systems, leading to mission failure.

Now that we’ve seen how buffer overflows can corrupt memory, disrupt system behavior, and even cause critical failures, the next step is understanding how to detect and fix them before they lead to serious issues.

How to Debug Buffer Overflows

Debugging buffer overflows in embedded systems can be complex, as their effects range from immediate crashes to silent data corruption, making them difficult to trace. A buffer overflow can cause either:

1. A system crash, which is easier to detect since it halts execution or forces a system reboot.

2. Unexpected behavior, which is much harder to debug as it requires tracing how corrupted data affects different modules.

This section focuses on embedded system debugging techniques using memory map files, debuggers (GDB/LLDB), and a structured debugging approach. Let’s look into the debuggers and memory map files.

Memory Map File (.map file)

A memory map file is generated during the linking process. It provides a memory layout of global/static variables, function addresses, and heap/stack locations. It provides a memory layout of Flash and RAM, including:

• Text section (.text): Stores executable code.

• Read-only section (.rodata): Stores constants and string literals.

• BSS section (.bss): Stores uninitialized global and static variables.

• Data section (.data): Stores initialized global and static variables.

• Heap and stack locations, depending on the linker script.

If a buffer overflow corrupts a global variable, the .map file can identify nearby variables that may also be affected, provided the compiler has not optimized the memory allocation. Similarly, if a function pointer is corrupted, the .map file can reveal where it was stored in memory.

Debuggers (GDB & LLDB)

Debugging tools like GDB (GNU Debugger) and LLDB (LLVM Debugger) allow:

• Controlling execution (breakpoints, stepping through code).

• Inspecting variable values and memory addresses.

• Getting backtraces (viewing function calls before a crash).

• Extracting core dumps from microcontrollers for post-mortem analysis.

If the system halts on a crash, a backtrace (bt command in GDB) can reveal which function was executing before failure. If the overflow affects a heap-allocated variable, GDB can inspect heap memory usage to detect corruption.

The Debugging Process

Now, let’s go through a step-by-step debugging process to identify and fix buffer overflows. Once a crash or unexpected behavior occurs, follow these techniques to trace the root cause:

Step 1: Identify the misbehaving module

If the system crashes, use GDB or LLDB backtrace (bt command) to locate the last executed function. If the system behaves unexpectedly, determine which software module controls the affected functionality.

Step 2: Analyze inputs and outputs of the module

Every function or module has inputs and outputs. Create a truth table listing expected outputs for all possible inputs. Check if the unexpected behavior matches any undefined input combination, which may indicate corruption.

Step 3: Locate memory corruption using address analysis

If a variable shows incorrect values, determine its physical memory location. Depending on where the variable is stored:

1. Global/static variables (.bss / .data): Look up the memory map file for nearby buffers.

2. Heap variables: Snapshot heap allocations using GDB.

   Here’s an example of using GDB to find corrupted variables:

    (gdb) print &my_variable  # Get memory address of the variable
    $1 = (int *) 0x20001000
    (gdb) x/10x 0x20001000   # Examine memory near this address, Display 10 memory words in hexadecimal format starting from 0x20001000
   

Step 4: Identify the overflowing buffer

If a buffer is located just before the corrupted variable, inspect its usage in the code. Review all possible code paths that write to the buffer. Check if any design limitations could cause an overflow under a specific use cases.

Step 5: Fix the root cause

If the buffer overflow happened due to missing bounds checks, add proper input validation to prevent it. Buffer design should enforce strict memory limits. The module should implement strict boundary checks for all inputs and maintain a consistent state.

In addition to GDB/LLDB, you can also use techniques like hardware tracing and fault injection to simulate buffer overflows and observe system behavior in real-time.

While debugging helps identify and fix buffer overflows, prevention is always the best approach. Let’s explore techniques that can help avoid buffer overflows altogether.

How to Prevent Buffer Overflows

You can often prevent buffer overflows through good software design, defensive programming, hardware protections, and rigorous testing. Embedded systems, unlike general-purpose computers, often lack memory protection mechanisms, which means that buffer overflow prevention critical for system reliability and security.

Here are some key techniques to help prevent buffer overflows:

Defensive Programming

Defensive programming helps minimize buffer overflow risks by ensuring all inputs are validated and unexpected conditions are handled safely.

First, it’s crucial to validate input size before writing to a buffer. Always check the write index by adding the size of data to be written prior to writing data to make sure more data is not written than the available buffer space.

Then you’ll want to make sure you have proper error handling and fail-safe mechanisms in place. If an input is invalid, halt execution, log the error, or switch to a safe state. Also, functions should indicate success/failure with helpful error codes to prevent misuse.

Sample Code:

   #include <stdint.h>
   #include <string.h>
   #include <stdbool.h>
   #include <stdio.h>

   #define BUFFER_SIZE 300

   static uint16_t sample_count = 0;
   static uint8_t buffer[BUFFER_SIZE] = {0};

   typedef enum
   {
       SUCCESS = 0,
       NOT_ENOUGH_SPACE = 1,
       DATA_IS_INVALID = 2,
   } buffer_err_code_e;


   buffer_err_code_e updateBufferWithData(uint8_t *data, uint16_t size)
   {
       if (data == NULL || size == 0 || size > BUFFER_SIZE)  
       {
           return DATA_IS_INVALID; // Invalid input size
       }

       uint16_t available_space = BUFFER_SIZE - sample_count;
       bool can_write = (available_space >= size) ? true : false;

       if (!can_write)  
       {
           return NOT_ENOUGH_SPACE;
       }

       // Copy data safely
       memcpy(&buffer[sample_count], data, size);
       sample_count += size;

       return SUCCESS;
   }

   int main()
   {   
       buffer_err_code_e ret;

       // Save 1 byte to buffer
       uint8_t data_to_buffer = 10;
       ret = updateBufferWithData(&data_to_buffer, sizeof(data_to_buffer));
       if (ret)  
       {
           printf("Buffer update didn't succeed, Err:%d\n", ret);
       }

       // Save an array of 20 bytes to buffer
       uint8_t data_to_buffer_1[20] = {5};
       ret = updateBufferWithData(data_to_buffer_1, sizeof(data_to_buffer_1));
       if (ret)  
       {
           printf("Buffer update didn't succeed, Err:%d\n", ret);
       }

       // Save an array of 50 x 8 bytes, Intentional buffer overflow
       uint64_t data_to_buffer_2[50] = {7};
       ret = updateBufferWithData((uint8_t*)data_to_buffer_2, sizeof(data_to_buffer_2));  
       if (ret)  
       {
           printf("Buffer update didn't succeed, Err:%d\n", ret);
       }

       return 0;
   }

Choosing the Right Buffer Design And Size

Some buffer designs handle overflow better than others. Choosing the correct buffer type and size for the application reduces the risk of corruption.

• Circular Buffers (Ring Buffers) prevent out-of-bounds writes by wrapping around. They overwrite the oldest data instead of corrupting memory. These are useful for real-time streaming data (for example, UART, sensor readings). This approach is ideal for applications where data loss is unacceptable.

• Ping-Pong Buffers (Double Buffers) use two buffers. One buffer fills up with data. Then, once it’s full, it switches to the second buffer while the first one is processed. This approach is beneficial for application that have strict requirements on no data loss. The buffer design should be based on the speed of write and read tasks.

Hardware Protection

Memory Protection Unit (MPU)

An MPU (Memory Protection Unit) helps detect unauthorized memory accesses, including buffer overflows, by restricting which regions of memory can be written to. It prevents buffer overflows from modifying critical memory regions and triggers a MemManage Fault if a process attemps to write outside an allowed region.

But keep in mind that, an MPU does not prevent buffer overflows – it only detects and stops execution when they occur. Not all microcontrollers have an MPU, and some low-end MCUs lack hardware protection, making software-based safeguards even more critical.

Modern C compilers provide several flags to identify memory errors at compile-time:

1. -Wall -Wextra: Enables useful warnings

2. -Warray-bounds: Detects out-of-bounds array access when the array size is known at compile-time

3. -Wstringop-overflow: Warns about possible overflows in string functions like memcpy and strcpy.

Testing and Validation

Testing helps detect buffer overflows before deployment, reducing the risk of field failures. Unit testing each function independently with valid inputs, boundary cases, and invalid inputs helps detect buffer-related issues early. Automated testing involves feeding random and invalid inputs into the system to uncover crashes and unexpected behavior. Static Analysis Tools like Coverity, Clang Static Analyzer help detect buffer overflows before runtime. Run real-world inputs on embedded hardware to detect issues.

Now that we've explored how to identify, debug, and prevent buffer overflows, it’s clear that these vulnerabilities pose a significant threat to embedded systems. From silent data corruption to catastrophic system failures, the consequences can be severe.

But with the right debugging tools, systematic analysis, and preventive techniques, you can effectively either prevent or mitigate buffer overflows in your systems.

Conclusion

Buffer overflows and corruption are major challenges in embedded systems, leading to crashes, unpredictable behavior, and security risks. Debugging these issues is difficult because their symptoms vary based on system state, requiring systematic analysis using memory map files, GDB/LLDB, and structured debugging approaches.

In this article, we explored:

• The causes and consequences of buffer overflows and corruptions

• How to debug buffer overflows using memory analysis and debugging tools

• Best practices for prevention

Buffer overflow prevention requires a multi-layered approach:

1. Follow a structured software design process to identify risks early.

2. Apply defensive programming principles to validate inputs and handle errors gracefully.

3. Use hardware-based protections like MPUs where available.

4. Enable compiler flags that help identify memory errors.

5. Test extensively, unit testing, automated testing, and code reviews help catch vulnerabilities early.

By implementing these best practices, you can minimize the risk of buffer overflows in embedded systems, improving reliability and security.

In embedded systems, where reliability and safety are critical, preventing buffer overflows is not just a best practice, it is a necessity. A single buffer overflow can compromise an entire system. Defensive programming, rigorous testing, and hardware protections are essential for building secure and robust embedded applications.

The first is general-purpose computing, where they handle a wide range of tasks, including running diverse applications and programs. Examples include laptops, desktops, servers, and supercomputers.

The second is embedded systems, which are specialized computers designed for specific functions. Commonly found in devices such as thermostats, refrigerators, cars, and other smart appliances, they rely on sensors to collect environmental data and execute their tasks efficiently.

The Role of Sensors

Sensors play a critical role in both types of computing. In embedded systems, sensors gather environmental data to help devices like autonomous vehicles, home appliances, and industrial machines perform tasks. In general-purpose computers, sensors primarily monitor internal conditions such as temperature and voltage, ensuring safe operation and preventing issues like overheating or electrical faults.

As Artificial Intelligence (AI) and the Internet of Things (IoT) evolve, sensors have become indispensable for gathering real-world data to support intelligent decision-making. Embedded systems leverage sensors to perceive their environment, transforming raw data into actionable insights that power automation and improve efficiency across industries.

This means that understanding sensor interfacing and designing robust sensor-driven software has become a vital skill for engineers and hobbyists alike.

Whether you're a beginner or experienced engineer, this guide will help you build a solid understanding of sensor interfacing software.

What You’ll Learn and Article Scope

In this article, you’ll learn how to connect sensors to microcontrollers (MCUs) and design sensor software pipelines that turn raw data into meaningful, usable information. You’ll also explore practical techniques for processing sensor data accurately and efficiently in embedded systems.

Here’s a breakdown of what we’ll cover:

• What sensors are and how they work – An introduction to sensors, common types, and how sensor pipelines help process sensor data.

• Key sensor characteristics – Important parameters like sensitivity, accuracy, precision, range, drift, and response time to help you choose the right sensor for your project.

• How to interface sensors with microcontrollers – Hardware connections and communication protocols like SPI, I²C, and GPIO that allow microcontrollers to read sensor data.

• Software architecture for sensor data – A high-level overview of the software pipeline that processes sensor data, including drivers, ADC support, scaling, calibration, and post-processing.

• Detailed design of pipeline components – A closer look at each step in the pipeline, focusing on scaling raw data, calibrating sensors, and applying filters to clean up noisy signals.

• Practical tips for power management – Best practices for handling power efficiently using low-power modes, FIFO buffers, and DMA when working with sensor data in embedded systems.

By the end of this article, you’ll know how to design and implement a complete sensor data pipeline for an embedded system, from reading raw sensor data to preparing it for real-world use in intelligent, connected devices.

Note: Advanced data processing, high-resolution ADCs, and hardware circuit design for sensors are outside the scope of this article.

Prerequisites

To get the most out of this article, you should have:

1. Basic knowledge of microcontrollers: Understanding of common peripherals like ADCs (Analog-to-Digital Converters), SPI (Serial Peripheral Interface), I2C (Inter-Integrated Circuit) and GPIO (General Purpose Input/Output). If you’re new to these protocols, this article provides a great overview.

2. Basic knowledge of electronics: Familiarity with circuits and signals, including analog and digital interfaces.

3. Programming in C: Familiarity in embedded software development, including driver development.

4. (Optional) Basic knowledge of sensors: Understanding different types of sensors (like temperature, pressure, motion) is helpful but not required.

Also, this article assumes the following:

• You are working with a microcontroller equipped with the peripherals needed for sensor integration. The details of microcontroller peripherals can be found in a reference manual for example for an STM32F4 series microcontroller will have all the details :

• You are familiar with compilers, debuggers, and IDEs used in embedded systems. Some common tools include:

  • Compilers: GCC, Clang,

  • Debuggers: GDB, LLDB

  • IDEs: Visual Studio Code (VSCode) is a popular choice, especially with extensions for embedded development and debugging.

• You aim to build reliable, sensor-driven embedded systems, capable of collecting and processing real-world data efficiently.

Table of Contents

• What is a Sensor and Sensor Pipeline?

• Sensor Characteristics

• How to Interface with a Microcontroller

• Software Architecture

  • High-Level Overview of Components

  • Accessing Data from the Sensor

  • Sensor Power Management

• Detailed Design of Components

  • 1. Sensor Driver

  • 2. ADC Support

  • 3. Scaling

  • 4. Calibration

  • 5. Data Post-Processing

• Conclusion

What is a Sensor and Sensor Pipeline?

A sensor detects changes in physical properties such as temperature, pressure, or light and converts them into electrical signals that can be measured or interpreted. For example, a thermistor is a type of resistor whose resistance changes with temperature. As the temperature varies, the resistance of the thermistor changes, altering the voltage across it. The system then interprets this voltage change to determine the temperature.

To better understand sensors, consider the natural sensors in the human body: the eyes, ears, skin, nose, and tongue. These natural sensors constantly send signals about the environment to the brain for processing. Different regions of the brain interpret these signals and use the information to drive actions and responses. Just like the brain processes signals from natural sensors, a microcontroller processes signals from electronic sensors using a sensor pipeline.

Sensors come in many types, each designed to detect specific physical properties. Some sensors have a sensing element that changes its properties in response to conditions like heat, light, or pressure. Examples include thermistors, infrared receivers, and photodiodes.

For detecting movement, such as acceleration and rotation, MEMS (Microelectromechanical Systems) sensors—like accelerometers and gyroscopes—are widely used.

To measure distance, sensors like sonars, ultrasonic sensors, and radars are common. These are just a few examples of the many types of sensors available.

Beyond the types of physical properties they detect, sensors also differ in their levels of integration. Some sensors are raw sensors, consisting only of a sensing element and a transducer with simple leads for direct connection to an external circuit.

Others, known as smart sensors, include additional components such as an ADC (analog-to-digital converter) and onboard processing capabilities, enabling them to handle more of the data processing independently.

The choice between a raw sensor and a smart sensor depends on your application requirements, including factors like cost, size, and the processing load on the interfacing microcontroller.

Returning to our human analogy, consider how vision works as a sensor pipeline. When light enters our eyes, photoreceptor cells (rods and cones) in the retina act as sensing elements, converting the light into electrical signals. These signals travel via the optic nerve to the brain’s visual cortex, where they undergo processing to form a recognizable image. The brain then interprets this information and initiates a response, like smiling when you see a beautiful scenery.

Similarly, a sensor pipeline for an embedded system can be defined as shown in the picture below:

Each of these steps may have different requirements based on the application. Creating a requirements document for the sensor is helpful when selecting the appropriate sensor and configuring the pipeline.

Sensor Characteristics

Before you dive into the blocks of the sensor pipeline, let’s review some important characteristics of a sensor.

Sensitivity

Sensitivity is the ability of a sensor to detect small changes in the physical property it’s designed to measure.

Sensitivity can vary based on factors like manufacturing processes, cost, and the design of the sensing element.

Sensors designed for a specific property often come in different sensitivity levels, allowing users to select an appropriate sensitivity based on the application requirements.

Accuracy

Accuracy is the degree to which a sensor’s measurement matches the true value of the physical property it’s measuring. Testing a sensor’s accuracy typically requires comparing its readings to those of a reference instrument.

A sensor may have gain and offset errors—issues that calibration can help correct. Calibration adjusts for these systematic errors, which are often due to manufacturing tolerances or design factors.

Once calibrated, the sensor’s output can be verified against a reference to confirm its accuracy. The required level of accuracy should be determined based on the application’s needs.

Precision

Precision refers to the consistency or repeatability of a sensor's measurements, regardless of how close those measurements are to the true value. It indicates the sensor's ability to produce the same output under identical conditions and how finely it can resolve and report values.

For example, if the true temperature of an object is 12.53°C:

• A precise sensor will consistently measure values like 12.52°C, 12.53°C, or 12.54°C, even if those values are slightly offset from the true temperature.

• A highly accurate sensor, on the other hand, will measure values close to 12.53°C but may lack precision if those readings vary widely (e.g., 12.50°C, 12.53°C, and 12.56°C).

For applications requiring exact measurements, a sensor with both high accuracy (closeness to the true value) and high precision (low variability) is essential. This is especially important in distinguishing small differences, such as between 12.5°C and 12.53°C.

In contrast, applications with less stringent requirements might use sensors with broader tolerances, such as ±1°C, which are sufficient for general monitoring purposes.

Range

The range of a sensor refers to the span between the maximum and minimum values of the physical property it can measure while maintaining its specified precision and accuracy. A sensor's operating range may extend beyond its measurement range, but the measurement range defines the limits within which the sensor reliably adheres to its specified sensitivity, accuracy, and response time.

Drift

Drift is when a sensor's output changes over time due to conditions like temperature or humidity. Components within the sensor, including the sensing element, may be sensitive to these conditions, leading to gradual shifts in measurements.

For example, many components are affected by temperature and humidity changes, which can alter sensor readings. Also, sensors with internal oscillators may experience time-based drift, impacting accuracy.

Regular calibration with an accurate external reference (such as a precise clock) can help correct for drift and maintain reliable measurements. For certain applications, selecting a sensor with acceptable drift characteristics is crucial.

Response Time

Response time is the duration a sensor takes to detect and reflect a change in the measured physical property. For example, if the temperature rises by 5°C, the response time indicates how long the temperature sensor takes to reflect this change in its output.

Response time depends on the sensor’s design, manufacturing quality, and internal components, such as the ADC (Analog-to-Digital Converter), averaging circuits, and filters within the sensor pipeline.

All the parameters mentioned above are thoroughly documented in the sensor’s data-sheet. In practice, it’s a good idea to create a sensor requirements document for each specific application, detailing these key parameters as a baseline for sensor selection.

Now that you’ve examined the key characteristics of sensors, let’s explore how you can connect them to a microcontroller for real-world applications.

How to Interface with a Microcontroller

Choosing a Communication Protocol

Another essential aspect of sensor requirements is specifying the communication interface between the sensor and the MCU or processor in the system. It’s important to understand how the sensor will be interfaced based on its output signal type and the available pins on the microcontroller.

For instance, certain sensors may connect directly to an analog or digital input pin on a microcontroller. A raw sensor, such as a temperature sensor, typically connects to an analog input pin, which is then read by the microcontroller’s internal ADC (Analog-to-Digital Converter).

In contrast, a digital-output sensor connects to a digital GPIO (General Purpose Input/Output) pin. For instance, speed sensors generate square waves with variable pulse widths to indicate speed. These signals are usually connected to a GPIO pin configured as an external interrupt or timer capture input, allowing the microcontroller to measure pulse width accurately.

A smart sensor, on the other hand, often supports communication protocols like SPI (Serial Peripheral Interface) or I2C (Inter-Integrated Circuit). These interfaces enable the microcontroller to configure the sensor, check its status, and retrieve data through register reads and writes.

Choosing the appropriate communication protocol for interfacing a sensor depends on the available pins in the system and the specific requirements of the application.

Tip: When working with protocols like I²C or SPI, using tools such as Saleae logic analyzers can greatly simplify debugging and validation. Logic analyzers capture and visualize communication signals, and tools like Saleae offer built-in protocol interpreters to help you decode sensor communication in real time. This can be especially helpful when troubleshooting configuration issues, timing problems, or communication errors during sensor interfacing.

Figure 2 below shows an example of a microcontroller connected to 4 sensors having different interfaces.

Determining Power Requirements

Power requirements are another key consideration when interfacing a sensor. Sensors may operate at different voltages (for example, 3.3V or 5V), so ensuring the microcontroller can accommodate these levels is essential. Level converters can bridge voltage mismatches, ensuring compatibility between the sensor and microcontroller voltage levels.

Timing and sampling requirements must also be evaluated, especially for sensors generating high-frequency data. Configuring external interrupts on GPIO pins can ensure timely data capture, while techniques like using DMA can streamline data transfer for sensors sampling at high frequencies without CPU involvement.

Now that you’ve learned about communication protocols and hardware connections, let’s focus on designing the software architecture that acquires, processes, and prepares sensor data for use. Designing effective software is crucial for obtaining clean, reliable data from the sensor.

Software Architecture

Now that we’ve chosen the sensor and communication protocol, let’s design the software architecture for the sensor pipeline. This software runs on the microcontroller connected to the sensor and processes raw data to make it clean and usable.

While application-level data processing is beyond the scope of this article, let’s focus on interfacing with the sensor and preparing the data for application use.

The sensor processing pipeline can be broken into the following components:

1. Sensor Driver

2. Analog-to-Digital Conversion (ADC) Support

3. Scaling

4. Calibration

5. Data Post-Processing

Let’s examine a high-level overview of these components for both smart and raw sensors.

High-Level Overview of Components

1. Sensor Driver

   1. Smart sensors: The driver configures the sensor, manages power, and handles read and write operations to the sensor registers over a communication protocol like SPI, I2C.

   2. Raw sensors: The driver may only control GPIOs for power management, as raw sensors typically lack registers.

2. Analog-to-Digital Conversion (ADC) Support

   1. Smart sensors: Include an onboard ADC, which is configured through the sensor driver.

   2. Raw sensors: Requires an external ADC, an ADC driver implemented in software to configure the ADC, initiate conversions, and retrieve data.

3. Scaling: Scaling is necessary for both smart and raw sensors. It converts digital counts after the analog to digital conversion into meaningful physical quantities using formulas provided in the sensor data sheet. For example, a temperature sensor will use a formula to convert digital counts to degree Celsius.

4. Calibration: Once the measured physical quantity is obtained, calibration adjusts the value by applying offsets, gains, or both to correct errors. This process ensures the sensor output aligns with reference values across its entire measurement range. A detailed discussion of the calibration process will follow in the next section.

5. Data Post-Processing: Post-processing techniques, such as filtering are applied to improve data quality and reduce noise. Common filters such as low-pass or high-pass filters can remove unwanted frequency components.

Accessing Data from the Sensor

The method of accessing data depends on the whether it’s a raw sensor or a smart sensor. Smart sensors will have onboard ADCs and FIFOs. Before delving into how data is accessed, it’s important to first understand sampling frequency.

Sampling Frequency:

The frequency of taking a measurement from the sensor must follow the Nyquist-Shannon sampling theorem. It states that the sampling rate must be twice the highest frequency component of the signal to be measured to accurately reconstruct the measured data.

The sampling frequency defines how often the sensor captures data, which affects how the data is accessed. Depending on whether the sensor is a raw sensor or a smart sensor, the approach to handling this sampled data varies.

Smart Sensors:

1. Data register: The sensor writes sampled data directly into a register based on the set sample frequency updated during setup. The microcontroller reads this data register based on a data conversion completion interrupt.

2. FIFObBuffer: Some sensors include FIFO (First-In, First-Out) buffers to store multiple data points. When enabled, the FIFO updates at the configured sampling frequency and trigger interrupts when it becomes full or reaches a predefined level.
   The benefits of FIFO include:

   1. Power efficiency: The MCU can process data in batches, reducing CPU overhead and allowing it to enter low-power mode during data collection.

   2. Sampling and processing rate matching: FIFO buffers help reconcile differences between the sensor’s sampling rate and the MCU’s data processing rate.

   3. For MCUs with Direct Memory Access (DMA), data transfer from the sensor to MCU memory can occur without CPU intervention, further reducing power consumption.

Raw Sensors:

For raw sensors, the MCU triggers ADC conversions at the sampling frequency, often using a timer interrupt. Data is read upon the ADC conversion complete interrupt, allowing the MCU to sleep during conversions and between samples to save power.

Sensor Power Management

Power management is critical for energy-sensitive applications. Strategies include:

1. Low-power modes: Many sensors support low-power modes configurable through sensor registers.

2. GPIO-controlled power cycling (Duty-Cycling): For sensors without built-in low-power modes, the microcontroller can toggle the sensor’s power line using a GPIO pin, reducing power consumption further. Figure 3 below shows the diagram of a raw temperature sensor whose power is controlled using a GPIO from the MCU. For example, a temperature sensor in sleep mode can be activated only when temperature readings are required.

The above techniques ensure efficient use of power while maintaining the required data sampling rate and sensor responsiveness.

With the high-level architecture in mind, we’ll now dive into the detailed design of each pipeline component.

Detailed Design of Components

In this section, you’ll delve into the key components of the sensor pipeline outlined in the Software Architecture section.

1. Sensor Driver

The sensor driver is responsible for managing communication, configuration, power, and data acquisition for both smart and raw sensors.

Smart Sensor Driver:

1. Communication driver: Generic I2C or SPI drivers on the MCU can be adapted using wrapper functions to handle sensor-specific requirements, such as 1-byte, 2-byte, or 4-byte transfers.

2. Configuration: Typical tasks include setting the sampling rate, configuring interrupts, managing FIFO buffers, and, if needed, clock settings.

3. Power management: APIs should allow higher software layers to transition sensors between power modes by writing to specific registers or controlling GPIO lines for sensors without built-in power modes.

Raw Sensor Driver:

For raw sensors, the driver primarily manages power, often through GPIO-controlled toggling.

2. ADC Support

ADC support is required only for raw sensors. In this article, we’re focusing on SAR ADCs, which are commonly embedded in microcontrollers.

How SAR ADCs Work?

A SAR ADC converts an analog signal to a digital value over multiple clock cycles, with the number of cycles equal to its bit resolution (for example, 10 cycles for a 10-bit ADC).

Key terms related to ADCs:

1. Reference Voltage (VRef): Represents the maximum voltage the ADC can measure. Analog signals exceeding this limit must be scaled down.

2. Resolution: Determines the smallest detectable voltage change. For example, a 10-bit ADC with a 3.3V VRef has a resolution of 3.22 mV

$$V_{\text{Res}} = V_{\text{Ref}} /2^{10}$$

The ADC result is stored in a data register, which can then be scaled to meaningful physical units.

3. Scaling

Scaling converts ADC counts into meaningful physical values, such as temperature (°C) or acceleration (g) depending on the sensor type. Sensor datasheets typically provide the necessary formulas or lookup tables.

For example, the method to convert a voltage measured by a raw temperature sensor to temperature value is shown below:

$$V_{\text{Measured}} = Counts_{\text{ADC}} / 2^{10} * V_{\text{Ref}} \quad \text{(Get V_Measured from ADC Counts)}$$

$$Temperature_{\text{Measured}} = V_{\text{Measured}} * T_{\text{C/mV}} \quad \text{(Get Temperature physical value)}$$

Similarly, a 3-axis accelerometer maps counts on the X, Y, and Z axes to acceleration values in g or milli-g.

4. Calibration

The figure above on the left (4a) is showing Calibration with gain and offset, while the figure above on the right (4b) is showing calibration with fixed offset.

$$x_{\text{calibrated}} = Gain * x_{\text{raw}} + Offset \quad \text{(Figure 4a - Linear Calibration)}$$

$$x_{\text{calibrated}} = x_{\text{raw}} + Offset \quad \text{(Figure 4b - Fixed offset Calibration)}$$

Calibration ensures the sensor’s output aligns with reference measurements, correcting for errors introduced by design, materials, or manufacturing.

Types of Errors:

1. Offset error: A constant deviation of the sensor’s output from the true reference value, regardless of input magnitude.

2. Gain error: A proportional error where the sensor’s output scale deviates from the expected value, causing the output to increase or decrease incorrectly relative to the input.

Calibration Methods:

1. 2/3-Point calibration: This type of calibration may involve either applying a fixed offset to the raw value or applying both gain and offset. Figure 4a illustrates an example of a gain/offset calibration, while Figure 4b depicts offset calibration. In both figures, the y-axis represents the reference value measured by an accurate instrument, while the x-axis represents the raw value measured by the sensor after ADC.

2. N-Point calibration: Involves multiple points for more complex, non-linear error correction.

Implementation:

1. Calibration points shall cover the sensor’s entire measurement range for accuracy.

2. Parameters like gain and offset once estimated shall be stored in a non-volatile memory in the system for persistence to be used across power cycles.

5. Data Post-Processing

Post-processing covered in this section talks about removing noise and unwanted signal components, which improves data reliability.

Filtering

Filtering is the process of removing unwanted frequency components from a signal to improve data quality. There are several different types of filters:

• Low-Pass Filters: Allows low-frequency signals to pass while attenuating high-frequency noise.

• High-Pass Filters: Allows high-frequency signals to pass while attenuating low-frequency noise. (for example, gravitational acceleration in accelerometer data).

• Band-Pass Filters: Retains only signals within a specific frequency range, removing both lower and higher frequencies outside the desired band.

These filters are often implemented as FIR (Finite Impulse Response) or IIR (Infinite Impulse Response) filters. IIR filters are easy to implement and computationally efficient while FIR filters are computationally intensive but have better control over the frequency response.

Here, we will explore a simple low-pass filter known as the Exponential Moving Average (EMA), a type of IIR filter. A moving average filter is a mathematical technique that smooths short-term fluctuations while highlighting longer-term trends.

Unlike other moving average filters, EMA does not require maintaining a buffer, making it more memory-efficient. It is also more responsive to data changes while still providing smoothing, making it well-suited for real-time filtering. EMA assigns greater weight to recent data samples than older ones, allowing it to adapt quickly to changes in sensor readings.

EMA can be calculated like this:

$$EMA_{\text{t}} = \alpha * x_{\text{t}} + (1 - \alpha) * EMA_{\text{t - 1}}$$

$$\alpha = 2 / (N + 1) \quad \text{(Smoothening Factor, N - filter window size)}$$

$$EMA_{\text{t}} \quad \text{(Exponential Moving Average in current iteration)}$$

$$x_{\text{t}} \quad \text{(New Data Sample in Current Iteration)}$$

$$EMA_{\text{t - 1}} \quad \text{(Exponential Moving Average in the last iteration)}$$

Now that we understand the Exponential Moving Average (EMA) filter, here are two key factors to consider when tuning it for an application:

• Smoothing vs. Responsiveness: A higher smoothing factor (closer to 1, smaller filter window size) gives more weight to recent data, making the filter more responsive to changes but less effective at noise reduction. A lower smoothing factor (closer to 0, larger filter window size) provides better noise reduction but reacts more slowly to data changes.

• Application-Specific Tuning: The smoothing factor should be chosen based on the sampling rate, sensor sensitivity, and application requirements. Real-time systems often require a balance between quick responsiveness and stable output.

Here’s a code sample for EMA:

#include <stdio.h>
#include <stdint.h>

// Exponential Moving Average (EMA) filter implementation
#define FILTER_WINDOW 5

// Function to calculate EMA
float calculateEMA(float ema, float new_value, float alpha) {
    return (alpha * new_value) + (1 - alpha) * ema;
}

int main() {
    float sensorReadings[] = {26.0, 27.5, 28.2, 27.0, 26.8, 26.5, 27.2};
    int numReadings = sizeof(sensorReadings) / sizeof(sensorReadings[0]);

    float alpha = 2.0f / (FILTER_WINDOW + 1); // Standard EMA formula
    float ema = sensorReadings[0];  // Initialize EMA with the first reading

    printf("EMA Filtered Sensor Data:\n");

    for (int i = 0; i < numReadings; i++) {
        ema = calculateEMA(ema, sensorReadings[i], alpha);
        printf("Reading %d: Raw = %.2f, EMA = %.2f\n", i + 1, sensorReadings[i], ema);
    }

    return 0;
}

Conclusion

In summary, sensors are the backbone of modern smart devices, bridging the gap between the physical world and digital systems. From consumer electronics to industrial automation and medical devices, they enable devices to perceive and interact with their environments.

Understanding how sensors work, the components of their data pipeline, and their integration with microcontrollers is essential for engineers and hobbyists alike. By designing effective pipelines, developers can ensure accurate, clean, and reliable data, enabling systems to meet performance and power efficiency goals.

If you have questions or want to talk more about this topic, feel free to reach out on Twitter or LinkedIn. Always happy to connect.

In this article, you'll learn how to control LEDs using a switch case statement in Arduino. You can also find the switch case statement in other programming languages, so this can serve as a practical example of how they work.

Here’s a demo of what you’ll be building:

You can watch the video version of this article here:

Hardware Components

Here are the components you'll need to follow along:

• Arduino board (Uno).

• Potentiometer.

• Breadboard.

• Three LEDs.

• Resistors for the LEDs.

• Jumper wires.

How to Use a Switch Case Statement in Arduino

Here's the syntax/structure of a switch statement:

switch (variable) {
  case value1:
    // code to be executed if variable == value1
    break;
  case value2:
    // code to be executed if variable == value2
    break;
  default:
    // code to be executed if variable doesn't match any case
    break;
}

Let's break it down:

• variable: This denotes the variable being evaluated. The value of the variable determines how the code blocks will be executed.

• case: Each case represents a value that may match the variable being evaluated. If the variable and a case have the same value, the code for that case will be executed. You can have as many cases as you want.

• break: After a code block in a case has been executed, the break keyword terminates the code. That is, it stops the code from moving on to other cases because a match has already been found.

• default: In a situation where none of the cases match the variable, the code in the default block will be executed.

Next, let's use a switch statement to control LEDs.

Switch Case in Arduino Example

Circuit Diagram

Here’s how to connect your components:

The goal here is to decide which LED (or a combination of LEDs) comes on based on the value of a variable.

Potentiometer Connection

• Connect the left terminal of the potentiometer to 5V.

• Connect the right terminal to GND.

• Connect the middle terminal to A0.

LED Connection

• For each LED, connect the shorter leg to GND.

• Connect each longer leg to a digital pin. I recommend using pin 8 (for the green LED), 9 (for the yellow LED), and 10 (for the red LED) to match what we have in the circuit diagram. We'll also use these values in the code.

Here's the full project code:

int greenLED = 8;
int yellowLED = 9;
int redLED = 10;
int potPin = A0; 
int potValue;
int mappedPotValue;
​
void setup() {
  pinMode(greenLED, OUTPUT);
  pinMode(yellowLED, OUTPUT);
  pinMode(redLED, OUTPUT);
  Serial.begin(9600);
}
​
void loop() {
  potValue = analogRead(potPin);
  mappedPotValue = map(potValue, 0, 1023, 0, 5);
​
  switch (mappedPotValue) {
    case 0:
      digitalWrite(greenLED, LOW);
      digitalWrite(yellowLED, LOW);
      digitalWrite(redLED, LOW);
      Serial.println(mappedPotValue);
      break;
    case 1:
      digitalWrite(greenLED, HIGH);
      digitalWrite(yellowLED, LOW);
      digitalWrite(redLED, LOW);
      Serial.println(mappedPotValue);
      break;
    case 2:
      digitalWrite(greenLED, LOW);
      digitalWrite(yellowLED, HIGH);
      digitalWrite(redLED, LOW);
      Serial.println(mappedPotValue);
      break;
    case 3:
      digitalWrite(greenLED, LOW);
      digitalWrite(yellowLED, LOW);
      digitalWrite(redLED, HIGH);
      Serial.println(mappedPotValue);
      break;
    case 4:
      digitalWrite(greenLED, HIGH);
      digitalWrite(yellowLED, HIGH);
      digitalWrite(redLED, HIGH);
      Serial.println(mappedPotValue);
      delay(500);
      digitalWrite(greenLED, LOW);
      digitalWrite(yellowLED, LOW);
      digitalWrite(redLED, LOW);
      Serial.println(mappedPotValue);
      delay(500);
      break;
  }
}

Let's break down the code.

Variable Initialization

int greenLED = 8;
int yellowLED = 9;
int redLED = 10;
int potPin = A0; 
int potValue;
int mappedPotValue;

We started by initializing variables to correspond with the hardware connections.

greenLED, yellowLED, and redLED have values of 8, 9, and 10, respectively. This matches the pins they were connected to on the Arduino board. Similarly, potPin, which is the variable for the potentiometer, has a value of A0.

You'll use the potValue variable to store the current value of the potentiometer. We also created a mappedPotValue variable to store the range of values needed for the LEDs in a minute.

pinMode and Serial Monitor

void setup() {
  pinMode(greenLED, OUTPUT);
  pinMode(yellowLED, OUTPUT);
  pinMode(redLED, OUTPUT);
  Serial.begin(9600);
}

In the setup() function, we set the LEDs as output pins and initialized the serial monitor.

Logic for switch case Statement

First, we read the value of the potentiometer using the analogRead() function and stored it in the potValue variable:

potValue = analogRead(potPin);

We then converted the values from the potentiometer to a range of 0 to 4 using the map function and stored them in the mappedPotValue variable:

mappedPotValue = map(potValue, 0, 1023, 0, 5);

Next, we created a switch statement—the value being evaluated is mappedPotValue. Recall that this is the variable where we stored the potentiometer values. So whenever you turn the potentiometer, the value changes and potentially matches a case:

  switch (mappedPotValue) {
    case 0:
      digitalWrite(greenLED, LOW);
      digitalWrite(yellowLED, LOW);
      digitalWrite(redLED, LOW);
      Serial.println(mappedPotValue);
      break;
    case 1:
      digitalWrite(greenLED, HIGH);
      digitalWrite(yellowLED, LOW);
      digitalWrite(redLED, LOW);
      Serial.println(mappedPotValue);
      break;
    case 2:
      digitalWrite(greenLED, LOW);
      digitalWrite(yellowLED, HIGH);
      digitalWrite(redLED, LOW);
      Serial.println(mappedPotValue);
      break;
    case 3:
      digitalWrite(greenLED, LOW);
      digitalWrite(yellowLED, LOW);
      digitalWrite(redLED, HIGH);
      Serial.println(mappedPotValue);
      break;
    case 4:
      digitalWrite(greenLED, HIGH);
      digitalWrite(yellowLED, HIGH);
      digitalWrite(redLED, HIGH);
      Serial.println(mappedPotValue);
      delay(500);
      digitalWrite(greenLED, LOW);
      digitalWrite(yellowLED, LOW);
      digitalWrite(redLED, LOW);
      Serial.println(mappedPotValue);
      delay(500);
      break;
  }

We passed mappedPotValue as a parameter to switch since it's the variable being compared to different cases: switch (mappedPotValue).

• For case 0, all the LEDs will be off.

• For case 1, only the green LED comes on.

• For case 2, only the yellow LED comes on.

• For case 3, only the red LED comes on.

• For case 4, all three LEDs will blink continuously.

Using a switch statement, you've successfully controlled the behavior of LEDs based on the value of a potentiometer!

Conclusion

In this article, you learned how to use a switch case statement in Arduino using a practical example.

You learned how to control different LEDs based on the value of a potentiometer. You achieved this by using different cases in a switch statement to match the potentiometer's current value and execute the corresponding code.

switch statements can be used in different ways to make a project more dynamic. Some use cases in Arduino include:

• Managing and interpreting the different values, modes, and states of a component or sensor.

• Performing actions based on specific commands. For example, rotating a robotic arm to a specific angle/direction.

• Mapping button presses to user input, and so on.

You can watch the video version of this project here. The full project code is available on GitHub.

Check out my blog for articles about embedded systems, IoT, and web development.

Happy coding!

You've most likely used a potentiometer before with appliances like radios, microwaves, blenders, electric fans, game controllers, and others.

They are generally used to provide or control different ranges of variable resistance in electronic circuits.

In this article, you'll learn the following:

• How to connect a potentiometer to an Arduino board.
• How to get the values of a potentiometer.
• How to control the brightness of an LED using a potentiometer.

You can also watch the video version of this article here:

Hardware Components

Here are the hardware components you'll need to follow along:

• Arduino board.
• Potentiometer.
• Breadboard.
• LED.
• 1K Ohm resistor.
• Jumper wires.

How to Connect a Potentiometer to an Arduino Board

The potentiometer is made up of three terminals: two outer terminals and the middle terminal. Either of the outer terminals can be connected to either 5V or GND (ground). That is:

• If you connect the left outer terminal to 5V, you have to connect the right outer terminal to GND.
• If you connect the left outer terminal to GND, you have to connect the right outer terminal to 5V.

The middle terminal serves as the output terminal. We'll connect it to an analog pin. You can read the varying values of the potentiometer from the output terminal.

Here's the circuit diagram:

circuit diagram

Here's how the potentiometer was connected in the diagram above:

• The left outer terminal of the potentiometer was connected to GND.
• The right outer terminal was connected to 5V.
• The middle terminal (output terminal) was connected to analog pin A0 on the Uno board.

Here's how the LED was connected:

• The shorter leg of the LED was connected to GND.
• The longer leg was connected to digital pin 6 through a 1K Ohm resistor.

Make sure to connect the LED to a digital pin with the ~ symbol. Such pins support pulse width modulation, which lets you send analog signals to digital pins.

int potPin = A0;
int potValue = 0;
int brightness = 0;
int ledPin = 6;

void setup() {
  Serial.begin(9600);
  pinMode(ledPin, OUTPUT);
}

void loop() {
  potValue = analogRead(potPin);
  brightness = (255.0/1023.0)*potValue;
  analogWrite(ledPin, brightness);
}

Let's break down the code.

Initialize Variables

We started by initializing our variables:

int potPin = A0;
int potValue = 0;
int brightness = 0;
int ledPin = 6;

The potPin variable has a value of A0. This represents the A0 pin connected to the output pin of the potentiometer.

We then declared a potValue variable, which will be used to store the values from potPin.

The brightness variable will be used to control the brightness of the LED.

The LED pin was connected to digital pin 6 on the Uno board, so we initialized a ledPin variable with a value of 6: int ledPin = 6;.

Serial Monitor and pinMode

Next, in the loop() function, we initialized the serial monitor and set the LED pin to serve as an output pin:

void setup() {
  Serial.begin(9600);
  pinMode(ledPin, OUTPUT);
}

Create Logic to Control LED Brightness

In the loop() function, we have three lines of code:

void loop() {
  potValue = analogRead(potPin);
  brightness = (255.0/1023.0)*potValue;
  analogWrite(ledPin, brightness);
}

In the first line, we used the analogRead() function to read the value of potPin. The read values were assigned to the potValue variable.

At this point, if you print potValue to the serial monitor using Serial.println(potValue);, you'll get a range of values from 0 to 1023 when you tune the knob of the potentiometer.

For the brightness variable, we converted the values from the potentiometer to fall with the range of 0 to 255: brightness = (255.0/1023.0)*potValue;. This is because the analogWrite() function only accepts values within that range, and not the default 0 to 1023 that the potentiometer produces.

Lastly, we used the analogWrite() function to send values to the LED: analogWrite(ledPin, brightness);.

The analogWrite() function's first parameter is the ledPin, which denotes the pin where the values should be sent. The second parameter is brightness, which denotes a range of values to be sent to the LED (ledPin).

When you upload the code to your board, the LED should have varying levels of brightness as you tune the potentiometer.

Conclusion

In this article, you learned how to control the brightness on an LED using a potentiometer. You also saw how to connect the components to digital and analog pins on an Arduino board.

Finally, you learned about how to make the components work together using code.

You can find the code for this project here. You can watch the video version here.

Check out my blog for articles about embedded systems, IoT, and web development.

Happy coding!

We talk about growing up in the European countryside, his early passion for computers, and ultimately his move to San Francisco, where he's founded several tech companies.

Bruno's super excited about embedded systems and custom hardware. He's building home appliances that incorporate open source software and open datasets.

We cover so many topics here. From Star Trek to the European Pirate Party.

I hope these weekly freeCodeCamp podcasts are firing you up about learning more about technology.

Be sure to follow The freeCodeCamp podcast in your favorite podcast app. And share this podcast with a friend. Let's inspire more folks to learn to code and build careers for themselves in tech.

You can listen to the podcast in Apple Podcasts, Spotify, or your favorite podcast app. You can also listen to the podcast below, right in your browser:

A couple interesting links from our discussion:

• "Only Amiga" song from Comdex 1987
• Halt and Catch Fire TV Show trailer

And here are some bonus photos from my time at Bruno's house in NYC:

Bruno at work in his office

A hardware project Bruno had prototyped in his bedroom. (This is the photo that I mentioned during the interview that I had snuck in to snap.)

Arduino can be applied in a variety of projects like:

• Home automation.
• Internet of Things (IoT).
• Audio and music.
• Automated and remote controlled systems.
• Automation in agriculture.
• Electronic prototyping.
• Wearable devices, and so much more.

The hardware part of Arduino comprises Arduino boards, input and output devices (including digital and analog pins, and sensors and actuators), shields, breadboards, jumper wires, and so on. These components can be combined together to create dynamic and interactive projects.

The software is made up of the development tools used to write, debug, compile, and upload code to Arduino boards. Most 0f the software tools can be found in the Arduino IDE (Integrated Development Environment).

This handbook will help you understand how Arduino works. You'll learn about the Arduino boards, the components that make up a board, and how to connect devices to them.

We'll talk about input and output peripherals which help the microcontroller (the brain of the Arduino board) process information coming from the physical environment, and send output based on programmed logic.

You'll learn about the Arduino IDE, how to code using the Arduino programming language, and how to use sensors, actuators, and other components to build projects as you learn.

You'll also learn about serial communication, which helps Arduino boards communicate with other computers.

This handbook is written for makers (students, artists, hobbyists, programmers) who are beginners.

Prerequisites

Although it would be helpful, you don't need prior programming knowledge to use this handbook. You will learn the basics of Arduino programming from scratch. This can also serve as your introduction to programming.

To make it beginner friendly, we won't talk about some aspects of electronics like current and voltage, resistance, circuits (series and parallel), and most basic electronic/electrical laws and requirements for students in STEM (Science, Technology, Engineering, Mathematics) fields.

Whether you have knowledge of these concepts or not, you can learn about Arduino using this handbook.

If you know how to use a breadboard and a resistor, then that's all the knowledge on electronics you'll need to follow along.

In summary, this handbook is for everyone. You don't need an engineering degree to become an Arduino maker!

Table of Contents:

• Prerequisites
• Chapter 1: Getting Started with Arduino
• Chapter 2: Basics of Arduino Programming
• Chapter 3: How to use Digital Pins in Arduino
• Chapter 4: How to use Analog Pins in Arduino
• Chapter 5: How to use Sensors and Actuators in Arduino
• Chapter 6: How to use the Serial Monitor in Arduino
• Chapter 7: How to use Displays in Arduino

Chapter 1: Getting Started with Arduino

The Arduino development and design process comprises both hardware and software. So knowing how they work together is important for building the right foundation for your journey.

In this chapter, you’ll learn about the different components that make up the Arduino Uno board. You'll also learn how to install the Arduino IDE and set up your development environment.

At the time of writing, a new Uno board was released — the Arduino Uno R4. This handbook will make use of the Uno R3 board, but you can follow along with either of them. The R4 board comes in two variants — Arduino Uno R4 WiFi and Arduino Uno R4 Minima — with cool additional features that you can read about here.

Components of the Arduino Uno R3 Board

There are many types of Arduino boards like Arduino Nano, Arduino Uno, Arduino Mega, Arduino Leonardo, and so on.

These boards have some common features — they all have digital and output pins, they’re programmable, and they all have a microcontroller.

But there are also some differences. Each board varies in size and shape, and usually has more or fewer components when compared to other boards.

The common boards you’ll come across as a beginner are the Nano, Uno, and Mega boards. The most commonly used is the Uno board, which we’ll use for this handbook.

Here are some of the components you'll find on the Uno R3 board:

• A power port.
• USB connector.
• Microcontroller (ATmega328).
• Analog pins.
• Digital pins.
• Reset button.
• TX and RX indicators.

Arduino Uno R3 board (https://store.arduino.cc/products/arduino-uno-rev3)

You’ll make use of most of the components listed above as you progress through this handbook.

How to Install and Set Up the Arduino IDE

You can use the Arduino IDE to program Arduino boards. That is, you write the code in the IDE, then upload it to the board.

In this section, you’ll learn how to set up the IDE, and create your first Arduino program (also called an Arduino sketch).

You can download the latest version of the Arduino IDE on the Arduino software download page. You can download the IDE for different operating systems — Windows, MacOS, and Linux.

The installation process is similar for the operating systems listed above. Here’s how to install it on a Windows machine:

Step #1 – Download the Arduino IDE

The first step is to download the IDE from the Arduino software download page. You should see a section of the page similar to the image below:

On the right side of the image above are different download options for specific operating systems. Make sure you download the option that suits your operating system.

I'll use the ZIP file option for Windows. If you decide to download an installer instead, then you can follow the installation steps after clicking the installation file.

Step #2 – Unzip the Downloaded File

Go on and unzip the downloaded file. This gives you access to all the resources needed to run the Arduino IDE.

After unzipping the file, you should see files like these:

Image showing the files you should see

To launch the Arduino IDE, click on the file that says “Arduino IDE”.

Step #3 – Overview of the Arduino IDE

Now that you’ve downloaded and installed the Arduino IDE, the next part is to get familiar with the development environment. In the next section, you’ll learn how to upload code to an Arduino Uno board using the IDE.

Before that, let’s have a look at some options you’ll find in the Arduino IDE. At the top left corner of the IDE are five options — File, Edit, Sketch, Tools, Help:

Screenshot showing these options (File, Edit, Sketch, Tools, Help)

The “File” option lets you do different things like creating a new sketch (we’ll talk about sketches in the next section), opening an existing sketch, Arduino practice examples for beginners, keyboard shortcuts, save options, and so on.

The “Edit” option gives you access to text formatting options like copy, paste, cut, comment/uncomment code, font size options, text search options, and so on.

You can use the “Sketch” option to verify and compile code, upload code to Arduino boards, optimize code, and add libraries.

You can use the “Tools” option to manage libraries, format code, access the serial monitor and plotter, select an Arduino board and port to upload code to, choose a processor, and so on.

The “Help” option provides resources for troubleshooting, information on IDE updates, guides on “getting started”, and so on.

Next, let’s look at some other parts and functionalities in the IDE that you’ll find useful. The image below, from the Arduino documentation, highlights them perfectly:

https://docs.arduino.cc/software/ide-v2/tutorials/getting-started-ide-v2

• Verify/Upload: You can use these options to compile and upload code to Arduino boards. You’ll get error messages if the code doesn't compile as expected.
• Select Board & Port: You can use this option to select a port and port number to upload your code too. The current version of the Arduino IDE automatically detects both boards and ports.
• Sketchbook: This gives you access to all the sketches created in your computer. You can also access sketches saved on Arduino Cloud (mostly used for creating IoT projects).
• Boards Manager: The Arduino IDE comes with support for different boards. As you progress through your journey, you’ll make use of different boards and some of them may not be supported by the IDE. The board manager tab lets you install and manage packages required to use these boards.
• Library Manager: You can use libraries to extend certain functionalities in code. Through the library manager, you can install numerous libraries that’ll help simplify the development process for you.
• Debugger: This is used for real time testing and debugging of Arduino programs.
• Search: You can use the search tool to find specific keywords in your code.
• Open Serial Monitor: You can use the serial monitor to communicate with Arduino boards, debug and test code, visualize data from your boards, interact with user input, and so on. We’ll look at the serial monitor in depth in a different chapter.
• Open Serial Plotter: The serial plotter is mostly used for real-time visualization of numerical data.

What Is an Arduino Sketch?

We mentioned the term “sketch” a couple of times in the last section, but what is it? A sketch is a program written with the Arduino programming language. It’s another way of referring to a code file written for Arduino projects.

The Arduino programming language is built upon the C/C++ language so they both share similar syntax and structure. You may come across resources that refer to Arduino code as “embedded C” or “embedded C++”.

How to Upload Code to an Arduino Board

To upload code to an Arduino board, you'll need both hardware and software. The hardware is the board which is the Uno board in our case, and the software is the Arduino sketch in the IDE.

Here are the steps:

Step #1 – Connect the Arduino Board

Connect the Arduino board to your computer using the USB cable. Without this step, you can't go further.

Step #2 – Create a Sketch

Now it's time to launch the IDE and write some code.

Here's a code example that makes an LED blink:

int ledPin = 13;

void setup() {
  pinMode(ledPin, OUTPUT);
}

void loop() {
  digitalWrite(ledPin, HIGH);
  delay(1000);
  digitalWrite(ledPin, LOW);
  delay(1000);
}

Don't worry if you don't understand the code — we'll cover everything as we go further.

Step #3 – Select the Board and Port

You can select the board to upload your code to from the IDE. Here's an image showing what that looks like:

Step #4 – Verify the Code

You can use the verify button to compile the code and check for errors. If errors exist, you'll get an error message to show you the possible cause.

Image showing the verify button

Step #5 – Upload the Code

You can upload the code using the upload button (the button after the verify button).

If there are no errors in your code, these steps will help upload code to your board. If you've uploaded the code above, you should have the built-in LED (it is connected to pin 13 by design) on the Uno board blinking.

In the next chapter, you'll learn the basics of the Arduino programming language.

Chapter 2: Basics of Arduino Programming

Before we dive into creating our own sketches and tinkering, you have to understand the logic that make these boards work as expected. To do that, you’ll have to know how to code using the Arduino programming language.

As discussed in the last chapter, the Arduino language is built upon C/C++. You’ll begin this chapter by learning the basics of programming. This will prepare you for every other chapter that involves writing code.

I’ve created this chapter with beginners in mind. If you’ve never written code before then this can serve as a starting point for you. This doesn’t mean you’ll learn how to code in C or C++. You’ll be learning how to write Arduino code which shares similar syntax with those languages.

At the end of this chapter, you should be able to understand and write Arduino code.

Variables and Data Types in Arduino

Variables and data types are used in most programming languages to store and manipulate data. You can think of variables as containers or storage units. Data types, like the name implies, are the type of data stored in variables.

In Arduino programming, you must specify the data type of a variable before using it. That is:

dataType variableName = variableValue

There are different types of data types in Arduino, and we’ll discuss each one along with code examples.

int Data Type in Arduino

The int data type is used to store integer values. The Uno board has a 16-bit integer capacity so it can store values that fall within the range of -32,768 to 32,767.

int redLED = 6;

In the code above, we created an integer variable called redLED with a value of 6.

The int data type can also store negative integers:

int redLED = -6;

long Data Type in Arduino

The long data type is similar to int but has a wider range of integer values. It has a 32-bit integer limit which falls within the range of -2,147,483,648 to 2,147,483,647.

long largeNumber = 6000;

float Data Type in Arduino

The float data type can be used to store numbers with decimals. Float variables can store values up to 3.4028235E+38 and values as low as -3.4028235E+38.

float num = 10.5;

Although the float data type is mainly used for decimal numbers, it can also accept whole numbers (integers without decimals). But it'll always return a float value. So if you store 10 in a float, it'll return 10.00.

String Data Type in Arduino

You can use the String data type to store and manipulate text. You'll work with strings occasionally to display information in the form of text when building projects.

Here's a code example:

String greeting = "Hello World!";

The value of strings are nested within double quotation marks as can be seen in the code above.

Note that when declaring a string, the "S" should always be in uppercase.

char Data Type in Arduino

The char data type stores single characters.

Here's an example:

char alphabet = 'A';

This is different from the String data type that can store multiple characters.

There are two main differences between char and String:

• char uses single quotes ('A') while string uses double quotes ("Arduino").
• char stores single characters while string stores multiple characters.

char can also accept integer values equivalent to the ASCII value of letters:

char charValue = 65;

In the code above, we initialized a char variable with the value of 65. When printed to the serial monitor (we'll talk about the serial monitor in Chapter 6: How to use the Serial Monitor in Arduino), A will be returned.

A is returned because 65 has an ASCII character of A.

bool and boolean Data Types in Arduino

You can use both bool and boolean to store/denote boolean values of either true or false.

bool roomIsCold = false;

Boolean values are mostly used with logical and comparison operators, and conditional statements (you'll learn about these later in this chapter) to manipulate and control different outcomes in an Arduino program.

byte Data Type in Arduino

The byte data type has an 8-bit unsigned integer limit that ranges from 0 to 255. Unsigned means that it can't store negative values.

byte sensorValue = 200;

The byte data type isn't the only data type that can be unsigned. You can also use the unsigned int, unsigned long, and unsigned char data types which all have their respective positive integer ranges.

Operators in Arduino

Operators are symbols or characters that can be used to perform certain operations on operands. An operand is simply any value(s) an operator acts on.

There are different categories of operators in Arduino like:

Arithmetic Operators

Arithmetic operators are used to perform mathematical operations like addition, subtraction, division, multiplication, and so on. Here are some arithmetic operators you should know:

Addition(+) Operator

The addition operator, denoted by the + symbol, adds two operands together:

int a = 5;
int b = 10;

// we use addition operator to add a and b below
int c = a + b;

Serial.print(c);
// 15

Subtraction(-) Operator

The subtraction operator subtracts the value of one operand from another operand. It is denoted by the - symbol:

int a = 5;
int b = 10;

// we use subtraction operator to subtract b from a below
int c = b - a;

Serial.print(c);
// 5

Multiplication (*) Operator

You can use the multiplication operator (*) to multiply two operands:

int a = 5;
int b = 10;

// we use multiplication operator to multiply a by b below
int c = a * b;

Serial.print(c);
// 50

Division(/) Operator

The division operator divides one operand by another:

int a = 5;
int b = 10;

// we use division operator to divide b by a below
int c = b / a;

Serial.print(c);
// 2

Modulus (%) Operator

The modulus operator returns the remainder of a division between two operands:

int a = 5;
int b = 10;

// we use division operator to divide b by a below
int c = b % a;

Serial.print(c);
// 0

Increment (++) Operator

The increment operator increases the value of a variable by 1:

int num = 5;
num++;

Serial.print(num);
// 6

Decrement (--) Operator

The decrement operator decreases the value of a variable by 1:

int num = 5;
num--;

Serial.print(num);
// 4

Assignment Operators

Assignment operators are mainly used to assign values to variables. You can also use them to update the value of variables.

The assignment (=) operator is used for assigning and updating variables. The = operator should not be confused for "equal to" — they aren't the same. We'll talk about the equal to (==) operator in the Comparison Operators section.

Here's an example that shows how to use the assignment operator:

int age = 1;

In the code above, we created a variable called age, and then assigned a value of 1 to it using the = operator.

But this isn't the only way to assign or update the value of variables when using the = operator. You can also use compound assignment operators.

Compound Assignment Operators

Compound assignment operators let you combine arithmetic operators and the = operator. This method provides a shorter way of writing code. Here is an example:

int x = 5;
x += 5;

Serial.print(x)
// 10

In the code above, we created an x variable and assigned a value of 5 to it. But on the second line, you'd see that we combined the arithmetic addition (+) operator and the = operator to assign a new value to x:

x += 5;

The line of code above is the same as this:

x = x + 5;

So compound operators combine two operators and let you do the same thing in a shorter way. There's nothing wrong with either method.

Here are other compound operator examples:

int a = 10;
a -= 5; // Equivalent to a = a - 5 (a becomes 5)

int b = 10;
b *= 5; // Equivalent to b = b * 5 (b becomes 50)

int c = 10;
c /= 5; // Equivalent to c = c / 5 (c becomes 2)

int d = 10;
d %= 5; // Equivalent to d = d % 3 (d becomes 0)

Comparison Operators

You can use comparison operators to compare two values/operands. Comparison operators return either true (1) or false (0) depending on the relationship between operands.

Comparison operators can help you make decisions based on their return values. You'll see them a lot when we start building projects.

Here are the comparison operators you'll come across occasionally:

Equal To (==) Operator

This operator compares the values of two variables. If the values are the same, it returns true. If the values are not the same, it returns false. Here's an example:

int x = 10;
int y = 5; 

Serial.print(x == y)
// returns 0

The return value of x == y in the code above is 0 (false) because the two variables are not the same. Remember that the == operator returns 1 (true) only when both variables have the same value.

Not Equal (!=) Operator

The not equal operator checks whether two values have different values. It's does the opposite of the == operator. That means it'll return 1 (true) if both values are not of the same value and 0 (false) if both values are the same.

Here's an example:

int x = 10;
int y = 5; 

Serial.print(x != y)
// returns 1

Greater Than (>) Operator

The greater than (>) operator checks if the operand on the left is greater than the operand on the right. If the left operand is greater, it returns 1. If the left operand is smaller, it returns 0.

int x = 10;
int y = 5; 

Serial.print(x > y)
// returns 1

Less Than (<) Operator

The less than (<) operator checks if the operand on the left is less than the operand on the left. If the left operand is smaller, it returns 1. If the left operand is greater, it returns 0.

int x = 10;
int y = 5; 

Serial.print(x < y)
// returns 0

Greater Than or Equal To (>=) Operator

Just like the name, the >= operator checks if the operand on the left is either greater than or equal to the operand on the right. It returns 1 if the left operand is greater than or equal to the right operand, and 0 if it isn't.

"Or" implies that either of the conditions can be used. If the left operand is not greater than the right operand but is equal to the right operand, you'll still get a value of 1.

int x = 10;
int y = 5; 

Serial.print(x >= y)
// returns 1

Less Than or Equal To (<=) Operator

The <= operator checks if the left operand is either less than or equal to the right operand. If the left operand is less than or equal to the right operand, it returns 1, and returns 0 if the left operand is neither less than nor equal to the right operand.

int x = 10;
int y = 5; 

Serial.print(x <= y)
// returns 0

Logical Operators

Logical operators are used in most programming languages to evaluate and determine the relationship between variables.

Here are the three logical operators you should know for Arduino programming:

Logical AND (&&) Operator

The logical AND (&&) operator returns 1 if both statements are true.

int x = 10;

Serial.print((x > 5) && (x > 3));
// returns 1

The expression above — (x > 5) && (x > 3) — returns 1 because both statements are true. That is, x > 3 and x > 3. If either or both of those statements were false, then we'd have a return value of 0.

Logical OR (||) Operator

The logical OR (||) operator returns 1 if one of both statements is true.

int x = 10;

Serial.print((x > 5) && (x > 15));
// returns 1

The code above returns 1 although one of the statements is false. This is because the || operator returns 1 if either or both statements are true.

Logical NOT (!) Operator

The NOT (!) operator negates or reverses the value of its operand. If the operand statement is true, it returns false, and returns false if the operand is true.

Here's an example:

int x = 10;

Serial.print(!(x > 5));
// returns 0

The code above returns 0, but why? x > 5 is true so the expected result is 1.

We got 0 because the ! operator reversed the return value of the operand from 0 to 1.

Conditional Statements in Arduino

You can use conditional statements to make decisions or execute code based on specific conditions. You can combine conditional statements and logic (like operators in the last section) to control how code is executed.

Let's take a look at some conditional statements and how to use them:

if Statement

The if statement is used to execute code if a condition is true. Here's what the syntax looks like:

if (condition) {
    // code to be executed if condition is true
}

In the syntax above, condition denotes a specified logic. If the condition is true then the code in the curly brackets will be executed. Here's an example:

int x = 5;
if (x < 10) {
  Serial.print("x is less than 10");
}

// x is less than 10

In the code above, we gave a condition— x < 10 — and a block of code within curly brackets that prints "x is less than 10". The code in the curly brackets will only run if the condition is true.

This is the same as saying "if x is less than 10 then print 'x is less than 10' to the serial monitor". Since x is less than 10, the condition evaluates as true and we get the message printed out.

But what if the condition is false? The code in the curly brackets won't run, so we'll need a different type of logic to handle situations like that. We can do this using the else statement.

else Statement

The else statement is used to execute code if a condition is false.

int score = 20;
if (score > 50 ) {
  Serial.print("You passed the exam!");
} else {
  Serial.print("You have to rewrite the exam!");
}

// You have to rewrite the exam

In the code above, the condition given is false. So the code for the else statement will be executed because the score variable is not greater than 50.

Remember: the code for the else statement only runs when the condition is false. If the condition is true then the code for the if statement will be executed.

else if Statement

You can use the else if statement to define multiple conditions to be checked. Here's the syntax:

if (condition1) {
    // code to be executed if condition1 is true
} else if (condition2){
    // code to be executed if condition2 is true
} else {
    // code to be executed if condition1 and condition2 are false
}

In the syntax above, there are two conditions (you can create more than two conditions). If condition1 is true, then code in the curly bracket for condition1 will be executed.

If condition1 is false, then condition2 will be evaluated. If condition2 is true, its block of code will be executed.

If both condition1 and condition2 are false, the else statement's code will be executed.

int score = 80;
if (score > 50 ) {
  Serial.print("You passed the exam!");
} else if (score < 50) {
  Serial.print("You have to rewrite the exam!");
} else {
  Serial.print("No records for your exam score found!");
}

// You passed the exam!

switch-case Statement

In the last section, we saw how to create multiple conditions using else if statements. Your code might become hard to read if you have many conditions. We can clean it up and make the code more readable using switch statements.

Here's what the syntax looks like:

switch (expression) {
    case 1:
        // Code to be executed if expression equals case 1
        break;
    case 2:
        // Code to be executed if expression equals case 2
        break;
     case 3:
        // Code to be executed if expression equals case 3
        break;
    default:
        // Code to be executed if expression doesn't match any case
        break;
}

Let's break the syntax down:

• The expression is compared to the value of each case.
• When a case matches the expression, the code for that case is executed.
• The break keyword stops the switch statement's iteration once a match for the expression has been found.
• The code for the default keyword is executed if none of the cases match the expression.

Here's an example:

int day = 2;

switch (day) {
  case 1:
    Serial.print("Monday");
    break;
  case 2:
    Serial.print("Tuesday");
    break;
  case 3:
    Serial.print("Wednesday");
    break;
  case 4:
    Serial.print("Thursday");
    break;
  case 5:
    Serial.print("Friday");
    break;
  case 6:
    Serial.print("Saturday");
    break;
  case 7:
    Serial.print("Sunday");
    break;
  default:
    Serial.print("Number out of range");
  }

//  Tuesday

The code above prints "Tuesday" because the expression which has a value of 2 matches case 2.

Loops in Arduino

You can use loops to execute code repeatedly until a certain condition is met. You can also use loops to iterate over a collection of data and execute code on all elements of the collection.

There are different type of loops you can use in Arduino like the for loop, while loop, and do-while loop. Let's take a look at their syntax along with some practical examples:

for loop

You can use the for loop to iterate through a collection or execute code until a certain condition is met. It is commonly used when you know the number of times the loop is supposed to run.

Here's the syntax:

for (initialization; condition; increment/decrement) {
   // code to be executed
}

There are three important keywords in the syntax above:

• initialization denotes an initial variable (usually an integer) which specifies the starting point of the loop.
• condition is used to control the number of times the loop is expected to run for. The loop stops when the condition is false.
• increment/decrement increases/decreases the value of the initial variable after every iteration.

Here's an example to help you understand the keywords:

for (int i = 0; i < 11; i++){
  Serial.println(i);
}

// 0
// 1
// 2
// 3
// 4
// 5
// 6
// 7
// 8
// 9
// 10

In the loop above, we created an initial variable called 1 with a value of 0.

The condition stated i < 11 which implies that the loop will continue to run as long as i is less than 11.

Using the increment operator i++, we increased the value of i by 1 every time the loop ran.

Lastly, we printed the value of i at every iteration. In the serial monitor, you'll notice the numbers from 0 to 10 printed out. This is because after the number 10, i is no longer less than 11 (the condition given), so the loop terminates.

while loop

The while loop works just like the for loop — it executes code as long as the given condition is true. But its often used when the number of times the loop is supposed to run is unknown.

Here's the syntax:

while (condition) {
    // Code to be executed
}

In the syntax above, the code will continue to run until the condition becomes false.

while (i < 11) {
  Serial.println(i);
  i++;
}

// 0
// 1
// 2
// 3
// 4
// 5
// 6
// 7
// 8
// 9
// 10

do-while Loop

The do-while loop is just like the while loop, but it executes its code block first before checking the validity of the condition given. That is, at the beginning of the loop, the code in curly brackets will run first even if the condition is false. After that, it starts checking if the condition is true or false, just like a normal loop.

In summary, the code for a do-while loop will run at least once, irrespective of the condition given. Here's an example:

do {
  // code block to be executed
}
while (condition);

Here's the code example:

int i = 0;

do {
  Serial.println(i);
  i++;
} while ( i < 11);

// 0
// 1
// 2
// 3
// 4
// 5
// 6
// 7
// 8
// 9
// 10

Arrays in Arduino

You can use arrays in Arduino to store multiple variables of the same data type in a single variable. Each element stored in an array can be accessed using its index number.

Array Declaration

Declaring an array simply means to create one. You can do that in Arduino using the syntax below:

dataType arrayName[arraySize]

In the syntax above:

• dataType represents the data types that'll be stored in the array. For instance, if the data type is int, then only integers can be stored in the array.
• arrayName denotes the name of the array.
• arraySize denotes the number of elements that can be stored in the array.

Here's an array declaration code example:

int ages[4];

In the code above, we created an array called ages. The array can only store four elements with an int data type.

Array Initialization

To initialize an array means to assign values to the array. In the last section, we created an array called ages. Now, let's assign some elements to it:

int ages[4] = {2, 4, 6, 8};

You can see from the example above that there are only four elements in the array — 2, 4, 6, 8. Assigning a fifth element would throw an error because we specified that the array can only have for integer elements: int ages[4];.

You can access the elements of the array using their index number. Indexes start at zero (0) – so the first element has an index of 0, the second element has an index of 1, the third element has an index of 2, and so on.

int ages[4] = {2, 4, 6, 8};

Serial.print(ages[0]);
// 2

As can be seen above, we accessed the first element using the array name and the index of the element in square brackets: ages[0].

You can also assign and reassign values to a particular element using its index:

ages[0] = 10;

Note that you can declare and initialize an array at the same time. I only divided them into separate sections to help you understand what each term means.

Functions in Arduino

In the last chapter, we discussed some built-in functions in Arduino that can be used to carry out a variety of tasks related to Arduino hardware and software components. All we did was write the function name and pass in parameters where necessary and we got the desired outcome.

For instance, the digitalWrite() function writes values to digital pins using two parameters (the pin number and the value to be sent to the pin). Under the hood, some code logic handles that operation.

Let's assume that the logic required to send values to digital pins was up to a hundred lines of code. Without functions, you'll have to write those hundred lines every time you wanted to send values to digital pins.

Functions prevent you from having to reinvent the wheel. They also help you break your code down into smaller, more readable and manageable parts.

Just like how built-in functions can be reused to perform a particular task repeatedly, you can also create your own functions for specific functionalities, and that's exactly what you'll learn in this chapter.

How to Declare Functions in Arduino

There are four main parts of a function in Arduino:

• The type of data the function returns.
• The name of the function.
• The function parameter(s) which is optional.
• The body of the function.

Here's what that looks like:

dataType functionName(optionalParameters) {
    // body of the function
}

So from the syntax above, dataType is the data type the function returns. It can be int, String, and so on. A function that has no return statement uses the void type as its data type.

The functionName is the name given to the function. The name is used to call the function to execute the logic defined in the body of the function. You'd see words like "call", "fire", and "invoke" associated with functions. They all mean the same thing — to execute the function's logic.

optionalParameters are variables that you define when creating a function. They enable functions to accept external data which can be used within the function body. Function parameters are defined along with their data types. You'll understand this when we look at some examples.

The body of the function is where all the logic goes to. What the function does when it is invoked is written in the body.

Now that we've seen the different parts of a function, let's create some functions!

How to Declare a Function with the void Type

In the last chapter, we discussed the void Setup() and void loop() functions. They are two built-in functions that you'll use in every Arduino sketch. These functions are defined using the void keyword because they return nothing

Here's what the syntax looks like for functions that use the void type:

void functionName(optionalParameters) {
    // code logic
}

In the syntax above, functionName denotes the name of the function. We can use that name to call the function in order to execute the code defined in the function.

optionalParameters are used to pass external data to the function while the code logic that runs when the function is called is written between the curly brackets.

Here's an example:

// function declaration
void printName(String userName) {
  Serial.println("Hello " + userName);
}

void setup() {
  Serial.begin(9600);
}

void loop() {
  printName("Ihechikara"); // function call
  delay(1000);
}

In the code above, we created a function called printName which accepts a string parameter called userName. The function's task is to print "Hello " along with whatever the parameter value is.

In the void loop(), we called the function and passed a parameter to it: printName("Ihechikara"). In the serial monitor, you'll see "Hello Ihechikara" printed.

To call a function, all you have to do is write the name of the function with parenthesis: printName(). Remember to pass in parameters when required: printName("Ihechikara").

Using a parameter that has the wrong data type will result in an error. For instance, we defined a string parameter in our example. Using an integer will raise an error because the function expects a string.

How to Declare a Function with a Return Data Type

In this section, I'll use the int data type to show you how functions declared without the void type are used. The logic here is the same with other functions that use the return statement.

// function declaration
int addNums(int a, int b) {

  int result = a + b;
  return result;
}

void setup() {
  Serial.begin(9600);
}

void loop() {
  Serial.println(addNums(2, 10)); // function call
  delay(1000);
}

In the code above, we declared a function with the int type: int addNums(int a, int b) {...}. This implies that the function is expected to return an integer value.

The function's logic adds the value of the two parameters (a and b) and returns their sum. We used the return statement to return the sum of the parameters.

We can now say that the task of the addNums function is to return the sum of two given parameters. This can be seen when we used the function in the void loop():

Serial.println(addNums(2, 10));

We called the function with two parameters and got their sum printed out in the serial monitor.

What You Should Know About the return Statement

In the last two sections, we saw how to use functions in two different ways — functions that use the return statement and functions that don't use it.

But what if you used the return statement in a void function? Would that break the code? The answer is no. I'll explain why.

The main use of the return keyword is to return a value from the function, and then terminate the function. Consider the example below:

int addNums(int a, int b) {

  int result = a + b;
  Serial.println(result);
  return result;

  // This part will be ignored
  Serial.println("Hello World");
}

The function above takes in two parameters — a and b — and returns their sum. You'll notice that we printed "Hello World" after the return statement. The part of the code that comes after the return statement will not be executed because the function terminates/stops its operation once it sees a return statement.

So you should always remember that anything that comes after the return statement will be ignored.

You can use the return statement in void functions but it is a convention not to. We simply use the void keyword to define functions that have no use for the return statement.

Commonly Used Built-in Functions in Arduino Sketch

In this section, we’ll discuss some of the commonly used built-in functions you’ll come across when writing or reading Arduino code. We'll make use of them in most of the upcoming chapters of this handbook.

We'll begin with the two main parts of an Arduino sketch — the setup() and loop() functions.

setup() and loop() Functions in Arduino

You can use the setup() function to configure analog and digital pins, initialize variables, and do other setup functionalities. The setup() function is executed once — when the board is powered on or reset.

The loop() function runs continuously. This part of the sketch is where you write all the code logic. You can use the loop() function to give the Arduino board instructions on different components and sensors.

void setup() {
  // put your setup code here, to run once:

}

void loop() {
  // put your main code here, to run repeatedly:

}

pinMode() Function in Arduino

The pinMode() function is used to configure pins as input or output pins. It can also be used to configure a resistor to act as either a pull-up or pull-down resistor. You'll understand more about this function in the Sensors and Actuators chapter.

Syntax

pinMode(pin, mode)

• pin denotes the pin number on an Arduino board.
• mode denotes the configuration of the pin which can be INPUT, OUTPUT, or INPUT_PULLUP.

digitalRead() Function in Arduino

You can use the digitalRead() function to read the state of digital pins. It returns either 0 (LOW) or 1 (HIGH).

Syntax

digitalRead(pin)

In the code above, pin denotes the pin number on an Arduino board.

digitalWrite() Function in Arduino

The digitalWrite() function assigns or writes values (either HIGH or LOW) to digital pins.

Syntax

digitalWrite(pin, value)

• pin denotes the pin number on an Arduino board.
• value denotes the value to be assigned to pin. Can be HIGH or LOW.

analogRead() Function in Arduino

The analogRead() function reads values from analog pins and returns values that fall within the range of 0 and 1023.

Syntax

analogRead(pin)

In the code above, pin denotes the pin number on an Arduino board.

analogWrite() Function in Arduino

This function writes or assigns an analog value to a pin.

Syntax

analogWrite(pin, value)

• pin denotes the pin number on an Arduino board.
• value denotes the value to be assigned to pin. Range from 0 to 255.

Serial Functions in Arduino

Serial communication enables an Arduino board to communicate with the computer and other devices using the built-in serial monitor. Here are some of the commonly used functions:

Serial.begin()

The Serial.begin() function initializes serial communication. It is the first function you use when working with the serial monitor. The function takes in a baud rate as its parameter.

In this case, baud rate represents the rate or speed of data transfer in serial communication.

Here's the syntax:

Serial.begin(baudRate)

Serial.print() and Serial.println()

You can use the print() and println() functions to print data to the serial monitor.

print(val)
println(val)

In the code above, val denotes the value to be printed.

We'll talk more about serial communication in Chapter 6: How to use the Serial Monitor in Arduino.

delay() Function in Arduino

You can use the delay() function to pause the Arduino program for a specified amount of time. Here's what the syntax looks like:

delay(ms)

In the code above, ms denotes the specified time in milliseconds.

Chapter 3: How to Use Digital Pins in Arduino

Digital pins are used to send and receive digital signals in two states — HIGH and LOW. The digital pins on a Arduino board can be configured as either input or output pins.

These states can also be represented using numbers (1 for HIGH and 0 for LOW), or in volts (V) (5V for HIGH and 0V for LOW).

The number and arrangement of pins differ in various Arduino boards, but they serve the same purpose. So if you understand how to use them in this chapter, you won't have a problem working with them in other boards.

The Uno board has 14 digital pins numbered from 0 to 13. Although each pin can be configured to serve as either a digital input or output pin, some of them come with extra functionalities like:

• Pins 0 (RX) and 1 (TX) enable the Arduino board to communicate serially. RX receives while TX sends.
• Pins that have the tilde (~) symbol beside them support PWM (Pulse Width Modulation) signals. This means that you can use these pins like analog pins (to receive analog values).
• Pins 2 and 3 can be used for interrupt-based functionalities.

How to Configure Digital Pins as Input or Output Pins

When a digital pin is configured as an INPUT pin, it serves as a point for receiving information from components. This way you get data from sensors, electronic components, and so on.

You can use the pinMode() function to configure a pin to serve as either an INPUT or OUTPUT pin. Note that the pins on an Uno board are set to INPUT by default so if you don't specify prior to using them, they'll serve as input pins.

In this section, you'll learn how to use digital pins as both input and output pins. You'll start by learning about them individually, and then see how to combine input and output signals to build a mini project.

We'll start with INPUT.

Digital Pins as INPUT

The information or signal sent to input pins can be read using the digitalRead() function. In this section, you'll learn how to configure and read signals from a digital input pin using different built-in functions.

We'll use the following hardware components:

• Arduino Uno.
• Breadboard.
• Pushbutton.
• Jumper wires.

Here's the diagram for the connection:

Configuration diagram – INPUT

Here's the code:

int pushBtn = 7;
int push_btn_state;

void setup(){
  pinMode(pushBtn, INPUT_PULLUP);
  Serial.begin(9600);
}

void loop(){
  push_btn_state = digitalRead(pushBtn);
  Serial.println(push_btn_state);
  delay(1000);
}

Let's break the code down.

We started by creating two integer variables — pushBtn and push_btn_state:

int pushBtn = 7;
int push_btn_state;

pushBtn was assigned a value of 7. We used this value to denote pin 7 on the Arduino board. We declared the push_btn_state variable but didn't assign any value to it because we'll use it to store the value of the push button later.

In our setup() function, we configured the push button to act as an input pin using the pinMode() function:

pinMode(pushBtn, INPUT_PULLUP);

The function took in two parameters — the pushBtn variable which denoted pin 7, and INPUT_PULLUP which sets the pin as an input with a pull-up resistor.

We also initialized the serial monitor using Serial.begin(9600).

At this point, we've configured the Arduino software and hardware to recognize pin 7 as an input pin.

Next, we used the digitalRead() function to read signals coming from pin 7. Remember the push_btn_state variable we created? That's where we stored the signal:

push_btn_state = digitalRead(pushBtn);

After that, we printed the value being read from pin 7 to the serial monitor using Serial.println(push_btn_state);.

When you open the serial monitor, you'll see 1 being printed out repeatedly. This is the initial state of the pushbutton using a pull-up resistor. When you press down the pushbutton, the value will become 0. When you release the button, the value will become 1.

0 denotes LOW while 1 denotes HIGH. With this example, you should understand how to configure, read, and display signals from an input pin.

Digital Pins as OUTPUT

The main use of an output pin is to send out signals. Since we're working with digital output, we can send either HIGH (5V) or LOW (0V) signals. We can do this for digital pins using the digitalWrite() function.

In this section, we'll use a LED (Light Emitting Diode) to demonstrate how to configure and send signals to components.

Here are the components we'll use:

• Arduino Uno.
• Red LED.
• 1k Ohm resistor.
• Jumper wires.
• Breadboard.

Here's the circuit diagram:

Configuration diagram – OUTPUT

int RedLED = 8;

void setup(){
  pinMode(RedLED, OUTPUT);
}

void loop(){
  digitalWrite(RedLED, HIGH);
  delay(1000);
  digitalWrite(RedLED, LOW);
  delay(1000);
}

In the code above, we configured the red LED, which is connected to pin 8 on the Uno board, as an output pin using the pinMode() function.

We then used the digitalWrite() function to send signals to the pin:

  digitalWrite(RedLED, HIGH);
  delay(1000);
  digitalWrite(RedLED, LOW);
  delay(1000);

With the HIGH parameter, we send 5V to the pin which makes the LED come on. With LOW, we send 0V which turns the LED off. So the LED comes on and off continuously with a delay of 1000 milliseconds. This example is commonly known as the "BLINK" example.

Digital I/O Project

Now that we've understood how to send and receive digital signals using Arduino, let's combine both of them to build an interactive project.

The idea is to control an LED using a pushbutton. When you press the button, the LED goes off, and comes back on when you release the button.

We'll combine the components used in the previous examples:

• Arduino Uno.
• Red LED.
• 1k Ohm resistor.
• Pushbutton.
• Jumper wires.
• Breadboard.

Configuration diagram - INPUT and OUTPUT project

Here's the code:

int pushBtn = 7;
int push_btn_state;
int RedLED = 8;

void setup(){
  pinMode(pushBtn, INPUT_PULLUP);
  pinMode(RedLED, OUTPUT);
}

void loop(){
  push_btn_state = digitalRead(pushBtn);

  if (push_btn_state == 1) {
    digitalWrite(RedLED, HIGH);
  } else {
    digitalWrite(RedLED, LOW);
  }
}

Here's a breakdown of the code above:

First, we connected the pushbutton to pin 7 and the red LED to pin 8.

Then we configured both pins — pin 7 as input, and pin 8 as output.

Next, we read the value of the pushbutton using the digitalRead() function and stored the value in a variable called push_btn_state.

Using an if statement, we checked for the state of the pushbutton. When push_btn_state is not being pushed down, it has a value of 1 (HIGH). We send 5V to the LED using digitalWrite().

When push_btn_state is pushed down, it has a value of 0, and we send 0V to the LED which turns it off.

Chapter 4: How to Use Analog Pins in Arduino

Analog pins can be used to receive and send voltage values from/to different sensors and components. Unlike digital signals that fall within two states of 0 (LOW) and 1 (HIGH), analog values have a wider range of values from 0 to 1023.

The Uno board has six analog pins — A0, A1, A2, A3, A4, and A5. These pins are INPUT pins by default.

Similar to digital pins, there are in-built functions for receiving and sending analog voltage signals. You can use the analogRead() function to read/receive analog values from pins while the analogWrite() function can be used to write to specific pins.

Note that the analogWrite() function doesn't write or send analog values to analog pins. It sends analog voltage values (that get converted to digital signals) to digital pins that support PWM (Pulse Width Modulation).

You can find PWM digital pins on the Arduino board by the (~) symbol beside them. On the Uno board, there are pins 3, 5, 6, 9, 10, and 11.

In the project for this section, you'll learn how to adjust the brightness of an LED using a potentiometer. We'll use the potentiometer as an input component, and send its voltage to an LED to increase/decrease the LED's brightness.

Here are the components we'll use:

• Arduino Uno.
• Yellow LED.
• Potentiometer.
• 1k Ohm resistor.
• Jumper wires.
• Breadboard.

Here's the circuit diagram:

Configuration diagram

Here's the code for the circuit:

int potentiometer = A0;
int pot_value = 0;
float pot_in_PWM;
int yelowLED = 6;

void setup(){
  pinMode(potentiometer, INPUT);
  pinMode(yelowLED, OUTPUT);
}

void loop(){
  pot_value = analogRead(potentiometer);
  pot_in_PWM = pot_value * (255.0 / 1023.0);

  analogWrite(yelowLED, pot_in_PWM);
}

In the code above, we set the potentiometer (connected to analog pin A0) as an input component. The yellow LED (connected to digital pin 6) was set as an output component. The LED is connected to a PWM pin because we'll be sending analog values to it.

Using the analogRead() function, we got the value of the potentiometer and stored it in a variable called pot_value. The potentiometer returns values from 0 to 1023.

We then converted the values returned by the potentiometer to a range of 0 to 255 which corresponds with PWM values. This is because the analogWrite() function sends PWM values that fall within that range (0 to 255). The new range of values was stored in a variable called pot_in_PWM.

Lastly, we used the analogRead() function to send PWM values to the LED: analogWrite(yelowLED, pot_in_PWM).

When you verify and upload the code to your Arduino board, you'll be able to control the brightness of the LED by turning the potentiometer's knob.

Chapter 5: How to Use Sensors and Actuators in Arduino

Sensors and actuators play a crucial role in developing projects using Arduino. They help microcontrollers get information about changes in the physical environment, and make decisions based on that information.

The goal of this chapter is to help you understand the difference between sensors and actuators, and how to use them. Of course, there are numerous sensors and actuators, but we'll focus on just a few. You should be able to explore other sensors on your own at the end of this chapter.

We'll first talk about what sensors are, types of sensors, and how to work with them using code. We'll then do the same for actuators, and conclude by looking at an example that uses both sensors and actuators in a project.

Let's get started!

What are Sensors in Arduino?

A sensor is a device that listens for or detects changes in the environment. Sensors convert the information they detect in the physical environment to electrical signals which are then sent to the microcontroller (the brain of the board).

You can look at sensors like the human sense organs — we use them to gather information about our environment. Each sense organ (like eyes and ears) in the human body has a specific function and mode of operation that's different from other organs.

In the same manner, sensors in Arduino have their specific functions and use cases. For example, we have sensors that can measure and detect temperature, motion, moisture, and so on.

Types of Sensors in Arduino

Here are some commonly used sensors you'll come across when working with Arduino:

• Temperature sensor: Measures the temperature and humidity of the environment.
• Light-Dependent Resistor (LDR): Measures/senses the intensity of light.
• Ultrasonic sensor: Measures the distance of an object from the sensor.
• Motion sensor: Generally detects motion by sensing changes in infrared energy/radiation.
• Soil moisture sensor: Measures the moisture level of the soil.
• Water sensor: Measures water level/detects water, and so on.

There are other types of sensors that you can use with Arduino, but we'll just focus on two: the LDR (light-dependent resistor) and ultrasonic sensor.

With the examples in this section, you'll be able to explore how other sensors work on your own.

How to use the Light-Dependent Resistor (LDR) in Arduino

The light-dependent resistor (LDR), also known as a photoresistor, is an electronic component that has varying levels of resistance depending on the intensity of light.

Basically, you can use this sensor to sense light. You can do cool things with it like creating an automated lighting system that turns on electric lights in your house when it's dark, or a weather station that tracks and monitors sunlight, and so on.

A LDR usually looks like this:

Diagram of an LDR

Here's what the circuit connection looks like:

Configuration diagram

In the circuit above, one leg of the LDR is connected to 5V (volts). The other leg is connected to a 1K Ohm resistor — one end of the resistor is connected to ground (GND) while the other end is connected to an analog pin (A1 in our circuit).

Here's the code:

int ldrPin = A1;
int ldrValue;

void setup() {
  pinMode(ldrPin, INPUT);
  Serial.begin(9600); 
}

void loop() {
  ldrValue = analogRead(ldrPin);
  Serial.println(ldrValue);
  delay(1000); 
}

In the code above, we created a variable called ldrpin with a value of A1. This denotes the connection made in the circuit where one leg of the LDR is connected to the analog pin. This pin will help us know the value of the sensor.

We then created an ldrValue variable which will be used to store the value of the sensor.

In the setup() function, we set the LDR as an INPUT pin. We also initialized the serial monitor.

Next, we read the value of the LDR using the analogRead function, and stored the value in the ldrValue variable:

ldrValue = analogRead(ldrPin);

Lastly, we printed the read value to the serial monitor with a delay of 1000 milliseconds (one second).

At this point, if you increase the exposure of light on the LDR, the value increases. If you decrease the light intensity or cover the sensor to block off light, the value will decrease or become zero, respectively.

How to Use the Ultrasonic Sensor in Arduino

The ultrasonic sensor is generally used to measure the distance of objects. You can use this sensor in a lot of applications like:

• Home automation (you can perform certain actions when the presence of an object/human is sensed, like turning on lights in a dark room).
• Automated doors.
• Security systems.
• Measurement of distance, and so on.

Just like other sensors, there are various types. In this example, we'll use the HC-SR04 ultrasonic sensor. Don't worry, the working principle is the same. So if you understand how to use this one, you can configure and use other types of ultrasonic sensors you come across.

Here's what the circuit diagram looks like:

Configuration diagram (from https://howtomechatronics.com/tutorials/arduino/ultrasonic-sensor-hc-sr04/)

The sensor has four pins — VCC, Trig, Echo, and GND.

• The VCC pin is connected to 5V on the Uno board
• The Trig pin is connected to digital pin 9.
• The Echo pin is connected to digital pin 10.
• The GND pin is connected to GND on the Uno board.

The trig pin is used to "trigger" the ultrasonic sensor while the echo pin is used to distance of objects based on the amount of time it takes for ultrasonic waves/signals to bounce back from an object.

Here's a code example:

int trigPin = 9; 
int echoPin = 10;  

long duration, distance;

void setup() {
  Serial.begin(9600);
  pinMode(trigPin, OUTPUT);
  pinMode(echoPin, INPUT);
}

void loop() {
  digitalWrite(trigPin, HIGH);
  delayMicroseconds(10);
  digitalWrite(trigPin, LOW);

  duration = pulseIn(echoPin, HIGH);

  distance = (duration / 2) * 0.0343;

  Serial.print("Distance: ");
  Serial.print(distance);
  Serial.println(" cm");

  delay(1000);
}

In the code above, the trig pin denotes digital pin 9, and the echo pin denotes digital pin 10.

We declared two variables — duration and distance — to be used to store their respective values.

We triggered the sensor by sending the HIGH signal to the trigger pin for 10 microseconds. Without this, the sensor might not work:

  digitalWrite(trigPin, HIGH);
  delayMicroseconds(10);
  digitalWrite(trigPin, LOW);

We measured and stored the duration of the ultrasonic pulse/signals using the pulseIn(echoPin, HIGH) function and stored it in the duration variable.

We then calculated the duration in centimeters and stored it in the distance variable.

Lastly, we printed the distance to the serial monitor with a delay of 1000 milliseconds.

At this point, you can place an object closer to or farther away from the sensor and see the value of the distance of the object change in the serial monitor.

What are Actuators in Arduino?

Actuators in Arduino are components that convert electrical signals into physical/mechanical motion.

Here are some actuators:

• LED (Light Emitting Diode): Used as light/visual indicators.
• Buzzer: Used to produce sound.
• Relay modules: Used to control high voltage devices.
• LCD (Liquid Crystal Display): Used as visual display for text, images, sensor data, and so on. We'll dedicate a separate chapter to displays.
• Servo motors: Used to control angular or rotational motion (an example is the movement of a robotic arm).

We'll focus on the buzzer and LED in this section.

How to use LEDs in Arduino

LEDs are usually the first components you learn about in Arduino. There are easy to connect and use.

Here's what an LED looks like:

Here's the circuit connection:

In the circuit above, the longer leg of the LED (the anode or positive leg) is connected to digital pin 7.

The shorter leg (cathode) is connected to ground.

Here's the code:

int redLED = 7;

void setup() {
  pinMode(redLED, OUTPUT);
}

void loop() {
  digitalWrite(redLED, HIGH);
  delay(1000);
  digitalWrite(redLED, LOW);
  delay(1000);
}

The code above will make the LED blink once the code has been uploaded.

How to use a Buzzer in Arduino

You can use buzzers to produce sound. There are generally two types of buzzers: active and passive buzzers.

Active buzzers usually have a predefined type of sound that they produce when voltage is supplied to them. The sound cannot be modified. Active buzzers have an internal circuit that triggers the sound production.

Passive buzzers are a bit more flexible when it comes to producing sound because they rely on external signals. This means that you can determine the frequency (or frequencies) of the sound produced by the buzzer.

Buzzers can be used in different applications like:

• Alarm systems.
• Sound indictor for home appliances.
• Doorbells.
• Communication devices to denote the start/end of a signal communication, and so on.

We'll work with the passive buzzer because of its flexibility. This is what it looks like:

The buzzer has a positive and negative terminal. The positive is connected to a digital pin while the negative is connected to ground.

Here's the circuit diagram:

In this example, we'll use the buzzer to produce the DO, RE, MI, FA, SOL, LA, TI music notes sound.

Here's the code:

int buzzerPin = 7;

int notes[] = {262, 294, 330, 349, 392, 440, 494};

int noteDurations[] = {400, 400, 400, 400, 400, 400, 400};

void setup() {
  pinMode(buzzerPin, OUTPUT);
}

void loop() {
  for (int i = 0; i < 8; i++) {
    tone(buzzerPin, notes[i], noteDurations[i]);
    delay(noteDurations[i] + 50);
    noTone(buzzerPin);
  }

  delay(1000);
}

In the code above, we used the buzzerPin variable to denote pin 7 on the Uno board.

We then created an array called notes[] which stores the respective frequency of each note, and another array called noteDurations which stores the duration of each note as they're being produced by the buzzer.

In the void loop() function, we looped through each note a played them using the tone() function. The function takes in three parameters — the pin connected to the buzzer, the frequency, and the duration.

We used the noTone() function to stop the generation of sound. Lastly, we added a delay of 1000 milliseconds which is the amount of time it'll take to play the notes from the start after the last note has been played.

If you've followed along up to this point, then you should have the DO, RE, MI, FA, SOL, LA, TI music notes playing through the buzzer.

Arduino Sensor and Actuator Example

Now that you understand what sensors and actuators are and how to use them, let's use them together in one project.

In many embedded systems, sensors and actuators work together to get a specific task/functionality done. Here's how that works:

• A sensor detects/senses changes in the environment, and sends signals to the microcontroller to notify it of the detected changes.
• The microcontroller processes the signals from the sensor. Depending on the existing logic (defined in the code), it sends signals to the actuator.
• The actuator converts the signal from the microcontroller to physical/mechanical motion.

Let's demonstrate these using an example. The idea here is to turn on an LED in a dark environment and turn it off when there's enough light.

The sensor will be the LDR sensor, the microcontroller will be the Arduino Uno board/software, and the actuator will be the LED.

Here's the circuit diagram:

Here's the code:

int redLED = 7;

int ldrPin = A1;
int ldrValue;

void setup() {
  pinMode(ldrPin, INPUT);
  pinMode(redLED, OUTPUT);
  Serial.begin(9600); 
}

void loop() {
  ldrValue = analogRead(ldrPin);

  if (ldrValue > 200) {
    digitalWrite(redLED, LOW);
  } else if (ldrValue < 200) {
    digitalWrite(redLED, HIGH);
  }

  Serial.println(ldrValue);
  delay(1000); 
}

As usual, we created variables to represent the Arduino pins connected to the LED and LDR. We then set the LDR as INPUT and the LED as output in the setup() function.

We read the value of the LDR using the analogRead() function and stored it in the ldrValue variable.

Next, we used the an if statement to check the value of the LDR.

If the value is greater than 200, we turn off the LED. If the value is less than 200, we turn on the LED.

Chapter 6: How to Use the Serial Monitor in Arduino

The serial monitor is a useful tool for every Arduino builder. You can use it for a variety of tasks like:

• Debugging and testing code/components.
• Serial communication between the Arduino board and the computer.
• Display sensor and component data and readings.

In this chapter, you'll learn how to initialize and use the serial monitor using the Arduino IDE. You'll learn about different built-in functions that can be used to send and receive data between the Arduino board and the computer.

Lastly, you'll build a project that uses values sent from the serial monitor to power specific LEDs connected to the Arduino board.

How to Initialize the Serial Monitor With Serial.begin()

You can use the Serial.begin() function to initialize the serial monitor. It takes in the baud rate as its parameter. Here's what the syntax looks like:

Serial.begin(baudRate)

The baud rate is the speed of data transfer between the Arduino board and the computer or any other device communicating with the Arduino board through the serial monitor.

The most commonly used baud rate is 9600, but you'll also come across resources that make use of 115200, 57600, and 38400, and so on. Whichever baud rate you specify in the Serial.begin() function should always match the baud rate seen in the serial monitor window.

For example, we can initialize the serial monitor in the IDE using the Serial.begin() function like this:

void setup() {
  Serial.begin(9600);

}

void loop() {
  // put your main code here, to run repeatedly:

}

In the code above, using the Serial.begin() function, we initialized the serial monitor in the setup() function with a baud rate of 9600.

At this point, nothing is happening. You can verify that the baud rates match by opening the serial monitor window. Once you have it open, you should have the serial monitor appear at the base of the IDE. You'll see the baud rate of the serial monitor within the window, usually written like "9600 baud".

Image showing baud rate

If you're using an older version of the IDE, the serial monitor may pop up as a separate window. The functionality is still the same.

Now that we've initialized the serial monitor, let's see how to send and receive data with it.

How to Send Data with Serial Monitor

You can use different built-in functions reserved for serial communication in Arduino. We won't discuss all the built-in serial functions in Arduino – we'll just look at some you'll use/come across regularly. You can see more functions here.

print() and println() Functions

The print() and println() functions both print data to the serial monitor. The difference between the two is that print() prints data on the same line while println() prints each data on a new line.

Here are some examples:

void setup() {
  Serial.begin(9600);
}

void loop() {
  Serial.print("Hello"); 
  delay(1000);
}

In the code above, we used the print() function to print "Hello" to the serial monitor repeatedly with a delay of 1000 milliseconds. Here's what the output looks like in the serial monitor:

Here's another example using the println() function:

void setup() {
  Serial.begin(9600);
}

void loop() {
  Serial.println("Hi");
  delay(1000);
}

Here's the output in the serial monitor:

How to Receive Data with Serial Monitor

We'll discuss four functions in this section — available(), readString(), parseInt(), and parseFloat(). You can read up on other serial functions here.

available() Function

The Serial.available() function checks the number of characters in the serial port. It is mostly use to run code only when data is available in the serial monitor.

Here's a code example:

int userInput;

void setup(){
  Serial.begin(9600);
}

void loop(){
  if (Serial.available() > 0) {
    userInput = Serial.parseInt();

    Serial.println(userInput);
  }

}

In the code above, the code block in the if statement will not run until the number of characters in the serial monitor is greater than 0.

When you initialize the serial monitor, the number of readable characters will be zero because there is no data available yet. The Serial.available() function checks and returns the number of characters available in the serial monitor. So it is used to create logic that only lets code be executed when data is available.

readString() Function

You can use the readString() function to read characters from the serial monitor. It returns a string object so whatever values/characters you input in the serial monitor will be seen as string values when using the readString() function.

String userInput;

void setup(){
  Serial.begin(9600);
}

void loop(){
  if (Serial.available() > 0) {
    userInput = Serial.readString();

    Serial.println(userInput);
  }

}

parseInt() Function

The parseInt() function returns valid integer values from incoming serial data. Non integer values will be returned as 0.

int userInput;

void setup(){
  Serial.begin(9600);
}

void loop(){
  if (Serial.available() > 0) {
    userInput = Serial.parseInt();

    Serial.println(userInput);
  }

}

parseFloat() Function

The parseFloat() function returns valid floating point numbers from incoming serial data.

float userInput;

void setup(){
  Serial.begin(9600);
}

void loop(){
  if (Serial.available() > 0) {
    userInput = Serial.parseFloat();

    Serial.println(userInput);
  }

}

Serial Monitor Project

In this section, you'll build a project that uses string values from the serial monitor to turn on LEDs. We'll make use of some of the serial functions discussed in previous sections.

Here's the circuit diagram:

In the circuit diagram above, we have three LEDs. The red LED is connected to pin 6, the blue LED to pin 5, and the yellow LED to pin 4.

Here's the code:

int redLED = 6;
int blueLED = 5;
int yellowLED = 4;

String userInput;

void setup(){
  pinMode(redLED, OUTPUT);
  pinMode(blueLED, OUTPUT);
  pinMode(yellowLED, OUTPUT);
  Serial.begin(9600);

  Serial.println("Choose an LED to turn on from the list below:");
  Serial.println("red");
  Serial.println("blue");
  Serial.println("yellow");
}

void loop(){

  if (Serial.available() > 0) {

    userInput = Serial.readString();

    if (userInput == "red") {
      digitalWrite(redLED, HIGH);
      digitalWrite(blueLED, LOW);
      digitalWrite(yellowLED, LOW);
    } 

    if (userInput == "blue") {
      digitalWrite(redLED, LOW);
      digitalWrite(blueLED, HIGH);
      digitalWrite(yellowLED, LOW);
    }

    if (userInput == "yellow") {
      digitalWrite(redLED, LOW);
      digitalWrite(blueLED, LOW);
      digitalWrite(yellowLED, HIGH);
    }

  }

}

Let's break down the code above.

Just like the connection in the circuit diagram, we initialized the three LEDs with their respective pin numbers. We also created a string variable called userInput that will be used to store the data from the serial monitor:

int redLED = 6;
int blueLED = 5;
int yellowLED = 4;

We then configured the three LEDs as output pins using the pinMode() function:

pinMode(redLED, OUTPUT);
pinMode(blueLED, OUTPUT);
pinMode(yellowLED, OUTPUT);

We initialized the serial monitor with a baud rate of 9600, and printed some strings to give users a hint at the expected values to be used in the serial monitor:

Serial.begin(9600);

Serial.println("Choose an LED to turn on from the list below:");
Serial.println("red");
Serial.println("blue");
Serial.println("yellow");

At this point, you'll have an output like this in the serial monitor:

Using the Serial.available() function, we check for availability of serial data before running the code in the if statement:

if (Serial.available() > 0) {
    ...
}

The next thing we did was to read and return the incoming data as string values, and store the data in the userInput variable. We did that using the readString() function:

userInput = Serial.readString();

Lastly, we used if statements to check which LED color value the user has typed in/sent through the serial monitor, then turn on/off the respective LEDs:

if (userInput == "red") {
  digitalWrite(redLED, HIGH);
  digitalWrite(blueLED, LOW);
  digitalWrite(yellowLED, LOW);
} 

if (userInput == "blue") {
  digitalWrite(redLED, LOW);
  digitalWrite(blueLED, HIGH);
  digitalWrite(yellowLED, LOW);
}

if (userInput == "yellow") {
  digitalWrite(redLED, LOW);
  digitalWrite(blueLED, LOW);
  digitalWrite(yellowLED, HIGH);
}

If you type and send red through the serial monitor, the red LED comes on while the others goes off. The same logic applies to sending blue or yellow through the serial monitor.

Chapter 7: How to Use Displays in Arduino

You can use display components in Arduino to represent data visually in different formats like text, images, and so on.

Displays provide an alternative to showing data other than the serial monitor. You can use them to display sensor values, instructions to users, user input, and so on.

There are various types of display components when it comes to building with Arduino but we'll focus on the LCD (Liquid Crystal Display).

To be more specific, we'll use the 16 x 2 LCD (HD44780). This is what it looks like:

From the image above, there are sixteen pins with different values. We'll use most of these pins to make connections to the Uno board which will enable us use and control the component.

There are generally two ways of connecting the LCD — using the built-in library or using a third-party library.

With the built-in LiquidCrystal library, you'll make use of about 12 pins on the LCD component. When it comes to using a third-party library like LiquidCrystal_I2C, you'll make use of just 2 pins.

We'll be using the first method — this will help you understand how the pins work. Now let's talk about those pins.

What are the pins in an LCD used for?

The pins control functionalities like memory management, data transfer, and power on an LCD. Let's talk about them in detail:

• The first pin from the left is the GND pin, which is the LCD's ground pin. It is connected to GND on the Arduino board. In some LCD modules, it may be written as VSS.
• The VCC is the power pin which is connected to 5V on the Arduino board. In some LCD modules, it may be written as VDD.
• The V0 pin is used to adjust the contrast of the LCD. It is connected to a potentiometer. Tuning the potentiometer changes the contrast of the data being displayed.
• The RS (Register Select) pin can be used to control the LCD's memory. It is usually connected to a digital pin.
• The RW (Read/Write) pin controls whether data is written or read to the LCD. It is usually connected to ground (GND) which sets the LCD to write mode — this enables you to send and display data with the LCD.
• The E (Enable) pin "enables" data transfer between the microcontroller (in our case, the Uno board) and the LCD. It is usually connected to a digital pin.
• The D0 to D7 (can also be written as DB0 to DB7) are the data pins. There are used to send data to the LCD in bits. In most cases, the D4 to D7 pins are used. The main difference is that data is sent in 4-bits with the D0 to D3 pins, while data is sent in 8-bits with the D4 to D7 pins. There are usually connected to digital pins.
• The LED pins are used to control the backlight of the LCD. In some LCD modules, they may be written as BLA and BLK, or A and K. The first LED pin is connected to 5V using a resistor while the second LED pin is connected to ground (GND).

Example #1 – How to Connect and Use an LCD with Arduino

In the last section, we talked about the meaning of the pins on an LCD. In this section, you'll see a practical example on how to connect them in a circuit, and write code to display data on the LCD.

We'll use the following components:

• Arduino Uno.
• 16 x 2 LCD.
• Potentiometer.
• Jumper wires.
• Resistors.
• Breadboard.

Here's what the circuit diagram looks like:

In the circuit above, we made the following connections from the LCD to the Uno board:

• The LCD's GND pin was connected to GND on the Uno board.
• The LCD's VCC pin was connected to 5V on the Uno board.
• The V0 pin was connected to the potentiometer.
• The RS pin was connected to digital pin 4 on the Uno board.
• The RW pin was connected to GND on the Uno board.
• The LCD's D4 was connected to digital pin 6 on the Uno board.
• The LCD's D5 was connected to digital pin 7 on the Uno board.
• The LCD's D6 was connected to digital pin 8 on the Uno board.
• The LCD's D7 was connected to digital pin 9 on the Uno board.
• The first LED pin was connected to 5V.
• The second LED pin was connected to GND.

For the potentiometer, one of the outer legs (either the left or right) was connected to 5V, the other outer leg was connected to GND. Then the middle leg of the potentiometer was connected to the V0 pin on the LCD. This will enable you to control the contrast of the LCD.

Next, we'll write some code!

#include <LiquidCrystal.h>

LiquidCrystal lcd(4, 5, 6, 7, 8, 9);

void setup() {
  lcd.begin(16, 2);
  lcd.print("freeCodeCamp!");
}

void loop() {

}

In the code above, we first included/imported the built-in LiquidCrystal library which can be used to interact with an LCD using Arduino code: #include <LiquidCrystal.h>.

We then initialized the lcd object with the required pin numbers: LiquidCrystal lcd(4, 5, 6, 7, 8, 9). The first number denotes the RS pin. The second number denotes the E pin. The last four numbers denote the data pins (D4 to D7).

Next, we initialized the number of columns and rows using the lcd.begin() function: lcd.begin(16, 2). We have 16 columns and 2 rows.

The lcd.print function prints data to the LCD. If you've made all the connections right, you should see "freeCodeCamp" displayed on the LCD.

In the next examples, we'll work with user input and sensor data, and also see other lcd functions.

Example #2 – How to Display User Input with LCD in Arduino

In this example, we'll accept input from the user using the serial monitor, and display the input along with a welcome message on the LCD screen.

We'll make use of the same circuit in the previous example. Here's the code:

#include <LiquidCrystal.h>
LiquidCrystal lcd(4, 5, 6, 7, 8, 9);

String userInput;

void setup() {
  Serial.begin(9600);
  lcd.begin(16, 2);

  lcd.print("Input name");

  lcd.setCursor(0, 1);
}

void loop() {
  if (Serial.available() > 0) {

    userInput = Serial.readString();
    lcd.print("Welcome " + userInput);
  }
}

In the code above, we created a string variable called userInput which will be used to store the value of the user's input.

We then initialized the serial monitor — Serial.begin(9600);

Next, we set the rows and columns of the LCD using the lcd.begin function.

Using the lcd.print function, we printed "Input name" on the LCD screen.

This was followed by the lcd.setCursor(0, 1) function which has two parameters — 0 and 1. 0 represents first column on the left while 1 represents the second row. So the cursor will be set to the second line of the LCD.

We set the cursor to be on the second line because we already have some text ("input name") on the first line so it would be more ideal in this case to print the message to the user on the next line. I'll show you an alternative way to do this after this example.

In the void loop() function, we used the Serial.readString() function to read the value of user's input and stored it in the userInput variable.

Lastly, we printed a welcome message and the user's name in the LCD. This will be printed on the second line because that's where we set the cursor to be.

Play around with the code and see what happens as you adjust the parameters of the lcd.setCursor() function. This will help you understand it better.

Example #2 Alternative

An alternative way to print the message to the user is by using the same line where the initial message was displayed. We can clear whatever is written on the first row of the LCD and display a different value.

Here's how:

#include <LiquidCrystal.h>
LiquidCrystal lcd(4, 5, 6, 7, 8, 9);

String userInput;

void setup() {
  Serial.begin(9600);
  lcd.begin(16, 2);

  lcd.print("Input name");
}

void loop() {
  if (Serial.available() > 0) {
    lcd.clear();
    userInput = Serial.readString();
    lcd.print("Welcome " + userInput);
  }
}

In this example, we used the lcd.clear() function to clear the initial request message and display/print the welcome message to the user. The rest of the logic is the same.

So when you type in a name in the serial monitor an hit enter, the request message ("Input name") will be cleared while the welcome message will be displayed with the name you inputted.

Example #3 – How to Display Sensor Data with LCD in Arduino

In this example, we'll display data from a sensor on the LCD. We'll use the temperature sensor. The logic here is to sense the temperature of a room and display the varying values on the LCD.

Here are the components we'll use:

• Arduino Uno.
• 16 x 2 LCD.
• Light-Dependent Resistor (LDR).
• Potentiometer.
• Jumper wires.
• Resistors.
• Breadboard.

Here's the circuit diagram:

The circuit diagram is almost the same as the ones in the previous examples, except that we added an LDR to measure the intensity of light. Using the LDR, we'll read, store, and display the value of light intensity on the LCD.

Here's the code:

#include <LiquidCrystal.h>
LiquidCrystal lcd(4, 5, 6, 7, 8, 9);

int ldrPin = A1;
int ldrValue;

void setup() {
  pinMode(ldrPin, INPUT);
  lcd.begin(16, 2);
}

void loop() {
  ldrValue = analogRead(ldrPin);

  lcd.clear();
  lcd.setCursor(0, 0);
  lcd.print("LDR value:");

  lcd.setCursor(0, 1);
  lcd.print(ldrValue);

  delay(1000);
}

Let's break down the code.

We started by including the built-in LiquidCrystal library and initialized the respective pins for working with the LCD (refer to the first example if you do not understand how the connection and initialization works).

We then created two variables: ldrPin which denoted the analog pin connected to the LDR, and ldrValue to store the readings from the LDR.

In void setup(), we configured the LDR as an input pin and set the columns and rows of the LCD.

Next, we read the value of the LDR using the analogRead function and stored that value in the ldrValue variable.

Using the lcd.clear() function, we clear the display after each reading. This helps in making the reading appear more accurately. If you remove the function, then some values will still be on display and make your results appear inaccurately (I spent about thirty minutes figuring this out 😂).

After that, we set the cursor on the first row and column of the LCD: lcd.setCursor(0, 0), and displayed a message ("LDR value:").

Using the lcd.setCursor(0, 1) function, we moved the cursor down to the second row, and used lcd.print(ldrValue) to print/display the value of the LDR on the LCD.

Lastly, we added a delay of 1000 milliseconds.

Now you can see the value of light intensity in your environment displayed on the LCD.

You can use a flashlight to increase and decrease the exposure of light on the LDR, and then observe the values change on the LCD. Pretty cool!

Conclusion

Congratulations! We've come to the end of this handbook. You now have enough knowledge to take on bigger projects.

And that should be your next goal — applying these concepts to projects that aid you and the people around you. There is no limit to what you can create. You can start by watching a couple of videos to see the type of projects people build, then you can come up with yours.

This handbook covered the necessary parts of Arduino (both hardware and software) that you'll need as a beginner to kickstart your journey.

The best way to improve and retain what you've learned is by practicing and building.

Happy tinkering!

Starting with an embedded "Hello World" equivalent, and advancing to a text-to-morse-code translator, this article will walk you through the process.

• How to Set Up the Pi
  • Format the SD Card
  • Flash the Distribution
    • Configure Wifi and SSH
  • Complete the Circuit
• How to Set Up Cross Compilation
  • Install the Target
  • Specify the Linker
• How to Program an Embedded "Hello World"
  • Successfully Exit the Program
• How to Cross Compile the Program
• How to Transfer the Binary to the Pi
• How to SSH into the Pi
  • Run the Program
• How to Code a Text-to-Morse-Code Translator
• Appendix
  • Targets

How to Set Up the Pi

Format the SD Card

Use the Raspberry Pi Imager which can be downloaded from the Raspberry Pi Software Webpage.

Flash the Distribution

A distribution I'd suggest is Raspberry Pi OS Lite. This is a headless distribution, which means it does not come with a GUI.

Configure Wifi and SSH

Once that is done, you can insert the SD card into the Raspberry Pi, and power it up.

Complete the Circuit

Circuit Diagram

Pi Pinout

Connect negative to ground, and positive to BCM pin 17 as shown below:

The pinout can be seen here: https://pinout.xyz/

How to Set Up Cross Compilation

Install the Target

Use rustup to install the necessary target for your Raspberry Pi:

my-pc$ rustup add target arm-unknown-linux-gnueabihf

Appendix for more information about targets in Rust.

Specify the Linker

Download the raspberrypi/tools repository into a directory named rpi_tools:

my-pc:~ $ git clone https://github.com/raspberrypi/tools $HOME/rpi_tools

Edit the ~/.cargo/config file using your favourite text editor:

my_pc:~ $ sudo nano ~/.cargo/config

Tell Cargo to use a specific linker version for your target:

[target.arm-unknown-linux-gnueabihf]
linker = "/rpi_tools/arm-bcm2708/arm-rpi-4.9.3-linux-gnueabihf/bin/arm-linux-gnueabihf-gcc"

How to Program an Embedded "Hello World"

Start by creating a new Rust project, and opening the main.rs file in your favourite text editor:

my-pc:~ $ cargo new blink
my-pc:~ $ cd blink
my-pc:~/blink $ nano src/main.rs

Import the rust_gpiozero crate, and program an LED to alternate between on and off every second:

use rust_gpiozero::*;

fn main() {
    // Create a new LED attached to Pin 17
    let mut led = LED::new(17);

    led.blink(1.0, 1.0);

    led.wait();
}

Be sure to add the dependency to the Cargo.toml file:

[dependencies]
rust-gpiozero = "0.2.1"

Successfully Exit the Program

Since rustc 1.61.0 ^[1], you can use the std::process::ExitCode struct to specify the status code returned to the process' parent:

use std::process::ExitCode;
fn main() -> ExitCode {
    // ...
    if error {
      return ExitCode::from(1);
    }
    ExitCode::SUCCESS
}

Otherwise, you can simply return a Result:

fn main() -> Result<(), std::io::Error> {
  // ...
  Ok(())
}

How to Cross Compile the Program

Build a release of your program, targeting the required architecture:

my-pc:~/blink $ cargo build --release --target=arm-unknown-linux-gnueabihf

How to Transfer the Binary to the Pi

Use scp to transfer the compiled binary from your host computer to the Raspberry Pi over SSH:

my-pc:~/blink $ scp target/arm-unknown-linux-gnueabihf/release/blink pi@192.168.1.199:~/blink

Note: The local IP of your Pi will likely be different.

How to SSH into the Pi

SSH and log in to the Raspberry Pi via its local IP address:

my-pc:~ $ ssh pi@192.168.1.199

Run the Program

From the Raspberry Pi, run the compiled binary:

pi:~ $ ./blink

How to Code a Text-to-Morse-Code Translator

Here is an example of an application that reads the stdin line by line, translates the input into Morse Code, and toggles the LED on and off based on the Morse Code for the characters.

use rust_gpiozero::*;
use std::io::{BufRead, self};
use std::collections::HashMap;
use std::thread::sleep;
use std::time::Duration;

fn main() -> Result<(), std::io::Error> {
    println!("Starting...\n- Type in text to turn into Morse Code\n- Type `quit()` to quit\n");
    // Create a new LED attached to Pin 17
    let led = LED::new(17);

    /// Length of a dot in milliseconds
    const DOT_DELAY: u64 = 80;
    /// Length of a dash in milliseconds
    const DASH_DELAY: u64 = DOT_DELAY * 3;
    /// Delay between inputs in milliseconds
    const PUSH_DELAY: u64 = DOT_DELAY;
    /// Delay between letters in milliseconds
    const LETTER_DELAY: u64 = DOT_DELAY * 3;
    /// Delay between words in milliseconds
    const WORD_DELAY: u64 = DOT_DELAY * 7;

    let morse_code_alphabet: HashMap<char, &'static str> =
    [
        ('a', ".-"),
        ('b', "-..."),
        ('c', "-.-."),
        ('d', "-.."),
        ('e', "."),
        ('f', "..-."),
        ('g', "--."),
        ('h', "...."),
        ('i', ".."),
        ('j', ".---"),
        ('k', "-.-"),
        ('l', ".-.."),
        ('m', "--"),
        ('n', "-."),
        ('o', "---"),
        ('p', ".--."),
        ('q', "--.-"),
        ('r', ".-."),
        ('s', "..."),
        ('t', "-"),
        ('u', "..-"),
        ('v', "...-"),
        ('w', ".--"),
        ('x', "-..-"),
        ('y', "-.--"),
        ('z', "--.."),
        ('1', ".----"),
        ('2', "..---"),
        ('3', "...--"),
        ('4', "....-"),
        ('5', "....."),
        ('6', "-...."),
        ('7', "--..."),
        ('8', "---.."),
        ('9', "----."),
        ('0', "-----"),
        ('.', ".-.-.-"),
        (',', "--..--"),
        ('?', "..--.."),
        ('\'', ".----."),
        ('!', "-.-.--"),
        ('/', "-..-."),
        ('(', "-.--."),
        (')', "-.--.-"),
        ('&', ".-..."),
        (':', "---..."),
        (';', "-.-.-."),
        ('=', "-...-"),
        ('+', ".-.-."),
        ('-', "-....-"),
        ('_', "..--.-"),
        ('"', ".-..-."),
        ('$', "...-..-"),
        ('@', ".--.-."),
        (' ', " "),
    ].iter().cloned().collect();

    // Read standard input per line
    for line_res in io::stdin().lock().lines() {
        let line = line_res?;
        if line == "quit()" {
            break;
        }
        // Turn line into morse code
        let mut morse = String::new();
        for c in line.chars() {
            if let Some(morse_code_char) = morse_code_alphabet.get(&c) {
                morse.push_str(morse_code_char);
                // Separate characters with a comma
                morse.push_str(",");
            }
        }
        // Blink LED based on characters
        for c in morse.chars() {
            match c {
                '.' => {
                    led.on();
                    sleep(Duration::from_millis(DOT_DELAY));
                    led.off();
                    sleep(Duration::from_millis(PUSH_DELAY));
                },
                '-' => {
                    led.on();
                    sleep(Duration::from_millis(DASH_DELAY));
                    led.off();
                    sleep(Duration::from_millis(PUSH_DELAY));
                },
                ',' => {
                    sleep(Duration::from_millis(LETTER_DELAY));
                },
                ' ' => {
                    sleep(Duration::from_millis(WORD_DELAY));
                },
                _ => {
                    println!("Unknown character: {}", c);
                    break;
                }
            }
        }
        sleep(Duration::from_millis(WORD_DELAY));
    }

    // Free the variable and associated resources
    led.close();

    Ok(())
}

Appendix

Targets

In Rust, the target is the platform (architecture) the program is compiled for. Cargo automatically detects the target, based on the file system layout ^[2].

You can see the list of built-in targets, by running:

rustc --print target-list
# OR
rustup target list

From here you can add a new target to your project, by running:

rustup target add <target>

The given target is often in the form of a triple ^[3]:

• The architecture
• The vendor
• The operating system type
• The environment type

This is refered to as a 'target triple', because the fourth part is optional.

One thing that’s always tripped me up as an IoT developer is figuring out the best way to transmit data. There are many different kinds of radios and mediums. On top of that, there are different protocols to boot.

As of this writing, there is one protocol that has reigned supreme in the IoT world:

MQTT.

Unlike an HTTP server, a device can connect, publish, and subscribe to topics. These topics then get sent to a broker and distributed to other subscribed devices. It also happens that MQTT on Nordic’s nRF9160 is well supported.

In this post, I’ll show you how to connect the nRF9160 Feather to a self-hosted Mosquitto instance. You’ll learn how to generate your own certificates, and get a hang of how to test your connections.

Ready to play? Let’s get to it.

Where to host?

If you want to host Mosquitto, you’ll need a server. Since Mosquitto is written in C it’s lightweight and can go almost anywhere. Plus it sips resources so you can install it on a budget VPS without much worry. That’s where a VPS provider like Digital Ocean or Vultr comes in.

To set up a new server here are some steps:

• Login to Digital Ocean. If you don’t have Digital Ocean and would like to support click here to create an account.
• Create a new Droplet

• Choose the FreeBSD 12.1 with UFS.

• Choose the $5 instance. That’s usually more than enough.

• Make sure you import your public key. Otherwise, you won’t be able to immediately use password-less login.

• Hit that green Create Droplet button, and let’s get this show on the road.

Important Extra step

For the certs to work with Mosquitto, you’ll have to set a domain to point to your VPS IP address. A CNAME or A record works. If you’re unsure how to do that, here’s a good guide. Note which (sub)domain you’ve used. We’ll need it in a bit…

Install Mosquitto

I run my servers inside of a FreeBSD jail using Bastille. In this tutorial, we’ll be skipping the jail part and focusing on getting the nRF9160 Feather working.

• You should be set with a Digital Ocean instance (or similar) using FreeBSD. If you haven't done that yet, head back up to the Where to host? section.
• Next, to install mosquitto on your droplet, run pkg install mosquitto. If you’re running something other than FreeBSD, this command may differ. apt-get install mosquitto works on Debian-based systems. If you want the most up to date repositories, make sure you run sudo apt-add-repository ppa:mosquitto-dev/mosquitto-ppa beforehand. Here’s what the full output on FreeBSD looks like:

$ pkg install mosquitto
The package management tool is not yet installed on your system.
Do you want to fetch and install it now? [y/N]: y
Bootstrapping pkg from pkg+http://pkg.FreeBSD.org/FreeBSD:12:amd64/quarterly, please wait...
Verifying signature with trusted certificate pkg.freebsd.org.2013102301... done
[mosquitto] Installing pkg-1.14.6...
[mosquitto] Extracting pkg-1.14.6: 100%
Updating FreeBSD repository catalogue...
[mosquitto] Fetching meta.conf: 100%    163 B   0.2kB/s    00:01
[mosquitto] Fetching packagesite.txz: 100%    6 MiB   6.6MB/s    00:01
Processing entries: 100%
FreeBSD repository update completed. 31943 packages processed.
All repositories are up to date.
Updating database digests format: 100%
The following 4 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
        c-ares: 1.16.1
        ca_root_nss: 3.55
        e2fsprogs-libuuid: 1.45.6
        mosquitto: 1.6.7

Number of packages to be installed: 4

The process will require 2 MiB more space.
682 KiB to be downloaded.

Proceed with this action? [y/N]: y
[mosquitto] [1/4] Fetching mosquitto-1.6.7.txz: 100%  226 KiB 231.1kB/s    00:01
[mosquitto] [2/4] Fetching ca_root_nss-3.55.txz: 100%  285 KiB 291.5kB/s    00:01
[mosquitto] [3/4] Fetching e2fsprogs-libuuid-1.45.6.txz: 100%   34 KiB  34.7kB/s    00:01
[mosquitto] [4/4] Fetching c-ares-1.16.1.txz: 100%  138 KiB 140.9kB/s    00:01
Checking integrity... done (0 conflicting)
[mosquitto] [1/4] Installing ca_root_nss-3.55...
[mosquitto] [1/4] Extracting ca_root_nss-3.55: 100%
[mosquitto] [2/4] Installing e2fsprogs-libuuid-1.45.6...
[mosquitto] [2/4] Extracting e2fsprogs-libuuid-1.45.6: 100%
[mosquitto] [3/4] Installing c-ares-1.16.1...
[mosquitto] [3/4] Extracting c-ares-1.16.1: 100%
[mosquitto] [4/4] Installing mosquitto-1.6.7...
===> Creating users
Using existing user 'nobody'.
[mosquitto] [4/4] Extracting mosquitto-1.6.7: 100%
=====
Message from ca_root_nss-3.55:

--
FreeBSD does not, and can not warrant that the certification authorities
whose certificates are included in this package have in any way been
audited for trustworthiness or RFC 3647 compliance.

Assessment and verification of trust is the complete responsibility of the
system administrator.

This package installs symlinks to support root certificates discovery by
default for software that uses OpenSSL.

This enables SSL Certificate Verification by client software without manual
intervention.

If you prefer to do this manually, replace the following symlinks with
either an empty file or your site-local certificate bundle.

  * /etc/ssl/cert.pem
  * /usr/local/etc/ssl/cert.pem
  * /usr/local/openssl/cert.pem
=====
Message from mosquitto-1.6.7:

--
The mosquitto MQTT Python driver is now provided by net/py-paho-mqtt

All installed package configuration lives at /usr/local/etc/mosquitto/. We’ll need to edit mosquitto.conf in that folder to use certificates. Here’s what it looks like:

# Daemon configuration
pid_file /var/run/mosquitto.pid
user nobody

# Port to use for the default listener.
port 8885

# At least one of cafile or capath must be defined.
cafile /root/pki/ca.crt

# Path to the PEM encoded server certificate.
certfile /root/pki/issued/mosquitto.crt

# Path to the PEM encoded keyfile.
keyfile /root/pki/private/mosquitto.key

# Path to CRL file
#crlfile /root/pki/crl.pem

# Each client has their own cert
require_certificate true
use_identity_as_username true

# listener port-number [ip address/host name]
listener 1883
protocol mqtt

# listener port-number [ip address/host name]
# listener 8080
# protocol websockets

# =================================================================
# Logging
# =================================================================
log_dest syslog

# Types of messages to log.
log_type all
#log_type warning
# websockets_log_level 127

# -----------------------------------------------------------------
# Default authentication and topic access control
# -----------------------------------------------------------------
# password_file /usr/local/etc/mosquitto/pwfile

Before we can start the server we’ll need to provision some RSA certificates. We’ll get to that in the next step.

Provision Certs

You can use easy-rsa to generate a CA server and client certs. (These instructions come from this guide.) For production, you should generate your keys and certs on an offline machine. That way your private keys are safe if your server becomes a target.

First, install easy-rsa:

$ pkg install easy-rsa
Updating FreeBSD repository catalogue...
FreeBSD repository is up to date.
All repositories are up to date.
The following 1 package(s) will be affected (of 0 checked):

New packages to be INSTALLED:
        easy-rsa: 3.0.7

Number of packages to be installed: 1

44 KiB to be downloaded.

Proceed with this action? [y/N]: y
[mosquitto] [1/1] Fetching easy-rsa-3.0.7.txz: 100%   44 KiB  44.8kB/s    00:01
Checking integrity... done (0 conflicting)
[mosquitto] [1/1] Installing easy-rsa-3.0.7...
[mosquitto] [1/1] Extracting easy-rsa-3.0.7: 100%

Then let's begin the cert creation process:

$ easyrsa init-pki

Note: using Easy-RSA configuration from: /usr/local/share/easy-rsa/vars

init-pki complete; you may now create a CA or requests.
Your newly created PKI dir is: /root/pki
$
$ easyrsa build-ca

Note: using Easy-RSA configuration from: /usr/local/share/easy-rsa/vars
Using SSL: openssl OpenSSL 1.1.1d-freebsd  10 Sep 2019

Enter New CA Key Passphrase:
Re-Enter New CA Key Passphrase:
Generating RSA private key, 2048 bit long modulus (2 primes)
......................+++++
..................................................................................+++++
e is 65537 (0x010001)
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Common Name (eg: your user, host, or server name) [Easy-RSA CA]:testserver.jaredwolff.com

CA creation complete and you may now import and sign cert requests.
Your new CA certificate file for publishing is at:
/root/pki/ca.crt

Note: You will be prompted for a password at the build-ca step. Make sure you keep this password handy.

Then to generate a server cert use:

$ easyrsa gen-req mosquitto nopass

Note: using Easy-RSA configuration from: /usr/local/share/easy-rsa/vars
Using SSL: openssl OpenSSL 1.1.1d-freebsd  10 Sep 2019
Generating a RSA private key
...............+++++
........................................+++++
writing new private key to '/root/pki/easy-rsa-82720.X2NVQ0/tmp.akOxhO'
-----
You are about to be asked to enter information that will be incorporated
into your certificate request.
What you are about to enter is what is called a Distinguished Name or a DN.
There are quite a few fields but you can leave some blank
For some fields there will be a default value,
If you enter '.', the field will be left blank.
-----
Common Name (eg: your user, host, or server name) [mosquitto]:testserver.jaredwolff.com

Keypair and certificate request completed. Your files are:
req: /root/pki/reqs/mosquitto.req
key: /root/pki/private/mosquitto.key
$
$ easyrsa sign-req server mosquitto

Note: using Easy-RSA configuration from: /usr/local/share/easy-rsa/vars
Using SSL: openssl OpenSSL 1.1.1d-freebsd  10 Sep 2019

You are about to sign the following certificate.
Please check over the details shown below for accuracy. Note that this request
has not been cryptographically verified. Please be sure it came from a trusted
source or that you have verified the request checksum with the sender.

Request subject, to be signed as a server certificate for 825 days:

subject=
    commonName                = testserver.jaredwolff.com

Type the word 'yes' to continue, or any other input to abort.
  Confirm request details: yes
Using configuration from /root/pki/easy-rsa-82744.hyuGzt/tmp.lZHLEH
Enter pass phrase for /root/pki/private/ca.key:
Check that the request matches the signature
Signature ok
The Subject's Distinguished Name is as follows
commonName            :ASN.1 12:'testserver.jaredwolff.com'
Certificate is to be certified until Nov  3 01:12:53 2022 GMT (825 days)

Write out database with 1 new entries
Data Base Updated

Certificate created at: /root/pki/issued/mosquitto.crt

You’ll be prompted for both the Common Name (i.e. your server name) and the CA cert password in the above step. Important: the Common Name needs to match the domain name of your server! (Remember, we wrote that down earlier?)

To generate the nRF9160 cert, use:

$ easyrsa gen-req nrf9160 nopass batch
$ easyrsa sign-req client nrf9160 batch

Follow the same procedure as earlier. The only difference is that we’re generating a client cert instead of a server cert.

Once complete, we’ll need some files. Here’s a full list:

For your Mosquitto Server

• /root/pki/ca.crt
• /root/pki/private/mosquitto.key
• /root/pki/issued/mosquitto.crt

For your nRF9160 Feather

• /root/pki/ca.crt
• /root/pki/private/nrf9160.key
• /root/pki/issued/nrf9160.crt

If you’re using the configuration from above, it’s already pointing to your server certificates. All we have to do now is start mosquitto!

$ service mosquitto start
Cannot 'start' mosquitto. Set mosquitto_enable to YES in /etc/rc.conf or use 'onestart' instead of 'start'.

If you get an error about mosquitto_enable simply run:

$ sysrc mosquitto_enable=YES
$ service mosquitto start
Starting mosquitto.

This enables mosquitto to start when your system starts.

Now, check if mosquitto is running by using ps aux:

$ ps aux
USER     PID %CPU %MEM   VSZ  RSS TT  STAT STARTED    TIME COMMAND
root   82401  0.0  0.2 11472 2424  -  SsJ  01:02   0:00.00 /usr/sbin/syslogd -ss
root   82457  0.0  0.2 11408 2284  -  IsJ  01:02   0:00.00 /usr/sbin/cron -J 60 -s
nobody 82900  0.0  0.6 16352 6212  -  SsJ  01:17   0:00.02 /usr/local/sbin/mosquitto -c /usr/local/etc/mosquitto/mosquitto.conf -d
root   82488  0.0  0.3 12096 2848  0  IJ   01:02   0:00.01 login [pam] (login)
root   82489  0.0  0.3 13092 3504  0  SJ   01:02   0:00.03 -csh (csh)
root   82902  0.0  0.3 11704 2540  0  R+J  01:17   0:00.00 ps aux

Now that we have a server loaded and running, let’s get the firmware working.

Firmware bits

Dealing with certificates on the nRF9160 Feather is a two-step process. The first step is to load the certificates using the at_client firmware. The second is to load the mqtt_simple library with added TLS support. Let’s tackle the certs first.

Program at_client first

Change directories to ncs/nrf/samples/nrf9160/at_client/ and start a fresh build:

$ west build -b circuitdojo_feather_nrf9160ns -p

Then flash to your board using:

$ west flash --erase
$ nrfjprog -r

We’ll need this sample on your board for the next step.

Add certs to device

To install our new certs, we’ll need nRF Connect Desktop installed. You can download it by going here.

You’ll also need a custom version of LTE Link Monitor. You can get the modified version on docs.jaredwolff.com.

First, Install nRF Connect Desktop app. Then, copy the LTE Link Monitor .tgz file to %USERPROFILE%\.nrfconnect-apps\local (on Windows) or $HOME/.nrfconnect-apps/local (on Linux/macOS). Here’s an example of where it is on Windows:

Close and re-open nRF Connect Desktop (if it’s open).

Then, click Open next to the v1.1.1 version of LTE Link Monitor. It will also have local written underneath it.

Next, let’s launch it!

Once you’ve opened the port, hit the reset button. Make sure you turn off Automatic requests.

Then in the command box send AT+CFUN=4. This will shut down your modem so it’s ready to upload certs. You can run AT+CFUN? to confirm your modem in that mode.

Open up the Certificate manager.

Make sure you set the security tag. In this case, I’m using 1234. This is an important identifier that you’ll need later. Make it whatever you want but I would avoid using 16842753. This is the default tag for NRF Cloud. You don’t want to blast away your nRF Cloud certs!

Copy and paste the contents of your ca.crt, nrf9160.crt and nrf9160.key into the boxes (in that order). You can easily get the certs by using cat on Unix/Linux:

$ cat cat.crt
-----BEGIN CERTIFICATE-----
MIIDdTCCAl2gAwIBAgIUDLkBxLLQO9wosNDtA7E9qvqHOxMwDQYJKoZIhvcNAQEL
BQAwJDEiMCAGA1UEAwwZdGVzdHNlcnZlci5qYXJlZHdvbGZmLmNvbTAeFw0yMDA3
MzEwMTExNDJaFw0zMDA3MjkwMTExNDJaMCQxIjAgBgNVBAMMGXRlc3RzZXJ2ZXIu
amFyZWR3b2xmZi5jb20wggEiMA0GCSqGSIb3DQEBAQUAA4IBDwAwggEKAoIBAQC3
de1v8k+FXzY/Im7Z2YKS7wwbBRft5CUxqP1sdYJgMvheS9LhFufk81URZ0lHD9pK
aNPxU1UEmnLvVDTGLJ+YAmMH08xn17FS1R1UVPYzi2ouwqRM2pR9EStsSlP9Zj44
1MsdizABnnlkZndUVLL/gjc4cNsNncMLBSEbsz6b5WzhtAGg3rOpdAxSSblZVSFw
bquCgg5hb2NUzy+JxGtUIsE5d6CxTDdSs4Z3FK/RRYjmCG6qsaya4N5W35yf8h5O
StfKRecl3kq2kCnWa6P+lErG4wuxIBtMkgz2zV+zd1tz4aHXxSdoZTqLz7dTVbFA
zEVnKD+ZReBG+4fwUL6rAgMBAAGjgZ4wgZswHQYDVR0OBBYEFIvdGnjrxRPzvXQi
7XJ70LzpZSOjMF8GA1UdIwRYMFaAFIvdGnjrxRPzvXQi7XJ70LzpZSOjoSikJjAk
MSIwIAYDVQQDDBl0ZXN0c2VydmVyLmphcmVkd29sZmYuY29tghQMuQHEstA73Ciw
0O0DsT2q+oc7EzAMBgNVHRMEBTADAQH/MAsGA1UdDwQEAwIBBjANBgkqhkiG9w0B
AQsFAAOCAQEAIzz1nSSDkPueNPlADRYMDOMFNkxoKA+gRXwDVa7y39As7IZp7Fqr
KAH79U1XtGyDlt6FPKTvDJ7jtd4y8auIGVQO7z3AG9pVU1imIWZHoIqgBUCEhsjp
uMxD23kRCX5kd9dsmF9WOGGxb4kkMv83Rh2rCONQmvnozuI3fJv2ZFX/ORoADGLP
OPSJPl11x+2rxPxiLi+T8RyzDh3DwqnPVsSnbRWV7hosaN0ip/cbnSTaIul9mbCY
ID6qm9leqlY/gha9aZfg+tv1Lm6PT6o8Pzek2VeDoIS5YERBMOwV84hQrZjV3vIE
jT6y663HGsl7KvqVaWdV3fM6Cr7f0QdR5A==
-----END CERTIFICATE-----

You’ll need everything from -----BEGIN CERTIFICATE----- to -----END CERTIFICATE-----

Check the Log area for more details. If all went well, it should say that your certificates updated.

Using the mqtt_simple example

We’ll be using the mqtt_simple sample within the nRF Connect SDK repository. The full path is: ncs/nrf/samples/nrf9160/mqtt_simple. We’ll need to make a few edits to add full TLS compatibility. All the files are within the mqtt_simple directory.

First, we’ll have to update the proj.conf file. See the highlighted differences:

The # Set the PDP Context section is especially important if you’re using a Hologram SIM card (included with the nRF9160 Feather). If you are using a SIM that doesn’t need it, you do not need this section.

Adopt your CONFIG_MQTT_BROKER_HOSTNAME to your hostname (configured at the beginning of this guide).

You’ll also have to add these lines in KConfig:

config SEC_TAG
    int "Security tag to use for the connection"
    default 1234

config PEER_VERIFY
    int "Peer verify parameter for mqtt_client"
    default 1
    help
            Set to 0 for VERIFY_NONE, 1 for VERIFY_OPTIONAL, and 2 for VERIFY_REQUIRED.

Finally in main add this block to the top of your file:

#if defined(CONFIG_MQTT_LIB_TLS)
static sec_tag_t sec_tag_list[] = { CONFIG_SEC_TAG };
#endif /* defined(CONFIG_MQTT_LIB_TLS) */

Then add this block to client_init under #if defined(CONFIG_MQTT_LIB_TLS)

  struct mqtt_sec_config *tls_config = &client->transport.tls.config;

    client->transport.type = MQTT_TRANSPORT_SECURE;

    tls_config->peer_verify = CONFIG_PEER_VERIFY;
    tls_config->cipher_count = 0;
    tls_config->cipher_list = NULL;
    tls_config->sec_tag_count = ARRAY_SIZE(sec_tag_list);
    tls_config->sec_tag_list = sec_tag_list;
    tls_config->hostname = CONFIG_MQTT_BROKER_HOSTNAME;

The changes should look like this:

Then build with:

$ west build -b circuitdojo_feather_nrf9160ns -p

Finally, flash it using west flash:

$ west flash --erase
$ nrfjprog -r

Open your serial terminal and double check that your nRF9160 Feather is connecting. You can also use LTE Link Monitor to view your progress as well (Example below).

Lots of the information above came from Nordic's post on the subject.

Sending a message

We’re almost there! You’ve configured your nRF9160 Feather to connect to Mosquitto using self-generated certificates. The last part is connecting another device to see if the nRF9160 Feather replies to a message.

I’ve created a new set of certs for this purpose. I called them test.

$ easyrsa gen-req test nopass batch
$ easyrsa sign-req client test batch

I copied them to my desktop using CyberDuck (a great little visual SFTP client):

You can also use something like scp if you're confident in your command line file transfer abilities. Then, open a terminal and run:

mosquitto_sub --cafile ca.crt --cert test.crt --key test.key -q 1 -d -h testserver.jaredwolff.com -p 8885 -t "/my/publish/topic" &
mosquitto_pub --cafile ca.crt --cert test.crt --key test.key -q 1 -d -h testserver.jaredwolff.com -p 8885 -t "/my/subscribe/topic" -m "hello there"

You should see an output like this:

$ mosquitto_sub --cafile ca.crt --cert test.crt --key test.key -q 1 -d -h testserver.jaredwolff.com -p 8885 -t "/my/publish/topic" &
$ mosquitto_pub --cafile ca.crt --cert test.crt --key test.key -q 1 -d -h testserver.jaredwolff.com -p 8885 -t "/my/subscribe/topic" -m "hello there"
Client mosq-CczskQKzMKdtTo4O4s sending CONNECT
Client mosq-CczskQKzMKdtTo4O4s received CONNACK (0)
Client mosq-CczskQKzMKdtTo4O4s sending PUBLISH (d0, q1, r0, m1, '/my/subscribe/topic', ... (11 bytes))
Client mosq-CczskQKzMKdtTo4O4s received PUBACK (Mid: 1, RC:0)
Client mosq-CczskQKzMKdtTo4O4s sending DISCONNECT
MacBook-Pro:Downloads jaredwolff$ Client mosq-qK8tMlJk0Qri4Z7jUo sending PINGREQ
Client mosq-qK8tMlJk0Qri4Z7jUo received PINGRESP
MacBook-Pro:Downloads jaredwolff$ Client mosq-qK8tMlJk0Qri4Z7jUo received PUBLISH (d0, q0, r0, m0, '/my/publish/topic', ... (11 bytes))
hello there

Booyah! You have an active working connection to your very own Mosquitto server.

Conclusion

We’ve made it to the end! By this point in the post, you should have a Mosquitto server running and an nRF9160 connected. Now you can use your new-found skills to add more devices to your deployments and more.

If you haven’t had a chance to play with the nRF9160 you should check out the nRF9160 Feather. It has Nordic Semiconductor’s nRF9160 LTE-M, NB IoT + GPS Combo, plus flexible power supply, external flash, and low power shutdown.

Oh, and did I mention it’s 100% open source? Learn more by checking out the campaign on GroupGets and Hackster Launch. ?

Photo credit to the awesome folks at GroupGets!

You can read this article and lots of other good stuff at jaredwolff.com.

This post is a bit lengthy. If you prefer, signup to get the full PDF here. ?

After Particle's Mesh deprecation announcement, many have been left to figure out how to deploy their low power sensor networks. There was always the option of using Particle's Built in Bluetooth stack but as it stands today it's not secure.

Previously I had helped form a very simple nRF SDK-based hub and spoke IoT deployment. Unfortunately, it was closed source and the company is no longer around.

So what's a guy to do?

Build another one and make it open. (BSD licensed to be exact!) Open and free for anyone to use adopt and improve upon. Plus if you're building a product that's using the code, you don't have to share your improvements or proprietary code with anyone.

In this post I'll be talking about how to get started with Pyrinas. It uses Nordic's time tested SDK as a basis for the kernel of the system. The main concept of Pyrinas is to abstract as much IoT cruft away so you can focus on your application.

So without further ado, let's chat about what Pyrinas is and what it isn't.

What Pyrinas is

• Is an embedded "kernel", written in C. It's open and permissive IoT development environment you can use for anything you want. Seriously. It's BSD licensed and can be used in closed source applications.
• Using the power of Bluetooth 5.0 Long Range, Pyrinas allows you to communicate with many peripheral devices at one time. Currently Pyrinas has been tested with 3 simultaneous peripheral connections. Theoretically, it can support up to 20 simultaneous connections. (Thanks to Nordic's S140 Softdevice)

• Pyrinas transports its data in two ways

  • In a familiar string format used by Particle
  • A custom Protocol Buffer for raw data transmission.

  That way you have a choice of how you want to process and publish your data!

What Pyrinas isn't

• Pyrinas is not a RTOS (real time operating system). If you have a need to run multiple threaded applications on embedded, Pyrinas is not for you.
• Pyrinas, at this time, does not support Mesh.
• An OS for every single kind of Bluetooth SoC on the market. Due to the tight coupling with Nordic's nRF SDK, Pyrinas only works with Nordic's SoCs.
• A turnkey solution for IoT. Pyrinas is early on it it's development process. The aim is for it to become a viable option for anyone to develop and publish IoT applications the way they want to. There's no vendor lock in. There's no surprises.

Things you'll need

There are a few things you'll need in order to get going with Pyrinas.

• At least 2 Particle Xenons
• At least 1 nRF Development board or J-link programmer
• Associated USB cables

Getting started with an example

Getting started with Pyrinas involves two repositories.

• The OS respository
• The template

The OS directory has all the source, SDK dependencies and toolchain you need to use Pyrinas.

The template is where you add all your application code. The template provides a starting point for you and your project.

Here's how everything goes together:

Clone the OS directory to some place on your machine:

git clone https://github.com/pyrinas-iot/pyrinas-os.git --recurse-submodules

Once complete, change directories to pyrinas-os and run make setup

cd pyrinas-os
make setup

This will download your toolchain and SDK dependencies.

In order to use OTA DFU, you will need to also generate the DFU key for the process:

make gen_key

The files generated by this process will be used later.

Next, we'll want to use the template to make two new projects. In this example we'll have one "hub" and one "sensor." Simply navigate to the template repository and click the Use this template button.

Then name your new repository. Click the Create repository from template button when you're happy with everything.

Then clone your repository to the same directory as pyrinas-os. Make sure you replace <your username> and <repo name> with your own.

cd ..
git clone https://github.com/<your username>/<repo name>.git hub

After this is done, go back and create a new repository from the template. We'll be using this one for the sensor node.

Clone this repository once you're done setting it up in the same place as your hub and pyrinas-os repositories.

Now that we have all our repositories, let's start with our sensor node.

Setting up the sensor node repository

Open up the sensor repository using a program like Microsoft's VS Code. If you have the command line shortcuts you can use code to open it from the terminal:

code sensor

Before we do anything we'll need to set up the symbolic link to pyrinas-os. Make sure you're in the sensor directory and then run ln -s ../pyrinas-os/ using the terminal.

cd sensor
ln -s ../pyrinas-os/ .

This allows your project to use all the code, SDK and toolchains within the pyrinas-os repository! As an added bonus you can do this as many times as you want. Have multiple Pyrinas projects? No problem.

Alright! Now, let's check out the Makefile. You'll want to customize some of the definitions within the file:

# Start: Your configuration!

# Set this to the directory of pyrinas-os
# If you used a symbolic link this points to
# the `pyrinas-os` folder in this repository
OS_DIR := pyrinas-os

# This should be the serial number of your Jlink programmer.
# To find simply run `jlinkexe`
PROG_SERIAL=1234678

# This is your debugger port for Jlink's RTT. If you
# have mulitple, you will have to change this on each app
# your're using
PROG_PORT=19021

# This is where you set your board type. Here are the supported boards:
# xenon - Particle Xenon
BOARD=xenon

# This is where you can name your app something. Make it specific
APP_FILENAME=pyrinas-template

# This determines whether or not you're using debug mode
# Comment this out or change to false
DEBUG=true

# End: Your Configuration