Most people think Jupyter is just for Python and data science stuff, but apparently you can run Rust in one, too.

In this tutorial, we’ll be taking a look at:

1. What is EvCxR?

2. How to Install the Rust Jupyter kernel

3. How to run your first Rust code in a notebook

4. Handy Tips and Tricks

5. Common Issues and Solutions

6. When NOT to Use Jupyter for Rust

Friendly Disclaimer: This tutorial assumes you know the basics of both Rust and Jupyter. If you break something, that's on you, mate 🙂.

So without further ado, let's jump in.

What is EvCxR?

EvCxR (pronounced "Evaluator" to my fellow linguists’ horror) is a Rust REPL and Jupyter kernel. It's basically the magic that lets you run Rust code interactively in Jupyter notebooks instead of the traditional compile-run-debug cycle.

The name stands for "Evaluation Context for Rust", and it’s an open source project actively maintained on GitHub. Here are a few things that make this terribly named tool absolutely brilliant:

1. Interactive development: It lets you test Rust snippets without creating a whole project 🧪

2. Prototyping: You can quickly try out ideas before committing to a full implementation 💡

3. Data visualisation: And yes, you can even plot charts with Rust (more on that later) 📊

How to Install the Rust Jupyter kernel

Prerequisites

Before we dive into the installation, make sure you have these sorted:

1. A Linux System: Or at least, Windows Subsystem for Linux (There’s a little note below for Windows users.)

2. The Rust toolchain: You can get it from rustup.rs if you haven't already

3. Jupyter: Install via pip – pip install jupyter

4. Patience: This might take a minute or two ⏱️

Once you’ve got all that, we can get rusty (pun intended).

Note: If you’re using Windows, you’ll need to do a little extra to get started. Here’s the quick rundown:

1. Go to https://visualstudio.microsoft.com/visual-cpp-build-tools/

2. Download and run the installer

3. Select "Desktop development with C++"

4. Install it (it's large, ~5GB)

Step 1: Install EvCxR

Open your terminal and run this command:

cargo install evcxr_jupyter

Now go grab a cup of joe ☕. This will take a few minutes as Cargo downloads and compiles everything. And don't panic if it seems stuck. Rust compilation is thorough but not particularly fast.

If you get any errors about missing system libraries, you might need to install some dependencies. On Ubuntu/Debian, try:

sudo apt install jupyter-notebook jupyter-core python-ipykernel
sudo apt install cmake

On macOS with Homebrew:

brew install cmake jupyter

Step 2: Install the Jupyter Kernel

Once the installation finishes, you’ll need to register the EvCxR kernel with Jupyter:

evcxr_jupyter --install

You should see output that looks something like this at the end:

Installation complete

Step 3: Launch Jupyter and Create a Rust Notebook

Let’s test out our baby. Fire up Jupyter:

jupyter notebook

Your browser should open automatically (if it doesn't, copy the URL from the terminal).

In the Jupyter interface:

1. Click New in the top right

2. Select Rust from the dropdown (or "evcxr" depending on your version)

3. A new notebook opens

Welcome to interactive Rust! 🦀

Step 4: Write Your First Rust Code

Let's start with a classic:

println!("Hello my fellow Rustaceans! 🦀");

Hit Shift + Enter to run the cell. You should see the output appear below the cell. Simple as that.

Note that notebooks execute code at the top level, so you don’t have to wrap it around the main() function. If you still want to do that, you’re going to have to call it like this:

fn main(){
    println!("Hello my fellow Rustaceans! 🦀");
}
//Calling the function
main()

Now let's try something more interesting:

fn fibonacci(n: u32) -> u32 {
    match n {
        0 => 0,
        1 => 1,
        _ => fibonacci(n - 1) + fibonacci(n - 2)
    }
}

for i in 0..10 {
    println!("fibonacci({}) = {}", i, fibonacci(i));
}

Run it and watch the Fibonacci sequence appear.

fibonacci(0) = 0
fibonacci(1) = 1
fibonacci(2) = 1
fibonacci(3) = 2
fibonacci(4) = 3
fibonacci(5) = 5
fibonacci(6) = 8
fibonacci(7) = 13
fibonacci(8) = 21
fibonacci(9) = 34

Handy Tips and Tricks

Functions aren’t the only things that behave differently when using Rust in notebooks. Here are a few other things you might want to keep in mind:

Variables Persist Between Cells

Unlike traditional Rust compilation, variables you define in one cell stick around for the next cells:

let mut counter = 0;

Then in the next cell:

counter += 1;
println!("Counter: {}", counter);

The output would be:

Counter: 1

This is great for building up complex examples step by step.

You Can Use External Crates

Add dependencies with the :dep command in one cell:

:dep serde = { version = "1.0", features = ["derive"] }
:dep serde_json = "1.0"

Then use them normally in the next:

use serde::{Serialize, Deserialize};

#[derive(Serialize, Deserialize, Debug)]
struct Person {
    name: String,
    age: u32,
}

let person = Person {
    name: "Amina".to_string(),
    age: 24,
};

let json = serde_json::to_string(&person).unwrap();
println!("{}", json);

Output:

{"name":"Amina","age":24}

Pretty neat, huh?

Visualisation Support

You can even create graphs. To get started, install the plotters crate:

:dep plotters = { version = "0.3", default-features = false, features = ["evcxr", "all_series", "bitmap_backend", "bitmap_encoder"] }

Then create a simple sine graph:

use plotters::prelude::*;

let root = SVGBackend::new("sine_wave.svg", (640, 480)).into_drawing_area();
root.fill(&WHITE).unwrap();

let mut chart = ChartBuilder::on(&root)
    .caption("Sine Wave", ("Arial", 20))
    .margin(5)
    .x_label_area_size(30)
    .y_label_area_size(30)
    .build_cartesian_2d(-3.14..3.14, -1.2..1.2)
    .unwrap();

chart.configure_mesh().draw().unwrap();

chart.draw_series(LineSeries::new(
    (-314..314).map(|x| {
        let x = x as f64 / 100.0;
        (x, x.sin())
    }),
    &RED,
)).unwrap();

root.present().unwrap();
println!("Plot saved to sine_wave.svg");

Output:

A word on plotting: You can actually display plots directly inline in your notebook. But if you're using WSL with VSCode (like I do), inline plotting may not work properly due to rendering issues on the notebook interface. That’s why I used it as an svg file that I can easily view in my text editor.

Checking Types

Not sure what type something is? Use :vars. This shows all variables and their types:

let x = vec![1, 2, 3];

:vars

Output:

Variable	    Type
       x	Vec<i32>

Common Issues and Solutions

Compilation Errors Everywhere

If you're getting weird compilation errors, remember:

• Each cell is compiled separately

• You might need to reimport things in each cell

Slow Execution

The first time you run code in a session, it's slow due to the compilation overhead. Subsequent runs are faster. If it's really slow, you might want to:

• Use release mode: :opt 2

• Reduce dependency features to only what you need

• Consider if Jupyter is the right tool for your use case

Dependencies Not Loading

If a crate won't load:

• Make sure the version exists on crates.io

• Check your internet connection (it needs to download)

• Try specifying features explicitly

• Clear the cargo cache if things get really wonky: rm -rf ~/.evcxr

When NOT to Use Jupyter for Rust

Jupyter notebooks are great for learning and experimenting, but they're not always the best choice in:

• Production code: Use proper projects with cargo

• Performance-critical code: The overhead isn't worth it

• Large applications: Notebooks get very messy, very fast

• Team collaboration: Version control with notebooks is quite the nightmare

Stick to notebooks for prototyping and quick experiments. For anything serious, fire up your favourite editor and create a proper Rust project.

Conclusion

Let's summarise what you've learned:

1. How to install the EvCxR Jupyter kernel

2. How to create and run Rust notebooks

3. How to use external crates in notebooks

4. Tips and tricks for interactive Rust development

Jupyter notebooks make Rust more accessible for learning and experimentation. Give it a go next time you want to try out a quick Rust snippet without the ceremony of creating a full project. And with that, we've come to the end of this tutorial.

Cheers.

Resources

1. EvCxR GitHub Repository

2. Rust Book

3. Jupyter Documentation

Acknowledgements

Thanks to Anuoluwapo Victor, Chinaza Nwukwa, Holumidey Mercy, Favour Ojo, Georgina Awani, and my family for the inspiration, support and knowledge used to put this post together.

And thanks to the EvCxR project maintainers for making this possible, the Rust community for being awesome, and to anyone reading this for wanting to learn. You inspire me daily.

In this guide, you’ll learn how to create a financial tracker that runs entirely in your terminal. You’ll use Rust to build a system that saves transactions to a local JSON file, ensuring that you have total ownership of your information.

Along the way, you’ll learn how to use the Rust type system to validate financial data and handle file errors gracefully. You’ll also use the Clap library to create a professional command line interface. By the time you finish, you’ll understand how to manage local state, serialize data with Serde, and structure a modular Rust application.

Table of Contents

1. Prerequisites

2. Commands You’ll Build

3. Step 1: Set Up the Project

4. Step 2: Design the Data Model

   • Add Methods to the TrackerData

5. Step 3: Handle Errors Properly

   • Map System Errors to Custom Errors

   • Prepare for Error Output

6. Step 4: Create File Operations

   • Add JSON Utility Functions

   • Register the Utilities

7. Step 5: Set Up the CLI Structure

   • The Command Architecture

   • Manage Paths with Global Context

   • Register the Command System

8. Step 6: Create Response Types

   • Define the Response Structures

   • Implement the Output Module

   • Update the Library Registration

9. Step 7: Create Argument Parsing Helpers

   • Implement Custom Data Parsers

10. Step 8: Implement the Init Command

11. Step 9: Implement the Add Command

12. Step 10: Implement the List Command

13. Step 11: Implement the Update Command

14. Step 12: Implement the Delete Command

15. Step 13: Implement Subcategory Commands

    • List Subcategories

    • Add Subcategories

    • Delete Subcategories

    • Rename Subcategories

16. Step 14: Implement the Total Command

17. Step 15: Wire Up the Main Function

18. Test Your Application

    • Install the Binary

19. What's Next and Advanced Features

    • Advanced Features to Explore

20. Conclusion

Prerequisites

To follow along with this tutorial, you should have a basic comfort level with Rust syntax. You don’t need to be an expert, but you should understand how to use variables, functions, and structs.

You’ll also need the following tools and knowledge:

• Rust installed (version 1.70 or later). If you don't have Rust installed, follow the official installation guide. You can verify your installation by running rustc --version in your terminal.

• Familiarity with command-line tools and terminal usage.

• Basic knowledge of the JSON format.

Commands You’ll Build

This tutorial will guide you on how to implement these commands step-by-step:

• init: Initializes a new tracker and creates your storage file.

• add: Saves new income or expense records to your data.

• list: Allows you to view and filter your saved transactions.

• update: Modifies existing records in your storage.

• delete: Removes specific records from your history.

• subcategory: Manages custom subcategories (list, add, delete, rename)

• total: Calculates your financial totals and net balance.

Step 1: Set Up the Project

To start, you need to create a new Rust project. Open your terminal and run these commands:

cargo new fintrack
cd fintrack

This creates a new directory called fintrack with a basic Rust project structure. cargo is Rust's package manager and build tool. It handles dependencies, compilation, and project management.

Now, open Cargo.toml in your editor. This file defines the metadata and libraries for your project. Add the following dependencies that your application will need:

[package]
name = "fintrack"
version = "1.0.0"
edition = "2021"

[dependencies]
chrono = "0.4.42"
clap = { version = "4.5.53", features = ["derive"] }
dirs = "6.0.0"
serde = { version = "1.0.228", features = ["derive"] }
serde_json = "1.0.148"
strum = { version = "0.26", features = ["derive"] }

Here’s what each dependency does in your project:

• chrono: Handles dates and times. You'll use it to parse dates from user input and format them for display.

• clap: A library for building command-line interfaces. It manages the process of parsing and validating the arguments you type into the terminal.

• dirs: Provides a cross-platform way to find the user's home directory, where you'll store the tracker data.

• serde and serde_json: serde is Rust's serialization framework. Combined with serde_json, it lets you convert Rust structs to JSON and back. This is how you'll save and load your tracker data.

• strum: Provides macros to automatically generate useful code for enums, like converting them to strings and parsing strings into enums.

The features = ["derive"] for clap and serde enables their derive macros, which will let you use attributes like #[derive(...)] to automatically generate the code needed for parsing and data conversion.

Step 2: Design the Data Model

Before writing any command logic, you’ll want to define the structure of the data your tracker will store. In Rust, you use structs to group related data much like a record in a database, and enums to represent values that can only be one of several fixed variants.

Create a new file src/models.rs and add the code to define a record:

use chrono::NaiveDate;
use serde::{Deserialize, Serialize};
use std::collections::HashMap;

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct Record {
    pub id: usize,
    pub category: usize,
    pub amount: f64,
    pub subcategory: usize,
    pub description: String,
    pub date: String,
}

This Record struct represents a single income or expense transaction. The #[derive(...)] attribute automatically implements traits that allow you to print the struct for debugging, copy it, and convert it to or from JSON. The pub keyword ensures that these fields are accessible to the other modules you will build.

Next, add the main data structure to the src/models.rs file:

#[derive(Debug, Clone, Serialize, Deserialize)]
pub struct TrackerData {
    pub version: u32,
    pub currency: String,
    pub created_at: String,
    pub last_modified: String,
    pub opening_balance: f64,
    pub categories: HashMap<String, usize>,
    pub subcategories_by_id: HashMap<usize, String>,
    pub subcategories_by_name: HashMap<String, usize>,
    pub next_subcategory_id: u32,
    pub records: Vec<Record>,
    pub next_record_id: usize,
}

This struct holds the state of the entire application. It uses a HashMap for categories and subcategories to allow for fast lookups by name or ID. All individual transactions are stored in the records vector, which can grow dynamically as you add more data.

Now, add enums to handle your fixed categories and supported currencies:

#[derive(clap::ValueEnum, Clone, Debug, strum::Display, strum::EnumString)]
#[strum(serialize_all = "lowercase", ascii_case_insensitive)]
pub enum Category {
    Income,
    Expenses,
}

#[derive(clap::ValueEnum, Clone, Debug, strum::Display, strum::EnumString)]
#[strum(serialize_all = "UPPERCASE", ascii_case_insensitive)]
pub enum Currency {
    NGN,
    USD,
    GBP,
    EUR,
    CAD,
    AUD,
    JPY,
}

These enums ensure the user can only input valid categories or currencies. The strum attributes handle the conversion between terminal input strings and your Rust code, while clap::ValueEnum allows these types to work directly with your command-line arguments.

Add Methods to the TrackerData

To interact with this data in the TrackerData struct, you need to add methods using an impl block. These methods will handle adding records and calculating totals:

impl TrackerData {
    pub fn push_record(&mut self, record: Record) -> &Self {
        self.records.push(record);
        self
    }

    pub fn category_id(&self, category: &str) -> usize {
        self.categories[category]
    }

    pub fn miscellaneous_subcategory_id(&self) -> Option<usize> {
        self.subcategories_by_name.get("miscellaneous").copied()
    }

    pub fn subcategory_id(&self, name: &str) -> Option<usize> {
        self.subcategories_by_name.get(&name.to_lowercase()).copied()
    }

    pub fn category_name(&self, id: usize) -> Option<&String> {
        self.categories.iter().find(|(_, v)| **v == id).map(|(k, _)| k)
    }

    pub fn subcategory_name(&self, id: usize) -> Option<&String> {
        self.subcategories_by_id.get(&id)
    }

    pub fn totals(&self) -> (f64, f64) {
        self.records.iter().fold((0.0, 0.0), |mut acc, r| {
            if r.category == 1 {
                acc.0 += r.amount;
            } else {
                acc.1 += r.amount;
            }
            acc
        })
    }
}

These methods utilize key Rust patterns to manage the tracker's state:

• &mut self is used when you need to modify the data, such as pushing a new record into the vector.

• Option handles cases where a value might not exist, returning Some(value) or None.

• iter() and fold are used in the totals() method to process all records and accumulate the total income and expenses into a single tuple (f64, f64) representing total income and total expenses.

Finally, add a helper function to create the default tracker JSON structure. Add this to src/models.rs:

pub fn default_tracker_json(currency: &Currency, opening_balance: f64) -> serde_json::Value {
    serde_json::json!({
        "version": 1,
        "currency": currency.to_string(),
        "opening_balance": opening_balance,
        "created_at": chrono::Utc::now().to_rfc3339(),
        "last_modified": chrono::Utc::now().to_rfc3339(),
        "categories": {
            "income": 1,
            "expenses": 2
        },
        "subcategories_by_id": {
            "1": "miscellaneous"
        },
        "subcategories_by_name": {
            "miscellaneous": 1
        },
        "records": [],
        "next_record_id": 1,
        "next_subcategory_id": 2
    })
}

Then, register this module in your src/lib.rs file so the rest of your application can use it:

pub mod models;

Step 3: Handle Errors Properly

In a financial application, error handling is critical to ensure you don’t lose or corrupt your data. Rust uses a Result type to handle operations that might fail. A Result is either an Ok containing the successful value or an Err containing the error details. This structure forces you to address potential failures explicitly before your code will compile.

Create a new file named src/error.rs and start with the necessary imports:

use std::io;

Now, define your custom error types using enums:

#[derive(Debug)]
pub enum ValidationErrorKind {
    AmountTooSmall { amount: f64 },
    InvalidDate { provided: String, expected_format: String },
    SubcategoryNotFound { name: String },
    SubcategoryAlreadyExists { name: String },
    RecordNotFound { id: usize },
    SubcategoryHasRecords { name: String, count: usize },
    CannotDeleteMiscellaneous,
    CategoryImmutable { category: usize },
    InvalidCategoryName { name: String, reason: String },
    InvalidName { name: String, reason: String },
    InvalidAmount { reason: String },
    TrackerAlreadyInitialized,
    InvalidSubcommand { subcommand: String },
}

#[derive(Debug)]
pub enum CliError {
    FileNotFound(String),
    InvalidJson(String),
    ValidationError(ValidationErrorKind),
    PermissionDenied(String),
    CorruptedData { backup_restored: bool, timestamp: String },
    FileAlreadyExists,
    Other(String),
}

This nested structure allows you to categorize every possible failure that can occur during the execution of your program. The CliError enum acts as the top-level container for all errors in the application. It handles errors like missing files, denied permissions, validation errors, file existence conflicts, and so on.

One specific variant, ValidationError, carries a ValidationErrorKind as its payload. This allows you to group all validation-specific failures (such as invalid date formats, duplicate subcategory names, or attempts to delete protected system categories) under a single error type while still preserving the specific details of what went wrong.

Structuring your errors this way allows you to report exactly what caused a failure alongside the specific data that triggered it. For example, a validation error can include the exact amount or date that failed your rules, while a system error can pinpoint the specific file path or permission issue that stopped the program.

Map System Errors to Custom Errors

To keep your application code clean, you can use the From trait to automatically convert low-level system errors into your custom CliError. This allows you to use the ? operator later in your logic to propagate errors gracefully.

Add these implementations to src/error.rs:

impl From<std::io::Error> for CliError {
    fn from(err: std::io::Error) -> Self {
        match err.kind() {
            std::io::ErrorKind::NotFound => CliError::FileNotFound(err.to_string()),
            std::io::ErrorKind::PermissionDenied => CliError::PermissionDenied(err.to_string()),
            std::io::ErrorKind::AlreadyExists => CliError::FileAlreadyExists,
            // ... add more here as is required.
            _ => CliError::Other(format!("IO error: {}", err)),
        }
    }
}

impl From<serde_json::Error> for CliError {
    fn from(err: serde_json::Error) -> Self {
        CliError::InvalidJson(err.to_string())
    }
}

The match block inside the std::io::Error implementation allows you to inspect the system error and categorize it correctly. If the system reports a "NotFound" error, your application transforms it into a CliError::FileNotFound. This ensures that your user-facing messages remain consistent.

Prepare for Error Output

Finally, add a method signature to the CliError block. This will later connect your error logic to a dedicated output module that formats these errors for the terminal:

impl CliError {
    pub fn write_to(&self, writer: &mut impl std::io::Write) -> io::Result<()> {
        crate::output::write_error(self, writer)
    }
}

The &mut impl std::io::Write parameter is a flexible way to say this method can write to any output stream, whether it’s the standard error stream in the terminal or a log file.

Register the error module in your src/lib.rs file so it’s available to the rest of your project:

pub mod models;
pub mod error;

Step 4: Create File Operations

To manage your tracker data, you need a reliable way to read and write JSON files. Instead of repeating file logic in every command, you’ll create a trait. In Rust, traits allow you to add new methods to existing types. Here, you’ll add custom file-handling methods directly to Path and PathBuf.

First, create a new directory named src/utils and create a file inside it called src/utils/file.rs. Start with the necessary imports:

use std::{
  fs::{self, File},
  io::{self, prelude::*},
  path::Path,
};

use serde_json::Value;

use crate::CliError;

Now, define and implement the FilePath trait:

pub trait FilePath: AsRef<Path> {
    fn create_file_if_not_exists(&self) -> io::Result<File> {
        let path = self.as_ref();
        if let Some(parent) = path.parent() {
            fs::create_dir_all(parent)?;
        }
        File::options().write(true).create_new(true).open(path)
    }

    fn read_file(&self) -> io::Result<File> {
        File::options().read(true).open(self.as_ref())
    }

    fn open_read_write(&self) -> io::Result<File> {
        File::options().read(true).write(true).open(self.as_ref())
    }

    fn open_read(&self) -> io::Result<File> {
        File::options().read(true).open(self.as_ref())
    }

    fn delete_if_exists(&self) -> io::Result<()> {
        let path = self.as_ref();
        if !path.exists() {
            return Ok(());
        }
        if path.is_dir() {
            fs::remove_dir_all(path)?;
        } else {
            fs::remove_file(path)?;
        }
        Ok(())
    }
}

impl<P: AsRef<Path>> FilePath for P {}

This "blanket implementation" at the end is powerful. It ensures that any type capable of representing a file path, like a PathBuf or a standard String, automatically gains these methods.

Throughout these methods, you use the ? operator. This is Rust’s shorthand for error propagation. If an operation like create_dir_all fails, the ? immediately returns the error from the function. If it succeeds, the program continues to the next line. This keeps your logic flat and readable without nested error checks.

Add JSON Utility Functions

Writing financial data to a file requires precision. You must ensure that you are completely overwriting the old data rather than just appending to it. Add this helper function to src/utils/file.rs:

pub fn write_json_to_file(json: &Value, file: &mut File) -> Result<(), CliError> {
    let json_string = serde_json::to_string_pretty(&json)?;

    file.seek(io::SeekFrom::Start(0))?;
    file.set_len(0)?;
    file.write_all(json_string.as_bytes())?;

    Ok(())
}

The seek call moves the file pointer back to the very beginning, and set_len(0) truncates the file to zero bytes. Using to_string_pretty ensures your JSON file is human-readable, which fits the local-first goal of keeping your data accessible.

Register the Utilities

To make these tools available to the rest of your application, you need to set up the module tree. Create src/utils.rs and add this line:

pub mod file;

Then, update your src/lib.rs file to include the new utils module and export the types you've built so far:

pub mod models;
pub mod error;
pub mod utils;

pub use error::{CliError, ValidationErrorKind};
pub use models::{Category, Currency, Record, TrackerData};

Step 5: Set Up the CLI Structure

In this step, you will organize the interface that allows users to interact with your code. Building a CLI is more than just reading strings. It involves mapping specific terminal commands to the internal logic of your application.

The Command Architecture

You’ll follow a modular pattern where each command has its own definition and execution logic. This separation ensures that adding a new feature in the future doesn’t break your existing commands.

Create a file named src/commands.rs. This file acts as a central dispatcher that declares your command modules and routes terminal input to the correct function:

use crate::{CliResult, command_prelude::*};
use clap::{ArgMatches, Command};

pub type Exec = fn(&mut GlobalContext, &ArgMatches) -> CliResult;

pub fn cli() -> Vec<Command> {
    vec![
        init::cli(),
        add::cli(),
        list::cli(),
        update::cli(),
        delete::cli(),
        subcategory::cli(),
        total::cli(),
    ]
}

pub fn build_exec(cmd: &str) -> Option<Exec> {
    match cmd {
        "init" => Some(init::exec),
        "add" => Some(add::exec),
        "list" => Some(list::exec),
        "update" => Some(update::exec),
        "delete" => Some(delete::exec),
        "subcategory" => Some(subcategory::exec),
        "total" => Some(total::exec),
        _ => None,
    }
}

pub mod init;
pub mod add;
pub mod list;
pub mod update;
pub mod delete;
pub mod subcategory;
pub mod total;

The Exec type alias defines a standard signature for all your command functions. Every command will receive the global context and the arguments parsed by clap, and every command will return a CliResult.

The Exec type alias defines a standard signature for all your command functions. Every command will receive the global context and the arguments parsed by clap. The build_exec function then uses pattern matching to return the specific execution logic associated with the user's input.

Manage Paths with Global Context

Since your application is local-first, it needs to know exactly where to find the data directory on different operating systems. You will create a GlobalContext struct to centralize these paths so you don’t have to rebuild them manually in every command module.

Now create src/utils/context.rs for managing file paths:

use std::path::PathBuf;

#[derive(Debug)]
pub struct GlobalContext {
    home_path: PathBuf,
    base_path: PathBuf,
    tracker_path: PathBuf,
}

impl GlobalContext {
    pub fn new(home_dir: PathBuf) -> Self {
        let base_path = home_dir.join(".fintrack");
        let tracker_path = base_path.join("tracker.json");

        GlobalContext {
            home_path: home_dir,
            base_path,
            tracker_path,
        }
    }

    pub fn tracker_path(&self) -> &PathBuf {
        &self.tracker_path
    }

    pub fn home_path(&self) -> &PathBuf {
        &self.home_path
    }

    pub fn base_path(&self) -> &PathBuf {
        &self.base_path
    }
}

The join() method is a cross-platform way to combine paths. It automatically uses the correct separator for your operating system, such as a backslash on Windows or a forward slash on Linux.

Register the Command System

To tie these components together, update your utility and library files. In src/utils.rs, add the context module:

pub mod file;
pub mod context;

Finally, update src/lib.rs to expose the command structures and the new context type. You’ll also define a CliResult type alias to keep your function signatures consistent throughout the project:

pub mod models;
pub mod error;
pub mod utils;
pub mod commands;

pub use error::*;
pub use models::*;
pub use utils::command_prelude;
pub use utils::context::GlobalContext;
pub use utils::parsers;

By defining the result type here, you ensure that every command follows the same error-handling and response rules you established in previous steps.

Step 6: Create Response Types

Commands in your tracker do more than just execute logic. They return data that must be formatted and displayed to the user.

In a command-line tool, your "user interface" is the text printed to the terminal, so you need a structured way to handle various results. You’ll create a ResponseContent enum to categorize these different outputs, such as single records, transaction lists, or financial totals. This ensures that your application communicates both successful results and informative error messages clearly.

Define the Response Structures

Open your src/models.rs file and add these structures to manage how the application packages its data:

Add to src/models.rs:

#[derive(Debug)]
pub enum ResponseContent {
    Message(String),
    Record {
        record: Record,
        tracker_data: TrackerData,
        is_update: bool,
    },
    List {
        records: Vec<Record>,
        tracker_data: TrackerData,
    },
    TrackerData(TrackerData),
    Total(Total),
    Categories(Vec<(usize, String)>),
    Subcategories(Vec<(usize, String)>),
}

#[derive(Debug, Clone)]
pub struct Total {
    pub currency: Currency,
    pub opening_balance: f64,
    pub income_total: f64,
    pub expenses_total: f64,
}

#[derive(Debug)]
pub struct CliResponse {
    content: Option<ResponseContent>,
}

impl CliResponse {
    pub fn new(content: ResponseContent) -> Self {
        CliResponse {
            content: Some(content),
        }
    }

    pub fn success() -> Self {
        CliResponse { content: None }
    }

    pub fn content(&self) -> Option<&ResponseContent> {
        self.content.as_ref()
    }

    pub fn write_to(&self, writer: &mut impl std::io::Write) -> std::io::Result<()> {
        crate::output::write_response(self, writer)
    }
}

pub type CliResult = Result<CliResponse, CliError>;

The CliResponse struct acts as a container for your output. By using an Option<ResponseContent>, you can represent a simple success message when the content is None, or provide more complex data like a Total struct when needed. This approach keeps your command logic consistent because every operation will return the same response type.

Implement the Output Module

Next, you need a central place to turn these Rust types into formatted text for the terminal. Create a new file named src/output.rs. This module will handle the printing logic for both successful responses and the errors you defined earlier.

use crate::{CliError, CliResponse, ResponseContent};

pub fn write_response(res: &CliResponse, writer: &mut impl std::io::Write) -> std::io::Result<()> {
    let Some(content) = res.content() else {
        writeln!(writer, "✓ Success")?;
        return Ok(());
    };

    match content {
        ResponseContent::Message(msg) => {
            writeln!(writer, "✓ {}", msg)?;
        }
        ResponseContent::Record { record, .. } => {
            writeln!(writer, "✓ Record created:")?;
            writeln!(writer, "  ID: {}", record.id)?;
            writeln!(writer, "  Amount: {}", record.amount)?;
            // More formatting later
        }
        ResponseContent::List { records, .. } => {
            for record in records {
                writeln!(writer, "{:?}", record)?;
            }
        }
        ResponseContent::Total(total) => {
            writeln!(writer, "Opening Balance: {} {}", total.opening_balance, total.currency)?;
            writeln!(writer, "Total Income: {} {}", total.income_total, total.currency)?;
            writeln!(writer, "Total Expenses: {} {}", total.expenses_total, total.currency)?;
            let net_balance = total.opening_balance + total.income_total - total.expenses_total;
            writeln!(writer, "Net Balance: {} {}", net_balance, total.currency)?;
        }
        _ => {}
    }
    Ok(())
}

pub fn write_error(err: &CliError, writer: &mut impl std::io::Write) -> std::io::Result<()> {
    match err {
        CliError::FileNotFound(msg) => writeln!(writer, "Error: File not found: {}", msg),
        CliError::InvalidJson(msg) => writeln!(writer, "Error: Invalid JSON: {}", msg),
        CliError::ValidationError(kind) => {
            match kind {
                crate::ValidationErrorKind::AmountTooSmall { amount } => {
                    writeln!(writer, "Error: Amount must be greater than 0, got {}", amount)
                }
                crate::ValidationErrorKind::SubcategoryNotFound { name } => {
                    writeln!(writer, "Error: Subcategory '{}' not found", name)
                }
                crate::ValidationErrorKind::RecordNotFound { id } => {
                    writeln!(writer, "Error: Record with ID {} not found", id)
                }
                _ => writeln!(writer, "Error: Validation failed"),
            }
        }
        CliError::FileAlreadyExists => {
            writeln!(writer, "Error: Tracker already initialized. Use 'fintrack clear' to start fresh.")
        }
        _ => writeln!(writer, "Error: {}", err),
    }
}

By centralizing the output logic in this module, you fulfill the goal of reporting the exact data that caused a failure alongside the error message itself. If a user enters an invalid amount, the error output clearly identifies the problematic value.

Update the Library Registration

To finalize this step, register the output module and export the new response types in your src/lib.rs file:

pub mod commands;
pub mod error;
pub mod models;
pub mod output;
pub mod utils;

pub use error::*;
pub use models::*;
pub use utils::command_prelude;
pub use utils::context::GlobalContext;
pub use utils::parsers;

Step 7: Create Argument Parsing Helpers

Extracting specific values like transaction amounts, dates, or categories from raw command-line input can quickly lead to repetitive code. While clap provides the basic parsing, you need a streamlined way to convert those inputs into the specific types used by your tracker. By creating a custom trait to extend clap, you handle type conversion and error reporting in a single, consistent place.

Create a new file named src/utils/cli.rs and add the following implementation:

use chrono::NaiveDate;
use clap::ArgMatches;
use crate::{Category, CliError, Currency};

const DEFAULT_F64: f64 = 0.0;
const DEFAULT_USIZE: usize = 0;
const DEFAULT_SUBCATEGORY: &str = "miscellaneous";

pub trait ArgMatchesExt {
    fn get_category(&self, id: &str) -> Result<&Category, CliError>;
    fn get_usize(&self, id: &str) -> Result<usize, CliError>;
    fn get_category_opt(&self, id: &str) -> Option<&Category>;
    fn get_f64_opt(&self, id: &str) -> Option<f64>;
    fn get_usize_opt(&self, id: &str) -> Option<usize>;
    fn get_string_opt(&self, id: &str) -> Option<String>;
    fn get_subcategory_opt(&self, id: &str) -> Option<String>;
    fn get_date_opt(&self, id: &str) -> Option<NaiveDate>;
    fn get_currency_opt(&self, id: &str) -> Option<&Currency>;
    fn get_f64_or_default(&self, id: &str) -> f64;
    fn get_usize_or_default(&self, id: &str) -> usize;
    fn get_string_or_default(&self, id: &str) -> String;
    fn get_subcategory_or_default(&self, id: &str) -> String;
    fn get_currency_or_default(&self, id: &str) -> &Currency;
    fn get_vec<T: Clone + Send + Sync + 'static>(&self, id: &str) -> Vec<T>;
    fn contains_id(&self, id: &str) -> bool;
}

impl ArgMatchesExt for ArgMatches {
    fn get_category(&self, id: &str) -> Result<&Category, CliError> {
        self.get_one::<Category>(id).ok_or_else(|| {
            CliError::ValidationError(crate::ValidationErrorKind::InvalidCategoryName {
                name: id.to_string(),
                reason: "Category not provided".to_string(),
            })
        })
    }

    fn get_usize(&self, id: &str) -> Result<usize, CliError> {
        self.get_one::<usize>(id).copied().ok_or_else(|| {
            CliError::Other(format!("Required argument '{}' not provided", id))
        })
    }

    fn get_category_opt(&self, id: &str) -> Option<&Category> {
        self.get_one::<Category>(id)
    }

    fn get_f64_opt(&self, id: &str) -> Option<f64> {
        self.get_one::<f64>(id).copied()
    }

    fn get_usize_opt(&self, id: &str) -> Option<usize> {
        self.get_one::<usize>(id).copied()
    }

    fn get_string_opt(&self, id: &str) -> Option<String> {
        self.get_one::<String>(id).cloned()
    }

    fn get_subcategory_opt(&self, id: &str) -> Option<String> {
        self.get_one::<String>(id).cloned()
    }

    fn get_date_opt(&self, id: &str) -> Option<NaiveDate> {
        self.get_one::<NaiveDate>(id).copied()
    }

    fn get_currency_opt(&self, id: &str) -> Option<&Currency> {
        self.get_one::<Currency>(id)
    }

    fn get_f64_or_default(&self, id: &str) -> f64 {
        self.get_one::<f64>(id).copied().unwrap_or(DEFAULT_F64)
    }

    fn get_usize_or_default(&self, id: &str) -> usize {
        self.get_one::<usize>(id).copied().unwrap_or(DEFAULT_USIZE)
    }

    fn get_string_or_default(&self, id: &str) -> String {
        self.get_one::<String>(id).cloned().unwrap_or_default()
    }

    fn get_subcategory_or_default(&self, id: &str) -> String {
        self.get_one::<String>(id)
            .cloned()
            .unwrap_or_else(|| DEFAULT_SUBCATEGORY.to_string())
    }

    fn get_currency_or_default(&self, id: &str) -> &Currency {
        self.get_one::<Currency>(id).unwrap_or(&Currency::NGN)
    }

    fn get_vec<T: Clone + Send + Sync + 'static>(&self, id: &str) -> Vec<T> {
        self.get_many::<T>(id)
            .map(|iter| iter.cloned().collect())
            .unwrap_or_default()
    }

    fn contains_id(&self, id: &str) -> bool {
        ArgMatches::contains_id(self, id)
    }
}

These helper methods allow you to decide exactly how to handle missing data. Methods like ok_or_else convert an empty input into a specific CliError that informs the user exactly which argument is missing. In contrast, unwrap_or_else allows the application to provide sensible fallbacks, such as defaulting to the "miscellaneous" subcategory if the user does not specify one.

Implement Custom Data Parsers

Standard command-line arguments are received as strings. To turn them into useful data types like dates or categories, you need specific parsing logic. Create a new file src/utils/parsers.rs:

use chrono::NaiveDate;
use crate::Category;

pub fn parse_date(s: &str) -> Result<NaiveDate, String> {
    NaiveDate::parse_from_str(s, "%d-%m-%Y")
        .map_err(|_| format!("'{}' is not in the format DD-MM-YYYY", s))
}

pub fn parse_category(s: &str) -> Result<Category, String> {
    s.parse::<Category>().map_err(|_| {
        format!("'{}' is not a valid category. Use 'income' or 'expenses'", s)
    })
}

We'll use these parsers to ensure that any input that doesn’t match your expected format is caught immediately with a clear error message before it ever reaches your core logic.

To finalize this setup, update src/utils.rs to include the new helper files:

pub mod cli;
pub mod command_prelude;
pub mod context;
pub mod file;
pub mod parsers;

This infrastructure ensures that when you begin implementing the actual financial commands, you can focus on the business logic instead of fighting with string conversions and repeating argument validation.

Next we'll begin implementing the business logic for the commands, starting with the init command.

Step 8: Implement the Init Command

The init command will set up the workspace for the financial tracker. It will handle the creation of the hidden directory structure in your home folder and generate the initial JSON file with default settings like the currency and starting balance.

Create the src/commands/init.rs file and add this code:

use clap::{Arg, ArgMatches, Command};

use crate::command_prelude::ArgMatchesExt;
use crate::utils::file::{FilePath, write_json_to_file};
use crate::{CliResponse, CliResult, Currency, GlobalContext, default_tracker_json};

pub fn cli() -> Command {
    Command::new("init")
        .about("Initialize a new financial tracker")
        .arg(
            Arg::new("currency")
                .short('c')
                .value_parser(clap::value_parser!(Currency))
                .default_value("ngn"),
        )
        .arg(
            Arg::new("opening")
                .short('o')
                .value_parser(clap::value_parser!(f64)),
        )
}

pub fn exec(gctx: &mut GlobalContext, args: &ArgMatches) -> CliResult {
    let currency = args.get_currency_or_default("currency");
    let opening_balance = args.get_f64_or_default("opening");

    let mut file = gctx.tracker_path().create_file_if_not_exists()?;

    let default_json = default_tracker_json(currency, opening_balance);
    write_json_to_file(&default_json, &mut file)?;

    Ok(CliResponse::success())
}

The cli function defines the command's interface. Command::new("init") sets the name of the subcommand the user types. Within the .arg() blocks, .short('c') and .long("currency") allow for two different ways to provide the same data. A user can choose the concise short form or the more descriptive long form:

fintrack init -c usd -o 5000

OR

fintrack init --currency usd --opening 5000

Both commands map to the same internal "currency" and "opening" arguments.

The exec function performs the actual initialization of the tracker. It uses the helpers built in previous steps to keep the logic concise. Specifically, it uses get_currency_or_default and get_f64_or_default from the ArgMatchesExt trait you created in src/utils/cli.rs.

When attempting to create the tracker file, it calls create_file_if_not_exists. This method belongs to the FilePath trait you implemented in src/utils/file.rs. Because that method was built using create_new(true), it acts as a guard that fails if a tracker already exists with an error std::io::ErrorKind::AlreadyExists. This failure is caught and converted into a CliError::FileAlreadyExists message, which was defined in your error handling logic in src/error.rs.

The default_tracker_json function builds the initial state for the application. It packages the base currency, opening balance, and default "miscellaneous" subcategory into a JSON structure. Finally, the write_json_to_file helper from src/utils/file.rs writes this data to the disk.

Step 9: Implement the Add Command

The add command will add a new record. To do this, you’ll implement code that will read the existing data from the JSON file, validate the new input, and save the updated record back to the JSON file.

Create src/commands/add.rs and insert this code:

use chrono::Local;
use clap::{Arg, ArgMatches, Command};

use crate::command_prelude::ArgMatchesExt;
use crate::utils::file::{FilePath, write_json_to_file};
use crate::utils::parsers::{parse_category, parse_date};
use crate::{
    CliError, CliResponse, CliResult, GlobalContext, Record, ResponseContent, TrackerData,
};

pub fn cli() -> Command {
    Command::new("add")
        .about("Record a new income or expense transaction")
        .arg(
            Arg::new("category")
                .index(1)
                .required(true)
                .value_parser(parse_category),
        )
        .arg(
            Arg::new("amount")
                .index(2)
                .required(true)
                .value_parser(clap::value_parser!(f64)),
        )
        .arg(
            Arg::new("subcategory")
                .short('s')
                .long("subcategory")
                .default_value("miscellaneous"),
        )
        .arg(
            Arg::new("description")
                .short('d')
                .long("description"),
        )
        .arg(
            Arg::new("date")
                .short('D')
                .long("date")
                .value_parser(parse_date),
        )
}

pub fn exec(gctx: &mut GlobalContext, args: &ArgMatches) -> CliResult {
    let mut file = gctx.tracker_path().open_read_write()?;
    let mut tracker_data: TrackerData = serde_json::from_reader(&file)?;

    let category = args.get_category("category")?;
    let amount = args.get_f64_or_default("amount");

    if amount <= 0.0 {
        return Err(CliError::ValidationError(
            crate::ValidationErrorKind::AmountTooSmall { amount },
        ));
    }

    let subcategory_name = args.get_subcategory_or_default("subcategory");
    let description = args.get_string_or_default("description");

    let category_str = category.to_string();
    let category_id = tracker_data.category_id(&category_str);

    let subcategory_id = tracker_data
        .subcategory_id(&subcategory_name)
        .ok_or_else(|| {
            CliError::ValidationError(crate::ValidationErrorKind::SubcategoryNotFound {
                name: subcategory_name,
            })
        })?;

    let date = args
        .get_date_opt("date")
        .map(|d| d.format("%d-%m-%Y").to_string())
        .unwrap_or_else(|| Local::now().format("%d-%m-%Y").to_string());

    let record_id = tracker_data.next_record_id;
    let record = Record {
        id: record_id,
        category: category_id,
        amount,
        subcategory: subcategory_id,
        description,
        date,
    };

    tracker_data.next_record_id += 1;
    tracker_data.last_modified = chrono::Utc::now().to_rfc3339();
    tracker_data.push_record(record.clone());

    let tracker_json = serde_json::json!(tracker_data);
    write_json_to_file(&tracker_json, &mut file)?;

    Ok(CliResponse::new(ResponseContent::Record {
        record,
        tracker_data,
        is_update: false,
    }))
}

The cli function here defines positional arguments using .index(1) and .index(2). This means users can provide the category and amount without specific flags. An example usage looks like this:

fintrack add income 1500 -s salary -d "Monthly pay"

In this command, "income" maps to the "category" and 1500 maps to the "amount". The parser logic for these inputs makes use of the parse_category and parse_date functions created in src/utils/parsers.rs.

The exec function here opens the data file with open_read_write from the FilePath trait (src/utils/file.rs) and extracts user input using the ArgMatchesExt trait (src/utils/cli.rs).

The date logic handles optional input through a chain of methods. get_date_opt returns an Option, such that when a date exists, .map transforms it into the required string format. When a date doesn’t exist, .unwrap_or_else provides the current system date as a default.

Once the Record struct is populated, the code updates the TrackerData state and saves the result using the write_json_to_file helper. The final CliResponse contains the record details for the output module in src/output.rs to display.

Step 10: Implement the List Command

The list command will provide a way to view and filter records. This logic involves loading the data file, applying criteria like date ranges or categories, and sorting the results chronologically.

Create src/commands/list.rs and add the following code:

use chrono::NaiveDate;
use clap::{Arg, ArgGroup, ArgMatches, Command};

use crate::command_prelude::ArgMatchesExt;
use crate::utils::file::FilePath;
use crate::utils::parsers::{parse_category, parse_date};
use crate::{CliResponse, CliResult, GlobalContext, Record, ResponseContent, TrackerData};

pub fn cli() -> Command {
    Command::new("list")
        .about("View and filter your transaction records")
        .arg(
            Arg::new("first")
                .short('f')
                .long("first")
                .value_parser(clap::value_parser!(usize)),
        )
        .arg(
            Arg::new("last")
                .short('l')
                .long("last")
                .value_parser(clap::value_parser!(usize)),
        )
        .group(
            ArgGroup::new("first_or_last")
                .args(["first", "last"])
                .multiple(false),
        )
        .arg(
            Arg::new("start")
                .short('S')
                .long("start")
                .value_parser(parse_date),
        )
        .arg(
            Arg::new("end")
                .short('E')
                .long("end")
                .value_parser(parse_date),
        )
        .arg(
            Arg::new("category")
                .short('c')
                .long("category")
                .value_parser(parse_category),
        )
        .arg(
            Arg::new("subcategory")
                .short('s')
                .long("subcategory"),
        )
}

pub fn exec(gctx: &mut GlobalContext, args: &ArgMatches) -> CliResult {
    let file = gctx.tracker_path().open_read()?;
    let tracker_data: TrackerData = serde_json::from_reader(&file)?;

    let start_date = args.get_date_opt("start");
    let end_date = args.get_date_opt("end");

    let category_filter = args
        .get_category_opt("category")
        .map(|cat| tracker_data.category_id(&cat.to_string()));

    let subcategory_filter = args
        .get_subcategory_opt("subcategory")
        .and_then(|name| tracker_data.subcategory_id(&name));

    let mut filtered_data: Vec<Record> = tracker_data
        .records
        .iter()
        .filter(|r| {
            let matches_category = category_filter
                .map(|expected_id| r.category == expected_id)
                .unwrap_or(true);

            let matches_subcategory = subcategory_filter
                .map(|expected_id| r.subcategory == expected_id)
                .unwrap_or(true);

            let matches_date = NaiveDate::parse_from_str(&r.date, "%d-%m-%Y")
                .map(|record_date| {
                    let after_start = start_date.map_or(true, |start| record_date >= start);
                    let before_end = end_date.map_or(true, |end| record_date <= end);
                    after_start && before_end
                })
                .unwrap_or(false);

            matches_category && matches_subcategory && matches_date
        })
        .cloned()
        .collect();

    filtered_data.sort_by(|a, b| {
        let date_a = NaiveDate::parse_from_str(&a.date, "%d-%m-%Y").unwrap_or(NaiveDate::MIN);
        let date_b = NaiveDate::parse_from_str(&b.date, "%d-%m-%Y").unwrap_or(NaiveDate::MIN);
        date_a.cmp(&date_b)
    });

    if args.contains_id("first") {
        let first = args.get_usize_or_default("first");
        if first > 0 {
            filtered_data.truncate(first);
        }
    } else if args.contains_id("last") {
        let last = args.get_usize_or_default("last");
        if last > 0 && filtered_data.len() > last {
            let start_idx = filtered_data.len() - last;
            filtered_data = filtered_data.into_iter().skip(start_idx).collect();
        }
    }

    Ok(CliResponse::new(ResponseContent::List {
        records: filtered_data,
        tracker_data,
    }))
}

The cli function utilizes an ArgGroup named "first_or_last". This ensures the user cannot request both the first N and last N records at the same time. The command supports multiple filtering flags, which allows a user to run queries like:

fintrack list -c expenses -S 01-01-2024 -E 31-01-2024

The command above filters for "expenses" specifically within the month of January 2024.

The exec function uses open_read from the FilePath trait (src/utils/file.rs) to access the tracker file without write permissions. The filtering logic makes use of methods like and_then and map_or to handle optional criteria. For example, the date filter uses map_or(true, ...) to include a record if no specific start or end date was provided.

The record sorting uses sort_by to compare record dates. Since dates are stored as strings in the JSON file, they are parsed into NaiveDate objects temporarily for an accurate chronological comparison. Finally, the function uses truncate or skip to limit the results based on the "first" or "last" arguments before returning a ResponseContent::List for the output module in src/output.rs to process.

Step 11: Implement the Update Command

The update command will allow the user to modify specific fields in an existing record. It will accept similar arguments as the add command but unlike the add command, every argument except the ID will be optional, enabling the user to change only what is necessary.

Create src/commands/update.rs and add the following code:

use clap::{Arg, ArgMatches, Command};

use crate::command_prelude::ArgMatchesExt;
use crate::utils::file::{FilePath, write_json_to_file};
use crate::utils::parsers::{parse_category, parse_date};
use crate::{CliError, CliResponse, CliResult, GlobalContext, ResponseContent, TrackerData};

pub fn cli() -> Command {
    Command::new("update")
        .about("Modify an existing transaction record")
        .arg(
            Arg::new("record_id")
                .index(1)
                .required(true)
                .value_parser(clap::value_parser!(usize)),
        )
        .arg(
            Arg::new("category")
                .short('c')
                .long("category")
                .value_parser(parse_category),
        )
        .arg(
            Arg::new("amount")
                .short('a')
                .long("amount")
                .value_parser(clap::value_parser!(f64)),
        )
        .arg(
            Arg::new("subcategory")
                .short('s')
                .long("subcategory"),
        )
        .arg(
            Arg::new("description")
                .short('d')
                .long("description"),
        )
        .arg(
            Arg::new("date")
                .short('D')
                .long("date")
                .value_parser(parse_date),
        )
}

pub fn exec(gctx: &mut GlobalContext, args: &ArgMatches) -> CliResult {
    let mut file = gctx.tracker_path().open_read_write()?;
    let mut tracker_data: TrackerData = serde_json::from_reader(&file)?;

    let record_id = args
        .get_usize("record_id")
        .map_err(|_| CliError::ValidationError(crate::ValidationErrorKind::RecordNotFound { id: 0 }))?;

    let category_id = args.get_category_opt("category").map(|category| {
        let category_str = category.to_string();
        tracker_data.category_id(&category_str)
    });

    let subcategory_id = args
        .get_subcategory_opt("subcategory")
        .map(|name| {
            tracker_data.subcategory_id(&name).ok_or_else(|| {
                CliError::ValidationError(crate::ValidationErrorKind::SubcategoryNotFound { name })
            })
        })
        .transpose()?;

    let record = tracker_data
        .records
        .iter_mut()
        .find(|r| r.id == record_id)
        .ok_or_else(|| {
            CliError::ValidationError(crate::ValidationErrorKind::RecordNotFound { id: record_id })
        })?;

    if let Some(cat_id) = category_id {
        record.category = cat_id;
    }

    if let Some(amount) = args.get_f64_opt("amount") {
        if amount <= 0.0 {
            return Err(CliError::ValidationError(
                crate::ValidationErrorKind::AmountTooSmall { amount },
            ));
        }
        record.amount = amount;
    }

    if let Some(subcat_id) = subcategory_id {
        record.subcategory = subcat_id;
    }

    if let Some(description) = args.get_string_opt("description") {
        record.description = description;
    }

    if let Some(date) = args.get_date_opt("date") {
        record.date = date.format("%d-%m-%Y").to_string();
    }

    tracker_data.last_modified = chrono::Utc::now().to_rfc3339();

    let updated_record = record.clone();

    let tracker_json = serde_json::json!(tracker_data);
    write_json_to_file(&tracker_json, &mut file)?;

    Ok(CliResponse::new(ResponseContent::Record {
        record: updated_record,
        tracker_data,
        is_update: true,
    }))
}

The cli function requires a positional "record_id" so the program knows which record to target. Users can find this ID by running the list command. An update command looks like this:

fintrack update 5 -a 2000 -d "Updated price"

The command above specifically updates the amount and description for record number 5, leaving all other fields unchanged.

The exec function uses iter_mut() and find() to locate the specific record in your data list. Because iter_mut() provides a mutable reference, any changes made to the record variable directly update the object inside tracker_data.records.

To handle the optional subcategory update, the code uses transpose(). This method is essential here because looking up a subcategory name is optional. But if a name is provided and it doesn't exist, the program should stop and return an error. transpose() flips the Option<Result> into a Result<Option>, allowing the ? operator to handle the error while still giving you an Option to work with.

The final state is saved back to the file using the write_json_to_file helper from src/utils/file.rs. The CliResponse indicates that an update occurred by setting is_update: true, which the output module uses to format the success message appropriately.

Step 12: Implement the Delete Command

The delete command will remove specific records from the tracker. This implementation will support multiple deletion strategies: targeting individual IDs, removing an entire category, or clearing a specific subcategory.

use std::collections::HashSet;

use clap::{Arg, ArgAction, ArgGroup, ArgMatches, Command};

use crate::{
    CliResponse, CliResult, GlobalContext, TrackerData,
    command_prelude::ArgMatchesExt,
    utils::file::{FilePath, write_json_to_file},
    utils::parsers::parse_category,
};

pub fn cli() -> Command {
    Command::new("delete")
        .about("Delete transaction records")
        .arg(
            Arg::new("ids")
                .short('i')
                .long("ids")
                .value_parser(clap::value_parser!(usize))
                .action(ArgAction::Append)
                .value_delimiter(','),
        )
        .arg(
            Arg::new("by-cat")
                .short('c')
                .long("by-cat")
                .value_parser(parse_category),
        )
        .arg(
            Arg::new("by-subcat")
                .short('s')
                .long("by-subcat"),
        )
        .group(
            ArgGroup::new("delete_by")
                .args(["ids", "by-cat", "by-subcat"])
                .multiple(false)
                .required(true),
        )
}

pub fn exec(gctx: &mut GlobalContext, args: &ArgMatches) -> CliResult {
    let mut file = gctx.tracker_path().open_read_write()?;
    let mut tracker_data: TrackerData = serde_json::from_reader(&file)?;

    if args.contains_id("ids") {
        let ids: Vec<usize> = args.get_vec::<usize>("ids");
        let ids_set: HashSet<usize> = ids.into_iter().collect();

        tracker_data.records.retain(|r| !ids_set.contains(&r.id));
    } else if args.contains_id("by-cat") {
        let category = args.get_category("by-cat")?;
        let category_str = category.to_string();
        let category_id = tracker_data.category_id(&category_str);

        tracker_data.records.retain(|r| r.category != category_id);
    } else if args.contains_id("by-subcat") {
        let subcategory_name = args
            .get_subcategory_opt("by-subcat")
            .ok_or_else(|| crate::CliError::Other("Subcategory not provided".to_string()))?;

        let subcategory_id = tracker_data
            .subcategory_id(subcategory_name.as_str())
            .ok_or_else(|| {
                crate::CliError::ValidationError(crate::ValidationErrorKind::SubcategoryNotFound {
                    name: subcategory_name.clone(),
                })
            })?;

        tracker_data
            .records
            .retain(|r| r.subcategory != subcategory_id);
    }

    tracker_data.last_modified = chrono::Utc::now().to_rfc3339();

    let tracker_json = serde_json::json!(tracker_data);
    write_json_to_file(&tracker_json, &mut file)?;

    Ok(CliResponse::success())
}

The cli function makes use of an ArgGroup to enforce that only one deletion method is used at a time ("ids, "by-cat", or "by-subcat"). The "ids" argument uses value_delimiter(','), allowing a user to pass multiple IDs separated by a comma (','). For example:

fintrack delete --ids 1,4,7

Also, a user can clear all records of a particular category or subcategory using the "by-cat" or "by-subcat" flag. For example:

fintrack delete --by-cat expenses

The exec function determines which records to target based on three possible inputs. If --ids is used, it collects the provided values directly into a HashSet. If --by-cat or --by-subcat is used, the code iterates through existing records and gathers the IDs of every record that matches that specific category or subcategory and stores it in a HashSet. Regardless of the flag used, the logic converges on a HashSet containing all IDs scheduled for removal.

Using a HashSet makes the final cleanup highly efficient because it allows the program to check if an ID exists in the "delete list" almost instantly. The retain method then keeps only the records whose IDs are not in that set, effectively pruning the data in place.

After the removal, the code calculates the difference between the initial and final record counts. If no records were removed, it returns a RecordNotFound error. Otherwise, it updates the last_modified timestamp and saves the updated JSON using the write_json_to_file helper from src/utils/file.rs.

Step 13: Implement Subcategory Commands

The subcategory command will serve as a parent for several nested subcommands, allowing users to organize their records beyond the basic "income" and "expenses" categories. This structure will use a modular approach, where each management task will live in its own dedicated file.

Create the entry point file at src/commands/subcategory.rs:

use clap::{ArgMatches, Command};

use crate::{CliResult, GlobalContext, commands::Exec, invalid_subcommand_error};

pub fn cli() -> Command {
    Command::new("subcategory")
        .about("Manage your subcategories")
        .subcommand_required(true)
        .subcommands(build_cli())
}

pub fn exec(gctx: &mut GlobalContext, args: &ArgMatches) -> CliResult {
    match args.subcommand() {
        Some((cmd, sub_args)) => {
            let exec_fn = build_exec(cmd).ok_or_else(|| invalid_subcommand_error(cmd))?;

            exec_fn(gctx, sub_args)
        }
        None => Err(invalid_subcommand_error("")),
    }
}

fn build_cli() -> Vec<Command> {
    vec![add::cli(), delete::cli(), list::cli(), rename::cli()]
}

fn build_exec(cmd: &str) -> Option<Exec> {
    match cmd {
        "add" => Some(add::exec),
        "delete" => Some(delete::exec),
        "list" => Some(list::exec),
        "rename" => Some(rename::exec),
        "update" => Some(rename::exec),
        _ => None,
    }
}

pub mod list;
pub mod add;
pub mod delete;
pub mod rename;

The cli function here sets subcommand_required(true). This means the user must specify an action. The exec function uses a match statement to delegate the logic to the appropriate module.

List Subcategories

Create src/commands/subcategory/list.rs:

use clap::{ArgMatches, Command};

use crate::{CliResponse, CliResult, GlobalContext, ResponseContent, TrackerData, utils::file::FilePath};

pub fn cli() -> Command {
    Command::new("list")
        .about("View all available subcategories")
}

pub fn exec(gctx: &mut GlobalContext, _args: &ArgMatches) -> CliResult {
    let file = gctx.tracker_path().open_read()?;
    let tracker_data: TrackerData = serde_json::from_reader(&file)?;

    let mut subcategories: Vec<(usize, String)> = tracker_data
        .subcategories_by_id
        .iter()
        .map(|(&id, name)| (id, name.clone()))
        .collect();

    subcategories.sort_by_key(|(id, _)| *id);

    Ok(CliResponse::new(ResponseContent::Subcategories(subcategories)))
}

The exec function first accesses the data file using the open_read helper method defined earlier in src/utils/file.rs. Once the file is open, it reads the JSON content into the TrackerData struct. The logic then pulls the specific list of IDs and names from the subcategories_by_id map found in src/models.rs and converts them into a simple list for the user to view.

Add Subcategories

Create src/commands/subcategory/add.rs:

use clap::{Arg, ArgMatches, Command};

use crate::{
    CliError, CliResponse, CliResult, GlobalContext, TrackerData,
    utils::file::{FilePath, write_json_to_file},
    utils::parsers::parse_label,
};

pub fn cli() -> Command {
    Command::new("add")
        .about("Create a new subcategory")
        .arg(
            Arg::new("name")
                .index(1)
                .required(true)
                .value_parser(parse_label),
        )
}

pub fn exec(gctx: &mut GlobalContext, args: &ArgMatches) -> CliResult {
    let mut file = gctx.tracker_path().open_read_write()?;
    let mut tracker_data: TrackerData = serde_json::from_reader(&file)?;

    let name = args
        .get_one::<String>("name")
        .ok_or_else(|| CliError::Other("Subcategory name not provided".to_string()))?;

    let name_lower = name.to_lowercase();
    let name_title = {
        let mut chars = name_lower.chars();
        match chars.next() {
            None => return Err(CliError::Other("Invalid name".to_string())),
            Some(first) => first.to_uppercase().collect::<String>() + &chars.as_str().to_lowercase(),
        }
    };

    if name_lower == "miscellaneous" {
        return Err(CliError::ValidationError(
            crate::ValidationErrorKind::CannotDeleteMiscellaneous,
        ));
    }

    if tracker_data.subcategories_by_name.contains_key(&name_lower) {
        return Err(CliError::ValidationError(
            crate::ValidationErrorKind::SubcategoryAlreadyExists {
                name: name_title.clone(),
            },
        ));
    }

    let subcategory_id = tracker_data.next_subcategory_id as usize;
    tracker_data.subcategories_by_id.insert(subcategory_id, name_title.clone());
    tracker_data.subcategories_by_name.insert(name_lower, subcategory_id);
    tracker_data.next_subcategory_id += 1;
    tracker_data.last_modified = chrono::Utc::now().to_rfc3339();

    let tracker_json = serde_json::json!(tracker_data);
    write_json_to_file(&tracker_json, &mut file)?;

    Ok(CliResponse::new(crate::ResponseContent::Message(format!(
        "Subcategory '{}' added (ID: {})",
        name_title, subcategory_id
    ))))
}

The exec function here makes use of open_read_write to load the data for modification. It retrieves the user's input through the get_string_opt helper from the ArgMatchesExt trait.

To maintain consistency, the normalize_name function ensures all names follow a standard title case format. Before saving, the logic checks the subcategories_by_name map from src/models.rs to ensure the name is unique.

Once validated, it updates the next_subcategory_id and writes the changes back to disk using write_json_to_file.

Delete Subcategories

Create src/commands/subcategory/delete.rs:

use clap::{Arg, ArgMatches, Command};

use crate::{
    CliError, CliResponse, CliResult, GlobalContext, TrackerData,
    utils::file::{FilePath, write_json_to_file},
    utils::parsers::parse_label,
};

pub fn cli() -> Command {
    Command::new("delete")
        .about("Delete a subcategory")
        .arg(
            Arg::new("name")
                .index(1)
                .required(true)
                .value_parser(parse_label),
        )
}

pub fn exec(gctx: &mut GlobalContext, args: &ArgMatches) -> CliResult {
    let mut file = gctx.tracker_path().open_read_write()?;
    let mut tracker_data: TrackerData = serde_json::from_reader(&file)?;

    let name = args
        .get_one::<String>("name")
        .ok_or_else(|| CliError::Other("Subcategory name not provided".to_string()))?;

    let name_lower = name.to_lowercase();

    if name_lower == "miscellaneous" {
        return Err(CliError::ValidationError(
            crate::ValidationErrorKind::CannotDeleteMiscellaneous,
        ));
    }

    let subcategory_id = tracker_data
        .subcategory_id(&name_lower)
        .ok_or_else(|| {
            CliError::ValidationError(crate::ValidationErrorKind::SubcategoryNotFound {
                name: name.to_string(),
            })
        })?;

    let record_count = tracker_data
        .records
        .iter()
        .filter(|r| r.subcategory == subcategory_id)
        .count();

    if record_count > 0 {
        return Err(CliError::ValidationError(
            crate::ValidationErrorKind::SubcategoryHasRecords {
                name: name.to_string(),
                count: record_count,
            },
        ));
    }

    tracker_data.subcategories_by_id.remove(&subcategory_id);
    tracker_data.subcategories_by_name.remove(&name_lower);
    tracker_data.last_modified = chrono::Utc::now().to_rfc3339();

    let tracker_json = serde_json::json!(tracker_data);
    write_json_to_file(&tracker_json, &mut file)?;

    Ok(CliResponse::new(crate::ResponseContent::Message(format!(
        "Subcategory '{}' deleted",
        name
    ))))
}

The exec function performs a safety check before removing any data. It first locates the ID of the target subcategory using the name provided by the user. Then, it iterates through the records vector in the tracker data to count if any records are currently linked to that ID. If the count is greater than zero, the operation stops and returns a SubcategoryHasRecords error, preventing you from accidentally creating "orphaned" records that point to a missing subcategory. If the check passes, the subcategory is removed from both HashMaps in src/models.rs.

Rename Subcategories

Create src/commands/subcategory/rename.rs:

use clap::{Arg, ArgMatches, Command};

use crate::{
    CliError, CliResponse, CliResult, GlobalContext, TrackerData,
    utils::file::{FilePath, write_json_to_file},
    utils::parsers::parse_label,
};

pub fn cli() -> Command {
    Command::new("rename")
        .about("Rename an existing subcategory")
        .arg(
            Arg::new("old")
                .index(1)
                .required(true)
                .value_parser(parse_label),
        )
        .arg(
            Arg::new("new")
                .index(2)
                .required(true)
                .value_parser(parse_label),
        )
}

pub fn exec(gctx: &mut GlobalContext, args: &ArgMatches) -> CliResult {
    let mut file = gctx.tracker_path().open_read_write()?;
    let mut tracker_data: TrackerData = serde_json::from_reader(&file)?;

    let old_name = args
        .get_one::<String>("old")
        .ok_or_else(|| CliError::Other("Old subcategory name not provided".to_string()))?;
    let new_name = args
        .get_one::<String>("new")
        .ok_or_else(|| CliError::Other("New subcategory name not provided".to_string()))?;

    let old_name_lower = old_name.to_lowercase();
    let new_name_lower = new_name.to_lowercase();
    let new_name_title = {
        let mut chars = new_name_lower.chars();
        match chars.next() {
            None => return Err(CliError::Other("Invalid new name".to_string())),
            Some(first) => first.to_uppercase().collect::<String>() + &chars.as_str().to_lowercase(),
        }
    };

    let subcategory_id = tracker_data
        .subcategory_id(&old_name_lower)
        .ok_or_else(|| {
            CliError::ValidationError(crate::ValidationErrorKind::SubcategoryNotFound {
                name: old_name.to_string(),
            })
        })?;

    if tracker_data.subcategories_by_name.contains_key(&new_name_lower) {
        return Err(CliError::ValidationError(
            crate::ValidationErrorKind::SubcategoryAlreadyExists {
                name: new_name_title.clone(),
            },
        ));
    }

    tracker_data
        .subcategories_by_id
        .insert(subcategory_id, new_name_title.clone());
    tracker_data.subcategories_by_name.remove(&old_name_lower);
    tracker_data
        .subcategories_by_name
        .insert(new_name_lower, subcategory_id);
    tracker_data.last_modified = chrono::Utc::now().to_rfc3339();

    let tracker_json = serde_json::json!(tracker_data);
    write_json_to_file(&tracker_json, &mut file)?;

    Ok(CliResponse::new(crate::ResponseContent::Message(format!(
        "Subcategory renamed: '{}' → '{}'",
        old_name, new_name_title
    ))))
}

The exec function implements a "swap" logic to preserve record history. It first finds the numeric ID associated with the current name. Instead of changing any individual record, it simply removes the old name from the subcategories_by_name HashMap and inserts the new name with the same ID. This ensures that all existing records in src/models.rs immediately reflect the new name because they reference the subcategory by ID rather than by a string.

Step 14: Implement the Total Command

The total command will aggregate every transaction record in your JSON file to provide a clear view of your ledger's standing. It will sum up all income and expenses to show you exactly how your balance has changed since you initialized the tracker.

Create src/commands/total.rs and add the following code:

use clap::{ArgMatches, Command};

use crate::{
    CliError, CliResponse, CliResult, Currency, GlobalContext, Total, TrackerData,
    utils::file::FilePath,
};

pub fn cli() -> Command {
    Command::new("total")
        .about("Display financial summary with totals")
}


pub fn exec(gctx: &mut GlobalContext, _args: &ArgMatches) -> CliResult {
  let file = gctx.tracker_path().open_read()?;
  let tracker_data: TrackerData = serde_json::from_reader(&file)?;

  let opening_balance = tracker_data.opening_balance;

  let currency = tracker_data
    .currency
    .parse::<Currency>()
    .map_err(|e| CliError::Other(format!("Invalid currency in tracker data: {}", e)))?;

  let (income_total, expenses_total) = tracker_data.totals();

  Ok(CliResponse::new(crate::ResponseContent::Total(Total {
    currency,
    opening_balance,
    income_total,
    expenses_total,
  })))
}

The cli function defines a straightforward interface without extra flags. It focuses entirely on processing the complete dataset.

The exec function first accesses the data file using the open_read helper. After parsing the JSON into the TrackerData struct, the logic calls the totals() method you implemented in src/models.rs. That method iterates through your records to return the raw sums of all income and expenses.

The Total struct contains the opening balance, income total, and expenses total. The net balance is calculated in the output module by adding income_total to the opening_balance and subtracting expenses_total. Finally you return a CliResponse which allows the output module to take these raw numbers and render them in the terminal.

Step 15: Wire Up the Main Function

This step brings every separate module back to the source. Until now, the models, error handling, and command logic existed as isolated parts. You will now create the main.rs file to establish the central entry point that connects these pieces, allowing the application to function as a unified binary.

First, update src/lib.rs to expose the internal modules:

pub mod commands;
pub mod error;
pub mod models;
pub mod output;
pub mod utils;

pub use error::*;
pub use models::*;
pub use utils::command_prelude;
pub use utils::context::GlobalContext;
pub use utils::parsers;

Next, create src/main.rs:

use std::io;

use clap::Command;
use fintrack::{GlobalContext, commands};

fn main() {
    let exit_code = match run() {
        Ok(_) => 0,
        Err(e) => {
            eprintln!("Error: {}", e);
            1
        }
    };
    std::process::exit(exit_code);
}

fn run() -> Result<(), String> {
    let home_dir = dirs::home_dir()
        .ok_or_else(|| "Failed to determine home directory".to_string())?;

    let mut gctx = GlobalContext::new(home_dir);

    let matches = Command::new("fintrack")
        .bin_name("fintrack")
        .about("A local-first CLI financial tracker for managing income and expenses")
        .version(env!("CARGO_PKG_VERSION"))
        .subcommand_required(true)
        .subcommands(commands::cli())
        .get_matches();

    let (cmd, args) = matches
        .subcommand()
        .expect("subcommand required but not found");

    let exec_fn = commands::build_exec(cmd)
        .ok_or_else(|| format!("Unknown command: {}", cmd))?;

    let exec_result = exec_fn(&mut gctx, args);
    process_result(&exec_result).expect("An error occurred displaying response");

    Ok(())
}

fn process_result(result: &fintrack::CliResult) -> io::Result<()> {
    match result {
        Ok(res) => res.write_to(&mut std::io::stdout()),
        Err(err) => err.write_to(&mut std::io::stderr()),
    }
}

The main function serves as the supervisor for the entire process. It triggers the run function and maps the final outcome to a standard system exit code. This informs the terminal whether the operation succeeded or encountered a failure.

The run function initiates a complete roundtrip through the architecture you built in previous steps. It starts by determining the user's home directory and passing it into GlobalContext::new(home_dir). This instantiation creates the gctx object from Step 5, which determines the cross platform paths for the .fintrack folder and the tracker.json file.

When a user types a command like fintrack add in the terminal, the process begins by calling commands::cli(). This function, which you have defined in your central dispatcher src/commands.rs from Step 5, collects the list of all available subcommands (init, add, list, etc). It pulls the specific configuration for each command into a single clap instance so the terminal can understand the user intent.

If the user provides the correct input and arguments and clap's validation is passed, it calls commands::build_exec(cmd) which uses the pattern matching logic also defined in Step 5. This function returns a pointer to the specific exec function for that command. For example, if the user typed fintrack add ..., it retrieves the exec function from src/commands/add.rs. The code then executes this function using a mutable reference to the gctx you just instantiated. This grants the command access to the file paths and data it needs.

The final execution phase happens in process_result. This function takes the CliResult returned by the command and calls the write_to method you defined earlier in the Step 6 output logic. Providing mutable references to std::io::stdout() for successes or std::io::stderr() for errors ensures the application prints the result or error message to the terminal.

Test Your Application

You can build and test your application using cargo run. The double dash -- tells Cargo to pass the following flags directly to your fintrack binary rather than interpreting them as Cargo arguments:

cargo build
cargo run -- init --currency USD --opening 1000
cargo run -- add income 500 --subcategory salary
cargo run -- add expenses 50 --subcategory groceries
cargo run -- list
cargo run -- total

Install the Binary

Running with cargo run is useful during development, but you can install the binary directly to your system to use the fintrack command globally. It will use fintrack because that's the value in the name field in your Cargo.toml file, which you set in the first step when you ran cargo new fintrack.

Run this to install fintack as a command:

cargo install --path .

When you run the installation command, Cargo compiles your code in release mode and places the executable in your Cargo bin folder (typically ~/.cargo/bin). Once installed, the operating system recognizes fintrack as a standalone command. You can now call your application from any directory without prefixing it with cargo:

fintrack total

What's Next and Advanced Features

Congratulations! You've built a complete local-first CLI financial tracker. The application you've created includes:

• Data persistence in JSON format

• Full CRUD operations for financial records

• Subcategory management

• Financial calculations

• Comprehensive error handling

• Type-safe command-line argument parsing

Advanced Features to Explore

The modular architecture you built makes this tool easily extensible. To add new commands, you simply follow the pattern established in previous steps: create a new command module, define its logic, and register it within the cli() and build_exec functions in src/commands.rs.

Consider implementing these features to enhance the tool:

• Export: Add an export.rs module to convert your JSON data into CSV format for analysis in spreadsheet applications.

• Describe: Create a command that uses terminal plotting libraries to generate visual charts of your spending patterns.

• Enhanced Output Formatting: Update src/output.rs with libraries like colored or tabled to add colors and professional borders to your terminal summaries.

You can find the complete implementation of fintrack with all features, including advanced output formatting, export functionality, and more, in the GitHub repository. The repository also includes installation instructions for downloading the binary or installing via Cargo.

Conclusion

In this tutorial, you've learned how to:

• Structure a Rust CLI application with proper error handling

• Use traits to extend functionality

• Work with JSON serialization

• Parse and validate command-line arguments

• Manage file I/O operations

• Implement a complete data model with relationships

The patterns you've learned here apply to many Rust applications. Traits, error handling with Result, and the ownership system are fundamental to writing idiomatic Rust code. These techniques ensure that as the application grows, the code remains maintainable and safe.

The modular nature of this tracker also means that the source code is now a template for any local-first tool. By swapping out the financial models for other data types, this same architecture can power a task manager, a personal wiki, or a time tracker.

Keep building, and happy tracking!

In the past year, frontier AI labs such as OpenAI, xAI, Anthropic, Meta, and Google have all released real-time voice services. Yet voice apps also have the highest requirements for latency, privacy, and customization. It’s difficult to have a one-size-fits-all voice AI solution.

In this article, we’ll explore how to use open-source technologies to create voice AI agents that utilize your custom knowledge base, voice style, actions, fine-tuned AI models, and run on your own computer.

What We’ll Cover:

• Prerequisites

• What it Looks Like

• Two Voice AI Approaches

• The Voice AI Orchestrator

  • Configure an ASR

  • Run and configure a VAD

  • Configure an LLM

  • Configure a TTS

  • Configure MCP and actions

• Local AI With LlamaEdge

• Conclusion

Prerequisites

You’ll need to have and know a few things to most effectively follow along with this tutorial:

• Access to a Linux-like system. Mac or Windows WSL suffice too.

• Be comfortable with command line (CLI) tools.

• Be able to run server applications on the Linux system.

• Have/get free API keys from Groq and ElevenLabs.

• Optional: be able to compile and build Rust source code.

• Optional: have/get an EchoKit device or assemble your own.

What it Looks Like

The key software component we will cover is the echokit_server project. It is an open-source agent orchestrator for voice AI applications. That means it coordinates services such as LLMs, ASR, TTS, VAD, MCP, search, knowledge/vector databases, and others to generate intelligent voice responses from user prompts.

The EchoKit server provides a WebSocket interface that allows compatible clients to send and receive voice data to and from it. The echokit_box project provides an ESP32-based firmware that can act as a client to collect audio from the user and play TTS-generated voice from the EchoKit server. You can see a couple of demos here. You can assemble your own EchoKit device or purchase one.

Of course, you can also use a pure software client that conforms to the echokit_server WebSocket interface. The project publishes a JavaScript web page that you can run locally to connect to your own EchoKit server as a reference.

In the rest of the article, I will discuss how it’s implemented and how to deploy the system for your own voice AI applications.

Two Voice AI Approaches

When OpenAI released its “realtime voice” services in October 2024, the consensus was that voice AI required “end-to-end” AI models. Traditional LLMs take text as input and then respond in text. The voice end-to-end models take voice audio data as input and respond in voice audio data as well. The end-to-end models could reduce latency since the voice processing, understanding, and generation are done in a single step.

But an end-to-end model is very difficult to customize. For example, it’s impossible to add your own prompt and knowledge to the context for each LLM request, or to act on the LLM's thinking or tool-call responses, or to clone your own voice for the response.

The second approach is to use an “agent orchestration” service to tie together multiple AI models, using one model’s output as the input for the next model. This allows us to customize or select each model and manipulate or supplement the model input at every step.

• The VAD model is used to detect conversation turns in the user's speech. It determines when the user is finished speaking and is now expecting a response.

• The ASR/STT model turns user speech into text.

• The LLM model generates a text response, including MCP tool calls.

• The TTS model turns the response text into voice.

The issue with multi-model and multi-step orchestration is that it can be slow. A lot of optimizations are needed for this approach to work well. For example, a useful technique is to utilize streaming input and output wherever possible. This way, each model doesn’t have to wait for the complete response from the upstream model.

The EchoKit server is a stream-everything, highly efficient AI model orchestrator. It is entirely written in Rust for stability, safety, and speed.

The Voice AI Orchestrator

The EchoKit server project is an open-source AI service orchestrator focused on real-time voice use cases. It starts up a WebSocket server that listens for streaming audio input and returns streaming audio responses.

You can build the echokit_server project yourself using the Rust toolchain. Or, you can simply download the pre-built binary for your computer.

# for x86 / AMD64 CPUs
curl -LO https://github.com/second-state/echokit_server/releases/download/v0.1.0/echokit_server-v0.1.0-x86_64-unknown-linux-gnu.tar.gz
unzip echokit_server-v0.1.0-x86_64-unknown-linux-gnu.tar.gz

# for arm64 CPUs
curl -LO https://github.com/second-state/echokit_server/releases/download/v0.1.0/echokit_server-v0.1.0-aarch64-unknown-linux-gnu.tar.gz
unzip echokit_server-v0.1.0-aarch64-unknown-linux-gnu.tar.gz

Then, run it as follows:

nohup ./echokit_server &

It reads the config.toml file from the current directory. At the top of the file, you can configure the port on which the WebSocket server listens. You can also specify a WAV file that is downloaded to the connected EchoKit client device as a welcome message.

addr = "0.0.0.0:8000"
hello_wav = "hello.wav"

Configure an ASR

When the EchoKit server receives the user's voice data, it first sends the data to an ASR service to convert it into text.

There are many compelling ASR models available today. The EchoKit server can work with any OpenAI-compatible API providers, such as OpenAI itself, x.ai, OpenRouter, and Groq.

In our example, we use Groq’s Whisper ASR service. Whisper is a state-of-the-art ASR model released by OpenAI. Groq provides specialized hardware chips to run it very fast. You will first get a free API key from Groq. Then, configure the ASR service as follows. Notice the “prompt” for the Whisper model. It is a tried-and-true prompt to reduce hallucination of the Whisper model.

[asr]
url = "https://api.groq.com/openai/v1/audio/transcriptions"
api_key = "gsk_XYZ"
model = "whisper-large-v3"
lang = "en"
prompt = "Hello\n你好\n(noise)\n(bgm)\n(silence)\n"

Run and configure a VAD

In order to carry out a voice conversation, participants must detect each other's intentions and speak only when a turn arises. VAD (Voice Activity Detection) is a specialized AI model used to detect activities and, in particular, when the speaker has finished and expects an answer.

In EchoKit, we have VAD detection on both the device and the server.

• Device-side VAD: It detects human language. The device ignores background noise, music, keyboard sounds, and dog barking. It only sends human voice to the server.

• Server-side VAD: It processes the audio stream in 100ms (0.1s) chunks. Once it detects that the speaker has finished, it sends all transcribed text to the LLM and starts waiting for the LLM’s response stream.

The server-side VAD is optional, since the device-side VAD can also generate “conversation turn” signals. But due to the limited computing resources on the device, adding the server-side VAD can dramatically improve the overall VAD performance.

We’re porting the popular Silero VAD project from Python to Rust, and creating the silero_vad_server project. Build the project as instructed. You can start the VAD server on your EchoKit server’s port 9094 as follows:

VAD_LISTEN=0.0.0.0:9094 nohup target/release/silero_vad_server &

You might be wondering: why port to Rust? While many AI projects are written in Python for ease of development, Rust applications are often much lighter, faster, and safer at deployment. So, we’ll leverage AI tools like RustCoder to port as much Python code as possible to Rust. The EchoKit software stack is largely written in Rust.

The VAD server is a WebSocket service that listens on port 9094. As we discussed, the EchoKit server will stream audio to this WebSocket and stop the ASR when a conversation turn is detected. Therefore, we’ll add the VAD service to the EchoKit server’s ASR config section in config.toml.

[asr]
url = "https://api.groq.com/openai/v1/audio/transcriptions"
api_key = "gsk_XYZ"
model = "whisper-large-v3"
lang = "en"
prompt = "Hello\n你好\n(noise)\n(bgm)\n(silence)\n"
vad_realtime_url = "ws://localhost:9094/v1/audio/realtime_vad"

Configure an LLM

Once the ASR service transcribes the user's voice into text, the next step in the pipeline is the LLM (Large Language Model). It’s the AI service that actually “thinks” and generates an answer in text.

Again, the EchoKit server can work with any OpenAI-compatible API providers for LLMs, such as OpenAI itself, x.ai, OpenRouter, and Groq. Since the voice service is highly sensitive to speed, we’ll choose Groq again. Groq supports a number of open-source LLMs. We’ll choose the gpt-oss-20b model released by OpenAI.

[llm]
llm_chat_url = "https://api.groq.com/openai/v1/chat/completions"
api_key = "gsk_XYZ"
model = "openai/gpt-oss-20b"
history = 20

The “history” field indicates how many messages should be kept in the context. Another crucial feature of an LLM application is the “system prompt,” where you instruct the LLM how it should “behave.” You can specify the system prompt in the EchoKit server config as well.

[[llm.sys_prompts]]
role = "system"
content = """
You are a comedian. Engage in lighthearted and humorous conversation with the user. Tell jokes when appropriate.

"""

Since Groq is very fast, it can process very large system prompts in under one second. You can add a lot more context and instructions to the system prompt. For example, you can give the application “knowledge” about a specific field by putting entire books into the system prompt.

Configure a TTS

Finally, once the LLM generates a text response, the EchoKit server will call a TTS (text to speech) service to convert the text into voice and stream it back to the client device.

While Groq has a TTS service, it’s not particularly compelling. ElevenLabs is a leading TTS provider that offers hundreds of voice characters. It can express emotions and supports easy voice cloning. In the config below, you’ll put in your ElevenLabs API key and select a voice.

[tts]
platform = "Elevenlabs"
token = "sk_xyz"
voice = "VOICE-ID-ABCD"

The ElevenLabs TTS models and API services are all great, but they are not open-source. A very compelling open-source TTS, known as GPT-SoVITS, is also available.

You can port GPT-SoVITS from Python to Rust and create an open-source API server project called gsv_tts. It allows easy cloning of any voice. You can run a gsv_tts API server by following its instructions. Then, you can configure the EchoKit server to stream text to it and receive streaming audio from it.

[tts]
platform = "StreamGSV"
url = "http://gsv_tts.server:port/v1/audio/stream_speech"
speaker = "michael"

Configure MCP and actions

Of course, an “AI agent” is not just about chatting. It is about performing actions on specific tasks. For example, the “US civics test prep” use case, which I shared as an example video at the beginning of this article, requires the agent to get exam questions from a database, and then generate responses that guide the user toward the official answer. This is accomplished using LLM tools and actions.

• The LLM detects that the user is requesting a new question.

• Instead of responding in natural language, it responds with a JSON structure that instructs the agent to "get a new question and answer."

• The EchoKit server intercepts this JSON response and retrieves the question and answer from a database.

• The EchoKit server sends the question and answer back to the LLM.

• The LLM formulates a natural language response based on the question and answer.

• The EchoKit server generates a voice response using its TTS service.

As you can see, the EchoKit server needs to perform a few extra steps behind the scenes before it responds in voice. The EchoKit server leverages the MCP protocol for this. The function to look up questions and answers is provided by an open-source MCP server called ExamPrepAgent.

The MCP protocol standardizes the tools and functions for LLMs to call. There are many MCP servers available for all kinds of different tasks. ExamPrepAgent is just one of them.

We are running this MCP server on port 8003. With the MCP server up and running, you only need to add the following configuration to EchoKit server’s config.toml.

[[llm.mcp_server]]
server = "http://localhost:8003/mcp"
type = "http_streamable"

With MCP integration, the EchoKit AI agent can now perform actions. It can call APIs to send messages, make payments, or even turn electronic devices on or off.

Local AI With LlamaEdge

You’ve now seen the open-source EchoKit device working with the open-source EchoKit server to understand and respond to users in voice. But the AI models we use, while also open-source, run on commercial cloud providers. Can we run AI models using open-source technologies at home?

LlamaEdge is an open-source, cross-platform API server for AI models. It supports many mainstream LLM, ASR, and TTS models across Linux, Mac, Windows, and many CPU/GPU architectures. It’s perfect for running AI models on home or office computers. It also provides OpenAI-compatible API endpoints, which makes them very easy to integrate into the EchoKit server.

To install LlamaEdge and its dependencies, run the following shell command. It will detect your hardware and install the appropriate software that can fully take advantage of your GPUs (if any).

curl -sSf https://raw.githubusercontent.com/WasmEdge/WasmEdge/master/utils/install_v2.sh | bash -s

Then, download an open-source LLM model. I am using Google's Gemma model as an example.

curl -LO https://huggingface.co/second-state/gemma-3-4b-it-GGUF/resolve/main/gemma-3-4b-it-Q5_K_M.gguf

Download the cross-platform LlamaEdge API server.

curl -LO https://github.com/second-state/LlamaEdge/releases/latest/download/llama-api-server.wasm

Start an LLamaEdge API server with the Google Gemma LLM model. by default, it listens on localhost port 8080.

wasmedge --dir .:. --nn-preload default:GGML:AUTO:gemma-3-4b-it-Q5_K_M.gguf llama-api-server.wasm -p gemma-3

Test the OpenAI compatible API on that server.

curl -X POST http://localhost:8080/v1/chat/completions \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"system", "content": "You are a helpful assistant. Try to be as brief as possible."}, {"role":"user", "content": "Where is the capital of Texas?"}]}'

Now, you can add this local LLM service to your EchoKit server configuration.

[llm]
llm_chat_url = "http://localhost:8080/v1/chat/completions"
api_key = "NONE"
model = "default"
history = 20

The LlamaEdge project supports more than LLMs. It runs the Whisper ASR model and the Piper TTS model as OpenAI-compatible API servers as well.

Conclusion

The voice AI agent software stack is complex and deep. EchoKit is an open-source platform that ties together and coordinates all those components. It provides a good vantage point for us to learn about the entire stack.

I can’t wait to see what you build!

So what are regular pointers? Regular pointers (just called “pointers”) are variables that hold memory addresses as their values. They allow programs to store, read, and write data to memory locations with their addresses.

Here’s a diagram to give an idea of what they are:

In programming languages like C, C++, and Rust, pointers are useful for accessing manually allocated memory, but they come with these limitations:

• The memory address that a pointer holds can be deallocated while the pointer still references it, making it a dangling pointer.

• The pointer doesn’t help with managing the memory allocation, which can cause memory leaks or other types of memory bugs in cases where handling memory allocations are complex.

Rust doesn’t give the same level of control of pointers as with C and C++. However, like C++, Rust provides smart pointers that overcome the limitations of regular pointers while providing extra functionalities.

In Rust, there are four major types of smart pointers: Box, Rc, Arc, and Weak. I’ll be discussing them in this article. I’ll also touch a little on RefCell, because it adds a specific functionality that is missing in other smart pointers.

Table of Contents

• Box Pointers

• Rc and Arc Pointers

• Weak Pointers

• RefCell

• Summary

Box Pointers

Box is the most straightforward type of a smart pointer. It allows you to manually allocate memory in the heap.

#[allow(dead_code)]
#[derive(Debug)]
struct Point {
    x: f32,
    y: f32,
}

fn main() {
    let point = Box::new(Point { x: 0.0, y: 0.0 });
    println!("{:?}", point);
}

You can access the contents of a Box pointer like you would with a regular variable:

println!("{}", point.x); // -> output: 0.0
println!("{}", point.y); // -> output: 0.0

It works almost identically to malloc in C and new in c++, with the exception that Box automatically gets freed when it goes out of scope, or when the program execution ends, as opposed to manually freeing the allocation in malloc and new.

Rc and Arc Pointers

I’m putting Rc and Arc together because they’re very similar in what they do and how they work.

Rc and Arc are reference counted pointers that allow multiple ownership of a memory allocation. Similar to Box, they allocate memory in the heap, but what differentiates them from Box is that they also include a reference count.

Rc and Arc allows you to create multiple clones of a reference to a memory allocation. This allows you to move those references to multiple scopes, and in the case of Arc, multiple threads, without borrowing. For example:

use std::sync::Arc;
use std::thread;
use std::thread::JoinHandle;

struct GameState {
    user_name: String,
}

impl GameState {
    fn new() -> Self {
        GameState { user_name: "Chigozie".to_string() }
    }
}

fn main() {
    let mut threads: Vec<JoinHandle<()>> = vec![];
    let game_state = Arc::new( GameState::new() );

    let g1 = Arc::clone(&game_state); // first clone
    threads.push(thread::spawn(move || {
        let username = &g1.user_name;
        // ...
    }));

    let g2 = Arc::clone(&game_state); // second clone
    threads.push(thread::spawn(move || {
        let username = &g2.user_name;
        // ...
    }));

    let g3 = Arc::clone(&game_state); // third clone
    threads.push(thread::spawn(move || {
        let username = &g3.user_name;
        // ...
    }));

    let g4 = Arc::clone(&game_state); // fourth clone
    threads.push(thread::spawn(move || {
        let username = &g4.user_name;
        // ...
    }));

    let g5 = Arc::clone(&game_state); // fifth clone
    threads.push(thread::spawn(move || {
        let username = &g5.user_name;
        // ...
    }));

    for th in threads {
        th.join().unwrap();
    }
}

In this example, I created an instance of a game struct in an Arc data structure, spawned five threads, then created and passed five more Arc references of the game struct to the five spawned threads.

The difference between Rc and Arc pointers is that references in Arc pointers are counted atomically, while references in Rc pointers are counted using the usual mathematical operations. This means that the operations that go into counting the references in Arc pointers are guaranteed to not be interrupted or overlapped by other threads or processes, making them very useful for multi-threaded environments.

One useful application of Rc and Arc pointers is in reference-based data structures, like linked lists, where each node has its value and a reference to the next node. For example:

use std::rc::Rc;

#[derive(Debug)]
struct Node {
    value: i32,
    next: Option<Rc<Node>>,
}

fn main() {
    // A chain of nodes
    let node1 = Rc::new(Node { value: 1, next: None });
    let node2 = Rc::new(Node { value: 2, next: Some(Rc::clone(&node1)) });
    let node3 = Rc::new(Node { value: 3, next: Some(Rc::clone(&node2)) });

    // Multiple owners of node2
    let another_ref_to_node2 = Rc::clone(&node2);

    println!("Node 3: {:?}", node3);
    println!("Another reference to Node 2: {:?}", another_ref_to_node2);
}

The memory allocations pointed to by Rc and Arc references are dropped when their reference counts goes to 0. The reference count of Rc and Arc pointers goes to 0 when it and all its clones have gone out of scope, or have been dropped manually.

Weak Pointers

Unlike Arc or Rc pointers, Weak pointers are non-owning references to memory allocations. This means that they don’t count towards ownership of the memory allocation and don’t stop memory allocations from being dropped.

Weak references are helpful in scenarios where you might prefer a reference to a memory allocation to prevent it from being deallocated. A good example of a scenario like this is a doubly linked list, where each node holds a reference to the next node and the previous node:

A scenario like this using Rc or Arc for both the next and previous nodes can cause reference cycles. Reference cycles prevent nodes from being deallocated because for one node to be deallocated all Arc or Rc references to it must be 0. Since the nodes in this case hold references to other nodes that also hold references back to it, both nodes can’t be deallocated automatically and they can end up stopping all other nodes in the data structure from being deallocated, causing a memory leak.

To prevent reference cycles while allowing nodes to both reference previous and next nodes, you can make each node’s reference to its previous node a Weak reference. For example:

use std::rc::{Rc, Weak};
use std::cell::RefCell;

#[derive(Debug)]
struct Node {
    value: i32,
    next: Option<Rc<RefCell<Node>>>,
    prev: Option<Weak<RefCell<Node>>>, // Weak reference to avoid cycles
}

fn main() {
    let node1 = Rc::new(RefCell::new(Node { value: 1, next: None, prev: None }));
    let node2 = Rc::new(RefCell::new(Node { value: 2, next: None, prev: Some(Rc::downgrade(&node1)) }));

    // Set node1's next to node2
    node1.borrow_mut().next = Some(Rc::clone(&node2));

    println!("Node 1: {:?}", node1);
    println!("Node 2: {:?}", node2);
}

However, since Weak references have non-owning references to memory allocations, they need to be upgraded to Rc or Arc references with .upgrade() to allow access to the memory allocation they point to.

Also, as you can see in code example below (as well as above on line 13), Rc and Arc references can be downgraded to Weak references with Rc::downgrade() or Arc::downgrade():

use std::rc::{Rc, Weak};

fn main() {
    let strong = Rc::new(5);
    let weak = Rc::downgrade(&strong);

    // Drop the weak reference
    drop(weak);

    // try to upgrade the weak reference
    if let Some(shared) = weak.upgrade() {
        println!("Data is still alive: {}", shared);
    } else {
        println!("Data has been dropped");
    }
}

Running this results in the following output:

Data has been dropped

This shows that only having weak references to a memory allocation doesn’t prevent it from being dropped. If a Weak pointer’s memory allocation is dropped, calling .upgrade() on the Weak pointer would return None.

RefCell

To ensure memory safety, Rust doesn’t allow you to mutate the data that smart pointers point to. This can prevent hidden mutations, but can become really inconvenient when you need to build something that is dynamically changing (for example the ability to add a new node to anywhere in a linked-list data structure).

RefCell allows you to overcome this limitation because it is a data structure that allows interior mutability of immutable variables by enforcing Rust’s borrowing rules at runtime.

You may have noticed it’s usage in the Weak pointer example earlier:

use std::rc::{Rc, Weak};
use std::cell::RefCell;

#[derive(Debug)]
struct Node {
    value: i32,
    next: Option<Rc<RefCell<Node>>>,
    prev: Option<Weak<RefCell<Node>>>,
}

fn main() {
    let node1 = Rc::new(RefCell::new(Node { value: 1, next: None, prev: None }));
    let node2 = Rc::new(RefCell::new(Node { value: 2, next: None, prev: Some(Rc::downgrade(&node1)) }));

    // Set node1's next to node2
    node1.borrow_mut().next = Some(Rc::clone(&node2));

    println!("Node 1: {:?}", node1);
    println!("Node 2: {:?}", node2);
}

You can call .borrow() and .borrow_mut() on a RefCell type to borrow references to its internal value at runtime, while keeping its own type as immutable making it useful in cases like this that require immutability.

Mutable and immutable borrows in a RefCell type work just like regular borrows that are checked at compile time, but they allow you to bypass compile time restrictions to be checked instead at runtime.

One major borrowing rule to look out for is the “single mutable ownership and multiple immutable ownership” rule. Borrowing two mutable references to a RefCell would result in a panic, crashing the application. For example:

#![allow(unused_variables)]
#![allow(dead_code)]
#![allow(unused_mut)]
use std::cell::RefCell;

fn main() {
    let counter = RefCell::new(100);
    let mut c1 = counter.borrow_mut();
    let mut c2 = counter.borrow_mut();

    println!("I'm done");
}

/**
 * output:
 *  thread 'main' panicked at src/main.rs:9:26:
 *  already borrowed: BorrowMutError
 *  note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
 */

Summary

To give an overview of the points made in this article, there are four common types of smart pointers in Rust:

• Box is used for manually allocating memory in the heap (similar to malloc and new in C and C++ respectively)

• Rc and Arc are used for allowing multiple ownership of a memory allocation. Arc is best for multi-threaded environments, and Rc is best for single-threaded environments.

• Weak is best used in giving multiple ownership of a memory allocation while preventing reference cycles.

• RefCell allows mutability in scenarios that require immutability, for example, in smart pointers.

I hope this article has provided clarity on smart pointers in Rust and how they work. Thanks for reading!

Even though they are important to Rust projects, lifetimes can be quite tricky to wrap your head around. So I created this guide to provide more clarity on what they are and when you should use them.

Prerequisites for this Tutorial

To get the most out of this tutorial, you'll need the following:

• At least beginner-level familiarity with Rust: This tutorial doesn't help with learning how to code in Rust. It only helps with understanding lifetimes in Rust and how they work

• Familiarity with generics: Generics in Rust work identically to how they do in popular programming languages. Knowledge of how generics work in any language would be helpful.

• Knowing how the borrow checker works isn't as much a requirement as the last two above, but it would be helpful. Knowledge of how lifetimes work also helps in understanding how the borrow checker works.

So, What are Lifetimes in Rust?

For Rust's borrow checker to ensure safety throughout your code, it needs to know how long all the data in the program will live during its execution. This becomes difficult to do in certain situations, and those situations are where you need to use explicit lifetime annotations.

Lifetimes in Rust are mechanisms for ensuring that all borrows that occur within your code are valid. A variable's lifetime is how long it lives within the program's execution, starting from when it's initialized and ending when it's destroyed in the program.

The borrow checker can detect the lifetimes of variables in many cases. But in cases where it can't, you have to assist it with explicit lifetime annotations.

The syntax for explicit lifetime annotations is a single quote followed by a set of characters for identification (for example, 'static, 'a) as in:

max<'a>

The lifetime annotation indicates that max should live at most as long as 'a.

Using multiple lifetimes follows the same syntax:

max<'a, 'b>

In this case, the lifetime annotations indicate that max should live at most as long as 'a and 'b.

Explicit lifetime annotations are handled similarly to how generics are. Let's take a look at an example:

fn max<'a>(s1: &'a str, s2: &'a str) -> &'a str {
    // return the longest string out of the two
}

In the example, the lifetime annotations indicate that max should live at most as long as the lifetimes of s1 or s2. It also indicates that max returns a reference that lives as long as s1.

A Rust project has many cases that would require explicit lifetime annotations, and in the next few sections, we'll go over each of them.

Lifetime Annotations in Functions

A function only needs an explicit lifetime annotation when it returns a reference from any of its arguments. Let's take an example:

fn max<'a>(s1: &'a str, s2: &'a str) -> &'a str {
    if s1.len() > s2.len() {
        s1
    } else {
        s2
    }
}

If you remove the lifetime annotations, you'll get an LSP (Language Server Protocol) warning to include the lifetime annotations. If you ignore LSP's warning message and compile the code, you'll get the same message as a compiler error. For example:

fn max(s1: &str, s2: &str) -> &str {
    if s1.len() > s2.len() {
        s1
    } else {
        s2
    }
}

/**
 * Output ->
 *
error[E0106]: missing lifetime specifier
  --> src/main.rs:44:31
   |
44 | fn max(s1: &str, s2: &str) -> &str {
   |            ----      ----     ^ expected named lifetime parameter
   |
   = help: this function's return type contains a borrowed value, but the signature does not say whether it is borrowed from `s1` or `s2`
help: consider introducing a named lifetime parameter
   |
44 | fn max<'a>(s1: &'a str, s2: &'a str) -> &'a str {
   |       ++++      ++           ++          ++

For more information about this error, try `rustc --explain E0106`.
error: could not compile `lifetime-test` (bin "lifetime-test") due to 1 previous error
 ***********************
 */

On the other hand, a function doesn't need explicit lifetimes if it isn't returning a reference in its arguments. For example:

fn print_longest(s1: &str, s2: &str) {
    if s1.len() > s2.len() {
        println!("{s1} is longer than {s2}")
    } else {
        println!("{s2} is longer than {s1}")
    }
}

A function returning a different value doesn't need explicit lifetime annotations either:

fn join_strs(s1: &str, s2: &str) -> String {
    let mut joint_string = String::from(s1);
    joint_string.push_str(s2);
    return joint_string;
}

You only need to specify lifetimes if a function returns a reference from one of its arguments that is a borrowed reference.

Lifetime Annotations in Structs

Structs require explicit lifetime annotations when any of their fields are references. This allows the borrow checker to ensure that the references in the struct's fields live longer than the struct. For example:

struct Strs<'a, 'b> {
    x: &'a str,
    y: &'b str,
}

Without the lifetime annotations, you'll get a similar LSP and compiler error message to the one in the previous section:

struct OtherStruct {
    x: &str,
    y: &str,
}

/**
* Output ->
**********************
error[E0106]: missing lifetime specifier
 --> src/main.rs:7:8
  |
7 |     x: &str,
  |        ^ expected named lifetime parameter
  |
help: consider introducing a named lifetime parameter
  |
6 ~ struct OtherStruct<'a> {
7 ~     x: &'a str,
  |

error[E0106]: missing lifetime specifier
 --> src/main.rs:8:8
  |
8 |     y: &str,
  |        ^ expected named lifetime parameter
  |
help: consider introducing a named lifetime parameter
  |
6 ~ struct OtherStruct<'a> {
7 |     x: &str,
8 ~     y: &'a str,
  |

For more information about this error, try `rustc --explain E0106`.
error: could not compile `lifetime-test` (bin "lifetime-test") due to 2 previous errors
**********************
*/

Lifetime Annotations in Methods

Lifetime annotations concerning methods can be done as annotations to standalone methods, impl blocks, or traits. Let's look at each of them:

Standalone Methods:

Annotating lifetimes on standalone methods is identical to annotating lifetimes in functions:

impl Struct {
    fn max<'a>(self: &Self, s1: &'a str, s2: &'a str) -> &'a str {
        if s1.len() > s2.len() {
            s1
        } else {
            s2
        }
    }
}

impl Blocks

Writing explicit lifetime annotations for impl blocks is required if the struct it is associated with has lifetime annotations in its definition. This is the syntax for writing impl blocks with explicit lifetime annotations:

struct Struct<'a> {
}

impl<'a> Struct<'a> {
}

This allows any method you write in the impl block to return a reference from Struct. For example:

struct Strs<'a> {
    x: &'a str,
    y: &'a str,
}

impl<'a> Strs<'a> {
    fn max(self: &Self) -> &'a str {
        if self.y.len() > self.x.len() {
            self.y
        } else {
            self.x
        }
    }
}

Traits

Lifetime annotations in traits are dependent on the methods that the trait defines.

Let's look at one example. A method inside a trait definition can use explicit lifetime annotations as a standalone method, and the trait definition won't require explicit lifetime annotations. Like so:

trait Max {
    fn longest_str<'a>(s1: &'a str, s2: &'a str) -> &'a str;
}

impl<'a> Max for Struct<'a> {
    fn longest_str(s1: &'a str, s2: &'a str) {
        if s1.len() > s2.len() {
            s1
        } else {
            s2
        }
    }
}

If a trait method requires references from the struct its associated with, the trait's definition would require explicit lifetime annotations. For example:

trait Max<'a> {
    fn max(self: &Self) -> &'a str;
}

Which can be implemented this way:

struct Strs<'a> {
    x: &'a str,
    y: &'a str,
}

trait Max<'a> {
    fn max(self: &Self) -> &'a str;
}

impl<'a> Max<'a> for Strs<'a> {
    fn max(self: &Self) -> &'a str {
        if self.y.len() > self.x.len() {
            self.y
        } else {
            self.x
        }
    }
}

Lifetime Annotations in Enums

Similar to structs, enums need explicit lifetime annotations if any of their fields are references. For example:

enum Either<'a> {
    Str(String),
    Ref(&'a String),
}

The 'static Lifetime

In many Rust projects, you'll likely have encountered variables that are 'static in lifetimes. In this section, we'll go over a brief overview of what a 'static lifetime is, how it works, and where it is commonly used.

'static is a reserved lifetime name in Rust. It signifies that the data that a reference points to lives from where it is initialized to the end of the program. This differs slightly from static variables, which are stored directly in the program's binary file. However, all static variables have a 'static lifetime.

Variables with 'static lifetimes can be created at runtime. But they can't be dropped, only coerced into shorter lifetimes. For example:

// The lifetime annotation 'a is the shorter lifetime of the
// two arguments s1 and s2
fn max<'a>(s1: &'a str, s2: &'a str) -> &'a str {
    if s1.len() > s2.len() {
        s1
    } else {
        s2
    }
}

fn main() {
    let first = "First string"; // Longer lifetime

    {
        let second = "Second string"; // Shorter lifetime

        // In the max function, the lifetime of first is
        // coerced into the lifetime of second
        println!("The biggest of {} and {} is {}", first, second, max(first, second));
    };
}

String literals are examples of values with 'static lifetimes. They are also stored in the program's binary file and can be created at runtime.

Rust allows you to declare static variables with the static keyword, using this syntax:

static IDENTIFIER: &'static str = "value";

Static variables can be declared in any scope, including the global scope. This means that you can use static variables as global variables. For example:

static FIRST_NAME: &'static str = "John";
static LAST_NAME: &'static str = "Doe";

fn main() {
    println!("First name: {}", FIRST_NAME);
    println!("Last name: {}", LAST_NAME);
}

Static variables can also be mutable or immutable. But working with mutable static variables is only allowed in unsafe blocks because they're unsafe.

static mut FIRST_NAME: &'static str = "John";
static LAST_NAME: &'static str = "Doe";

fn main() {
    unsafe {
        println!("First name: {}", FIRST_NAME);
    }
    println!("Last name: {}", LAST_NAME);
    unsafe {
        FIRST_NAME = "Jane";
        println!("First name changed to: {}", FIRST_NAME);
    }
}

Summary

Lifetimes in Rust help the borrow checker ensure that all borrowed references are valid. The borrow checker can detect the lifetimes of variables in many cases, but in cases where it can't you have to assist it with explicit lifetime annotations.

Explicit lifetime annotations are those 'a, 'b, and 'static things you see in many Rust projects. You only need to use them in structures (structs, enums, traits, and impls) that deal with references, and in functions or methods that receive and return references.

In this guide, you learned about explicit lifetime annotations and saw some examples of how to use them. I it gave you some clarity on the topic, and helped you understand lifetimes better.

Thanks for reading!

Developers rely on tools like compilers and bundlers to transform their code, optimize performance, and ensure compatibility across different environments.

These tools are essential for modern JavaScript applications, enabling developers to write clean, maintainable code while leveraging the latest language features.

This article will help you understand what Speedy Web Compiler (SWC) is and how it helps optimize the performance of your web apps.

Table of Contents

• Introduction to Speedy Web Compiler

• Background of SWC

• How Speedy Web Compiler Works

• Benefits of Using Speedy Web Compiler

• How to Set Up Speedy Web Compiler in Your Project

• SWC Integration in Popular Frameworks

• Conclusion

What is Speedy Web Compiler?

First, let me break it down.

SWC stands for Speedy Web Compiler, and when broken down:

Speedy - This means it's fast! SWC processes and transforms JavaScript code, making it efficient to use in big projects.

Web - It’s all about web development. It focuses on improving how JavaScript (the language of the web) handles it.

Compiler - It takes code written in one form and transforms it into another form that can be better understood or used by computers.

Background of SWC

kdy1, a South Korean developer and maintainer of Next.js, created SWC as a faster tool for handling JavaScript code.

The motivation was the need for speed and efficiency, as web projects grow larger and more complex.

With numerous websites and apps depending on JavaScript, SWC helps developers save time and work more efficiently.

How Speedy Web Compiler Works

SWC uses Rust, a programming language known for its speed and safety.

SWC works by taking your JavaScript or TypeScript code and transforming it into a version that can run efficiently in various environments.

Understanding how SWC achieves this involves examining its core steps: parsing, transforming, and generating code.

How Does SWC Parse Code?

The first step in the compilation process is parsing.

Begins by reading and analyzing the code to understand its structure.

This is akin to taking a complex sentence and breaking it down into its grammatical components—subject, verb, object, etc.

In technical terms, SWC converts your code into an Abstract Syntax Tree (AST).

The AST is a tree-like representation of the source code, where each node in the tree corresponds to a construct occurring in the code, such as expressions, statements, and functions.

This tree structure allows SWC to process and understand the code’s logic in a way that is both efficient and scalable.

How Does SWC Transform Code?

After creating the AST, SWC moves on to the transformation phase.

This is where the magic happens—SWC applies various optimizations and changes to the code based on the target environment.

For instance, if you're targeting older browsers that don't support modern JavaScript features, SWC will transform your ES6+ code into a backward-compatible version.

During this phase, SWC also handles TypeScript transformations. It strips away TypeScript-specific syntax, such as types and interfaces, converting the code into pure JavaScript that gets executed by any JavaScript engine.

SWC can apply custom transformations based on plugins or specific configurations, making it highly versatile.

How Does SWC Generate Optimized Code?

After the transformations are complete, SWC proceeds to the final step: code generation.

In this step, SWC takes the transformed AST and converts it back into executable code.

In contrast, this process is not just a reversal of the parsing process.

SWC takes special care to generate code that is both functionally correct and optimized for performance.

For instance, SWC might remove dead code (code that will never be executed) or inline certain functions to reduce overhead.

The goal is to produce code that is as clean and efficient as possible, ensuring that it runs quickly and reliably in production environments.

Benefits of Using Speedy Web Compiler

Performance

One of the major benefits of using SWC is its outstanding speed. SWC achieves this exceptional speed by using Rust.

Developers can expect a significant performance improvement when compiling their code for large projects or codebases.

This speed greatly reduces build times, enhancing the efficiency and responsiveness of the development process. It is so fast, you might think it’s late for a meeting!

Optimized Output

SWC compiles your code and guarantees that the output is highly optimized for performance in production environment by removing dead code, in-lining functions, and reducing the size of the output.

This makes using SWC cost-effective, saving you from extra expenses during production.

The result is a leaner, faster, and more efficient code that can enhance loading times and performance in web applications.

Compatibility

SWC is fully compatible with modern JavaScript libraries and frameworks.

You do not have to worry about using ES6+ or TypeScript. This makes SWC a versatile choice for your projects.

How to Set Up Speedy Web Compiler

Installation

To install SWC in your JavaScript or TypeScript project, follow these steps:

1. Initialize your project: If you haven't already, start by initializing a new project. In your terminal, run:

npm init -y

2. Install SWC with npm: Run the following command to download the pre-built binaries:

npm install -D @swc/cli @swc/core

3. Create a JavaScript file: Create a src/index.js file with some code:

const greet = (name) => {
  return `Hello, ${name}!`;
};

console.log(greet("World"));

4. Compile with SWC: Run SWC from the command line to compile your JavaScript file:

npx swc src/index.js -o dist/index.js

5. Resulting Code: The resulting JavaScript code in dist/index.js will look like this:

"use strict";
var greet = function(name) {
    return "Hello, " + name + "!";
};
console.log(greet("World"));

This is the transpiled ES5 code produced by SWC, suitable for environments that require backward compatibility with older JavaScript versions.

SWC Integration in Popular Frameworks

If you are using Next.js, Deno, Vite, Remix, Parcel, or Turbopack, SWC is already integrated.

Notable improvements were made on Next.js, a popular React framework, since version 12 (Source: Next.js 12: The SDK for the Web)

Conclusion

In the constantly changing realm of JavaScript development, having the correct tools can make a significant difference.

SWC, the Speedy Web Compiler, distinguishes itself as a strong solution for converting and optimizing JavaScript and TypeScript code.

Its impressive speed, owing to its Rust-based implementation, along with its efficient management of code transformations and optimizations, positions it as a powerful tool for modern web development.

If you would like to stay in touch:

Connect with me on LinkedIn

Follow me on X

To help your intuition, it's best to imagine an image as a mathematical graph of pixel values plotted along the x and y coordinates. The top right pixel in an image is your origin, which corresponds to an x value of 0 and a y value of 0.

Once you imagine this, any pixel in an image can be read or modified using it's coordinate in this x-y graph. For example, for a square image of size 5px x 5px, the coordinate of the center pixel is 2, 2. You may have expected it to be 3, 3, but image coordinates in this context work similar to array indexes and start from 0 for both axis.

Approaching image processing this way also helps you address each pixel individually, making the process much simpler.

Prerequisites

The focus of this article is for you to understand and learn how to blend images using the Rust programming language, without going into the details of the language or it's syntax. So being comfortable writing Rust programs is required.

If you're not familiar with Rust, I highly encourage you to learn the basics. Here's an interactive Rust course that can get you started.

Table Of Contents

1. Introduction

2. How Image Blending Works

3. Project Setup

4. How to Read Pixel Values

5. How to Blend Functions

   1. Average Blend

   2. Multiply Blend

   3. Lighten Blend

   4. Darken Blend

   5. Screen Blend

   6. Addition Blend

   7. Subtraction Blend

6. How to Apply Blend Functions To Images

7. Putting It All Together

8. Glossary

Introduction

Image blending refers to the technique of merging pixels from multiple images to create a single output image that is derived from all of its inputs. Depending on which blending operation is used, the image output can vary widely given the same inputs.

This technique serves as the basis for many complex image processing tools, some of which you may already be familiar with. Things such as removing moving people from images if you have multiple images, merging images of the night sky to create star trails, and merging multiple noise-heavy images to create a noise reduced image are all examples of this technique at play.

To achieve the blending of images in this tutorial, we will make use of "pixel math", which while not being a truly standard term, refers to the technique of performing mathematical operations on a pixel or set of pixels to generate an output pixel.

For example, to blend two images using the "average" blend mode, you will perform the mathematical average operation on all input pixels at a given location, to generate the output at the same location.

Pixel math is not limited to point operations, which are basically operations performed during image processing that generate a given output pixel based on input pixel from single or multiple images from the same location in the x-y coordinate system.

In my experience so far, the entirety of image processing field is 99% mathematics and 1% black magic. Mathematical operations on pixels and it's surrounding pixels is the basis of image manipulation techniques such as compression, resizing, blurring and sharpening, noise reduction, and so on.

How Image Blending Works

The technique is technically simple to implement. Let's take the example of a simple average blend. Here's how it works:

1. Read the pixel data of both images into memory, usually into an array for each image.

   • The array is usually 2 dimensional. Each entry in array is another array for color images, the secondary array holds the 3 pixel values corresponding to Red, Green, and Blue color channels.

2. For each pixel location:

   1. For each channel:
      a. Take the value of the channel from the 2nd image, let's consider it y.
      b. Perform the averaging operation x/2 + y/2.
      c. Save the output value of this operation as the value of the output channel

   2. Save the result of previous operation as the value of the output pixel.

3. Construct the output image with the same dimensions from the computed data.

You'll notice that pixel math is performed on a per-channel basis. This is always true for the blend modes we cover in this tutorial, but many techniques involve applying blends between the channels themselves and many times within the same image.

Project Setup

Let's get started by setting up a project that gives us a good baseline to work with.

cargo new --bin image-blender
cd image-blender

You will also need a single dependency to help you perform these operations:

cargo add image

image is a Rust library we'll use to work with images of all of the standard formats and encodings. It also helps us convert between various formats, and provides easy access to pixel data as buffers.

For more information on the image crate, you can refer to the official documentation.

To follow along, you can use any two images, the only requirement being that they should be of the same size and in the same format. You can also find the images used in this tutorial, along with complete code, in the GitHub repository here.

How to Read Pixel Values

The first step is to load the images and read their pixel values into a data structure that facilitates our operation. For this tutorial, we're going to use a Vec of arrays (Vec<[u8; 3]>). Each entry in the outer Vec represents a pixel, and the channel-wise values of each pixel are stored in [u8; 3] array.

Let's start by creating a new file to hold this code called io.rs.

// src/io.rs

use image::GenericImageView;

pub struct SourceData {
    pub width: usize,
    pub height: usize,
    pub image1: Vec<[u8; 3]>,
    pub image2: Vec<[u8; 3]>,
}

pub fn read_pixel_data(image1_path: String, image2_path: String) -> SourceData {
    // Open the images
    let image1 = image::open(image1_path).unwrap();
    let image2 = image::open(image2_path).unwrap();

    // Compute image dimensions
    let (width, height) = image1.dimensions();
    let (width, height) = (width as usize, height as usize);

    // Create arrays to hold input pixel data
    let mut image1_data: Vec<[u8; 3]> = vec![[0, 0, 0]; width * height];
    let mut image2_data: Vec<[u8; 3]> = vec![[0, 0, 0]; width * height];

    // Iterate over all pixels in the input image, along with their positions in x & y
    // coordinates.
    for (x, y, pixel) in image1.to_rgb8().enumerate_pixels() {
        // Compute the raw values for each channel in the RGB pixel.
        let [r, g, b] = pixel.0;

        // Compute linear index based on 2D index. This is basically computing index in
        // 1D array based on the row and column index of the pixel in the 2D image.
        let index = (y * (width as u32) + x) as usize;

        // Save the channel-wise values in the correct index in data arrays.
        image1_data[index] = [r, g, b];
    }

    // Iterate over all pixels in the input image, along with their positions in x & y
    // coordinates.
    for (x, y, pixel) in image2.to_rgb8().enumerate_pixels() {
        // Compute the raw values for each channel in the RGB pixel.
        let [r, g, b] = pixel.0;

        // Compute linear index based on 2D index. This is basically computing index in
        // 1D array based on the row and column index of the pixel in the 2D image.
        let index = (y * (width as u32) + x) as usize;

        // Save the channel-wise values in the correct index in data arrays.
        image2_data[index] = [r, g, b];
    }

    SourceData {
        width,
        height,
        image1: image1_data,
        image2: image2_data,
    }
}

How to Blend Functions

The next step is to implement the blending functions, which are pure functions that take two pixel values as input and return the output value. This is implemented through the BlendOperation trait defined below. Let's create a new file to host all the operations called operations.rs.

// src/operations.rs

pub trait BlendOperation {
    fn perform_operation(&self, pixel1: [u8; 3], pixel2: [u8; 3]) -> [u8; 3];
}

Next, we need to implement this trait for all of the blending methods we want to support.

For showcasing the result of each of the blending modes, the following two input images are blended together

Average Blend

An average blend involves channel-wise averaging the input pixel values to get the output pixel.

// src/operations.rs

pub struct AverageBlend;

impl BlendOperation for AverageBlend {
    fn perform_operation(&self, pixel1: [u8; 3], pixel2: [u8; 3]) -> [u8; 3] {
        [
            pixel1[0] / 2 + pixel2[0] / 2,
            pixel1[1] / 2 + pixel2[1] / 2,
            pixel1[2] / 2 + pixel2[2] / 2,
        ]
    }
}

Multiply Blend

A multiply blend involves channel-wise multiplication of input pixel values after they've been normalized[¹] to get the output pixel. The output pixel is then rescaled back to the original range by multiplying with 255.

// src/operations.rs

pub struct MultiplyBlend;

impl BlendOperation for MultiplyBlend {
    fn perform_operation(&self, pixel1: [u8; 3], pixel2: [u8; 3]) -> [u8; 3] {
        [
            ((pixel1[0] as f32 / 255. * pixel2[0] as f32 / 255.) * 255.) as u8,
            ((pixel1[1] as f32 / 255. * pixel2[1] as f32 / 255.) * 255.) as u8,
            ((pixel1[2] as f32 / 255. * pixel2[2] as f32 / 255.) * 255.) as u8,
        ]
    }
}

Lighten Blend

Lighten blend involves channel-wise comparison of input pixel values, selecting the pixel with higher value (intensity) as the output pixel.

// src/operations.rs

pub struct LightenBlend;

impl BlendOperation for LightenBlend {
    fn perform_operation(&self, pixel1: [u8; 3], pixel2: [u8; 3]) -> [u8; 3] {
        [
            pixel1[0].max(pixel2[0]),
            pixel1[1].max(pixel2[1]),
            pixel1[2].max(pixel2[2]),
        ]
    }
}

Darken Blend

Darken blend is the opposite operation of lighten blend. It involves channel-wise comparison of input pixel values, selecting the pixel with least value (intensity) as the output pixel.

// src/operations.rs

pub struct DarkenBlend;

impl BlendOperation for DarkenBlend {
    fn perform_operation(&self, pixel1: [u8; 3], pixel2: [u8; 3]) -> [u8; 3] {
        [
            pixel1[0].min(pixel2[0]),
            pixel1[1].min(pixel2[1]),
            pixel1[2].min(pixel2[2]),
        ]
    }
}

Screen Blend

Screen blend refers to multiplying the inverse of two images, and then inverting the result. In our implementation, the pixels first need to be normalized[¹]. The normalized[¹] values are then inverted by subtracting them from 1, then they're multiplied and inverted again.

Finally, the output is multiplied by 255 to de-normalize the output pixel value.

// src/operations.rs

pub struct ScreenBlend;

impl BlendOperation for ScreenBlend {
    fn perform_operation(&self, pixel1: [u8; 3], pixel2: [u8; 3]) -> [u8; 3] {
        [
            ((1. - ((1. - (pixel1[0] as f32 / 255.)) * (1. - (pixel2[0] as f32 / 255.)))) * u8::MAX as f32) as u8,
            ((1. - ((1. - (pixel1[1] as f32 / 255.)) * (1. - (pixel2[1] as f32 / 255.)))) * u8::MAX as f32) as u8,
            ((1. - ((1. - (pixel1[2] as f32 / 255.)) * (1. - (pixel2[2] as f32 / 255.)))) * u8::MAX as f32) as u8,
        ]
    }
}

Addition Blend

Addition blend involves adding the input values and then clamping the result to the maximum range of the color depth we're targeting. In this case, that would be 0-255 as we're targeting 8-bit color depth.

We also have to convert the values to u16 in order to avoid loss of value due to overflow. We can also use normalized[¹] values here to achieve the same result.

// src/operations.rs

pub struct AdditionBlend;

impl BlendOperation for AdditionBlend {
    fn perform_operation(&self, pixel1: [u8; 3], pixel2: [u8; 3]) -> [u8; 3] {
        [
            (pixel1[0] as u16 + pixel2[0] as u16).clamp(0, u8::MAX as u16) as u8,
            (pixel1[1] as u16 + pixel2[1] as u16).clamp(0, u8::MAX as u16) as u8,
            (pixel1[2] as u16 + pixel2[2] as u16).clamp(0, u8::MAX as u16) as u8,
        ]
    }
}

Subtraction Blend

Addition blend involves subtracting the input values and then clamping the result to the maximum range of the color depth we're targeting. In this case, that would be 0-255 as we're targeting 8-bit color depth.

We also convert the values to i16 in order to avoid loss of value due to overflow and lack of sign. We can also use normalized[¹] values here to achieve the same result.

// src/operations.rs

pub struct SubtractionBlend;

impl BlendOperation for SubtractionBlend {
    fn perform_operation(&self, pixel1: [u8; 3], pixel2: [u8; 3]) -> [u8; 3] {
        [
            (pixel1[0] as i16 - pixel2[0] as i16).clamp(0, u8::MAX as i16) as u8,
            (pixel1[1] as i16 - pixel2[1] as i16).clamp(0, u8::MAX as i16) as u8,
            (pixel1[2] as i16 - pixel2[2] as i16).clamp(0, u8::MAX as i16) as u8,
        ]
    }
}

How to Apply Blend Functions To Images

The final step is to actually use the blending operations we created previously and apply them to pairs of images.

To achieve this, we need a function that can take the SourceData type we defined previously as input, along with a blending operation as the arguments, and gives us the final output buffer. Let's start by creating a new file for it called blend.rs.

// src/blend.rs

use image::{ImageBuffer, Rgb};
use crate::{operations::BlendOperation, SourceData};

impl SourceData {
    pub fn blend_images(&self, operation: impl BlendOperation)  -> ImageBuffer<Rgb<u8>, Vec<u8>> {
        let SourceData {
            width,
            height,
            image1,
            image2,
        } = self;

        // Create a new buffer that has the same size as input images, which will serve as our output data
        let mut buffer = ImageBuffer::new(*width as u32, *height as u32);

        // Iterate over all pixels in the output buffer, along with their coordinates
        for (x, y, output_pixel) in buffer.enumerate_pixels_mut() {
            // Compute linear index form x & y coordinates. In other words, you have the
            // row and column indexes here, and you want to compute the array index based
            // on these two positions.
            let index = (y * *width as u32 + x) as usize;

            // Store pixel values in the given position into variables
            let pixel1 = image1[index];
            let pixel2 = image2[index];

            // Compute the blended pixel and convert it into the `Rgb` type, which is then
            // assigned to the output pixel in the buffer.
            *output_pixel = Rgb::from(operation.perform_operation(pixel1, pixel2));
        }

        buffer
    }
}

Putting It All Together

It's now time to make use of all the new things you've learnt so far, and put them together in main.rs file.

// src/main.rs

mod blend;
mod io;
mod operations;

use io::*;
use operations::{
    AdditionBlend, AverageBlend, DarkenBlend, LightenBlend, MultiplyBlend, ScreenBlend,
    SubtractionBlend,
};

fn main() {
    let source_data = read_pixel_data("image1.jpg".to_string(), "image2.jpg".to_string());

    let output_buffer = source_data.blend_images(AdditionBlend);
    output_buffer.save("addition.jpg").unwrap();

    let output_buffer = source_data.blend_images(AverageBlend);
    output_buffer.save("average.jpg").unwrap();

    let output_buffer = source_data.blend_images(DarkenBlend);
    output_buffer.save("darken.jpg").unwrap();

    let output_buffer = source_data.blend_images(LightenBlend);
    output_buffer.save("lighten.jpg").unwrap();

    let output_buffer = source_data.blend_images(MultiplyBlend);
    output_buffer.save("multiply.jpg").unwrap();

    let output_buffer = source_data.blend_images(ScreenBlend);
    output_buffer.save("screen.jpg").unwrap();

    let output_buffer = source_data.blend_images(SubtractionBlend);
    output_buffer.save("subtraction.jpg").unwrap();
}

You can now run the program using the following command, and you should have all the images generated and saved in the project folder:

cargo run --release

As you might have guessed already, this implementation only works for 8-bit RGB images. This code, however, can be extended very easily to support the other color formats such as 8-bit Luma (Monochrome), 16-bit RGB (Many RAW camera images), and so on.

I highly encourage you to try that out. You can also reach out to me for help with anything in this tutorial or with extending the code in this tutorial. I'd be happy to answer all your queries. Email is the best way to reach me, you can email me at anshul@anshulsanghi.tech.

Glossary

Normalization refers to the process of rescaling the pixel values so that the values are in floating point format and are in the range of 0-1. For example, for an 8 bit image, the color black is represented by 0 (0 in de-normalized value) and the color white is represented by 1 (255 in de-normalized value). Intermediary decimal values between 0 & 1 represent different intensities of the pixel between black and white. Normalization is done for many different reasons such as:

• Preventing overflows during calculations.

• Re-scaling images to the same range irrespective of their individual color depth.

• Expanding possible dynamic range of the image.

Enjoying my work?

Consider buying me a coffee to support my work!

Till next time, happy coding and wishing you clear skies!

In this article, I'll give you a simple overview of how asynchronous programming works in Rust. You'll learn about futures as well as async/.await.

This article isn't a beginner's guide to Rust or asynchronous programming, so you'll need to have some experience with programming in Rust and familiarity with asynchronous programming to get the most out of this guide.

With that said, let's begin!

When Should You Use Asynchronous Programming?

Asynchronous tasks work like a more integrated version of threads. You can use them in a lot of the same scenarios where you can use multiple threads. The benefits async programming provides over multiple threads is that multi-threaded applications have a larger overhead to work on multiple tasks compared to asynchronous applications.

But this doesn’t make asynchronous applications better than multithreaded applications. Multi-threaded programs can run multiple computing-intensive tasks simultaneously – and multiple times faster than if you ran all the tasks in a single thread.

With asynchronous coding, trying to run multiple computing-intensive applications simultaneously will be much slower than just running every task on a single thread.

Asynchronous programming is very good for building applications that make many network requests or many IO operations like file reading and writing.

I can’t cover every case when you’ll want to use asynchronous techniques. But I can say that it’s most beneficial for tasks that have a lot of idle time (for example, waiting for a server to respond to a network request).

During the idle time of an asynchronous task, instead of blocking the thread, the event handler works on other tasks in the program until the asynchronous task is ready to make progress.

Overview of Asynchronous Rust – Futures

The basics of asynchronous Rust are Futures. Futures are similar to promises in JavaScript. They are Rust's way of saying "hey, I'm going to give you the result later, but just hold on to this (the future instance) to keep track of if the result is ready."

Futures are traits, with a poll state that is either Poll::Pending to signify that the future is in the process of executing its task, or Poll::Ready to signify that the future is done with its task.

Futures are lazy. They run when you .await them (we'll look into how to .await them in the next section). .awaiting a future pauses the execution of an asynchronous thread, and begins running its task. At this point the result of the poll method is Poll::Pending. When the future is done with its task, poll's state will become Poll::Ready, and the future will allow it's thread to proceed.

If you want to get more into the technical details about Futures, you can check out the documentation.

async/.await in Rust

async and .await are the primary ways you'll be working with asynchronous code in Rust. async is a keyword for declaring asynchronous functions. Within those functions, the .await keyword pauses its execution until the result of the future is ready.

Let's take a look at an example:

async fn add(a: u8, b: u8) -> u8 {
    a + b
}

async fn secondFunc() -> u8 {
    let a = 10;
    let b = 20;
    let result = add(a, b).await;
    return result;
}

Any asynchronous function declared with async fn wraps its return value in a future. On the third line of secondFunc, we .await the future from add(a, b) to get its result before proceeding to return it.

How to Work with Asynchronous Operations in main

Rust doesn’t allow you to async fn main functions. Running asynchronous operations from a non-asynchronous function may lead to some operations not fully concluding before the main thread is exited.

In this section, we’ll look at two ways solve this issue. The two ways to do this are with futures and tokio. Let's look at both.

tokio

tokio is a platform that provides tools and APIs for performing asynchronous applications. tokio also allows you to easily declare an asynchronous main function, which will help with the issue I described earlier.

To install tokio in your project, add this line to its Cargo.toml:

[dependencies]
tokio = { version = "1", features = ["full"] }

After adding the line, you can write your main functions like this:

async fn add(a: u8, b: u8) -> u8 {
    a + b
}

#[tokio::main]
async fn main() {
    let a = 10;
    let b = 20;
    let result = add(a, b).await;
    println!("{result}");
}

The futures library

futures is a library that provides methods for working with asynchronous Rust. For our use case, futures provides futures::executor::block_on(). This method works similar to .await in asynchronous functions. It blocks the main thread, until the result of future is ready. futures::executor::block_on() also returns the result of the completed future.

To install futures in your project, add this line to its Cargo.toml:

[dependencies]
futures = "0.3"

After installing the library, you can do something like this:

use futures::executor::block_on;

async fn add(a: u8, b: u8) -> u8 {
    a + b
}

fn main() {
    let a = 10;
    let b = 20;
    let result = block_on(add(a, b));
    println!("{result}"); 
}

First, we import the block_on method on the first line and use the method to block the main thread until the result of the add(a, b) function was ready. We also didn’t have to make the main function asynchronous like we did with tokio.

Conclusion

Asynchronous programming allows you to run operations in parallel and with less overhead and complexity compared to traditional multi-threading. In Rust, it allows you to build I/O and network applications more efficiently.

While this article should help you become familiar with the basics asynchronous programming in Rust, it’s still just an overview. There are some cases where you'll need to use other constructs in Rust like Pinning, Arcs, and so on.

If you have any questions or thoughts, feel free to reach out to me. Thanks for reading!

One great combination is using Rust with the Gear Protocol. In this guide, I'll show you how to build and deploy a smart contract using Rust and the Gear Protocol. Whether you're new to this or have some experience, this article will help you get started with clear and easy-to-follow steps.

Prerequisites

1. Have basic Rust knowledge.
2. Having a basic understanding of decentralization.

Table of Contents

1. Introduction to Vara Network & Gear Protocol.
2. Why Use the Web2 Analogy?
3. Message-based Communication.
4. Illustration
5. Vara Network's Role.
6. First Project – Reading a Joke
7. Next Project – input-msg
8. Metadata & State
9. Third Project – Building Messages
10. Final Project – Battle Showdown
11. Conclusion.

Introduction to Vara Network & Gear Protocol.

Vara Network

Think of Vara as the sturdy foundation of blockchain technology. It's a layer-1 blockchain, meaning that it's at the core of transactions, ensuring that they are secure and decentralized. Vara uses Nominated Proof-of-Stake (NPoS) for agreement, making it reliable and efficient.

Furthermore, Vara Network distinguishes itself through its novel Actor Model, an architecture characterized by isolation and asynchronous messaging. This paradigm shift in smart contract execution imbues Vara Network with unparalleled security and scalability, setting it apart from conventional blockchain platforms.

Gear Protocol

Gear Protocol is like a toolbox for developers. It's a smart contract engine that makes building decentralized apps (dApps) faster, safer, and cheaper. By using substrate technology and WebAssembly (Wasm), Gear makes it easy for developers to create dApps that run smoothly and securely.

Gear's utilization of the Wasm virtual machine serves as a cornerstone of its efficiency. By harnessing the power of Wasm, developers can transcend language barriers, seamlessly integrating existing codebases and accelerating the development lifecycle. This fusion of familiarity and performance paves the way for a new era of dApp creation, where speed, security, and scalability converge harmoniously.

In simpler terms, Vara Network and Gear Protocol work together to make blockchain technology more user-friendly and secure for building and using decentralized apps.

Why Use the Web2 Analogy?

Understanding message-based communication, particularly within the context of Gear Protocol, can be quite challenging. To gain a clearer understanding, I delved into the documentation and conducted additional research. Eventually, I stumbled upon an analogy that made it all click: the analogy of web HTTP requests, specifically the POST method.

Let's dissect this analogy step by step. Consider the familiar scenario of a user visiting a website like google.com and interacting with the search bar. When the user enters a search query and hits enter, what's happening behind the scenes is akin to a POST HTTP request being sent.

Here's how it unfolds:

1. User Interaction: The user initiates the action by typing a search query into the search bar and hitting enter. This action triggers a request for information.
2. Client Acknowledgment: Google's website, acting as the client-side user interface (UI), acknowledges the user's input and prepares to send a request to the server for processing.
3. Request Sent: Just like when you hit enter after typing a query, Google's website sends a POST request to its server, conveying the user's search query.
4. Server Processing: Upon receiving the POST request, Google's server processes the query, searching its vast index for relevant information.
5. Response Generation: After processing the query, Google's server generates a response containing the search results.
6. Response Sent: Finally, Google's server sends the response back to the client (the user's web browser), completing the communication cycle.

In this analogy, the user represents the initiator of the communication, the client (UI) serves as the intermediary between the user and the server, and the server acts as the responder, processing requests and generating responses.

By drawing parallels between message-based communication in Gear Protocol and the familiar concept of web HTTP requests, we can better grasp the dynamics at play. Just as understanding how web requests facilitate communication between users and servers is essential for navigating the internet, comprehending message-based communication in Gear Protocol is crucial for building and interacting with decentralized applications effectively.

how the POST method works

Message-based Communication

Similarly, the Gear Protocol operates based on user or program interactions.
Note: Programs on Gear can also interact with each other.
So here is a detailed explanation to the whole communication flow in Gear.

User Interaction and @gear-js/api

When a user (actor) interacts with the dApp's UI elements (like buttons or forms), _@gear-js/api_ (which is integrated into the UI) captures these interactions. Based on the interactions, it extracts information and potentially pre-defined message formats, and then contracts a message object containing the user's intent or request.

How to Send Messages

The constructed message object encapsulates the user's input and becomes the data @gear-js/api transmits across the Vara Network to the Gear crate within the program.

How the Program Receives and Processes Messages

Gear (crate) delivers the message object to the appropriate program deployed on the Vara Network based on the location the user initiated the action. The Gear crate within the program utilizes functions like msg::load() and access the delivered message object, which the program extracts information (such as payload, source, messsageId) from, and process it according to how it's designed by the developer.

How to Generate a Reply

Based on the processed input, the program creates a new message object containing a reply (response in web2) to the user's action or interaction (called reply) to or for the user.

Note, the program typically doesn't send the original message object back, it generates a new one based message received, which a reply is sent back to be received by @gear-js/api using the gstd crate from the program utilizing functions like the msg::reply or msg::reply_bytes.

UI Update

@gear-js/api, within the dApp, receives the reply message object delivered by the gstd crate from the program across the Vara Network and extracts the response data from the reply object, and finally updates the UI reflecting the program's response to the user's interaction.

And that's pretty much the communication between the Users, Client(dApp), Gear Protocol(gstd), and finally Vara Network.

Illustration

Let's discuss more about the diagrams below, and how they each interact with each other.

UI Update

This illustration above is just a bird's eye view of how communication flows from the user to the program. I'll provide a complete illustration for more clarity. But before that, let's break the overview illustration into three stages.

Initial Interaction Stage

As said earlier, this is when the user interacts with the program, both @gear-js/api and gstd.

Initial Interaction Stage

Business/Program Logic

This section depicts the communication between the program and Gear within Vara Network. The gstd is used by the program to access the transmitted message (msg::load()) from the initial stage to perform business logic.

Business/Program Logic

Reply (Response)

This final stage shows how user feedback is delivered to the user or program. @gear-js/api translates it if necessary, and then updates the dApp's UI with the results. This allows the user to see the outcome of their action within the dApp.

Reply(Response)

That's great, right? This should help you understand how messages are passed from the client to the program. But what does Vara Network role mean here? Earlier I said that, the message object get transmitted across the Vara Network, but I didn't say how. Let's explain that.

Vara Network's Role

In Vara, all participants, including user interfaces (through @gear-js/api) and smart contracts (programs & gstd), are considered as actors. Another point to know is that, actors don't directly call functions within other actors (as in, programs interacting with other programs or even users).

Instead, they send messages containing data or instructions. So in our explanation of the message-based communication, Vara serves as the underlying decentralized network infrastructure for communication of our system (dApps). It provides a secure and reliable platform for message transmission across distributed network of nodes. And since Vara utilize a consensus mechanism NPoS (Nominated Proof-of-Stake), it ensures network security and transaction validation.

Getting Our Hands Dirty

In order to build upon the above information I provided, you and I need to get our hands dirty by building and deploying programs with additional explanation for a clearer understanding.

Let's get started.

First Project - Reading a Joke

In this project, you're going to interact with and deploy your smart contract on Vara Network, and receive a reply message back.

This is just a simple project, and nothing too complex. I chose this example project because it aligns with the analogy I gave earlier.

Currently, this project should be fine when running it on your Windows system. In case you get an error, scroll to the part of this article with a guide for setting up a Windows Subsystem for Linux (WSL), since it would allow you to run a Linux environment, including command-line tools and applications, directly on Windows, without the overhead of a traditional virtual machine or dual boot setup.

To get started, create a directory named freecodecamp-gear-protocol. Since you'll be building about fours projects, and I think it is important on how you can setup your projects for Gear Protocol.

So in your freecodecamp-gear-protocol directory, create a Cargo.toml file with the following code:

[workspace]
resolver = "2"
members = []


[workspace.package]
name = "freecodecamp-gear-protocol"
version = "0.1.0"
edition = "2021"
authors = ["Rocky Essel"]
license = "MIT"
publish = false

[workspace.dependencies]
# Internal Crates
# External Crates

For someone new to Rust or used to creating single projects, I'll guide you through understanding and setting up a workspace in Rust, making it easy to grasp.

Understanding Your Workspace

A workspace in Rust is a set of packages (crates) that are managed together. Let's break down the key sections: [workspace], members, [workspace.package], and [workspace.dependencies]. So think of this like a cabin for your shoes, where each pair of shoes is a crate (package) that you want to keep organized.

[workspace] Section

The [workspace] section defines the overall workspace. It typically contains multiple members.

resolver = "2": Specifies the version of Cargo's feature resolver to use, improving how dependencies are managed across the workspace.

members: Lists the crates that are part of the workspace. When you add a project with cargo new --lib sneakers or boots, the members section of the Cargo.toml is populated with the name of the project you created.

If not added automatically, you can add them yourself.

For example:

members = ["sneakers", "boots"]

[workspace.package] Section

This section provides metadata for the entire workspace as if it were a single package.

• name: The name of the workspace package.
• version: The version of the workspace package.
• edition: The Rust edition being used (e.g., "2021").
• authors: List of authors.
• license: The license for the workspace package.
• publish: Indicates whether the workspace package should be published to crates.io.

Example:

[workspace.package]
name = "my-shoe-collection"
version = "0.1.0"
edition = "2021"
authors = ["Your Name"]
license = "MIT"
publish = false

[workspace.dependencies] Section

Lists dependencies that apply to the entire workspace. Meaning that every crate whether external or internal added to the [workspace.dependencies] is accessible to every project you create under project workspace. So below is how both external and internal crate are made accessible to other project.

Note: For internal crate, you need to add them yourself.

Internal Crates: Add internal crates like this:

sneakers = { path = "sneakers" }
boots = { path = "boots" }

External Crates: Add external crates like this:

polish = "1.0"

Example Cargo.toml

Here's an example combining these sections:

[workspace]
resolver = "2"
members = ["sneakers", "boots"]

[workspace.package]
name = "my-shoe-collection"
version = "0.1.0"
edition = "2021"
authors = ["Your Name"]
license = "MIT"
publish = false

[workspace.dependencies]
# Internal crate
sneakers = { path = "sneakers" } 
boots = { path = "boots" } 

# External crate
polish = "1.0"

So, here's how you set up a workspace for your project to manage multiple crates (sub-projects) and share dependencies and configuration settings across them. I spent quite some time understanding this, so I thought I'd share it with you all to make it easier.

To build your first smart contract, run the command below in your parent directory (freecodecamp-gear-protocol) on your terminal.

cargo new --lib receive-joke

.freecodecamp-gear-protocol
├── Cargo.toml
└── receive-joke
    ├── Cargo.toml
    └── src
        └── lib.rs

2 directories, 3 files

Head over to your freecodecamp-gear-protocol/receive-joke/Cargo.toml, and this is how you access crates, and configuration from the workspace directory(main) using .workspace=true, like below;

[package]
name ="receive-joke"
version.workspace = true
edition.workspace = true
authors.workspace = true
license.workspace = true
publish.workspace = true

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]

Next, create a build file in your receive-joke directory with path likereceive-joke/build.rs, and paste the code below. Now, the build.rs helps you to build your project into .wasm file, that is used to deploy your smart contract.

build.rs:

fn main() {
    gear_wasm_builder::build();
}

Currently, you have't install the nesscessary crate to help create your smart contract. Therefore, add the following crate to your workspace dependency.

Cargo.toml:

[workspace]
resolver = "2"
members = ["receive-joke"]


[workspace.package]
name = "freecodecamp-gear-protocol"
version = "0.1.0"
edition = "2021"
authors = ["Rocky Essel"]
license = "MIT"
publish = false

[workspace.dependencies]
# Internal Crates

# External Crates
gstd = "1.4.1"
gmeta = "1.4.1"
gtest  = "1.4.1"
gear-wasm-builder = "1.4.1"
parity-scale-codec = { version = "3.6.12", default-features = false }
scale-info = { version = "2.11.3", default-features = false }

For your first project, only gstd would be used, so like add that external crate to your receive-joke's Cargo.toml. Like below:

[dependencies]
gstd.workspace = true


[build-dependencies]
gear-wasm-builder.workspace = true

If you reached here without any errors, well done my friend. Next, is to clear any code in freecodecamp-gear-protocol/receive-joke/src/lib.rs. Let's move on to the next step.

In Gear Protocol, there are entry points. An entry point serves as a gateway or door to your code. Gear has a few entry points, namely:

state(),
handle(),
handle_reply(),
init(),
handle_signal(),

Each entry point plays a significant role. For example, init() is called when the smart contract(.wasm) is deployed, allowing you to set certain conditions or variables or even other functions that need to be executed for smooth sailing of your smart contract or program.

However, it's optional, meaning that you can choose to include or exclude it depending on your project, but it still gets executed, and it is the first message you'll see once you deploy your smart contract.

The handle() method is crucial as it contains most of the business logic. It's mandatory to include your program. More light will be shared on the entry points as you move forward.

Now, paste the following code into your receive-joke/src/lib.rs:

#![no_std]

use gstd::msg;

#[no_mangle]
extern "C" fn handle() {
    // Send a reply(in HTTP GET Request, you'd use "response").
    msg::reply_bytes(
        "What did the dirt say to the rain? If you keep this up, my name will be mud!",
        0,
    )
    .expect("Unable to reply");
}

The code above defines a function handle that, when called, sends a message you've defined as a response using the gstd::msg functionality. This gstd is a crate provided by Gear Protocol, to send and receive messages, and this is crucial for programs running on Vara Network to communicate with each other and external systems. And the reply_bytes send a new message as a reply to the message that is currently being processed.

Time to deploy and send your first message and recieve your joke reply. In your terminal, run the following command to build your program into .wasm.

Usually, I use cargo check for check for errors first, before using the build command below, either way is fine.

cargo build --release

After the build is completed, follow the structure below to locate your .wasm file in the path below:

.freecodecamp-gear-protocol
├── Cargo.lock
├── Cargo.toml
├── receive-joke
│   └── ...
└── target
    ├── ...
    └── wasm32-unknown-unknown
        ├── ...
        └── release
            ├── receive_joke.opt.wasm <--- Optimized for deployment.
            └── receive_joke.wasm

How to Deploy Your Smart Contract

Just like in other blockchain tools, that help you deploy your smart contract from the terminal, IDEA is where you deploy your smart contract and interact with it. We'll be exploring the interface in a bit. So finally, head over to IDEA so start familiarizing yourself with your deployment environment.

Smart Contract Deplyment Web App- IDEA

1. Step one, click on Upload program, then select or drag your .opt.wasm inside the modal. This takes you to the upload page, where you can change names, enter values for the payload, or change the payload type. For now, let's leave everything as it is, and click on the Calculate, which will enter a 0.00015 gas fee value for uploading your program.

Note: You can either set the gas limit yourself, or click on Calculate to allow the program to generate one for you.

Upload Page Details

At this point, click on the Upload Program, and click on the Submit button.

Transaction Details- PopUp

When you submit, you'll be prompted you sign into a your wallet, and approve.

Wallet - SubWallet

After the approval, a toast message should be displayed at the right-hand corner your computer/laptop screen for you to see the status of your program, whether it failed or succeeded.

Assuming it's a success, click on the Programs on the sidebar, then BOOM!, there's your program. Click on it, and let's explore.

Upon deploying, the first thing you see is the program ID, but after a few seconds, the name of your program would be shown.

Smart Contract Block - Page

Earlier, I said that when you deploy a program, the init() function is executed regardless of whether you defined it in your project or not, and that's what you see in the Messages section. Below is a simple illustration for you to familiarize and understand the information about your program.

Page Illustration

Now, it's time to send a message to your program and receive a reply back, which is our joke. Remember, you're not inputting any values, you're just performing a simple action to receive a reply from the program. So click on Calculate and press the Send Message (that's the action or interaction) button.

Performing an action - Initial Stage

After the message has been sent, a success toast will popup. Then head back to your program by clicking on the Cancel button, and you'll see two additional messages.

Remember, the blue color with the arrow represent the message you sent, and the green represents the reply you've received. So click on the replied message to see the joke, which says: What did the dirt say to the rain? If you keep this up, my name will be mud!.

Receiving Reply & More Illustration

Now, you're finally done with this project. In the next project, you'll be sending data or information to your program, and having it return a reply with your entered value attached to it. Here is the program deployed on the Vara Network.

Important Information

Though I've provided some context to the picture above, I want to expand on it. Both the Source and Destination takes an address that can be either a user (actor) and a program (actor), or even a message object.

Next Project – input-msg

Just like the illustration earlier, you're going to interact with your program by sending an input value to your smart contract deployed on [IDEA](https://idea.gear-tech.io/programs?node=wss%3A%2F%2Ftestnet.vara.network). IDEA is your deployment environment where you deploy your smart contract on the Vara Network.
The point here is for you to load input values from your user, and process it by concatenating a string to the user's input value: "We've received your query. {user's-input}".

This is the reply you'll send back to the user that sends a message (input value).

So in your freecodecamp-gear-protocol directory, run the command below to add another member to your freecodecamp-gear-protocol/Cargo.toml.

cargo new --lib input-msg

After adding another member or project in the freecodecamp-gear-protocol, your path should be freecodecamp-gear-protocol/input-msg.

Earlier, I made mention of how to access input values into the smart contract or program by using gstd, which has a function or method called load(). For the next step, clear your freecodecamp-gear-protocol/input-msg/src/lib.rs, and paste the following code and run cargo check.

#![no_std]

use gstd::{msg, prelude::*};

#[no_mangle]
extern "C" fn handle() {
    let new_msg = msg::load().expect("Unable to create string");
    let reply_msg = format!("We've received your query {}", new_msg);
    msg::reply_bytes(reply_msg, 0).expect("Unable to reply.");
}

The check fails, but why? Well, the load() function has a type of unknown. And since Rust is a strongly typed language, it has to always know the type before hand, which wasn't the case, so it failed to build the project.

This should tell you that the load() doesn't have a type, and it is up to you to set the right data type, and failure to do so would result in some frustrating errors like below.

Debugging

Now if you were to use a single project and not a workspace, then debugging the error might have easy like below.

  error[E0282]: type annotations needed
   --> C:\Users\user\Desktop\2024\web3\re-gear\input-msg\src\lib.rs:7:9
    |
  7 |     let new_msg = msg::load().expect("Unable to create string");
    |         ^^^^^^^
    |
  help: consider giving `new_msg` an explicit type
    |
  7 |     let new_msg: /* Type */ = msg::load().expect("Unable to create string");
    |                ++++++++++++

But since you and I are using a workspace, it makes debugging a little difficult. This is my error message i got, when dubgging this error.

  error[E0275]: overflow evaluating the requirement `gstd::parity_scale_codec::Compact<_>: gstd::Decode`
    |
    = help: consider increasing the recursion limit by adding a `#![recursion_limit = "256"]` attribute to your crate (`input_msg`)
    = note: required for `gstd::parity_scale_codec::Compact<_>` to implement `gstd::Decode`
    = note: 125 redundant requirements hidden
    = note: required for `gstd::parity_scale_codec::Compact<<_ as CompactAs>::As>` to implement `gstd::Decode`

  For more information about this error, try `rustc --explain E0275`.
  error: could not compile `input-msg` (lib) due to 1 previous error
  warning: build failed, waiting for other jobs to finish...
  error: cargo command run failed: exit status: 101
warning: build failed, waiting for other jobs to finish...

And if you look closely, you can tell that the input-msg is what creating the error. In this case, run rustc --explain E0275, which output an suggestion like this

An evaluation of a trait requirement overflowed.

Erroneous code example:

trait Foo {}

struct Bar<T>(T);

impl<T> Foo for T where Bar<T>: Foo {}

This error occurs when there was a recursive trait requirement that overflowed before it could be
evaluated. This often means that there is an unbounded recursion in resolving some type bounds.

To determine if a T is Foo, we need to check if Bar<T> is Foo. However, to do this check, we need to
determine that Bar<Bar<T>> is Foo. To determine this, we check if Bar<Bar<Bar<T>>> is Foo, and so on. This
is clearly a recursive requirement that can't be resolved directly.

Consider changing your trait bounds so that they're less self-referential.

Now, though, compared to the first error message, this message doesn't provide a direct solution, it does tells you that, there's a type error in your code. And the reason is that, the load() can load any data type, so you should always defined a type for it.

#![no_std]

use gstd::{msg, prelude::*};

#[no_mangle]
extern "C" fn handle() {
    let new_msg: String = msg::load().expect("Unable to create string");
    let reply_msg = format!("We've received your query {}", new_msg);
    msg::reply_bytes(reply_msg, 0).expect("Unable to reply.");
}

In the above code, you've added a type String to the new_msg because that's the type you're expecting. Now run the build command, and deploy the file .opt.wasm to IDEA.

.freecodecamp-gear-protocol
├── receive-joke
├── Cargo.toml
├── input-msg
│   └── ...
└── target
    ├── ...
    └── wasm32-unknown-unknown
        ├── ...
        └── release
            ├── input_msg.opt.wasm <--- Optimized for deployment.         
            ├── input_msg.wasm
            ├── receive_joke.opt.wasm
            └── receive_joke.wasm

When you're done, go to your program and click on the Send Messages. Type any value into the payload field, and it should be a type of String.

Submit and approve and head back to your program, then select your reply_message box and you should see your reply message.

Smart Contract - Reply Message

You can find the program here on the Vara Network.

Metadata & State

Metadata and state goes hand in hand with each other. In order for your client application to allow users to interact or request data(state) from your smart contract, you need to define both the metadata and state, and even if the state is defined, and the metadata was not provided, you cannot access the any data.

So let's take each step by step.

Metadata

In the Gear Protocol world, metadata is like a blueprint for defining how different parts of a decentralized app (dApp) talk to each other. It's similar to how interfaces or types work in TypeScript. These blueprints describe how things like initial data type to expect, handling messages, and swapping data happen in the dApp, whether In, Out, and InOut.

When we make clear blueprints, it helps developers make sure that all the different parts of the dApp understand each other's data formats. This makes it easy for the smart contract (program-actor) and the client side app to share data smoothly.

To create these blueprints for your program, we use the gmeta tool. It helps us define these blueprints by outlining how different interactions work and what kinds of data they involve.

So, think of metadata in your program as similar to how interfaces/types work in TypeScript. They help organize how the different parts of your dApp communicate and understand each other's data.

Example Of A Metadata

use gmeta::{InOut, Metadata, Out};

pub struct ProgramMetadata;

// Be sure to describe all the types.
// But if any of the endpoints is missing in your program, you can use ();
// as indicated in the case of `type Signal`.

impl Metadata for ProgramMetadata {
    type Init = InOut<MessageInitIn, MessageInitOut>;
    type Handle = InOut<MessageIn, MessageOut>;
    type Others = InOut<MessageAsyncIn, Option<u8>>;
    type Reply = String;
    type Signal = ();
    type State = Out<Vec<Wallet>>;
}

The above is an example of how it is defined. Don't worry if you don't understand it now, I'll cover more into details later. Now let's talk about the state.

State

In Gear Protocol, the state function serves as a dedicated storage space within a program. This storage allows us to store and retrieve data as needed. Since this data is stored in persistent memory, it remains accessible even after the contract stops running. What's fascinating is that anyone with access to the blockchain can view this stored data. The state function doesn't alter or modify the blockchain itself. Instead, it simply provides a way to access stored data within the program.

Here is an example of a state function:

// Describe state structure
#[derive(TypeInfo, Decode, Encode, Clone)]
pub struct Messages {
    pub id: ActorId,
    pub content: String,
}

// Declare and initialize the state
static mut MESSAGES: Vec<Messages> = Vec::new();

#[no_mangle]
extern "C" fn state() {
    msg::reply(unsafe { MESSAGES.clone() }, 0).expect("Failed to share state");
}

When the state function is called, it returns a list of wallets data stored within the program. This means that once a program is deployed on the blockchain, anyone can read its state.

Additionally, developers have the flexibility to create custom programs that can read the state. This empowers you and I to tailor our data access methods according to the specific needs for our dApp, even if the primary program undergoes changes.

The key takeaway is that, the state function facilitates access to data stored in smart contracts. It's worth noting that both users and other programs can access the state of a program, providing a versatile means of interacting with stored data.

Third Project - Building Messages

In our last project input-msg, we didn't keep track of the messages that got sent. So in this project, we'll cover the metadata and state.

Run the command below to create your project in /freecodecamp-gear-protocol/:

cargo new --lib messages

Next, add your build.rs file, and make the workspace dependencies available to the /freecodecamp-gear-protocol/messages.

Adding Metadata to Messages

To setup a metadata for your project, you need to create an additional crate to manage that, so cd into messages, and run the command below.

cargo new --lib io

In your freecodecamp-gear-protocol/messages/io/Cargo.toml, copy and paste the following code:

[package]
name = "messages-io"
version.workspace = true
edition.workspace = true


[dependencies]
gstd.workspace = true
gmeta.workspace = true

Here, I changed the name from io to messages-io, and the reason is for me to identify, and separate it for other io's in the workspace. And add the dependencies.

In order to use the io in your workspace, you need to go the freecodecamp-gear-protocol/Cargo.toml, and add a path from your io to your workspace, which you can then use in any of the projects that need struct, enum, and method.

In freecodecamp-gear-protocol/Cargo.toml:

[workspace]
resolver = "2"
members = ["receive-joke","input-msg"]


[workspace.package]
name = "freecodecamp-gear-protocol"
version = "0.1.0"
edition = "2021"
authors = ["Rocky Essel"]
license = "MIT"
publish = false

[workspace.dependencies]
# Internal Crates
messages-io={path = "messages/io"} < ---- Here

# External Crates
gstd = "1.4.1"
gmeta = "1.4.1"
gtest  = "1.4.1"
gear-wasm-builder = "1.4.1"
parity-scale-codec = { version = "3.6.12", default-features = false }
scale-info = { version = "2.11.3", default-features = false }

And that's the Internal Crate I talked about earlier. Next, you need to include the messages-io in your messages project, like below:

[package]
name="messages"
version.workspace = true
edition.workspace = true
authors.workspace = true
license.workspace = true
publish.workspace = true


[dependencies]
gstd.workspace = true
messages-io.workspace = true <---

[build-dependencies]
gear-wasm-builder.workspace = true
messages-io.workspace = true < ---

The reason for adding messages-io.workspace to both the [dependencies] and [build-dependencies] is to make the struct, enums, pub variables and methods accessible to messages/src/lib.rs, and messages/build.rs using messages-io.workspace.

Metadata in io/src/lib.rs

#![no_std]

use gmeta::{InOut, Metadata, Out};
use gstd::{prelude::*, ActorId, Vec};

pub struct MessageMetadata;

pub static mut MESSAGES: Vec<User> = Vec::new();

pub struct Message {
    pub id: ActorId,
    pub content: String,
}

impl Metadata for MessageMetadata {
    type Init = InOut<Message, String>;
    type Handle = InOut<Message, String>;
    type State = Out<Vec<Message>>;
    type Reply = ();
    type Others = ();
    type Signal = ();
}

To implement the logic of the message-handling system for your program or smart contract, understanding how to set the metadata of your program is crucial.
Therefore, much attention is needed here.

The MessageMetadata struct you've defined implements the Metadata trait, which then structures the message metadata for the program. Also, a mutable static variable MESSAGES is declared to store all the messages you and your users send to the program. And since it’s a mutable static variable, unsafe code will be required to modify it due to Rust's safety guarantees around mutable static variables.

The Message struct is defined with two fields: id (sender's identifier) and content (the message text).

The Metadata trait is implemented for MessageMetadata, defining several associated types. The Init type is set to InOut<MessageInit, String>, specifying the input-output types for the initialization phase. \

This means that when the contract is initialized, it will accept a MessageInit type and return a String. The Handle type is set to InOut<Message, String>, specifying the input-output types for handling messages. It accepts a Message type as input and returns a String.

The State type is set to Out<Vec<Message>>, defining the state output type, meaning that the state of the contract will be a vector of Message objects, and it doesn’t accept any input to retrieve this state. The Reply, Others, and Signal types are all set to (), indicating no additional reply, other types, or signals are used in this case.

Further Context of its usage.

In this system, the metadata definition specifies how the smart contract should handle initialization and message handling. During the initialization phase (Init), when the contract is deployed on the Vara Network, it uses the Init type to set up the initial state. The input is expected to be of type MessageInit, and the output will be a String. During deployment, you provide your ID and message content, which the contract processes using the init() method.

After deployment, the contract can handle new messages using the Handle type, which expects a Message type as input and returns a String as a response. This functionality is useful for adding new messages to the MESSAGES vector. For state management (State), the contract’s state is a list of messages (Vec<Message>), and it doesn’t accept any input to retrieve the state but outputs the current state when queried.

So to summarize, the code in freecodecamp-gear-protocol/messages/io/src/lib.rs defines the structure and behavior of a message-handling smart contract, specifying how it initializes, handles messages, and manages state.

Building the Metadata

In order to build your project with the metadata, you need to modify the build.rs, which initially looks like below:

fn main() {
    gear_wasm_builder::build();
}

There's nothing wrong with using the above code, but if you plan to build your program and deploy on the blockchain to use it on the client or anywhere else, it would be impossible to interact with your smart contract if the metadata is not defined. Think of it like ABI in other blockchain environment.

So replace the code with:

use messages_io::MessageMetadata;

fn main() {
    gear_wasm_builder::build_with_metadata::<MessageMetadata>();
}

Finally, you would be handling the logic for your smart contract in the messages/src/lib.rs using the handle() function.

Here is the code for the lib.rs:

#![no_std]

use gstd::{exec, msg, prelude::*, ActorId};

use messages_io::*;

#[no_mangle]
extern "C" fn init() {
    let init: Message = msg::load().expect("Unable to decode Message");
    let init_message = Message {
        id: init.id,
        content: init.content,
    };

    unsafe { MESSAGES.push(init_message) };    
    msg::reply_bytes("Successfully initialized", 0).expect("Failed to initialize successfully.");
}


#[no_mangle]
extern "C" fn handle() {

    let message_handler: Message = msg::load().expect("Unable to decode Message");
    let message = Message {
        id: message_handler.id,
        content: message_handler.content,
    };
    unsafe { MESSAGES.push(message) };
    msg::reply_bytes("Message sent successfully.", 0).expect("Failed to send  reply message.");
}


#[no_mangle]
extern "C" fn state() {
    msg::reply(unsafe { MESSAGES.clone() }, 0).expect("Failed to share state");
}

Initialization Function (init)

#[no_mangle]
extern "C" fn init() {
    let init: Message = msg::load().expect("Unable to decode Message");
    let init_message = Message {
        id: init.id,
        content: init.content,
    };

    unsafe { MESSAGES.push(init_message) };
    msg::reply_bytes("Successfully initialized", 0).expect("Failed to initialize successfully.");
}

The init function is the entry point for initializing the smart contract. It is marked with #[no_mangle] to prevent Rust from applying name mangling, making the function accessible from the smart contract runtime.

The function begins by loading the initial message from the input payload using msg::load(). This message is expected to be of type Message, and if decoding fails, the function will panic with an error message. Next, a new Message instance is created from the loaded data. This new message is then added to the global MESSAGES vector, which is a mutable static variable marked as unsafe due to potential data races. Finally, the function sends a reply indicating successful initialization using msg::reply_bytes. If this reply fails, the function will panic.

Message Handling Function (handle)

#[no_mangle]
extern "C" fn handle() {
    let message_handler: Message = msg::load().expect("Unable to decode Message");
    let message = Message {
        id: message_handler.id,
        content: message_handler.content,
    };
    unsafe { MESSAGES.push(message) };
    msg::reply_bytes("Message sent successfully.", 0).expect("Failed to send  reply message.");
}

The handle function is designed to handle incoming messages after the contract is deployed. Like the init function, it is marked with #[no_mangle] to ensure it can be called from the smart contract runtime. The function begins by loading the incoming message from the input payload. This message is decoded into a Message struct, and if decoding fails, the function will panic.

A new Message instance is then created from the decoded data and added to the global MESSAGES vector using an unsafe block. The function then sends a reply indicating that the message was sent successfully. If the reply fails, the function will panic.

State Query Function (state)

#[no_mangle]
extern "C" fn state() {
    msg::reply(unsafe { MESSAGES.clone() }, 0).expect("Failed to share state");
}

The state function allows querying the current state of the smart contract. It is also marked with #[no_mangle] for the same reasons as the previous functions. The function replies with a cloned version of the global MESSAGES vector, containing all the messages that have been added so far. This is done within an unsafe block due to the mutable static variable. If the function fails to send the state, it will panic.

So this code simply defines a smart contract with three main functions: init for initialization, handle for processing incoming messages, and state for querying the current state of the contract. Where each function carefully manages the global MESSAGES vector.

Deployment on the Vara Network

Now you're done with this project, and hope you learned and have understood most of what I've written. In our next project, you'll be building something a bit more complex than this. So let's begin.

Here is the program deployed on Vara Network, and this is the entire repository for the 3 projects we've built so far. The next project is going to be stand-alone project so you won't use the workspace.

Final Project

In this final project, you'll build a very simple game: where you (player) fights with the boss. So here is a simple layman's explanation of the game mechanics.

Game Description

This game is a one-on-one battle between a player and a boss. To begin, the player selects their character from three options: Warrior, Mage, or Archer. Once the player's character data is stored in the program, the game begins.

In the game, the player immediately faces the boss, who starts with 10 lives (represented as an integer), while the player begins with 10 lives by default. The objective is to defeat the boss using a specific rule: the boss has weaknesses represented by letters (X, Y, Z), each associated with a random number.

During gameplay, if the player enters one of these letters, for example, 'Y' with a value of 4, and the boss starts with 0 lives, the program subtracts 4 lives from the boss, leaving 6. Similarly, when the player makes a move with a letter, the boss also makes a move with the a random letter with an associated value added to it.

The player progresses to the next level upon defeating the boss, continuing the battle with new challenges. I call this game Battle Showdown 🤣😁😂.

Battle Showdown Mechanics Summary

Player and Boss Lives

• Player starts with 10 lives.
• Boss starts with 10 lives.

Weaknesses and Values

• Boss and the player has weaknesses represented by letters ( X, Y, Z), each associated with a random number.

Gameplay

• Player inputs a letter (for example, 'Y') and the associated value (for example, 4) is subtracted from the boss's lives.
• Boss retaliates with a letter and the same value is subtracted from the player's lives.
• The objective is for the player to reduce the boss's lives to 0 to progress to the next level.

Match Equation

Let's define the key variables:

• (Lp) = Player's current lives.
• (Lb) = Boss's current lives.
• (V) = Value associated with the letter representing the attack.

Initial conditions:

• ( Lp = 10 )
• ( Lb = 10 )

Player's turn:

• Player selects a letter with an associated value (V).
• Boss's lives are reduced: (Lb = Lb - V).

Boss's retaliation:

• Boss selects a letter (same value (V)).
• Player's lives are reduced: (Lp = Lp - V).

This continues until either ( Lb ) (boss's lives) or ( Lp ) (player's lives) reaches 0.

Equations

After the player's attack and the Boss's retaliation:
[Lb = Lb - V]
[Lp = Lp - V]

The game loop continues with the player and boss exchanging moves. Repeat until Lb <= 0 or Lp <= 0

Example

If the player inputs 'Y' with a value of 4:

• Initial: ( Lp = 10 ), ( Lb = 10 )
• Player's attack: ( Lb = 10 - 4 = 6 )
• Boss's retaliation: ( Lp = 10 - 4 = 6 )

Next move:

• If the player inputs another value, let's say: 'X' with a value of 5:
• Player's attack: ( Lb = 6 - 5 = 1 )
• Boss's retaliation: ( Lp = 6 - 5 = 1 )

The player wins as the Boss's lives ( Lb ) have reached 0.

The match equation for each round of the game can be summarized as:
[Lb = Lb - V]
[Lp = Lp - V]
This process is repeated until either the player's lives (( Lp )) or the Boss's lives (( Lb )) reach 0, determining the winner of the battle.

Windows Error

If you use Windows, you may encounter an error with the link.exe. Honestly, I cannot explain the reason behind the error, but in the Gear docs, it was made clear that Windows users might experience some problems when building their project.

But rest assured, there's a solution, and I'm going to guide you through it. So please follow these steps carefully so you don't have to deal with bugs along the way.

Step 1 - Install WSL via Command Prompt

Open your CLI with administrative privileges, and run the command below:

wsl --install

After excutting the command, run the command below to list other Linux releases.

wsl -l -o

This command shows a list of other Linux distros, and you can select anyone you've used before. If you're new to Linux distros, I recommend selecting the Ubuntu-22.04.

These are just lists and are read-only. In order to select your system, run the command below.

wsl --install -d {Distribtion Name here(Ubuntu-22.04)}

After you're done with the installation, restart your PC. Wait a little while for the terminal to popup and prompt you for your details such as your username and password. If the terminal doesn't open, go to your start menu, and you will find something similar to this in your Start menu.

Ubuntu in start menu

After that, the next thing to do is to install Rust on your WSL.

How to Install Rust On Your WSL

To install Rust on your machine, I recommend that whenever you want to install any package, it is best practice to install updates and upgrades to the system before continuing with the installation.

sudo apt update && sudo apt upgrade -y

When you run sudo apt update && sudo apt upgrade -y, you first update the package lists to get the latest information about available software packages. Then, if the update is successful, it proceeds to upgrade the installed packages to their latest versions, automatically confirming the upgrades to avoid manual intervention. This is a common and recommended practice to keep your Linux system up-to-date and secure.

Essential Dependencies.

The command below installs essential development tools (build-essential, gcc, and make) and the curl utility for making HTTP requests and downloading files. These packages are commonly required for software development, compilation, and system administration tasks.

 sudo apt install curl build-essential gcc make -y

After that, run the command in the terminal

 curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

In the installation process, you'll be prompted a question: go with the default when it happens.

1. Proceed with installation (default) --> Enter
2. Customize installation
3. Cancel installation.

After this prompt, you have successfully installed Rust on the Ubuntu System. Now the next step is to restart your terminal, simply close the current terminal. Open a new one, and run the command below.

source "$HOME/.cargo/env"

What this command source "$HOME/.cargo/env" does is to activate the Rust environment. The reason is that the Rust environment comprises essential variables and configurations required for effective Rust usage. Now, once run, there's no output, so you can verify the installation by running the command below.

 rustc -V

Expected output:

rockyessel@UBUNTU-ROCKY:~$ rustc -V rustc 1.73.0 (cc66ad468 2024-02-07)

When you're done, there're also additional dependencies we have to install. So here, install them.

// Install the following.

 --> rustup toolchain add nightly-2023-09-18
 --> rustup target add wasm32-unknown-unknown --toolchain nightly-2023-09-18

After successfully installing them, head to the next section, which is building a game project.

In your WSL terminal, create your project name battle-showdown, and adding all the necessary toml files, and dependences. After that, cd into your project battle-showdown and added another program called io, this is where you write your metadate and other complex or simple data-type for your dApp.

battle-showdown
.
├── Cargo.toml
├── io
│   ├── Cargo.toml
│   └── src
│       └── lib.rs
└── src
    └── lib.rs

So head-over to to your ./io/src/lib.rs and paste the follow code:

#![no_std]

use gmeta::Metadata;

pub struct BattleShowdown;

impl Metadata for BattleShowdown {
    type Init = ();
    type Handle = ();
    type State = ();
    type Reply = ();
    type Others = ();
    type Signal = ();
}

Here, you have defined a new public struct named BattleShowdown. Structs are used to create custom data types by grouping fields of various types together. But in this case, you're providing an implementation for the required types of the Metadata trait for the BattleShowdown struct: impl Metadata for BattleShowdown.

type Init = (), type Handle = (), type State = (), type Reply = (), type Others = (), and type Signal = () specifies that the handlers or functions data type for BattleShowdown is of type (), which in Rust's unit type, equivalent to void in other language such as TypeScript.

So for now, we're saying that these handlers do not send or receive data as such. Hence, the code just specifies how BattlwShowdown interacts with the system. However, it is worth mentioning that, the BattleShowdown struct itself doesn't have any specific initialization data, state, handling behavior, replies, signals, or other associated types defined.

Building Our Game

First off, let's make register the io in your parent directory cargo.toml. So head over to ./cargo.toml and paste the code below:

workspace = { members = ["io"] }
[package]
name = "battle-showdown"
version = "0.1.0"
edition = "2021"

[dependencies]
gstd = "1.4.1"
battle-showdown-io={path = "io"}



[build-dependencies]
gear-wasm-builder = "1.4.1"
battle-showdown-io={path = "io"}

I've ensured that the "battle-showdown-io" path is included in both the dependencies and build-dependencies sections. This decision was intentional because when it's added solely to build-dependencies, only the structs, enums, and other data types or functions within the build.rs file gain access to them, not the dependencies in your ./src/lib.rs. This is important because I'll be utilizing battle-showdown-io in both build-dependencies (build.rs) and dependencies (./src/lib.rs).

This step is crucial because overlooking it can lead to frustrating import errors.

Next, is the file ./io/cargo.toml, paste the following code below.

[package]
name = "battle-showdown-io"
version = "0.1.0"
edition = "2021"

# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
gstd = "1.4.1"
gmeta = "1.4.1"
parity-scale-codec = { version = "3.6.12", default-features = false }
scale-info = { version = "2.11.3", default-features = false }

Explaining Metadata For BattleShown

It's crucial to pay closer attention to this section, as I'll shed more light on explaining the metadata types for BattleShown and deploying it on the Vara Network.

#![no_std]

use gmeta::Metadata;

pub struct BattleShowdown;

impl Metadata for BattleShowdown {
    type Init = ();
    type Handle = ();
    type State = ();
    type Reply = ();
    type Others = ();
    type Signal = ();
}

Defining Initialization – Init

To define the types for this purpose, consider whether your program or smart contract needs to store data or perform actions before the user can interact with it. In the case of this game, the answer is yes.

The game assumes that only one person is playing and does not allow users to create their characters or players. This means that you need to store data before proceeding to use this program, and in this scenario, we need information about the person/developer/actor/user deploying the contract or program, which is you.

Here is the information you'll want to store:

• playerId - Type: ActorId
  The playerId is actually the address associated with your account, which has the type of ActorId.
• playerName - Type: String
  This has a type of string, pretty much straightforward.
• playerCharacterType - Type: Enum
  The playerCharacterType is an enum that gives the actor an option to select which type they want to be, with options including Mage, Warrior, and Archer.

#![no_std]

use gmeta::Metadata;

pub struct BattleShowdown;

impl Metadata for BattleShowdown {
    type Init = InOut<InitBattleShowdown, String>;
    type Handle = ();
    type State = ();
    type Reply = ();
    type Others = ();
    type Signal = ();
}

#[derive(Debug, Clone, Copy, Encode, Decode, TypeInfo)]
pub enum CharacterType {
    Warrior,
    Mage,
    Archer,
}


#[derive(Default, Debug, Encode, Decode, TypeInfo)]
pub struct InitBattleShowdown {
    pub player_id: ActorId,
    pub player_character_type: CharacterType,
    pub player_name: String,
}

Although I've previously discussed metadata, you might be curious about what type Init = InOut<InitBattleShowdown, String>; means. Well, it's nothing complex. Here, we're simply stating that the init handle will accept a data type of InitBattleShowdown and will respond with a data type of String.

Before you continue, one more step remains: implementing a default trait for the enum CharacterType. This ensures that if the CharacterType is not explicitly defined, it defaults to Warrior. Simply add the following code to the existing code above:

impl Default for CharacterType {
    fn default() -> Self {
        CharacterType::Warrior
    }
}

Defining the Handle

Defining a type for the handle function mirrors the process for the init function, but the actual implementation is left to the developer, which in this case, is you. After reviewing code and experimenting with different approaches, I discovered a method used by Gear Protocol (which shares similarities with some of their projects) that made more sense.

Action & Event

In their implementation, they utilized Actions and Events. Actions represent a set of operations that the program can perform, while Events are the outcomes of these Actions.

For example, in the context of this game, you could have an action named Attack with a corresponding Event named Attacked. These could potentially accept parameters and return results.

Therefore, to define the handle type, include the following code in your existing codebase:

#[derive(Debug, Encode, Decode, TypeInfo)]
pub enum BattleShowdownAction {
    Attack {
        character_hit_power_value: PlayerHitPowerValue,
    },
}

Previously, when describing the game mechanics, I introduced a mechanic involving a letter with a randomly assigned number to inject an element of randomness. In this context, these letters correspond to an ENUM state of X, Z, and Y.

Therefore, to implement this mechanic, add the following code:

...

#[derive(Debug, Clone, Copy, Encode, Decode, TypeInfo)]
pub enum PlayerHitPowerValue {
    X,
    Y,
    Z,
}

When an actor or user decides to attack the boss, they can select from the options provided above, each with a random value. Consequently, each attack on the boss will yield different outcomes due to the variability in these values. Similar to how you implemented a default trait for the CharacterType, you should follow suit here.

impl Default for PlayerHitPowerValue {
    fn default() -> Self {
        PlayerHitPowerValue::X
    }
}

Event

As mentioned earlier, events are the outcomes of actions. Unlike the BattleShowdownAction, which only had one action, the BattleShowdownEvent will encompass more than two actions. Why? Because the game's logic dictates that when the user attacks, the boss also counterattacks. This results in three possible outcomes: either the user loses, the boss is defeated, or the battle continues.

However, the third outcome is contingent upon the first two outcomes.
Therefore, for the BattleShowdownEvent, you will need to define:

#[derive(Debug, Encode, Decode, TypeInfo)]
pub enum BattleShowdownEvent {
    Attacked {
        id: ActorId,
        character_type: CharacterType,
        name: String,
        player_lives: u32,
        boss_livesL: u32,
    },

    PlayerLost {
        id: ActorId,
        character_type: CharacterType,
        boss_livesL: u32,
        player_lives: u32,
        message: String,
    },

    BossLost {
        character_type: CharacterType,
        player_lives: u32,
        boss_livesL: u32,
        message: String,
    },
}

You have one action, but there are three possible events, correct? When the user/actor attacks the boss and the boss counterattacks, if either of them is defeated, the "Attacked" event is returned. However, if the player successfully defeats the boss, the "BossLost" event is returned.
Now that you have a solid understanding, let's incorporate both the input and output types for the Handle function: type Handle = InOut<BattleShowdownAction, BattleShowdownEvent>;.

Defining State

As previously mentioned, the state stores information within your program. For BattleShowdown, the state you'd want to store includes information about the player, the boss, and the current level.

#[derive(Default, Debug, Encode, Decode, TypeInfo)]
pub struct BattleShowdownState {
    pub player_id: ActorId,
    pub player_character_type: CharacterType,
    pub current_level: u32,
    pub player_lives: u32,
    pub player_name: String,
    pub boss_lives: u32,
    pub player_hit_power: u32,
    pub boss_hit_power: u32,
}

Therefore, whenever you call the state function, you should expect to see the result in this format. Now, add the BattleShowdownState to the state in the metadata, like so: type State = Out<BattleShowdownState>;.

With that, the setup is complete. Here is the entire code for the ./io/src/lib.rs file.

#![no_std]

use gmeta::{In, InOut, Metadata, Out};
use gstd::{prelude::*, ActorId};

// Define the main struct for the BattleShowdown
pub struct BattleShowdown;

// Implementing Metadata for BattleShowdown
impl Metadata for BattleShowdown {
    // Define the type for initialization messages
    type Init = InOut<InitBattleShowdown, String>;
    // Define the type for handle messages
    type Handle = InOut<BattleShowdownAction, BattleShowdownEvent>;
    // Define the type for state messages
    type State = Out<BattleShowdownState>;
    type Reply = ();
    type Others = ();
    type Signal = ();
}

// Struct for initializing the BattleShowdown
#[derive(Default, Debug, Encode, Decode, TypeInfo)]
pub struct InitBattleShowdown {
    pub player_id: ActorId,
    pub player_character_type: CharacterType,
    pub player_name: String,
}

// Struct representing the state of the BattleShowdown
#[derive(Default, Debug, Encode, Decode, TypeInfo)]
pub struct BattleShowdownState {
    pub player_id: ActorId,
    pub player_character_type: CharacterType,
    pub current_level: u32,
    pub player_lives: u32,
    pub player_name: String,
    pub boss_lives: u32,
    pub player_hit_power: u32,
    pub boss_hit_power: u32,
}

// Enum representing different character types
#[derive(Debug, Clone, Copy, Encode, Decode, TypeInfo)]
pub enum CharacterType {
    Warrior,
    Mage,
    Archer,
}

// Enum representing different values for player hit power
#[derive(Debug, Clone, Copy, Encode, Decode, TypeInfo)]
pub enum PlayerHitPowerValue {
    X,
    Y,
    Z,
}

// Implementing Default for PlayerHitPowerValue
impl Default for PlayerHitPowerValue {
    fn default() -> Self {
        PlayerHitPowerValue::X
    }
}

// Implementing Default for CharacterType
impl Default for CharacterType {
    fn default() -> Self {
        CharacterType::Warrior
    }
}

// Enum representing different actions in the BattleShowdown
#[derive(Debug, Encode, Decode, TypeInfo)]
pub enum BattleShowdownAction {
    Attack {
        character_hit_power_value: PlayerHitPowerValue,
    },
}

// Enum representing different events in the BattleShowdown
#[derive(Debug, Encode, Decode, TypeInfo)]
pub enum BattleShowdownEvent {
    Attacked {
        id: ActorId,
        character_type: CharacterType,
        name: String,
        player_lives: u32,
        boss_lives: u32,
    },
    PlayerLost {
        id: ActorId,
        character_type: CharacterType,
        boss_lives: u32,
        player_lives: u32,
        message: String,
    },
    BossLost {
        character_type: CharacterType,
        player_lives: u32,
        boss_lives: u32,
        message: String,
    },
}

Build.rs

Import BattleShowdown to the build.rs from your parent directory at ./src/build.rs. If you encounter an import error, make sure that in your ./cargo.toml, you're registering battle-showdown-io={path = "io"} there.

use battle_showdown_io::BattleShowdown;

fn main() {
    gear_wasm_builder::build_with_metadata::<BattleShowdown>();
}

That's it for the build.rs, and what it does is to build your project into wasm and then build the metadata for BattleShown for you.

Game Logic Implementation - ./src/lib.rs

For this section, I'll write the code below, then I'll explain this as we go. There's going to be a problem I'd want you to solve, which will be about the state.

#![no_std]

use gstd::{exec, msg, prelude::*, ActorId};

use battle_showdown_io::*;

// Function to generate random number between 1 and 3
fn get_random_u32() -> u32 {
    let salt = msg::id();
    let (hash, _num) = exec::random(salt.into()).expect("get_random_u32(): random call failed");
    (u32::from_le_bytes([hash[0], hash[1], hash[2], hash[3]]) % 3) + 1 // Generate random number between 1 and 3
}

#[derive(Debug, Default)]
pub struct BattleShowdown {
    pub player_id: ActorId,
    pub player_character_type: CharacterType,
    pub current_level: u32,
    pub player_lives: u32,
    pub player_name: String,
    pub boss_lives: u32,
    pub character_hit_power_value: PlayerHitPowerValue,
    pub player_hit_power: u32,
    pub boss_hit_power: u32,
    pub game_state: String,
}

static mut BATTLESHOWNDOWN: Option<BattleShowdown> = None;

#[no_mangle]
unsafe extern "C" fn init() {
    let init: InitBattleShowdown = msg::load().expect("Unable to decode InitBattleShowdown");

    let battle_showdown = BattleShowdown {
        player_id: msg::source(),
        player_character_type: init.player_character_type,
        player_name: init.player_name,
        boss_lives: 10,
        player_lives: 10,
        ..Default::default()
    };

    BATTLESHOWNDOWN = Some(battle_showdown);

    msg::reply_bytes("Successfully initialized", 0).expect("Failed to initialize successfully.");
}

impl Encode for BattleShowdown {
    fn encode(&self) -> Vec<u8> {
        let mut encoded = Vec::new();

        // Encode each field of BattleShowdown struct
        encoded.extend_from_slice(&self.player_id.encode());
        encoded.extend_from_slice(&self.player_character_type.encode());
        encoded.extend_from_slice(&self.current_level.encode());
        encoded.extend_from_slice(&self.player_lives.encode());
        encoded.extend_from_slice(&self.player_name.encode());
        encoded.extend_from_slice(&self.boss_lives.encode());
        encoded.extend_from_slice(&self.character_hit_power_value.encode());
        encoded.extend_from_slice(&self.player_hit_power.encode());
        encoded.extend_from_slice(&self.boss_hit_power.encode());
        encoded.extend_from_slice(&self.game_state.encode());

        encoded
    }
}

impl BattleShowdown {
    // Placeholder for the `attack` method
    fn attack(&mut self, _character_hit_power_value: PlayerHitPowerValue) -> BattleShowdownEvent {
        // Implement this method according to your game logic
        // For now, just returning an empty event

        // Calculate total hit power for player based on character type and random values
        let character_hit_power = match &self.player_character_type {
            CharacterType::Warrior => 4,
            CharacterType::Mage => 3,
            CharacterType::Archer => 2,
        };

        let player_hit_power = match &self.character_hit_power_value {
            PlayerHitPowerValue::X => character_hit_power + get_random_u32(),
            PlayerHitPowerValue::Y => character_hit_power + get_random_u32(),
            PlayerHitPowerValue::Z => character_hit_power + get_random_u32(),
        };

        // Placeholder for boss attack logic
        // Update boss hit power to a random value for each attack
        self.boss_hit_power = get_random_u32();

        self.player_hit_power = player_hit_power;

        // Reduce boss's lives based on player's hit power
        self.boss_lives = self.boss_lives.saturating_sub(self.player_hit_power);
        // Reduce player's lives based on boss's hit power
        self.player_lives = self.player_lives.saturating_sub(self.boss_hit_power);

        // Check if player or boss has lost
        if self.player_lives == 0 {
            // Player lost
            self.game_state = "Player lost.".to_string();
            return BattleShowdownEvent::PlayerLost {
                id: self.player_id,
                boss_lives: self.boss_lives,
                character_type: self.player_character_type,
                message: "".to_string(),
                player_lives: self.player_lives,
            };
        } else if self.boss_lives == 0 {
            // Boss lost
            self.game_state = "Boss lost.".to_string();
            return BattleShowdownEvent::BossLost {
                boss_lives: self.boss_lives,
                character_type: self.player_character_type,
                player_lives: self.player_lives,
                message: "You've defeated the boos".to_string(),
            };
        }

        self.game_state = "The games continues.".to_string();
        BattleShowdownEvent::Attacked {
            boss_lives: self.boss_lives,
            character_type: self.player_character_type,
            id: self.player_id,
            name: self.player_name.clone(),
            player_lives: self.player_lives,
        }
    }
}

#[no_mangle]
extern "C" fn handle() {
    let battle_showdown_action: BattleShowdownAction =
        msg::load().expect("Could not load BattleShowdownAction");
    let battle_showdown = unsafe {
        BATTLESHOWNDOWN
            .as_mut()
            .expect("`BattleShowdown` is not initialized.")
    };
    let result: BattleShowdownEvent = match battle_showdown_action {
        BattleShowdownAction::Attack {
            character_hit_power_value,
        } => battle_showdown.attack(character_hit_power_value),
    };
    msg::reply_bytes(result.encode(), 0)
        .expect("Failed to encode or reply with `BattleShowdownEvent`.");
}

#[no_mangle]
extern "C" fn state() {
    let battle_showdown = unsafe {
        BATTLESHOWNDOWN
            .take()
            .expect("Unexpected error in taking state")
    };

    msg::reply(battle_showdown, 0).expect("Unable to share the state");
}

At first glance this might seem a lot, but it isn't, so don't get too intimidated. Before you start, make sure you understand the whole logic of the game description I gave earlier since you'll be implementing it here.

Above, we have some important functions, struct, and impl, and here is an overview of what they do.

1. With the get_random_u32 function, we generated a random number between 1 and 3.
2. The BattleShowdown struct in the /src/lib.rs represents the main state of the game. It holds information such as player and boss stats, current game level, and game state. The static mut BATTLESHOWNDOWN: Option<BattleShowdown> = None; is a static mutable variable that holds the current state of the game. It's wrapped in an Option to indicate whether the game has been initialized yet or not, which you'll use later in your implementation.
3. unsafe extern "C" fn init() is responsible for initializing the game state when called after the contract has been uploaded. It loads an initialization message, constructs a BattleShowdown instance based on that message, and sets BATTLESHOWNDOWN to Some with the constructed instance.
4. impl Encode for BattleShowdown: this trait is implemented for BattleShowdown, enabling it to be encoded into a byte representation. This is useful for serialization and sending the game state over the network. And there's a way to also implement the trait without creating an impl for BattleShowdown.
5. impl BattleShowdown: this impl for BattleShowdown is where the entire logic happens, and for now, we've only added an attack method for it. It worth noting that we'll be adding more as we continue this project.
6. So what does the attack method do? Well, The attack method simulates a combat encounter between the player and the boss character in our game. It calculates the hit power for both entities based on character type and randomness, manages their health points accordingly, and generates game events to reflect the outcome of the encounter.
7. extern "C" fn handle(): In our case, the handle function is used to process incoming messages, specifically BattleShowdownAction. So depending on the action perform by the actor, it dispatches a result of the action to the appropriate methods of BattleShowdown, such as attack, and sends back the resulting game events to the actor. Like disccued in the illustration.
8. And lastly, extern "C" fn state() simply retrieves the current game state represented by BattleShowdown and sends it as a reply.

This is the overall explanation to the code in the file. But leaving with this isn't enough for even me. Let's disccus more below.

Understanding theinit()

#[derive(Debug, Default)]
pub struct BattleShowdown {
    pub player_id: ActorId,
    pub player_character_type: CharacterType,
    pub current_level: u32,
    pub player_lives: u32,
    pub player_name: String,
    pub boss_lives: u32,
    pub character_hit_power_value: PlayerHitPowerValue,
    pub player_hit_power: u32,
    pub boss_hit_power: u32,
    pub game_state: String,
}

static mut BATTLESHOWNDOWN: Option<BattleShowdown> = None;

#[no_mangle]
unsafe extern "C" fn init() {
    // Load initialization data
    let init: InitBattleShowdown = msg::load().expect("Unable to decode InitBattleShowdown");

    // Create a BattleShowdown instance with initial values
    let battle_showdown = BattleShowdown {
        player_id: msg::source(),
        player_character_type: init.player_character_type,
        player_name: init.player_name,
        boss_lives: 10,
        player_lives: 10,
        ..Default::default()
    };

    // Store the BattleShowdown instance
    BATTLESHOWNDOWN = Some(battle_showdown);

    // Reply to signal successful initialization
    msg::reply_bytes("Successfully initialized", 0).expect("Failed to initialize successfully.");
}

The function loads data from an initialization message (InitBattleShowdown) sent by the developer or player. This data includes the player's chosen character type and name. Based on the initialization data, a BattleShowdown instance is created with initial values, which is stored in battle_showdown.

This instance represents the state of the game, including player and boss stats, current level, and game state. The created BattleShowdown instance is stored in the BATTLESHOWNDOWN static variable, allowing the game logic to access and manipulate the game state throughout the gameplay. Finally, a reply message is sent back to the developer or player to indicate successful initialization of the game contract.

This function sets up the initial state of the game, paving the way for further interactions and gameplay logic.

Understanding the handle()

#[no_mangle]
extern "C" fn handle() {
    // Load the action from the message
    let battle_showdown_action: BattleShowdownAction =
        msg::load().expect("Could not load BattleShowdownAction");

    // Retrieve the current game state
    let battle_showdown = unsafe {
        BATTLESHOWNDOWN
            .as_mut()
            .expect("`BattleShowdown` is not initialized.")
    };

    // Execute the appropriate action on the game state and get the result
    let result: BattleShowdownEvent = match battle_showdown_action {
        BattleShowdownAction::Attack {
            character_hit_power_value,
        } => battle_showdown.attack(character_hit_power_value),
    };

    // Send back the result as a reply message
    msg::reply_bytes(result.encode(), 0)
        .expect("Failed to encode or reply with `BattleShowdownEvent`.");
}

The handle() function plays a crucial role in processing incoming messages and orchestrating the game's actions. It serves as the bridge between player interactions and the game's internal logic. When invoked, handle() begins by loading the action sent by the player from the message.

This action, encapsulated as BattleShowdownAction, dictates the player's intended move, such as attacking the boss. Next, the function retrieves the current game state from the BATTLESHOWNDOWN variable. This state holds essential information about the player, the boss, and the overall game environment.

With both the action and the game state at hand, handle() proceeds to execute the appropriate action. For instance, if the player's action is an attack, the function triggers the attack() method on the battle_showdown instance. This method calculates the outcome of the player's attack, considering factors like the player's hit power and the boss's remaining health points.

Crucially, the attack() method requires a parameter: character_hit_power_value. This parameter corresponds to the player's choice between three options: X, Y, and Z, each associated with different hit power values as disccused in earlier sections.

Once the action is executed, handle() generates an event, encapsulated as BattleShowdownEvent, reflecting the outcome of the player's move. This event encapsulates important details, such as changes in player and boss health points. Finally, handle() responds to the player by replying with the result of the action as a byte-encoded message. This message contains the updated game state, allowing the player to understand their current situation, including their health status and that of the boss.

Understanding the impl BattleShowdown for attack

impl BattleShowdown {
    fn attack(&mut self, _character_hit_power_value: PlayerHitPowerValue) -> BattleShowdownEvent {
        // Calculate total hit power for player based on character type and random values
        let character_hit_power = match &self.player_character_type {
            CharacterType::Warrior => 4,
            CharacterType::Mage => 3,
            CharacterType::Archer => 2,
        };

        let player_hit_power = match &self.character_hit_power_value {
            PlayerHitPowerValue::X => character_hit_power + get_random_u32(),
            PlayerHitPowerValue::Y => character_hit_power + get_random_u32(),
            PlayerHitPowerValue::Z => character_hit_power + get_random_u32(),
        };

        // Placeholder for boss attack logic
        // Update boss hit power to a random value for each attack
        self.boss_hit_power = get_random_u32();

        self.player_hit_power = player_hit_power;

        // Reduce boss's lives based on player's hit power
        self.boss_lives = self.boss_lives.saturating_sub(self.player_hit_power);
        // Reduce player's lives based on boss's hit power
        self.player_lives = self.player_lives.saturating_sub(self.boss_hit_power);

        // Check if player or boss has lost
        if self.player_lives == 0 {
            // Player lost
            self.game_state = "Player lost.".to_string();
            return BattleShowdownEvent::PlayerLost {
                id: self.player_id,
                boss_lives: self.boss_lives,
                character_type: self.player_character_type,
                message: "".to_string(),
                player_lives: self.player_lives,
            };
        } else if self.boss_lives == 0 {
            // Boss lost
            self.game_state = "Boss lost.".to_string();
            return BattleShowdownEvent::BossLost {
                boss_lives: self.boss_lives,
                character_type: self.player_character_type,
                player_lives: self.player_lives,
                message: "You've defeated the boss".to_string(),
            };
        }

        self.game_state = "The game continues.".to_string();
        // Return event indicating attack occurred
        BattleShowdownEvent::Attacked {
            boss_lives: self.boss_lives,
            character_type: self.player_character_type,
            id: self.player_id,
            name: self.player_name.clone(),
            player_lives: self.player_lives,
        }
    }
}

The attack method within the BattleShowdown implementation simulates a pivotal moment in the game: a combat encounter between the player and the boss character.

Here's how it works:

Firstly, the method calculates the total hit power for the player based on their character type (character_hit_power) and randomness (player_hit_power). Different character types (Warrior, Mage, or Archer) have different base hit powers.

Next, a random hit power value is added to the character's base hit power. This adds an element of unpredictability to each attack. The method then updates the boss's hit power (self.boss_hit_power = get_random_u32();) to a random value, representing the boss's retaliatory strike against the player.

After calculating the hit powers, the method reduces the boss's lives based on the player's hit power and vice versa, updating their respective health points accordingly.

        // Reduce boss's lives based on player's hit power
        self.boss_lives = self.boss_lives.saturating_sub(self.player_hit_power);
        // Reduce player's lives based on boss's hit power
        self.player_lives = self.player_lives.saturating_sub(self.boss_hit_power);

The game state is then checked to determine if either the player or the boss has lost the battle. If the player's health points reaches zero, the game state is updated to indicate that the player has lost. Conversely, if the boss's health points reach zero, the game state reflects the boss's defeat.

        // Check if player or boss has lost
        if self.player_lives == 0 {
            // Player lost
            self.game_state = "Player lost.".to_string();
            return BattleShowdownEvent::PlayerLost {
                id: self.player_id,
                boss_lives: self.boss_lives,
                character_type: self.player_character_type,
                message: "".to_string(),
                player_lives: self.player_lives,
            };
        } else if self.boss_lives == 0 {
            // Boss lost
            self.game_state = "Boss lost.".to_string();
            return BattleShowdownEvent::BossLost {
                boss_lives: self.boss_lives,
                character_type: self.player_character_type,
                player_lives: self.player_lives,
                message: "You've defeated the boss".to_string(),
            };
        }

        self.game_state = "The game continues.".to_string();
        // Return event indicating attack occurred
        BattleShowdownEvent::Attacked {
            boss_lives: self.boss_lives,
            character_type: self.player_character_type,
            id: self.player_id,
            name: self.player_name.clone(),
            player_lives: self.player_lives,
        }

Finally, if neither the player nor the boss has lost, the game state is updated to indicate that the battle continues.

Understanding the State()

#[no_mangle]
extern "C" fn state() {
    let battle_showdown = unsafe {
        BATTLESHOWNDOWN
            .take()
            .expect("Unexpected error in taking state")
    };

    msg::reply(battle_showdown, 0).expect("Unable to share the state");
}

For this instance there's nothing more to share, it retrieves the current state of the game, represented by the BattleShowdown struct, from a static mutable variable BATTLESHOWNDOWN, and sends a reply message containing the game state back to the player. If there is an error sending the reply message, it will panic with an error message indicating the inability to share the state.

And that's that for this project. There are some exciting features you can consider if you want to extend this project. Imagine the possibility of resetting the game state, accommodating multiple players, or even resetting the game for a single player. And for the ambitious, you could even tackle the challenge of resetting the state for the entire game. These additions can offer new dimensions to the project and provide excellent opportunities for you to challenge yourself.

Short Recording of what we've built - Demo

In the video you could see I added another method for resetting everything back to it inital state. Though I didn't guide you through the process of doing that, you should know it is easy to implement, and I've added a GitHub repository for the entire code.

Conclusion

As demonstrated, developing a smart contract with Gear Protocol becomes straightforward once you grasp the communication message concepts. By following the steps outlined, you can start building your own projects with confidence.

While this article didn't delve into handling transactions such as token transfers, minting, or NFTs, I will cover these topics in a future article.

For now, you can explore the repository of the project we built together: Battle-Showdown, and if you have any question to ask, feel free to reach @rockyessel on X.

In this tutorial, I aim to help you understand how they actually work, the many ways they're usually used, and how powerful they are by writing a JSON parser that heavily uses these features.

Disclaimer

The goal of this tutorial is to create a real-world library that extensively uses match patterns and iterators. The goal is not to write a performant or fully-compliant JSON parser.

If you're very familiar with JSON, you will notice many things that are missing in this code, the biggest one being error handling when invalid tokens are encountered, and giving feedback to the user or helpful suggestions on what's wrong with the JSON.

This program also doesn't handle escape characters and sequences within string literals, for example. The code, for the most part, assumes that you have a valid JSON.

Prerequisites

Although this tutorial can be consumed by Rust programmers of any experience, previous experience or understanding of basic iterators and match patterns in Rust is helpful.

It is also assumed that you're familiar with the most basic Rust concepts such as traits, structs, enums, for loops, impl blocks, and so on. The tutorial does introduce you to iterator and match, so you don't need to be familiar with those to benefit from this tutorial.

Table Of Contents

1. What are Iterators in Rust?
   1. How to implement iterators in Rust
   2. What are peekable iterators in Rust?
2. What is The Match Statement in Rust?
   1. How to use iterators in match statements in Rust
   2. What are match guards in Rust?
   3. What is binding in Rust?
3. How to Build a JSON Parser – Stage 1: Reader
   1. What is the UTF-8 byte encoding?
   2. How to read the data
   3. How to implement the iterator for JsonReader
4. How to Build a JSON Parser – Stage 2: Prepare Intermediate Data Types
   1. The value type
   2. How to add helpful conversion methods
5. How to Build a JSON Parser – Stage 3: Tokenization
   1. How to define expected valid tokens
   2. How to implement the tokenizer struct
   3. How to tokenize an iterator of characters
   4. How to parse string tokens
   5. How to parse number tokens
   6. How to parse boolean tokens
   7. How to parse Null Literal
   8. How to parse delimiters
   9. How to parse a terminating character
6. How to Build a JSON Parser – Stage 4: From Tokens To Value
   1. How to parse primitives
   2. How to parse arrays
   3. How to parse objects
7. How to Use the JSON parser
8. Wrapping Up

What are Iterators in Rust?

Iterators are not a new concept, neither are they specific to Rust. It's both a pattern that is also implemented as an object in most programming languages for working with lists (such as arrays or vectors) or collections (such as HashMaps) that allows you to traverse through these data types and act on individual entries in them.

In Rust, iterators are a very powerful feature. The official Rust book describes it as:

The iterator pattern allows you to perform some task on a sequence of items in turn. An iterator is responsible for the logic of iterating over each item and determining when the sequence has finished. When you use iterators, you don’t have to reimplement that logic yourself.

In Rust, iterators are lazy, meaning they have no effect until you call methods that consume the iterator to use it up.

An iterator is an object that facilitates sequential access to the elements of a collection, like an array or a vector, without exposing the implementation details.

How to implement iterators in Rust

Iterators are implemented in Rust using a collection of traits, the most basic of which is the Iterator trait. It is implemented for all collections in the standard library, and can be implemented for custom types as well.

It requires the implementation of a single method: next(). This method returns an Option<T>, where T is the type of element the iterator is for. When next() is called (the call is implicit in most cases and you generally use higher level methods), the iterator produces Some(value) for the next element in the sequence or None when the iteration is complete. In most cases, whether the value is Some or None is also implicit.

For example, anything that implements the Iterator trait, can be used with a for loop directly, which implicitly handles both calling the next method as well as handling whether the value is Some or None. A None value triggers the loop to end. This is true for the built-in types such as arrays, slices, vectors, and hash-maps as well.

For example, let's implement the iterator trait on a simple custom type. You need to store the current state of the iterator in the type. You can also store any additional information you need. Here, we just need to know the max number after which the iteration should end:

use std::iter::Iterator;

struct CustomType {
    current: usize,
    max: usize,
}

impl CustomType {
    fn new(max: usize) -> Self {
        Self {
            current: 0,
            max,
        }
    }
}

impl Iterator for CustomType {
    type Item = usize;

    fn next(&mut self) -> Option<Self::Item> {
        if self.current >= self.max {
            None
        } else {
            self.current += 1;
            Some(self.current)
        }
    }
}

fn main() {
    let custom = CustomType::new(10);

    for item in custom {
        println!("Item is {item}");
    }
}

# Output

Item is 1
Item is 2
Item is 3
Item is 4
Item is 5
Item is 6
Item is 7
Item is 8
Item is 9
Item is 10

Rust iterators are also lazily evaluated, meaning they do not do anything unless they're used. This means that until you actually want to get the next value and do something with it, it won't even compute what the next value is.

This also means that if you have a chain of operations, such as a map and a filter, each item will go through the entire pipeline first, before the code processes the next item. This is unlike many other languages which support map and filter as methods, where first the entire map will be processed for all operations, and then the filter will be performed.

If you think about it carefully, iterators allow us to write parallel processing pipelines in a much easier way than the counterpart.

Since Iterator is just a trait, it allows for iterators to be chain-able and transformable to other iterators using various adapter methods (either the ones in standard library, or the ones that you can implement yourself).

What are peekable iterators in Rust?

Many times, you need the ability to know what the next element will be for deciding what to do, without actually modifying the iterator state for it to move to the next element. This is especially necessary when working with an iterator of tokens for parsing, as we'll do later in this tutorial.

This is where the Peekable struct comes in. You can convert any iterator into a peekable iterator by calling the peekable method on it.

Let's take the previous example and see how peekable works in action:

use std::iter::Iterator;

struct CustomType {
    current: usize,
    max: usize,
}

impl CustomType {
    fn new(max: usize) -> Self {
        Self {
            current: 0,
            max,
        }
    }
}

impl Iterator for CustomType {
    type Item = usize;

    fn next(&mut self) -> Option<Self::Item> {
        if self.current >= self.max {
            None
        } else {
            self.current += 1;
            Some(self.current)
        }
    }
}

fn main() {
    let mut custom = CustomType::new(2).peekable();

    let first = custom.peek();
    println!("{first:?}");

    let second = custom.next();
    println!("{second:?}");

    let third = custom.next();
    println!("{third:?}");

    let fourth = custom.next();
    println!("{fourth:?}");
}

# Output

Some(1)
Some(1)
Some(2)
None

I also wanted to show you how you can use iterators manually without a for loop, which is why you see all the calls to next method, and also that it returns Option instead of the value directly.

Also notice that the first and second variables are both Some(1). This is because we called peek the first time which returned the first element but without modifying the state of the iterator.

What is The Match Statement in Rust?

The match statement is a pattern-matching syntax in Rust that allows you to conditionally run code based on complex conditions in a concise syntax. You can think of it as a switch statement from other languages, but on steroids.

A very simple example of a match statement is:

let value = true;

match value {
    true => {
        println!("Value is true")
    },
    false => {
        println!("Value is false")
    }
}

The various conditions defined above, namely true and false, are known as branches. Each branch can have a single match, multiple matches separated by the pipe | operator, and ranges. They also can have guards and binding for each arm. Let's see what each of these mean:

// Multiple conditions per branch

let value = "some_string";

match value {
    "some_string1" | "some_string2" | "some_string3" => {
        println!("Bad match");
    }
    "some_string" => {
        println!("Good match");
    }
    _ => {
        println!("No match");
    }
}

Notice the _ branch in the above example. Match statements require you to cover all possible cases. In the first example, since the value was a boolean, there were only two possible values, true and false. This is why in the first case, we already covered all possible values.

In the 2nd example however, the value we're matching against is a string ( &str to be more precise). A string can be any value. It's impossible to write a match statement that can ever cover all possible cases for this example. Good thing is, Rust has a special matcher _ that matches any value.

If you're experienced with JavaScript or C (or many other languages that have the traditional switch syntax), _ is equivalent to the default case in switch, but you don't need to use _, you can also bind it to a variable and handle it differently. We'll see how to do this shortly.

How to use iterators in match statements in Rust

A match statement allows you to use iterators as branches. A successful match occurs if the value being matched is one of the values in the iterator. For example, say you are matching if a char type is a digit or not. You can write a simple iterator of characters that contains all the digit characters and use that as a branch:

let value: char = '5';

match value {
    '0'..='9' => {
        println!("Character is a digit");
    }
    _ => {
        println!("Character is not a digit");
    }
}

The above example will print "Character is a digit". If you're not familiar with the ..= syntax, it's a shorthand to create iterators over a range. In the example above, the iterator starts at '0' character and ends at '9' character, including all of the characters in between.

You can also use 1..5 to create a iterator over the range between 1 and 5 but excluding 5, so that the iterator will contain 1, 2, 3, 4.

You can also use a variable that holds the iterator as the value, meaning that the iterators do not need to be created inline:

let list = vec!["1, 2", "3, 4"].iter();
    let value = "3, 4";

    match value {
        list => {
            println!("Matched");
        }
        _ => {
            println!("No matches");
        }
    }

Note that the example calls .iter() on the vec to store the iterator in list variable and not the vector. Match arms cannot have method calls, so it's important to convert the value to an iterator outside of the match statement.

What are match guards in Rust?

Guards in match statements are additional conditions for a particular branch that the branch must satisfy to consider a successful match. For example, if you want to match over a range of numbers, but also match on whether they're odd or even, match guards can come in handy.

The syntax is also pretty intuitive. It is of the form <pattern> if <condition> => {}.

let value: u8 = 5;

match value {
    0..=9 if value % 2 == 0 => {
        println!("Value is even");
    }
    0..=9 if value % 2 == 1 => {
        println!("Value is odd");
    }
    _ => {
        println!("Unexpected value");
    }
}

The above code will print "Value is odd".

What is binding in Rust?

Binding allows you to store values in variables that can be used within the branch where the binding is present. It's basically assigning variables to certain parts of the pattern.

Pattern Binding

A very simple example is binding the catch-all pattern to a variable instead of ignoring its value using _.

let value: u8 = 5;

match value {
    0..=9 if value % 2 == 0 => {
        println!("Value is even");
    }
    0..=9 if value % 2 == 1 => {
        println!("Value is odd");
    }
    other_value => {
        println!("Unexpected value: {other_value}");
    }
}

Notice that in this example, we used the variable other_value to bind the value of value in the last pattern, which is a catch-all if it doesn't match any of the previous patterns. We can then use the variable in logic for that arm. Here we just print it the console.

Some other examples of binding are:

let value: Option<i32> = Some(43);

match value {
    Some(matched_value) => println!("The value is {matched_value}"),
    None => println!("There is no value")
}

In this example, we bound the value within the Some pattern for storing the inner value of the option, and use it in our logic.

pub struct Person {
    name: String,
    age: u32,
}

let value: Option<Person> = Some(Person {
    name: "Name".to_string(),
    age: 23,
});

match value {
    Some(Person { name: person_name, age }) => {
        println!("{person_name} is {age} years old");
    },
    None => {
        println!("The value is empty");
    }
}

We see two different types of binding in this example. The first is assigning a different name to a struct field by destructuring it ( name field), and the other is using the same name as the name of the field ( age field).

The @ Binding

The official Rust book describes it as:

The at operator @ lets us create a variable that holds a value at the same time as we’re testing that value for a pattern match.

In our example of pattern matching against a range of values, or against an iterator, we can bind the matched value to a variable using this syntax to use it within that branch:

let value: u8 = 5;

match value {
    digit @ 0..=9 => {
        println!("The matched value is {digit}");
    }
    _ => {
        println!("Unexpected value");
    }
}

Here we are binding the matched value from the iterator to the digit variable, that we then use within the branch to read the actual value.

How to Build a JSON Parser – Stage 1: Reader

Before we can parse incoming JSON data, we need to be able to read it in a way which facilitates parsing it. To be able to tokenize the incoming JSON, we need to analyse each character as they come in, and based on whether they represent a literal value, or a delimiter, (or an invalid value), decide what to do with them as well as subsequent characters.

This is a really good use case for using a combination of iterators and Rust's match syntax.

Our reader needs to hold two pieces of data. A buffered reader using which we can iterate over the input, and a character_buffer, which will hold the current character being decoded.

At this point, you may ask why we need to hold the character buffer in the reader and the reason is that JSON is UTF-8 encoded.

What is the UTF-8 byte encoding?

A UTF-8 character can be anywhere between 1 byte to 4 bytes long. We need to be able to parse all of the valid characters because the JSON spec supports these characters. This means that JSON characters can be either of 1-byte, 2-bytes, 3-bytes or 4-bytes long.

For each iteration, we need to read 4 bytes at a time, decide how many characters the 4 bytes contain (for example, these 4 bytes can contain 4 1-byte characters), finish iterating over them and then move on to the reading next 4 bytes and repeating the process. To store this intermediary piece of information, we need the character buffer.

It is also possible that we only have part of the character in the current 4 byte. For example, if you consider 2 1-byte characters followed by 1 3-byte character like 23€, the first 4 bytes will contain 2 valid characters and only part of the next valid character. You also need to be able to handle this, which will involve rewinding the iterator.

It's possible to handle this in a way where we do not need allocations, and for performance reasons it's in fact better to do so. But I will leave that to you as a reader to think about how to implement it in this case, as it is not the focus of this article.

I hope that it's now clear why we iterators are the best tool for the job here.

How to read the data

We are going to support two different readers. One is directly from a buffered reader (which is most commonly created from a file), and the other is from a iterator over bytes.

These are going to be pretty straightforward. For reading from a file, you need to create a buffered cursor over the underlying file data:

let file = File::create("dummy.json").unwrap();
let reader = BufReader::new(file);

Let's start by implementing the JSON Reader struct and these methods on it:

// src/reader.rs

use std::collections::VecDeque;
use std::io::{BufReader, Cursor, Read, Seek};
use std::str::from_utf8;

/// A struct that handles reading input data to be parsed and
/// provides an iterator over said data character-by-character.
pub struct JsonReader<T>
where
    T: Read + Seek,
{
    /// A reference to the input data, which can be anything
    /// that implements [`Read`]
    reader: BufReader<T>,

    /// A character buffer that holds queue of characters to
    /// be used by the iterator.
    ///
    /// This is necessary because UTF-8 can be 1-4 bytes long.
    /// Because of this, the reader always reads 4 bytes at a 
    /// time. We then iterate over "characters", irrespective of 
    /// whether they are 1 byte long, or 4.
    ///
    /// A [`VecDeque`] is used instead of a normal vector 
    /// because characters need to be read out from the start 
    /// of the buffer.
    character_buffer: VecDeque<char>,
}

impl<T> JsonReader<T>
where
    T: Read + Seek,
{
    /// Create a new [`JsonReader`] that reads from a file
    ///
    /// # Examples
    ///
    ///

/// use std::fs::File; /// use std::io::BufReader; /// use json_parser::reader::JsonReader; /// /// let file = File::create("dummy.json").unwrap(); /// let reader = BufReader::new(file); /// /// let json_reader = JsonReader::new(reader); /// ``` pub fn new(reader: BufReader) -> Self { JsonReader { reader, character_buffer: VecDeque::with_capacity(4), } }

/// Create a new [JsonReader] that reads from a given byte stream /// /// # Examples /// /// /// use std::io::{BufReader, Cursor}; /// use json_parser::reader::JsonReader; /// /// let input_json_string = r#"{"key1":"value1","key2":"value2"}"#; /// /// let json_reader = JsonReader::<Cursor<&'static [u8]>>::from_bytes(input_json_string.as_bytes()); ///

#[must_use] pub fn from_bytes(bytes: &[u8]) -> JsonReader> { JsonReader { reader: BufReader::new(Cursor::new(bytes)), character_buffer: VecDeque::with_capacity(4), } } }

### How to implement the iterator for `JsonReader`

Next, you are going to need to implement the `Iterator` trait on this `JSONReader` which will facilitate parsing.

First, if the character buffer isn't empty already, you can return the first character in buffer from iterator:

```rust
if !self.character_buffer.is_empty() {
    return self.character_buffer.pop_front();
}

If it is empty, you need to create a new buffer and read into that buffer from the reader:

let mut utf8_buffer = [0, 0, 0, 0];
let _ = self.reader.read(&mut utf8_buffer);

Here, you are creating a new array of size 4, and you'll be reading 4 bytes into it from the reader.

Next, you need to parse it as UTF-8. Rust provides you with a from_utf8 function that will try to parse the given bytes as UTF-8. It returns a string containing parsed characters if it was valid.

It returns an error with number of invalid bytes as part of the error information, which you can use to backtrack the reader to only retain the valid characters, and try the next 4 characters from the point of failure.

If that didn't make too much sense, looking at the code will make things clear:

match from_utf8(&utf8_buffer) {
    Ok(string) => {
        self.character_buffer = string.chars().collect();
        self.character_buffer.pop_front()
    }
    Err(error) => {
        // Read valid bytes, and rewind the buffered reader for 
        // the remaining bytes so that they can be read again in the
        // next iteration.

        let valid_bytes = error.valid_up_to();
        let string = from_utf8(&utf8_buffer[..valid_bytes]).unwrap();

        let remaining_bytes = 4 - valid_bytes;

        let _ = self.reader.seek_relative(-(remaining_bytes as i64));

        // Collect the valid characters into character_buffer
        self.character_buffer = string.chars().collect();

        // Return the first character from character_buffer
        self.character_buffer.pop_front()
    }
}

Here's the complete implementation of the Iterator trait:

// src/reader.rs

impl<T> Iterator for JsonReader<T>
where
    T: Read + Seek,
{
    type Item = char;

    #[allow(clippy::cast_possible_wrap)]
    fn next(&mut self) -> Option<Self::Item> {
        if !self.character_buffer.is_empty() {
            return self.character_buffer.pop_front();
        }

        let mut utf8_buffer = [0, 0, 0, 0];
        let _ = self.reader.read(&mut utf8_buffer);

        match from_utf8(&utf8_buffer) {
            Ok(string) => {
                self.character_buffer = string.chars().collect();
                self.character_buffer.pop_front()
            }
            Err(error) => {
                // Read valid bytes, and rewind the buffered reader for
                // the remaining bytes so that they can be read again in the
                // next iteration.

                let valid_bytes = error.valid_up_to();
                let string = from_utf8(&utf8_buffer[..valid_bytes]).unwrap();

                let remaining_bytes = 4 - valid_bytes;

                let _ = self.reader.seek_relative(-(remaining_bytes as i64));

                // Collect the valid characters into character_buffer
                self.character_buffer = string.chars().collect();

                // Return the first character from character_buffer
                self.character_buffer.pop_front()
            }
        }
    }
}

And that's all you need to do for reading the input data for parsing. It's time to move on to the next stage in the process.

How to Build a JSON Parser – Stage 2: Prepare Intermediate Data Types

This isn't really a stage in the parsing pipeline, but it is a prerequisite for the next steps. We need to define Rust types that map to all of the possible types that JSON supports.

JSON supports the following data types:

• String
• Number
• Boolean
• Array
• Object
• Null

A number can further be either an integer, or a floating-point number. While you can use f64 as the Rust type for all JSON numbers, practically it's not feasible without littering your code with type casts everywhere when you try to use it.

So in this tutorial, we're going to indeed make that distinction and record that fact.

The value type

Enums are the ideal way to store state like this, where each variant needs to have some identifier as metadata (in this case the type of JSON value), and optionally some data attached to it. The data you're going to attach to these variants will be the actual value of that type in JSON.

// src/value.rs

use std::collections::HashMap;

#[derive(Debug, Copy, Clone, PartialEq)]
pub enum Number {
    I64(i64),
    F64(f64),
}

#[derive(Debug, PartialEq, Clone)]
pub enum Value {
    String(String),
    Number(Number),
    Boolean(bool),
    Array(Vec<Value>),
    Object(HashMap<String, Value>),
    Null,
}

The first few variants are pretty straightforward, you define the variant and the data it holds is a corresponding Rust type. The last variant is even simpler, representing the null value which doesn't need further data to be stored.

The Array and Object variants though are a bit more interesting, since they are recursively storing the Enum itself. This makes sense, as arrays in JSON can have any value type that JSON spec supports. And objects in JSON always have string keys and any JSON supported value, including other objects.

How to add helpful conversion methods

You will also need a way to convert the enum type into the underlying types, and throw an error if the underlying data isn't what you expected. This is mostly boilerplate code, so I'm just going to put it all together without further explanation:

// src/value.rs

impl TryFrom<&Value> for String {
    type Error = ();

    fn try_from(value: &Value) -> Result<Self, ()> {
        match value {
            Value::String(value) => Ok(value.clone()),
            _ => Err(()),
        }
    }
}

impl TryFrom<&Value> for i64 {
    type Error = ();

    #[allow(clippy::cast_possible_truncation)]
    fn try_from(value: &Value) -> Result<Self, ()> {
        match value {
            Value::Number(value) => match value {
                Number::I64(value) => Ok(*value),
                Number::F64(value) => Ok(*value as i64),
            },
            _ => Err(()),
        }
    }
}

impl TryFrom<&Value> for f64 {
    type Error = ();

    fn try_from(value: &Value) -> Result<Self, ()> {
        match value {
            Value::Number(value) => match value {
                Number::F64(value) => Ok(*value),
                Number::I64(value) => Ok(*value as f64),
            },
            _ => Err(()),
        }
    }
}

impl TryFrom<&Value> for bool {
    type Error = ();

    fn try_from(value: &Value) -> Result<Self, ()> {
        match value {
            Value::Boolean(value) => Ok(*value),
            _ => Err(()),
        }
    }
}

impl<'a> TryFrom<&'a Value> for &'a Vec<Value> {
    type Error = ();

    fn try_from(value: &'a Value) -> Result<Self, ()> {
        match value {
            Value::Array(value) => Ok(value),
            _ => Err(()),
        }
    }
}

#[allow(clippy::implicit_hasher)]
impl<'a> TryFrom<&'a Value> for &'a HashMap<String, Value> {
    type Error = ();

    fn try_from(value: &'a Value) -> Result<Self, ()> {
        match value {
            Value::Object(value) => Ok(value),
            _ => Err(()),
        }
    }
}

How to Build a JSON Parser – Stage 3: Tokenization

The next step is to take the input data and tokenize it.

Tokenization is the process of splitting a large chunk of input into smaller, more digestible units that can then be analysed independently. This also allows you to work with them much more easily than just byte streams and they help represent the incoming data as a standard form, and allow for mapping tokens to output value types.

The parser can then recursively process all tokens until there's nothing to process, giving us the parsed data once it finishes.

How to define expected valid tokens

There is going to be some duplication here compared to the value type you looked at previously, but that's to be expected, since the token representation of any literal value will be that value itself. There's no way to break it down to smaller units in that case.

Once again, Enum is the right data type for this since we need both metadata (as the token type), and optionally data associated with it.

The tokens representing literal values can be defined in this way:

// src/token.rs

use std::io::{Read, Seek};
use std::iter::Peekable;
use crate::reader::JsonReader;

#[derive(Debug, Copy, Clone, PartialEq)]
pub enum Number {
    I64(i64),
    F64(f64),
}

#[derive(Debug, Clone, PartialEq)]
pub enum Token {
    String(String),
    Number(Number),
    Boolean(bool),
    Null,
}

Apart from these, we also have a lot of other tokens in JSON that form the "grammar" of the JSON format. These are:

• Curly braces ({ or } ) that represent opening and closing of an object respectively.
• Square brackets ([ or ]) that represent opening and closing of an array respectively.
• Colon (:) for separating key-value pairs within the object.
• Comma (,) for separating values.
• Quotes (") that represent opening/closing of the string literal values.

All of these do not need to have any data associated with them, so they're going to be unit variants in the enum. Adding these in, the complete enum will be:

// src/token.rs

use std::io::{Read, Seek};
use std::iter::Peekable;
use crate::reader::JsonReader;
use crate::value::Number;

#[derive(Debug, Clone, PartialEq)]
pub enum Token {
    CurlyOpen,
    CurlyClose,
    Quotes,
    Colon,
    String(String),
    Number(Number),
    ArrayOpen,
    ArrayClose,
    Comma,
    Boolean(bool),
    Null,
}

How to implement the tokenizer struct

You are going to need a JsonTokenizer struct that can facilitate the process while also responsible for holding the state of the tokenizer process:

// src/token.rs

pub struct JsonTokenizer<T>
    where
        T: Read + Seek,
{
    tokens: Vec<Token>,
    iterator: Peekable<JsonReader<T>>,
}

impl<T> JsonTokenizer<T>
where
    T: Read + Seek,
{
    pub fn new(reader: File) -> JsonTokenizer<File> {
        let json_reader = JsonReader::<File>::new(BufReader::new(reader));

        JsonTokenizer {
            iterator: json_reader.peekable(),
            tokens: vec![],
        }
    }

    pub fn from_bytes<'a>(input: &'a [u8]) -> JsonTokenizer<Cursor<&'a [u8]>> {
        let json_reader = JsonReader::<Cursor<&'a [u8]>>::from_bytes(input);

        JsonTokenizer {
            iterator: json_reader.peekable(),
            tokens: Vec::with_capacity(input.len()),
        }
    }
}

In this case, we've made it generic over where the input comes from. The type T needs to implement Read & Seek traits, the reason for which is explained shortly.

The iterator also needs to be Peekable, which basically means we should be able to read the next item in the iterator without advancing the iterator itself.

How to tokenize an iterator of characters

Once you've defined all of the expected tokens, you need to take your character iterator and convert it into a list of tokens, where each entry is a variant of the Token enum defined in the last section.

We'll start by writing a skeleton function that matches on the incoming character and panics if it encounters an invalid token:

// src/token.rs

impl<T> JsonTokenizer<T> where
    T: Read + Seek, {
    pub fn tokenize_json(&mut self) -> Result<&[Token], ()> {
        while let Some(character) = self.iterator.peek() {
            match *character {
                // Parse all other tokens here
                // ...
                character => {
                    if character.is_ascii_whitespace() {
                        continue;
                    }

                    panic!("Unexpected character: ;{character};")
                }
            }
        }

        Ok(&self.tokens)
    }
}

There are two noteworthy things here, let's start with the easy one. If your match block doesn't encounter any known characters (you will implement this shortly), you need to have a "catch-all" condition that matches any character.

Here, we are going to ignore any whitespace characters and continue to the next iteration if it encounters one. If the character isn't a whitespace, then you need to panic (or return error) here.

The next noteworthy thing here is self.iterator.peek(). To facilitate parsing of different kinds of tokens from delimiters to literal values, it is important that the iterator is not advanced when reading out the next character. This needs to happen so that you can conditionally advance it based on what character is next.

You also need to delegate parsing of certain sets of tokens to different functions, which will have their own logic of advancing the iterator.

A good example is parsing the null literal value. If the match encounters a n character and is not within a string, object, number, and so on, then you need to ensure that the next three characters are u, l, l respectively to form the literal value null and then advance the iterator by four so that the next loop starts parsing after the null character and not in the middle of it.

How to parse string tokens

We're going to start by parsing strings. Let's stop for a second and think what needs to happen step-by-step:

• Check if match encounters a " character. If it does, push Token::Quote to your list of output tokens.
• Advance the iterator by one so the next steps start from after the " character.
• Parse all characters as part of the string until you encounter another " character which indicates closing of the string value.
• Advance the iterator by however many characters are parsed as part of the string, and one addition to also jump over the closing " character.
• Push Token::String with the parsed value to your list of output tokens.
• Push Token::Quote to your list of output tokens.

Hopefully, that isn't too confusing. But the code should help you understand it better:

// src/token.rs

impl<T> JsonTokenizer<T>
    where
        T: Read + Seek,
{
    // ...

    pub fn tokenize_json(&mut self) -> Result<&[Token], ()> {
        while let Some(character) = self.iterator.peek() {
            match *character {
                '"' => {
                    // Pushed opening quote to output tokens list.
                    self.tokens.push(Token::Quotes);

                    // Skip quote token since we already added it to the tokens list.
                    let _ = self.iterator.next();

                    // Delegate parsing string value to a separate function.
                    // The function should also take care of advancing the iterator properly.
                    let string = self.parse_string();

                    // Push parsed string to output tokens list.
                    self.tokens.push(Token::String(string));

                    // Pushed closing quote to output tokens list.
                    self.tokens.push(Token::Quotes);
                }
                // ...
            }
        }

        Ok(&self.tokens)
    }

    fn parse_string(&mut self) -> String {
        // Create new vector to hold parsed characters.
        let mut string_characters = Vec::<char>::new();

        // Take each character by reference so that they
        // aren't moved out of the iterator, which will
        // require you to move the iterator into this
        // function.
        for character in self.iterator.by_ref() {
            // If it encounters a closing `"`, break
            // out of the loop as the string has ended.
            if character == '"' {
                break;
            }

            // Continue pushing to the vector to build
            // the string.
            string_characters.push(character);
        }

        // Create a string out of character iterator and
        // return it.
        String::from_iter(string_characters)
    }
}

As I've previously mentioned, we're not going to look at handling escape characters in this tutorial, as they do not add much value towards learning the topic at hand, but if you're interested, it will be a good exercise for you to add that in on top of the implementation.

That takes care of parsing the string, we can move on to a more interesting value type next.

How to parse number tokens

Numbers in JSON spec have a lot of variation. They can either be positive or negative, and integers or decimals. They can also be defined as scientific notation (for example negative exponential 3.7e-5 or positive exponential 3.7e5). And we need to parse all of these variations.

As always, we'll start with the easy bit. If we encounter any character that can be a valid character in number, you need to delegate parsing to a parse_number function. But also, any valid number can only start with either a digit, or a negative sign. A number cannot begin with a decimal character or an epsilon character, so it makes things easier for us.

// src/token.rs

impl<T> JsonTokenizer<T>
    where
        T: Read + Seek,
{
    // ...

    pub fn tokenize_json(&mut self) -> Result<&[Token], ()> {
        while let Some(character) = self.iterator.peek() {
            match *character {
                // ...

                '-' | '0'..='9' => {
                    let number = self.parse_number()?;
                    self.tokens.push(Token::Number(number));
                }

                // ...
            }
        }

        Ok(&self.tokens)
    }

    // ...
}

Next, we'll implement the parse_number method:

// src/token.rs

impl<T> JsonTokenizer<T>
    where
        T: Read + Seek,
{
    // ...

    fn parse_number(&mut self) -> Result<Number, ()> {
        // Store parsed number characters.
        let mut number_characters = Vec::<char>::new();

        // Stores whether the digit being parsed is after a `.` character
        // making it a decimal.
        let mut is_decimal = false;

        // Stores the characters after an epsilon character `e` or `E`
        // to indicate the exponential value.
        let mut epsilon_characters = Vec::<char>::new();

        // Stores whether the digit being parsed is part of the epsilon
        // characters.
        let mut is_epsilon_characters = false;

        while let Some(character) = self.iterator.peek() {
            match character {
                // Match the negative sign character that indicates whether number is negative
                '-' => {
                    if is_epsilon_characters {
                        // If it's parsing epsilon characters, push it to the epsilon
                        // character set.
                        epsilon_characters.push('-');
                    } else {
                        // Otherwise, push it to normal character set.
                        number_characters.push('-');
                    }

                    // Advance the iterator by 1.
                    let _ = self.iterator.next();
                }
                // Match a positive sign, which can be treated as redundant and ignored since
                // positive is the default.
                '+' => {
                    // Advance the iterator by 1.
                    let _ = self.iterator.next();
                }
                // Match any digit between 0 and 9, and store it into the `digit`
                // variable.
                digit @ '0'..='9' => {
                    if is_epsilon_characters {
                        // If it's parsing epsilon characters, push it to the epsilon
                        // character set.
                        epsilon_characters.push(*digit);
                    } else {
                        // Otherwise, push it to normal character set.
                        number_characters.push(*digit);
                    }
                    // Advance the iterator by 1.
                    let _ = self.iterator.next();
                }
                // Match the period character which indicates start of the fractional
                // part of a decimal number.
                '.' => {
                    // Push the decimal character to numbers character set.
                    number_characters.push('.');

                    // Set the current state of number being decimal to true.
                    is_decimal = true;

                    // Advance the iterator by 1.
                    let _ = self.iterator.next();
                }
                // Match any of the characters that can signify end of the number
                // literal value. This can be a comma which separates key-value pair,
                // closing object character, closing array character, or a `:` which
                // separates a key from its value.
                '}' | ',' | ']' | ':' => {
                    break;
                }
                // Match the epsilon character which indicates that the number is in
                // scientific notation.
                'e' | 'E' => {
                    // Panic if it's already parsing an exponential number since this would
                    // mean there are 2 epsilon characters which is invalid.
                    if is_epsilon_characters {
                        panic!("Unexpected character while parsing number: {character}. Double epsilon characters encountered");
                    }

                    // Set the current state of number being in scientific notation to true.
                    is_epsilon_characters = true;

                    // Advance the iterator by 1.
                    let _ = self.iterator.next();
                }
                // Panic if any other character is encountered.
                other => {
                    if !other.is_ascii_whitespace() {
                        panic!("Unexpected character while parsing number: {character}")
                    } else {
                        self.iterator.next();
                    }
                },
            }
        }

        if is_epsilon_characters {
            // if the number is an exponential, perform the calculations to convert it
            // to a floating point number in rust.

            // Parse base as floating point number.
            let base: f64 = String::from_iter(number_characters).parse().unwrap();

            // Parse exponential as floating point number.
            let exponential: f64 = String::from_iter(epsilon_characters).parse().unwrap();

            // Return the final computed decimal number.
            Ok(Number::F64(base * 10_f64.powf(exponential)))
        } else if is_decimal {
            // if the number is a decimal, parse it as a floating point number in rust.
            Ok(Number::F64(
                String::from_iter(number_characters).parse::<f64>().unwrap(),
            ))
        } else {
            // Parse the number as an integer in rust.
            Ok(Number::I64(
                String::from_iter(number_characters).parse::<i64>().unwrap(),
            ))
        }
    }
}

It is advisable for you to go through the code and read the comments to understand this function. You shouldn't encounter any new syntax that is not either covered already or assumed to be known by the reader.

How to parse boolean tokens

Parsing booleans is going to be the simplest one we look at so far. All we need to do is match t or f as the first character, and then check the next few characters to ensure they form the literal value true or false.

// src/token.rs

impl<T> JsonTokenizer<T>
    where
        T: Read + Seek,
{
    // ...

    pub fn tokenize_json(&mut self) -> Result<&[Token], ()> {
        while let Some(character) = self.iterator.peek() {
            match *character {
                // ...

                // Match `t` character which indicates beginning of a boolean literal.
                't' => {
                    // Advance iterator by 1.
                    let _ = self.iterator.next();

                    // Assert next character is `r` while advancing the iterator by 1.
                    assert_eq!(Some('r'), self.iterator.next());
                    // Assert next character is `u` while advancing the iterator by 1.
                    assert_eq!(Some('u'), self.iterator.next());
                    // Assert next character is `e` while advancing the iterator by 1.
                    assert_eq!(Some('e'), self.iterator.next());

                    // Push the literal value to token list.
                    self.tokens.push(Token::Boolean(true));
                }
                'f' => {
                    // Advance iterator by 1.
                    let _ = self.iterator.next();

                    // Assert next character is `a` while advancing the iterator by 1.
                    assert_eq!(Some('a'), self.iterator.next());
                    // Assert next character is `l` while advancing the iterator by 1.
                    assert_eq!(Some('l'), self.iterator.next());
                    // Assert next character is `s` while advancing the iterator by 1.
                    assert_eq!(Some('s'), self.iterator.next());
                    // Assert next character is `e` while advancing the iterator by 1.
                    assert_eq!(Some('e'), self.iterator.next());

                    // Push the literal value to token list.
                    self.tokens.push(Token::Boolean(false));
                }

                // ...
            }
        }

        Ok(&self.tokens)
    }
}