In the past year, frontier AI labs such as OpenAI, xAI, Anthropic, Meta, and Google have all released real-time voice services. Yet voice apps also have the highest requirements for latency, privacy, and customization. It’s difficult to have a one-size-fits-all voice AI solution.

In this article, we’ll explore how to use open-source technologies to create voice AI agents that utilize your custom knowledge base, voice style, actions, fine-tuned AI models, and run on your own computer.

What We’ll Cover:

• Prerequisites

• What it Looks Like

• Two Voice AI Approaches

• The Voice AI Orchestrator

  • Configure an ASR

  • Run and configure a VAD

  • Configure an LLM

  • Configure a TTS

  • Configure MCP and actions

• Local AI With LlamaEdge

• Conclusion

Prerequisites

You’ll need to have and know a few things to most effectively follow along with this tutorial:

• Access to a Linux-like system. Mac or Windows WSL suffice too.

• Be comfortable with command line (CLI) tools.

• Be able to run server applications on the Linux system.

• Have/get free API keys from Groq and ElevenLabs.

• Optional: be able to compile and build Rust source code.

• Optional: have/get an EchoKit device or assemble your own.

What it Looks Like

The key software component we will cover is the echokit_server project. It is an open-source agent orchestrator for voice AI applications. That means it coordinates services such as LLMs, ASR, TTS, VAD, MCP, search, knowledge/vector databases, and others to generate intelligent voice responses from user prompts.

The EchoKit server provides a WebSocket interface that allows compatible clients to send and receive voice data to and from it. The echokit_box project provides an ESP32-based firmware that can act as a client to collect audio from the user and play TTS-generated voice from the EchoKit server. You can see a couple of demos here. You can assemble your own EchoKit device or purchase one.

Of course, you can also use a pure software client that conforms to the echokit_server WebSocket interface. The project publishes a JavaScript web page that you can run locally to connect to your own EchoKit server as a reference.

In the rest of the article, I will discuss how it’s implemented and how to deploy the system for your own voice AI applications.

Two Voice AI Approaches

When OpenAI released its “realtime voice” services in October 2024, the consensus was that voice AI required “end-to-end” AI models. Traditional LLMs take text as input and then respond in text. The voice end-to-end models take voice audio data as input and respond in voice audio data as well. The end-to-end models could reduce latency since the voice processing, understanding, and generation are done in a single step.

But an end-to-end model is very difficult to customize. For example, it’s impossible to add your own prompt and knowledge to the context for each LLM request, or to act on the LLM's thinking or tool-call responses, or to clone your own voice for the response.

The second approach is to use an “agent orchestration” service to tie together multiple AI models, using one model’s output as the input for the next model. This allows us to customize or select each model and manipulate or supplement the model input at every step.

• The VAD model is used to detect conversation turns in the user's speech. It determines when the user is finished speaking and is now expecting a response.

• The ASR/STT model turns user speech into text.

• The LLM model generates a text response, including MCP tool calls.

• The TTS model turns the response text into voice.

The issue with multi-model and multi-step orchestration is that it can be slow. A lot of optimizations are needed for this approach to work well. For example, a useful technique is to utilize streaming input and output wherever possible. This way, each model doesn’t have to wait for the complete response from the upstream model.

The EchoKit server is a stream-everything, highly efficient AI model orchestrator. It is entirely written in Rust for stability, safety, and speed.

The Voice AI Orchestrator

The EchoKit server project is an open-source AI service orchestrator focused on real-time voice use cases. It starts up a WebSocket server that listens for streaming audio input and returns streaming audio responses.

You can build the echokit_server project yourself using the Rust toolchain. Or, you can simply download the pre-built binary for your computer.

# for x86 / AMD64 CPUs
curl -LO https://github.com/second-state/echokit_server/releases/download/v0.1.0/echokit_server-v0.1.0-x86_64-unknown-linux-gnu.tar.gz
unzip echokit_server-v0.1.0-x86_64-unknown-linux-gnu.tar.gz

# for arm64 CPUs
curl -LO https://github.com/second-state/echokit_server/releases/download/v0.1.0/echokit_server-v0.1.0-aarch64-unknown-linux-gnu.tar.gz
unzip echokit_server-v0.1.0-aarch64-unknown-linux-gnu.tar.gz

Then, run it as follows:

nohup ./echokit_server &

It reads the config.toml file from the current directory. At the top of the file, you can configure the port on which the WebSocket server listens. You can also specify a WAV file that is downloaded to the connected EchoKit client device as a welcome message.

addr = "0.0.0.0:8000"
hello_wav = "hello.wav"

Configure an ASR

When the EchoKit server receives the user's voice data, it first sends the data to an ASR service to convert it into text.

There are many compelling ASR models available today. The EchoKit server can work with any OpenAI-compatible API providers, such as OpenAI itself, x.ai, OpenRouter, and Groq.

In our example, we use Groq’s Whisper ASR service. Whisper is a state-of-the-art ASR model released by OpenAI. Groq provides specialized hardware chips to run it very fast. You will first get a free API key from Groq. Then, configure the ASR service as follows. Notice the “prompt” for the Whisper model. It is a tried-and-true prompt to reduce hallucination of the Whisper model.

[asr]
url = "https://api.groq.com/openai/v1/audio/transcriptions"
api_key = "gsk_XYZ"
model = "whisper-large-v3"
lang = "en"
prompt = "Hello\n你好\n(noise)\n(bgm)\n(silence)\n"

Run and configure a VAD

In order to carry out a voice conversation, participants must detect each other's intentions and speak only when a turn arises. VAD (Voice Activity Detection) is a specialized AI model used to detect activities and, in particular, when the speaker has finished and expects an answer.

In EchoKit, we have VAD detection on both the device and the server.

• Device-side VAD: It detects human language. The device ignores background noise, music, keyboard sounds, and dog barking. It only sends human voice to the server.

• Server-side VAD: It processes the audio stream in 100ms (0.1s) chunks. Once it detects that the speaker has finished, it sends all transcribed text to the LLM and starts waiting for the LLM’s response stream.

The server-side VAD is optional, since the device-side VAD can also generate “conversation turn” signals. But due to the limited computing resources on the device, adding the server-side VAD can dramatically improve the overall VAD performance.

We’re porting the popular Silero VAD project from Python to Rust, and creating the silero_vad_server project. Build the project as instructed. You can start the VAD server on your EchoKit server’s port 9094 as follows:

VAD_LISTEN=0.0.0.0:9094 nohup target/release/silero_vad_server &

You might be wondering: why port to Rust? While many AI projects are written in Python for ease of development, Rust applications are often much lighter, faster, and safer at deployment. So, we’ll leverage AI tools like RustCoder to port as much Python code as possible to Rust. The EchoKit software stack is largely written in Rust.

The VAD server is a WebSocket service that listens on port 9094. As we discussed, the EchoKit server will stream audio to this WebSocket and stop the ASR when a conversation turn is detected. Therefore, we’ll add the VAD service to the EchoKit server’s ASR config section in config.toml.

[asr]
url = "https://api.groq.com/openai/v1/audio/transcriptions"
api_key = "gsk_XYZ"
model = "whisper-large-v3"
lang = "en"
prompt = "Hello\n你好\n(noise)\n(bgm)\n(silence)\n"
vad_realtime_url = "ws://localhost:9094/v1/audio/realtime_vad"

Configure an LLM

Once the ASR service transcribes the user's voice into text, the next step in the pipeline is the LLM (Large Language Model). It’s the AI service that actually “thinks” and generates an answer in text.

Again, the EchoKit server can work with any OpenAI-compatible API providers for LLMs, such as OpenAI itself, x.ai, OpenRouter, and Groq. Since the voice service is highly sensitive to speed, we’ll choose Groq again. Groq supports a number of open-source LLMs. We’ll choose the gpt-oss-20b model released by OpenAI.

[llm]
llm_chat_url = "https://api.groq.com/openai/v1/chat/completions"
api_key = "gsk_XYZ"
model = "openai/gpt-oss-20b"
history = 20

The “history” field indicates how many messages should be kept in the context. Another crucial feature of an LLM application is the “system prompt,” where you instruct the LLM how it should “behave.” You can specify the system prompt in the EchoKit server config as well.

[[llm.sys_prompts]]
role = "system"
content = """
You are a comedian. Engage in lighthearted and humorous conversation with the user. Tell jokes when appropriate.

"""

Since Groq is very fast, it can process very large system prompts in under one second. You can add a lot more context and instructions to the system prompt. For example, you can give the application “knowledge” about a specific field by putting entire books into the system prompt.

Configure a TTS

Finally, once the LLM generates a text response, the EchoKit server will call a TTS (text to speech) service to convert the text into voice and stream it back to the client device.

While Groq has a TTS service, it’s not particularly compelling. ElevenLabs is a leading TTS provider that offers hundreds of voice characters. It can express emotions and supports easy voice cloning. In the config below, you’ll put in your ElevenLabs API key and select a voice.

[tts]
platform = "Elevenlabs"
token = "sk_xyz"
voice = "VOICE-ID-ABCD"

The ElevenLabs TTS models and API services are all great, but they are not open-source. A very compelling open-source TTS, known as GPT-SoVITS, is also available.

You can port GPT-SoVITS from Python to Rust and create an open-source API server project called gsv_tts. It allows easy cloning of any voice. You can run a gsv_tts API server by following its instructions. Then, you can configure the EchoKit server to stream text to it and receive streaming audio from it.

[tts]
platform = "StreamGSV"
url = "http://gsv_tts.server:port/v1/audio/stream_speech"
speaker = "michael"

Configure MCP and actions

Of course, an “AI agent” is not just about chatting. It is about performing actions on specific tasks. For example, the “US civics test prep” use case, which I shared as an example video at the beginning of this article, requires the agent to get exam questions from a database, and then generate responses that guide the user toward the official answer. This is accomplished using LLM tools and actions.

• The LLM detects that the user is requesting a new question.

• Instead of responding in natural language, it responds with a JSON structure that instructs the agent to "get a new question and answer."

• The EchoKit server intercepts this JSON response and retrieves the question and answer from a database.

• The EchoKit server sends the question and answer back to the LLM.

• The LLM formulates a natural language response based on the question and answer.

• The EchoKit server generates a voice response using its TTS service.

As you can see, the EchoKit server needs to perform a few extra steps behind the scenes before it responds in voice. The EchoKit server leverages the MCP protocol for this. The function to look up questions and answers is provided by an open-source MCP server called ExamPrepAgent.

The MCP protocol standardizes the tools and functions for LLMs to call. There are many MCP servers available for all kinds of different tasks. ExamPrepAgent is just one of them.

We are running this MCP server on port 8003. With the MCP server up and running, you only need to add the following configuration to EchoKit server’s config.toml.

[[llm.mcp_server]]
server = "http://localhost:8003/mcp"
type = "http_streamable"

With MCP integration, the EchoKit AI agent can now perform actions. It can call APIs to send messages, make payments, or even turn electronic devices on or off.

Local AI With LlamaEdge

You’ve now seen the open-source EchoKit device working with the open-source EchoKit server to understand and respond to users in voice. But the AI models we use, while also open-source, run on commercial cloud providers. Can we run AI models using open-source technologies at home?

LlamaEdge is an open-source, cross-platform API server for AI models. It supports many mainstream LLM, ASR, and TTS models across Linux, Mac, Windows, and many CPU/GPU architectures. It’s perfect for running AI models on home or office computers. It also provides OpenAI-compatible API endpoints, which makes them very easy to integrate into the EchoKit server.

To install LlamaEdge and its dependencies, run the following shell command. It will detect your hardware and install the appropriate software that can fully take advantage of your GPUs (if any).

curl -sSf https://raw.githubusercontent.com/WasmEdge/WasmEdge/master/utils/install_v2.sh | bash -s

Then, download an open-source LLM model. I am using Google's Gemma model as an example.

curl -LO https://huggingface.co/second-state/gemma-3-4b-it-GGUF/resolve/main/gemma-3-4b-it-Q5_K_M.gguf

Download the cross-platform LlamaEdge API server.

curl -LO https://github.com/second-state/LlamaEdge/releases/latest/download/llama-api-server.wasm

Start an LLamaEdge API server with the Google Gemma LLM model. by default, it listens on localhost port 8080.

wasmedge --dir .:. --nn-preload default:GGML:AUTO:gemma-3-4b-it-Q5_K_M.gguf llama-api-server.wasm -p gemma-3

Test the OpenAI compatible API on that server.

curl -X POST http://localhost:8080/v1/chat/completions \
  -H 'accept: application/json' \
  -H 'Content-Type: application/json' \
  -d '{"messages":[{"role":"system", "content": "You are a helpful assistant. Try to be as brief as possible."}, {"role":"user", "content": "Where is the capital of Texas?"}]}'

Now, you can add this local LLM service to your EchoKit server configuration.

[llm]
llm_chat_url = "http://localhost:8080/v1/chat/completions"
api_key = "NONE"
model = "default"
history = 20

The LlamaEdge project supports more than LLMs. It runs the Whisper ASR model and the Piper TTS model as OpenAI-compatible API servers as well.

Conclusion

The voice AI agent software stack is complex and deep. EchoKit is an open-source platform that ties together and coordinates all those components. It provides a good vantage point for us to learn about the entire stack.

I can’t wait to see what you build!

But some significant barriers remain that make it difficult for them to do this:

• OpenAI provides a simple stateless API for ChatGPT. The developer needs to keep track of the history and context of each conversation in a cache or database managed by the application. The developer also needs to manage and safeguard the API keys. There is a lot of boilerplate code unrelated to the application’s business logic.

• The “natural” UI for the ChatGPT API application is a threaded chat. But it is difficult to create a “chat UI” in a traditional web or app framework. In fact, the most commonly used chat UI already exists in messaging apps like Slack, Discord, and even forums (for example, GitHub Discussions). We need a simple way to connect ChatGPT API responses to an existing messaging service.

In this article, I will show you how to create a serverless GitHub bot. The bot allows GitHub users to chat with ChatGPT and each other in GitHub Issues. You can try it by asking a question, or joining another conversation thread by leaving a comment. In other words, this project uses GitHub Issues’ threaded messages UI as its own chat UI.

Figure 1. Learning Rust with ChatGPT. see https://github.com/second-state/chat-with-chatgpt/issues/31

The bot is a serverless function written in Rust. Just fork the example, deploy your fork on flows.network, and configure it to interact with your own GitHub repos and OpenAI keys. You will have a fully functional GitHub bot in 10 minutes. There is no need to set up a web server, or a webhook for GitHub API, or a cache / database server.

How to Fork the Template Repo

First, fork this template repo from GitHub.

The src/lib.rs file contains the bot application (also known as the flow function). The run() function is called upon starting up. It listens for issue_comment and issues events from the GitHub repo owner/repo. Those events are emitted when a new issue comment or a new issue is created in the repo.

#[no_mangle]
#[tokio::main(flavor = "current_thread")]
pub async fn run() {
    // Setup variables for
    //   ower: GitHub org to install the bot
    //   repo: GitHub repo to install the bot
    //   openai_key_name: Name for your OpenAI API key
    // All the values can be set in the source code or as env vars

    listen_to_event(&owner, &repo, vec!["issue_comment", "issues"], |payload| {
        handler(&owner, &repo, &openai_key_name, payload)
    })
    .await;
}

The handler() function processes the events received by listen_to_event(). If the event is a new comment in an issue, the bot calls OpenAI's ChatGPT API to add the comment text into an existing conversation identified by the issue.number. It receives a response from ChatGPT, and adds a comment in the issue.

The flow function here automatically and transparently manages the conversation history with the ChatGPT API in a local storage. The OpenAI API key is also stored in the local storage so that instead of putting the secret text in the source code, the key can be identified by a string name in openai_key_name.

EventPayload::IssueCommentEvent(e) => {
    if e.comment.user.r#type != "Bot" {
        if let Some(b) = e.comment.body {
            if let Some(r) = chat_completion (
                    openai_key_name,
                    &format!("issue#{}", e.issue.number),
                    &b,
                    &ChatOptions::default(),
            ) {
                if let Err(e) = issues.create_comment(e.issue.number, r.choice).await {
                    write_error_log!(e.to_string());
                }
            }
        }
    }
}

If the event is a new issue, the flow function creates a new conversation identified by issue.number, and requests a response from ChatGPT.

EventPayload::IssuesEvent(e) => {
    if e.action == IssuesEventAction::Closed {
        return;
    }

    let title = e.issue.title;
    let body = e.issue.body.unwrap_or("".to_string());
    let q = title + "\n" + &body;
    if let Some(r) = chat_completion (
            openai_key_name,
            &format!("issue#{}", e.issue.number),
            &q,
            &ChatOptions::default(),
    ) {
        if let Err(e) = issues.create_comment(e.issue.number, r.choice).await {
            write_error_log!(e.to_string());
        }
    }
}

How to Deploy the Serverless Flow Function

As we can see, the flow function code calls SDK APIs to perform complex operations. For example,

• The listen_to_event() function registers a webhook URL through GitHub API so that the handler() function will be called when certain events occur in GitHub.

• The chat_completion() function calls the ChatGPT API with the named API key and past history / context of the specified conversation. The API key and conversation history are stored in a Redis cache.

The webhook server and the Redis cache are both external services the SDK depends on. That means the flow function must run inside a managed host environment that provides such external services. Flows.network is a PaaS (Platform as a Service) host for the flow function SDKs.

In order to deploy the flow function on flows.network, you simply need to import its source code to the PaaS.

First, sign into flows.network from your GitHub account. Import your forked GitHub repo that contains the flow function source code and choose "With Environment Variables".

Note that this is NOT the GitHub repo where you want to deploy the bot. This is the repo for your forked flow function source code.

Figure 2. Import the GitHub repo you forked from the flow function template into flows.network.

Set the environment variables to point the flow function to the OpenAI API key name (open_ai_key) and GitHub repo (owner and repo).

The GitHub owner and repo variables here point to the GitHub repo where you want to deploy the bot, NOT the repo for the flow function source code.

Figure 3. Set the environment variables for the GitHub repo where you want to deploy the bot, as well as the OpenAI API key name.

Flows.network will fetch the source code and build the Rust source code into Wasm bytecode using the standard cargo toolchain. It will then run the Wasm flow function in the WasmEdge Runtime.

How to Connect the Flow Function to GitHub and OpenAI

While the flow function requires connections to the OpenAI and GitHub APIs, the source code has no hardcoded API keys, access tokens, or OAUTH logic. The flows function SDKs have made it easy and safe for developers to interact with external SaaS API services.

Flows.network discovers that the flow function requires connections to the OpenAI and GitHub APIs. It presents UI workflows for the developers to:

• Log into GitHub, authorize access to events, and register the flow function as the webhook for receiving those events.

• Associate an OpenAI API key with the name openai_key_name.

Figure 4. The external services required by the flow function are connected and turned green.

Once the external SaaS APIs are connected and authorized, they turn green on the flow function dashboard. The flow function will now receive the events it listen_to_event() for. It will also get transparent access to Redis for the named OpenAI API key and the cached conversation context to support the chat_completion() SDK function.

What's next

The GitHub bot is just one of many bot types the flows.network can support. By connecting the flow function to a Slack channel, you can get ChatGPT to participate in your group discussion. Here is an example of a Slack-based ChatGPT bot.

Figure 5. The Slack ChatGPT bot.

Another example is to have ChatGPT answering legal questions in a Slack channel. The flow function prepends the legal question with a prompt.

Figure 6. The Slack robo lawyer bot.

Besides GitHub and Slack, there are many SaaS products you can integrate into flows.network through their APIs.

While the example flow functions are written in Rust, we aim to support JavaScript-based flow function SDKs. In another word, platform SDK functions such as listen_to_event() and chat_completion() will have a JavaScript version. The JavaScript flow function runs inside the WasmEdge Runtime on the flows.network platform through the WasmEdge-QuickJS module.

It also leverages local or even private data centers, CDN networks, and telecomm data centers (for example 5G MECs) to provide compute services.

Successful examples of edge clouds include Cloudflare, Fastly, Akamai, fly.io, Vercel, Netlify, and many others.

But the edge cloud is also a resource-constrained environment compared with big public clouds. If the edge microservices themselves are slow or bloated or insecure, they will defeat the whole purpose of deploying on the edge cloud.

In this article, I will show you how to create lightweight and high-performance web services in the WebAssembly sandbox, and then deploy them for free on edge cloud provider fly.io.

Fly.io is a leading provider of VM services on the edge cloud. It has edge data centers around the world. The fly.io VMs support app servers, databases, and in our case, lightweight runtimes for microservices.

I will use the WasmEdge Runtime as the security sandbox for those microservices. WasmEdge is a WebAssembly runtime specifically optimized for cloud-native services.

We will package the microservice application, written in Rust or JavaScript, in WasmEdge-based Docker images.

There are several compelling advantages to this approach:

• WasmEdge runs sandboxed applications at near-native speed. According to a peer-reviewed study, WasmEdge runs Rust programs at nearly the same speed as Linux runs native machine code.

• WasmEdge is a highly secure runtime. It protects your app against both external and internal threats.

• The attack surface of the WasmEdge runtime is dramatically reduced from a regular Linux OS runtime.

• The risk of software supply chain attack is greatly reduced since the WebAssembly sandbox only has access to explicitly declared capabilities.

• WasmEdge provides a complete and portable application runtime environment at a memory footprint that is only 1/10 of a standard Linux OS runtime image.

• The WasmEdge runtime is cross-platform. That means the development and deployment of machines do not have to be the same. And once you created a WasmEdge application, you can deploy it to anywhere WasmEdge is supported including fly.io infrastructure.

The performance advantages are amplified if the application is complex. For example, a WasmEdge AI inference application would NOT require a Python install. A WasmEdge node.js application would NOT require a Node.js and v8 install.

In the rest of this article, I will demonstrate how to run:

• an async HTTP server (in Rust)

• a very fast image classification web service (in Rust), and

• a node.JS web server

• stateful microservices with database connections

All of them run fast and securely in WasmEdge while consuming 1/10 of the resources required by regular Linux containers.

Prerequisites

First, if you already have Docker tools installed on your system, that's great. If not, please follow the first section of this handbook to install Docker now. Then we will use online installers to install WasmEdge, Rust, and the flyctl tool for fly.io.

Install WasmEdge. See details here.

curl -sSf https://raw.githubusercontent.com/WasmEdge/WasmEdge/master/utils/install.sh | bash -s -- -e all

Install Rust. See details here.

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

Install the flyctl tool for fly.io. See details here.

curl -L https://fly.io/install.sh | sh

Once you installed flyctl, follow the instructions to sign up for a free account at fly.io. You are now ready to deploy web services on the edge cloud!

A Simple Microservice in Rust

Our first example is a simple HTTP service written in Rust. It demonstrates a modern web application that can be extended to support arbitrarily complex business logic.

Based on the popular tokio and hyper crates, this microservice is fast, async (non-blocking), and very easy for developers to create.

The fully statically linked WasmEdge image is only 4MB as opposed to 40MB for a base Linux image. That is sufficient to run an async HTTP service written in Rust’s tokio and hyper frameworks.

Run the following two CLI commands to create and then deploy a fly.io app from our slim Docker image for WasmEdge.

$ flyctl launch --image juntaoyuan/flyio-echo
$ flyctl deploy

That’s it! You can use the curl command to test whether the deployed web service actually works. It echoes back whatever data you post to it.

$ curl https://proud-sunset-3795.fly.dev/echo -d "Hello WasmEdge on fly.io!"
Hello WasmEdge on fly.io!

The Dockerfile for the juntaoyuan/flyio-echo Docker image contains the complete package of the WasmEdge runtime and the custom web application wasmedge_hyper_server.wasm.

FROM wasmedge/slim-runtime:0.11.0
ADD wasmedge_hyper_server.wasm /
CMD ["wasmedge", "--dir", ".:/", "/wasmedge_hyper_server.wasm"]

The Rust source code project to build the wasmedge_hyper_server.wasm application is available on GitHub. It uses the tokio API to start an HTTP server.

When the server receives a request, it delegates to the echo() function to process the request asynchronously. That allows the microservice to accept and handle multiple concurrent HTTP requests.

#[tokio::main(flavor = "current_thread")]
async fn main() -> Result<(), Box<dyn std::error::Error + Send + Sync>> {
    let addr = SocketAddr::from(([0, 0, 0, 0], 8080));

    let listener = TcpListener::bind(addr).await?;
    println!("Listening on http://{}", addr);
    loop {
        let (stream, _) = listener.accept().await?;

        tokio::task::spawn(async move {
            if let Err(err) = Http::new().serve_connection(stream, service_fn(echo)).await {
                println!("Error serving connection: {:?}", err);
            }
        });
    }
}

The asynchronous echo() function is as follows. It utilizes the HTTP API provided by hyper to parse the request and generate the response. Here, the response is simply the request data body.

async fn echo(req: Request<Body>) -> Result<Response<Body>, hyper::Error> {
    match (req.method(), req.uri().path()) {
        ... ...
        (&Method::POST, "/echo") => Ok(Response::new(req.into_body())),
        ... ...

        // Return the 404 Not Found for other routes.
        _ => {
            let mut not_found = Response::default();
            *not_found.status_mut() = StatusCode::NOT_FOUND;
            Ok(not_found)
        }
    }
}

Now let's add to the basic microservice to do something impressive!

An AI Inference Microservice in Rust

In this example, we will create a web service for image classification. It processes an uploaded image through a Tensorflow Lite model.

Instead of creating a complex (and bloated) Python program, we will use WasmEdge’s Rust API to access Tensorflow, which runs the inference task at full native machine code speed (for example, utilizing the GPU hardware if available).

Through the WASI-NN standard, WasmEdge’s Rust API can work with AI models in Tensorflow, PyTorch, OpenVINO, and other AI frameworks.

For AI inference applications with full Tensorflow Lite dependencies included, the WasmEdge footprint is less than 115MB. That compares to over 400MB for the standard Tensorflow Linux image.

Run the following two CLI commands to create and then deploy a fly.io app from our slim Docker image for WasmEdge + Tensorflow.

$ flyctl launch --image juntaoyuan/flyio-classify
$ flyctl deploy

That’s it! You can use the curl command to test whether the deployed web service actually works. It returns the image classification result with a confidence level.

$ curl https://silent-glade-6853.fly.dev/classify -X POST --data-binary "@grace_hopper.jpg"
military uniform is detected with 206/255 confidence

The Dockerfile for the juntaoyuan/flyio-classify Docker image contains the complete package of the WasmEdge runtime, the entire Tensorflow libraries and their dependencies, and the custom web application wasmedge_hyper_server_tflite.wasm.

FROM wasmedge/slim-tf:0.11.0
ADD wasmedge_hyper_server_tflite.wasm /
CMD ["wasmedge-tensorflow-lite", "--dir", ".:/", "/wasmedge_hyper_server_tflite.wasm"]

The Rust source code project to build the wasmedge_hyper_server_tflite.wasm application is available on GitHub. The tokio-based async HTTP server is in the async main() function as in the previous example.

The classify() function processes the image data in the request, turns the image into a tensor, runs the Tensorflow model, and then turns the return values (in a tensor) into text labels and probabilities for the possible classifications.

async fn classify(req: Request<Body>) -> Result<Response<Body>, hyper::Error> {
    let model_data: &[u8] = include_bytes!("models/mobilenet_v1_1.0_224/mobilenet_v1_1.0_224_quant.tflite");
    let labels = include_str!("models/mobilenet_v1_1.0_224/labels_mobilenet_quant_v1_224.txt");
    match (req.method(), req.uri().path()) {

        (&Method::POST, "/classify") => {
            let buf = hyper::body::to_bytes(req.into_body()).await?;
            let flat_img = wasmedge_tensorflow_interface::load_jpg_image_to_rgb8(&buf, 224, 224);

            let mut session = wasmedge_tensorflow_interface::Session::new(&model_data, wasmedge_tensorflow_interface::ModelType::TensorFlowLite);
            session.add_input("input", &flat_img, &[1, 224, 224, 3])
                .run();
            let res_vec: Vec<u8> = session.get_output("MobilenetV1/Predictions/Reshape_1");
            ... ...

            let mut label_lines = labels.lines();
            for _i in 0..max_index {
              label_lines.next();
            }
            let class_name = label_lines.next().unwrap().to_string();

            Ok(Response::new(Body::from(format!("{} is detected with {}/255 confidence", class_name, max_value))))
        }

        // Return the 404 Not Found for other routes.
        _ => {
            let mut not_found = Response::default();
            *not_found.status_mut() = StatusCode::NOT_FOUND;
            Ok(not_found)
        }
    }
}

In the last section of this article, we will discuss how to add more functionalities, such as database clients and web services clients, to the Rust microservice.

A Simple Microservice in Node.js

While Rust-based microservices are light and fast, not everyone is a Rust developer (yet).

If you are more comfortable in JavaScript, you can still take advantage of WasmEdge’s security, performance, small footprint, and portability in the edge cloud. Specifically, you can use Node.js APIs to create microservices for WasmEdge!

For Node.js applications, the WasmEdge footprint is less than 15MB. That compares to over 150MB for the standard Node.js Linux image.

Run the following two CLI commands to create and then deploy a fly.io app from our slim Docker image for WasmEdge + Node.js.

$ flyctl launch --image juntaoyuan/flyio-nodejs-echo
$ flyctl deploy

That’s it! You can use the curl command to test whether the deployed web service actually works. It echoes back whatever data you post to it.

$ curl https://solitary-snowflake-1159.fly.dev -d "Hello WasmEdge for Node.js on fly.io!"
Hello WasmEdge for Node.js on fly.io!

The Dockerfile for the juntaoyuan/flyio-nodejs-echo Docker image contains the complete package of the WasmEdge runtime, the QuickJS runtime wasmedge_quickjs.wasm, Node.js modules, and the web service application node_echo.js.

FROM wasmedge/slim-runtime:0.11.0
ADD wasmedge_quickjs.wasm /
ADD node_echo.js /
ADD modules /modules
CMD ["wasmedge", "--dir", ".:/", "/wasmedge_quickjs.wasm", "node_echo.js"]

The complete JavaScript source code for the node_echo.js application is as follows. As you can clearly see, it uses just standard Node.js APIs to create an asynchronous HTTP server that echoes back the HTTP request body.

import { createServer, request, fetch } from 'http';

createServer((req, resp) => {
  req.on('data', (body) => {
    resp.end(body)
  })
}).listen(8080, () => {
  print('listen 8080 ...\n');
})

WasmEdge’s QuickJS engine provides not only Node.js support, but also Tensorflow inference support. We wrapped the Rust Tensorflow and WASI-NN SDKs into JavaScript APIs so that JavaScript developers could easily create AI inference applications.

Stateful Microservices on the Edge

With WasmEdge, it is also possible to create stateful microservices backed by databases. This GitHub repo contains examples of tokio-based non-blocking database clients in WasmEdge applications.

• The MySQL client allows WasmEdge applications to access most cloud databases.

• The anna-rs project is an edge-native KV store with adjustable sync and consistency levels on edge nodes. WasmEdge applications can use anna-rs as an edge cache or database.

You can now build a wide variety of web services on the edge cloud using WasmEdge SDKs and runtimes. Cannot wait to see your creations!

The meme economy is where FOMO meets YOLO. — Felix Salmon from AXIOS

However, it takes time to create a meme. How about a web app that allows anyone to customize a meme and generate a new one? That is a meme-as-a-service (MaaS)!

Why Serverless?

As a developer, it is probably not difficult to create a web application that adds text captions to images. However, a meme-as-a-service has some additional requirements.

• The imaging processing task is often computationally intensive and requires high performance.

• The meme service can see very little use or it could explode in popularity. In other words, it needs to be scalable and the developer only pays for actual use.

There are solutions to the above issues. First, we will use a modern high-performance programming language to write the image and text manipulation function. We'll use Rust for this purpose, which delivers native performance but with memory safety.

Next, we can best address the scalability requirement with a serverless function in a public cloud. A serverless function is free when it is not used and can quickly scale to millions of users.

While it is possible to run a native program compiled from Rust as a serverless function, a better way is to run the Rust program in a WebAssembly VM as a serverless function.

The WebAssembly VM acts as a compatibility layer and the security sandbox between the native application and the serverless host environment. It allows the Rust program to be more portable as the WebAssembly VM is pre-configured to run in a variety of operating systems and container images required by public cloud serverless runtimes.

In addition, WebAssembly makes it easy for Rust programs to securely access software libraries written in C/C++. An example is to access Tensorflow libraries in legacy operating systems from Rust.

Quick start

In this tutorial, we will use a template project on GitHub to deploy our Rust meme-as-a-service on Tencent Cloud. The template is based on the open source Serverless Framework.

The Rust program compiles to and runs on the Second State VM (SSVM), which is a WebAssembly VM optimized to run in cloud-based host environments.

While we use Tencent Cloud in this example, the Serverless Framework is agnostic to all leading public clouds. You can easily deploy to AWS or Azure with minor changes.

First, you will need to install the Serverless Framework and create a free account on Tencent Cloud. Next, fork or clone the template GitHub repo and then cd into its directory.

$ git clone https://github.com/second-state/tencent-meme-scf
$ cd tencent-meme-scf

You can now use the Serverless Framework to deploy the cloud function, an API gateway for the function service, and a static HTML page that uses the function service. It is configured to deploy to Tencent Cloud in the .env file.

Just follow the on screen instructions to log into Tencent Cloud and deploy.

$ sls deploy
... ...
  website:       https://sls-website-ap-hongkong-kfdilz-1302315972.cos-website.ap-hongkong.myqcloud.com
  vendorMessage: null

63s › tencent-meme-scf › "deploy" ran for 3 apps successfully.

Load the website URL in a browser to see your deployed meme-as-a-service in action! It allows you to customize the text, position, and size for each piece of caption / watermark on the meme picture.

How the application works

The overall architecture of the application is a typical JAMStack app. The backend logic for image processing is deployed as a serverless function, and is available via an API.

The front end UI is a static HTML and JavaScript web page. The front end interacts with the backend API through JavaScript Ajax calls.

The backend serverless function for adding captions (or watermarks) to the meme background image is written in Rust in the main.rs file. It first reads the font file for the watermark text and the background image file.

It then reads the input from the user application that specifies the text, position, and size of each watermark. The input is in the form of JSON text, and it is parsed into an array of Rust structs using the serde_json library.

The _watermark() function adds each watermark to the image. The function outputs the final image and base64 encodes it. The serverless runtime’s API gateway returns the base64 encoded image to the calling JavaScript to be displayed on the web page.

const FONT_FILE : &[u8] = include_bytes!("PingFang-Bold.ttf") as &[u8];
const TEMPLATE_BUF : &[u8] = include_bytes!("bg.png") as &[u8];

fn main() {
  let mut buffer = String::new();
  io::stdin().read_to_string(&mut buffer).expect("Error reading from STDIN");
  let obj: FaasInput = serde_json::from_str(&buffer).unwrap();

  let mut img = image::load_from_memory(TEMPLATE_BUF).unwrap();

  let memes: Vec<Watermark> = serde_json::from_str(&(obj.body)).unwrap();
  for m in memes {
    _watermark(m, &mut img);
  }

  let mut buf = vec![];
  img.write_to(&mut buf, image::ImageOutputFormat::Png).unwrap();
  println!("{}", base64::encode_config(buf, base64::STANDARD));
}

The _watermark() function adds a piece of watermark text to the image. It utilizes the standard Rust image processing libraries (known as crates) to manipulate the meme image.

fn _watermark(w: Watermark, img: &mut image::DynamicImage) {
  let font_size = w.font_size;

  let font = Vec::from(FONT_FILE);
  let font = Font::try_from_vec(font).unwrap();

  let scale = Scale {
    x: font_size,
    y: font_size,
  };
  drawing::draw_text_mut(img, image::Rgba([0u8, 0u8, 0u8, 255u8]), w.left, w.top, scale, &font, &w.text);
}

The frontend JavaScript takes the user input text, position, and font size from the HTML form, submits the input data in JSON to the cloud function, and then displays the returned base64 image.

var memes = [];
memes[0] = {};
memes[0].text = $('#top-says').val();
memes[0].left = parseInt($('#top-left').val());
memes[0].top = parseInt($('#top-top').val());
memes[0].font_size = parseInt($('#top-font').val());
... ...

$.ajax({
  url: window.env.API_URL,
  type: "post",
  data : JSON.stringify(memes),
  dataType: "text",
  success: function (data) {
    const img_url = "data:image/png;base64," + data;
    $('#wm_img').prop('src', img_url);
  },
  error: function(jqXHR, exception){
    console.log("Error Status: " + jqXHR.statusText);
  }
});

Build your own meme-as-a-service

With the source code template, you can create your own meme-as-a-service. You can change the meme background image, and change the UI for adding text watermarks.

To do that, first make sure that you have installed the Rust compiler and ssvmup build tools.

Make changes to the Rust code and the HTML document. Compile the Rust function to WebAssembly, and copy the result to the scf/ folder.

$ ssvmup build --enable-aot
$ cp pkg/scf.so scf/

Run the serverless framework command to deploy the application.

$ sls deploy
... ...
  website:       https://sls-website-ap-hongkong-kfdilz-1302315972.cos-website.ap-hongkong.myqcloud.com
  vendorMessage: null

63s › tencent-meme-scf › "deploy" ran for 3 apps successfully.

What’s next

Now that you have created and published your own meme-as-a-service, congratulations!

There is much more you can do with the JAMStack application and serverless functions. For example, you can create serverless functions for Tensorflow inference to incorporate advanced image recognition and processing to your application.

The Raspberry Pi is a very powerful computer in a tiny package. The cheapest option, the Raspberry Pi Zero, is capable of running a fully featured Linux distribution and driving a high definition display. It is the size of two coins (US Quarters) and costs $5.

At $10, the Raspberry Pi Zero W comes with integrated WiFi and Bluetooth.

The $10 Raspberry Pi Zero W has a powerful CPU, WiFi, Bluetooth, and all kinds of connectors

At the “high end”, you can purchase a Raspberry Pi 4 desktop kit for less than $100. It has a 4-core ARM CPU running at 1.5GHz, GPU, 2GB (up to 8 GB) of RAM, 16GB (up to 2TB) of storage on MicroSD cards, wifi and Ethernet connectors, USB ports, HDMI ports that can drive 4K displays, as well as a keyboard and mouse.

The Raspberry Pi is also more than a standard computer. It is fun and hackable. The Raspberry Pi exposes a row of GPIO (General Purpose Input Output) pins. You can attach simple sensors (eg. temperature, humidity, light) to those pins, and capture their data from your applications running on the Pi.

You could also attach LED lights and motors to those pins, and use your Pi application to drive those peripheral devices.

For more complex sensors or devices, such as camera modules, you can also connect to the Pi via USB or Wifi, and access them in software. The Pi is a great device for learning and hardware hacking. Because of this, it is widely used in educational settings.

However, fun and learning are not just for kids. With so much computing power and easy networking, the Raspberry Pi can easily become a personal application server for you.

For example, you can put a web application (for example, a collaborative note taking app, or just some documents / videos to share) on a Pi, bring it to a meeting, and make it accessible to everyone in the room. You do not even need the Internet. It is completely decentralized and censorship-resistant.

The personal server is especially useful for developers. You can have a separate environment to deploy and test your server-side applications without having to mess with your laptop. A personal dev server is like Docker on steroids. In this article, I will teach you how to set one up.

First, get a Raspberry Pi

If this is your first Raspberry Pi, the easiest (and most expensive) way to set up is just to buy a desktop kit for around $100. It comes with everything you need for a computer except for the display.

If you are using the Pi as a personal dev server, you would NOT need a display after the initial setup. You can just SSH into it from your laptop once it is turned on!

Learn how to get your Raspberry Pi starter kit for free when you participate in this high performance web application learning exercise.

Of course, if you have spare computer parts, such as MicroSD cards, USB power supply, a keyboard, and a mouse laying around, you could save money by purchasing only the boards. You could get a Raspberry Pi Zero board for $5 and a Raspberry Pi 4 board for $35.

But the thing missing from the board is a MicroSD card that acts as the “hard drive” for storing the operating system and data. You can purchase a 16GB MicroSD card for $10 online, a MicroSD card reader, and use the Raspberry Pi Imager to load an operating system onto the MicroSD card from your laptop.

The two popular choices are Raspberry Pi OS and Ubuntu Linux. Both are Debian-based Linux distributions. Most starter kits pre-install the Raspberry Pi OS on their MicroSD cards (it is called NOOBS).

In the next two sections, I will talk you through both operating systems.

How to set up Raspberry Pi OS

Once you put in the MicroSD card with NOOBS, and connect a display, a keyboard, and a mouse, you can turn on the power!

From there, just follow on screen instructions to install Raspberry Pi OS (previously known as the Raspbian OS). Then setup a password for the user pi, and setup the wifi connection.

After you are logged in, go to the Preferences → Raspberry Pi Configuration menu and enable SSH. That will allow you to log into the Pi from another computer.

Note: in order to use the Pi as a “headless” server, you could request a static IP address from your router. In the future, you can just power on the Pi, and connect to it via SSH from your other computers or phones.

The Raspberry Pi OS is derived from the Debian Linux distribution. It comes with a full desktop UI environment with a modern web browser, a command line terminal, and learning programs such as IDEs for Python, Java, and Scratch.

My Raspberry Pi 4 with Raspberry Pi OS setup. Notice how small the actual computer is.

For our purposes, we are mostly interested in installing dev and server software through the command line terminal.

At this point, you could also find out the IP address of the Pi on your local network by running the following command. Then you can SSH into the Pi using that local IP address, username pi, and the password you gave pi during setup.

$ hostname -I
192.168.2.108 172.17.0.1

You can find a complete list of software packages installed on the Raspberry Pi OS here. It is always a good idea to update and upgrade to the latest packages. Run the command below and be patient. It could take an hour.

$ sudo apt update && sudo apt upgrade

How to set up Ubuntu Server 20.04

The Raspberry Pi OS is primarily geared toward a desktop experience. For developers who just want to use the device as a server or IoT device, the Ubuntu Linux is a much better choice. It has the latest software packages and libraries, and could be far more efficient without the desktop windows, web browser, Java, games, and learning tools.

You can download Ubuntu Server images for Raspberry Pi from the web, and load it on a MicroSD card. But perhaps a much easier way is just to use the Raspberry Pi Imager, select Ubuntu Server 20.04 TLS from the menu, and write into an empty MicroSD card.

Once the MicroSD card is prepared, you should follow these instructions to put in your WiFi network name and password. This allows the Raspberry Pi device to connect to the network as soon as it boots.

Basically, you can just put the MicroSD card into the Raspberry Pi, connect USB power, then wait for it to come online. You can find the raspberrypi device IP from your WiFi router, and then SSH into from any computer on your network.

The initial username and password are ubuntu / ubuntu. There is no need to even connect a monitor or keyboard. That’s it for a completely headless setup!

Note: if, for some reason, your Raspberry Pi cannot connect to WiFi at startup, you can hook up an HDMI display and a USB keyboard to it. Then follow these instructions to debug and set up WiFi on the running system.

Next, let's install the developer tool stack on the Pi.

Install Git

I always install Git on all my development environments because a lot of software can be directly retrieved from Git repositories. It saves me the trouble of downloading and copying.

Git also allows me to save and backup my own work in private repositories. For a small computer like Raspberry Pi, I would recommend that you save work in Git in case you lose the device or MicroSD card.

The following command installs Git:

$ sudo apt install git

Install Node.js

To turn the Raspberry Pi into a personal dev server for web applications, you need to install a modern web application runtime.

For most developers today, the best starting point is Node.js, which allows you to write server-side applications in JavaScript. The following two commands install Node.js on your Pi.

$ curl -sL https://deb.nodesource.com/setup_10.x | sudo bash -
$ sudo apt install nodejs

You can verify the installation is done correctly by running the following two commands. Both node and npm are now available.

$ node -v
v10.19.0
$ npm -v
5.8.0

From here, you can use npm to install modules. For example, a commonly used npm module is the express framework for web applications.

$ npm install express

Now, you can go through the ExpressJS hello world example to create a web server on your Pi, and use web browsers from any computer on your network to access the application!

Install Rust

Rust is a fast growing programming language for writing both systems and web applications. It is close to the hardware, high performance, and memory safe. That makes Rust a great language for writing applications on resource constrained devices like the Raspberry Pi.

Also, Rust is the most beloved programming language by StackOverflow users for the past 5 years in a row. It is well worth your time to learn it!

An important use case of Rust is to compile Rust functions into WebAssembly and run them inside Node.js applications to achieve performance, safety, and code portability. It is a great choice for running computationally intensive web applications on a small Raspberry Pi device. In fact, you could get a free Raspberry Pi starter kit if you learn how to do that.

Note: strictly speaking, you do not need to install Rust tools on the Pi. You typically only need to run Rust programs in the Pi. You can compile your Rust program on any computer and then copy the compiled binaries to the Pi.

But still, with the powerful CPU, you can compile Rust programs on the Raspberry Pi. So why not?

The following command installs the Rust compiler toolchain on the Pi.

$ curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh

Run the following command to set up the correct path without logging out and back in again.

$ source $HOME/.cargo/env

The above command also installs the Rust package manager called cargo. Most Rust developers use cargo to build and share their work.

$ cargo -V
cargo 1.44.1 (88ba85757 2020-06-11)

Next, you can clone our Rust learning repository, and learn from examples.

$ git clone https://github.com/second-state/wasm-learning.git

Here is the hello world example. Have fun!

$ cd wasm-learning/rust/hello
$ cargo build
   Compiling hello v0.1.0 (/home/pi/Dev/wasm-learning/rust/hello)
    Finished dev [unoptimized + debuginfo] target(s) in 4.35s
$ target/debug/hello
Hello, world!

Check out the official Rust web site and the Rust by Example books for more learning resources.

Learn Docker

We have seen that the Raspberry Pi OS and Ubuntu Server are both very capable Linux distributions with lots of software packages.

But what if I want to test applications on other OSes? Do I need to wipe clean and reinstall a different OS on the MicroSD card? The answer is no. You can just use Docker! The following two commands install docker on the Raspberry Pi:

$ curl -fsSL https://get.docker.com -o get-docker.sh
$ sudo sh get-docker.sh

Run the following command so that you can use Docker as the pi user:

$ sudo usermod -aG docker pi

The Docker info command shows that Docker is now installed on an ARM system with Raspberry Pi OS.

$ docker info
... ...
 Kernel Version: 4.19.118-v7l+
 Operating System: Raspbian GNU/Linux 10 (buster)
 OSType: linux
 Architecture: armv7l
 CPUs: 4
 Total Memory: 3.814GiB
 Name: raspberrypi
 ID: XERI:ZVVZ:XQVA:HXSH:KRPI:6GL2:5QRE:E7GZ:Z72Q:6SGF:CEI6:GKTC
 Docker Root Dir: /var/lib/docker
... ...

Next, you can pull a Docker image for the latest Ubuntu distribution, run it, and log into Ubuntu as a command line user.

$ docker pull ubuntu
... ...
$ docker run -it ubuntu bash
root# ... enter commands ...

What’s next?

In this article, we have touched on the basics and learned how to turn your Raspberry Pi 4 device into a personal dev server for software developers.

There is much to learn about Git, Node.js, Rust, WebAssembly and Docker. There are also many other developer stacks you can install on the Raspberry Pi.

Grab your free Raspberry Pi kit and let us know what you did with it!

Subscribe to our newsletter and stay in touch.

A gif showing the setup process of a GitHub Codespace

In the recent GitHub Satellite online conference, one of the most exciting product announcements was GitHub Codespaces. The idea is to have a code button on every repository.

When you click on it, it launches a fully featured VSCode IDE with all the necessarily software dependencies, including operating system level libraries, to build and run the project. This VSCode IDE runs entirely in your browser, and will not install any software or change any configuration to mess up your computer.

Sounds too good to be true? Well, watch the Github Satellite keynote segment about Codespaces yourself!

A key benefit of Github Codespaces is how quickly you can on-board new developers to a project. A new developer can get set up in minutes, instead of days, and immediately start contributing to the project. It is a great learning tool for new languages, frameworks, and software tools.

Under the hood, it starts a Docker container on a remote server, installs the entire software stack required by the project, and runs tasks like compiling and debugging in the remote Docker.

The web browser acts as a front end UI for the Docker instance. This approach requires no software install on the developer’s machine. But the trade-off is that all software installation from the operation system all the way to the final application happens on the server.

GitHub must start a fresh server for each Codespaces instance. That requires a lot of data center resources. In fact, the GitHub Codespaces web page has a waiting list as of today (June 2020).

Personally, I cannot wait for GitHub Codespaces to become available. But a time machine does exist. You can experience all the features in GitHub Codespaces today, for free.

VS Codespaces

The software behind GitHub Codespaces is actually based on a Microsoft VSCode product called VS Codespaces. VS Codespaces is available today to all Microsoft Azure users. And yes, it allows you to open GitHub repositories in VSCode IDE directly from a browser window.

In this tutorial, I will show you how to use Codespaces in your own development work today.

To make Codespaces available in your GitHub repositories, you just need to add the following HTML button anywhere on your web pages.

When an Azure user clicks on the button, it asks the user to log into VS Codespaces and walks the user through opening the repository in the online IDE. You can see how it works in the examples in the next section.

<p>
  <a href="https://online.visualstudio.com/environments/new?name=My%20Project&repo=username/reponame">
    <img src="https://img.shields.io/endpoint?style=social&url=https%3A%2F%2Faka.ms%2Fvso-badge">
  </a>
</p>

VS Codespaces runs entirely in your browser and costs around $1 per work day. It is cheaper than a cup of coffee in the office.

Examples

Now, let's look into several examples of how you might learn new programming skills using VS Codespaces.

Rust is one of the fastest growing programming languages today. It is voted as the most beloved programming language by Stackoverflow users four years in a row.

But to experiment with Rust requires a complex toolchain of compiler, linker, package manager, tool manager and so on.

VS Codespaces provides an easy way to learn Rust. Just click on the VS Codespaces button in this repository and you now have a working Rust project to experiment with!

https://github.com/second-state/learn-rust-with-github-actions

As a system language, Rust is well positioned to build high performance server side applications. The most promising stack is to compile and run Rust functions in a WebAssembly container, and then access those high performance functions from an existing web application framework, such as Node.js.

However, as you can already see, this “best practice” setup requires a complex stack of software.

Clicking on the VS Codespaces button in this repository gives you a fully functional Node.js project that uses Rust functions in WebAssembly as modules. You can immediately start to modify the Rust and JavaScript code and run the Node.js application from inside the web browser IDE.

https://github.com/second-state/ssvm-nodejs-starter

Server-side Rust and WebAssembly sound cool. But do we actually have a more complete example that demonstrate the power and performance of Rust beyond a simple hello world?

This repository is such an example. Open it in VS Codespaces and you will have a project for a Rust + JavaScript app that utilizes Tensorflow to perform image recognition. Since the app runs inside Node.js, it provides a template for AI-as-a-Service web applications.

https://github.com/second-state/rust-wasm-ai-demo

What if you want to stay on the bleeding edge and use Rust-based web runtime Deno instead of the C-based Node.js? Well, there is a VS Codespaces template for running Deno as an Azure Function too!

How it works

If you look closely, each VS Codespaces-enabled repository has a .devcontainer folder. Inside that folder, the Dockerfile specifies how to build the Docker container for the development environment.

For example, the Node.js Docker container is based on Ubuntu Linux with Node.js and selected NPM packages pre-installed. Check out an example here.

The devcontainer.json file specifies the configuration for the VSCode IDE on the remote Docker. For example, it configures the VSCode extensions to install, the terminal and debugger commands to use, and the host ports to forward to for testing and debugging.

Microsoft provides quite a few .devcontainer templates for you to modify and use. They cover most of the common software development stacks today.

You could further customize the user’s VSCode experience by providing launch and tasks definitions in the .vscode folder. Check them out!

Conclusion

With VS Codespaces, and GitHub Codespaces when it launches, the barriers and friction for software development are further reduced. You can get started with complex software stacks without leaving your web browser. Try it today!

Finally, watch the full length GitHub Satellite presentation on GitHub Codespaces.

Subscribe to my newsletter and stay in touch.

Rust is one of the hottest ? programming languages today. It is StackOverflow's most beloved programming language for the past 4 years. Yet, it still has a reputation as the programming language for alpha geeks.

By some estimate, there are 600,000 Rust developers world-wide, which is a significant number. But it's still dwarfed when compared with tens of millions of JavaScript, Java, and Python developers.

Who are those Rust developers? What are they using Rust for? Why do they love Rust so much? And most important, how do you join their ranks and see for yourself why Rust is so beloved? Don't get left behind.

In order to answer those questions, the Rust community has conducted annual developer surveys from rust-lang.org since 2016. The site recently released its 2019 survey results based on responses from nearly 4000 Rust developers. Here are the top 8 things I learned from the survey.

??‍? Rust is for professional programmers

The Rust programming language is not designed to be “easy to get started”. Rather, it is designed to be powerful and safe at the same time. It aims to be the developer productivity language for professional programmers. It is challenging, fun, and rewarding. That shows in the survey.

Very few respondents call themselves as Rust experts. Most people rate their Rust expertise as 7/10 or below, despite the fact that over 68% of them write Rust code on a weekly basis. It is clearly a language that takes time to master and excel.

About 37% of Rust users felt productive in Rust in less than a month of use - this is not too different from the percentage last year (40%). Over 70% felt productive in their first year. Unfortunately, like last year, there is still a struggle among users - 21% indicated they did not yet feel productive.

At the same time, when asked why not using Rust on some projects, the learning curve is cited as the #2 most common reason. The #1 reason, of course, is the company’s decision whether to use a particular programming language in a project.

? Documentation is critical for adoption

How do developers overcome Rust's learning curve and fall in love with it? Well, not unexpectedly, most developers cited “better documentation” as the driver for adoption.

But true to “professional programmers”, the most sought after Rust documentation is intermediate-level content that helps developers improve their Rust skills and productivity.

While the survey is biased toward developers who already knew the basics of Rust, it seems that there is a thirst for knowledge and self-improvement in this crowd.

? Developers do not want tomes of text

Traditional software documentation typically consists of entire books and websites. New generations of developers want more and better documentation. As a “new” language, Rust is already leading the innovation when it comes to programming language documentation.

For example, the Rust compiler is a self-documenting tool. One of the most unique and beloved features of Rust is its aggressive compiler that helps you ensure correctness and safety before the program even runs. As a result, Rust developers can write highly performant yet safe programs.

When you encounter a compiling error in Rust, the compiler gives you an immediate explanation of the error, and suggestions on how to fix the error based on the context of your program.

This starter project in GitHub gets you started with the Rust compiler and the Cargo system without having to install any software toolchain. You can use the VSCode online IDE directly with this project.

Rust documentation web sites like docs.rs and Rust by Example (and its Extended Edition) use the Rust Playground to run Rust example code directly from the browser. Those interactive books are much better than simple text.

However, as the survey finds out, developers want more. Developers are thirsty for more video content, for example. We can look forward to more coding videos and live broadcasts from the community soon.

?️ Most people use Rust for web apps, srsly!

As a system-level language intended to replace C and C++, most people assume that Rust would be used in infrastructure programming, such as operating systems, native libraries, and runtime platforms.

Yet, the survey clearly shows that by a large margin, most Rust developers today are working on web app backends. No wonder crates like hyper, actix-web, and Rocket are among the most popular with Rust developers.

To be sure, most software developers are working on web applications. It is not surprising that, as Rust gains mainstream adoption, Rust projects will mirror the bigger software industry.

However, that does present opportunities for projects and tools that integrates Rust into popular web application runtimes. For example, the Rust + JavaScript hybrid application approach is gaining momentum.

? Blockchain is a Rusty hotbed

When it comes to infrastructure software, Rust really shines as a programming language for blockchain systems.

For all software related industry sectors, the survey shows that blockchain only ranks the 35th for all software developers, but 11th for Rust developers. That is in no small part due to aggressive Rust adoption by large blockchain projects such as Polkadot / Substrate, Oasis, Solana, and Second State etc.

In many ways, blockchains are perfect fit for Rust. Blockchains represent the community effort to re-build the internet infrastructure in a decentralized manner. They require high performance software that is also very safe. If you are interested in a career as a blockchain engineer, Rust is a must-have skill today.

Rust ❤️ WebAssembly

The survey reveals that WebAssembly is a popular runtime environment for Rust programs. Rust and WebAssembly were both invented at Mozilla.

Rust is focused on performance and memory safety, while WebAssembly is focused on performance and runtime safety. As a runtime container, WebAssembly also makes Rust programs cross-platform and more manageable. There are indeed a lot of synergy between the two technologies.

WebAssembly was originally invented as a client side virtual machine to run in-browser applications. But as Java and JavaScript before it, WebAssembly is now making the migration from the client side to the server side.

Rust-in-WebAssembly bodes well with the trend of accelerating Rust adoption on backend web applications. You can get started with Rust and WebAssembly application development from a starter project in this GitHub repository.

? Asynchronous programming is taking off

In recent years, two new programming languages have gained significant traction among developers. One is Rust, and the other is Go. A big part of their success is their superior support for concurrency programming models.

In fact, an early tagline of Rust is "fearless concurrency". It promises developer productivity in writing asynchronous multi-threaded programs optimized for today’s multi-core CPU architectures. As Node.js demonstrated, easy asynchronous programming is crucial for a language or framework’s success on the server side.

The survey shows that 4 of the 10 most important Rust crates (ie third party libraries), tokio, async, futures, and hyper, are frameworks for asynchronous multi-thread applications.

? R, Python, and JavaScript

As the adoption of Rust grows, developers increasingly need to integrate Rust programs with programs written in other languages. In the past, C and C++ were the most common languages to “talk” to Rust as they are all used in infrastructure software projects.

As Rust grows into application software projects, more language level interfaces and bridges are needed now. A good example is the Rust JavaScript bridge that supports Rust functions in Node.js applications.

The survey found that, besides C/C++ and JavaScript, Rust developers are interested in integrating with R and Python. That indicates developer interests in machine learning, big data, and artificial intelligence (AI) applications. In fact, many Python and R machine learning and statistical packages are implemented in native binary modules.

Rust is one of the best programming languages to write native modules. This example shows how to use Rust to execute Tensorflow models in a Node.js application. In the future, we envision such Rust modules to run in high performance managed containers such as WebAssembly.

Conclusion

2019 was a year of growth and incremental improvements for Rust. As Rust becomes a mainstream programming language, we look forward to more documentation, more tools, more ecosystem support, more interoperability with other languages, and a gentler learning curve.

And most important of all, we are eager to make more friends and have fun with the most beloved programming language in the world!

About the author

Dr. Michael Yuan is the author of 5 books on software engineering. His latest book Building Blockchain Apps was published by Addison-Wesley in Dec 2019. Dr. Yuan is the co-founder of Second State, a VC-funded startup that brings WebAssembly and Rust technologies to cloud, blockchain, and AI applications. It enables developers to deploy fast, safe, portable, and serverless Rust functions on Node.js.

Prior to Second State, Dr. Yuan was a long time open source contributor at Red Hat, JBoss, and Mozilla. Outside of software, Dr. Yuan is a Principal Investigator at the National Institutes of Health, with multiple research awards on cancer and public health research. He holds a PhD in astrophysics from the University of Texas at Austin.

Rust is one of the hottest programming languages today. Alpha geeks love it. It is Stackoverflow's most beloved programming language for the past 4 years in a row.

One of the most unique and beloved features of Rust is its aggressive compiler that helps you ensure correctness and safety before the program even runs. As a result, Rust developers can write highly performant yet safe programs. Rust eliminates whole classes of programming bugs, especially those hard-to-debug runtime bugs.

If you have not tried it, try it! It is magical. I believe Rust could be the next Java or Ruby -- the programming language everyone must learn in the future.

However, learning Rust typically requires you to install a bunch of command line tools on your computer. The Rust compiler is slow because the whole Rust paradigm is set up to deeply analyze the source code and find bugs at compile time, instead of crashing at runtime.

Online Rust IDEs, like the Rust Playground and REPL.it, are simple tools that do not take full advantages of the Rust ecosystem of third party compiler targets and libraries.

So you might wonder - can I try and learn Rust without having to install all those software packages on my computer?

Well, with GitHub actions, you can! You can learn and experiment with Rust code directly in your web browser. Let's get started!

GitHub Actions makes it easy to automate all your software workflows, now with world-class CI/CD. Build, test, and deploy your code right from GitHub. Make code reviews, branch management, and issue triaging work the way you want. The source code and workflow actions for the Hello World example can be found in this GitHub repository.

Hello world

First, create a new GitHub repository, and add a Rust source file. Let's add a src/main.rs file with the following content.

fn main() {
    println!("Hello, world!");
}

Next, go back to the root / directory of the GitHub repository, and add a Cargo.toml file. This file describes how the Rust cargo system should build and package our project.

[package]
name = "hello"
version = "0.1.0"
authors = ["ubuntu"]
edition = "2018"

[dependencies]

Now we have a complete Rust project. Let's build and run it now.

Github actions

On GitHub's Actions tab, we can add workflows associated with this project. Those are actions Github automatically performs when certain events occur, such as code push or commit. In our case, we would like GitHub to automatically build and run our main.rs, and show us the results.

The workflow actions and their event triggers are defined in yml files under the .github/workflows directory. You can write your own yml files, or pick from one of the ready-made templates.

The Rust workflow template in GitHub Actions

Here we choose the Rust template. GitHub allows you to edit the rust.yml file before checking it into the repository.

The default Rust actions template. You can edit it to your own liking.

Let's take a minute here to explain how GitHub Actions work. The default rust.yml file says that

• Whenever a user pushes code or accepts pull requests into this repository, the actions in this rust.yml workflow will be triggered.

• The workflow will create a virtual machine running the latest Ubuntu operating system. On this Ubuntu system, it will then perform the following steps.

• It will checkout the code from the master branch.

• It will run the cargo build --verbose command to compile and build the Rust code.

• It will run the cargo test --verbose command to execute the test cases.

• All the standard and console outputs on the Ubuntu system from the above two commands will be captured by GitHub Actions and displayed on the web.

You can edit the last line in rust.yml to perform cargo run, which runs the compiled binary program. Our updated rust.yml file is as follows.

name: Rust

on:
  push:
    branches: [ master ]
  pull_request:
    branches: [ master ]

jobs:
  build:

    runs-on: ubuntu-latest

    steps:
    - uses: actions/checkout@v2
    - name: Build
      run: cargo build --verbose
    - name: Run
      run: cargo run

Now whenever you push code to this repository, the actions in rust.yml are performed. You can see the results under the Actions tab.

Every push to the GitHub repo will trigger the actions to run

You can click on a result, and click on the build tab to the left to see the details. The build and run sections provide the most relevant details. The Run section shows successful printing of hello world!

The Run action shows Hello World! printed to the console!

Next, you can add third party dependencies in Cargo.toml, and build complex Rust applications in main.rs. Every time someone pushes code, we will be able to see the results.

Test Driven Development (TDD)

Of course, very few developers actually run their programs to print text to the console. The cargo run above is just a show. In reality, most developers write functions and test cases for those functions. The most frequent task after compiling and building is to run test cases. Let's see how it is done.

Create a new GitHub repository, and then add a src/lib.rs file below. As you can see, it defines a Rust function and a few test cases. It can be built and released as a Rust library package.

pub fn say(s: &str) -> String {
  let r = String::from("hello ");
  return r + s;
}

#[cfg(test)]
mod tests {
  use super::*;

  #[test]
  fn say_hello() {
    let result = say("ssvm");
    assert!(result.contains("hello ssvm"));
  }
}

Then, go back to the root / directory of the GitHub repository, and add the following Cargo.toml file.

[package]
name = "hello"
version = "0.1.0"
authors = ["ubuntu"]
edition = "2018"

[lib]
name = "hello_lib"
path = "src/lib.rs"
crate-type =["cdylib"]

[dependencies]

Click on the Actions tab and add the default Rust workflow. As you recall, the default Rust workflow ends with cargo test, which is exactly what we need here.

The default Rust workflow actions are what we need here.

The workflow runs every time new code is pushed into this repository. You can click open a successful build, and see the output from the build and test actions.

Github actions build and test your Rust program

What's next

Now you can experiment with your Rust code, and have GitHub build, test, and run it for you with complete console outputs, for free and without ever leaving your browser!

Resources

• Learn to program in Rust

• Learn more about GitHub Actions

• Of course, the best way to run Rust programs on the server is inside a WebAssembly Virtual Machine. Checkout the open source Second State VM for that!

• Learn about the BUIDL online IDE to code and deploy decentralized web apps on public blockchains

About the author

Dr. Michael Yuan is the author of 5 books on software engineering. His latest book Building Blockchain Apps was published by Addison-Wesley in Dec 2019. Dr. Yuan is the co-founder of Second State, a VC-funded startup that brings WebAssembly and Rust technologies to cloud, blockchain, and AI applications. It enables developers to deploy fast, safe, portable, and serverless Rust functions on Node.js.

Prior to Second State, Dr. Yuan was a long time open source contributor at Red Hat, JBoss, and Mozilla. Outside of software, Dr. Yuan is a Principal Investigator at the National Institutes of Health, with multiple research awards on cancer and public health research. He holds a PhD in astrophysics from the University of Texas at Austin.