But these skills have a hidden limitation: they're static. If you download a skill someone else wrote and you want it to behave a little differently, you'll have to copy the whole thing and edit it by hand. And as you may have noticed lately, there are many "skills" forks floating around that are difficult to maintain.

In this tutorial, I'll show you I built a small convention that fixes this. It lets any Agent Skill read a per-project config file, so you can adopt any skill and customize how it behaves by editing a few lines of YAML (without ever touching the skill itself).

You'll build it step by step, test it, and see how to share it so other people can plug into it.

Table of Contents

• What You Will Build

• Prerequisites

• What Are Antigravity Agent Skills?

• Why Static Skills Are a Problem

• The Configurable Skills Solution

• How to Build the Config Loader

• How to Make a Skill Configurable

• How to Add Project Overrides

• How to Test Your Configurable Skill

• Two More Example Skills

• How to Share Your Agent Skills With Others

• Wrapping Up

What You Will Build

You will build a tiny, reusable layer called Configurable Agent Skills. It has three parts:

1. A small Python script, resolve_config.py, that merges a skill's default settings with your project settings and prints the result.

2. A convention: each skill ships 2 files, a config.default.yaml file with its "knobs" and a SKILL.md file. They both guide the agent's behavior.

3. A per-project file, .agent/skills.config.yaml, where anyone using your skill sets their own values.

By the end, you'll have a working git-commit-formatter skill that one team can run in Conventional Commits mode and another team can switch to gitmoji mode, all using the exact same skill files with no forking.

Prerequisites

To follow along, you'll need:

• Google Antigravity installed (the IDE, CLI, or SDK. Any of them work, since skills are just files.).

• Python 3 installed, with PyYAML. You can install PyYAML with python -m pip install pyyaml.

• Basic comfort with the terminal and YAML. You don't need to be an expert in either.

If you've never written an Agent Skill before, the next two sections will bring you up to speed.

What Are Antigravity Agent Skills?

A Skill in Antigravity is a folder that contains a SKILL.md file and, optionally, some scripts, templates, or examples. The SKILL.md file has a short block of YAML "frontmatter" at the top (a name and a description), followed by a set of instructions written in plain Markdown.

Here's the important part: skills are loaded on demand. The agent reads only the short description of each skill at first. When your request matches that description, the agent pulls in the full instructions and follows them. This keeps the agent's context small and focused.

A minimal skill that enforces Conventional Commits looks like this:

---
name: git-commit-formatter
description: Formats git commit messages using the Conventional Commits specification. Use this when the user asks to commit changes or write a commit message.
---

# Git Commit Formatter

When writing a commit message, follow the Conventional Commits format:
`type(scope): description`

Allowed types: feat, fix, docs, style, refactor, perf, test, chore.

Drop that in your skills folder, ask the agent to "commit these changes," and it will write a properly formatted message. Simple and useful, right?

Why Static Skills Are a Problem

Now look closely at that skill. The allowed types (feat, fix, docs, and so on) are baked directly into the instructions.

That's fine until someone wants something slightly different. Maybe your team also uses a ci type. Maybe you prefer gitmoji, where each commit starts with an emoji. Maybe you want to require a scope on every commit.

With a static skill, there's only one way to get any of that: copy the whole skill and edit the Markdown. When you do this across a team, everyone ends up with their own private fork. When the original author ships an improvement, none of the forks get it. The skill stops being something you share and becomes something everyone rewrites.

The core issue is that there's no clean line between the skill's logic (which everyone should share) and its settings (which each project wants to control). How do we solve this?

The Configurable Skills Solution

The idea is simple. Instead of hard-coding settings in the instructions, the skill will:

1. Ship its settings and their defaults in a separate config.default.yaml file.

2. Read a merged config (defaults plus any project-level overrides) before it acts.

The project-level overrides live in a file called .agent/skills.config.yaml, which sits at the root of the user's project:

# .agent/skills.config.yaml 
# (edit this file in your project instead of the skill globally)
git-commit-formatter:
  style: gitmoji
  extra_types: [ci, build]
  scope_required: true

That's the easy flow. Drop the skill in, set a few keys, and you're done. The skill's own files never change.

To make this work, you need a script that reads both files, merges them, and hands the result to the agent. Let's build it.

How to Build the Config Loader

Create a file called resolve_config.py. Its job is to take a skill's name, load that skill's config.default.yaml, find the user's .agent/skills.config.yaml, and merge the two so that user values win.

Start with a deep-merge helper. This is the heart of the loader:

def deep_merge(base, override):
    """Recursively merge override onto base.

    Dicts merge key by key. Anything else (scalars, lists) is replaced
    wholesale by the override value.
    """
    if isinstance(base, dict) and isinstance(override, dict):
        merged = dict(base)
        for key, value in override.items():
            merged[key] = deep_merge(merged[key], value) if key in merged else value
        return merged
    return override

Notice the deliberate choice here: dictionaries merge key by key, but lists are replaced, not appended. That keeps the behavior predictable. If you want to handle "defaults plus extras", use the explicit extra_types key in the skill as you'll see in the example below.

Next, you need to find your "per-project" config. The loader walks up from the current directory looking for an .agent/skills.config.yaml file:

from pathlib import Path

def find_project_config(start: Path):
    """Walk upward from start looking for .agent/skills.config.yaml."""
    start = start.resolve()
    for folder in [start, *start.parents]:
        candidate = folder / ".agent" / "skills.config.yaml"
        if candidate.is_file():
            return candidate
    return None

Now put it together. The loader locates the skill's defaults (which sit next to the script), loads your overrides for that skill's name, merges them, and prints the result:

import sys, yaml
from pathlib import Path

def resolve(skill_name, skill_dir, project_root):
    defaults = yaml.safe_load((Path(skill_dir) / "config.default.yaml").read_text()) or {}

    user_path = find_project_config(Path(project_root))
    user_all = yaml.safe_load(user_path.read_text()) if user_path else {}
    user_cfg = (user_all or {}).get(skill_name, {}) or {}

    return deep_merge(defaults, user_cfg)

That completes the whole idea. The full version in the sample repo adds a command-line interface, JSON output, and clear error messages, but the logic above is all you really need.

How to Make a Skill Configurable

Now you'll convert the static commit skill into a configurable one. This takes two files.

First, create config.default.yaml next to the skill. It lists every setting and a safe default, so the skill works even when the user has no config at all:

# Default configuration for the git-commit-formatter skill.
style: conventional          # conventional | gitmoji
types:                       # base set of allowed commit types
  - feat
  - fix
  - docs
  - style
  - refactor
  - perf
  - test
  - chore
extra_types: []              # additional types, merged on top of `types`
scope_required: false        # if true, require a scope: type(scope): ...
max_subject_length: 72       # hard cap on the subject line

Second, update SKILL.md so that its very first instruction is to resolve the config and apply it. This is the key move: you're telling the agent to read the settings before it does anything else:

---
name: git-commit-formatter
description: Formats git commit messages to a team's chosen convention (Conventional Commits or gitmoji). Use this when the user asks to commit changes or write a commit message. Reads per-project settings so teams customize commit style without editing this skill.
---

# Git Commit Formatter (Configurable)

## Step 1 - Resolve configuration (always do this first)

Run the loader and read its output:

`python scripts/resolve_config.py git-commit-formatter --project-root .`

Apply exactly those settings:

- `style`: `conventional` or `gitmoji`.
- `types` + `extra_types`: the full set of allowed commit types.
- `scope_required`: if true, a scope is mandatory.
- `max_subject_length`: hard cap on the subject line.

## Step 2 - Compose the message

Pick the primary type from `types` + `extra_types`, build the subject in the
chosen `style`, and enforce `scope_required` and `max_subject_length`.

This pattern ("make the agent run a script and obey its output") is the same one Antigravity's own validation skills use. It keeps the behavior deterministic instead of leaving it to the model's memory.

Notice how extra_types solves the additive-list question. The default list stays put, and the user's extras are simply added on top by the skill. No fork is required to add a ci type.

How to Add Project Overrides

Let's say you want gitmoji commits with two extra types. Create a single file in your project:

# .agent/skills.config.yaml
git-commit-formatter:
  style: gitmoji
  extra_types: [ci, build]
  scope_required: true

You just changed three lines of config and didn't open the skill or fork any code. The next time the agent commits, it will use this project settings.

And a different project, with no config file at all, keeps getting the sensible Conventional Commits defaults. You have one skill with many behaviors.

How to Test Your Configurable Skill

You don't need the agent to check that the merge works. Run the loader directly and read the output.

With no overrides, you get the defaults:

$ python scripts/resolve_config.py git-commit-formatter --project-root .
style: conventional
scope_required: false
...

Now add the .agent/skills.config.yaml override from the last section and run it again:

$ python scripts/resolve_config.py git-commit-formatter --project-root . --print-sources
style: gitmoji
scope_required: true
extra_types:
- ci
- build
types:
- feat
- fix
- docs
...

The style flipped to gitmoji, scope_required became true, and your extra types appeared (while the base types list stayed intact). That confirms the merge does exactly what you want.

It's worth writing a small automated test too, so a future change to the loader can't silently break the merge. A test can create a fake skill and a fake project config in a temp folder, run the loader, and assert that user values override defaults while untouched defaults survive.

Two More Example Skills

The same pattern works for any skill. Here are two more to show the range.

A Changelog Generator

Its config.default.yaml exposes the output format (like Keep a Changelog), which commit types to include, and whether to link commit hashes to a repo URL. One project can generate a formal changelog grouped by type, while another can generate a simple bulleted list. It's the same skill with a different config.

# changelog-generator config.default.yaml (excerpt)
format: keepachangelog       # keepachangelog | conventional | simple
include_types: [feat, fix, perf]
include_authors: false
repo_url: ""                 # if set, hashes link to commits

A License-Header Adder

Its config exposes the license (Apache-2.0, MIT, or custom), the holder, and a map of file extensions to comment styles. A company sets the holder once in their project config, and every new file gets the right header in the right comment style, without editing the skill.

# license-header-adder config.default.yaml (excerpt)
license: apache-2.0          # apache-2.0 | mit | custom
holder: "Your Name or Org"
year: auto                   # auto = current year

The lesson is that almost any skill has a few decisions baked into it. When you pull those decisions into a config.default.yaml, you convert a one-off skill into a tool that anyone can reuse and tune.

How to Share Your Agent Skills With Others

Once your agent skills follow the convention, they compose into something bigger. To make your agent skills easy for others to adopt, you have to:

• Keep each skill self-contained: Vendor a copy of resolve_config.py inside each skill's scripts/ folder, so someone can copy a single skill folder anywhere and it just works.

• Document every config key in the SKILL.md, so users know exactly what they can tune.

• Publish a small index: A simple index.json that lists each skill's name, path, and config keys makes it easy for others to discover what you've built and contribute their own.

Because the convention is just "read a config file first," anyone can publish a compatible skill. Each new configurable skill makes the whole ecosystem more useful. In addition to shipping a skill, you're shipping a small standard that other people can build on.

Wrapping Up

You started with a static skill whose behavior was frozen in Markdown, and you turned it into a configurable one that anyone can tune from a single project file.

The entire setup is relatively small. It has the merge function, one convention, and a config.default.yaml per skill.

It also changes how skills are shared. Instead of forking a skill to change one setting, you can keep the shared logic and adjust your own config. Improvements to the skill flow to everyone, and everyone still gets the behavior they want.

If you want to try it, build the git-commit-formatter skill from this tutorial, drop it into your Antigravity skills folder, and add an .agent/skills.config.yaml to a project. Then flip style from conventional to gitmoji and watch the same skill behave differently.

From there, make one of your own skills configurable. Find the settings you baked into the instructions, move them into a config.default.yaml, and let your users take it from there.

The full sample code (the loader, its tests, and all three example skills) is on GitHub at github.com/keepdeploying/configurable-agent-skills.

Thanks for reading. If you build a configurable skill of your own, share it. Let's keep the ecosystem growing.

The question is never whether systems need to communicate. The question is always how.

For years, REST over HTTP with JSON was the default answer. It works, it's simple, and the tooling is everywhere. But as systems grow in scale (in the number of services talking to each other, the volume of data being exchanged, and the need for real-time communication), REST starts to show its limits.

This is where Remote Procedure Calls, Protocol Buffers, and gRPC enter the picture.

In this handbook, you'll learn what RPC is and the problem it was designed to solve. You'll also understand Protocol Buffers: what they are, why they exist, and how they work.

You'll then see how Google combined these ideas into gRPC, one of the most powerful communication frameworks in modern distributed systems. You'll learn all four gRPC communication patterns, see code generated across multiple languages from a single contract file, and walk through a complete end-to-end Flutter implementation with production-grade concerns including authentication, error handling, and timeouts.

By the end, you won't just know what gRPC is. You'll understand when to use it, when not to, and how to think about service communication as a systems engineer.

Table of Contents

• What is a Remote Procedure Call?

• The Problem RPC Solves

• Why gRPC Over REST: The Real Case

• Protocol Buffers: A New Language for Data

• The Proto File

• JSON vs Protocol Buffers

• The Protoc Compiler and Code Generation

• What is gRPC?

• Why HTTP/2 Matters for gRPC

• The Four gRPC Communication Patterns

• The Protobuf Repository: Organizational Best Practice

• Building a Complete gRPC System with Dart and Flutter

• Production Concerns

• gRPC vs REST vs WebSockets: When to Use What

• The Hybrid Architecture

• Conclusion

What is a Remote Procedure Call?

To understand RPCs, you first need to understand what a procedure call is.

A procedure call means invoking a procedure or function so that its code executes. For example, in Dart:

double calculateTax(double amount) {
  return amount * 0.075;
}

final tax = calculateTax(50000); // local procedure call

You call calculateTax, pass an argument, and get a result back. The function lives on the same machine, in the same process, in the same memory space. This is a local procedure call.

A Remote Procedure Call takes this same idea and stretches it across a network. The function you're calling lives on a different machine, in a different process, and potentially in a different country. But from the caller's perspective, it feels exactly like calling a local function.

// This looks like a local function call
final tax = await taxService.calculateTax(amount: 50000);

// But under the hood, this:
// 1. Serializes the argument into a binary format
// 2. Sends it over a network connection to a remote server
// 3. The server executes calculateTax with your argument
// 4. Serializes the result
// 5. Sends it back over the network
// 6. Deserializes it into a Dart object
// 7. Returns it to you as if it were local

The network complexity is completely hidden. You call a function. You get a result. Everything in between is handled by the RPC framework.

This is the fundamental idea behind RPC: make calling a remote function feel as natural as calling a local one.

The Problem RPC Solves

To appreciate why RPC matters, you need to understand what the alternative looks like.

Without RPC, calling a remote service looks like this:

Future<double> calculateTax(double amount) async {
  final response = await http.post(
    Uri.parse('https://tax-service.internal/api/v1/calculate'),
    headers: {
      'Content-Type': 'application/json',
      'Authorization': 'Bearer $token',
    },
    body: jsonEncode({'amount': amount}),
  );

  if (response.statusCode != 200) {
    throw Exception('Tax calculation failed: ${response.statusCode}');
  }

  final data = jsonDecode(response.body);
  return (data['tax'] as num).toDouble();
}

Every service call requires you to:

• Know and hardcode the endpoint URL

• Know the correct HTTP method

• Manually serialize your request to JSON

• Handle HTTP status codes yourself

• Manually deserialize the response from JSON

• Cast dynamic types to the types you actually expect

• Hope the field names in the response match what you think they are

Now multiply this by every single service call in your application. An authentication service, a user service, a payment service, a notification service, a transaction service. Every one of them requires the same manual boilerplate. Every one of them introduces the possibility of a typo in a field name, a wrong status code assumption, or a JSON deserialization failure that only surfaces at runtime.

With RPC:

// Feels like a local function call
final tax = await taxService.calculateTax(
  TaxRequest(amount: 50000),
);
// tax is already a strongly typed TaxResponse object
// No URLs. No HTTP methods. No JSON parsing. No casting.

The framework handles everything. The function signature is defined in a contract file that both the client and server use. The types are enforced at compile time. If the server changes the response shape, the client fails to compile before anything reaches production.

This is what RPC solves: it removes the accidental complexity of network communication and lets you focus on what you're actually trying to do.

Why gRPC Over REST: The Real Case

Before going into the technical details of Protocol Buffers and gRPC, it's important to make a solid case for why you'd choose gRPC over REST in specific scenarios. This isn't a claim that gRPC is always better. It's a clear look at where it genuinely wins.

Large Payloads Called by Many Internal Systems

Consider an internal enterprise API in a large telecommunications company. A single request and response payload for a plan registration or activation flow can contain over a thousand fields. This endpoint is called by dozens of internal applications: billing systems, CRM platforms, customer-facing mobile apps, internal dashboards, and partner portals.

With REST and JSON, every one of those applications sends and receives that thousand-field payload as text. Field names like subscription_activation_status, rate_plan_identifier, and network_provisioning_reference travel over the wire as strings on every single request. A significant portion of every payload isn't data. It's labels for data.

With Protocol Buffers, field names never appear in the payload at all. Only field numbers and values travel over the wire. That thousand-field payload shrinks dramatically. For an endpoint called millions of times per day by dozens of systems, the bandwidth saving is enormous and translates directly to infrastructure cost reduction.

Beyond size, the generated client guarantee is equally important. With REST, each of those dozens of applications reads the API documentation and builds its own understanding of the contract. When the backend changes a field name or type, not every application finds out immediately. Some find out in production when they break.

With a shared .proto file, every application generates its own strongly typed client from the same source. A contract change means every application regenerates. The compiler immediately reports where the breaking change affects each codebase. Nothing reaches production in a broken state.

Low Bandwidth and Remote Network Conditions

This is one of the most under-appreciated advantages of gRPC in markets where network quality varies significantly.

In many regions, a substantial portion of mobile users are on 2G or 3G connections. On a 2G connection, bandwidth can be as low as 50 to 100 kilobits per second. A REST JSON response that is 80 kilobytes takes over 6 seconds to download on a 2G connection. The equivalent protobuf binary, which can be 3 to 10 times smaller, takes under 2 seconds.

That difference isn't a technical footnote. It's the line between an application that feels usable and one that feels broken to a significant portion of your users. For any product that operates in markets with variable network conditions, protobuf's binary efficiency is a direct competitive advantage.

Beyond size, gRPC runs over HTTP/2 which maintains a single persistent connection rather than opening a new connection for every request. On slow networks where connection establishment (the TCP handshake and TLS negotiation) can itself take hundreds of milliseconds, reusing a single connection across many calls saves significant time over a session.

Microservice to Microservice Communication

When two internal services need to communicate, you have options. A message bus like Kafka or RabbitMQ is excellent when you don't need an immediate response, when the operation can happen asynchronously, and when you're broadcasting something that happened to multiple consumers.

But many service-to-service calls are synchronous by nature. An authentication service needs to validate a token right now before the request proceeds. A fraud detection service needs to assess a transaction right now before the payment is authorized. A pricing service needs to calculate a rate right now before the quote is generated. These operations can't publish an event and wait.

For synchronous service-to-service calls at high frequency, gRPC over HTTP/2 with protobuf encoding is significantly more efficient than REST. The persistent multiplexed connection means no connection setup overhead per call. The binary encoding means no JSON serialization and deserialization overhead on every hop. The generated clients mean both services compile against the same contract.

At scale, when two services are calling each other thousands of times per second, these efficiency differences compound into real performance and cost differences.

Managing API Contracts Across Multiple Teams

In a large engineering organization, multiple teams build services that others depend on. REST API contracts live in documentation. Documentation goes stale. The backend team changes a field name. The mobile team finds out when users report crashes. The data team finds out when their pipeline throws an error at 2am.

gRPC's protobuf repository approach transforms contract management from a documentation problem into a code problem. Contract changes go through pull requests. Every dependent team reviews the change. Breaking changes are caught at compile time. Nobody is surprised in production.

This governance benefit scales with team size. The larger the organization, the more valuable it becomes.

Real-Time Communication

REST is request-response. The client asks and the server answers. The conversation ends. For real-time features, you either poll (wasteful) or bolt on a separate WebSocket server alongside your REST API (two different systems to maintain).

gRPC's streaming patterns handle real-time communication natively within the same framework you use for regular calls. A live balance update, a real-time transaction notification, or a bidirectional chat session all use the same generated client, the same connection, and the same protobuf encoding as your regular unary calls.

One framework with all communication patterns. No separate infrastructure.

Protocol Buffers: A New Language for Data

RPC is a concept. To implement it, you need two things: a way to define the contract between client and server, and a way to serialize data efficiently for transmission over the network.

This is where Protocol Buffers comes in.

Protocol Buffers, commonly called protobuf, is a language-neutral, platform-neutral, extensible mechanism for serializing structured data. It was developed at Google in 2001, used internally for years, and open-sourced in 2008.

The JSON Problem at Scale

JSON is the dominant data format for web APIs. It's human-readable, flexible, and universally supported. For many use cases, it's the right choice.

But JSON has structural inefficiencies that become painful at scale.

Consider a user profile response:

{
  "id": "usr_001",
  "first_name": "John",
  "last_name": "Smith",
  "email": "john@example.com",
  "phone_number": "+2348012345678",
  "account_type": "savings",
  "balance": 500000.00,
  "currency": "NGN",
  "is_verified": true,
  "is_active": true,
  "kyc_level": 3,
  "created_at": "2024-01-15T10:30:00Z",
  "last_login": "2026-07-20T09:15:00Z"
}

Every field name travels over the network as a string on every single response. "first_name", "account_type", "phone_number" aren't data. They're labels for data. But they consume bytes on every request.

Now consider an internal enterprise API with over a thousand fields in its request and response payload, being called by dozens of internal applications thousands of times per day. A significant portion of every payload is field name strings, not actual data. The overhead accumulates into real bandwidth and processing costs.

Beyond size, JSON has another problem: it has no schema at the network level. Nothing prevents a backend engineer from renaming "first_name" to "firstName" in a new deployment. The client breaks at runtime in production with real users.

What Protocol Buffers Do Differently

Protocol Buffers solve both problems with a fundamentally different approach to data encoding.

Instead of encoding data as human-readable text with field names, protobuf encodes data as compact binary using only field numbers and values. Field names never travel over the network.

Here's the same user profile defined in protobuf:

message UserProfile {
  string id = 1;
  string first_name = 2;
  string last_name = 3;
  string email = 4;
  string phone_number = 5;
  string account_type = 6;
  double balance = 7;
  string currency = 8;
  bool is_verified = 9;
  bool is_active = 10;
  int32 kyc_level = 11;
  string created_at = 12;
  string last_login = 13;
}

When protobuf encodes this data, the output is binary that no human can read. But to a machine, it's extremely compact and fast to parse. The field numbers (1, 2, 3...) identify each field. The names never appear in the encoded output at all.

The result: the same user profile that's approximately 280 bytes in JSON is approximately 95 bytes in protobuf. That's three times smaller. For a thousand-field enterprise payload, this difference is enormous.

And because the schema is defined in a .proto file that both client and server compile against, field name changes are caught at compile time, not at runtime.

The Proto File

The .proto file is the heart of everything in the protobuf and gRPC ecosystem. It's where you define your data models and your service contracts.

It's written in Protocol Buffer Language (proto3) – not Go, not Dart, not Python, not Java. You write it in any text editor. VS Code with the vscode-proto3 extension gives you syntax highlighting, autocomplete, and inline validation.

Here's a complete .proto file for a fintech platform:

syntax = "proto3";

package banking;

option go_package = "./banking";
option java_package = "com.fintech.banking";



service BankingService {
  // Unary: one request, one response
  rpc Login (LoginRequest) returns (LoginResponse);

  // Unary: fetch user profile
  rpc GetProfile (ProfileRequest) returns (UserProfile);

  // Server streaming: real-time balance updates
  rpc WatchBalance (BalanceRequest) returns (stream BalanceResponse);

  // Server streaming: live transaction feed
  rpc StreamTransactions (TransactionRequest) returns (stream Transaction);

  // Client streaming: upload KYC documents in chunks
  rpc UploadDocument (stream DocumentChunk) returns (UploadResponse);

  // Bidirectional streaming: live chat support
  rpc Chat (stream ChatMessage) returns (stream ChatMessage);
}



message LoginRequest {
  string email = 1;
  string password = 2;
}

message LoginResponse {
  string token = 1;
  string user_id = 2;
  int64 expires_at = 3;
}

message ProfileRequest {
  string user_id = 1;
}

message UserProfile {
  string id = 1;
  string first_name = 2;
  string last_name = 3;
  string email = 4;
  string phone_number = 5;
  string account_type = 6;
  double balance = 7;
  string currency = 8;
  bool is_verified = 9;
  int32 kyc_level = 10;
}

message BalanceRequest {
  string user_id = 1;
}

message BalanceResponse {
  double balance = 1;
  string currency = 2;
  int64 timestamp = 3;
}

message TransactionRequest {
  string user_id = 1;
  int32 limit = 2;
}

message Transaction {
  string id = 1;
  double amount = 2;
  string description = 3;
  string type = 4;
  int64 timestamp = 5;
}

message DocumentChunk {
  bytes data = 1;
  string document_type = 2;
  int32 chunk_index = 3;
  bool is_last = 4;
}

message UploadResponse {
  bool success = 1;
  string document_id = 2;
  string message = 3;
}

message ChatMessage {
  string sender_id = 1;
  string content = 2;
  int64 timestamp = 3;
}

Let's walk through every part of this file carefully.

The Syntax Declaration

syntax = "proto3";

This tells the protobuf compiler which version of the Protocol Buffer language you're using. proto3 is the current standard. It must be the first non-comment line in every .proto file.

The Package Declaration

package banking;

The package name prevents naming conflicts when you have multiple .proto files across different services. It functions like a namespace. If two services both define a UserProfile message, the package name distinguishes them: banking.UserProfile versus auth.UserProfile.

Language-Specific Options

option go_package = "./banking";
option java_package = "com.fintech.banking";

These options tell the compiler how to organize the generated code for specific languages. They don't affect the proto file itself, only the generated output.

The Service Definition

service BankingService {
  rpc Login (LoginRequest) returns (LoginResponse);
  rpc WatchBalance (BalanceRequest) returns (stream BalanceResponse);
}

The service block defines the RPC contract. Think of it exactly like an abstract class in any object-oriented language. It declares what functions exist, what they accept, and what they return.

Each rpc line defines one remote procedure. The stream keyword before a type indicates that multiple messages will flow rather than just one.

Message Definitions

message LoginRequest {
  string email = 1;
  string password = 2;
}

A message is a data structure. Think of it as a class with only fields: no methods, no logic. Each field has three parts.

The type can be string, int32, int64, double, bool, bytes, or another message type.

The name is the field name as it appears in generated code. This is for human readability only. It never appears in the binary encoding.

The field number (= 1, = 2, = 3) is the unique identifier that protobuf uses in the binary output instead of the field name. This is critical: once you assign a field number, you must never change it or reuse it. The binary encoding uses these numbers, not names. If you change a field number, old encoded data becomes unreadable.

You can safely add new fields with new numbers, remove fields (the number stays reserved, never reuse it), and rename fields (names don't appear in binary). You must never change a field number, reuse a removed field's number, or change a field's type.

JSON vs Protocol Buffers

Now that you understand both formats, let's make a direct comparison.

Size Comparison

Let's look at the same login request in both formats:

JSON (text):

{
  "email": "john@example.com",
  "password": "securepassword123"
}

Approximately 55 bytes.

Protobuf binary:

Field 1 (email): tag + length + value bytes. Field 2 (password): tag + length + value bytes.

Approximately 38 bytes.

For a simple two-field message, the difference is modest. Now consider a thousand-field enterprise payload. Field names alone in JSON can account for 40-60% of the total payload size. In protobuf, field names contribute zero bytes to the payload.

On a 2G connection where bandwidth can be as low as 50 kilobits per second, the difference between an 80 kilobyte JSON response and a 15 kilobyte protobuf response is the difference between a 13-second load and a 2-second load. For users in areas with limited network infrastructure, this isn't a performance metric. It's a usability threshold.

Speed Comparison

Protobuf serialization and deserialization is significantly faster than JSON parsing because binary parsing requires no string tokenizing, quote handling, whitespace skipping, or type inference. The parser reads a field number, reads the value type, reads the value, and moves to the next field. It's a direct binary read.

JSON parsing must tokenize a string character by character, identify keys and values by their surrounding quotes and delimiters, infer types from the value format, and construct objects from dynamic maps.

On a mobile device handling hundreds of responses per session, this parsing difference translates to measurable CPU and battery savings.

Schema and Type Safety

JSON has no schema enforcement at the network level. A backend can change "balance" to "current_balance" and the client only discovers this when the app crashes in production.

Protobuf schemas are enforced at compile time. If the .proto file changes in a way that breaks the client, the client fails to compile. The problem is caught before it reaches any user.

The Honest Comparison

The Protoc Compiler and Code Generation

The protoc compiler reads your .proto file and generates code in any language you specify. This is where the universal contract becomes reality.

Generating Go code (for the backend server):

protoc \
  --go_out=. \
  --go-grpc_out=. \
  proto/banking.proto

Generates:

banking.pb.go        <- the message structs
banking_grpc.pb.go   <- the server interface

Generating Dart code (for the Flutter client):

protoc \
  --dart_out=grpc:lib/generated \
  proto/banking.proto

Generates:

lib/generated/
  banking.pb.dart        <- the message classes
  banking.pbgrpc.dart    <- the client stub

Generating Python code (for a data service):

protoc \
  --python_out=. \
  --grpc_python_out=. \
  proto/banking.proto

Generates:

banking_pb2.py         <- the message classes
banking_pb2_grpc.py    <- the client and server classes

Generating TypeScript code (for a web frontend):

protoc \
  --ts_out=. \
  proto/banking.proto

Generates:

banking.ts             <- typed message classes and client

All of this from the same single banking.proto file.

The Go backend engineer never writes serialization code. The Flutter engineer never writes deserialization code. The Python data engineer never parses binary manually. The TypeScript web engineer never constructs HTTP requests. All of that is generated automatically from the contract that every team agreed on.

Here's what the generated code looks like in each language to make this concrete:

Generated Go server interface (the backend implements this):

// Generated — do not edit
type BankingServiceServer interface {
    Login(context.Context, *LoginRequest) (*LoginResponse, error)
    WatchBalance(*BalanceRequest, BankingService_WatchBalanceServer) error
    mustEmbedUnimplementedBankingServiceServer()
}

// The Go backend engineer writes this implementation
type bankingServer struct {
    pb.UnimplementedBankingServiceServer
}

func (s *bankingServer) Login(
    ctx context.Context,
    req *pb.LoginRequest,
) (*pb.LoginResponse, error) {
    token, err := authService.Login(req.Email, req.Password)
    if err != nil {
        return nil, status.Errorf(codes.Unauthenticated, "invalid credentials")
    }
    return &pb.LoginResponse{
        Token:  token,
        UserId: user.Id,
    }, nil
}

Generated Python client (the data team uses this):

import grpc
import banking_pb2
import banking_pb2_grpc

channel = grpc.secure_channel(
    'api.fintech-platform.com:50051',
    grpc.ssl_channel_credentials()
)
stub = banking_pb2_grpc.BankingServiceStub(channel)

response = stub.Login(banking_pb2.LoginRequest(
    email='john@example.com',
    password='password123'
))

print(f"Token: {response.token}")
print(f"User ID: {response.user_id}")

Generated Dart client (you use this in Flutter):

import 'package:grpc/grpc.dart';
import 'generated/banking.pbgrpc.dart';
import 'generated/banking.pb.dart';

final channel = ClientChannel('api.fintech-platform.com', port: 50051);
final client = BankingServiceClient(channel);

final response = await client.login(
  LoginRequest(email: 'john@example.com', password: 'password123'),
);

print('Token: ${response.token}');
print('User ID: ${response.userId}');

Three different languages. Three different teams. One .proto file. All of them are generated, strongly typed, and guaranteed to be in sync with the server.

What is gRPC?

gRPC is Google's open-source Remote Procedure Call framework. It was open-sourced in 2016 and is now a Cloud Native Computing Foundation (CNCF) graduated project. This means it's been production-proven at the highest level of the cloud-native ecosystem.

gRPC combines three things:

1. Remote Procedure Calls as the programming model: calling remote functions like local ones.

2. Protocol Buffers as the interface definition language and data serialization format: strongly typed contracts and compact binary encoding.

3. HTTP/2 as the transport protocol: multiplexed, persistent connections with binary framing.

The combination of these three produces a framework that's faster than REST, more structured than WebSockets, and more powerful than any of its predecessors.

gRPC is used internally at Google for virtually all service-to-service communication. Netflix, Uber, Square, Dropbox, Lyft, and hundreds of other organizations use it for their internal microservice communication. Official support exists for Go, Java, Python, C++, C#, Ruby, Node.js, PHP, Dart, Kotlin, and more.

Why HTTP/2 Matters for gRPC

gRPC is built exclusively on HTTP/2. Understanding what HTTP/2 provides is essential to understanding why gRPC performs the way it does.

HTTP/1.1, which powers most REST APIs, has fundamental performance constraints. Each request must complete before the next one begins on the same connection. Headers are sent as verbose text on every request. The server can't send data unless the client asks first.

HTTP/2 was designed to fix these constraints at the protocol level.

Multiplexing

HTTP/2 introduces streams within a single connection. Multiple independent requests can travel over the same TCP connection simultaneously.

Single TCP connection to api.fintech-platform.com

Stream 1: Login request ---------> Login response
Stream 2: Profile request -------> Profile response
Stream 3: Balance request -------> Balance stream (ongoing)
Stream 4: Transactions request --> Transaction stream (ongoing)

All four streams active simultaneously over ONE connection

In HTTP/1.1, you would need four separate connections or wait for each to complete before starting the next. HTTP/2 handles all four over a single persistent connection with no waiting.

This is the foundation of gRPC's streaming capabilities. A persistent multiplexed connection is what allows the server to keep pushing balance updates and transaction notifications while the client continues making other calls.

Binary Framing

HTTP/1.1 sends everything as text. HTTP/2 sends everything as binary frames. Binary is more compact and significantly faster for machines to parse.

Every gRPC message is broken into binary frames and sent over the HTTP/2 connection. Combined with protobuf's binary encoding, gRPC data travels in the most compact form possible at every layer.

Header Compression (HPACK)

HTTP/1.1 sends full headers on every request. An Authorization header carrying a JWT token can be 500 bytes or more, repeated on every request.

HTTP/2 uses HPACK compression. Headers sent on previous requests are cached. Subsequent requests only send headers that changed. The Authorization header, once sent, is referenced by a short index rather than retransmitted in full.

On a mobile application making dozens of authenticated requests per session, this compression is a meaningful bandwidth saving, especially on slow networks where every byte matters.

Server Push

HTTP/2 allows the server to proactively send data to the client without waiting for a request. The client opens a stream and the server keeps pushing messages through it as events occur.

This is the mechanism behind gRPC server streaming. The client sends one WatchBalance request and the server pushes a new BalanceResponse every time the balance changes. No polling or repeated requests. The connection stays open and the server speaks whenever it has something new to say.

The Four gRPC Communication Patterns

This is the most important section of this article. gRPC doesn't have one communication model. It has four. Each one is defined precisely in the .proto file and serves different use cases.

Pattern 1: Unary RPC

One request from the client and one response from the server. This is identical to a REST API call in terms of the request-response flow.

rpc Login (LoginRequest) returns (LoginResponse);

Client ----LoginRequest----> Server
Client <---LoginResponse---- Server
Done.

When to use Unary RPC: Login, profile fetch, payment initiation, data creation, configuration retrieval: any operation that follows a simple ask-and-answer pattern.

Dart implementation:

Future<LoginResponse> login(String email, String password) async {
  try {
    return await _client.login(
      LoginRequest(email: email, password: password),
    );
  } on GrpcError catch (e) {
    throw _mapGrpcError(e);
  }
}

Go server implementation:

func (s *bankingServer) Login(
    ctx context.Context,
    req *pb.LoginRequest,
) (*pb.LoginResponse, error) {
    user, err := s.authService.Login(req.Email, req.Password)
    if err != nil {
        return nil, status.Errorf(codes.Unauthenticated, "invalid credentials: %v", err)
    }
    token, _ := s.tokenService.Generate(user.Id)
    return &pb.LoginResponse{
        Token:  token,
        UserId: user.Id,
    }, nil
}

Pattern 2: Server Streaming RPC

One request from the client and a continuous stream of responses from the server. The connection stays open and the server pushes messages as they become available.

rpc WatchBalance (BalanceRequest) returns (stream BalanceResponse);
rpc StreamTransactions (TransactionRequest) returns (stream Transaction);

Client ----BalanceRequest----> Server
Client <---BalanceResponse---- Server (balance: 500000)
Client <---BalanceResponse---- Server (balance: 495000, after a debit)
Client <---BalanceResponse---- Server (balance: 995000, after a credit)
[stream stays open, server pushes on every change]

When to use Server Streaming: Live account balance, real-time transaction notifications, live stock prices, sports scores, news feeds, system monitoring dashboards: anything where the server has an ongoing series of updates to deliver.

Dart implementation:

Stream<BalanceResponse> watchBalance(String userId) {
  return _client.watchBalance(
    BalanceRequest(userId: userId),
  );
}

In Flutter, consume this with a StreamBuilder:

StreamBuilder<BalanceResponse>(
  stream: _dataSource.watchBalance(currentUserId),
  builder: (context, snapshot) {
    if (snapshot.connectionState == ConnectionState.waiting) {
      return const CircularProgressIndicator();
    }

    if (snapshot.hasError) {
      return Text('Error: ${snapshot.error}');
    }

    if (!snapshot.hasData) {
      return const Text('Waiting for balance...');
    }

    final balance = snapshot.data!;
    return Column(
      children: [
        Text(
          '${balance.currency} ${balance.balance.toStringAsFixed(2)}',
          style: const TextStyle(
            fontSize: 36,
            fontWeight: FontWeight.bold,
          ),
        ),
        Text(
          'Updated: ${DateTime.fromMillisecondsSinceEpoch(balance.timestamp.toInt())}',
        ),
      ],
    );
  },
)

Every time the server pushes a new balance, the StreamBuilder calls builder again and the widget shows the updated value. There's zero polling logic or manual refresh. The server speaks and the widget listens.

Go server implementation:

func (s *bankingServer) WatchBalance(
    req *pb.BalanceRequest,
    stream pb.BankingService_WatchBalanceServer,
) error {
    ticker := time.NewTicker(time.Second)
    defer ticker.Stop()

    for {
        select {
        case <-stream.Context().Done():
            return nil
        case <-ticker.C:
            balance, err := s.accountService.GetBalance(req.UserId)
            if err != nil {
                return status.Errorf(codes.Internal, "failed to fetch balance: %v", err)
            }

            if err := stream.Send(&pb.BalanceResponse{
                Balance:   balance.Amount,
                Currency:  balance.Currency,
                Timestamp: time.Now().UnixMilli(),
            }); err != nil {
                return err
            }
        }
    }
}

Pattern 3: Client Streaming RPC

The client sends a stream of messages to the server. The server processes them all and responds once at the end.

rpc UploadDocument (stream DocumentChunk) returns (UploadResponse);

Client ----Chunk 1 (bytes 0-1024)-----> Server
Client ----Chunk 2 (bytes 1024-2048)--> Server
Client ----Chunk 3 (bytes 2048-3072)--> Server
Client ----Chunk 4 (last chunk)-------> Server
Client <---UploadResponse-------------- Server (document_id: "doc_001")

When to use Client Streaming: Uploading large files (KYC documents, profile photos) in chunks, sending a batch of sensor readings, submitting bulk records to a server.

Dart implementation:

Future<UploadResponse> uploadDocument(
  List<Uint8List> chunks,
  String documentType,
) async {
  try {
    Stream<DocumentChunk> chunkStream() async* {
      for (int i = 0; i < chunks.length; i++) {
        yield DocumentChunk(
          data: chunks[i],
          documentType: documentType,
          chunkIndex: i,
          isLast: i == chunks.length - 1,
        );
      }
    }

    return await _client.uploadDocument(chunkStream());
  } on GrpcError catch (e) {
    throw _mapGrpcError(e);
  }
}

chunkStream() is an async generator function. The async* keyword means it yields values over time rather than returning a single value. Each yield produces one DocumentChunk message that gRPC sends to the server. The server receives these one by one and processes them all before sending a single UploadResponse at the end.

Go server implementation:

func (s *bankingServer) UploadDocument(
    stream pb.BankingService_UploadDocumentServer,
) error {
    var allData []byte
    var documentType string

    for {
        chunk, err := stream.Recv()
        if err == io.EOF {
            break
        }
        if err != nil {
            return status.Errorf(codes.Internal, "failed to receive chunk: %v", err)
        }

        allData = append(allData, chunk.Data...)
        documentType = chunk.DocumentType
    }

    docId, err := s.documentService.Store(allData, documentType)
    if err != nil {
        return status.Errorf(codes.Internal, "failed to store document: %v", err)
    }

    return stream.SendAndClose(&pb.UploadResponse{
        Success:    true,
        DocumentId: docId,
        Message:    "Document uploaded successfully",
    })
}

Pattern 4: Bidirectional Streaming RPC

Both the client and server stream messages simultaneously. Both sides can send at any time. Neither waits for the other.

rpc Chat (stream ChatMessage) returns (stream ChatMessage);

Client ----"Hello"---------------------------> Server
Server <---"Hi, how can I help?"-------------- Client
Client ----"What is my account balance?"-----> Server
Server <---"Your balance is NGN 500,000"------ Client
Server <---"New transaction alert: -5,000"---- Client (server-initiated)
Client ----"Thanks"--------------------------> Server
[both sides communicate freely and simultaneously]

When to use Bidirectional Streaming: Real-time chat, live collaborative document editing, multiplayer game state synchronization, interactive trading terminals, real-time customer support sessions.

Dart implementation:

void startChat(String userId) {
  final outgoing = StreamController<ChatMessage>();

  final incoming = _client.chat(outgoing.stream);

  incoming.listen(
    (message) {
      print('${message.senderId}: ${message.content}');
    },
    onError: (error) {
      print('Chat error: $error');
    },
    onDone: () {
      print('Chat session ended');
    },
  );

  outgoing.add(ChatMessage(
    senderId: userId,
    content: 'Hello, I need help with my account',
    timestamp: DateTime.now().millisecondsSinceEpoch,
  ));
}

StreamController manages the outgoing message stream. You add messages to outgoing whenever the user sends something. The incoming stream delivers messages from the server. Both run simultaneously over the same HTTP/2 connection.

Go server implementation:

func (s *bankingServer) Chat(stream pb.BankingService_ChatServer) error {
    for {
        msg, err := stream.Recv()
        if err == io.EOF {
            return nil
        }
        if err != nil {
            return err
        }

        response := s.chatService.Process(msg)
        if err := stream.Send(&pb.ChatMessage{
            SenderId:  "support_agent",
            Content:   response,
            Timestamp: time.Now().UnixMilli(),
        }); err != nil {
            return err
        }
    }
}

The Protobuf Repository: Organizational Best Practice

In a small project, the .proto file can live inside the backend repository. The mobile engineer clones the backend repo to get it. This works at small scale.

In any organization of meaningful size, this approach breaks down. The backend repo becomes the source of truth, giving the backend team unilateral control over the contract. Other teams find out about changes when their builds break.

The industry best practice is a dedicated protobuf repository: a standalone repository that belongs to everyone and is owned exclusively by no one.

fintech-api-contracts/
  proto/
    auth/
      auth.proto
    banking/
      banking.proto
    payments/
      payments.proto
    notifications/
      notifications.proto
    kyc/
      kyc.proto
  scripts/
    generate_dart.sh
    generate_go.sh
    generate_python.sh
  README.md

How Contract Changes Work

Every API change follows the same process:

Engineer proposes a change to banking.proto
          |
          raises a Pull Request in fintech-api-contracts
          |
Flutter team lead reviews:
  "Does this break our client? Do we need to update?"

Go backend lead reviews:
  "Is this implementable? Does it follow our conventions?"

React web lead reviews:
  "Does the web client need changes?"

Python data lead reviews:
  "Does this affect our data pipelines?"
          |
All teams approve
          |
PR merges — the change is now the law
          |
Every team runs their code generation script
          |
Builds fail where breaking changes exist
Changes are caught at compile time
Before any code reaches production

This process gives you something REST with documentation can never provide: guaranteed contract synchronization across every team, enforced by the compiler, before anything reaches users.

Building a Complete gRPC System with Dart and Flutter

Now let's put everything together in a complete, production-structured example.

Project Setup

Add the gRPC dependency to your Flutter project:

# pubspec.yaml
dependencies:
  flutter:
    sdk: flutter
  grpc: ^3.2.4
  protobuf: ^3.1.0

dev_dependencies:
  protoc_plugin: ^21.1.2

Install the protoc compiler and the Dart plugin:

# macOS
brew install protobuf

# Install the Dart protoc plugin
dart pub global activate protoc_plugin

Generate the Dart code from the proto file:

protoc \
  --dart_out=grpc:lib/generated \
  -I proto \
  proto/banking/banking.proto

The Data Source Layer

// lib/features/banking/data/datasources/banking_remote_datasource.dart

import 'package:grpc/grpc.dart';
import '../../../../generated/banking.pb.dart';
import '../../../../generated/banking.pbgrpc.dart';
import '../../../../core/error/app_exception.dart';

class BankingRemoteDataSource {
  late final BankingServiceClient _client;
  late final ClientChannel _channel;

  BankingRemoteDataSource({
    required String host,
    required int port,
    required String authToken,
  }) {
   
    _channel = ClientChannel(
      host,
      port: port,
      options: const ChannelOptions(
        credentials: ChannelCredentials.secure(),
        connectionTimeout: Duration(seconds: 10),
      ),
    );

   
    _client = BankingServiceClient(
      _channel,
      options: CallOptions(
        metadata: {'authorization': 'Bearer $authToken'},
        timeout: const Duration(seconds: 30),
      ),
    );
  }

  Future<LoginResponse> login(String email, String password) async {
    try {
      return await _client.login(
        LoginRequest(email: email, password: password),
      );
    } on GrpcError catch (e) {
      throw _mapGrpcError(e);
    }
  }

  Future<UserProfile> getProfile(String userId) async {
    try {
      return await _client.getProfile(
        ProfileRequest(userId: userId),
      );
    } on GrpcError catch (e) {
      throw _mapGrpcError(e);
    }
  }

  Stream<BalanceResponse> watchBalance(String userId) {
    return _client
        .watchBalance(BalanceRequest(userId: userId))
        .handleError((error) {
      if (error is GrpcError) throw _mapGrpcError(error);
      throw error;
    });
  }

  Stream<Transaction> streamTransactions(String userId, {int limit = 20}) {
    return _client
        .streamTransactions(
          TransactionRequest(userId: userId, limit: limit),
        )
        .handleError((error) {
      if (error is GrpcError) throw _mapGrpcError(error);
      throw error;
    });
  }

  Future<UploadResponse> uploadDocument(
    List<Uint8List> chunks,
    String documentType,
  ) async {
    try {
      Stream<DocumentChunk> chunkStream() async* {
        for (int i = 0; i < chunks.length; i++) {
          yield DocumentChunk(
            data: chunks[i],
            documentType: documentType,
            chunkIndex: i,
            isLast: i == chunks.length - 1,
          );
        }
      }
      return await _client.uploadDocument(chunkStream());
    } on GrpcError catch (e) {
      throw _mapGrpcError(e);
    }
  }

  ResponseStream<ChatMessage> startChat(Stream<ChatMessage> outgoing) {
    return _client.chat(outgoing);
  }

  Future<void> dispose() async {
    await _channel.shutdown();
  }

  AppException _mapGrpcError(GrpcError error) {
    switch (error.code) {
      case StatusCode.unauthenticated:
        return AppException.unauthorized(
          message: error.message ?? 'Unauthorized',
        );
      case StatusCode.notFound:
        return AppException.notFound(
          message: error.message ?? 'Not found',
        );
      case StatusCode.deadlineExceeded:
        return AppException.timeout(message: 'Request timed out');
      case StatusCode.unavailable:
        return AppException.serverUnavailable(
          message: 'Service unavailable',
        );
      default:
        return AppException.unknown(
          message: error.message ?? 'Unknown error',
        );
    }
  }
}

Let's walk through the important decisions in this data source.

The Channel:

_channel = ClientChannel(
  host,
  port: port,
  options: const ChannelOptions(
    credentials: ChannelCredentials.secure(),
    connectionTimeout: Duration(seconds: 10),
  ),
);

The channel is the physical HTTP/2 connection to the server. You create it once and reuse it for every call. ChannelCredentials.secure() enables TLS encryption. connectionTimeout prevents the app from waiting indefinitely if the server is unreachable.

The channel is the core of gRPC's performance advantage. A single persistent channel multiplexes all requests through one HTTP/2 connection. Creating a new channel per request would eliminate this advantage entirely and perform worse than REST.

Authentication via Metadata:

_client = BankingServiceClient(
  _channel,
  options: CallOptions(
    metadata: {'authorization': 'Bearer $authToken'},
    timeout: const Duration(seconds: 30),
  ),
);

gRPC uses metadata (key-value pairs) for what HTTP uses headers. Passing the auth token as metadata on the CallOptions means every single call made through this client automatically includes the Authorization metadata. You write it once. It applies everywhere.

Error Mapping:

AppException _mapGrpcError(GrpcError error) {
  switch (error.code) {
    case StatusCode.unauthenticated:
      return AppException.unauthorized(...);
    case StatusCode.deadlineExceeded:
      return AppException.timeout(...);
    ...
  }
}

gRPC has its own set of status codes similar to HTTP status codes but not identical. Mapping them to your application's exception types at the data source layer means the rest of your code (use cases, notifiers, widgets) never deals with gRPC-specific errors directly. Your domain layer stays clean and framework-independent.

The Repository Layer

abstract class BankingRepository {
  Future<Result<UserProfile, AppException>> getProfile(String userId);
  Stream<BalanceResponse> watchBalance(String userId);
  Stream<Transaction> streamTransactions(String userId);
  Future<Result<UploadResponse, AppException>> uploadDocument(
    List<Uint8List> chunks,
    String documentType,
  );
}


class BankingRepositoryImpl implements BankingRepository {
  final BankingRemoteDataSource _dataSource;

  BankingRepositoryImpl(this._dataSource);

  @override
  Future<Result<UserProfile, AppException>> getProfile(String userId) async {
    try {
      final profile = await _dataSource.getProfile(userId);
      return Result.success(profile);
    } on AppException catch (e) {
      return Result.failure(e);
    }
  }

  @override
  Stream<BalanceResponse> watchBalance(String userId) {
    return _dataSource.watchBalance(userId);
  }

  @override
  Stream<Transaction> streamTransactions(String userId) {
    return _dataSource.streamTransactions(userId);
  }

  @override
  Future<Result<UploadResponse, AppException>> uploadDocument(
    List<Uint8List> chunks,
    String documentType,
  ) async {
    try {
      final response = await _dataSource.uploadDocument(chunks, documentType);
      return Result.success(response);
    } on AppException catch (e) {
      return Result.failure(e);
    }
  }
}

The Riverpod Providers

part 'banking_providers.g.dart';

@riverpod
Stream<BalanceResponse> balanceStream(BalanceStreamRef ref, String userId) {
  final repository = ref.watch(bankingRepositoryProvider);
  return repository.watchBalance(userId);
}

@riverpod
Stream<Transaction> transactionStream(
  TransactionStreamRef ref,
  String userId,
) {
  final repository = ref.watch(bankingRepositoryProvider);
  return repository.streamTransactions(userId);
}

With Riverpod, a provider that returns a Stream automatically becomes an AsyncValue that widgets can watch. Every new value pushed from the gRPC server stream triggers a widget rebuild automatically.

The UI

// lib/features/banking/presentation/pages/dashboard_page.dart
class DashboardPage extends ConsumerWidget {
  final String userId;

  const DashboardPage({required this.userId, super.key});

  @override
  Widget build(BuildContext context, WidgetRef ref) {
    final balanceAsync = ref.watch(balanceStreamProvider(userId));
    final transactionsAsync = ref.watch(transactionStreamProvider(userId));

    return Scaffold(
      appBar: AppBar(title: const Text('Dashboard')),
      body: Column(
        children: [
          balanceAsync.when(
            data: (balance) => BalanceCard(
              amount: balance.balance,
              currency: balance.currency,
            ),
            loading: () => const BalanceShimmer(),
            error: (e, _) => ErrorCard(message: e.toString()),
          ),

          const SizedBox(height: 24),

          Expanded(
            child: transactionsAsync.when(
              data: (transaction) => TransactionTile(
                transaction: transaction,
              ),
              loading: () => const TransactionShimmer(),
              error: (e, _) => ErrorCard(message: e.toString()),
            ),
          ),
        ],
      ),
    );
  }
}

Both the balance and transactions arrive through gRPC server streams. Both update in real-time as the server pushes new data. Both are handled identically through Riverpod's AsyncValue pattern. One framework with all patterns covered.

Production Concerns

Authentication with Interceptors

For more granular authentication control, such as refreshing an expired token and retrying automatically, you implement a client interceptor:

class AuthInterceptor extends ClientInterceptor {
  final TokenService _tokenService;

  AuthInterceptor(this._tokenService);

  @override
  ResponseFuture<R> interceptUnary<Q, R>(
    ClientMethod<Q, R> method,
    Q request,
    CallOptions options,
    ClientUnaryInvoker<Q, R> invoker,
  ) {
    final token = _tokenService.currentToken;
    final authenticatedOptions = options.mergedWith(
      CallOptions(metadata: {'authorization': 'Bearer $token'}),
    );
    return invoker(method, request, authenticatedOptions);
  }

  @override
  ResponseStream<R> interceptServerStreaming<Q, R>(
    ClientMethod<Q, R> method,
    Q request,
    CallOptions options,
    ClientServerStreamingInvoker<Q, R> invoker,
  ) {
    final token = _tokenService.currentToken;
    final authenticatedOptions = options.mergedWith(
      CallOptions(metadata: {'authorization': 'Bearer $token'}),
    );
    return invoker(method, request, authenticatedOptions);
  }
}

Pass the interceptor when creating the client:

_client = BankingServiceClient(
  _channel,
  interceptors: [AuthInterceptor(tokenService)],
);

The interceptor fires on every call automatically. The data source code never touches auth logic directly.

Error Handling: gRPC Status Codes

gRPC defines a standard set of status codes that every implementation follows:

AppException _mapGrpcError(GrpcError error) {
  switch (error.code) {
    case StatusCode.unauthenticated:
      return AppException.unauthorized(message: 'Session expired');
    case StatusCode.permissionDenied:
      return AppException.forbidden(message: 'Access denied');
    case StatusCode.notFound:
      return AppException.notFound(message: error.message ?? 'Not found');
    case StatusCode.deadlineExceeded:
      return AppException.timeout(message: 'Request timed out');
    case StatusCode.unavailable:
      return AppException.serverUnavailable(message: 'Service unavailable');
    case StatusCode.resourceExhausted:
      return AppException.rateLimited(message: 'Too many requests');
    case StatusCode.invalidArgument:
      return AppException.validation(
        message: error.message ?? 'Invalid input',
      );
    default:
      return AppException.unknown(
        message: error.message ?? 'An error occurred',
      );
  }
}

Deadlines and Timeouts

Every gRPC call should have a deadline. Without deadlines, a slow server can make your app hang indefinitely.

Per-call deadline:

Future<UserProfile> getProfile(String userId) async {
  return await _client.getProfile(
    ProfileRequest(userId: userId),
    options: CallOptions(timeout: const Duration(seconds: 10)),
  );
}

Default deadline for all calls:

_client = BankingServiceClient(
  _channel,
  options: CallOptions(
    timeout: const Duration(seconds: 30),
    metadata: {'authorization': 'Bearer $authToken'},
  ),
);

When the deadline is exceeded, the call throws a GrpcError with StatusCode.deadlineExceeded, which your error mapper handles appropriately.

Logging Interceptor

class LoggingInterceptor extends ClientInterceptor {
  @override
  ResponseFuture<R> interceptUnary<Q, R>(
    ClientMethod<Q, R> method,
    Q request,
    CallOptions options,
    ClientUnaryInvoker<Q, R> invoker,
  ) {
    final stopwatch = Stopwatch()..start();
    debugPrint('[gRPC] --> ${method.path}');

    final response = invoker(method, request, options);

    response.then((_) {
      stopwatch.stop();
      debugPrint(
        '[gRPC] <-- ${method.path} (${stopwatch.elapsedMilliseconds}ms)',
      );
    }).catchError((error) {
      stopwatch.stop();
      debugPrint(
        '[gRPC] ERROR ${method.path}: $error (${stopwatch.elapsedMilliseconds}ms)',
      );
    });

    return response;
  }
}

gRPC vs REST vs WebSockets: When to Use What

When to Use REST

The API is consumed by third-party developers or external partners. JSON over HTTP is the universal language that every developer in every language can access immediately without learning new tooling.

The operation is simple request-response with no streaming requirements and only one client platform. REST is simpler to implement, simpler to debug, and simpler to test for straightforward CRUD operations.

Public documentation and human readability matter. REST with OpenAPI/Swagger gives you browsable, testable documentation that developers can explore in a browser.

Caching is important. REST GET responses can be cached at every layer: CDN, reverse proxy, browser cache. gRPC requests can't leverage standard HTTP caching.

Use WebSockets When

You need true bidirectional real-time communication and gRPC isn't already in your stack. Chat applications, multiplayer games, and collaborative tools where both sides need to speak freely are natural WebSocket use cases.

Browser support without a proxy layer is required. WebSockets work natively in every modern browser. gRPC in the browser requires gRPC-Web and a proxy layer.

When to Use gRPC

Multiple platform teams share the same service contract. When Flutter, React, Go, and Python services all call the same backend, a .proto file enforced by the compiler prevents contract drift across every team.

Large payloads are called by many internal applications. The more fields in the payload and the more applications consuming it, the stronger the case for protobuf's binary encoding and generated clients.

Network conditions are variable and payload size matters. Users on 2G or 3G connections benefit directly from protobuf's compact binary format. The same data in protobuf can be 3 to 10 times smaller than JSON, translating to faster load times and lower data consumption for users on limited data plans.

Real-time streaming is required and you want one unified framework for all communication patterns. gRPC's four patterns cover every scenario without requiring a separate WebSocket server alongside your API.

Service-to-service communication at high frequency is involved. Two internal services calling each other thousands of times per second over a persistent multiplexed HTTP/2 connection with binary protobuf encoding will significantly outperform REST with JSON over HTTP/1.1.

The Hybrid Architecture

The mature engineering decision is not choosing gRPC over REST or REST over gRPC. It's knowing where each belongs and using both deliberately.

Most organizations of meaningful scale end up with a hybrid:

The public REST API serves external consumers who need simplicity and JSON. The internal gRPC network handles high-frequency, high-performance service-to-service calls. The mobile gRPC endpoints give Flutter clients real-time capabilities over efficient binary connections.

Each layer uses the right tool for its specific requirements. No ideological commitment to one protocol. Pure engineering pragmatism.

Conclusion

Remote Procedure Calls began with a simple observation: network communication shouldn't require developers to think about network communication. Calling a function on another machine should feel like calling a function on your own.

Protocol Buffers took this further by solving the data problem. JSON is readable but verbose. Binary encoding with a compiler-enforced schema produces payloads that are smaller, faster to parse, and guaranteed to match the contract every team agreed on. For users on slow networks and internal systems processing millions of requests daily, this efficiency is a business advantage.

gRPC combined RPC semantics, Protocol Buffer encoding, and HTTP/2 transport into a framework that supports four distinct communication patterns: unary request-response, server streaming, client streaming, and bidirectional streaming. All from the same generated client, using the same persistent connection, and enforced by the same .proto contract.

The organizational practice of a shared protobuf repository transforms gRPC from a technical tool into an engineering discipline. Contract changes go through review. Breaking changes are caught by the compiler. Every team generates their own strongly typed client from the same source of truth and stays in sync automatically, regardless of programming language.

In Flutter specifically, gRPC server streams integrate naturally with Dart's Stream type and Riverpod's stream providers. Real-time balance updates and live transaction feeds that would require polling with REST or a separate WebSocket implementation become simple stream subscriptions. The server pushes, the widget listens, and nothing else is required.

The decision of when to use gRPC versus REST versus WebSockets isn't about preference. It's about matching the tool to the requirement. Public APIs belong behind REST. High-frequency internal service communication belongs on gRPC. Large payloads consumed by many internal systems belong in protobuf. Real-time bidirectional features belong on gRPC streaming. Users on variable networks deserve the smallest payloads you can give them.

Understanding all of these tools, understanding why they exist, and knowing when to reach for each one is what separates engineers who use tools from engineers who think in systems.

Happy Coding!

For those unfamiliar, a PIP at Google is a two-month plan to show improvement – a final chance to prove yourself. You’re given a project and a strict deadline. Deliver successfully, or you’re fired. There are no extensions, no middle ground.

Scary thoughts about providing for my family’s finances raced through my mind. But my deeper fear was this: what story would I carry about myself if I tried to persevere and failed?

If I got fired, I would need to face job interviews. And I knew the question would come: “Tell me about a project you worked on at Google that you’re most proud of.” The honest answer was that I didn’t have one. I hadn’t yet excelled at a project, hadn’t gone deep enough into any system to truly own it. I imagined myself sitting in an interview, with a blank face, with nothing to say.

That dreadful image became my motivation. I wanted a project I could truly own, something I could explain inside and out, regardless of how the PIP ended. I’m also not the type of person who simply backs down when things get tough. I needed to prove to myself that I could rise up. I was gonna give the project everything I had, week after week after week. That singular commitment became the start of my transformation into a more focused, disciplined engineer.

In this guide, you’ll learn how to turn professional setbacks into catalysts for growth. While examining my journey on Google’s Performance Improvement Plan, I’ll show you how to face underperformance head-on, rebuild your confidence, and come out stronger than before. You’ll see how focus, discipline, and gratitude can turn the lowest points of your career into launch ramps for acceleration.

Here’s what I’ll cover:

1. The Backstory

2. The PIP Begins

3. The Project

4. Fatherhood

5. Letting Go

6. What’s Next

7. Closing

The Backstory

To understand how I landed in that chair across from my manager, you need to know where I came from. Before Google, I’d worked at Meta. I was hired there as an IC3 (entry-level engineer) and promoted after a year to IC4 (mid-level). But that promotion didn't come from technical excellence. It came from my connecting our engineering projects to the business’ needs.

I had worked on a payments system used by large enterprises. By sitting with the operations staff and customer service reps, I spotted inefficiencies and built small features that saved time, reduced errors, and allowed the team to scale. Those changes had a big impact, and they earned me a promotion. But, in hindsight, my success had come from soft skills such as teamwork and business awareness. I hadn’t actually developed the technical knowledge expected of an IC4.

On my team at Google, technical mastery was the main thing we were measured on, while business awareness was a side point. On top of that, I had recently immigrated from the United States to Israel and needed to learn the local language of Hebrew and the local culture. It was a lot all at once: new country, new language, new company, and strong engineering expectations.

The gap between my technical skills and those of my peers eventually led to my being placed on a Performance Improvement Plan.

The PIP Begins

When the PIP started, I increased my working hours to 60 hours per week. I cut out almost everything else in my life – news, side projects, YouTube – and focused only on my work. When you’re falling behind, cutting distractions isn’t punishment. Rather, it’s how you create the quiet needed to actually improve.

It was brutal. Despite all the intense hours, I was slower than my coworkers. They shipped code confidently while I second-guessed myself. They reviewed my work and pointed out ways to improve it.

Some nights I walked away from my desk ready to cry. I was exhausted, and even after pouring in all those hours, I still wasn’t keeping pace. I felt defeated.

But I kept grinding away, day after day, week after week. Ignoring every side project and distraction forced me to confront the real issue: my lack of depth in the systems I was working on.

The Project

For confidentiality reasons, I won’t describe the actual project I was assigned. But here’s a similar example.

Imagine Google had a small music game built into Google Search. My task was to add a line of text above the game’s start button telling players how many people had reached the next level. The goal was to encourage more people to keep playing.

The text addition would run as an experiment. We’d launch it to a small percentage of users, measure the impact, and then either shut it down or roll it out to everyone.

The problem? At that time, Google Search didn’t even store how many players completed each level. So before I could add the text, I had to design and run a new data pipeline to track how many players completed each level of the game.

At first, I got lost. Just before my PIP began, our broader organization had been through a reorg, and my small team of five engineers was newly assigned to focus on games within Search. None of us had touched the gaming code before, and the particular game I was assigned hadn't had any meaningful updates in several years.

I spent days combing through design docs from 3-7 years earlier, only to find that the original authors had long since moved on. Each time I reached out, I’d get referred to someone else. Eventually, I found the current owner of the data storage systems for gaming. She had recently inherited them and hadn’t built the systems herself, but she helped me understand their current state.

With that clarity, I was finally making headway. But I realized I needed to rethink my priorities. Data pipeline code could be reviewed and deployed relatively quickly, while code changes to Google Search required slower, more comprehensive quality assurance checks. If I wanted to have any chance of meeting the PIP deadline, I had to shift focus to the Search-side work first.

As I dug deeper, another issue surfaced: the people who were listed as the engineering owners of the game hadn’t touched it in years and didn’t want to be involved anymore. I learned that our team would be running many more experiments on that code. Therefore, after consulting with my manager, I became the game’s code owner, the person ultimately responsible for all engineering decisions.

Taking ownership didn’t mean I suddenly moved fast or flawlessly. Some of my choices slowed me down. I aimed for near-perfect data accuracy when “mostly accurate” would have been enough for an experiment. I also wasted days digging through outdated documentation instead of simply reaching out to the people behind it.

After several days reading a four-year-old doc, I finally messaged the author. They immediately redirected me to someone else, who then forwarded me again. The third person turned out to be the current owner, and within minutes, they shared with me their private notes which clarified a ton.

But those mistakes were part of the learning curve. Each week, I dove deeper into the engineering tasks, internalized more of the systems, and made more progress than the week before.

By the time the final two weeks of the PIP arrived, I was operating at a whole new level. While the first month had felt like drowning, the last two weeks had felt like flying. I was excitedly diving into the code, unblocking myself, and helping teammates navigate the codebase.

That turnaround, from tears of frustration to the thrill of ownership, was exhilarating. For the first time at Google, I was independently driving my project forward. And I loved it.

When the PIP deadline arrived, though, I hadn’t yet delivered the full project. I was just a few hours of engineering work away from getting a working end-to-end flow with hardcoded data, but the actual data collection and experiment launch would have required about nine more days of engineering work.

On a PIP, “almost there” isn’t good enough. I was called in for a hearing with my director and HR, where I was given an opportunity to explain my case.

I didn’t walk into that final meeting empty-handed. I brought a detailed handoff plan listing the current state of the project, the remaining steps, and every contact and document another engineer would need to continue. I also brought a plan for improving collaboration amongst our various gaming engineers by creating a doc that would function as a centralized directory of all gaming systems, their owners, and their design docs. I offered to maintain this directory as a side effort, building it up naturally through my ongoing engineering work and conversations with past owners of the systems.

The hearing was an hour. I walked my director and HR through what I had shipped, my handoff plan, and my roadmap for unblocking future projects. I left the meeting proud of all that I’d learned over the previous two months. “Whatever will be will be,” I told myself.

A few days later, HR and my director called me back with their decision. Their feedback was straightforward: I had shown steady improvement, but I hadn’t delivered the final project on time, and therefore I hadn’t met expectations for my level.

Their feedback didn’t mention my handoff plan, my roadmap, nor my becoming the game’s code owner. That’s because a PIP isn’t a coaching program, it’s an evaluation. It doesn’t measure acceleration, it measures completion. It’s binary: You either deliver within the two months, or you don’t. And I hadn’t.

Upon hearing their decision, I thanked my director and the HR representative for having given me a final chance. I told them that the PIP had succeeded: it had built within me an internal engine of ownership over my engineering career. The fact that I would no longer be at Google was irrelevant. There would be no break in my internal transformation.

Fatherhood

The official decision closed one chapter. But the habits I’d built during the PIP of focus, ownership, and accountability began reshaping more than just my work. They changed how I saw myself as a father and husband.

Before the PIP, I’d take my toddler to the playground after work. During it, I was often too drained for that. I’d sit him in front of the TV while I caught up on writing code or reading documentation. Date nights with my wife slipped away too. For a while, I wondered: What kind of father and husband does that make me?

One night, I was listening to financial coach Dave Ramsey, a religious Christian who often brings faith into his talk show. He spoke about a father’s responsibility to provide for his family. It reframed how I saw my long hours. Had I made more disciplined decisions and strengthened my engineering skills months earlier, I never would have been placed on the PIP. The newer, more focused, harder-working version of me wasn’t the problem, it was the solution.

So as my son sat in front of the TV, I reminded myself: An earlier version of me had made decisions which resulted in my now being less available for my family. The new me, the disciplined me, hadn’t made that choice. My current unavailability was a course correction that needed to happen for me to become the type of father and husband I wanted to become.

Letting Go

When I was let go, I felt a little lost. One of my biggest worries was financial. Not only did I lose a high-paying job, but I was also frustrated that I wouldn’t receive the yearly bonus Google gives its employees. I had plans for how I would use it, and letting go of that expectation was difficult.

I spoke with my Rabbi about being let go. He told me: In Judaism, we believe that everything happens for a reason. If I lost the job, then God wanted me to lose it. He encouraged me to view my overall experience at Google in positive terms, and to focus on appreciation to God for having a plan for me. It made logical sense, but I was still frustrated about the loss of the bonus income.

The inner peace came later, when I realized something simple: I hadn’t earned that yearly bonus. My performance before the PIP hadn’t justified it. It made sense, in fact it felt right, that I didn’t receive it.

With that acceptance came space for gratitude, especially toward my former coworkers and managers. During those final two months, they reviewed my work, pointed out ways to improve it, answered my questions, and patiently explained how Google’s internal engineering systems worked. I will always be grateful for how much they taught me.

That gratitude extended to my manager as well. Several weeks after being let go, I visited the office one last time to say goodbye to my team. My manager explained that the decision to let me go had been a difficult one. He told me he liked me as a person and recognized how much I had improved during the PIP. But keeping me on would have required certainty that I was already operating at the expected engineering level, and that was something he wasn’t sure of. I understood his position. If I had been in his shoes, I would have made the same decision.

Because of the Performance Improvement Plan I had gained growth, humility, and clarity. Letting go was about moving forward with gratitude for what had gone right.

What’s Next

The PIP had given me something invaluable: structure and accountability. During those eight weeks, I lived by a project plan timeline, and when my time at Google ended, I didn’t let that habit go. The very first thing I did afterward was set up a new timeline, this time for my job hunt. Tasks were ordered by priority, with time estimates and due dates, so that my search itself became a disciplined project. My wife, or anyone else, could hold me accountable just as my manager once had.

As an example, the below table is a snippet from my job hunt timeline:

The key to creating my job hunting timeline was being clear on my priorities regarding what type of engineering position and what type of company I’d prefer to work for.

At Google, a company with tens of thousands of engineers, I used coding frameworks and technologies that were custom-built for Google engineers and used by no one outside of Google. I felt isolated from the greater world and to engineers outside of the company. So I want my next engineering role to be at a smaller company, where I’ll use popular open-source technologies and software used by engineers throughout the world.

To prepare myself for my next engineering role, I'm now laser-focused on upskilling my technical knowledge. I’ve been interviewing engineers at local startups about the technologies they use and then sharing the lessons publicly on LinkedIn. Each 1:1 interview and write-up helps close the skills gap that led to my firing at Google.

By following my written timeline and by knowing my end goal, I’ve been able to sustain long-term momentum in my job hunt.

Closing

Whatever will be will be. I’m grateful for the PIP experience, because it caused me to claw my way out of underperformance. It stripped away distractions, forced me to confront my engineering weaknesses head-on, and gave me the discipline to close the gap.

The momentum I built during those eight weeks never stopped. There was no break between week eight and week nine, just continuous acceleration. Week eight was about my PIP project, and week nine was about my job hunt. The external goals changed, but the internal engine kept running.

While my momentum softened the blow of getting fired, it didn’t erase the emotions that came with it. Sharing my story of being fired for underperformance has felt awkward and vulnerable, but also has given me a feeling of pride. Pride at who I’ve become. And pride in my giving back to the greater community, by enabling others facing similar struggles to learn from my story.

The eight weeks of the PIP were my launch ramp, and my acceleration continued long after the official PIP was over. To quote someone I know, “Like the mythical Phoenix, I believe in rising from the ashes, no matter how daunting the obstacle.” The PIP was my ashes, but it was also my fire.

Naturally, I had to prepare my source code as per instruction. In this article, I will share how I packaged and submitted my Add-on's source code in Ubuntu OS.

But before that, let's talk a little about Tier 2 CASA assessment.

What is Tier 2 CASA Security Assessment?

The Tier 2 CASA (Cloud Application Security Assessment) is a self-service security assessment process for applicants seeking access to Google Workspace data or to comply with specific Google Workspace policies.

It allows developers to scan their applications and submit the results for verification without an external assessor accessing the code or infrastructure.

Importance Tier 2 CASA security assessment

Tier 2 CASA is important for several reasons:

• Security Assurance: It provides independent verification of your application's security posture, reducing the risk of data breaches and protecting user privacy.
• Compliance: It helps meet security requirements for accessing Google Workspace data or adhering to Google policies, like the Workspace Marketplace Terms of Service.
• Efficiency: It's a faster and more cost-effective alternative to Tier 1 assessments, which involves external assessors directly examining your application.
• Trust: If your addon is published without verification it'll display an "unverified" message to the clients while installing the addon, which creates distrust and can lead to the installation process of your addon to be abandoned.

In the context of my Google Workspace Addon Scan Me, the use of restrictive OAuth scope auth/drive of Google Drive API likely triggered the need for a Tier 2 assessment. This scope grants your addon access to see, edit, create, and delete all of your Google Drive files, which falls under Google's security and privacy requirements.

Additional Resources

• CASA Tier 2 Overview
• CASA Documentation
• Google Workspace Marketplace Terms of Service

Disclaimer: While I'll explain the Tier 2 CASA process, it's crucial to consult the official documentation and Google's security guidelines for specific requirements and guidance.

The assessment certification is free, by the way. To prepare your addon for the CASA assessment process follow the following steps.

Step 1 – Sign up for the new Assessment Procedure

If you're using restrictive scopes, you'll receive an email from Google's Verification team at some point requesting to verify the scopes after you've submitted your add-on for verification.

This email is the notification document. So, you need to download this email as a PDF, which must be submitted in the application form later on.

In that email, you'll find the following instructions for Tier 2 evaluation. You'll find a link to register or log-in to the CASA portal. Click the link and register to the site. Then click on Start New Assessment> Create New Assessment.

Fill in the information asked carefully. Upload the previously downloaded email where you're asked for a Tier 2 notification pdf.

Starting New CASA Assesment of the Addon

Note: For Google Workspace Addon, the type of application is Local App.

Caution: As shown in the image above, even though "Project ID" is asked in the input field, they are asking for the Project Number included in the email, not the Project ID of your Google Cloud Console project.

After you carefully fill in the details and submit the form, you'll arrive at a new screen – Application Screening – where there are two things that you should download:

Download Scan Cenral Package and Setup Insruction

1. Fortify Scan Central Package.
2. Instruction on compressing your application's source code for initial assessment.

Step 2 – Download and Setup Java JDK

To use the Scan Central package as mentioned in the instructions, a minimum of JDK 11 is required.

For setting up the path for the Java environment in Linux, I followed this instruction on StackOverflow.

Step 3 – Setup Path for Scan Central

Now let's add the path to the Scan Central in our system.

In your CLI, open .bashrc file with sudo nano ~/.bashrc. Append the following path at the end of the file:

# SCAN Central 
# Path looks like following
#/home/<username>/Fortify_ScanCentral_Client_22.2.1_x64/bin

 export PATH=$PATH:<Path To bin folder in Scan Central>

Save (CTRL+S) and exit (CTRL + X) the file.

Open .profile with sudo nano ~/.profile and add the same path as above. You can check the version of Scan Central in your CLI with the command scancentral -version, to make sure the setup was successful.

Step 4 – Packaging Source Code for Assessment

To package the source code for your Google Workspace Addon, go to the root directory of your project. If you're following the instruction manual, go to the section for JavaScript code packaging.

In the root directory run any of the following commands:

#cmd 1 
scancentral package -bt none -o myPackage.zip
# or cmd 2
scancentral package -bt none --scan-node-modules -o myPackage.zip

Note: The command scancentral.bat is for Windows users.

As mentioned in the instruction, command 2 increases the size of the package and is not necessary for Node.js or Angular. I created Workspace Addon so I don't have node-modules in my source code.

After that, you'll see a compressed package named myPackage in the directory where you ran the packaging operation.

Step 5 – Initiate the Scan Process

After packaging, go back to the CASA portal and click on your assessment ID link in the list, and open up the Application Screening window. Here:

1. Click the Begin Scan Process button.
2. Upload the package you just compressed.
3. Click the Upload File & Initiate Scan button.

Upload Source Code To Fortify Scan

This will initiate auto scanning of your application which is the beginning of assessment for your Addon.

Reminder: As I've personally experienced, if your source code uses the Math.random() method, then the auto-scanner will not pass your code.

If you pass this phase, the manual verification process will begin where you'll have to fill in forms for the survey. Go to this link for the questions that'll be asked in the CASA survey. Here, choose the Local App option for App Type for a Google Workspace Add-on. I want to remind you that they will change based on the answer provided.

Conclusion

Alright, I'm hoping this blog helped you reduce the time and confusion that I had to encounter when I was trying to assess my addon. And please don't give up midway during the evaluation otherwise your months of hard work will be in vain.

My addon Scan Me, scans the Google Drive and prepares an audit report in a spreadsheet file of your choosing in your Google Drive. It makes it extremely easy for you to analyze your Google Drive from one place, and it also offers a free quota. If you're looking for a similar addon I hope you'll try this addon.

This is Nibesh khadka, have a good day.

Large Language Models, or LLMs, let users enter a prompt (some question or set of instructions) and then the LLM generates a response. People in tech are using them for everything from creating code examples to generating graphs and charts and beyond.

Sounds useful, right? Well, I'm sure you've heard talk about AI taking over a whole bunch of jobs. And as developers and content creators, we want to carry out our roles without worrying about some AI stepping in.

After all, robots are now being used in some parts of the world as waiters, and executive assistants are also being replaced by digital virtual assistants. And of course creatives such as technical writers are not left out.

So we need to learn how to use and take advantage of these new tools. In this way, they can help us, not push us out.

Even though technology is racing down the fast lane, there are many ways for you to stay relevant. In this article, I'll talk about two of the major players in the AI/LLM space – ChatGPT and Bard AI – and explain how you can use them to your benefit.

Here's what we'll cover:

• How AI works
• AI tools and their applications
• The major AI tools used in writing
• Google Bard vs ChatGPT

How AI Works – An Overview

Before we dive into the tools themselves, we should understand briefly how Artificial intelligence works at a high level.

Artificial Intelligence is the technology that allows computers (or machines generally) to "think" and behave independently of human input.

Although there are some fundamental concepts and frameworks behind it such as data science and machine learning, artificial intelligence is now being used throughout various fields, both technical and not traditionally technical.

The AI works by being fed data so that it can train ("learn") and produce some output based on its training. So the output depends on the data it learns on, to a large extent.

This is true for LLMs as well as other types of AI models – if you don't ask the LLM the right question or feed it the right prompt, you won't get the output you desire.

For example, if I want to generate a professional summary for a Web Developer Role and I accidentally request one for a Graphic Designer instead, it will generate the professional summary based on the role of a Graphics Designer. So you'll need to learn how to ask the LLM the right questions.

As long as you feed the AI the right data, it can do all sorts of things – from photo generation to content idea generation to helping you create useful code examples for your technical tutorials. So in fact, AI can help Creatives, Software Developers, and many others be more productive and save time so they can focus on the truly important problems.

Here's something to remember, though: as a writer, irrespective of your genre, you should not rely solely on AI to do the writing for you. While AI is helpful in generating outlines and ideas, it doesn't have your background knowledge, it might not understand the context or your audience, and sometimes its writing just won't be as good as yours.

So embrace your skills and your authenticity, and use AI as an enhancer to help you write better.

A you read this guide, note that my intention is to show you how you can use the AI tools listed here to do your work more effectively. You should not use these tools or technologies for any nefarious or dishonest purposes.

AI Tools and Their Applications

Almost everyone uses AI in some form these days, even without realizing it. Irrespective of your career field, there’s likely some AI software to help ease your workload.

Although there are different classes of AIs based on their learning methods, this article won't bore you with too many technical terms. So I'll just briefly go over the different types of AI software that can help make your tasks easier and faster.

Below is a table showing some popular AI applications and their use cases:

These applications are grouped based on the functions they perform, even though they're built by different companies.

AI Tools for Writing

With the advent of writing assistants such as OpenAI’s ChatGPT and Google’s Bard, some people have begun to wonder – will writers start losing their jobs? But as one of my connections said on LinkedIn recently, tech is an enabler, and so is AI.

Some writers have started using these AI tools to generate topic ideas and help them improve their skills, as we briefly discussed above. But other writers are using AI tools to entirely write content for them.

This is not a good idea for many reasons – not least of which being that the AI could be plain wrong. It also won't have your finesse, your awareness of context and other subtleties, and it may sound stiff and overly formal. Plus, it'll take away a lot of the benefit you get as a writer and developer from researching and writing your own articles.

Just always remember that moderation is key and human intelligence will always still stand out. So learn and use these tools, but don't overly rely on them to do your job for you...or they might end up doing just that in the long run.

Let’s take a look at two of the major generative AIs used by writers and developers: ChatGPT and Google’s Bard.

What is ChatGPT?

In November 2022, OpenAI alongside Microsoft launched a Large Language Model (LLM) that could generate responses based on the user’s input (questions), whether that input was text or code.

This model that is based on a text generative pre-trained model known as ChatGPT. Prior to this model of chatbot, there had been several chatbots such as the instant responder you find on websites and also different virtual help desks used by Fintechs.

From generating a résumé to generating software code in different programming languages, ChatGPT became the “darling” everyone sought to know.

How does ChatGPT work?

The ChatGPT takes in your instructions, also referred to as prompts. These instructions are then used to generate a result relative to your prompt.

The more prompts the LLM receives, the better it gets at learning. This implies that the model is trained based on the user's input.

Example of a prompt:

In 5 sentences, explain how AI works

Output from ChatGPT

You should also bear in mind that your prompt should be well constructed and clear, otherwise the generated result might what you want (or might not be that helpful).

Note that I used the free version of ChatGPT for this article. More features are available on the premium version.

You can use ChatGPT in many different ways depending on your needs. Here are some examples of ways you can use it.

• ChatGPT can help you debug your code. You can enter a prompt requesting a code snippet or example for a certain task, and the LLM will generate it for you. This is not to say that you can depend solely on ChatGPT to build or debug your projects. Just like enzymes that speed up reactions, ChatGPT helps you debug codes faster by pointing you in the direction you should go.
• Also, the Assistant API from ChatGPT can be consumed to build an instant messaging AI in your project. Although still in the beta stage, the Assistant API can help Software Developers build mini-ChatGPTs in their individual projects. That way, your users can ask questions and get responses based on the underlying LLM's abilities and knowledge.
• Writers often complain about idea generation and writers' block. One way ChatGPT can work for you as a Technical Writer is to help you generate content ideas. It just depends, as always, on the prompt you enter.
• It can also help you start writing about your ideas. Some writers have issues getting down that first sentence or making an outline for a draft. With ChatGPT, you can perform these tasks more quickly and use the LLMs ideas as jumping off points.

Again, this is not to encourage you to depend solely on AI for content generation, as the human touch still stands out. But it can certainly help you get started.

While ChatGPT can help you in many ways, the earlier models (like GPT-3) also had their downsides. But now, the latest model of GPT-4 has many improvements, including being able to use plugins to get more current information. But you may need to pay for premium usage in some cases.

There are other ways you can use ChatGPT, and you can read more here to find out.

Google's BARD AI

Google's BARD is the newest LLM kid on the block.

As expected, in the first quarter of 2023, Google released its own language based-model chatbot called Bard AI. Although still in the Beta stage (as at the time of this article), Google Bard AI is gradually catching the attention of the tech community.

How does Bard work?

Like other LLMs, Bard accepts prompts and generates results in response to the prompt. Bard is built on Google's LaMDA (Language Model for Dialogue Applications), which generates finely-tuned responses and improves specificity.

Screenshot showing Bard responding to a prompt

Bard can perform various operations such as:

• Review software development code errors and suggest solutions. Just remember that AI shouldn't be used as the ultimate error finder or solver for your code reviews. The LLM may not understand the entire architecture of your program based on the piece of code you feed it via the prompt.
• Bard prides itself on its specific and fine-tuned responses, which of course are quite helpful in Technical Writing. Bard tries as much possible to keep prompt responses concise and authentic. In cases where it sources from any external web-page, the URL is referenced.

ChatGPT vs BARD AI

Both of these tools can be helpful to you as a dev or technical writer – it just depends on how you use them and what your needs are.

If you'd like to see them compared directly, here's a live demo of both ChatGPT and Google Bard:

Note: The information provided here is based on what was available at the time, and may not reflect the most up-to-date details about ChatGPT and BARD AI.

Wrapping Up

Technologies can help enhance our capabilities as human beings. And Artificial Intelligence tools should be no different. You can use them to help you perform basic tasks or build creative solutions. This is why you should get acquainted with how they work and try them out in your projects.

Whether it is ChatGPT or Google’s Bard AI, the goal remains achieving efficiency and genuine results in the simplest way possible.

I am too, but I'm also intrigued by the technology behind it.

If you'd like to dive into some of the details behind how AI actually works and take a break from the many creepy AI examples on the internet, check this out:

Google has released a free set of training courses for generative AI. Here's their announcement post.

Picture for generative AI courses

Video Walkthrough

I've made a short video of the coursework if you'd prefer a visual walkthrough of my first impressions:

Overview

There are 10 courses in this free learning path. Many are aimed at beginners with no prerequisite knowledge. Some suggest a background in Python, SQL, and/or machine learning. All of them deliver content through a combination of videos, articles, labs and quizzes.

There are badges to show that you've completed them, and I'm happy to see a freely available set of courses around the technology behind AI.

Screenshot of Google course badge

The amount of content produced by and highlighting AI is overwhelming right now, but it's a safe bet that this is going to stick around. As such, it's best to arm yourselves with as much information about it as you can so that you can responsibly create tools and content for the future.

Courses Material

I've created a Google Sheet with all the course info pulled from each of these courses. Check it out here. Google has done a fine job of creating a brief FAQ for each of the courses, and I've pulled them all into one sheet for easy comparison.

Screenshot of Google Sheet summary of all courses

Inside each course, you'll be presented with a straightforward navigation for the content. It's broken into video instruction, a curated collection of readings to peruse, and quizzes. Some courses also have online labs to complete in a certain amount of time.

Video content for the Create Image Captioning Models course

The curated list of readings is quite comprehensive, and you could easily exceed the estimated 45 minutes of time to complete the modules if you read through every one. But it is nice to have such a large list of resources compiled in one place.

The list of readings for the Intro to Large Language Models course

There are also Labs hosted in some of the courses. If you've taken any of Google's Coursera courses, you'll be familiar with this delivery method. You're given a handful of tasks and a countdown timer, in this case 1hr, will begin when you start the lab.

Screenshot of lab work

Here is the full list of courses:

1. Introduction to Generative AI – designed to be an overview of what generative AI is and how it differs from machine learning methods.
2. Introduction to Large Language Models – explores what large language models are, where they are used, and how to use prompt tuning. (If you haven't noticed, prompt writing is being touted as a skill of the future right now.)
3. Introduction to Responsible AI – an ethical course on what responsible AI is, how it's implemented in Google products, and why it's important. This course introduces Google's 7 AI Principles. I didn't know this was a thing, but there's a whole page devoted to it. Covering topics from social responsibility, to safety, accountability and privacy design principles, I was happy to see that there is large effort being paid to build in solid principles.
4. Generative AI Fundamentals – A quiz covering topics from the first three courses.
5. Introduction to Image Generation – An introduction to diffusion models which are a family of models used in image generation. Some pre-existing knowledge of machine learning, deep learning, convolutional neural nets and/or Python programming is suggested.
6. Encoder-Decoder Architecture – overview of a machine learning architecture for tasks like machine translation, text summarization, and question answering. Python and Tensorflow knowledge is suggested as a prerequisite.
7. Attention Mechanism – a technique that allows neural networks to focus on specific parts of an input sequence. Some pre-existing knowledge of machine learning, deep learning, natural language processing, and/or Python programming is suggested.
8. Transformer Models and BERT Model – Bidirectional Encoder Representations from Transformers...this is what BERT stands for in case you didn't know. You'll learn the main components of the Transformer architecture and intermediate machine learning experience as well, and knowledge of Python and TensorFlow are recommended.
9. Create Image Captioning Models – how to create an image captioning model using deep learning. Deep learning, machine learning, natural language processing, computer vision and Python are recommended prerequisites.
10. Introduction to Generative AI Studio – you'll walk through demos of the Generative AI Studio which helps prototype and customize generative AI models. There is a hands-on lab at the end.

My Thoughts

After spending some time browsing through all of these courses, it's a nice mix of truly beginner-friendly content and some intermediate level stuff that requires previous knowledge of machine learning, Python, deep learning, and/or natural language processing.

I appreciate how there are answers to many common questions in the drop down menus for each course (and I've compiled all of those in one place in this Google Sheet).

Screenshot of Google Cloud certifications

This is a great starting point if you are truly interested in the inner-workings of AI. It also looks to be a potential on-ramp to some of Google Cloud's larger certifications as there are links to further training sprinkled in the courses.

What do you think?

👋My name is Eamonn Cottrell, and I'm building content around Google Sheets & Workspace.

Get my 📨 newsletter here. Find me on 📺 YouTube here.

Please ask questions in the comments, or send me a message; I love making new spreadsheet connections!

It follows five other successful entry-level certificate programs in IT Support, Data Analytics, Digital Marketing & E-commerce, Project Management and UX Design.

I made a first impressions video to accompany this article that you can find here:(more links in the summary at the bottom of the article, too)

What is "Grow with Google"?

The Grow with Google program is for people interested in "training, tools and resources to grow skills, careers, or businesses". You can find all the certificate programs and resources here.

Screenshot of Grow with Google website

Grow with Google's mission is to enable anyone to have access to quality technology training. They've been training digital skills since 2017, and currently partner with Coursera for course delivery.

Coursera's been in the MOOC (Massive Open Online Course) space since 2012, and has grown to be a leader in the online education world.

Screenshot of FAQ from Grow With Google

Though Google has released a few Advanced level courses recently (I wrote about this here) in the Data Analytics, Business Intelligence and IT Automation tracts, the new Cybersecurity certificate has been designed with beginners in mind.

There is no experience necessary to begin. The structure of the certificate program starts in a foundational introductory module, works through some technical topics like SQL, Linux, and Python, and then culminates with a module on preparing for Cybersecurity Jobs.

Get Job Ready

This is also part of Google's Grow With Me pathways: they are partnering with businesses and universities to pair together jobs and program graduates.

The initial job platform was deprecated as of February 7th, 2023, but the new one for this year is maintained on CareerCircle.

Screenshot of CareerCircle

This is a strong draw to participants, especially as the entry-level salaries for many of the fields approach and exceed six figures. Cybersecurity has a $100K+ amount listed in the marketing material.

Screenshot of Cybersecurity marketing material for median entry salaries

Naturally, these amounts are easily skewed and may not be realistic depending on the applicant's background. But it's no secret that on the whole, these technical jobs do lead to higher paying salaries.

What's the Coursework Like?

I took a look through the 8 courses that make up the Cybersecurity certificate, and found it to be a similar makeup to the first Data Analytics certificate that I completed a couple years ago.

It begins with some basic overviewing of the field, and then moves into an assortment of high level concepts and technical skill building. Here is the full list:

• Foundations of Cybersecurity
• Play it Safe: Manage Security Risks
• Connect and Protect: Networks and Network Security
• Tools of the Trade: Linux and SQL
• Assets, Threats, and Vulnerabilities
• Sound the Alarm: Detection and Response
• Automate Cybersecurity Tasks with Python
• Put it to Work: Prepare for Cybersecurity Jobs

As you can see from some of the titles, Google does get a little general in their topics, which is to be expected in an introductory course.

Screenshot of Cybersecurity Professional Certificate coursework

They have also been long-winded in the video courses in my experience, and I recommend increasing the video playback speed to 1.5x or 2.0x normal speed. I found I had little difficulty in keeping up with the presenters when I did this in other courses.

More Course Details and $$$

Coursera would like you to pay $49 a month for this certificate program, and that's perfectly reasonable. Especially since Google estimates you can complete it in 6 months by working only 7 hours a week

...and I estimate you can complete it in a max of 3 months working 7-10 hours a week. 😀

But, if you're strapped for cash, you can click into any one of the individual courses and audit the whole thing for FREE.

Screenshot of Coursera course audit

You won't get the certificate, or access to the CareerCircle network, but if you're just leveling up your knowledge, you can do that for free.

You can also see each module's details within each of the courses. This detailed view breaks down each lesson to exactly what topics will be covered and how much time will be spent on each video, reading, or quiz.

This is a great place to look in case you're not starting from zero and wonder if it will be repeating information you already know. I found that many of the items were very beginner-based (as they stated), especially in the Python course.

Summary, Links, and Video

This looks to be another solid offering from Google into a field that promises to be intriguing to many people.

If you're interested in Cybersecurity and have been wondering where to start, I believe this is as good an entry point as any.

• Check out my first impressions video here.
• Check out the Coursera page here.
• Check out Google's Cybersecurity program overview page here.

Have a great one!

In this article, I'll introduce you to the Google Certifications and detail what's new. I'll focus on the Data Analytics pathway, but will include resources to their other programs.

Video Walkthrough

Here's a video walkthrough of what we'll discuss below.

Grow With Google Overview

Career Certifications from Google are a part of their "Grow with Google" initiative. A portion of their mission page is in the picture below 👇.

Grow with Google is an assortment of training and learning resources for small business owners, job seekers, veterans, educators, developers and startups to learn vital digital skills.

From Grow with Google's mission: https://grow.google/our-mission/

Part of Grow with Google are the Career Certificates pathways which offer flexible online training programs in Digital Marketing & E-commerce, IT Support, Data Analytics, Project Management, and UX Design.👇

picture of Grow with Google's Career Certificates page

The proposition is attractive: no experience necessary to begin the programs, learn at your own pace, have a well-regarded certification from Google at the end of it and be qualified for a new entry-level job in careers with a median salary of $72,000 across the certificate fields.

The median salaries range from $57,000 for IT Support to $112,000 for UX Design.

The entire certificate program is a partnership between Google, who has crafted the courses, and Coursera, which is one of the OG MOOCS that hosts the coursework.

This of course begs the question: how much does it cost? 🤔

gif of man saying, how much money?

The good news is that you can nab the knowledge for free. And if you fork over a little bit, you can have the official certificate to display proudly on that portion of your LinkedIn page rarely ever scrolled to. 😀

pic of my LinkedIn licenses & certifications

The Free Version:

Yes, the knowledge is free. You can audit any of these courses at Coursera for free. So, while there's not an option to audit the whole certificate, you can click into the individual courses and audit them one by one.

picture of audit option in coursera courser

I've clicked into the Foundations of Data Science course and selected audit. In the screenshot below, you can see that I now have access to all the coursework.

screenshot of Foundations of Data Science coursework on coursera

What don't you get with the free version?

Well, you don't get that nice certificate "proving" you did the work. But if you've got the knowledge, you can muster together projects that will do a better job of proving that anyway.

And you also may not be able to turn in certain assignments for grades, depending on the course.

The Not Free Version:

Coursera costs $39 a month for any of the individual Google certificates, including the new advanced ones. And since it's at your own pace, if you work fast, you can finish in one, two, or three months instead of the projected six...saving you some $$$. 💵💰

pic of Coursera's individual course pricing

As is now common, Coursera also offers a more expensive Plus program for $59 a month or $399 a year. This gives unlimited access to courses and could be a better deal if you plan on plowing through many courses during a year.

But, you'd have to be flying through them for this to be valuable considering you could by 10 individual courses for $390.

pic of coursera plus costs

My opinion: if you're strapped for cash, start knocking out the free audit versions. But, it's pretty cheap to grab a certificate if that's important to you. Particularly if you work hard and fast 👉 it could cost as little as one month's $39 fee.

My Experience with the Data Analytics Cert

I took the Data Analytics Certification a while back, and can attest that it was a fantastic overview of the field. I'd been using Excel heavily before taking the course, and was familiar with programming and database concepts, so I was able to move at a brisk pace.

I spent 2-3 months working through the "6 month" course, and certainly could have gone faster. Six months is conservative, even for a novice, in my opinion.

screenshot of my data analytics certificate

This certificate took some flak because it used R instead of Python for the programming portion. But I thought it was well laid out and offered a good introduction to data analytics.

Would it make someone hirable?

Well, it's just one piece in that puzzle. Relying on any of these certifications alone to land a job is perhaps unwise. It will take a combination of the skills learned, the networking you're willing to do, and your continued work through projects to prove your skills and land a job.

On the whole, I recommend it, especially if you're new to Excel, SQL, or programming.

And if you're wondering, yes, I ponied up the cash and enrolled in the program. It took me a little over two months and so I paid $117 +tax.

My payment history for Data Analytics coursework

Advanced Certificates

My foray into the data analytics was one of curiosity more than anything. I was not actively seeking new employment – I simply wanted to sharpen and expand my skills. And that's exactly what piqued my interest in the new, advanced certificates that are now available.

pic of banner advertising new advanced Certificates

In addition to Industry Specializations (added in October 2022), there are now three advanced certificates (as of April 2023) with more likely in the future: Advanced Data Analytics, Business Intelligence, and IT Automation with Python.

pic of advanced certificate listings from Grow with Google

According to Google's announcement post, the Data Analytics Certificate program was the most popular professional certificate on Coursera, so it's unsurprising that this field has received the first advanced follow-up courses.

Excerpt from announcement post

And the field is well paid, making it an attractive aspiration for career transitioners and students alike.

_pic of advanced data analytics certificate details from https://grow.google/certificates/data-analytics/?advanced#?modalactive=none

Of particular note, Google has wisely opted to go with Python and Jupyter Notebook (industry standard tools) as detailed in the above card for the Advanced Data Analytics Certificate.

Below, you can see similar details for the Business Intelligence Certificate which focuses more heavily on modeling, dashboards, and engaging with stakeholders via SQL, Tableau, and BigQuery.

_pic of details from business intelligence certificate from https://grow.google/certificates/data-analytics/?advanced#?modalactive=none

I'm excited to try out both courses and see how deep they go after the introductory, pretty surface level course. And right now, it has that same fresh, new, adventurous feel as when a new WoW server launches 🧙‍♂️. Only 3 enrolled in the Data Analytics Advanced degree!😲😅

pic of advanced data analytics certificate page on Coursera

And there are only 4 in the Business Intelligence program! This is sure to change quickly, and indeed, there are a few hundred in the individual courses already. Looks like many will indeed opt to audit the courses before jumping into the paid program.

Coursework Overview

Here are the respective courses in the two new advanced certificates. I encourage you to go check them out yourselves for further details, or watch my video walkthrough, but this will give you an idea of what to expect.

Data Analytics Courses

Go here to check out the overview of each data analytics course.

Click into each course below to explore in more detail and see all the modules within each course. This program is estimated by Google to take six months to complete.

1. Foundations of Data Science
2. Get Started with Python
3. Go Beyond the Numbers: Translate Data into Insights
4. The Power of Statistics
5. Regression Analysis: Simplify Complex Data Relationships
6. The Nuts and Bolts of Machine Learning
7. Google Advanced Data Analytics Capstone

You can see from the screenshot below the estimated hours to complete each module as well as the higher $118,000 median salary for entry-level roles:

screenshot of coursework for Advanced Data Analytics

Business Intelligence Courses

Go here to check out the overview of each business intelligence course.

Click into each course below to explore in more detail and see all the modules within each course. This shorter program is estimated by Google to take two months to complete.

1. Foundations of Business Intelligence
2. The Path to Insights: Data Models and Pipelines
3. Decisions, Decisions: Dashboards and Reports

screenshot of Business Intelligence certificate coursework

After the Certificate

Google has also developed an employer consortium of over 150 companies which will consider recent graduates of Google's programs for hiring. While this is certainly no guarantee of employment, if you're in the market it is a nice touch, and will be another helpful tool in your search.

Summary

I hope this has been a helpful overview for you!

I make weekly spreadsheet and coding videos on my YouTube channel. Come check it out if you find this helpful! Here's a link to my Google Sheets playlist where I walk through basic and advanced topics alike:

You can also find me on LinkedIn. Have a great one!👋

With the crawling capabilities of Google, it can also be a powerful tool for pen testers. Google can help us find exposed files, scripts and other critical resources in web applications.

To find this type of sensitive information, hackers use specific search terms in Google. We call them Google Dorks.

Google Dorks are special search terms that help locate information which is not found through regular web searches.

In this article, we will look at what Google Dorks are and how they can help us in penetration testing.

What are Google Dorks?

A Google Dork is a special search term. These terms, when used with regular search keywords, can help us discover hidden resources crawled by Google.

These resources include sensitive information such as usernames, passwords, credit card numbers, email addresses, shell scripts, user accounts, and so on.

These Dorks are not limited to Google. We can also use them with search engines like Bing and Yahoo. The results might vary, but they still serve the same purpose.

To harness the full potential of Google Dorking, we’ll need to master some specialized search operators. These operators will fine-tune our search results and help us find exactly what we are looking for.

Let’s try a few Google dorks.

Common Google Dorks

Some of the common query operators in Google Dorking include search modifiers. These search modifiers allow us to find specific information that may not be accessible through traditional search methods.

Here are some of the most common operators used in Google Dorking.

Intitle operator

The “intitle” operator searches for web pages with specific words or phrases in the title tag. For instance, if you’re looking for pages that contain the phrase “password” and have “index of” in the title, you would use the search term:intitle:”index of” password.

In title. Image by the author.

Inurl operator

The “inurl” operator searches for web pages that contain specific words or phrases in the URL. For example, if you’re looking for pages that contain “admin.php” in the URL, you would use the search term:inurl:admin.php.

In url. Image by the author.

Site operator

The “site” operator allows you to search within a specific website or domain. For instance, if you’re looking for pages on the example.com domain that contain the word “Steganography”, you would use the search term:site:yeahhub.com “Steganography”

In site. Image by the author.

Filetype operator

The “filetype” operator allows you to search for specific file types, such as PDFs or Word documents. For example, if you’re looking for PDF files that contain the phrase “confidential report”, you would use the search term:filetype:pdf "Advanced Network Security"

Filetype. Image by the author.

Intext operator

The “intext” operator searches for pages that contain specific words or phrases within the body of the page. For instance, if you’re looking for pages that contain both the words “login” and “password” within the body of the page, you would use the search term:intext:"about" contact.

In text. Image by the author.

Link operator

The “link” operator searches for web pages that link to a specific URL. For example, if you’re looking for web pages that link to the example.com domain, you would use the search term:link:”example.com”

Link operator. Image by the author.

Cache operator

The “cache” operator is used to retrieve the cached version of a web page. When you search for a website using Google, Google creates a cached version of that page in its system. This version can be useful if the original website is temporarily down or if you want to view an older version of the website.

Here is the syntax to find the cached version of yahoo.com.cache:https://www.yahoo.com

Cached version of yahoo.com. Image by author.

Related operator

The “related” operator is used to find web pages that are related to a specific URL. Here is the syntax to use the “related” operator to find sites similar to yahoo.com.

Related operator. Image by author.

By combining these operators in creative ways, you can find specific types of information on the web that can be useful for penetration testing and other purposes.

Structure of Query Operators

Google Dorking query operators have a structure similar to regular Google search query operators. This technique involves using advanced operators and search queries to uncover information that is not typically available through regular searches.

The general structure of query operators in Google Dorking includes three elements:

1. Operator: A specific keyword or symbol that instructs Google what to search for. For instance, the “inurl” operator searches for pages that contain a particular keyword in their URL.
2. Keyword: The search term or phrase that you want to find. If you are looking for a specific password file, then “password” is your keyword.
3. Modifier: An additional search parameter that you can use to further refine your search. For example, the “filetype” modifier searches for a specific file type, such as a PDF.

Here’s an example of a query operator structure in Google Dorking: intitle: “index of” site:example.com password filetype:pdf

This query uses the “intitle” operator to search for pages with “index of” in their title, the “site” operator to search within the example.com domain, the keyword “password,” and the “filetype” modifier to search for PDF files.

By utilizing query operators in Google Dorking, we can find useful and often vulnerable information that might not be accessible through regular searches.

Google Hacking Database (GHDB)

The Google Hacking Database (GHDB) is a compilation of search queries and query operators that help us in Google Dorking.

Google hacking database. Image generated by author.

Johnny Long, a well-known security researcher and author, established the GHDB. It has since become a valuable resource for security engineers like you and me.

The GHDB has several search queries and operators that can uncover numerous sensitive files, vulnerable web servers, and applications. It can also discover default login pages and credentials, as well as network and security devices that may be prone to attack.

GHDB is arranged into categories such as “Files containing passwords” “Vulnerable servers” “Footholds” and “Error Messages”. Each category contains several search queries and operators crafted to reveal specific information about a target.

Please note that search queries and operators in the GHDB might produce false positives or outdated information. Always verify the information obtained through these search operators.

A Dorking Scenario

Let’s assume you have to conduct a pentesting audit for a client. Here is a sample dorking scenario.

1. Use the “site” operator to limit your search to the company’s website: site:example.com. This returns all pages on the example.com website.
2. Use the “intitle” operator to search for pages containing specific keywords in the title: intitle:”login” site:example.com. This helps identify potential login pages vulnerable to attack.
3. Use the “filetype” operator to search for specific file types: filetype:pdf site:example.com. This helps identify potential documents or reports containing sensitive information.
4. Use the “inurl” operator to search for specific URLs: inurl:”admin” site:example.com. This helps identify potential administrative pages vulnerable to attack.
5. Use the “cache” operator to view the cached version of a webpage Google has indexed: cache:example.com/login.php. This provides access to the page contents even if the original page is removed or no longer accessible.
6. Use the “related” operator to find similar websites: related:example.com. This helps identify potential partners or third-party vendors with access to the company’s network.

Summary

Google Dorking is a powerful technique that allows us to perform advanced searches on Google. We can use Google Dorks to find specific information and publicly exposed vulnerabilities. It is an essential tool in a pentester’s toolkit.

Google Hacking Database (GHDB) provides a collection of pre-defined Google Dorks. Given the harm that someone can cause using dorking, it is important to use it ethically and with permission. Ensure that you have permission and follow ethical guidelines when using dorking for security audits.

We just published a video course on the freeCodeCamp.org YouTube channel that will teach you how to use Google like a pro.

Seth Goldin developed this course. Seth has been using Google to help him develop software for many years and he's currently a CS student a Yale University.

In this course, you'll learn how the search engine works and how to ask the right questions to get the best results. We'll also delve into the world of search operators, which are special commands that can help you narrow down and customize your search results.

Throughout the course, we'll cover a variety of search operators, including matching operators, date operators, source operators, boolean operators, and in-(URL/title/text/anchor) operators. We'll also explore utility operators and show you how to use and combine these operators to power up your search.

In addition to search operators, we'll also discuss other useful features of Google, such as Google Lens and image search. We'll also introduce you to some specialized search engines that can be helpful for specific types of information.

To wrap things up, we'll go over some handy keyboard shortcuts and give you some final tips and tricks for becoming a pro at Googling.

By the end of this course, you'll be a proficient searcher, able to find the information you need quickly and easily. So let's get started on our journey to becoming a Google pro!

Watch the full course on the freeCodeCamp.org YouTube channel (1-hour watch).

And while you might think it's obvious, many internet users don't know how to use Google search efficiently and effectively.

In truth, Googling is an art.

To get the right answers, you need to ask the right questions. And to get the right answers quickly, you must know how to ask the right questions.

Everyone should learn the how part, and that's what this tutorial is all about. Below are some helpful tips and tricks you can use to find correct answers to your questions effectively and efficiently.

How to Search Google Effectively

1. Use quotes to get an "EXACT" match

Generally, when we do a regular search on Google, it makes a shallow search. This may result in pages that may or may not contain all the words you mentioned in your search query.

But if you wrap your search query or question (either a term or a phrase) in quotes like this: "your question goes here" then Google will do a deep search. This means that all the results will include pages that contain all the terms in your question that you put within quotes.

This is helpful when you absolutely want a particular term or phrase to be there in your search results.

2. Search within a specific site with site:

If you want Google to return results from within a particular website, just add site: before the question you are searching.

This is also helpful when you want to search for something within a website that doesn't have an internal search option or perhaps doesn't have a very effective one.

For example, searching for site:freecodecamp.org react will populate search results only from the www.freecodecamp.org website on the topic of React.

3. Exclude a term from search results with -

If you don't want a term or phrase to appear in your search results, then just add - in front of that word.

For e.g. a search on => how to write components in React -class will return all the search results that don't have the term "class" in them. So, (if you know a bit of React) it'll give you only ways to write functional components in React.

4. Search images of a particular size with imagesize:

If you want to search images of a particular size, then use the tag imagesize: in your search query along with the width and height in pixels.

Dimension should be in pixels only – for example, imagesize:500x500 will populate image results which have a dimension of 500px x 500px. So a search on => "cute dog images imagesize:500x600" (coz why not) will result in pictures of cute dogs with an image size of 500px x 600px.

5. Search for a particular filetype with filetype:

If you want to get search results that contain a particular file type such as PDF or PPT, then add filetype:<extension> (without the angular brackets). For example, react tutorial filetype:pdf will generate results that contains PDF in them, as seen in the following image:

6. Use wildcard * to make searches

If you are unsure about or have forgotten any term in your search query/ question, then use the wildcard * charachter. Google will replace it for you with relevant terms.

For example, a search on => the of money will populate the following results. It will be a bunch of pages that include the phrase such as "the exchange of money", "the use of money", "the role of money", "the psychology* of money", and so on.

7. Combine searches with OR, AND logic

If you want your search results to contain two terms, then put the AND keyword in between them. For example, the search on => React AND Angular will fetch results that have both the terms react and angular in it.

Similarly, if you want either of the terms in your search results, then use the OR keyword in between them. For example, the search on => React OR Angular will fetch results that have either of the terms or even both.

8. Filter out searches with AFTER:, BEFORE: or .. between two numbers

If you want Google to populate search results that were published after a particular year, then use the tag AFTER:. For example, the search on => React tutorials AFTER:2020 will populate search results published after 2020.

Similarly adding the tag BEFORE: will return results published before a particular year.

You can also search for results published in a certain year range, or for that matter between any numbers. Just add .. between the two numbers you want to search between along with the units, if any.

9. Check out related websites using related:

If you want to know what all other websites are available on the internet that are similar to a particular website, then use the related: tag.

For example, a search on related:google.com will fetch all the websites that are similar to Google like Bing, Yahoo, DuckDuckGo, and so on.

10. Use cache: to see Google's cached version of a website

Google stores a cached version of websites to provide search results quickly. To see if Google has cached any site or not, just use the tag cache: in front of the website's URL.

This is especially helpful for web developers if they want to check if they are currently viewing a cached site or the latest version of a website after they have pushed some changes to the website. For example, a search for => cache:www.sohamderoy.dev on the day of writing this tutorial (that is, 4th Aug 2022) returns the following result:

As you see, even though I am searching for the site on 4th August 2022, I still see a copy of the site from 3rd August 2022, as that is when Google has cached the website.

It is important to note that "Not all search operators return exhaustive results." as mentioned by Google itself.

Still, I believe that these are some really handy tips that help you make an effective and efficient Google search. I hope I've explained them properly and have encouraged you to use them in your regular Google search.

Wrapping Up

Thanks for reading! I really hope you enjoyed reading this short article about how to google efficeiently and effectively and found this tutorial useful.

Do consider sharing it with your friends – I'd really appreciate that. Follow me on LinkedIn and Twitter and stay tuned for more amazing content! Peace out! 🖖

Social Links

• LinkedIn
• Website
• Other Blogs by me
• Twitter

If you have heard of Google Summer of Code but aren't sure how to apply, then read on.

Whether you are a college student or a working professional, GSoC now is open to both.

What is GSoC and How Does it Work?

GSoC is a program that encourages contribution to open source projects. It starts with the application process. Students apply once the organizations are announced. Then GSoC declares which projects are accepted into the program and the coding period begins.

If you are relying only on your college grades or work experience to get into the program, think again. It is not an interview that will help you get your foot in the door based on your qualifications or résumé. You have to submit a project idea. And it needs to be an idea that the organization is excited about and that the program wants to fund.

Don't forget that you will work on this project for 12 weeks or even more (if your project needs more time and the organization agrees). So make sure you propose something you want to build, fix, or revamp. It should be a project that fuels you to deliver your best.

What happens when the GSoC results are out?

If you are in the Indian timezone it is around midnight when the GSoC project announcement emails start pouring in. Depending on the result, you might be having a pizza party or blinking away tears.

If the results went your way, congratulations for making it into the program. For the next 3 to 4 weeks you have a community bonding period. Here you get to know your mentors and team.

You might kickstart or brainstorm your project. If you are someone like me who changed your personal commitment if you were to get selected, you'll need a change of plans. Devise a new plan for the coming months and make new goals. Keep exploring different projects, and by the time the GSoC application period arrives again, you will know which organization you want to apply to.

Once the contributing period ends, the coding period starts. This period is divided into two phases. You have to propose two major deliverables for each phase.

You need a thorough plan for phase 1. Make sure you meet your weekly goals. In the next 4 to 5 weeks it will be time for your first evaluation. Before the evaluation, you should have delivered your first deliverable. Your mentor will submit your evaluation to the GSoC team. Once the evaluation is submitted 45% of your stipend will be released. This marks the end of phase 1.

In phase 2 you will start working on your second deliverable. This final phase is of 6 to 8 weeks. Along with completing your project, you also have to document, test, demo, and hand it over to the organization. Your mentor will evaluate your performance and submit it to the GSoC team. This marks the end of your contribution period.

Your organization then decides whether your project was a success, and depending on this your pending payment is released.

What's new in GSoC 2022?

For the past 17 years, GSoC was solely focused on university students. The aim was to give them some industry experience in the summer and prepare them for real-world software development.

In 2022 this has changed. GSoC has opened its doors to contributors from all walks of life. You may be a student, working professional, or retired veteran – if you can code you are welcome to the program.

Also starting this year two types of projects have been announced: medium-sized and large-sized projects. A medium-sized project would be ~175 hours and a large-sized project would be ~350 hours. Previously you had to do a large size project for 12 weeks.

In one of the blogs, Stephanie shares what is new in GSoC 2022, and the thought that goes behind it.

We realize there are many folks that could benefit from the GSoC program that are at various stages of their career, recent career changers, self-taught, those returning to the workforce, etc. so we wanted to allow these folks the opportunity to participate in GSoC. – Stephanie Taylor, Program Manager, Google Open Source

You can extend your project for a few more weeks depending on your organization's discretion and the complexity of your project

These new changes cater to the different needs of the organization and contributors.

Why should I apply to GSoC?

• GSoC allows you to contribute to projects that have made a significant mark in the world.
• You get to network with some of the best engineers around the world.
• You are assigned a mentor. With a mentor as your guide, you can speed up things that otherwise might have taken you a long time to figure out by yourself.
• You are paid a stipend (to help you sustain and concentrate on the project) for the time you dedicate to the project.
• You will learn many best practices of coding, and write production quality code.

Nothing matches learning from the experts of the industry.

Am I Eligible for GSoC?

A contributor is eligible for GSoC if:

1. They are 18 years or older.
2. They are a beginner in open source.
3. They are not restricted legally or geographically from contributing.

GSoC has no other criteria. As this is a summer coding program that does not require office visits, you need to have a setup where you can code along with a good internet connection (at least video call quality).

How to Apply to GSoC

You can think of a GSoC proposal as a university SOP. Submitting the SOP takes a few seconds. You start preparing for your SOP a year or so in advance by doing internships, participating in various programs, and working on personal projects. This will help your SOP be strong and stand out.

You'll need to put in quite a bit of effort and time to the GSoC proposal as well. As GSoC is for open source beginners, you could have been be contributing to open source for a year or less and are eligible to apply for the program.

I started contributing to open source in Feb 2022. I came across the Github repository that has a list of beginner-friendly projects along with the label names you should look at. That's how I found my first organisation to contribute to open source: freeCodeCamp.

After making minor document changes, I wrote test cases for freeCodeCamp's CodeRadio Client, and this was my first actual code contribution.

The GSoC organizations were announced in March. I was looking for Python and JavaScript projects. I used the search bar on the organization's listing page to list the organizations in my preferred technology. I read their GSoC page (every org has its own GSoC page, so make sure you read it thoroughly) and went through their Github repositories. I cloned their repositories, used their product, read the issues list, and so on.

That's how came across Open Food Facts, a food database that holds details about each food product. The information comes from the manufacturer's nutrition details. The project is turning 10 this year and has 2.3M+ food products, 2.7M+ monthly visitors, and has reached 182 countries (Source: Open Food Facts blog).

How did I select my first GSoC organization?

Selecting the right organization is as important as writing a good proposal. If your goals do not align with the organization there is a high chance your proposal won't be accepted.

Here are the reasons why I selected Open Food Facts as my first GSoC organization:

• I was able to set up and run Open Food Facts Hunger games repository easily. Previously I had tried to set up Chrome, Brave, and Django but wasn't successful. I was aware of how difficult things can get at the setup stage itself.
• It was easy to understand what they did. I quickly found a beginner-friendly issue and commented on it for further guidance. (A beginner-friendly issue is already picked by the organization and reserved for beginners. It is usually under the label: first-timers only, beginner-friendly)
• Mentors here were super supportive. I am grateful for their guidance and trust. It was because of their timely guidance and mentorship that I completed my proposal in time.
• I understood their technology. This does not mean I knew everything. The main part of the project that reads nutrition data from the food label is in Python AI/ML. The main website is in Perl. I am not a Perl or ML student. I mostly go for Python and JavaScript projects. They have labels that say no ML knowledge required and I pick those to work on.
• Coming to non-technical reasons, I had started late in March and the deadline to submit the proposal was the third week of April. I did not have enough time to explore many organizations. I focussed on quality instead of quantity. You can submit at most 3 proposals to GSoC. I worked hard on only one proposal and gave all my time and energy to it.
• Open Food Facts has scope for both ~175 hour and ~350 hour projects.

How I wrote my first proposal

The proposal is the heart of your GSoC application. The strength of your proposal is directly proportional to the chances that you'll get selected.

A proposal is divided into the following parts:

Topic background:

You should be able to prove that you know what you are talking about. You should demonstrate the motivation behind your project, and how it will help the community.

Why is it worthy of your mentor's time? What is the long-term impact? You should be clear about what you are doing and why.

The deliverables:

Describe your deliverables in detail. Depending on your project you might have one or two major deliverables, and minor deliverables each week.

Each deliverable should be broken down into subtasks that you will complete each week. A delivery should include: detail description of your solution, any unforseen challenges, and how you plan to cope with them.

Timeline:

Deliverables with a timeline is the best way to convince your organization that your goals are well planned. Everything might not go to plan, so it is a good idea to leave some room for uncertainties.

Be clear about the size of your project, the number of hours you will be available each week, and what is the best time for the mentor to contact you.

Proof of your capabilities:

The organization does not know you, so it's up to you to prove that you are capable of achieving everything you claim.

Share links to all the open-source code you have written. Write about your background, what you like to do in your free time, other non-coding activities you do, and how they will help you succeed in GSoC.

Background on yourself:

Mention your commitment to other things (university, jobs) that will keep you away from GSoC, or any personal plans to take some time off.

Share you contribution to other open source projects, as this will help the organization trust you that you know to work with remote teams, and will stay committed to the project.

GNU Radio has an established road map to becoming a successful GSoC applicant.

How I Applied to GSoC with a Full-time Job

Whether you are a university student or a working professional, time is limited for everyone. When I started open source contribution I was working from home. This saved traveling time and I spent my weekends contributing to GSoC.

If you are someone with a full-time commitment I would suggest the following:

• Identify how much time you can spare each week.
• Plan your project around that time.
• Let your mentors know about your commitment, available hours, and the best time to contact you.

I had proposed a ~175-hour project. I also mentioned in my proposal that I could contribute 16-18 hours each week and my maximum availability would be on weekends.

What if I'm Not Selected?

Being turned down does not mean you cannot apply next year. Asking for feedback from the mentors is the first step in identifying your weak areas. The second step is working towards them.

GSoC is a part of the open-source journey. You can always contribute to your favorite organizations and keep exploring.

To those that didn’t make it into the 2022 edition, my message is “don’t give up”. Rather, reflect on what you can do to have a stronger proposal for the next GSoC edition. – Source: Jenkins message to GSoC 2022 Applicants

Where Can I Get Some Help and Guidance?

There are people there who will help if you ask. No one can write your proposal for you, but they can guide you while you're writing it.

How do I get help as a beginner in the organisation?

When you are working on an issue and get stuck, you can post your questions and doubts in the organization's chatroom (Slack channels, Discord channels, IRC channels, Google groups). Someone will answer your questions.

Posting in a group is always better than messaging someone directly. In a DM you're relying on one person, while in a group many people will read your question, and you might get a faster response.

Whom do I ask help for writing/reviewing my proposal?

Once you have completed your proposal it is always a good idea to get it reviewed before the final submission.

Proofread it yourself first. You can then ask your friends or family to proofread it again. A fresh pair of eyes can catch a lot of things you might have missed.

Make friends with people in GSoC and the open source community. You can ask fellow GSoC participants to review your proposal. You may find open source enthusiasts who are open to guiding you on LinkedIn, Twitter and other social media platforms. Search for them with hashtag #gsoc.

Tips to Help You Succeed in GSoC

Here's what I would advise if you're applying to GSoC:

1. Be nice: You are not interacting with robots, but rather humans at the other end who may be separated by timezones. They might have other full-time commitments apart from the project. Be kind when asking questions and waiting for a reply. No one wants to work with someone who doesn't treat them fairly.
   
   
2. Be patient: A famous quote says: good things come to those who wait. Learning new things is never easy. It requires you to get out of your comfort zone, push yourself, and manage your time.
   
   Understanding a new project demands both - time and patience. Don't try to understand the whole project at once. Start with the part that interests you and go up.
   
   
3. It is not a rat race: GSoC is not a competition or a race. It is an experience, an opportunity to contribute to products you have been knowingly or unknowingly using in your day-to-day life. You can continue doing the good work without GSoC as well.
   
   As a GSoC contributor, you will be accountable to someone. There will always be a mentor to guide you within the organization when you get stuck. Enjoy the process while you are at it.
   
   
4. Start early: A strong proposal is well-thought out and well-written. This requires the exchange of ideas. Some organizations are open to proposals that are outside their ideas list. Discuss it beforehand, submit PRs, and assure the organization that you can pull off your idea in the proposed time period.
   
   Start as early as possible. If your proposal is ready before time, say before the application window opens, you can ask the organization to review your proposal. Get their feedback and incorporate it. You cannot make changes to your proposal once the application window closes.
   
   
5. Discuss and ask questions: Mentors understand that you cannot figure out everything by yourself. Before asking a question, make sure you've read the documentation well, and show that it is well-researched. Communication plays a very important role here.

Wrapping up

It's time to say goodbye now. Before leaving you, I want to share that I was not selected for GSoC this year. I did ask for feedback from my mentor and I'm working on improving myself.

I will continue contributing to open source, and I'll explore new organizations and apply again next year. So should you!

I also encourage you to read Dheeraj Singh Jodha's story and learn from his GSoC 2022 journey.

I was looking for a job right after graduating from my college and landed an internship which was exciting and highly challenging.

Plus, one important thing is that I didn't know that I'm even eligible to participate in GSoC 2022 till Oct 2021 when Google announced that they were expanding their program to everyone. Otherwise, after graduating in June last year, I believed that I would never be participating in GSoC. I'm glad Google took this big step to make this program more inclusive.

As they say "Life doesn't make sense as we go on, but makes perfect sense when we look back." – Dheeraj Singh Jodha, GSoC 2022 contributer @Jenkins project

GSoC has extended the eligibility criteria and allows multiple projects of varying time spans. This is a great opportunity to find a project that suits you best. What's keeping you from getting in the program?

Google is the most powerful tool in the world if you know how to use it properly. Let me show you how to get better at Googling.

1. Use Quotes for Exact-Match Search

You can use quotes to force an exact-match search. This is super useful if you know exactly the search phrase you're looking for. You'll simply get more accurate results if you do this.

"what is javascript"

2. Use the AND Operator

The AND operator will return only results related to both terms. In the example bellow you can see the perfect use-case of this operator.

html AND css

3. Use the OR Operator

You can use the OR operator to get results related to one of the search terms you enter.

(javascript OR python) free course

4. Use the - (minus) Operator

The - operator will exclude results that contain a term or phrase. So, in our case we want to get JavaScript related results, but exclude any CSS results from them.

javascript -css

5. Use (*) wildcards as placeholders

You can use the (*) wildcards as placeholders, which will be replaced by any word or phrase.

This one is my favorite. I use it when I know almost the whole search phrase and just replace the part I don't know with asterisk. Super useful.

"how to start * in 6 months"

6. How to Search Inside a Single Website

This is another one I use frequently. If I'm searching for something in particular on a specific website, this is my approach.

site:freecodecamp.org

7. How to Find a Specific File Type

You can also use this very useful feature to help you find a specific file type. It's handy if you're looking for some materials like PDFs. If you haven't used it before, from now on you will.

filetype:pdf learn css

8. How to Search for a Range of Numbers

This can be anything at all. If you're looking to buy something that falls within a specific price range, or you're searching for results that include a specific range of years, this is for you.

ecmascript 2016..2018

9. Use the before Operator

Use the before operator to return only results before a given date.
You must provide year-month-day dates or only a year. For example:

javascript before:2020

10. Use the after Operator

Use the after operator to return only results after a given date. You must provide year-month-day dates or only a year. Super useful if you want exclude outdated results.

web development after:2020

As you can see, Google can be a powerful tool if you know how to use it properly.

If you have any questions, you can contact me on Twitter.

You can find a ton of real-life tips and resources on my blog.

This will not only help you understand basic ML/DS concepts, but it will also help you make a long-lasting, robust career as a data professional.

This is a short and precise guide for all self-taught devs and beginners in the field of Data Science and Machine Learning.

There's a common question that pops up in all my training programs, LinkedIn courses, videos on YouTube, or newsletters. It's that when people start learning DS/ML, after a certain point they feel lost in mathematics or statistics and sometimes programming.

And I have always recommended learning or refreshing some mathematical concepts that underpin ML as it helps you build intuition which keeps you curious throughout your learning journey.

To back up this claim, here are the prerequisites and prework Google recommends before taking their Machine Learning Crash Course:

Google ML course prerequisites

I’d recommend that you go through this article first and then look up all the links one by one and use this blog as a reference.

After going through the complete list of concepts and skills that are mentioned in the Google article, I also went through several books (Deep Learning by Ian Goodfellow, Deep Learning with Python by Francois Chollet, and several others).

From them, I tried to distill the essentials into three branches that you'll need to build a solid foundation for a career as a Data Analyst/Scientist/ML Engineer.

Following are the three pillars along with the a list of concepts that make for a good starter program:

From my course here.

Programming for Complete Beginners in Data Science and Machine Learning

Programming means telling a computer predefined rules that help it process input data and then get the results.

Machine learning, on the other hand, is giving the machine the results and data to find the rules that best approximate the relationship between the data and the results.

Programming offers that base platform which you can use to automate, verify, and solve problems of any scale.

The next question is which language should you learn?

Since most of the courses, libraries, and books are written to support Python infrastructure, I recommend learning Python and so does Google’s guide. Which language you use is a personal choice and a lot of it depends on the type of problem you’re trying to solve.

Most beginners prefer Python as it is the best way to develop end-to-end projects and there is a very large community of fellow developers who can help you. Chances are that ~90% of the problems that you’ll encounter in your journey (especially in the beginning phase) are already solved and documented for you.

1. Essential Python Programming for Machine Learning

Most data roles are programming-based except for a few like business intelligence, market analysis, and product analysis.

I am going to focus on technical data jobs that require expertise in at least one programming language. I personally prefer Python over any other language because of its versatility and ease of learning  – hands-down a good pick for developing end-to-end projects.

Here are some of the topics/libraries you should study for data science/ML:

• Common data structures (data types, lists, dictionaries, sets, tuples), writing functions, logic, control flow, searching and sorting algorithms, object-oriented programming, and working with external libraries.

• Writing Python scripts to extract, format, and store data into files or back to databases.

• Handling multi-dimensional arrays, indexing, slicing, transposing, broadcasting and pseudorandom number generation using NumPy.

• Performing vectorized operations using scientific computing libraries like NumPy.

• Manipulate data with Pandas  –  series, dataframe, indexing in a dataframe, comparison operators, merging dataframes, mapping, and applying functions.

• Wrangling data using pandas  –  checking for null values, imputing it, grouping data, describing it, performing exploratory analysis, and so on.

• Data Visualization using Matplotlib  – the API hierarchy, adding styles, color, and markers to a plot, knowledge of various plots and when to use them, line plots, bar plots, scatter plots, histograms, boxplots, and seaborn for more advanced plotting.

2. Essential Mathematics for Data Science and Machine Learning

There are practical reasons why Math is essential for folks who want a career as an ML practitioner, Data Scientist, or Deep Learning Engineer.

Use linear algebra to represent data

_An image from the course: [https://www.wiplane.com/p/foundations-for-data-science-ml](https://www.wiplane.com/p/foundations-for-data-science-ml" data-href="https://www.wiplane.com/p/foundations-for-data-science-ml" class="markup--anchor markup--figure-anchor" rel="noopener ugc nofollow noopener" target="blank)

ML is inherently data-driven – data is at the heart of machine learning. We can think of data as vectors , an object that adheres to arithmetic rules. This leads us to understand how rules of linear algebra operate over arrays of data.

Use calculus to train ML models

Image from the course: [https://www.wiplane.com/p/foundations-for-data-science-ml](https://www.wiplane.com/p/foundations-for-data-science-ml" rel="noopener ugc nofollow noopener)

Model training happens does not happen “automatically”. Calculus is what drives the learning of most ML and DL algorithms.

One of the most commonly used optimization algorithms  ( gradient descent)  is an application of partial derivatives.

A model is a mathematical representation of certain beliefs and assumptions. It is said to learn (approximate) the process (linear, polynomial, etc) how the data is provided, was generated in the first place, and then make predictions based on that learned process.

Important topics include:

• Basic algebra  –  ****  variables, coefficients, equations, functions — linear, exponential, logarithmic, etc.

• Linear Algebra  –  scalars, vectors, tensors, Norms(L1 & L2), dot product, types of matrices, linear transformation, representing linear equations in matrix notation, solving linear regression problem using vectors and matrices.

• Calculus  –  **** derivatives and limits, derivative rules, chain rule (for backpropagation algorithm), partial derivatives (to compute gradients), the convexity of functions, local/global minima, the math behind a regression model, applied math for training a model from scratch.

3. Essential Statistics for Data Science

Every organisation today is striving to become data-driven. To achieve that, Analysts and Scientists need to be able to use data in different ways in order to drive decision making.

Describing data — from data to insights

Data always comes in raw and ugly. The initial exploration tells you what’s missing, how the data is distributed, and what’s the best way to clean it to meet the end goal.

In order to answer the questions you've defined, descriptive statistics enables you to transform each observation in your data into insights that make sense.

Quantifying uncertainty

Furthermore, the ability to quantify uncertainty is the most valuable skill that is highly regarded at any data company. Knowing the chances of success in any experiment/decision is crucial for all businesses.

Here are a few of the main staples of statistics that constitute the bare minimum:

_Image from the lecture on Poisson distribution — [https://www.wiplane.com/p/foundations-for-data-science-ml](https://www.wiplane.com/p/foundations-for-data-science-ml" data-href="https://www.wiplane.com/p/foundations-for-data-science-ml" class="markup--anchor markup--figure-anchor" rel="noopener ugc nofollow noopener" target="blank)

• Estimates of location  – mean, median and other variants of these.

• Estimates of variability

• Correlation and covariance

• Random variables  – discrete and continuous

• Data distributions –  PMF, PDF, CDF

• Conditional probability –  bayesian statistics

• Commonly used statistical distributions –  Gaussian, Binomial, Poisson, Exponential.

• Important theorems  – Law of large numbers and Central limit theorem.

_Image from the lecture on Poisson distribution — [https://www.wiplane.com/p/foundations-for-data-science-ml](https://www.wiplane.com/p/foundations-for-data-science-ml" data-href="https://www.wiplane.com/p/foundations-for-data-science-ml" class="markup--anchor markup--figure-anchor" rel="noopener ugc nofollow noopener" target="blank)

Every beginner-level data science enthusiast should focus on these three pillars before diving into any core data science or core ML course

How to Learn these Foundational DS and ML Concepts

I created a learning roadmap that you can find here. It also told you what to learn and was also loaded with resources, courses, and programs that you can check out.

But there are a few inconsistencies in the recommended resources and the roadmap that I charted out.

Problems with Data Science or ML Courses

1. Every data science course that I listed in that article requires students to have a decent understanding of Programming, Math, or Statistics. For example, the most famous course on ML by Andrew Ng also relies heavily on the understanding of vector algebra and calculus.

2. Most courses that cover Math and Statistics for Data Science are just a checklist of concepts required for DS/ML with no explanation on how they are applied and how they are programmed into a machine.

3. There are exceptional resources to dive deep into Math but most of us are not made for it and you don't need to be a gold medalist to learn data science.

Bottom line: you need a resource that covers just enough applied math or statistics or programming to get started with data science or ML is missing.

Wiplane Academy — wiplane.com

https://www.wiplane.com

So, I decided to give in and develop the course myself. I have spent months designing and developing a curriculum that will provide a solid foundation for your career as a…

• Data Analyst

• Data Scientist

• Or an ML Practitioner/Engineer

Here's the course – Foundations for Data Science or ML — First Steps to learn Data Science and ML

It's a comprehensive yet compact and affordable course that not only covers all the essentials, pre-requisites, and pre-work but also explains how each concept is used computationally and programmatically ( in Python).

And that’s not it – I will keep updating the course content every month based on your inputs. Learn more here.

When you land on the page, right above the search box, you'll see Google's logo. On many occassions the logo will be different and accompanied by what's called a Doodle.

These Doodles range from small, simple and temporary alterations to the traditional logo. But more often that not, they are very creative and artistic expressions.

They highlight world issues, major historical events, worldwide holidays, local celebrations, anniversaries and important figures' brithdays (including pioneers, artists, and scientists) along with their contributions to society.

When you click on the Doodle animation you can find more information about that specific topic and a little game or interaction sometimes sometimes accompanies the doodle.

The First Google Doodle

The first doodle was created when Google co-founders Larry Page and Sergey Brin attended the annual Burning Man Festival in the Nevada desert in the United States on 30 August 1998.

They put a stick figure icon, resembling the Burning Man logo, behind the second "O" as an "Out Of Office" message to let users know of their absence in a fun and creative way.

Google was very young at the time – it had only been a couple of years since the search engine had been built by these two Stanford students. And this was just less than a week before it would be offically incorporated as a company.

You can also see that the logo was different, with an exclamation mark at the end.

The Development of Google Doodles

Since then, this new idea of altering the logo when celebrating notable events was born, and the Doodles progressed.

Nearly two years later, Page and Brin asked Dennis Hwang – an intern at the time – to create a doodle in honor of Bastille Day, on July 14 2000. It was very well received and Dennis Hwang was appointed Chief Doodler.

Doodles started to appear more regularly on the homepage.

Since then, Google has a whole dedicated team of talented illustrators (called doodlers) responsible for the doodle that appears on Google's homepage for the entire globe.

The early doodles were fairly simple. When you hovered over the doodle, a little pop up text would appear noting what was being commemorated.

For Halloween in 2000, guest artist Lorie Loeb created the very first animated doodle. It featured two jack-o-lanterns in place of the "O"’s and a spider was dangling from the "L".

Another notable tech-related, nostalgic doodle was that of March 12 2019, where Google commemorated the 30th anniversary of the World Wide Web.

You can view a historical archive of all the Google Doodles that have appeared on the homepage, letting you relive any Doodle and discover new ones you may not have seen already.

The First Google Doodle Game

The doodle technology kept on improving, and on May 21st 2010 Google released its first interactive game – the first ever playable Doodle Google game.

This release celebrated the 30th anniverary of one of the most popular and classic arcade games, Pac Man, which was released in 1980.

It was an immense development for Google and it must have brightend many peoples' days by bringing back such a fun and addictive throwback game. It was estimated that over 1 billion people played the game in the first three days of its release.

The PacMan doodle game brings up 8-bit nostalgic memories as it uses the same graphics, and the same original colors, sounds, and iconic characters. It even incudes the same original game logic and bugs (!).

If you want to take a trip down memory lane, or be introduced to the game for the first time, you can play it here.

Just press "Insert Coin" and use the arrow keys on your keyboard. You are a yellow hero aiming to eat all the dots without getting caught by ghosts of different colors.

Be prepared to spend one too many hours on this one.

How to Play Doodle Google Games

Since then there have been many more fun and interactive games and puzzles.

The doodles and games usually stay up for one day to commemorate a specific event. But Google archives all its old doodles and doodle games on the dedicated archive page instead of just getting rid of them.

Thanks to this, you can play any of the games that have been featured on the search engine's homepage.

Playing them is easy and fuss free.

They are all web based and can be played on both desktop and mobile without needing to download anything or set up a console. You don't need any equipment – just a browser and an internet connection.

Click on the links to go to the page for the specific game and then click to start the game.

In most cases you'll use your mouse or trackpad and keyboard. If you're playing from a mobile you'll be swipping or tapping.

Popular Doodle Google Games

Below is a list of some of the most well known and popular doodle games, grouped together based on different categories.

Doodle brain teaser games

Rubik's Cube

The Rubik's cube appeared on the homepage on May 19, 2014 and is one of the biggest and most fascinating mental challenges.

The colors move horizontally, but you can move its pieces by clicking anywhere on the cube to turn it on all its sides.

Crossword puzzles

On December 21st 2013, Google came out with a tribute to crosswords to commemorate 100 years of crossword puzzles.

With the help of Merl Reagle, one of the best and most well known crossword instructors, Google created their own version of a crossword with clues. It also shared bits of puzzle history throughout.

Doodle coding games

Coding for carrots

Coding for carrots introduces childern (or you!) to the world of programming by connecting blocks that represent coding concepts. This helps guide a rabbit across 6 levels to get to its favorite food – a carrot – and collect it.

The game was released on 4 December 2017 during Computer Science Education week in collaboration with MIT to celebrate 50 years of Kids Coding and of coding languages for kids. It's based on the Scratch programming language.

Doodle games for sports fans

Cricket

Cricket was introduced to celebrate the 2017 ICC Champions Trophy and you play cricket as an animated cricket.

Just click on the bat to swing it when you see the ball coming towards you and want to hit it to score. It's a clever re-creation of the sport.

Baseball

Baseball was introduced in honor of 4th of July, US Independence day, in 2019. The players are classic American snacks like hot dogs, fries, and ketchup.

Click to swing the bat and if you're like I am, be prepared to hear "You're Out" too many times (otherwise you'll see plenty of fireworks).

Basketball

Basketball came out in 2012, honoring the 2012 Olympic Summer games with this interactive basketball game. You try to make as many baskets as you can in 24 seconds.

Besides this game, for that period Google released three other separate interactive sports games, like Soccer. With this one you're a goal keeper, using the space key to stop the incoming ball from reaching the net and the arrow keys to change your position.

2020 Tokyo Olympics

In celebration of the 2020 Tokyo Olympics,Google released its most ambitious, elaborate, and complex game to date – a full on anime sports game called the Doodle Champion Island Games.

It's a homage to retro 16-bit Japanese video games, with 7 mini Olympic game sporting event themes to choose from (including table tennis, skateboarding, and artistic swimming among others). It's the largest game Google has released.

You play as a Calico ninja warrior cat named Lucky that arrives on the island where the festival is underway. You can choose from one of 4 teams - the Blue, Red, Yellow, or Green team - represented by Ushi, Karasu, Inari, or Kappa.

To play, use the arrow keys and space bar and pay tribute by celebrating Japanese history, culture, and sports throughtout the game. It was made in partnership with Japanese animation studio Studio 4°C.

Doodle games for music lovers

Celebrating Oskar Fischinger

Fischinger was created on 17 June 2017 to celebrate Oskar Fischinger's 117th birthday.

To honor the filmmaker and visual artist, Google released an interactive design with a visual music composition tool, letting users click and create their own visual pieces of music with the wide variety of settings you can change.

Celebrating the birth of Hip Hop

Hip Hop was released to commemorate the 44th anniversary of the birth of hip-hop, on 11th August 2017.

With the crossfader you can mix legendary beats together, change the volume and intensity of playback speed, and pick tracks. The classic DJ interactive turntable will let you cut and scratch tracks and relive hip-hop history.

The graffiti was made by artist Cey Adams, and it's narrated by hip-hop icon Fab Five Freddy. It pays tribute to Kool Herc and Coke La Rock, the two founding fathers of hip hop.

Rockmore

Rockmore simulates the experience of the Theremin. This is an electronic musical instrument that works without physical contact – only by gesture controls.

The game celebrates the anniversary of Clara Rockmore’s 105th birthday, the inventor and the instrument's most well known performer, on 9th March 2016.

It will let you create music and experiment with the different settings and replicate different notes. Move your cursor or mouse over the notes on the screen to create music using a virtual Theremin.

Doodle games for creatives

Quick draw

Quick draw! is a game that got its own website. You are given instructions on the screen about what to draw and a 20 second timeframe to do so. Google's neural network then makes guesses and tells you what your drawing resembles.

The game is built with Machine Learning and is one of the most popular Doodle games.

Doodle games for science-fiction enthusiasts

Doctor Who

The Doctor Who doodle game, released on 23 November 2013, was created to honor the 50th anniversary of the classic and popular British television show, Doctor Who.

It's a simple but impressive adventure puzzle game, featuring all of the 13 doctors at the time it was made.

The goal is to retrieve all of the 6 letters from Google logo that the Daleks stole from the doctors, going through the 6 levels of the game.

Doodle food games

Scoville was released on 22nd January 2016 to celebrate Wilbur Scoville’s 151st birthday.

The Scoville Scale is a system that measures how hot a pepper is. In this game, you're an ice cream fighting against spicy peppers by throwing scoops at them. Peppers start progressively getting hotter each time.

Doodle games for people who love Halloween

The Magic Cat Academy

The Magic Cat Academy celebrated Halloween in 2016 and follows a cat named Momo who wants to restore peace at the Magic Cat Academy where she's a student so she can rescue the school.

The muse for this game was a black cat that belongs to the Doodler who built the game, Juliana Chen.

Momo has to go through 5 levels in her school – the library, cafeteria, classroom, gym, and building rooftop. And along the way she has to swipe her wand to trace symbols above the heads of the ghosts that appear! This way, Momo casts spells and won't get her spellbook taken away from her by the ghosts that are lurking.

Use your keypad to swipe above the ghosts's heads, turning them into thin air.

Doodle games for those who enjoy card games

Loteria

Loteria is a traditional Mexican card game. You can start a match with your friends or with random people from all around the globe.

It was released on 9th December 2019, celebrating this traditional game with a virtual version. There is a tutorial before your first trial round if you've never played Lotería before.

Conclusion

I hope you enjoyed this write-up of some of Google's most popular Doodle Games for you to try.

There are many more available on the dedicated archive page, so go have a look and see what interests you.

Thanks for reading!

Copyright: All images belong to Google