Yet most developers have never built one.

That's a missed opportunity. CLI tools are one of the most practical things a developer can ship. They automate repetitive workflows, standardise processes across teams, and, when published, become tangible artifacts that the developer community can discover, install, and use.

In this handbook, you'll go from zero to building a fully distributed Dart CLI tool. We'll start with the fundamentals – how CLIs work, how Dart receives and processes terminal input, and the core syntax you need to know. Then we'll build three progressively complex CLIs, starting with the basics and finishing with a real-world API request runner. Finally, we will cover every distribution path available, from pub.dev to compiled binaries, Homebrew taps, Docker, and local team activation.

By the end of the guide, you'll understand both how to build a CLI tool in Dart as well as how to ship it so other developers can actually use it.

Table of Contents

• Prerequisites

• What is a CLI and Why Should You Build One?

• CLI Syntax Anatomy

• How Dart Receives Terminal Input

• Core CLI Concepts in Dart

  • stdout, stderr, and stdin

  • Exit Codes

  • Environment Variables

  • File and Directory Operations

  • Running External Processes

  • Platform Detection

  • Async in CLI

• Setting Up Your Dart CLI Project

• CLI 1 — Hello CLI: The Fundamentals

• CLI 2 — dart_todo: A Terminal Task Manager

  • Introducing the args Package

  • Building dart_todo

• CLI 3 — dart_http: A Lightweight API Request Runner

  • Building dart_http

• Adding Color and Polish to Your CLI

• Testing Your CLI Tool

• Deploying and Distributing Your CLI

  • Mode 1: pub.dev — Public Package Distribution

  • Mode 2: Local Path Activation

  • Mode 3: Compiled Binary via GitHub Releases

  • Mode 4: Homebrew Tap

  • Mode 5: Docker

• Choosing the Right Distribution Mode

• Conclusion

Prerequisites

Before starting, you should have:

• Dart SDK installed (dart --version should work in your terminal)

• Basic familiarity with Dart syntax

• Comfort with the terminal and running commands

• A pub.dev account (for the publishing section)

• A GitHub account (for the binary distribution section)

What is a CLI and Why Should You Build One?

A CLI (or Command Line Interface) is a program you interact with entirely through text commands in a terminal, rather than through buttons and screens in a graphical interface.

Many of the tools you likely already rely on as a developer are CLI tools:

flutter build apk
git commit -m "fix: auth flow"
dart pub get
npm install

Flutter, Git, Dart, npm – all CLIs. You are already a CLI user every single day. This article is about becoming a CLI builder.

There are three strong reasons to build CLI tools as a developer:

1. Automating repetitive work: Anything you type more than twice a week is a candidate for automation. Generating boilerplate folder structures, running sequences of commands, scaffolding files, checking environments before a build a CLI turns a seven-step manual process into a single command.

2. Standardising team workflows: Instead of a README that says "run these commands in this order," you ship one command that does all of it – consistently, every time, with no room for human error or a missed step.

3. Building and publishing tooling. A published Dart CLI package is a tangible artifact. It shows up on pub.dev, gets installed and used by other developers, and communicates real engineering depth in a way that a portfolio or resume cannot.

CLI Syntax Anatomy

Before writing a single line of code, it helps to understand the structure of a CLI command. Every command follows a consistent pattern:

tool [subcommand] [arguments] [options/flags]

Breaking down a real example:

flutter build apk --release --obfuscate
│       │     │   │
tool    sub   arg  flags

• Tool — the program itself (flutter, dart, git)

• Subcommand — the action being performed (build, run, pub)

• Arguments — what the action operates on (apk, main.dart, a filename)

• Flags and Options — modifiers that change behaviour

There are two types of options:

--release              # Boolean flag — either present or absent

--output=build/app     # Key-value option — name and a value
-v                     # Short flag — single hyphen, single character

This is the anatomy your CLIs will follow. Understanding it before writing any code means you will design your commands intentionally rather than stumbling into structure by accident.

How Dart Receives Terminal Input

In Dart, everything the user types after your tool name is passed into your program through the main function:

void main(List<String> args) {
  print(args);
}

Run it:

dart run bin/mytool.dart hello world --name=Seyi
# [hello, world, --name=Seyi]

That List<String> args is just a list of strings. Each word or flag the user typed becomes an element in that list. Everything else you build on top of a CLI subcommands, flags, validation — is ultimately just processing this list.

Core CLI Concepts in Dart

Before building anything, there's a set of foundational concepts that every CLI developer needs to understand. These are the building blocks that everything else sits on top of.

stdout, stderr, and stdin

Most developers use print() for all output when they start building CLIs. That works for learning but it's incorrect in production.

There are two separate output streams in a terminal program:

• stdout — regular output, meant for the user

• stderr — error output, meant for diagnostic messages and failures

import 'dart:io';

void main(List<String> args) {
  if (args.isEmpty) {
    stderr.writeln('Error: no arguments provided');
    exit(1);
  }

  stdout.writeln('Processing: ${args[0]}');
}

Keeping these separate matters because users can redirect stdout to a file without errors polluting it:

dart run bin/tool.dart > output.txt
# Errors still appear in the terminal
# Normal output goes cleanly to the file

Tools like git, flutter, and curl all do this correctly. Your CLI should too.

stdin is the third stream — reading input from the user interactively at runtime:

import 'dart:io';

void main() {
  stdout.write('Enter your name: ');
  final name = stdin.readLineSync();

  if (name == null || name.trim().isEmpty) {
    stderr.writeln('Error: no name provided');
    exit(1);
  }

  stdout.writeln('Hello, $name!');
}

stdout.write (without ln) keeps the cursor on the same line so the user types right after the prompt. stdin.readLineSync() blocks until the user presses Enter and returns the typed string, or null if the stream closes unexpectedly. Always handle the null case.

Exit Codes

Every program returns an exit code when it finishes. This is how the shell – and any script or CI system calling your tool – knows whether it succeeded or failed.

import 'dart:io';

void main(List<String> args) {
  if (args.isEmpty) {
    stderr.writeln('Error: please provide an argument');
    exit(1); // failure
  }

  stdout.writeln('Done');
  exit(0); // success — also the default if you don't call exit()
}

The conventions are:

• 0 — success

• 1 — general failure

• 2 — incorrect usage (wrong arguments, missing flags)

Exit codes are critical when your CLI is called inside shell scripts or GitHub Actions workflows. A non-zero exit code stops a pipeline immediately. That's exactly the behaviour you want from a quality gate or a validation step.

Environment Variables

Your CLI can read environment variables set in the user's shell:

import 'dart:io';

void main() {
  final token = Platform.environment['API_TOKEN'];

  if (token == null) {
    stderr.writeln('Error: API_TOKEN environment variable is not set');
    exit(1);
  }

  stdout.writeln('Token found — proceeding...');
}

Set it in the terminal and run:

export API_TOKEN=mytoken123
dart run bin/tool.dart
# Token found — proceeding...

This pattern is essential for CLI tools that interact with APIs, cloud services, or CI environments where credentials should never be hardcoded.

File and Directory Operations

Many CLI tools read from or write to the file system. Dart's dart:io library covers everything you need:

import 'dart:io';

void main(List<String> args) {
  if (args.isEmpty) {
    stderr.writeln('Usage: tool <filename>');
    exit(2);
  }

  final file = File(args[0]);

  if (!file.existsSync()) {
    stderr.writeln('Error: "${args[0]}" not found');
    exit(1);
  }

  final contents = file.readAsStringSync();
  stdout.writeln(contents);

  final output = File('output.txt');
  output.writeAsStringSync('Processed:\n$contents');
  stdout.writeln('Written to output.txt');
}

Working with directories:

import 'dart:io';

void main() {
  // Where the command was run from
  final cwd = Directory.current.path;
  stdout.writeln('Working directory: $cwd');

  // Create a directory relative to current location
  final dir = Directory('$cwd/generated');

  if (!dir.existsSync()) {
    dir.createSync(recursive: true);
    stdout.writeln('Created: ${dir.path}');
  } else {
    stdout.writeln('Already exists: ${dir.path}');
  }
}

The recursive: true flag on createSync means it creates all intermediate directories — equivalent to mkdir -p in bash.

Running External Processes

One of the most powerful things a CLI can do is call other programs. Your Dart CLI can run git, flutter, dart, or any shell command programmatically:

import 'dart:io';

void main() async {
  // Run a command and wait for it to finish
  final result = await Process.run('dart', ['pub', 'get']);

  stdout.write(result.stdout);

  if (result.exitCode != 0) {
    stderr.write(result.stderr);
    exit(result.exitCode);
  }

  stdout.writeln('Dependencies installed successfully');
}

For long-running commands where you want output to stream live as it happens:

import 'dart:io';

void main() async {
  final process = await Process.start('flutter', ['build', 'apk']);

  // Pipe output directly to the terminal in real time
  process.stdout.pipe(stdout);
  process.stderr.pipe(stderr);

  final exitCode = await process.exitCode;
  exit(exitCode);
}

Process.run — waits for completion, returns all output at once. Use for short commands.

Process.start — streams output live as it arrives. Use for long-running commands where the user needs to see progress.

Platform Detection

Sometimes your CLI needs to behave differently depending on the operating system it is running on:

import 'dart:io';

void main() {
  if (Platform.isWindows) {
    stdout.writeln('Running on Windows');
  } else if (Platform.isMacOS) {
    stdout.writeln('Running on macOS');
  } else if (Platform.isLinux) {
    stdout.writeln('Running on Linux');
  }

  // Useful for path handling across operating systems
  stdout.writeln(Platform.pathSeparator); // \ on Windows, / elsewhere
  stdout.writeln(Platform.operatingSystem); // 'macos', 'linux', 'windows'
}

This matters when your CLI creates files, resolves paths, or calls shell commands that differ between operating systems.

Async in CLI

Dart CLIs support async/await natively. Any main function can be made async:

import 'dart:io';

void main() async {
  stdout.writeln('Starting...');

  await Future.delayed(const Duration(seconds: 1)); // simulating async work

  stdout.writeln('Done');
}

Any operation involving file I/O, HTTP requests, or spawning processes will be asynchronous. Get comfortable with async main functions early — you'll use them constantly.

Setting Up Your Dart CLI Project

Create a new Dart console project:

dart create -t console my_cli_tool
cd my_cli_tool

This generates a clean structure:

my_cli_tool/
  bin/
    my_cli_tool.dart    ← entry point
  lib/                  ← shared library code
  test/                 ← tests
  pubspec.yaml
  README.md

The bin/ directory is where your executable entry point lives. The lib/ directory is where you put everything else — commands, utilities, models — that bin/ imports and uses.

Open pubspec.yaml. You'll need to add an executables block before publishing:

name: my_cli_tool
description: A sample CLI tool built with Dart
version: 1.0.0

environment:
  sdk: '>=3.0.0 <4.0.0'

executables:
  my_cli_tool: my_cli_tool  # executable name: bin file name

dependencies:
  args: ^2.4.2

dev_dependencies:
  lints: ^3.0.0
  test: ^1.24.0

The executables block is what makes dart pub global activate my_cli_tool work. It tells Dart which script in bin/ to expose as a runnable command after installation.

CLI 1 — Hello CLI: The Fundamentals

This first CLI uses pure Dart — no packages. The goal is to get comfortable with args, subcommands, input validation, and exit codes before introducing any external dependencies.

Replace the contents of bin/my_cli_tool.dart:

import 'dart:io';

void main(List<String> args) {
  if (args.isEmpty) {
    printHelp();
    exit(0);
  }

  final command = args[0];

  switch (command) {
    case 'greet':
      handleGreet(args.sublist(1));
    case 'time':
      handleTime();
    case 'echo':
      handleEcho(args.sublist(1));
    case 'help':
      printHelp();
    default:
      stderr.writeln('Unknown command: "$command"');
      stderr.writeln('Run "mytool help" to see available commands.');
      exit(1);
  }
}

void handleGreet(List<String> args) {
  if (args.isEmpty) {
    stderr.writeln('Usage: mytool greet <name>');
    exit(2);
  }

  final name = args[0];
  stdout.writeln('Hello, $name! Welcome to your first Dart CLI.');
}

void handleTime() {
  final now = DateTime.now();
  stdout.writeln(
    'Current time: ${now.hour.toString().padLeft(2, '0')}:'
    '${now.minute.toString().padLeft(2, '0')}:'
    '${now.second.toString().padLeft(2, '0')}',
  );
}

void handleEcho(List<String> args) {
  if (args.isEmpty) {
    stderr.writeln('Usage: mytool echo <message>');
    exit(2);
  }

  stdout.writeln(args.join(' '));
}

void printHelp() {
  stdout.writeln('''
mytool — a simple Dart CLI

Usage:
  mytool <command> [arguments]

Commands:
  greet <name>      Greet someone by name
  time              Show the current time
  echo <message>    Echo a message back to the terminal
  help              Show this help message

Examples:
  mytool greet Seyi
  mytool echo "Hello from the terminal"
  mytool time
  ''');
}

Run it:

dart run bin/my_cli_tool.dart help

dart run bin/my_cli_tool.dart greet Seyi
# Hello, Seyi! Welcome to your first Dart CLI.

dart run bin/my_cli_tool.dart time
# Current time: 14:32:10

dart run bin/my_cli_tool.dart echo "Dart CLIs are powerful"
# Dart CLIs are powerful

dart run bin/my_cli_tool.dart unknown
# Unknown command: "unknown"
# Run "mytool help" to see available commands.

Three things this CLI demonstrates that are worth internalising:

1. Subcommands are just a switch on args[0]. The pattern is simple and scalable — add a new case to add a new command.

2. args.sublist(1) passes remaining args to the handler. When greet receives ['greet', 'Seyi'], it calls handleGreet(['Seyi']) — clean and isolated.

3. Every error path has a message and a non-zero exit code. The user always knows what went wrong and what to do next.

CLI 2 — dart_todo: A Terminal Task Manager

This CLI introduces the args package, JSON file persistence, and structured terminal output. It's meaningfully more complex than CLI 1 and reflects real patterns you will use in production tools.

Introducing the args Package

Manually parsing List<String> args works for simple cases, but breaks down quickly when you add flags like --priority=high, boolean options like --done, or commands with multiple optional arguments.

The args package handles all of that cleanly.

Add it to your pubspec.yaml:

dependencies:
  args: ^2.4.2

Run:

dart pub get

The core concept in args is the ArgParser. You define what your CLI accepts, and args handles parsing, validation, and generating help text automatically:

import 'package:args/args.dart';

void main(List<String> arguments) {
  final parser = ArgParser()
    ..addCommand('add')
    ..addCommand('list')
    ..addFlag('help', abbr: 'h', negatable: false);

  final results = parser.parse(arguments);

  if (results['help'] as bool) {
    print(parser.usage);
    return;
  }
}

For more complex CLIs with subcommands that each have their own flags, use ArgParser per command:

final parser = ArgParser();

final addCommand = ArgParser()
  ..addOption('priority', abbr: 'p', defaultsTo: 'normal');

parser.addCommand('add', addCommand);

Building dart_todo

Create a fresh project:

dart create -t console dart_todo
cd dart_todo

Update pubspec.yaml:

name: dart_todo
description: A terminal task manager built with Dart
version: 1.0.0

environment:
  sdk: '>=3.0.0 <4.0.0'

executables:
  dart_todo: dart_todo

dependencies:
  args: ^2.4.2

dev_dependencies:
  lints: ^3.0.0
  test: ^1.24.0

Run dart pub get.

Create the folder structure:

dart_todo/
  bin/
    dart_todo.dart
  lib/
    models/
      task.dart
    storage/
      task_storage.dart
    commands/
      add_command.dart
      list_command.dart
      complete_command.dart
      delete_command.dart
      clear_command.dart
  pubspec.yaml

Step 1 — The Task Model (lib/models/task.dart)

class Task {
  final int id;
  final String title;
  final String priority;
  final bool isComplete;
  final DateTime createdAt;

  Task({
    required this.id,
    required this.title,
    required this.priority,
    this.isComplete = false,
    required this.createdAt,
  });

  Task copyWith({bool? isComplete}) {
    return Task(
      id: id,
      title: title,
      priority: priority,
      isComplete: isComplete ?? this.isComplete,
      createdAt: createdAt,
    );
  }

  Map<String, dynamic> toJson() => {
        'id': id,
        'title': title,
        'priority': priority,
        'isComplete': isComplete,
        'createdAt': createdAt.toIso8601String(),
      };

  factory Task.fromJson(Map<String, dynamic> json) => Task(
        id: json['id'] as int,
        title: json['title'] as String,
        priority: json['priority'] as String,
        isComplete: json['isComplete'] as bool,
        createdAt: DateTime.parse(json['createdAt'] as String),
      );
}

Step 2 — Storage (lib/storage/task_storage.dart)

This class handles reading and writing tasks to a local JSON file so they persist between CLI runs:

import 'dart:convert';
import 'dart:io';

import '../models/task.dart';

class TaskStorage {
  static final _file = File(
    '${Platform.environment['HOME'] ?? Directory.current.path}/.dart_todo.json',
  );

  static List<Task> loadAll() {
    if (!_file.existsSync()) return [];

    try {
      final content = _file.readAsStringSync();
      final List<dynamic> json = jsonDecode(content) as List<dynamic>;
      return json
          .map((e) => Task.fromJson(e as Map<String, dynamic>))
          .toList();
    } catch (_) {
      return [];
    }
  }

  static void saveAll(List<Task> tasks) {
    final json = jsonEncode(tasks.map((t) => t.toJson()).toList());
    _file.writeAsStringSync(json);
  }
}

Tasks are stored in a hidden JSON file in the user's home directory — a common pattern for CLI tools that need lightweight local persistence.

Step 3 — Commands

lib/commands/add_command.dart:

import 'dart:io';

import '../models/task.dart';
import '../storage/task_storage.dart';

void runAdd(List<String> args, String priority) {
  if (args.isEmpty) {
    stderr.writeln('Usage: dart_todo add <title> [--priority=high|normal|low]');
    exit(2);
  }

  final title = args.join(' ');
  final tasks = TaskStorage.loadAll();

  final newTask = Task(
    id: tasks.isEmpty ? 1 : tasks.last.id + 1,
    title: title,
    priority: priority,
    createdAt: DateTime.now(),
  );

  tasks.add(newTask);
  TaskStorage.saveAll(tasks);

  stdout.writeln('Added task #\({newTask.id}: "\)title" [$priority]');
}

lib/commands/list_command.dart:

import 'dart:io';

import '../storage/task_storage.dart';

void runList() {
  final tasks = TaskStorage.loadAll();

  if (tasks.isEmpty) {
    stdout.writeln('No tasks yet. Add one with: dart_todo add <title>');
    return;
  }

  stdout.writeln('');
  stdout.writeln('  ID   Status      Priority   Title');
  stdout.writeln('  ───  ──────────  ─────────  ────────────────────────');

  for (final task in tasks) {
    final status = task.isComplete ? 'done  ' : 'pending';
    final id = task.id.toString().padRight(4);
    final priority = task.priority.padRight(9);
    stdout.writeln('  \(id \)status  \(priority  \){task.title}');
  }

  stdout.writeln('');
}

lib/commands/complete_command.dart:

import 'dart:io';

import '../storage/task_storage.dart';

void runComplete(List<String> args) {
  if (args.isEmpty) {
    stderr.writeln('Usage: dart_todo complete <id>');
    exit(2);
  }

  final id = int.tryParse(args[0]);
  if (id == null) {
    stderr.writeln('Error: "${args[0]}" is not a valid task ID');
    exit(1);
  }

  final tasks = TaskStorage.loadAll();
  final index = tasks.indexWhere((t) => t.id == id);

  if (index == -1) {
    stderr.writeln('Error: No task found with ID $id');
    exit(1);
  }

  if (tasks[index].isComplete) {
    stdout.writeln('Task #$id is already complete.');
    return;
  }

  tasks[index] = tasks[index].copyWith(isComplete: true);
  TaskStorage.saveAll(tasks);

  stdout.writeln('Task #\(id marked as complete: "\){tasks[index].title}"');
}

lib/commands/delete_command.dart:

import 'dart:io';

import '../storage/task_storage.dart';

void runDelete(List<String> args) {
  if (args.isEmpty) {
    stderr.writeln('Usage: dart_todo delete <id>');
    exit(2);
  }

  final id = int.tryParse(args[0]);
  if (id == null) {
    stderr.writeln('Error: "${args[0]}" is not a valid task ID');
    exit(1);
  }

  final tasks = TaskStorage.loadAll();
  final index = tasks.indexWhere((t) => t.id == id);

  if (index == -1) {
    stderr.writeln('Error: No task found with ID $id');
    exit(1);
  }

  final title = tasks[index].title;
  tasks.removeAt(index);
  TaskStorage.saveAll(tasks);

  stdout.writeln('Deleted task #\(id: "\)title"');
}

lib/commands/clear_command.dart:

import 'dart:io';

import '../storage/task_storage.dart';

void runClear() {
  stdout.write('Are you sure you want to delete all tasks? (y/N): ');
  final input = stdin.readLineSync()?.trim().toLowerCase();

  if (input != 'y') {
    stdout.writeln('Cancelled.');
    return;
  }

  TaskStorage.saveAll([]);
  stdout.writeln('All tasks cleared.');
}

Step 4 — Entry Point (bin/dart_todo.dart)

import 'dart:io';

import 'package:args/args.dart';

import '../lib/commands/add_command.dart';
import '../lib/commands/clear_command.dart';
import '../lib/commands/complete_command.dart';
import '../lib/commands/delete_command.dart';
import '../lib/commands/list_command.dart';

void main(List<String> arguments) {
  final parser = ArgParser();

  // Add subcommand parsers
  final addParser = ArgParser()
    ..addOption(
      'priority',
      abbr: 'p',
      defaultsTo: 'normal',
      allowed: ['high', 'normal', 'low'],
      help: 'Task priority level',
    );

  parser
    ..addCommand('add', addParser)
    ..addCommand('list')
    ..addCommand('complete')
    ..addCommand('delete')
    ..addCommand('clear')
    ..addFlag('help', abbr: 'h', negatable: false, help: 'Show help');

  ArgResults results;

  try {
    results = parser.parse(arguments);
  } catch (e) {
    stderr.writeln('Error: $e');
    stderr.writeln(parser.usage);
    exit(2);
  }

  if (results['help'] as bool || results.command == null) {
    printHelp(parser);
    exit(0);
  }

  final command = results.command!;

  switch (command.name) {
    case 'add':
      runAdd(command.rest, command['priority'] as String);
    case 'list':
      runList();
    case 'complete':
      runComplete(command.rest);
    case 'delete':
      runDelete(command.rest);
    case 'clear':
      runClear();
    default:
      stderr.writeln('Unknown command: "${command.name}"');
      exit(1);
  }
}

void printHelp(ArgParser parser) {
  stdout.writeln('''
dart_todo — a terminal task manager

Usage:
  dart_todo <command> [arguments]

Commands:
  add <title>        Add a new task
    -p, --priority   Priority: high, normal, low (default: normal)
  list               List all tasks
  complete <id>      Mark a task as complete
  delete <id>        Delete a task
  clear              Delete all tasks

Examples:
  dart_todo add "Write the CLI article" --priority=high
  dart_todo list
  dart_todo complete 1
  dart_todo delete 2
  dart_todo clear
  ''');
}

Run it:

dart run bin/dart_todo.dart add "Write the CLI article" --priority=high
# Added task #1: "Write the CLI article" [high]

dart run bin/dart_todo.dart add "Review PR comments"
# Added task #2: "Review PR comments" [normal]

dart run bin/dart_todo.dart list
#   ID   Status      Priority   Title
#   ───  ──────────  ─────────  ────────────────────────
#   1    ⬜ pending  high       Write the CLI article
#   2    ⬜ pending  normal     Review PR comments

dart run bin/dart_todo.dart complete 1
# Task #1 marked as complete: "Write the CLI article"

dart run bin/dart_todo.dart delete 2
# Deleted task #2: "Review PR comments"

dart_todo demonstrates the patterns that form the backbone of almost every real CLI tool — argument parsing with args, JSON persistence, interactive prompts, structured output, and clean error handling across every command.

CLI 3 — dart_http: A Lightweight API Request Runner

This is the most complex CLI in this article – and the most immediately useful. dart_http lets developers make HTTP requests directly from the terminal, with pretty-printed JSON responses, response metadata, header support, and the ability to save responses to a file.

dart_http get https://jsonplaceholder.typicode.com/users/1
dart_http post https://jsonplaceholder.typicode.com/posts --body='{"title":"Hello"}'
dart_http get https://jsonplaceholder.typicode.com/users --save=users.json
dart_http get https://api.example.com/me --header="Authorization: Bearer mytoken"

Building dart_http

Create the project:

dart create -t console dart_http
cd dart_http

Update pubspec.yaml:

name: dart_http
description: A lightweight API request runner for the terminal
version: 1.0.0

environment:
  sdk: '>=3.0.0 <4.0.0'

executables:
  dart_http: dart_http

dependencies:
  args: ^2.4.2
  http: ^1.2.1

dev_dependencies:
  lints: ^3.0.0
  test: ^1.24.0

Run dart pub get.

Project structure:

dart_http/
  bin/
    dart_http.dart
  lib/
    runner/
      request_runner.dart
    printer/
      response_printer.dart
    utils/
      headers_parser.dart
  pubspec.yaml

Step 1 — Headers Parser (lib/utils/headers_parser.dart)

Map<String, String> parseHeaders(List<String> rawHeaders) {
  final headers = <String, String>{};

  for (final header in rawHeaders) {
    final index = header.indexOf(':');
    if (index == -1) continue;

    final key = header.substring(0, index).trim();
    final value = header.substring(index + 1).trim();
    headers[key] = value;
  }

  return headers;
}

Step 2 — Response Printer (lib/printer/response_printer.dart)

import 'dart:convert';
import 'dart:io';

void printResponse({
  required int statusCode,
  required String body,
  required int durationMs,
  required int bodyBytes,
}) {
  final statusLabel = _statusLabel(statusCode);
  final size = _formatSize(bodyBytes);

  stdout.writeln('');
  stdout.writeln('\(statusLabel | \){durationMs}ms | $size');
  stdout.writeln('─' * 50);

  try {
    final decoded = jsonDecode(body);
    const encoder = JsonEncoder.withIndent('  ');
    stdout.writeln(encoder.convert(decoded));
  } catch (_) {
    // Not JSON — print as plain text
    stdout.writeln(body);
  }

  stdout.writeln('');
}

String _statusLabel(int code) {
  if (code >= 200 && code < 300) return '✅ $code';
  if (code >= 300 && code < 400) return '↪️  $code';
  if (code >= 400 && code < 500) return '❌ $code';
  return '$code';
}

String _formatSize(int bytes) {
  if (bytes < 1024) return '${bytes}b';
  if (bytes < 1024 * 1024) return '${(bytes / 1024).toStringAsFixed(1)}kb';
  return '${(bytes / (1024 * 1024)).toStringAsFixed(1)}mb';
}

Step 3 — Request Runner (lib/runner/request_runner.dart)

import 'dart:io';

import 'package:http/http.dart' as http;

import '../printer/response_printer.dart';

Future<void> runRequest({
  required String method,
  required String url,
  required Map<String, String> headers,
  String? body,
  String? saveToFile,
}) async {
  final uri = Uri.tryParse(url);

  if (uri == null) {
    stderr.writeln('Error: "$url" is not a valid URL');
    exit(1);
  }

  stdout.writeln('→ \({method.toUpperCase()} \)url');

  http.Response response;
  final stopwatch = Stopwatch()..start();

  try {
    switch (method.toLowerCase()) {
      case 'get':
        response = await http.get(uri, headers: headers);
      case 'post':
        response = await http.post(uri, headers: headers, body: body);
      case 'put':
        response = await http.put(uri, headers: headers, body: body);
      case 'patch':
        response = await http.patch(uri, headers: headers, body: body);
      case 'delete':
        response = await http.delete(uri, headers: headers);
      default:
        stderr.writeln('Error: unsupported method "$method"');
        exit(2);
    }
  } catch (e) {
    stderr.writeln('Error: request failed — $e');
    exit(1);
  }

  stopwatch.stop();

  printResponse(
    statusCode: response.statusCode,
    body: response.body,
    durationMs: stopwatch.elapsedMilliseconds,
    bodyBytes: response.bodyBytes.length,
  );

  if (saveToFile != null) {
    final file = File(saveToFile);
    file.writeAsStringSync(response.body);
    stdout.writeln('Response saved to $saveToFile');
  }
}

Step 4 — Entry Point (bin/dart_http.dart)

import 'dart:io';

import 'package:args/args.dart';

import '../lib/runner/request_runner.dart';
import '../lib/utils/headers_parser.dart';

void main(List<String> arguments) async {
  final parser = ArgParser();

  for (final method in ['get', 'post', 'put', 'patch', 'delete']) {
    final commandParser = ArgParser()
      ..addMultiOption('header', abbr: 'H', help: 'Request header (repeatable)')
      ..addOption('body', abbr: 'b', help: 'Request body (for POST/PUT/PATCH)')
      ..addOption('save', abbr: 's', help: 'Save response body to a file');

    parser.addCommand(method, commandParser);
  }

  parser.addFlag('help', abbr: 'h', negatable: false, help: 'Show help');

  ArgResults results;

  try {
    results = parser.parse(arguments);
  } catch (e) {
    stderr.writeln('Error: $e');
    printHelp();
    exit(2);
  }

  if (results['help'] as bool || results.command == null) {
    printHelp();
    exit(0);
  }

  final command = results.command!;
  final method = command.name!;
  final rest = command.rest;

  if (rest.isEmpty) {
    stderr.writeln('Error: please provide a URL');
    stderr.writeln('Usage: dart_http $method <url>');
    exit(2);
  }

  final url = rest[0];
  final rawHeaders = command['header'] as List<String>;
  final body = command['body'] as String?;
  final saveToFile = command['save'] as String?;

  final headers = parseHeaders(rawHeaders);

  // Default Content-Type for requests with a body
  if (body != null && !headers.containsKey('Content-Type')) {
    headers['Content-Type'] = 'application/json';
  }

  await runRequest(
    method: method,
    url: url,
    headers: headers,
    body: body,
    saveToFile: saveToFile,
  );
}

void printHelp() {
  stdout.writeln('''
dart_http — a lightweight API request runner

Usage:
  dart_http <method> <url> [options]

Methods:
  get       Send a GET request
  post      Send a POST request
  put       Send a PUT request
  patch     Send a PATCH request
  delete    Send a DELETE request

Options:
  -H, --header    Add a request header (repeatable)
  -b, --body      Request body (JSON string)
  -s, --save      Save response body to a file
  -h, --help      Show this help message

Examples:
  dart_http get https://jsonplaceholder.typicode.com/users
  dart_http get https://api.example.com/me --header="Authorization: Bearer token"
  dart_http post https://api.example.com/posts --body=\'{"title":"Hello"}\'
  dart_http get https://api.example.com/users --save=users.json
  ''');
}

Run it:

dart run bin/dart_http.dart get https://jsonplaceholder.typicode.com/users/1

# → GET https://jsonplaceholder.typicode.com/users/1
# 200 | 87ms | 510b
# ──────────────────────────────────────────────────
# {
#   "id": 1,
#   "name": "Leanne Graham",
#   "username": "Bret",
#   "email": "Sincere@april.biz"
# }

dart run bin/dart_http.dart get https://jsonplaceholder.typicode.com/users --save=users.json
# → GET https://jsonplaceholder.typicode.com/users
# 200 | 143ms | 5.3kb
# ──────────────────────────────────────────────────
# [ ... ]
# Response saved to users.json

dart run bin/dart_http.dart post https://jsonplaceholder.typicode.com/posts \
  --body='{"title":"Hello from dart_http","userId":1}'
# → POST https://jsonplaceholder.typicode.com/posts
# 201 | 312ms | 72b

Adding Color and Polish to Your CLI

The CLIs above are functional, but terminal output can be made significantly more readable with color. The ansi_styles package provides ANSI escape code support for coloring text in the terminal.

Add it to pubspec.yaml:

dependencies:
  ansi_styles: ^0.3.0

Using it:

import 'package:ansi_styles/ansi_styles.dart';

stdout.writeln(AnsiStyles.green('✅ Success'));
stdout.writeln(AnsiStyles.red('❌ Error: something went wrong'));
stdout.writeln(AnsiStyles.yellow('⚠️  Warning: check your config'));
stdout.writeln(AnsiStyles.bold('dart_http — API request runner'));
stdout.writeln(AnsiStyles.cyan('→ GET https://api.example.com/users'));

Apply color intentionally and consistently:

• Green — success states, completed operations

• Red — errors and failures

• Yellow — warnings and non-blocking issues

• Cyan — informational output, URLs, paths

• Bold — headers, tool names, important values

Avoid coloring everything. Color loses meaning when it is everywhere. Use it to draw the user's eye to what actually matters.

Testing Your CLI Tool

CLI tools are testable, and they should be tested. The most reliable approach is to test the logic inside your commands directly — not the terminal output formatting, but the behaviour.

Add test to your dev dependencies if it's not already there:

dev_dependencies:
  test: ^1.24.0

Testing command logic:

import 'package:test/test.dart';

import '../lib/models/task.dart';

void main() {
  group('Task model', () {
    test('copyWith updates isComplete correctly', () {
      final task = Task(
        id: 1,
        title: 'Write tests',
        priority: 'high',
        createdAt: DateTime.now(),
      );

      final completed = task.copyWith(isComplete: true);

      expect(completed.isComplete, isTrue);
      expect(completed.title, equals('Write tests'));
      expect(completed.id, equals(1));
    });

    test('toJson and fromJson round-trips correctly', () {
      final task = Task(
        id: 2,
        title: 'Ship the tool',
        priority: 'normal',
        createdAt: DateTime.parse('2025-01-01T00:00:00.000'),
      );

      final json = task.toJson();
      final restored = Task.fromJson(json);

      expect(restored.id, equals(task.id));
      expect(restored.title, equals(task.title));
      expect(restored.priority, equals(task.priority));
    });
  });
}

Testing the headers parser:

import 'package:test/test.dart';

import '../lib/utils/headers_parser.dart';

void main() {
  group('parseHeaders', () {
    test('parses a single header correctly', () {
      final result = parseHeaders(['Authorization: Bearer mytoken']);
      expect(result['Authorization'], equals('Bearer mytoken'));
    });

    test('parses multiple headers', () {
      final result = parseHeaders([
        'Authorization: Bearer token',
        'Accept: application/json',
      ]);
      expect(result.length, equals(2));
      expect(result['Accept'], equals('application/json'));
    });

    test('ignores malformed headers without a colon', () {
      final result = parseHeaders(['malformed-header']);
      expect(result.isEmpty, isTrue);
    });
  });
}

Run your tests:

dart test

Deploying and Distributing Your CLI

Building a CLI tool is half the work. Getting it into the hands of developers is the other half. There are five distribution paths available, each suited to a different use case.

Mode 1: pub.dev — Public Package Distribution

Publishing to pub.dev makes your tool installable by anyone in the Dart and Flutter community with a single command.

Prepare your package:

Your pubspec.yaml needs to be complete:

name: dart_http
description: A lightweight API request runner for Dart developers.
version: 1.0.0
homepage: https://github.com/yourname/dart_http

environment:
  sdk: '>=3.0.0 <4.0.0'

executables:
  dart_http: dart_http

The executables block is critical. It tells pub.dev which script in bin/ to expose as a runnable command.

You also need:

• README.md — what the tool does, how to install it, usage examples

• CHANGELOG.md — version history

• LICENSE — an open source license (MIT is standard)

Validate before publishing:

dart pub publish --dry-run

This runs all validation checks without actually publishing. Fix any warnings before proceeding.

Publish:

dart pub publish

You will be prompted to authenticate with your pub.dev account. Once published, your tool is available globally:

dart pub global activate dart_http
dart_http get https://api.example.com/users

Mode 2: Local Path Activation

For internal team tools that you don't want to publish publicly, activate directly from a local or cloned repository:

dart pub global activate --source path /path/to/dart_http

Any developer on the team clones the repo and runs this command once. The tool is then available globally in their terminal without needing a pub.dev publish.

This is the right distribution mode for:

• Internal company tooling

• Tools that depend on private packages

• Work-in-progress tools shared within a team before a public release

Mode 3: Compiled Binary via GitHub Releases

Dart can compile to a self-contained native executable — no Dart SDK required on the target machine. This makes your tool accessible to developers outside the Dart ecosystem.

Compile:

# macOS
dart compile exe bin/dart_http.dart -o dist/dart_http-macos

# Linux
dart compile exe bin/dart_http.dart -o dist/dart_http-linux

# Windows
dart compile exe bin/dart_http.dart -o dist/dart_http-windows.exe

The compiled binary is fully self-contained. Copy it to any machine and run it — no Dart installation needed.

Automate with GitHub Actions:

Create .github/workflows/release.yml:

name: Release

on:
  push:
    tags:
      - 'v*'

jobs:
  build:
    strategy:
      matrix:
        os: [ubuntu-latest, macos-latest, windows-latest]
    runs-on: ${{ matrix.os }}

    steps:
      - uses: actions/checkout@v3

      - uses: dart-lang/setup-dart@v1
        with:
          sdk: stable

      - name: Install dependencies
        run: dart pub get

      - name: Compile binary
        run: |
          mkdir -p dist
          dart compile exe bin/dart_http.dart -o dist/dart_http-${{ runner.os }}

      - name: Upload binary to release
        uses: softprops/action-gh-release@v1
        with:
          files: dist/dart_http-${{ runner.os }}
        env:
          GITHUB_TOKEN: ${{ secrets.GITHUB_TOKEN }}

Every time you push a version tag (v1.0.0), GitHub Actions compiles binaries for all three platforms and attaches them to the GitHub Release automatically.

Write an install script:

#!/usr/bin/env bash
set -euo pipefail

VERSION="1.0.0"
OS=$(uname -s | tr '[:upper:]' '[:lower:]')
BINARY="dart_http-$OS"
INSTALL_DIR="/usr/local/bin"

curl -L "https://github.com/yourname/dart_http/releases/download/v\(VERSION/\)BINARY" \
  -o "$INSTALL_DIR/dart_http"

chmod +x "$INSTALL_DIR/dart_http"
echo "dart_http installed successfully"

Developers install it with:

curl -fsSL https://raw.githubusercontent.com/yourname/dart_http/main/install.sh | bash

Mode 4: Homebrew Tap

Homebrew is the standard package manager for macOS and is widely used on Linux. A Homebrew tap makes your tool installable with brew install — the most familiar installation pattern for macOS developers.

Create your tap repository:

Create a new GitHub repository named homebrew-tools (the homebrew- prefix is required by Homebrew's naming convention).

Write the formula:

Create Formula/dart_http.rb in that repository:

class DartHttp < Formula
  desc "A lightweight API request runner for the terminal"
  homepage "https://github.com/yourname/dart_http"
  version "1.0.0"

  on_macos do
    url "https://github.com/yourname/dart_http/releases/download/v1.0.0/dart_http-macOS"
    sha256 "YOUR_SHA256_HASH_HERE"
  end

  on_linux do
    url "https://github.com/yourname/dart_http/releases/download/v1.0.0/dart_http-Linux"
    sha256 "YOUR_SHA256_HASH_HERE"
  end

  def install
    bin.install "dart_http-#{OS.mac? ? 'macOS' : 'Linux'}" => "dart_http"
  end

  test do
    system "#{bin}/dart_http", "--help"
  end
end

Generate the SHA256 hash for each binary:

shasum -a 256 dist/dart_http-macOS

Install from the tap:

brew tap yourname/tools
brew install dart_http

When you release a new version, update the url and sha256 values in the formula and push the change. Users run brew upgrade dart_http to update.

Mode 5: Docker

Docker distribution is best suited for CI environments, teams that standardise on containers, or tools with complex dependencies.

Write a Dockerfile:

FROM dart:stable AS build

WORKDIR /app
COPY pubspec.* ./
RUN dart pub get

COPY . .
RUN dart compile exe bin/dart_http.dart -o /app/dart_http

FROM debian:stable-slim
COPY --from=build /app/dart_http /usr/local/bin/dart_http

ENTRYPOINT ["dart_http"]

This uses a multi-stage build: the first stage compiles the binary using the Dart SDK image, and the second stage copies only the binary into a minimal Debian image. The final image has no Dart SDK — just the compiled binary.

Build and run:

docker build -t dart_http .
docker run dart_http get https://jsonplaceholder.typicode.com/users/1

Publish to Docker Hub:

docker tag dart_http yourname/dart_http:1.0.0
docker push yourname/dart_http:1.0.0

Users can then run your tool without installing anything locally:

docker run yourname/dart_http get https://api.example.com/users

Choosing the Right Distribution Mode

For most tools, the practical recommendation is:

• Start with pub.dev if your audience is Dart developers

• Add compiled binary + GitHub Releases once you want broader adoption

• Add a Homebrew tap when macOS developers start asking for it

• Use Docker only when it is already part of your team's workflow

Conclusion

You've gone from understanding what a CLI is to building three progressively complex tools and distributing them across five different channels.

The foundational skills – args, stdin, stdout, stderr, exit codes, file I/O, and process spawning – are the same building blocks that tools like flutter, git, and dart themselves are built on. Everything else is composition.

The three CLIs we built (Hello CLI, dart_todo, and dart_http) each introduced a new layer: raw Dart fundamentals, the args package with JSON persistence, and real-world HTTP interaction. The distribution section ensures that whatever you build next, you have a clear path to getting it in front of the developers who will use it.

Dart is a powerful language for CLI development. Its strong typing, async support, native compilation, and pub.dev ecosystem make it a serious choice for building developer tooling, not just mobile apps.

The next step is building something that solves a real problem for you or your team, and shipping it.

Happy coding!!

OpenCode, an open-source project, takes a different path. It brings an AI assistant directly into your terminal, letting you write, debug, and refactor code using natural language. All this without ever leaving the command line.

It combines the power of large language models with real-world developer workflows, so you can build software faster, with fewer distractions.

Table of Contents

• What is OpenCode?

• Why Use OpenCode?

• How OpenCode Works

• The Power of Context

• Privacy and Control

• Real-World Use Cases

• Community and Ecosystem

• The Future of AI-Powered Development

• Conclusion

What is OpenCode?

OpenCode is an open-source AI coding assistant that works right inside your terminal. It’s built for developers who prefer command-line speed but still want the intelligence of modern AI models.

You can think of it as having ChatGPT or Claude built into your local development environment, except it’s fully open and under your control.

Imagine typing a command like opencode fix error in main.go, and the AI instantly reads your code, finds the problem, and suggests a clean fix. That’s the magic of OpenCode.

The project is hosted at github.com/sst/opencode and has quickly become one of the most popular open-source AI tools for developers. As of October 2025, it has more than 26,000 stars on GitHub, proving that developers are hungry for coding tools that blend automation and simplicity.

Why Use OpenCode?

Most AI coding assistants work inside editors like VS Code or JetBrains. OpenCode, on the other hand, lives in your terminal. This means it can work with any language, any editor, and any environment.

You can use it while building a backend in Go, a frontend in React, or even while managing infrastructure scripts.

OpenCode uses your project context to understand your code deeply. It scans through your files, recognizes dependencies, and maintains context across multiple commands. This allows it to perform complex operations like:

• Refactoring multiple files in one go

• Adding new features based on natural language instructions

• Explaining errors and suggesting fixes

• Reviewing your code before you commit

All of this happens without needing to upload your code to the cloud. Everything stays local, which is a major advantage for teams handling private or sensitive codebases.

How OpenCode Works

OpenCode connects your local files, git history, and LLMs together. When you run a command like opencode explain this function, it gathers context from the files you’re working on, passes that to an AI model, and shows the response directly in your terminal.

The setup is simple. You can install OpenCode using a single command:

curl -fsSL https://opencode.ai/install | bash

Once installed, you can start using it by running:

opencode

The tool will open an interactive terminal interface where you can chat, run tasks, and even let the AI make edits automatically. It supports multiple model providers, including OpenAI, Anthropic, and local models like Ollama.

Behind the scenes, OpenCode uses a plugin-like architecture that makes it easy to extend. Developers can write “actions” or “skills” that teach the AI how to perform domain-specific tasks — like generating Kubernetes manifests, writing API routes, or setting up unit tests.

Once Opencode is installed, go to the project repository and type opencode . Then type /init for Opencode to analyze your repository and created agents.md file.

You can then start asking questions like “What does this repository do?”. Here is a sample output from my portfolio website.

You can use the the OpenCode documentation to learn more tips and tricks to work with OpenCode.

The Power of Context

One of OpenCode’s biggest strengths is how it handles context. Traditional chatbots lose track of what you’re working on after a few turns. OpenCode doesn’t.

OpenCode remembers your codebase, understands imports, and keeps track of related files. This allows it to work more like a real developer assistant.

Let’s say you tell it: “Add authentication to my Express.js app.”

OpenCode will scan your project, identify where routes are defined, create middleware for login, and even suggest where to store tokens securely. It’s this blend of contextual awareness and natural language understanding that makes OpenCode feel more like a teammate than a tool.

Privacy and Control

A major reason developers love OpenCode is control. Unlike cloud-based assistants, OpenCode doesn’t send your code to remote servers by default.

You choose the model provider and what data gets shared. If you run a local model, your entire workflow stays private.

This is especially important for companies with strict data rules. With OpenCode, teams can integrate AI safely into their workflows without breaking compliance or risking leaks.

The tool also integrates with version control systems like Git. Every change the AI suggests can be previewed before committing. You can accept, reject, or modify them just like a pull request. This ensures transparency and keeps human developers in charge.

Real-World Use Cases

Developers are using OpenCode in many creative ways. Backend engineers use it to generate API routes. Frontend teams use it to fix TypeScript errors. DevOps engineers rely on it to generate Terraform scripts and Dockerfiles.

Even researchers and students find it useful for exploring new codebases. By simply asking questions like “What does this repository do?” or “Where is the entry point?”, they can get clear, AI-driven summaries of complex projects.

The flexibility of OpenCode means it can fit almost any workflow. It doesn’t replace your tools , it enhances them. Whether you’re using Vim, VS Code, or JetBrains IDEs, OpenCode works alongside your setup.

Community and Ecosystem

OpenCode is not just a tool, it’s a growing community. The project’s GitHub discussions and Discord are full of contributors sharing workflows, plugins, and even model configuration tips.

The maintainers, part of the SST team, are known for building tools that make cloud and AI development simpler. They continue to release frequent updates and listen closely to community feedback.

Recent updates have added features like persistent sessions, better error recovery, and support for local models. The roadmap includes plans for even deeper IDE integration and team collaboration features.

The Future of AI-Powered Development

As AI coding agents mature, the line between writing code and describing what you want will continue to blur. Tools like OpenCode show us what that future might look like , one where developers spend less time fighting syntax and more time building ideas.

Imagine a day when you start a new project by typing: “Create a REST API for a todo app with user authentication and SQLite support.”

And within seconds, your project structure, database, and routes are ready and reviewed by your AI assistant, tested, and documented.

That’s the vision OpenCode is moving toward: AI tools that don’t just generate code, but understand context, handle complexity, and let humans stay in control.

Conclusion

OpenCode represents a turning point in how developers interact with AI. It’s open-source, private, and deeply integrated into the terminal, making it both powerful and flexible. While other assistants are built around specific IDEs or cloud services, OpenCode keeps developers at the center.

With its growing community, smart architecture, and commitment to privacy, OpenCode isn’t just another coding assistant. It’s a glimpse into the future of AI-driven software development, a future where your terminal becomes the smartest part of your workflow.

Hope you enjoyed this article. Signup for my free AI newsletter TuringTalks.ai for more hands-on tutorials on AI. You can also find visit my website.

In this tutorial, you’ll to learn what the GitHub CLI is, how to install and set it up, and how to use it for everyday tasks such as creating repositories, managing issues and pull requests, working with GitHub Actions, and automating tasks using custom aliases. You’ll learn how to replace some functionalities on GitHub’s web interface with quick commands in your terminal.

Here’s what we’ll cover:

• Overview of GitHub CLI

• Key Features

• Benefits of Using GitHub CLI

• Installation and Setup

• Authenticating with a GitHub Account

• Navigating the GitHub CLI

• Managing Repositories

• Working with Pull Requests and Issues

• Pushing and Pulling Changes

• Working with GitHub Actions

• Managing Gists

• Releases and Tags

• Custom Scripts and Aliases

• Troubleshooting

• Conclusion

Overview of GitHub CLI

You can use the GitHub CLI to bridge the gap between GitHub's web interface and your local environment. You can perform various tasks such as creating issues, managing repositories, or even checking the status of your GitHub Actions workflows using the CLI. Using the CLI, you can perform almost all the tasks that you might complete on the GitHub website.

Key Features of GitHub CLI

• Repository management: Easily create, clone, view, and manage repositories.

• Pull requests and issues: Manage pull requests and issues directly from the terminal, including creating, merging, and listing them.

• GitHub Actions: Interact with workflows and manage workflow runs.

• Authentication: Provides a secure way to authenticate with your GitHub account, supporting SSH keys, tokens, and OAuth.

• Custom scripting: Lets you create custom scripts and aliases to automate repetitive tasks and streamline development processes.

Benefits of Using GitHub CLI

Suppose you’re working on a project, and you need to create a new issue on GitHub. Normally, you would switch to your browser, log in to GitHub, navigate to the repository, click on the “Issues” tab, and then click “New Issue.” With GitHub CLI, you can do all of this by typing a single command, without ever leaving your terminal. This makes your workflow faster and saves time.

Installation and Setup

To install GitHub CLI on Windows, you can use the winget package manager. Winget is a command-line tool that allows you to install software easily.

Installing GitHub CLI on Windows, macOS, and Linux

Windows:

Run the command given below:

winget install --id GitHub.cli

• winget install: Tells Windows to install a new software package.

• --id GitHub.cli: Specifies the exact package ID for GitHub CLI.

After running this command, GitHub CLI will be installed on your Windows system.

macOS:

You can use Homebrew to install GitHub CLI on macOS. Open your terminal and run:

brew install gh

Linux:

On Linux, you can use your package manager. For example, on Ubuntu, you can run:

sudo apt install gh

Authenticating with a GitHub Account

After installing GitHub CLI, the next step is to authenticate it with your GitHub account.

Run Authentication Command:

Type gh auth login in the terminal and press Enter.

gh auth login

You’ll then be prompted to select an authentication method. The recommended option is to authenticate via a web browser.

If you select the browser method, GitHub CLI will open a link in your default browser, where you can log in to GitHub.

Complete Authentication:

After logging in, the browser will confirm that the GitHub CLI is connected to your account.

You can verify the authentication status by running:

gh auth status

Navigating the GitHub CLI

The GitHub CLI is easy to navigate, and its command structure is intuitive.

Command Structure and Syntax

GitHub CLI commands follow a simple and straightforward pattern:

gh [command] [subcommand] [flags]

• Command: The main action you want to perform (for example, repo, issue, pr).

• Subcommand: A specific task within the command (for example, create, list, view).

• Flags: Optional parameters that modify the command's behavior (for example, --title, --body).

Commonly Used Commands and Flags

Here are some common GitHub CLI commands:

• Creating a repository: gh repo create

• Listing issues: gh issue list

• Creating a pull request: gh pr create

• Viewing a repository's details: gh repo view

To see all available commands and options, you can always run:

gh help

How to Manage Repositories with the GitHub CLI

Let’s go through examples of some of the commands you’ll use the most often.

Creating and Cloning Repositories

To create a new GitHub repository directly from the terminal, just use the following command:

gh repo create my-repo-name

To clone an existing repository, use the following command:

gh repo clone owner/repo-name

Managing Branches and Pull Requests

GitHub CLI allows you to handle issues and pull requests (PRs) without leaving the terminal.

Switching branches or creating pull requests is simple. To create a new branch:

git checkout -b new-branch-name

Then, to create a pull request:

gh pr create --title "Your PR Title" --body "Description of your PR"

Pushing and Pulling Changes

Push your changes to GitHub with this command:

git push origin branch-name

And pull the latest changes with:

git pull

Working with GitHub Actions

GitHub CLI also supports GitHub Actions, allowing you to manage workflows directly from your terminal.

You can manually trigger workflows using the following:

gh workflow run workflow-name

And you can monitor the status of workflows with:

gh run list

To see detailed logs of a workflow, run this:

gh run view run-id --log

Cloning and Forking Repositories

Cloning and forking are essential tasks when working on projects from other repositories.

To clone a repository, use this command:

gh repo clone <repository-name>

To fork a repository, do this:

gh repo fork <repository-url>

Example:

Here’s what it would look like:

gh repo clone example-repo

gh repo fork https://github.com/username/repository-name

How to Work with GitHub Actions

Using the GitHub CLI, you can also manage GitHub Actions, which are automated tasks you can run in response to certain events in your repository.

Triggering and Monitoring Workflows

You can trigger a workflow manually like this:

gh workflow run <workflow-name>

And you can monitor workflow runs with this:

gh run list

Managing Workflow Runs and Logs

If you want to check the details of a specific workflow run, you can view logs directly from the CLI:

gh run view <run-id> --log

You can also use GitHub CLI commands to enhance your Continuous Integration/Continuous Deployment (CI/CD) pipelines, ensuring smooth automation and better control over our workflows.

How to Update the GitHub CLI

To make sure that you’re using the latest version of GitHub CLI with all the latest features and fixes, you can update it using winget.

winget upgrade --id GitHub.cli

• winget upgrade: Checks for updates for the specified package.

• --id GitHub.cli: Identifies the GitHub CLI package for the upgrade.

Advanced GitHub CLI Features and Integrations

The GitHub CLI is not only useful for performing basic tasks. You can also perform some advanced operations with its help.

How to Manage Gists with the GitHub CLI

Gists are a simple way to share snippets of code. You can create, list, and manage your Gists right from the CLI. Here’s how you can create a gist:

gh gist create my-code-snippet.py

To list your gists:

gh gist list

Interacting with Releases and Tags

To manage releases and tags, GitHub CLI provides commands to create, list, and delete releases. Here’s an example of creating a release:

gh release create v1.0.0

How to Extend the GitHub CLI with Custom Scripts and Aliases

You can write your own scripts and integrate them into GitHub CLI, or create aliases for commands you use frequently to save time. Aliases let you create shortcuts for commands that you use often. For example, the command given below creates an alias prlist that will show all pull requests, regardless of their state:

gh alias set prlist "pr list --state all"

In the same manner, you can create a shortcut co to quickly check out a pull request branch without typing the full command each time. The command is given below:

gh alias set co "pr checkout"

Troubleshooting Common Issues

If you face any issues, you can troubleshoot by checking the command syntax, ensuring your GitHub CLI is up to date, or consulting the documentation using the command:

gh help <command>

Conclusion

GitHub CLI is an excellent tool that helps developers work directly from the terminal. It lets you manage repositories, handle pull requests and issues, trigger and monitor GitHub Actions, and even work with Gists.

You can save time and improve productivity as developers using this powerful tool. Keep exploring its new features and stay updated with the latest version.

In this article, we’ll go through a collection of CLI / TUI tools that have been widely adopted in the developer community, spanning various categories such as version control, system utilities, text editors, and more. I wanted to give you a diverse selection that caters to different needs and workflows.

For each tool, I’ll include an overview, highlighting its key features and use cases, along with clear and concise installation instructions for various operating systems, ensuring you can quickly get up and running with these valuable command-line companions.

Table of Contents

• Kubernetes Tools

• Container Tools

• File and Text Tools

• Git Tools

• Development Tools

• Networking Tools

• Workstation Tools

Kubernetes Tools

k9s — Kubernetes CLI To Manage Your Clusters In Style

K9s is a must-have tool for anyone working with Kubernetes. Its intuitive terminal-based UI, real-time monitoring capabilities, and powerful command options make it a standout in the world of Kubernetes management tools.

The K9s project is designed to continually watch Kubernetes cluster for changes and offer subsequent commands to interact with observed resources. This makes it easier to manage applications, especially in a complex, multi-cluster environment. The project’s aim is to make Kubernetes management more accessible and less daunting, especially for those who are not Kubernetes experts.

Just launch k9s in your terminal and start exploring the Kubernetes resources with ease.

To install K9s:

# via Homebrew for macOS
brew install derailed/k9s/k9s

# via snap for Linux
snap install k9s --devmode

# via Chocolatey for Windows
choco install k9s

# via go install
go install github.com/derailed/k9s@latest

kubectx — switch between contexts (clusters) on kubectl faster.

Kubectx is the most popular tool for switching Kubernetes contexts, but it has the fewest features! It displays all the contexts in your Kubernetes config as a selectable list and lets you pick one. That’s it!

This project comes with 2 tools:

• kubectx is a tool that helps you switch between contexts (clusters) on kubectl faster.

• kubens is a tool to switch between Kubernetes namespaces (and configure them for kubectl) easily.

These tools make it very easy to switch between Kubernetes clusters and namespaces if you work with many of them daily. Here you can see it in action:

To install kubectx:

# via Homebrew for macOS
brew install kubectx

# via apt for Debian
sudo apt install kubectx

# via pacman for Arch Linux
sudo pacman -S kubectx

# via Chocolatey for Windows
choco install kubens kubectx

kubescape — Kubernetes security platform for your IDE, CI/CD pipelines, and clusters.

I hope you take the security of your Kubernetes clusters seriously. If so, kubescape is really great for testing if your Kubernetes cluster is deployed securely according to multiple frameworks.

Kubescape can scan clusters, YAML files, and Helm charts and detects the misconfigurations according to multiple sources.

I usually use it in my CI/CD to scan for vulnerabilities automatically when changing Kubernetes manifests or Helm templates.

To install kubescape:

# via Homebrew for macOS
brew install kubescape

# via apt for Debian
sudo add-apt-repository ppa:kubescape/kubescape
sudo apt update
sudo apt install kubescape

# via Chocolatey for Windows
choco install kubescape

Container Tools

ctop — A top-like interface for container metrics.

ctop is basically a better version of docker stats. It provides a concise and condensed overview of real-time metrics for multiple containers. It comes with built-in support for Docker and runC, and connectors for other container and cluster systems are planned for future releases.

Using ctop is simple. Once you have the tool open, you’ll see all of your currently active containers listed.

To install ctop:

# via Homebrew for macOS
brew install ctop

# via pacman for Arch Linux
sudo pacman -S ctop

# via scoop for Windows
scoop install ctop

lazydocker — A simple terminal UI for both docker and docker-compose.

While Docker's command-line interface is powerful, sometimes you might want a more visual approach without the overhead of a full GUI. This is especially true when managing Docker containers on a headless Linux server where installing a web-based GUI might be undesirable.

Lazydocker was created by Jesse Duffield to help make managing docker containers a bit easier. Simply put, Lazydocker is a terminal UI (written in Golang) for the docker and docker-compose commands.

To install lazydocker:

# via Homebrew for macOS
brew install lazydocker

# via Chocolatey for Windows
choco install lazydocker

# via go install
go install github.com/jesseduffield/lazydocker@latest

dive — A tool for exploring each layer in a Docker image.

A Docker image is made up of layers, and with every layer you add on, more space will be taken up by the image. Therefore, the more layers in the image, the more space the image will require.

That’s where dive shines, it helps you explore your Docker image and layer contents. It can also help you find ways to shrink the size of your Docker/OCI image.

To install dive:

# via Homebrew for macOS
brew install dive

# via pacman for Arch Linux
pacman -S dive

# via go install
go get github.com/wagoodman/dive

File and Text Tools

jq — Command-line JSON processor.

You may be aware of this one already as it’s well known in the developer community.

Unfortunately, shells such as Bash can’t interpret and work with JSON directly. That’s where you can use jq as a command-line JSON processor that’s similar to sed, awk, grep, and so on for JSON data. It’s written in portable C and doesn’t have any runtime dependencies. This lets you slice, filter, map, and transform structured data with ease.

To install jq, you can download the latest releases from the GitHub release page.

bat — A cat(1) clone with wings.

This is the most used CLI on my machine currently. A few years ago it was cat, which is great but doesn’t provide syntax highlighting, or Git integration

Bat’s syntax highlighting supports many programming and markup languages, helping you make your code more readable directly in the terminal. Git integration lets you see modifications in relation to the index, highlighting the lines you’ve added or changed.

Simply run bat filename and enjoy its output.

To install bat:

# via Homebrew for macOS
brew install bat

# via apt for Debian
sudo apt install bat

# via pacman for Arch Linux
pacman -S bat

# via Chocolatey for Windows
choco install bat

ripgrep — Recursively search directories for a regex pattern while respecting your gitignore.

ripgrep is definitely becoming a popular alternative (if not the most popular) to the grep command. Even some editors like Visual Studio Code are using ripgrep to power their search offerings.

The major selling point is its default behavior for recursive search and speed.

I now rarely use grep on my personal machine, as ripgrep is much faster.

To install ripgrep:

# via Homebrew for macOS
brew install ripgrep

# via apt for Debian
sudo apt-get install ripgrep

# via pacman for Arch Linux
pacman -S ripgrep

# via Chocolatey for Windows
choco install ripgrep

Git Tools

lazygit — Simple terminal UI for git commands.

lazygit is another great terminal UI for Git commands developed by Jesse Duffield using Go.

I don’t mind using the Git CLI directly for simple things, but it is famously verbose for more advanced use cases. I am just too lazy to memorize longer commands.

And lazigit has made me a more productive Git user than ever.

To install lazygit:

# via Homebrew for macOS
brew install jesseduffield/lazygit/lazygit

# via pacman for Arch Linux
pacman -S lazygit

# via scoop for Windows
scoop install lazygit

Development Tools

ATAC — A simple API client (Postman-like) in your terminal.

ATAC stands for Arguably a Terminal API Client. It’s based on popular clients like Postman, Insomnia, and Bruno, but it runs inside your terminal without needing any particular graphical environment.

It works best for developers who need an offline, cross-platform API client right at their fingertips (terminal).

To install ATAC:

# via Homebrew for macOS
brew tap julien-cpsn/atac
brew install atac

# via pacman for Arch Linux
pacman -S atac

k6 — A modern load testing tool, using Go and JavaScript.

I’ve used many load-testing tools in my career, such as vegeta or even ab in the past. But now I mostly use k6s as it has everything I need and has a great GUI and TUI.

Why it works well for me:

• k6 has really good documentation

• Many integrations available: Swagger, JMeter scripts, and so on.

• Results reporting is quite good

To install k6:

# via Homebrew for macOS
brew install k6

# via apt for Debian
sudo apt-get install k6

# via Chocolatey for Windows
choco install k6

httpie — modern, user-friendly command-line HTTP client for the API era.

Don’t get me wrong, curl is great, but not very human-friendly.

HTTPie has a simple and expressive syntax, supports JSON and form data, handles authentication and headers, and displays colorized and formatted output.

To install httpie:

# via Homebrew for macOS
brew install httpie

# via apt for Debian
sudo apt install httpie

# via pacman for Arch Linux
pacman -Syu httpie

# via Chocolatey for Windows
choco install httpie

asciinema — Terminal session recorder.

I call it a terminal YouTube :)

asciinema is a great tool when you want to share your terminal sessions with someone else, instead of recording heavy videos.

I use it often when I develop some CLI tools and want to share the demo of how they work (on GitHub, for example).

To install asciinema:

# via Homebrew for macOS
brew install asciinema

# via apt for Debian
sudo apt install asciinema

# via pacman for Arch Linux
sudo pacman -S asciinema

Networking

doggo — A command-line DNS client.

It's totally inspired by dog which is written in Rust.

In the past I would use dig to inspect the DNS, but its output is often verbose and difficult to parse visually.

doggo addresses these shortcomings by offering two key improvements:

• doggo provides the JSON output support for easy scripting and parsing.

• doggo offers a human-readable output format that uses color-coding and a tabular layout to present DNS information clearly and concisely.

To install doggo:

# via Homebrew for macOS
brew install doggo

# via scoop for Windows
scoop install doggo

# via go install
go install github.com/mr-karan/doggo/cmd/doggo@latest

gping — Ping, but with a graph.

The well-known ping command is not the most interesting to look at, and interpreting its output in a useful way can be difficult.

gping gives a plot of the ping latency to a host, and the most useful feature is the ability to run concurrent pings to multiple hosts and plot all of them on the same graph.

To install gping:

# via Homebrew for macOS
brew install gping

# via Chocolatey for Windows
choco install gping

# via apt for Debian
apt install gping

Workstation

tmux — A terminal multiplexer.

Why is tmux such a big deal?

You may have run into situations where you need to view multiple terminal consoles at the same time. For example, you may have a few servers running (for example, web, database, debugger) and you might want to monitor all the output coming from these servers in real-time to validate behavior or run commands.

Before tmux, you might have just opened a few different tabs in the terminal and switched between them to see the output.

Thankfully, there’s an easier way — tmux.

In a nutshell, here are some of its most popular features:

• Window/Pane management

• Session management with persistence

• Sharable sessions with other users

• Scriptable configurations

To install tmux:

# via Homebrew for macOS
brew install tmux

# via apt for Debian
apt install tmux

# via pacman for Arch Linux
pacman -S tmux

zellij — A terminal workspace with batteries included.

Since I listed tmux here, it also makes sense to include a new competitor, Zellij, which has been gaining traction in the developer community. Both have their own unique features and purposes.

Compared to traditional terminal multiplexers, zellij offers a more user-friendly interface, modern design elements, built-in layout systems, and a plugin system, making it easier for newcomers to get started.

I still like tmux. It has a special place in my heart because it has served a great purpose for years. But zellij is another good option.

To install zellij:

# via Homebrew for macOS
brew install zellij

# via apt for Debian
apt install zellij

# via pacman for Arch Linux
pacman -S zellij

btop — A monitor of resources.

I can’t live without btop, and it’s installed on all my machines via my personal dotfiles. I rarely use now built-in OS GUIs to check the resource utilization on my host machine, because btop can do it much better.

I use to to quickly explore what uses the most memory, monitor and kill some processes, and more.

To install btop:

# via Homebrew for macOS
brew install btop

# via snap for Debian
sudo snap install btop

Conclusion

These CLIs/TUIs should work well in any modern terminal. I personally use Ghostty currently and it works great, but other popular options like iTerm2, Kitty, and the default terminal applications on macOS and Linux should also provide a seamless experience. The key is to ensure your terminal supports features like 256-color palettes and UTF-8 encoding for optimal display of these tools.

There’s a huge amount of CLIs/TUIs out there, and I couldn’t list them all (though I tried to list some of the best). This selection represents a starting point for exploring the rich ecosystem of command-line tools available to developers. I encourage you to explore further, discover new tools that fit your specific needs, and contribute back to the community by sharing your findings.

Explore more articles on packagemain.tech

Two of these powerful features are piping and redirection. These features enable you to redirect the output and input of commands to and from other commands and files.

In this article, you'll learn what piping and redirection in Linux mean, followed by a deep dive into how to use these features yourself.

Prerequisites

To fully understand this guide, you'll need to at least have:

• Basic understanding of the Linux operating system.
• Basic experience working with the Linux command line.
• Access to a Linux command line to try out the commands.

Take a look at this Linux command line tutorial if you're new or need a refresher.

What is Piping in Linux?

Before we dive into the how, what does piping even mean? Piping is the act of directing the output of one Linux command as an input to another command. You can direct the standard output or standard error from one command to another using piping.

A simple example of piping is when you take the output of one command and use it as an input to another command. The pipe (|) metacharacter is used to achieve this.

If you're new to the concept of metacharacters, this is just a fancy name for characters with special meaning in the command line. There are other metacharacters in Linux aside from pipe (|). Common examples include less than (<), greater than (>), and ampersand (&), to name a few.

The Basics of Piping

The basic syntax of using the pipe command is as follows:

command1 | command2 | command3 | ... | commandN

In the syntax above, the terminal will execute the commands from left to right. It will start with command1, and then the output will be input to command2. Outputs of command2 will then be used as inputs of command3 and so on. The good thing about piping is that you can chain as many commands as you'd like.

Piping Examples

Below are several examples of using the pipe command to perform various tasks.

1. Counting the number of files and directories

ls -l | wc -l

In the example above, the first section lists all the files and directories in the current directory using the ls command. The additional -l option tells the ls command to list the contents in a long list format.

The output of the ls -l command is then piped to the second section. The wc -l command counts the number of lines from the output of the ls -l command in the first section and prints the result to the terminal.

2. Sorting a list of files and directories

ls | sort

In the command above, the ls command will output the lists of the files and directories in the current directory. The list is then piped to the sort command, which sorts them alphabetically and prints the result to the terminal.

3. Sort and display unique words from a file

cat words.txt | sort -r | uniq

The third example includes three separate commands connected by two pipes. The first command outputs the contents of the words.txt file, which contains a list of words.

The output is piped to the sort -r command, which sorts the words in reverse alphabetical order. Finally, the sorted words are piped to the uniq command, which removes duplicates and outputs the unique sorted words.

Why and When Should You Apply Command Piping Linux?

Piping takes more than just knowing the syntax. The syntax is pretty simple. To effectively use piping, you should understand its essence.

The purpose of piping is to help you chain commands, using the output of one as the input of another.

Piping is not meant to be a way to chain unrelated commands that you want to be executed sequentially. If you need to do that, write your commands in the terminal and separate them with semicolon (;) using the following syntax:

command1 ; command2 ; ... ; commandN

What Is Redirection in Linux?

Redirection is the act of dictating where the inputs or outputs of your commands go. By default, commands receive data from standard input and then output the results in standard output.

One of the main areas where redirection proves useful is when working with commands and files. You can, for example, redirect the output of a command to a file instead of printing the output in the terminal. Alternatively, you can declare a certain file as an input to a command.

Like piping, Linux provides special characters to perform redirection. Here are important file-redirection characters in Linux and what they do:

• > – directs the output of a command to a given file.
• < – directs the contents of a given file to a command.
• >> – directs the output of a command to a given file. Appends the output if the file exists and has content.
• 2> – directs error messages from a command to a given file.
• 2>> – directs an error message from a command to a given file. Appends the error message if the file exists and has content.
• &> – directs standard output and error to a given file.
• &>> – directs standard output and error to a given file. Appends to the file if it exists and has contents.

Let's look at each file-redirection character in-depth.

1. Output Redirection With >

The > symbol enables you to redirect the output of a command to a certain file. Using the symbol, you can redirect the output to any existing file. If it doesn't exist, the output redirection character will automatically create a new one.

However, you should be careful when writing to an existing file because its contents will be overwritten without a warning.

You can perform output redirection using the following syntax:

command > file

The output of running command will be written to the file instead of the standard output (or, in other words, printed to the terminal). Here's an example of how you can do output redirection:

ls -a > contents.txt

In the command above, the list of items in the current directory (including dotfiles, directories, and files) will be written to the contents.txt file. You won't see any output in the terminal due to the redirection.

2. Output Redirection With >>

>> lets you redirect the output of a command to a file. But, unlike using a single greater than character (>), >> will append the output if you try to write to an existing file (instead of overwriting its contents). If the file doesn't exist, it will create a new one.

The syntax is as follows:

command >> file

Here's an example of using output redirection with >> to perform the same action as before:

ls -a >> contents.txt

3. Input Redirection With <

The < character in the command line lets you redirect the input to a command from a file instead of the keyboard. The syntax of input redirection using < is as follows:

command < file

Here's an example of using input redirection:

wc -w < files.txt

In the example above, we're passing files.txt as the input to the wc -w command, which counts the number of words in the file. But you don't need to use the input redirection character in many scenarios because it's the default behavior. For instance, the command above is similar to the following:

wc -w files.txt

4. Error Redirection With 2> and 2>>

When working on the command line, you may run into errors. For instance, if you want to execute a file without appropriate permissions. Instead of letting the terminal throw an error by printing it, you can use the error redirection character to dictate where the error message should go.

A good place to redirect errors is a file dedicated to storing errors. Here's a simple example of a command trying to access the list of files in a directory that doesn't exist:

ls nonexistent 2> error.txt

In the example above, the terminal will throw an error since there's no file named nonexistent. But instead of printing it to the console, it will be stored in the error.txt file due to the error redirection character. However, the file will have nothing if there's no error.

If you must append an error to an existing file instead of overwriting its contents, use 2>> instead of 2>.

5. Output and Error Redirection With &> and &>>

Instead of choosing whether to redirect standard output or standard errors to a file, &> allows you to redirect both simultaneously. You can think of &> as a shorthand for combining the output redirection (>) and error redirection (2>) characters.

The syntax of using &> is as follows:

command &> output.txt

The error or output of the command is written to the output.txt file. Here's an example:

ls sample &> output.txt

In the command above, if the sample directory is not available or is available but you don't have read permissions, an appropriate error will be written in the output.txt file. But if it exists and you have read permissions, its contents will be output in the output.txt file.

Using &>> allows you to append the output to the file if it exists and has contents.

How to Combine Piping and Redirection to Unleash the Power of the Command Line

Using piping and redirection in combination can let you perform complex operations without heavy lifting. By learning how to combine the two, you can create complex commands to perform various actions with less typing.

Take the following command as an example:

ls | grep ".txt" > text_files.txt

The ls command lists the contents of the current directory. The output is piped to the grep command to filter text files. Finally, the output of the grep command is redirected to the text_files.txt file.

This simple yet powerful example shows that when it comes to piping and redirection, the only limit is your mind.

Conclusion

In this tutorial, you learned the basics of piping and redirection in Linux. We covered the basics together with examples of how you can use the two.

The two features in isolation can be powerful, but you can go further by combining them in your commands, as shown in the last section.

We will cover the following:

• find

• xargs and nproc

• taskset

• numactl

• watch

• inotify-tools

I will present you with a challenge and the tools demonstrating how to solve each problem.

What You'll Need:

• A Linux distribution

• Curiosity

How to Handle Directories with Many Files

You may have encountered this problem before: you tried to do a ls on a directory with a very large number of files, but the command threw an 'argument list too long' error:

josevnz@orangepi5:/data/test_xargs$ ls *
-bash: /usr/bin/ls: Argument list too long

This is because POSIX-compatible systems have a limit for the maximum number of bytes you can pass as an argument:

[josevnz@dmaf5 Documents]$ getconf ARG_MAX
2097152

2 Million bytes may seem like a lot – or not enough depending on whom you ask. But it's also a protection against attacks or innocent mistakes with bad consequences.

In any case, how can you bypass this limitation? Well, there are many ways to do so.

Using Shell built-in

Bash built-in doesn't have the ARG_MAX limitation:

josevnz@orangepi5:/data/test_xargs$ echo *|ls
...
test_file055554  test_file111110  test_file166666  test_file222222  test_file277778  test_file333334  test_file388890  test_file444446
test_file055555  test_file111111  test_file166667  test_file222223  test_file277779  test_file333335  test_file388891  test_file444447
test_file055556  test_file111112  test_file166668  test_file222224  test_file277780  test_file333336  test_file388892  test_file444448

This is probably the simplest solution, but let's see another way.

Using find when you want formatting options

Or you can use this well known find flag:

find /data/test_xargs -type f -ls -printf '%name'

Or with formatting, to mimic ls:

find /data/test_xargs -type f -printf '%f\n

This is fast and also the most complete solution. But before moving on I'll show you yet another way.

Using xargs

The following works:

find /data/test_xargs -type f -print0 | xargs -0 ls

But it's inefficient, as you are forking 3 processes to display the contents of the directory. And on top of that, xargs is throttling how many files will be passed to the ls command.

Let's move on and check out a different problem.

How to Run More Programs Without Crashing the Server

First you walk then you run: Do it serially

So say that you want to compress all the files on the given directory from our previous example. A first try would be like this:

gzip *

Which will take a long time as gzip will process one file at the time.

You might think to do something like this to compress files in parallel:

josevnz@orangepi5:/data/test_xargs$ for file in $(ls data/test_xargs/*); do gzip $file &; done
-bash: /usr/bin/ls: Argument list too long

Again, ARG_MAX strikes again.

We know xargs or find now, so what if we do this:

for file in $(find $PWD); do echo gzip $file &; done
wait
echo "All files compressed?"

That will either make your server run out of memory or crush it under very heavy CPU utilization because you are forking a gzip instance for every file found.

Our first attempt at parallelism and throttling (the art of self control)

What you need is a way to throttle your compression requests, so you don't launch more processes than the number of CPUS you have.

Let's try that again with find and xargs:

find /data/test_xargs -type f -print0| xargs -0 -P $(($(nproc)-1)) -I % gzip %

Oh. That looks like a fancy one-liner. Let me explain how it works:

1. Use find to get all files on the given directory, use the null character as a separator to be able to process weird named ones.

2. nproc will tell you how many CPUS you have, then subtract 1 using Bash arithmetic like this using sub-shells: $(($(nproc)-1))

3. Finally, xargs will run no more than -P processes (In my case 8 CPUS - 1 = 7 jobs), replacing the '%' with the name of the file to compress

Note: There are other ways to get the number of CPUS on the machine, like parsing /proc/cpuinfo. There are other more efficient compression out there but gzip is available on pretty much any Linux/ Unix out there.

OK, time to see our next problem.

CPU Affinity with taskset to Maximize Execution Time

Despite limiting the number of CPUs, some intensive jobs can slow down other processes on your machine when looking for resources. There are a few things you can do to keep the performance of the server under control, like using taskset:

The taskset command is used to set or retrieve the CPU affinity
of a running process given its pid, or to launch a new command
with a given CPU affinity. CPU affinity is a scheduler property
that "bonds" a process to a given set of CPUs on the system.

In general, we always want to leave one of the CPUS 'free' for operating system tasks. The Kernel is normally pretty good keeping running processes glued to a specific CPU to avoid context switching, but if you want to enforce on which CPUS your process will run you can use tasket

taskset -c 1,2,3,4,5,6,7 find /data/test_xargs -type f -print0| xargs -0 -P $(($(nproc)-1)) -I % gzip %

taskset the only game in town? not so numactl fast!

What is NUMA and why you should care?

There are physical limitations to hardware that are encountered when many CPUs and lots of memory are required. The important limitation is that there is limited communication bandwidth between the CPUs and the memory.

One architecture modification that was introduced to address this is Non-Uniform Memory Access (NUMA).

So most simple desktop machines only have a single NUMA node, like mine:

[josevnz@dmaf5 ~]$ numactl --hardware
available: 1 nodes (0)
node 0 cpus: 0 1 2 3 4 5 6 7
node 0 size: 15679 MB
node 0 free: 5083 MB
node distances:
node   0 
  0:  10
# Or with lscpu
[josevnz@dmaf5 ~]$ lscpu |rg NUMA
NUMA node(s):                    1
NUMA node0 CPU(s):               0-7

If you have more than one NUMA node, you may want to 'pin' or set the affinity of your program to use the CPUS and memory of the same node.

For example, on a machine with 16 cores, 0-7 on node 0, 8-15 on node 1, we could force our compression program to run on all the CPUS on node 1, and use the memory of node 1 like this:

numactl --physcpubind 8-15 --membind=1 find /data/test_xargs -type f -print0| xargs -0 -P $(($(nproc)-1)) -I % gzip %

Keeping an Eye on Things

Just watch what I do

The watch command allows you to periodically run a command, and even show you the differences before calls:

Every 10.0s: ls                                                                                                         orangepi5: Wed May 24 22:46:33 2023

test_file000001.gz
test_file000002.gz
test_file000003.gz
test_file000004.gz
test_file000005.gz
test_file000006.gz
test_file000007.gz
test_file000008.gz
test_file000009.gz
test_file000010.gz
...

Shows me the output of the ls command every 10 seconds. To detect changes on a directory this is simple, but not easy to automate and definitely not efficient.

Wouldn't be nice if the kernel was able to tall me about changes on my directories?

A better way to know about changes on the filesystem, with inotify-tools

You may need to install this separately, but it should be easy to do. On Ubuntu:

sudo apt-get install inotify-tools

On Fedora:

sudo dnf install -y inotify-tools

So how we can monitor for events on a given directory?

On one terminal we can run inotifywait:

josevnz@orangepi5:/data/test_xargs$ inotifywait --recursive /data/test_xargs/
Setting up watches.  Beware: since -r was given, this may take a while!
Watches established.

And on another terminal we can touch some files to simulate an event:

josevnz@orangepi5:/data/test_xargs$ pwd
/data/test_xargs
josevnz@orangepi5:/data/test_xargs$ touch test_file285707.gz test_file357136.gz test_file428565.gz

The original terminal will get the first event and exit:

Watches established.
/data/test_xargs/ OPEN test_file285707.gz

To make it listen for even forever we do this:

josevnz@orangepi5:/data/test_xargs$ inotifywait --recursive --monitor /data/test_xargs/

If we touch the file again on a separate terminal then this time we will see all the events:

Setting up watches.  Beware: since -r was given, this may take a while!
Watches established.
/data/test_xargs/ OPEN test_file285707.gz
/data/test_xargs/ ATTRIB test_file285707.gz
/data/test_xargs/ CLOSE_WRITE,CLOSE test_file285707.gz
/data/test_xargs/ OPEN test_file357136.gz
/data/test_xargs/ ATTRIB test_file357136.gz
/data/test_xargs/ CLOSE_WRITE,CLOSE test_file357136.gz
/data/test_xargs/ OPEN test_file428565.gz
/data/test_xargs/ ATTRIB test_file428565.gz
/data/test_xargs/ CLOSE_WRITE,CLOSE test_file428565.gz

This is less taxing to the operating system than asking for directory changes every time, and filtering just the differences ourselves.

What's Next

There is so much more to explore. The tips above introduced you to some important concepts, so why not to learn much more about them?

• The Ubuntu forum has a great conversation about xargs, find, ulimit and other things. Knowledge is power.

• RedHat as a nice page about NUMA, taskset, interrupt handling. If you are serious about fine-tuning the performance of your processes, please read it.

• You liked inotify and want to use it from your Python script. Then take a look at pynotify.

• Find may be intimidating, but this tutorial will make it easier to understand.

• Source code for this tutorial can be found here.

This article provides an overview of WP-CLI, focusing on its capabilities to create, modify, and delete users, as well as handle plugins with ease. This article will show you how WP-CLI can significantly elevate your WordPress management experience.

What is WP-CLI?

WP-CLI is a command-line tool designed for managing WordPress installations. It empowers developers, administrators, and site owners to interact with their websites directly from the command line, bypassing the need for manual interventions through the web interface.

It's built on PHP, and offers a wide range of commands that you can execute right from the terminal.

By using WP-CLI, you'll be able to manage your WordPress sites much more efficiently. Let's see a few examples of how WP-CLI commands can simplify your workflow:

WP-CLI Commands

Site Information Retrieval

The wp site info command provides a quick overview of your WordPress site's important details, including the site's URL, the number of posts and pages, the active theme, and more.

For instance, by running wp site info, you can promptly gather essential information about your site without navigating through the WordPress admin dashboard.

Database Management

WP-CLI allows you to manage your WordPress database seamlessly. Use the wp db export command to create a database export file, ensuring a backup of your site's data.

If you need to import data, the wp db import command facilitates this process. For example, if you have a database backup named backup.sql, executing wp db import backup.sql restores the database to a previous state.

Theme Manipulation

Manipulating themes is extremely efficient with WP-CLI. For example, the wp theme install command lets you install a theme directly from the official WordPress theme repository. To install the "Twenty Twenty-One" theme, you can use the command wp theme install twentytwentyone.

Post and Page Creation

Generating new content is made easier using WP-CLI. The wp post create and wp post generate commands enable you to create and populate posts and pages with content.

For example, wp post create --post_type=post --post_title="New Post" creates a new post with the specified title.

These examples illustrate the versatility and power of WP-CLI in managing various aspects of your WordPress site. By harnessing its capabilities, you can enhance your efficiency, reduce manual tasks, and gain greater control over your website's management.

How to Install WP-CLI

Before diving into more features of WP-CLI, let's understand the installation process.

You can install WP-CLI globally on your system, making it accessible from any directory.

To install WP-CLI, make sure you have PHP installed, along with a compatible version of WordPress. Download the Phar archive, place it in a directory reachable through your system's PATH, and you're ready to go.

You can verify the installation by typing wp --info in your terminal. Here you can find the doc with the URL to download WP-CLI with a wget.

How to Manage Users with WP-CLI

Managing users is a fundamental task when overseeing a WordPress site. WP-CLI simplifies user management with various commands that make creating, modifying, and deleting users much easier.

How to Create Users

The wp user create command lets you quickly create users directly from the command line.

To illustrate, let's create a new user named "Alice" with the email address "alice@example.com" and the role of editor. Simply enter:

wp user create alice alice@example.com --role=editor

How to Modify Users

WP-CLI also streamlines user modifications. Use the wp user update command to adjust a user's details.

For instance, let's change Alice's display name to "Alice Johnson" using the following command:

wp user update 123 --display_name="Alice Johnson"

In this example, "123" is Alice's ID.

How to Delete Users

When user accounts become obsolete or require removal for security reasons, WP-CLI simplifies the process.

To delete a user, use the wp user delete command. To remove Alice's account, simply execute:

wp user delete 123 --reassign=567

In this case, again "123" is Alice's ID and "567" is the ID of the user you want to assign Alice's content to (for example posts, pages, and so on).

How to Manage Plugins with WP-CLI

Plugins play a crucial role in enhancing WordPress websites. WP-CLI extends its capabilities to manage plugins, making tasks such as installation, activation, deactivation, and updates incredibly efficient.

How to Install Plugins

Use the wp plugin install command to seamlessly install plugins from the WordPress repository.

For instance, let's install the "Akismet" anti-spam plugin:

wp plugin install akismet

How to Activate and Deactivate Plugins

Managing plugin status is quite easy with WP-CLI. Activate or deactivate plugins using the wp plugin activate and wp plugin deactivate commands respectively.

To activate the "Akismet" plugin, type the following command:

wp plugin activate akismet

How to Update Plugins

Keeping plugins up to date is vital for security and performance. The wp plugin update command makes updates hassle-free.

To update all installed plugins, simply run:

wp plugin update --all

How to List Installed Plugins

WP-CLI offers an overview of installed plugins with the wp plugin list command. This provides a quick snapshot of each plugin's status, version, and available updates:

wp plugin list

Conclusion

WP-CLI is an invaluable asset in the world of WordPress management. Its extensive command set helps you manage users and plugins – and much more – with remarkable ease. This saves you time and minimizes manual interventions.

By harnessing the power of WP-CLI, administrators and developers can streamline workflows, improve security, and ensure their WordPress websites operate seamlessly.

A computer is basically a piece of electronic circuitry that performs tasks as instructed by its users.

But for a human to interact with this hardware, they must really know and understand how it works. The person must also know the order in which to give the computer various tasks to produce a meaningful result.

But most of us don't know these things. What happened?

In this article, we're going to look at:

• The history of early computing

• How operating systems were developed

• Modern operating systems

• The development of kernels and shells

• The history of shells

• Why CLI tools are still important

The Early Days of Computers

In the 1800's, computers were mostly used to work on large amounts of numerical data. They were basically programmable calculators the size of small factories.

The data they worked on were also very physical. They were manually fed as punched cards into the machine's card readers. The computers were then programmed by physically rewiring cables, plugboards, and switches to determine what operations would be carried out on the input data. Some of the computers even processed logic through partially mechanical means.

Plugboards allowed you write (or wire) a program and store it so when you needed that program, you'd remove the current plugboard and install a new one. The output of the programs were printed out by line printers, saved on tapes, or punched on cards.

These programming and memory technologies were used in many forms of technological designs.

For example, the Enigma machine which was used to scramble letters and encipher top secret military and diplomatic communication had a plugboard you would rewire to change settings.

The IBM International Daily Dial Attendance Recorder also counted and recorded staff attendance on punched cards. These were more special-purpose equipment.

Early Operating Systems

The first-ever speech synthesis was done in 1962 on an IBM 704, a computer without an operating system.

At the time of the first computers, nobody ever thought of operating systems. People wrote their programs in machine language (or wired them on plugboards) and ran them. There was no way to run two programs at the same time or perform error detection/correction. Your program ran until it either crashed or completed.

IBM 704 computer with two operators in 1957

Initially, computers didn't ship with any software. Later on, IBM included FORTRAN and COBOL compilers in their mainframes. Then came 'resident monitors,' the precursor to operating systems. They literally got the name because they were software that resided in the company's memory and monitored tasks. They basically performed tasks like cleaning up after programs and helping to sequence jobs on the computers.

The first operating systems were made by computer owners who were tired of under-utilizing their computers. They didn't want to have to wait for a program to complete before they manually loaded in another set of data and programs.

One of these early operating systems was the Input/Output System of General Motors and North American Aviation (GM-NAA I/O). Its main aim was to execute the next program automatically after the current one was finished (batch processing). It was created in 1956, and is known as the first-ever working operating system.

IBM later modified one of its customer's operating systems and distributed it as IBSYS. Then, IBM thought that it wasn't good for different computers to have different (machine language) instruction sets, so they stopped all production and started a new line called System/360.

With this, they aimed to build just one operating system for all their computers. It didn't quite go as planned, so they ended up with a family of operating systems, OS/360.

OS/360 included features like time-sharing, error detection/correction, and device management, which are all features implemented in modern operating systems. The OS/360 was introduced in 1964 and was in use until 1977.

Early time-sharing technologies enabled multiple people to access the same mainframe from their different terminals at once. The first computer terminals were teletypewriters (or teletypes). Teletypes were developed around the 1830's but weren't used as computer terminals until the 1970's.

Modern Operating Systems

The MULTiplexed Information and Computing Service (MULTICS) operating system was initially developed for General Electric's 645 mainframe, and later Honeywell's mainframes when they bought over GE's computer division.

It was jointly developed in 1964 by GE, MIT, and Bell Labs, mainly as a successor to the Compatible Time-Sharing System (CTSS).

It came with a lot of new ideas to the computing world like access control, dynamic linking, support for ASCII, and dynamic reconfiguration.

Dynamic reconfiguration allowed the operators to remove and add memories and CPUs without logging out or disrupting users. With the advent of the Compatible Time-Sharing System (CTSS), computers were looked at as a utility.

The idea was that computers were too big and expensive and could be used by only one person at a time. But what if that expensive hardware could be used more efficiently by allowing more than one person to use them at the same time? That way, the computer could become a public utility, accessed from home with terminals.

With a public utility, you need to be able to carry out maintenance on different components without disrupting the service. That was the usefulness of dynamic reconfiguration.

That idea of computers becoming a public utility seems to have lived on with the use of servers, and now, cloud computing.

At a point, MULTICS didn't work out and the companies went their separate ways. Ken Thompson, one of the programmers, later described MULTICS as 'over-designed,' 'overbuilt,' and close to unusable. In an interview with Peter Seibel, he said that the things he liked enough to take from it were the hierarchical filesystem and the shell.

He later went on to create the UNIX operating system with Dennis Ritchie.

The Development of Kernels and Shells

One of the major features of the UNIX operating system was the splitting of the operating system into shell and kernel. The OS kernel was then meant to handle all the system calls, while the shell was used to access the system's services without exposing the inner workings of the operating system.

This concept was first introduced by French computer scientist Louis Pouzin in 1965. He also coined the term shell. This is how he described it in a document titled, The SHELL: A Global Tool for Calling and Chaining Procedures in the System.

We may envision a common procedure called automatically by the supervisor whenever a user types in some message at his console . . . we shall refer to that procedure as the "SHELL".

Louis Pouzin's Document on Shells

According to the document, when the user types in a command, the supervisor initiates a new stack to save the command, then calls the shell with the command as an argument.

He also envisioned a future where people created their own shells and used them without having to meddle with the underlying structure of the shell.

"An important facility is that the SHELL being itself a common procedure may be replaced by a private one supplied by the user. On that way, not only a particular procedure can be replaced on user's choice, but all conventions about typing commands may be tailor-made to user's wishes just by providing his own SHELL."

This idea was first implemented in MULTICS, and is one of the only things Ken Thompson admitted to liking enough to take from the project.

Now, in modern operating systems, the shell is the software that lets you communicate with the operating system, and access the OS services which are things like program execution, file management, and I/O management. It could be a command-line shell (CLI) or it could be a graphical user interface (GUI).

The kernel is the part that handles all the system calls, manages computer resources like memory, CPU, and storage, and handles timesharing between processes.

History of CLI Shells

Shells as we know them now really began with UNIX, and one of the first was the Thompson shell.

Thompson Shell (1971)

Ken Thompson, of course, went on to create his own shell. It was a simple command interpreter, not designed for scripting. It introduced the < > symbols for redirecting the input or output of a command, and the | symbol for piping.

C Shell (1978)

The C Shell (or csh) was created by Bill Joy and enabled scripting. The objective was to create a command language that looked more like the C programming language.

Bourne Shell (1979)

Stephen Bourne started work on the Bourne Shell in 1976 as a replacement for the Thompson Shell. It was intended as a scripting language. One of the major goals was to enable using scripts as filters.

Korn Shell (1983)

Developed by David Korn, the Korn Shell was based on source code from the Bourne Shell, and attempted to integrate the features of C Shell and Bourne Shell. It is POSIX.2 compliant.

Bourne Again SHell aka BASH (1989)

BASH was written by Brian Fox in 1989 and released under the GNU license as a free and open-source version of the Bourne Shell.

It was one of the first programs Linus Torvalds ported to Linux and is the default shell for most Linux distributions. It was built for scripting and is POSIX compliant.

Z Shell (1990)

The Z Shell (or Zsh) was created by Paul Falstad as an extended and improved Bourne Shell. The name was derived from the name of a teaching assistant at Paul's university named Zhong Shao.

It also enables scripting and is aesthetically superior to Bash. It supports plugins and extensions, auto-completion, and some other globing capabilities that are unavailable in Bash.

There are many other popular shells used today, but I'm not going to talk about them.

All these shells mentioned did not apply to Windows (until recently). Windows had two command-line shells: Command Prompt (cmd) and PowerShell. The PowerShell was created as an extension to Command Prompt. It supports cmdlets, scripting, piping, and many more features.

Then in 2019, that changed with Windows Terminal, a new terminal emulator for Windows that can run Bash on what is called Windows Subsystem for Linux (WSL). It also supports Command Prompt, PowerShell, and Azure Cloud shell out-of-the-box.

Why Are These CLI Tools Important When We Have Graphical User Interfaces (GUIs)?

You may wonder why we would take the pains to go through this whole evolution and yet still love to use CLI tools and shells. What's so interesting about typing commands into terminals?

The first and most compelling reason is that CLI tools are lightweight because they're text-based. In cases where you have servers and other devices where you need to optimize the use of resources, it's not very wise to use up most of the resources to run GUI interfaces.

Using the CLI can also provide you with a great deal of flexibility and speed that you won't get with a GUI. For example, you may want to search for all the images in a folder with 1000 files and rename them in a certain order. Doing that with your GUI will be time-consuming and maybe frustrating. With the CLI, you can just type a few commands.

Scripts are another important feature. Sometimes, there are tasks you want to carry out multiple times and you don't want to have to navigate through menus all the time or even type commands multiple times. You can write scripts that you run to automatically carry out the tasks repetitively.

In cases where you have to access devices remotely, like web servers or network devices, the CLI is also the most favoured method. Also, for some tasks, it's just easier to do them with a few commands than using your GUI—updating your apps on Linux for example.

Wrapping Up

We've really come a long way from purely mechanical computers to purely electronic computers. The way we interact with computers has changed over the decades, but command-line interfaces aren't going anywhere soon.

Thanks for reading. I hope you enjoyed this article. You can connect with me on LinkedIn.

In this article, I'll introduce you to some lesser-known Linux commands that will help you become a more productive and effective developer.

Whether you’re looking to boost your productivity, tighten system security, or just want to show off your Linux chops to fellow devs, these commands have you covered.

Chroot: Run Commands With a Different Root Directory

Ever wanted to run commands in a different root directory? The chroot command allows you to do just that.

Chroot, short for “change root,” lets you run commands with a different root directory. So if you wanted to test out software in a contained environment or build a minimal Linux system, chroot is your friend.

To use chroot, first you’ll need to set up a directory to act as the new root. Then you run the chroot command, passing it the path to that directory. For example, to chroot into /home/testdir, you’d run:

chroot /home/testdir

After that, any commands you run will be relative to / in the /home/testdir directory. So you could install packages, run shell scripts, compile programs, and so on. When you’re done, just exit the chroot with exit.

Chroot creates an isolated environment, so any changes you make won’t affect the rest of your system. It’s a handy tool for testing and development or for emergency recovery. Once you get familiar with chroot, you’ll find all sorts of uses for it.

Crontab: Schedule Tasks to Run Automatically

Crontab is a handy Linux command that lets you schedule tasks to run automatically at specific times. You’ll wonder how you ever lived without it!

To get started, open the crontab file with crontab -e. This will open your default text editor where you can create schedule entries, called cron jobs. Each entry has five-time fields – minute, hour, day of month, month, day of week, followed by the command to run.

For example, 0 0 * * * /home/user/daily_script.sh would run the daily_script.sh script at midnight every day. You can also do 30 8 * * 1-5 /home/user/work_script.sh to run work_script.sh at 8:30am every weekday.

Crontab allows a ton of flexibility. You can schedule jobs to run:

• Minutely, hourly, daily, weekly, monthly or yearly
• At a specific minute/hour
• On certain days of the week/month
• With wildcards to run, for example every 3 hours or every Monday and Thursday

The possibilities are endless! You can schedule backup scripts, system maintenance, email reports and so much more.

Df: Check Disk Space Usage

Ever run out of disk space and wonder where it all went? The df command is here to help. df stands for “disk free” and shows you exactly how much space is used and available on your Linux system.

Check Usage for All Mounted Filesystems

To see an overview of disk space usage for all mounted drives and partitions, simply run:

df -h

The -h flag formats the output in human-readable format, showing sizes in GB and MB instead of bytes. The output will show:

• Filesystem: The drive or partition name
• Size: Total space
• Used: Space currently in use
• Available: Free space still available
• Use%: Percentage of space used

This gives you a quick snapshot of where you have space and where it's running low.

Check Usage for a Specific Filesystem

To check space on a specific drive or partition, pass its mount point to df:

df -h /home

This will show usage stats just for your /home partition.

Keeping an eye on disk usage with df and related tools is important for any Linux system administrator. No one wants to run out of space unexpectedly.

Dmesg: View Kernel Messages

Ever wonder what’s happening behind the scenes in your Linux system? The dmesg command lets you peek under the hood and view messages from the kernel, the core of your operating system.

When your Linux system boots up, the kernel initializes hardware, loads drivers, starts services, and performs other startup tasks. The dmesg command displays the messages that get logged during this process so you can see what’s going on.

To view the kernel messages, just open your terminal and run the dmesg command. You'll see pages and pages of status updates, diagnostics, errors, and more as your system came to life. Skim through to check for any issues, or search for specific keywords like the name of your Wi-Fi adapter or other hardware components.

The dmesg output can also provide clues to solving any problems you're having. For example, if your network isn’t working, check dmesg for error messages related to your Ethernet or wireless card. You may spot something like “error initializing network device eth0” that points you to a driver issue.

The dmesg command is a handy diagnostic tool for any Linux system administrator or power user. It offers an inside look at your Linux system and can help uncover the source of both major malfunctions and minor annoyances. Ps: you can feel like a hacker.

Grep: Search for Patterns in Files

The grep command allows you to search for patterns in files and text. It’s ideal when you need to find something specific in a sea of data.

Say you have a massive log file full of information, but you only want to see the lines containing the word “error.” Just run:

grep error log.txt

This will print only the lines in log.txt that contain the word “error.”

You can also use grep to search for patterns rather than just words. For example, to find all lines in a file that start with “A” followed by a number, use:

grep ^A[0-9] log.txt

The ^A anchors the match to the start of the line, and [0-9] matches any digit.

Grep has many more advanced features as well. You can use:

• Regex patterns for complex searches
• -i to ignore case
• -v to invert the search and show lines that don’t match
• -c to just get a count of matches
• -r to recursively search all files in a directory

The next time you need to search through files, don’t do it manually—let grep do the work for you.

Head/Tail: View the First/Last Part of a File

Have you ever needed to quickly check just the first or last few lines of a long file? The head and tail commands are perfect for that.

The head command shows you the first 10 lines of a file by default. You can specify how many lines you want to see by using the -n flag, like head -n 5 filename to show the first 5 lines.

The tail command shows you the last 10 lines of a file by default. Again, use the -n flag to specify how many lines you want to see, like tail -n 20 filename to show the last 20 lines.

Both head and tail are useful when you want to do a quick check of the start or end of a long file without having to scroll through the entire thing. Some other uses of these commands:

• Checking log files for recent errors or warnings
• Viewing email headers
• Previewing configuration files
• And more!

Give head and tail a try – you'll be amazed at how much simpler they make tasks you do all the time.

Ps: List Running Processes

The Ps command allows you to see info about the processes running on your system. This includes programs, commands, and daemons that are currently active. Using Ps is a quick way to get an overview of what your Linux system is doing at the moment and the system resources each process is using.

To see a basic list of running processes, enter:

ps aux

This will show you:

• A: All processes
• U: User
• X: Processes without terminals

The output will contain info like:

• USER: The owner of the process
• PID: The process ID
• %CPU: The CPU usage
• %MEM: The memory usage
• VSZ: The virtual memory usage
• TTY: The terminal associated with the process
• STAT: The process state (Running, Sleeping, Zombie, and so on.)
• START: The start time of the process
• TIME: The CPU time used
• COMMAND: The command that started the process

You can also filter the Ps output by:

• Username: ps aux | grep root
• Process name: ps aux | grep cron
• PID: ps aux | grep 555

The Ps command allows you to quickly check in on what your system is doing and ensure there are no runaway or zombie processes hogging resources. For any Linux user, Ps is an indispensable tool for system monitoring and troubleshooting.

Rsync: Sync Files and Folders

As a Linux user, you’ve probably found yourself needing to sync files and folders between locations. Maybe you have files on your desktop you need to transfer to your laptop, or you want to backup your most important folders to an external drive. The rsync command makes syncing and backing up your files a breeze.

Rsync is a fast and versatile file copying tool. It can copy and sync files and folders locally or remotely over SSH. It’s smart enough to only transfer the differences between two locations, saving time and bandwidth.

To use rsync, open your terminal and enter the command:

rsync [options] source destination

• The source is the location of the files you want to copy. This could be a folder on your desktop or a remote server.
• The destination is where you want to copy the files. This could be an external drive mounted on your system or a folder on another server.
• Options allow you to specify items like:
• -a: Archive mode which preserves permissions, timestamps, group, owner and symlinks
• -v: Verbose output so you can see what’s being copied
• -z: Compression to speed up transfers over slow networks
• -h: Human-readable sizes (e.g. 1K, 234M, 2G)

Rsync is a must-have tool for any Linux user. Once you get the hang of it, you’ll be syncing and backing up your files with confidence.

For broad reading please refer to RSync Examples – Rsync Options and How to Copy Files Over SSH.

The Powerful Pipe Viewer (pv)

Ever wanted to see the progress of data through a pipe? The pv command allows you to do just that. It's a pipe viewer that shows you the progress of data through a pipeline.

Say you have a large file that you want to compress, like a video or backup file. Rather than staring at a blinking cursor while gzip does its thing, you can see the progress with pv. Just pipe the data through pv, then to gzip:

cat mylargefile.mp4 | pv | gzip > mylargefile.mp4.gz

pv will display the throughput and estimated time remaining as your data is compressed. It's a simple way to get feedback on long-running commands.

You can also use pv to see throughput and transfer rates of data over the network. For example, when copying a file with scp or rsync, add pv to the pipeline:

rsync -avz myfiles user@host:/backup | pv

Now you'll see the progress of your files copying over the network. pv gives you information like:

• Transferred bytes
• Transfer rate
• ETA
• Progress
• And more

It's a handy tool to give you more insight into what's happening in those long-running terminal commands.

mtr: Network Diagnostics

Have you ever needed to diagnose network issues but didn’t have access to expensive tools? mtr is a simple but powerful Linux network diagnostic tool. It combines the functionality of the ‘traceroute’ and ‘ping’ programs in a single network diagnostic tool.

mtr sends ICMP echo requests to test the network connectivity between the host mtr runs on and a user-specified destination host. It prints the response times and packet loss statistics for each router along the path. This allows you to quickly pinpoint network issues.

To use mtr, open a terminal and enter:

mtr [domain name or IP address]

For example, to trace the route to google.com, enter:

mtr google.com

mtr will start tracing the route and printing results that update in real time. It will show:

• The IP address and hostname of each router along the path
• The percentage of packet loss for each router
• The response times in milliseconds for each router

The output will continue to update until you press Ctrl+C to stop the trace.

mtr is a simple but useful tool for any Linux network administrator. It can save you hours of troubleshooting time when the network goes down by helping you identify the source of latency or packet loss.

jq: Parse JSON

Ever come across a messy JSON file and wish you had an easy way to parse through it? Meet jq, a command line tool that lets you filter and analyze JSON data with ease.

jq works like a filter. You pass it JSON data on stdin, and it passes the filtered/transformed data to stdout.

For example, say you have a JSON file called data.json with an array of objects. You can filter it to only show objects where name is equal to John like this:

cat data.json | jq '.[] | select(.name == "John")'

This will print only the John objects to the console.

jq supports many more filters than just select(), here are a few more useful ones:

• .key: Access a key from objects
• .[10:]: Show elements from index 10 onwards
• .[10:15]: Show elements from index 10 to 15
• length: Print the length of an array
• map(.): Apply a filter to each element of an array
• group_by(.key): Group objects by a key

With jq, you can manipulate JSON data in pretty much any way you want right from the command line. jq may seem niche, but JSON is used everywhere on the web, so being able to analyze and transform it efficiently is a useful skill.

tac: View Config Files in Reverse

Have you ever made a mistake while editing a config file and saved the changes, only to realize you preferred the previous version? The tac command allows you to quickly view config files in reverse, so you can see what the file looked like before your edits.

Tac simply prints files in reverse order, line by line. To view a file called config.txt in reverse, run:

tac config.txt

This will print the last line of the file first, then the second last line, and so on until it reaches the first line.

• Use tac when you want to quickly view a log file in reverse to see the latest entries first.
• Tac can also be useful when editing config files through the command line. If you make a mistake, run tac to see what the file looked like before so you can revert your changes.

Tac is a simple but useful utility to have in your Linux toolkit.

perf: Analyze CPU Performance

If you've ever wondered why your Linux system seems slower over time, the perf command can help you figure it out. Perf is a profiling tool in Linux that can analyze the performance of your CPU to help identify any bottlenecks.

To get started, run the basic perf list command to see a list of the events you can monitor. There are hundreds! Some of the most useful ones for profiling CPU performance are:

• cpu-clock: Measure the CPU clock cycles
• task-clock: Measure the time spent on task execution
• cache-misses: Count the number of cache misses
• branch-misses: Count the number of branch prediction misses

Pick an event you want to monitor, then run a command like:

perf stat -e cpu-clock sleep 5

This will run the sleep 5 command and measure the cpu-clock event while it's running. Perf will then give you a summary of the stats for that event.

To get more detailed profiling info, use the perf record command. For example, to profile a script called script.sh, run:

perf record script.sh

This will run the script and record the profiling data. You can then view the results with:

perf report

This gives you an interactive report to analyze the results. You'll see things like:

• The percentage of time spent in each function
• The actual time spent in each function
• The number of calls to each function

Using perf, you have a powerful tool to optimize the performance of your Linux system. Perf really is an amazing (if underused) tool.

Conclusion

So there you have it, some helpful Linux commands that will make you feel like a power user in no time.

With these tricks up your sleeve, you'll be zipping around Linux like a pro. The next time you're stuck or frustrated, give one of these a shot. You might just surprise yourself with what you can accomplish.

Linux is an incredibly powerful operating system if you know how to speak its language. Consider this your starter guide to becoming fluent.

I am Rahul, 19, Hustler. I am building Fueler.io a platform for generalists to create proof of work profile and land opportunities. Let’s connect on Twitter.

But then Vagrant came along, a command-line or shell tool that generally works with Type 2 hypervisors. You use it to create and manage virtual machines. It is a powerful tool that can help simplify the setup and management of your development environment.

Vagrant can be really helpful if you work on a team or with multiple people. This is because it guarantees consistency in your development environment by ensuring that everyone utilizes the same environment, preventing compatibility issues.

This tutorial will guide you through the process of setting up a single Ubuntu Linux virtual machine with Vagrant and configuring a web server inside it.

Prerequisites for this tutorial include:

• A computer with at least 8GB of RAM

• Basic knowledge of the Linux operating system

Required Tools and Installation

• Oracle VirtualBox: Go to the Oracle VirtualBox website, find the version of VirtualBox that is compatible with your operating system, and follow the instructions to download and install it. Virtual Box will provide the virtual environment, while Vagrant will set it up and manage it.

• Vagrant: Visit the Vagrant website and follow the instructions to download and install the binary that is suitable for your operating system. In this tutorial, we'll be utilizing the open-source Vagrant binary.

To check if the installation was successful, launch your preferred command line tool and enter the following command to output the installed version number:

$ vagrant --version

How to Create a Development Environment with Vagrant

To create a Vagrant project, start by creating a new project directory in your preferred location for Vagrant configuration and related files.

$ mkdir vagrant-project && cd vagrant-project

Within this directory, create a new Vagrantfile. Vagrant uses the configuration in the Vagrantfile to build the VM. By default, Vagrant syncs the project directory where the Vagrantfile is initialized to /vagrant. This eliminates the need to worry about volumes for persisting data.

Vagrant uses the concept of boxes. Boxes are a complete base image of an operating system. The public vagrant box repository contains a list of possible boxes. Choosing a box that matches the operating system used in your production environment is good practice.

A Vagrant box has the name of the user or organization that created it and the box name user/boxname. To initialize the Vagrant configuration file with an Ubuntu box, run the command:

$ vagrant init ubuntu/trusty64

This generates a Vagrantfile with a Ubuntu/trusty64 box in the current directory. The Vagrantfile, which is written in Ruby, contains the kind of VM to be used and various additional commented options such as network, port forwarding, disc capacity, and so on to assist in configuring the development environment.

You can add the --minimal flag to the initialization command of the Vagrantfile to generate a Vagrantfile without any additional settings.

Open the Vagrantfile with any editor of your choice. I will use the Vim editor in this tutorial.

 $ vim Vagrantfile

Removing the informational comments and some advanced configurations will leave the file like this:

# -*- mode: ruby -*-
# vi: set ft=ruby :

Vagrant.configure("2") do |config|
  config.vm.box = "ubuntu/xenial64"
    config.vm.network "forwarded_port", guest: 8000, host: 8000
    config.vm.provider "virtualbox" do |vb| vb.memory = "1024"
 end
  config.vm.provision :shell, path: "simple-node-project.sh", privileged: false
end

The simple-node-project.sh is a bash script that installs Node.js and Git, clones a project that creates a simple Node.js web server, and starts the server.

#!/bin/bash

 sudo apt-get update -y

 ## Git ##
 echo '###Installing Git..'
 sudo apt-get install git -y

 git clone https://github.com/Aijeyomah/simple-node-app.git

# Installing latest Node and npm version
 sudo apt-get install -y curl software-properties-common

# Add Node.js PPA
curl -sL https://deb.nodesource.com/setup_14.x | sudo -E bash -

# Install Node.js and npm
sudo apt-get install -y nodejs

# Verify installation
node -v
npm -v

echo "Node.js has been installed successfully."

# navigate to app directory and start app
cd simple-node-app
node index.js &

This Vagrant configuration sets up the following:

• ubuntu/trusty64 is specified as the virtual box base image

• Forwards port 8000 of the VM to port 8000 of the host machine.

• Allocates 1GB of memory to the VM

• Runs simple-node-project to provision the VM

• For the shell provisioner to run the script as a non-root user in a login shell, privileged is set to false

Save the Vagrantfile and start the virtual machine by running the following command:

$ vagrant up

The first time this command is run, it will download the latest version of the specified box, and it will configure and start the VM. This process might take some time, but when the Ubuntu box exists in the local machine the VM will start immediately.

Once the VM is running, you can access the web page by opening a web browser and navigating to http://localhost:8000. You should see the Hello World message page if everything was set up correctly.

How to Manage Vagrant

You can use Vagrant to manage the running virtual machine. Here are some useful Vagrant commands:

vagrant up: Launches the virtual machine and provisions it according to the settings in the Vagrantfile. This command will simply connect to the virtual machine if it is already running.

vagrant halt: Stops the virtual machine by delivering a shutdown signal to the guest operating system. This command is similar to shutting down a real computer.

vagrant reload: Restarts the virtual machine and re-provisions it depending on any changes in the Vagrantfile.

vagrant ssh: Connects to the virtual machine via SSH. This command is useful for accessing the command line interface of the virtual machine.

vagrant status: Shows the current status of the virtual machine, including whether it's running, stopped, or suspended.

vagrant destroy: Deletes the virtual machine and all associated resources. This command is useful for cleaning up your development environment.

Conclusion

In this article, we have learned how to utilize Vagrant to set up a reproducible and consistent development environment.

Using Vagrant can help you set up a virtual development environment that closely mimics your production environment. This allows you to test and develop your code in a consistent and isolated environment.

If you're looking for a tutorial to teach you the basics of Nodejs and the command line, you're in the right place.

You can use JavaScript to build almost any software (web, mobile, bots, and so on). The reason is that computers no longer depend on browsers only to understand JavaScript code. Check out this article to see How Computers Understand JavaScript Code.

Node.js is used to run backend applications that are written in JavaScript.

I will teach you how to build a command line application that can read, write, edit, and delete data using Node.js. It will not require connecting to external databases like MySQL, MongoDB, Postgresql, and so on. See the project here.

By the end of the article, you should be able to set up a basic Node project, manipulate files, use modules, navigate promises, collect input, and so on.

I also added videos to enhance your learning.

Prerequisites

You do not need any prior programming knowledge to understand this tutorial.

How the Finished Project Will Function

The application that we are going to build will be able to do the following:

• Check if a database exists
• Retrieve data from the database
• Add new data to the database
• Update database with new data
• Remove data from the database

How to Install Node and NPM

Please go to the Nodejs website and download the one recommended for all users. Install it after the download is complete.

Use the command below to check if the installation was successful:

• For Node

node -v

• For npm

npm -v

The installation is successful if each command returns a version number.

How to Setup a Node.js Project

The steps below will help you set up your project:

Open your terminal or CMD and create the project directory:

mkdir node_CLI_app

Navigate into the directory:

cd node_CLI_app

Initialize the project:

npm init

It will bring up some questions. Press the Enter button for every prompt.

A new file (package.json) should have been added to your project directory. We will use the file to add external code (module) to the project.

Add the following line before the closing curly brace (}) to enable ES6 import:

"type": "module"

That completes the project setup!

How to Install Dependencies

You will recall that the package.json file is for adding external code to the project. This external code is also called Dependencies, Modules, or Packages. It's written by other programmers (usually free of charge) to help others build applications faster. You will find a lot of these on the npm website.

We need two (2) packages for this project: inquirer and uuid. I will show you how to install them in this section.

Installing of packages follows this pattern:

npm install <package_name>

We can install more than one package at a time by separating the package names with a space:

npm install <package_name_1> <package_name_2> <package_name_3>

The install command can be replaced with i for convenience.

So run the following command to install the packages:

npm i inquirer uuid

Your package.json file should have the following lines of code added to it when the installation is complete:

  "dependencies": {
    "inquirer": "^9.1.4",
    "uuid": "^9.0.0"
  }

The version numbers may differ, but that’s fine.

You will also notice that a file (package.lock.json) and a folder (node_modules) have been added. You do not have to worry about them. Just know that they help to manage the external code we just added.

That completes the installation of dependencies!

How to Create a New File from the Terminal

You can create a file from the terminal using the following command:

• On Mac:

touch <file_name>

• On Windows:

echo.><file_name>

You can also create more than one file at a time by separating the files with a space. That is how we will generate the files for this project:

touch addData.js removeData.js retrieveData.js updateData.js queryDB.js dbFileCheck.js

Each of those files will play a unique role in the application.

• addData.js adds data to the database. It also fabricates the database file if it doesn’t exist.
• removeData.js removes selected data.
• retrieveData.js fetches all data.
• updateData.js edits data.
• queryDB.js checks if a database exists and executes a function passed to it.
• dbFileCheck.js confirms if the database file has been created.

There is one more file that we did not create: the database file (db.json). We will auto-generate it using our code.

How to Check if a File Exists

Node.js provides a built-in module for manipulating files – file system (fs). One of the methods that the module contains is the existsSync method. It returns true or false depending on if the file has been created. I will use that to check if the db.json file is in the project. See the code below:

  import fs from "fs";
  import { exit } from "process";  

  if (fs.existsSync("db.json")) {
    console.log("File exists!");
    exit(1);
  }

The exit method terminates a process.

That is what we want to do in the dbFileCheck.js file. However, we want to invert the result by adding a ! before the fs.existsSync method like this:

  import fs from "fs";
  import { exit } from "process";  

  if (!fs.existsSync("db.json")) {
    console.log("File does not exist!");
    exit(1);
  }

This will be necessary when building other functionalities like updating and deleting data.

So how will those files gain access to this code? It is through a modular structure. That entails bundling this code and making it accessible through other files in the project. The export command makes this possible:

import fs from "fs";
import { exit } from "process";

export default function dbFileCheck() {
  if (!fs.existsSync("db.json")) {
    console.log("Database is Empty. Create some data!");
    exit(1);
  }
}

The code above puts the code in a function (dbFileCheck) and exports it using the export command. This function can now be imported and used in other files within this project.

Note that the default keyword is necessary for the first export from a file.

How to Query the Database

Another method the file system has is the readFile method. It returns the content of any file passed to it.

The readFile method takes in two parameters: the file to be read and a callback function that returns the result of the operation.

We will use the following code to retrieve data from our database:

import fs from "fs";

fs.readFile("db.json", function (err, data) {
  if (err) {
    console.log(err);
  }
  console.log(data.toString());
});

The code above imports the file system module and tries to read the database file. If there is an error, it will return the error. If there are no errors, it will return the data it got.

The .toString() method attached to the data returned (data.toString()) is because the data retrieved is of the type buffer by default, which is not readable.

buffer output

Open your project in a code editor and paste that code into the queryDB.js file.

Run the command below to see if it is working:

node queryDB

The command above will execute every code in the queryDB.js. The .js extension is optional. It will run either way.

The result of that operation will be an error because we do not have the file.

You can create the db.json file, add some content, and check the output.

But we do not want our app to try to read the database file if it has not been generated. So use the existsSync method to check if the file has been created. See how I use it in the code below:

    import fs from "fs";

    if (fs.existsSync("db.json")) {
      fs.readFile("db.json", function (err, data) {
        if (err) {
          console.log(err);
        }
        console.log(data.toString());
      });
    } else {
      console.log("No data available!");
    }

The code above checks if the file exists by passing the file name (db.json) to the fs.existsSync method. If it is true, then it reads the file. If it is false, it returns a string.

Now that we have that clean code, we want to make it even more robust.

Since we are building a CRUD application, we must have access to and keep track of the data returned. I have introduced a new variable in the code below to make that happen:

    import fs from "fs";

    let info = [];
    if (fs.existsSync("db.json")) {
      fs.readFile("db.json", function (err, data) {
        if (err) {
          console.log(err);
        }
        info = JSON.parse(data.toString());
        console.log(info);
      });
    } else {
      console.log("No data available!");
    }

The code above now has a variable, info. It keeps track of the data returned. I passed the data through the JSON.parse method to convert the data from string to array.

We can now manipulate the info as we deem fit. We have to export the code and accept a function to make this possible. That function will take the info variable as input and then use it as required.

import fs from "fs";

export default async function queryDB(externalFunction) {
  try {
    let info = [];

    if (fs.existsSync("db.json")) {
      await fs.readFile("db.json", function (err, data) {
        info = JSON.parse(data.toString());
        console.log(info);

        if (err) {
          console.log(err);
          return;
        }

        if (externalFunction && !err) {
          externalFunction(info);
          return;
        }
      });
    } else {
      if (externalFunction) {
        externalFunction(info);
        return;
      }
    }
  } catch (error) {
    console.error(`Something Happened: ${error.message}`);
  }
}

This code takes in a function and executes it only if there are no errors. However, there will be one exception – when adding data. In that case, the database will be created.

The try…catch… block helps pick out errors, and the async await keywords are used when executing code that will take a long time to run.

The queryDB.js will no longer return any output when we execute the node queryDB command. But that is fine. We will trigger it from other files.

How to Add Data to a File

In this section, I will teach you how to add data to the store. The file for this section is the addData.js file.

We will start by importing all necessary modules:

import inquirer from "inquirer";
import fs from "fs";
import { v4 as uuidv4 } from "uuid";
import queryDB from "./queryDB.js";

Next, create the boilerplate for the function:

export default async function addData(info) {
  try {

  } catch (error) {
    console.log("Something went wrong!", error);
  }
}

This is similar to what we have done before now. All the code we would write next will go into the try… block. The info array passed into the function is coming from the queryDB file.

The first thing to do in the try… block is to collect the data to be stored. We will use inquirer to do that with the code below:

const answers = await inquirer.prompt([
      {
        type: "input",
        name: "name",
        message: "What's your name?",
      },
      {
        type: "number",
        name: "phone",
        message: "What's your phone?",
      },
      {
        type: "list",
        name: "age",
        message: "Are you an adult?",
        choices: [
          { name: "Y", value: "Adult" },
          { name: "N", value: "Minor" },
        ],
      },
    ]);

The inquirer package helps in building interactive command line interfaces. It contains a method called prompt used to ask for inputs. It takes an array of questions in object format. Each object must have the name, type, and message keys.

The choices key is optional. It is used when there is a list of options.

So the code above collects three (3) inputs (name, phone, and age) and stores them in a variable called answers.

Next, we assign this set of input a unique ID by calling the uuidv4() function and push everything into the info array:

    const data = {
      id: uuidv4(),
      name: answers.name,
      phone: answers.phone,
      age: answers.age,
    };
    info.push(data);

Finally, we check if the database file exists. We will update the database with the new data if the file has been created or create it and add the new data if it is false.

    if (fs.existsSync("db.json")) {
      createDetails(info);
    } else {
      fs.appendFile("db.json", "[]", (err) => {
        if (err) {
          console.log("Could not create db.json", err);
          return;
        }
        createDetails(info);
      });
    }

The createDetails function will be used to overwrite the database data. I will create that in a bit.

In the else (if the file doesn’t exist) block, I created the file using the file system’s appendFile method. This method is used to create a file if it is not there already and add or append data to the bottom of the file.

I am appending [] to the file. So the newly created file will have just [] in it. I called the createDetails function next to add the data collected from the CLI.

Please type the following code for the createDetails function below the addData function.

async function createDetails(info) {
  await fs.writeFile("db.json", JSON.stringify(info), function (err) {
    if (err) {
      console.log(err);
    }
    console.log("saved!");
  });
}

This function uses the file system’s writeFile method to overwrite the database with current data. I am using JSON.stringify to convert the info variable to a string because that is the acceptable format when writing to a file. That also explains why I used JSON.parse to convert it to the original type when I retrieved it in the previous section.

writeFile is different from appendFile because writeFile overwrites a file while appendFile adds to the existing content. They are similar in that both methods will create the file if it has not been created.

The last thing to do in this file is invoke or call the function. The way to do that is by typing its name followed by opening and closing braces like this:

addData();

This might work fine, but it wouldn’t do what we want. We need to pass the function as an argument into the queryDB like this:

queryDB(addData);

Now, the addData will be able to communicate with the queryDB function.

How to Test the addData File

Run the command below to test if the addData file works as expected:

node addData

It will prompt you for answers. Fill in the answers. After that, your screen should look like mine:

addData output

It should also have auto-generated a db.json file with the data you just added.

And that is it for the addData file!

How to Retrieve Data

This section shows how to get the content of the database. All you need to achieve this functionality is the code below:

import queryDB from "./queryDB.js";

queryDB();

What the code does is import the queryDB function and invoke it. Paste the code in the retrieveData.js file and execute the file with:

node retrieveData

It will return an output similar to this:

retrieveData.js output

You might wonder: why did I not call this function in the queryDB file? The reason is that the file performs more than just retrieving data. Calling the queryDB function in its file will alter the result of other files.

How to Edit Data

This section will now focus on how to update data. The pattern to follow here is:

• Collect the user’s ID.
• Search for the user.
• Return the user’s data if the user exists, and set it as the default option.
• Ask for updated info. The initial value will be kept if the user presses the Enter key.
• Finally, overwrite the database with the updated information.

Let’s get to it…

Navigate into the updateData.js file and paste the following code:

import inquirer from "inquirer";
import fs from "fs";
import queryDB from "./queryDB.js";
import dbFileCheck from "./dbFileCheck.js";

export default async function updateData(info) {
  dbFileCheck();

  try {

  } catch (error) {
    console.log("Something went wrong!", error);
  }
}

This is the boilerplate for the updateData function. The new thing here is the dbFileCheck function (made it a while ago) to terminate an operation if the database has not been created.

The rest of the code will be inside the try… block.

The first thing is to collect the user ID with this code:

    const answers = await inquirer.prompt([
      {
        type: "input",
        name: "recordID",
        message: "Enter Record ID",
      },
    ]);

The second is to search for the user:

    let current;

    info.forEach((element) => {
      if (element.id === answers.recordID) {
        current = element;

        updateDetails(current, info);
      }
    });

The code above searches through the users (info) and sets the user found to the current variable. Finally, the updateDetails is called to collect the updated information and overwrite the database with the new data.

How to build the updateDetails function

This part will show how to keep track of a user’s initial data while collecting new data. It will update the database with the new data afterward.

The following code is the boilerplate for the function:

async function updateDetails(current, info) {
  try {

  } catch (error) {
    console.log("Something went wrong!", error);
  }
}

This code goes under the updateData operation.

The code below goes into the try… block. It is for collecting updated information from the user.

  const feedbacks = await inquirer.prompt([
      {
        type: "input",
        default: current.name,
        name: "name",
        message: "What's your name?",
      },
      {
        type: "number",
        default: current.phone,
        name: "phone",
        message: "What's your phone?",
      },
      {
        type: "list",
        default: current.age,
        name: "age",
        message: "Are an adult?",
        choices: [
          { name: "Y", value: "Adult" },
          { name: "N", value: "Minor" },
        ],
      },
    ]);

The default key is new. It holds input that will be used if the user doesn’t provide one. It keeps track of the user’s current data for this code. So the user can hit the Enter button instead of entering the former value again.

The user’s details should be updated accordingly using this code:

    current.name = feedbacks.name;
    current.phone = feedbacks.phone;
    current.age = feedbacks.age;

Finally, overwrite the database with the new value:

await fs.writeFile("db.json", JSON.stringify(info), function (err) {
      if (err) {
        console.log(err);
      }
      console.log("updated");
    });

Call the updateData function to bring everything together:

queryDB(updateData)

How to test the updateData file

Run the command below to see how the updateData function performs:

node updateData

It will prompt you for an ID. Enter the ID of any record in the database.

It will then prompt for updated information about the user. Fill in the information and press Enter for any detail that does not need an update. The terminal should look like this at the end:

updateData Output

That wraps it up for the updateData file. Records can now be updated.

How to Delete Data

A CRUD application cannot be completed without the DELETE part. That will be the focus here. It will borrow a bit from the previous sections. So it will not be too much to grasp.

You'll begin by typing the following code in the removeData.js file:

import inquirer from "inquirer";
import fs from "fs";
import queryDB from "./queryDB.js";
import dbFileCheck from "./dbFileCheck.js";

export default async function removeData(info) {
  dbFileCheck();

  try {
    const answers = await inquirer.prompt([
      {
        type: "input",
        name: "recordID",
        message: "Enter Record ID",
      },
    ]);

    let remnantData = [];
    info.forEach((element) => {
      if (element.id !== answers.recordID) {
        remnantData.push(element);
      }
    });

    fs.writeFile("db.json", JSON.stringify(remnantData), function (err) {
      if (err) {
        console.log(err);
      }
      console.log("Deleted!");
    });
  } catch (error) {
    console.log("Something went wrong!", error);
  }
}

First, the code above collects an ID.

Next, it uses the ID to search the database data and keeps only data with a different ID in the remnantData array.

Finally, it overwrites the database with the updated data.

Call the removeData function at the bottom to put everything together like this:

queryDB(removeData)

Now try to test the file using this command:

node removeData

The is my output:

removeData Ouput

Conclusion

The best way to learn to program is by building. And some of the best projects to build are CRUD applications because they cover the basics of what a professional project requires. That is what I have done in this tutorial.

I have taught you the basics of Nodejs by building an application that creates, reads, updates, and deletes records. I covered concepts such as reading files, writing to files, loops, conditional statements, modules, and CLI operations.

All the files for this tutorial are on GitHub.

You now have all the basics needed to start building applications using Nodejs. Something I will like you to try is to update or delete more than one record at a time.

I have some other tutorials on Nodejs, and I suggest that you follow them in this order to grow your skill:

• How to Build a Secure Server with Node.js and Express and Upload Images with Cloudinary
• How to Build and Deploy a Backend App with Express, Postgres, Github, and Heroku
• How to Build a Full-Stack Authentication App With React, Express, MongoDB, Heroku, and Netlify

Happy Building!

But whenever different users share a system, problems of privacy can easily arise. The first user may not wish the next user to view, edit, or delete their files, for example.

The Linux Terminal possesses some superpowers when it comes to handling file permissions. You can grant or revoke permissions for every file and directory from your Linux Terminal.

What are File Permissions in Linux?

File permissions control which actions can be performed by which users. Read, Write, and Execute are the three actions possible for every file.

Users are classified under three broad categories: Normal users, Groups, and Others. Linux allows users to set permissions at a very granular level. You can secure your file or directory in every possible location of a file system.

This seems useful, right?

There are three important commands you'll use when managing file permissions:

1. chmod (Change mode)
2. chown (Change ownership)
3. chgrp (Change group)

Among these, chmod is one of the most important commands. We'll discuss the chmod command in this tutorial, and I'll get into the others in upcoming articles.

Let’s deep dive into the chmod command 🏊.

Actions you can perform on a file

Before we proceed further, I want to make sure you're clear about how the Read, Write, and Execute actions of a file work. Read and write are pretty are self-explanatory – they determine whether a user can read or write to a file.

But, what’s an executable file?

A file is said to be executable if it has a sequence of instructions to achieve something. A good example is scripting files (Shell Scripts).

What is the chmod Command?

chmod is a command that lets you change the permissions of a file or directory to all types of users.

Here’s the syntax of the chmod command:

chmod <Operations> <File/Directory Name>

You can grant or revoke the permission by replacing the Operations in the above command.

What are the operations you can perform?

The Operations in the above syntax are divided into 2 categories. Let's explore them below.

User Level permissions

These operations control permissions on the user level. Here's the commands you can use:

• u – Grant permission to a user
• g – Grant permission to a group (A Group of users)
• o – Grant permission to others (who do not come under either of the above).

Note: If this option is left empty, permissions will be applied to the logged-in user. Most of the time it'll be left empty.

File Level permissions

These control permissions on the file level.

• r – Grants read permission
• w – Grant write permission
• x – Grant execute permission

These operations need to be preceded with a '+' or '-' operator.

'+' indicates adding a new permission, and '-' indicates removing an existing permission.

Here's an example:

chmod +r sample.txt

The above command adds read permission for the sample.txt file.

Pretty straightforward, right? Let's continue.

How to Make a File Executable in Linux

I can explain this more clearly with an example from my experience.

Linux is the default operating system of my team. We recently hired an intern, who has zero knowledge of Linux but was curious to learn and explore. We started to train him initially by asking him to write some shell scripts, because most servers run Linux OS. He found the entire code on the internet and copied it (we gave such a task intentionally).

He saved the file but was not able to run the script. He didn't know the actual problem. He started removing a few blocks of code and tried to run it again and again.

He repeatedly got the error stating "Command not found".

Finally, he reached the 1st line. He replaced that line with a print statement (the "echo" command) and ran the file with the hope to see the output. But he still hadn't found that error.

With some frustration, he asked for help.

Let's see the issue now.

Basically, we can execute .sh files by just running them like this:

./install.sh

Let's see the code inside install.sh

echo "This is executable file 🎉"

He ran the same command but it did not work. This is because the file was not in executable format. So I ran the magic command to make the file executable:

chmod +x install.sh

Terminal command to make a file executable

Now it is executable. He stared at me as if I was a hacker 😂. But really, it's a pretty simple and basic concept.

How to Remove Permissions from a File in Linux

I work with my colleague Divad on lots of projects, and he likes to try to fool me. We work together on many hobby projects and we often write shell scripts for quick deployment.

Whenever he writes shell scripts, he always removes all the permissions from the file and pushes the changes to the remote repo. So every time I have to grant permissions using the above commands for whatever action I have to do.

Let's have a quick look at the command he uses to remove file permissions.

Here we have a file named install.sh which has all permissions (Read, Write, Execute). Let's remove the execute permission for this script file.

chmod -x install.sh

Terminal command to remove execute permission from a file

You'll not be able to execute this file now. Trying so will give you an error as shown in the above screenshot.

Let's remove the read permission from the file.

chmod -r install.sh

Alright, with this command we've removed the read permission. Let's try to read that file using Nano (the file editor for the Linux terminal). You will be able to see the "Permission Denied" error at the bottom.

The same applies to removing write permission from the file:

chmod -w install.sh

You can achieve all the above together using the below command:

chmod -rwx install.sh

This is the core part of handling file permissions in Linux. Remember that we have barely scratched the surface of it, though. Try to understand it and play around with some sample files. Because who knows – in the future, you may get a colleague like Divad. :)

How to Add or Remove Permissions for Directories (Folders) in Linux

If you work with Linux, you might have come across various directories such as /etc, /var, /opt, and others. But you may not be aware of why these directories exist.

There's one thing in common for all these folders, though: that is, you'll not be able to create a file or folder inside them without root permission.

This setting will be pre-configured in your system when Linux OS is installed.

But, you might wonder, can I restrict my folder in a /home directory similar to the above directories? The answer is yes. You can achieve this by changing the permission of the directory using the chmod command.

Let's understand this with an example.

I created a directory named locked_directory and removed read permission from this directory. If I try to read the contents of the folder using the ls command, I'll end up seeing the "Permission Denied" error message.

chmod -r locked_directory/

chmod command to remove read permission from a directory

But, did you know that I can create another directory inside locked_directory named dir1 and read the files and folders in dir1?

Then what's the purpose of the command we just ran before? Removing the read permission on the parent should remove the same on child directories too, right?

Well. That's the exact thing I told you earlier. Linux manages a very granular level of file permissions.

If you want to apply the permissions to the parent directory and all its child directories, you need to pass an exclusive flag with the chmod command.

That flag is -R. It basically means applying the same permissions recursively to all sub-directories (child directories). So this permission will apply to the end child of a file/directory.

Here's the syntax for that:

sudo chmod -R <permission> <filename>

Remember that running the command to do a recursive operation needs root permission. So you need to add sudo at the beginning of this command. Here's what it looks like:

sudo chmod -R -r locked_directory

Terminal Command to remove read permission from a directory Recursively

From the above screenshot, you can see that trying to view the child directory files has failed after removing the read permission recursively from the parent directory.

Another Way to Handle File Permissions in Linux

Alternatively, you can use Octal representation to control the file permissions.

We can use numbers to represent file permissions (the method most commonly used to set permissions). When you change permissions using the Octal mode, you represent permissions for each triplet using a number (4, 2, 1, or combination of 4, 2, and 1).

Let's see the syntax for using octal mode:

chmod <user><group><others> install.sh

Here's an example of octal mode:

chmod 777 install.sh

How can I remove permissions using Octal Mode?

We can use 0 to remove permissions from a file. Here's an example:

chmod 000 install.sh

The table shows the Octal code for each file permission:

You might be confused 😖. Read further to understand clearly.

Let's consider a scenario.

You want to grant read, write, and execute permissions to users and read-only permission for groups and others to the install.sh file.

Let's see how to do that using the above two methods.

How to manage permissions in Symbolic Mode

chmod u+rwx,go+r install.sh

Let's dismantle each part and try to understand them:

• u+rwx represents adding read, write, and execute permissions for users
• go+r represents adding read permission for groups and others

How to manage permissions in Octal Mode

 chmod 744 install.sh

Let's dismantle each of these numbers and try to understand them:

• The first number (7) represents permission for a user: 7 = ( 4 (read) +2 (write) +1(execute) )
• The second number (4) represents permissions for a group: 4 (read)
• The third number (4) represents permissions for others: 4 (read)

Which Mode is Best?

It turns out that symbolic mode is more powerful than octal mode.

The reason is, in the symbolic mode we can mask out the permission bits we want to change. But in octal mode, permission modes are absolute and can't be used to change individual bits.

How to Find Permissions of a File

We can find the existing permissions of a file using ls command.

I hope you all know about ls command. Adding the -l flag and file name with the ls command shows some more info about the file, including the permissions.

ls -l install.sh

Terminal command to see the existing permission of a file

Look at the first part of the output (-rwxrwxrwx) from the above screenshot. Let's explore what it means:

Existing permissions output description

The first character indicates the type of input.

• "-" indicates a file
• "d" indicates a directory
• "i" indicates a link (a symlink, which is a shortcut to a file/directory)

You group the next set of letters, at a maximum of 3 for each group. These groups represents corresponding permissions for user, group, and others.

Conclusion

In this article, you have learned about handling basic file and folder permissions.

I hope you enjoyed reading this tutorial. I have one request to all: give it a try on your own with some complicated scenarios like having permutations and combinations of permissions 😂. It'll definitely be helpful in your entire career.

Subscribe to my newsletter by visiting my site and also have a look at the consolidated list of all my blogs.

Cheers!

The Bandit wargame from OverTheWire is aimed at absolute beginners. It is a game you connect to through SSH that will help you will improve your command line skills, your linux skills, and you hacker skills.

We just published a course on the freeCodeCamp.org YouTube channel that will provide you with a complete walkthrough of the 34 levels that are part of the Bandit wargame.

Sabyasachi Paul created this tutorial. He has a lot of experience in the field of cybersecurity. Sabyasachi will guide you through the series of security challenges involving the Linux shell, remote connections, and SSH.

Before watching each challenge solution, consider first trying to solve each challenge yourself.

In this command line game, you start at Level 0 and try to “beat” or “finish” it. Finishing a level results in information on how to start the next level.

If you are following along with the game yourself, you may encounter many situations in which you have no idea what you are supposed to do. There are several things you can try when you are unsure how to continue in the game (before watching the solution).

First, if you know a command, but don’t know how to use it, try the manual (man page) by entering man <command>. For example, man ls to learn about the “ls” command. The “man” command also has a manual, try it! When using man, press q to quit (you can also use / and n and N to search).

Second, if there is no man page, the command might be a shell built-in. In that case use the help <X> command. For example, help cd.

You can find instructions for the game here: https://overthewire.org/wargames/bandit/

Watch the full walkthrough on the freeCodeCamp.org YouTube channel (2-hour watch).

The operating system (or OS) is a piece of software that controls all other application programs and helps you manage the hardware and software of your computer.

Examples of popular operating systems are Windows, Linux, MacOS, and Android. In this tutorial, we'll focus on the Linux OS.

operating system diagram

You will learn the most frequently used Linux commands and operators. You'll also get a high level grasp of the Linux operating system and its various distributions, which are referred to as "distros" in this article.

Table of Contents

• Why learn the Linux command line?
• History of operating systems
• The rise of the GNU Project
• How Linux works and its basic components
• What are Linux distributions?
• How to choose a Linux distribution
• Basic Linux commands to run in the terminal
• How to work with directories in Linux
• Commands to work with files in Linux
• Commands to work with file contents
• Linux command operations

Why Learn the Linux Command Line?

There are lot of reason why you should learn about the Linux command line. Some of these are:

• More Control Over Your Machine: You have a great deal of power and control with the command line. You can run commands to change permissions, view hidden files, interact with databases, start servers, and more.
• It's Faster: You can complete tasks much more quickly with the basic commands in your toolbox than you could with a Graphical User Interface (GUI). Just keep in mind that it might be slower while learning the CLI.
• Automate Many Tasks: You may speed up your work by using a single command to create 10,000 files, each with a unique name. With a GUI, this process is laborious.
• Available Everywhere: The instructions you issue will automatically run similarly on Linux and Mac computers. And with a little tweaking, they will also function on Windows.
• Basic requirement: You NEED to use the command line if you want to advance your knowledge in any coding-related technology field, including development, data analysis, devops engineering, system administration, security, machine learning engineering, and others.

History of Operating Systems

Most OSs are generally divided into two families: Unix-based and Microsoft NT descendants.

Unix was an OS developed in the mid 1960s. It's the "grandparent" of many modern operating system that we frequently use now, such as Linux.

The Unix operating system was a closed source project (meaning its code and files weren't made public). And this led to the rise of the "Free software" movement led by Richard Stallman. It argued that users should have the freedom to run, copy, distribute, and collaborate on the source code of a project.

Microsoft NT descendants were proprietary graphical operating systems that Microsoft created. The Windows NT descendants don't natively have similar Linux commands, unlike Unix and Unix-based Operating Systems, which do. Instead, Microsoft NT has its own set of commands and default shells.

Microsoft NT's offspring includes Windows, Xbox OS, Windows Phone/Mobile, and others.

The Rise of the GNU Project

Richard Stallman wanted to create a free software alternative to Unix. He worked with some other developers in 1984 to create a full operating system that would be free. So they started working on the GNU project.

At same time, another developer called Linus Torvalds was creating his own kernel known as Linux. At that time, many GNU pieces were completed but they lacked a kernel. Torvalds combined his kernel with the existing GNU components to create a full OS.

Some developers strongly feel that the name should be GNU/Linux instead of just Linux, as it reflects the joining of the Linux kernel with the GNU project.

How Linux Works and its Basic Components

In this section, you will learn how Linux works by understanding its fundamental components. We'll now discuss these elements.

What is a kernel?

A kernel is a part of an OS that facilitates interactions between the hardware and software. It's an essential element of an operating system for a computer.

The core of the OS alone is responsible for providing all other components with necessary services. It helps with device control, networking, file system management, process and memory management, and it acts as the main interface between the OS and the hardware.

kernel

What is a Shell?

A shell is a computer interface to an operating system. The shell exposes the services of the OS to users or other programs. The shell takes your commands and gives them to the OS so it can perform them.

It's named a shell because its the outer layer around the OS – like the shell around an oyster!

What is the Terminal?

A terminal is a program that runs a shell. This is where we run most of our commands that tell the OS what to do.

You install the terminal in the following ways on different operating systems:

• Linux Distro users – the Bash shell is installed by default
• Mac Users – Terminal is installed by default and can execute similar Linux commands
• Windows Users – Download Windows Subsystem for Linux (WSL) or use git bash and run all Linux command from there.

What are Linux Distributions?

Linux distributions (popularly called distros) are flavors of the Linux operating system. These distros are built based on Linux's open source software.

Some examples of these are:

Debian Family

Since it was founded in 1993, Debian has released new versions far more slowly than Ubuntu and mint. It is one of the most reliable Linux distributors as a result.

Debian is the foundation of Ubuntu, which was created to expeditiously enhance Debian's fundamental components and make it more user-friendly.

Ubuntu was created by Canonical in 2004 and gained popularity immediately. Canonical wants Ubuntu to be used as a simple, command-line-free graphical Linux desktop. It's the most well-known Linux distribution.

Ubuntu is simple for beginners to use. It has a large number of pre-installed applications and convenient repository libraries.

Red Hat Family

Red Hat is a professional Linux distributor. Red Hat Enterprise Linux (RHEL) and Fedora are their products, both of which are open source.

Fedora offers faster updates and no support, but RHEL is thoroughly tested before release and supported for seven years after the release.

Red Hat uses trademark law to stop the redistribution of its software. Red Hat Enterprise Linux source is used in CentOS, a community effort that eliminates all of Red Hat's trademarks and makes it publicly available. In other words, it is a free version of RHEL and offers a long-lasting reliable platform.

SUSE Family

SUSE created its own operating system for computers. It is supplied with system and application software from other open source projects and is developed on top of the free and open source Linux kernel.

SUSE Linux was primarily developed in Europe and is of German origin. The name SUSE is an acronym for "Software und System-Entwicklung." SUSE is one of the oldest commercial distributions still in use because the initial version debuted in early 1994.

Fedora Family

This is a project that offers the most recent software versions and mostly focuses on free software. It uses 'upstream' applications instead of developing its own desktop environment. It comes with the GNOME3 desktop environment by default. Although less reliable, it offers the newest information.

How to Choose a Linux Distribution

Now let's talk about some commands you can run to interact with the shell.

Basic Linux Commands to Run in the Terminal

The whoami command

This command prints the name of the currently logged in user to the terminal session.

caesarsage@caesarsage:$ whoami

The man command

This command prints the manual or information about a command, configuration files, and so on. This command is very useful when it comes to getting more information about any command.

caesarsage@caesarsage:$ man whoami

The clear command

Clears all previous commands that were run in the current terminal. This clears the screen from previous commands in the terminal.

caesarsage@caesarsage:$ clear

How to open files

Mac Users:

open <filename or directory name>

The open command lets you open a file or directory in the graphical user interface (GUI) outside the terminal.

Linux Users

xdg-open <filename or directory name"

Windows WSL users

You can open files in a similar way to Linux, but you need to install the xdg-open package.

Example for Linux and Windows users:

caesarsage@caesarsage:$ xdg-open clean-code-architecture.pdf

Now that we have covered the basic commands, let's learn a few other commands you'll use a lot.

How to Work with Directories in Linux

Now let's see some of the most common commands you'll use to work with directories.

Directories are like folders, and you can create, delete, and perform all functions on them through your system interface with a mouse or cursor.

Here we will be doing something similar but from the comfort of our terminal. The following commands let you perform different operations on directories:

• pwd (present working directory)
• cd (current directory)
• ls (list)
• mkdir (make directory)
• rmdir (remove directory)

Let's look at what each one does:

The pwd command

Whenever you feel lost in the filesystem, call the pwd command to know where you are. It takes no argument.

casesarsage@caesarsage:~/Documents/github.com$ pwd

It should print the current folder/directory path where you currently are.

The cd <path> command

You can change your current directory with the cd command (Change Directory). Just like going back and forth between folders when using the GUI.

caesarsage@caesarsage$: cd Documents/articles

This command takes me to a folder called articles inside my Documents folder.

Let's see what else you can do with cd.

cd ~

The cd is also a shortcut to get back into your home directory. Just typing cd without a target directory will put you in your home directory. Typing cd ~ has the same effect.

caesarsage@caesarsage:~/Documents/github.com$ cd ~

This takes your to you home directory from the github.com folder

cd ..

To go to the parent directory (the one just above your current directory in the directory tree), type cd ..:

caesarsage@caesarsage:~/Documents/github.com$ cd ..

The ls command

Inside a folder you can list all the files that the folder contains using the ls command. It takes no arguments.

caesarsage@caesarsage:~/Documents/mycatfolder$ ls

Just like with cd, there are some other options you can use with ls:

ls -a

A frequently used option with ls is -a to show all files. Showing all files means including the hidden files.

When a file name on a Linux file system starts with a dot, it is considered a hidden file and it doesn't show up in regular file listings. This command will show those files.

ls -l

Many times you will be using options with ls to display the contents of the directory in different formats or to display different parts of the directory.

Typing just ls gives you a list of files in the directory. Typing ls -l gives a long listing and permission as (rwx - read, write, execute).

The mkdir <directoryName> command

Walking around the Unix file tree is fun, but it is even more fun to create your own directories/folders with mkdir.

You have to give at least one parameter to mkdir – the name of the new directory to be created. Think before you type a leading / .

caesarsage@caesarsage:~/Documents$ mkdir cats

The rmdir <directoryName> command

When a directory is empty, you can use rmdir to remove or delete the directory.

caesarsage@caesarsage~/Documents$ rmdir cats

rmdir -p <directoryName>

When you want to delete nested directories, you can used the -p flag. You use rmdir -p to recursively remove directories. This is similar to creating nested directories with mkdir -p.

caesarsage@caesarsage:~/Documents$ rmdir -p articles/drafts

Commands to Work with Files in Linux

In this section, you'll learn how to recognize, create, remove, copy, and move files using the following commands:

• touch
• rm
• cp
• mv
• rename

The touch <filename> command

One easy way to create an empty file is with touch like this:

caesarsage@caesarsage:~$ touch file1.txt file2.md file3

The above creates three files (text and markdown files).

The rm <filename> command

When you no longer need a file, use rm to remove it.

Note that unlike some graphical user interfaces, the command line in general does not have a waste bin or trash from where you can recover files. When you use rm to remove a file, the file is gone. So be careful when removing files!

caesarsage@caesarsage:~$ rm file1.txt

Here are some more specific ways to use rm:

rm -v <filename>

This flags gives you feedback of what it did (deleting a file).

rm -i <filename>

To prevent yourself from accidentally removing a file, you can type rm -i. This will show a prompt to confirm if you really want to delete the file or not.

rm -rf <filename>or<directory>

By default, rm -r will not remove non-empty directories. However rm accepts several options that will allow you to remove any directory.

The rm -rf statement is famous because it will erase anything (providing that you have the permissions to do so). When you are logged in as root, be very careful with rm -rf (the f means force and the r means recursive), since being root implies that permissions don't apply to you. You can literally erase your entire file system by accident.

The cp <fileold> <newfile> command

To copy a file, use cp with a file name and a new file name argument.

caesarsage@caesarsage:$ cp text2.md text2Copy.md

cp <source> <destination>

Use this option to copy a file to another directory (destination).

If the target is a directory, then the source files are copied to that target directory.

caesarsage@caesarsage:~$ mkdir dir3
caesarsage@caesarsage:~$ cp file2.md dir3

cp -r directorySource dirTarget

To copy complete directories, use cp -r (the -r option forces recursive copying of all files in all sub-directories).

caesarsage@caesarsage:~$ cp -r dir1/dir2  dir3

The mv source destination command

You can use the mv command to move and rename directories.

caesarsage@caesarsage:~/Documents/$ mv cat catFolder

caesarsage@caesarsage:~/Documents/$ mv newarticle.txt articles

Commands to Work with File Contents

You can use the following commands to look at the contents of text files:

• head
• tail
• cat
• less
• echo
• wc
• grep

The head <file> command

This command prints the first part of the files. By default it gives the first 10 lines of a file, but you can override that by adding the -n flag.

caesarsage@caesarsage:$ head /etc/passwd

The tail <file> command

This command prints the last 10 lines of a file. You can also override the default similarly by passing the -n flag.

The tail file also has an -f flag that helps you keep printing extra additions to a file. This is useful for logs and errors files that keep changing in your system so you can monitor them.

caesarsage@caesarsage:$ tail /etc/passwd

The cat <filename> command

cat can add content to a file which makes it super powerful. In its simplest usage, cat prints a file's contents to the standard outputs.

caesarsage@caesarsage:$ cat file

You can print the content of multiple files as well.

And using the operator > (we will see what this does later – for now, know it takes terminal output into a file) you can concatenate the content of multiple files into a new file:

caesarsage@caesarsage:$ cat file2.txt file3.txt > combine.txt

You can also use it to create files:

caesarsage@caesarsage:$ cat > newfile.txt

The less <filename> command

The less command shows the content stored inside a file in a nice and interactive UI.

caesarsage@caesarsage:$ less /etc/passwd

Use b to scroll one page, G to the go to end, g to go to the start and q to quit the cmd.

The echo command

This command prints to the output the argument passed to it.

caesarsage@caesarsage:$ echo 'Hello world'

The wc <input> command

wc stands for word count, and this command gives information about input (for example a file) like number of lines, number of words, number of bytes for content, and so on.

wc -l

This option prints only the newline count.

wc -m

This option prints only the character count.

wc -c

This option prints only the byte count.

wc -w

This option prints only the word count.

The grep command

The command grep is probably the most widely used text manipulation command. It lets you filter the content of a file for display.

If, for instance, you want to see all lines that include the word output in your file, you could use cat and ask it to display only those lines.

caesarsage@caesarsage:$ cat /etc/snort/snort.conf | grep output

You will learn more about the pipe (|) operator in the next section.

Linux Command Operations

Some common commands you can use to manipulate Linux commands are:

• >: redirects standard outputs

Most of the commands we have seen so far print something out for us on the terminal. For example the PWD prints out our current directory, and so on.

These outputs can be stored and redirected to a file with the use of ">". It overrides the current file contents when you run it multiple times.

caesarsage@caesarsage: whoami > file.txt
caesarsage@caesarsage: pwd > file.txt
caesarsage@caesarsage: cat > file.txt

• >>: redirects standard outputs and appends new contents.

Unlike the '>' operation, >> doesn't override previously stored output in a file.

caesarsage@caesarsage: whoami >> file.txt
caesarsage@caesarsage: pwd >> file.txt
caesarsage@caesarsage: cat file.txt

• |: this operator is called pipe.

This takes the output of one command and passes it as the input for another command. Here's how you use it:

caesarsage@caesarsage:$ cat /etc/snort/snort.conf | grep output

Summary

In this article, you learned about the Linux operating system at a high level. You also learn to use the Linux command line to interact with the OS.

As always, I hope you enjoyed the article and learned something new. If you want, you can also follow me on LinkedIn or Twitter.

Cheers and see you in the next one! ✌️

And knowing how to use the command line interface (also known as the command prompt or terminal, depending on your operating system) is one of those skills.

Whether you're doing Web development, Game Development, Application Development, Cloud Engineering, DevOps, or many other disciplines in tech, you'll likely use the command line quite often.

History of the Command Line

In the early days of computers, developers used MS-DOS to navigate around the computer system.

Disk Operating System (DOS) is a kind of operating system that uses commands (words) to interact with the computer.

The DOS didn’t use mouse pointers, icons, or any graphics, so users were stuck with using text-commands to operate the computer system.

For instance, to go to a particular folder, you'd type:

Cd <name of folder>

The command line interface (or CLI for short) is similar to DOS in that it uses commands to perform various operations, like creating files, creating folders, installing programs, and what have you.

The advancement of technology over the years brought about the popular GUI (Graphical User Interface) and made Operating Systems less stressful to use.

Although developers (and non-technical users) often use the GUI these days, sometimes working directly from the CLI is useful or necessary, irrespective of your stack.

What is the CLI? Is it a Programming Language?

_Photo by [Unsplash](https://unsplash.com/@angshu_purkait?utm_source=ghost&utm_medium=referral&utm_campaign=api-credit">Angshu Purkait / <a href="https://unsplash.com/?utm_source=ghost&utm_medium=referral&utmcampaign=api-credit)

If you're new to software development, it's easy to get carried away with the various terminology you'll need to learn. Worry less, you are not doing a spelling bee competition. Rather, you are meant to learn what these terms refer to, how they actually work, and how you can use them.

One of the terms you'll hear quite often is the Command Line Interface (also referred to as Command Prompt or Terminal).

The Command Line Interface (CLI) is an editing environment that is text-based. It uses specified text (known as commands) to interact with the computer and perform numerous operations, including installing and working with programs.

Every operating system comes with an inbuilt command prompt. Some application packages such as Nodejs, Anaconda, Git, and so on also come with their own command prompt.

The same thing goes for Cloud Providers such as AWS, GCP, Azure. Although the CLI bears different names across different platforms or packages, its purpose remains the same: to let you interact freely with the software package or the computer system using text-based instructions known as commands.

So, the CLI is a tool, not a programming language.

A basic knowledge of CLI will help you along your tech journey, especially if you are into Software Development. In fact, you can completely build a program and run it right from the CLI.

This article will:

• Explain how the CLI Works
• Help you locate your CLI according to your OS
• Show you how to perform basic operations using the CLI
• Help you differentiate between the CLI and GUI

How the CLI Works

As briefly discussed earlier, the CLI uses text-based instructions to perform operations. These commands have specific syntax you need to follow and certain text must be written on the same line – otherwise it'll throw an error message.

In the case of CLI in application packages, the commands may be relative to the package in question. But all CLIs follow the same rule of being semantic and on the same line.

To use your CLI:

• Locate the CLI in your PC
• Open it
• Type in your desired commands
• Press the enter key

Later in the tutorial, we'll execute some commands using the CLI so you can better understand how it works.

The Command Line in Different Operating Systems

Every Operating System comes with its default Command Line Interface, though you can choose to install a more advanced CLI depending on your needs.

Some Operating Systems and their respective CLIs are listed below:

• Windows: Command Prompt
• Linux: Linux Bash Shell
• MacOs: Mac Terminal
• Google Cloud Platform: PowerShell, Cloud shell
• Amazon Web Services: AWS Command Prompt
• Microsoft Azure: Azure CLI bash

A picture of Windows command prompt, AWS Cloudshell, MacOs Terminal and Linux Terminal

I currently use Windows OS and Windows Command Prompt, but I'll also show you how you can locate your own Terminal or Command Prompt based on popular Operating systems.

How to Locate your CLI

In Windows

You can access the command prompt in Windows OS using the program directory or using shortcut keys.

Using the program directory, go to your search bar (next to the Windows icon) and type cmd. This will pop up a list of all the command prompts available on your machine, including the default windows cmd. You can now select the one you wish.

Using a keyboard shortcut, press Windows + R on your keyboard and type in cmd on the dialog box that pops up.

A picture showing the run terminal

In MacOS

As in the case of Windows, you can also open the CLI in a Mac OS using the program directory or a keyboard shortcut.

To use the program directory, locate the launchpad and type in Terminal. This will bring it up.

To use the keyboard shortcut, type the combination of the cmd + space bar keys.

Here's how to locate MacOs Terminal and create a file in a directory:

How to Perform Basic Operations Using the CLI

Back in Mathematics class, it was quite easy to memorize formulas and solve equations using those memorized formulas. But knowing when to apply those equations in real life scenarios was – and often continues to be – difficult for many students.

Knowing where your CLI is located and how it works is a good step in the right direction. But let me show you how to get started with the CLI by practicing simple operations right from your Command Prompt.

How to navigate through your PC with the CLI

Navigating through your PC simply means moving from one folder or file to another. If you don't want to use your mouse to direct your cursor, you can move around your PC using the CLI.

For instance in Windows, if you want to open the desktop, take the following steps:

1. Open the CLI (CMD) as explained earlier
2. Next, type in cd Desktop on the command prompt which will take you to your desktop

Keep in mind that this command is for Windows OS (for Mac, for example, it'll be slightly different).

Here's what you'll see:

How to create a folder using the CLI

You may know the usual method of right-clicking on your screen and selecting the "New folder" option on the drop-down menu. But there's a way of creating a new folder using the CLI.

1. First, open your CLI
2. Navigate to the folder or location where you want to create the new folder
3. Enter mkdir
4. Press Enter
5. You can now enter the folder by typing cd

How to install a package using the CLI

Installing a software package or application using the CLI depends on the package in question.

Packages like Node.js (a development package for BackEnd Development) require that you install the Node Package Manager (npm). This package manager is what you'll use to install and run Node and other similar packages.

Some commands for installing software packages via the CLI include:

1. Install <name of package>
2. run <name of package

For instance to install a new instance of a React app:

• Open the terminal

• locate your local environment by typing:

cd Desktop

• Create a directory for the app by typing:

cd Desktop mkdir my_directory

• create the react-app right in the directory by using the command:

npx create-react-app my-app

A terminal showing the commands for installing create-react-app using Windows Command Prompt

Differences Between the CLI and GUI

GUI vs CLI image

As we've discussed, the CLI uses commands to generally interact with the computer.

On the other hand, the Graphical User Interface (GUI) is a method of interacting with the computer using icons, menus, mouse clicks, and pointers.

An Operating System that is GUI-based allows users to freely operate the computer by clicking, dragging, dropping, and other visual methods of interacting with the computer.

Unlike the GUI, the CLI uses less RAM space and interacts with the operating system directly.

To use the GUI, no programming knowledge is required. But to use the CLI, you need to have a certain amount of knowledge of programming and command operations.

Conclusion

As you begin or advance in your tech journey, you'd need to install a lot of programs that are foreign to your Local Machine. Such programs may not have GUI installation methods and may require you to run a line of code or more on your CLI. At such a point, knowledge of using the CLI comes in handy.

There's a general notion that the CLI is difficult to use – and it does take some getting used to. But once you get acquainted with the operations of CLI you'll find it much easier to manage.