Zaira Hira - freeCodeCamp.org

How to Get Information About Your Linux System Through the Command Line

Zaira Hira — Wed, 11 Jun 2025 10:15:20 +0000

Whether you’ve just gained access to a new Linux system, ethically hacked into one as part of a security test, or you’re just curious to know more about your current machine, this article will guide you through the process.

You’ll learn how you can get information related to your OS (operating system), kernel, CPU, memory, processes, disks, networks, and installed software. You’ll explore the commands and their outputs in detail.

Why It's Important to Understand Your Linux System
How to Get Your OS & Kernel Information in Linux
How to Get Your CPU Information in Linux
How to Get Your Memory Information in Linux
How to Get Your Disk & Filesystem Information in Linux
How to Get Your Hardware Information in Linux
How to Get Your Network Interfaces & Status Information in Linux
How to Get Your Software & Services Information in Linux
How to Get Your Logs & Dmesg In formation in Linux
How to Get Your Security/User Audit Information in Linux
Visually Appealing Commands
Conclusion

Why It's Important to Understand Your Linux System

System Administration

System administrators need to have an understanding of the system so they are able to:

Manage users, groups, and permissions effectively.
Configure services like web servers, databases, and so on.
Automate repetitive tasks with scripts and cron jobs.

Troubleshooting

When the system is in a problematic state, a solid understanding of the system specification and configuration helps you to:

Identify and resolve system errors quickly.
Analyze system logs and monitor performance.
Diagnose network and hardware issues.

Security Auditing

If you are in a security related role, knowing your system in depth helps you to:

Monitor logs for unauthorized access.
Configure firewalls and security policies.
Detect and remove malicious processes or software.

Performance Optimization

If you know how to gather information related to system resources, you can measure them and create a projection for the future use. You can also:

Tune system parameters for better efficiency.
Monitor resource usage (CPU, memory, disk, I/O).
Eliminate bottlenecks and optimize workloads.

Proactive Maintenance

It is a good practice to be able to prevent issues before they occur. Once you know your system well, you can:

Schedule regular updates and backups.
Ensure system reliability and uptime.

Understanding your Linux system gives you greater control, enhances system stability, and improves your overall effectiveness as a system administrator or power user.

In the next section, we’ll discuss some essential commands for gathering system information.

How to Get Your OS & Kernel Information in Linux

`uname -a` Command

uname -a provides full kernel information:

uname -a
Linux ip-172-31-90-178 6.8.0-1024-aws #26-Ubuntu SMP Tue Feb 18 17:22:37 UTC 2025 x86_64 x86_64 x86_64 GNU/Linux

Here is what each part means in the above command:

Linux: The kernel name.
ip-172-31-90-178: The network hostname of the system.
6.8.0-1024-aws: The kernel version and AWS-specific build.
#26-Ubuntu: The kernel build number.
SMP: Symmetric Multi-Processing, indicating that the kernel is compiled for multiple processors.
Tue Feb 18 17:22:37 UTC 2025: The date and time when the kernel was compiled.
x86_64 x86_64 x86_64: The machine hardware name (architecture), processor type, and platform type, all indicating 64-bit x86 architecture.
GNU/Linux: The operating system name.

Based on this output, I’m running on an AWS EC2 instance with a 64-bit Ubuntu Linux distribution using a kernel that was specifically built for AWS infrastructure.

`uname -r` and `uname -s` Commands

The uname -r and uname -s commands specify the kernel version and OS type information:

uname -r
6.11.0-25-generic

uname -s
Linux

`cat /etc/os-release` Command

The cat /etc/os-release command provides distribution information:

cat /etc/os-release
PRETTY_NAME="Ubuntu 24.04.2 LTS"
NAME="Ubuntu"
VERSION_ID="24.04"
VERSION="24.04.2 LTS (Noble Numbat)"
VERSION_CODENAME=noble
ID=ubuntu
ID_LIKE=debian
HOME_URL="https://www.ubuntu.com/"
SUPPORT_URL="https://help.ubuntu.com/"
BUG_REPORT_URL="https://bugs.launchpad.net/ubuntu/"
PRIVACY_POLICY_URL="https://www.ubuntu.com/legal/terms-and-policies/privacy-policy"
UBUNTU_CODENAME=noble
LOGO=ubuntu-logo

Here is what each part means in the above command:

PRETTY_NAME="Ubuntu 24.04.2 LTS": The user-friendly name of the distribution including version and LTS (Long Term Support) designation.
NAME="Ubuntu": The name of the Linux distribution.
VERSION_ID="24.04": The version number of the Ubuntu release (Year/Month format).
VERSION="24.04.2 LTS (Noble Numbat)": The complete version information including:

• 24.04: Major version (released April 2024)

• .2: Point release number

• LTS: Long Term Support

• Noble Numbat: The release codename
VERSION_CODENAME=noble: The codename for this Ubuntu release ("Noble").
ID=ubuntu: The machine-readable name of the operating system.
ID_LIKE=debian: Indicates that Ubuntu is based on Debian Linux.
HOME_URL, SUPPORT_URL, BUG_REPORT_URL, PRIVACY_POLICY_URL : Various official URLs for Ubuntu resources.
UBUNTU_CODENAME=noble: Reiterates the codename of this Ubuntu release.
LOGO=ubuntu-logo: Specifies the logo identifier for the distribution.

This output shows that I’m running Ubuntu 24.04.2 LTS (codenamed "Noble Numbat"), which is a Long Term Support release of Ubuntu. Being an LTS version means it will receive security updates and support for an extended period (typically 5 years for Ubuntu LTS releases).

`hostnamectl` Command

hostnamectl shows the hostname, OS, and kernel info:

hostnamectl
 Static hostname: ip-172-31-90-178
       Icon name: computer-vm
         Chassis: vm 🖴
      Machine ID: ec272830b6dca2da0d11e41b292cfc99
         Boot ID: dd12f48ff01b44a796991d99ce1bcfde
  Virtualization: xen
Operating System: Ubuntu 24.04.2 LTS              
          Kernel: Linux 6.8.0-1024-aws
    Architecture: x86-64
 Hardware Vendor: Xen
  Hardware Model: HVM domU
Firmware Version: 4.11.amazon
   Firmware Date: Thu 2006-08-24
    Firmware Age: 18y 9month 1w 2d

In the above command, here is what each part means:

Static hostname: "ip-172-31-90-178": This is the permanent hostname of the system, stored in /etc/hostname.
Icon name: "computer-vm": A symbolic icon identifier for the system, used by some desktop environments.
Chassis: "vm": Indicates this is running in a virtual machine environment.
Machine ID: "ec272830b6dca2da0d11e41b292cfc99": A unique identifier for this system, stored in /etc/machine-id.
Boot ID: "dd12f48ff01b44a796991d99ce1bcfde": A unique identifier that changes with each system boot.
Virtualization: "xen": Shows that this system is running on Xen virtualization (common for AWS instances).
Operating System: "Ubuntu 24.04.2 LTS": The current OS distribution and version.
Kernel: "Linux 6.8.0-1024-aws": The current Linux kernel version, specifically an AWS-optimized kernel.
Architecture: "x86-64": The CPU architecture of the system.
Hardware Vendor: "Xen" Hardware Model: "HVM domU": Indicates this is a Xen HVM (Hardware Virtual Machine) domain user instance.
Firmware Details:
- Version: 4.11.amazon: This is the version of the firmware/BIOS specifically customized for AWS environments.
- Date: Thu 2006-08-24: This is the release date of the firmware. The date might seem old (2006) but this is normal for AWS instances.
- Age: 18y 9month 1w : This shows how old the firmware is relative to the current date calculated from the firmware date (2006) to now (2025). While the firmware seems old, it is still maintained and secure.

This overall output shows that I’m running Ubuntu 24.04.2 LTS on an AWS EC2 instance using Xen virtualization. The system is using an AWS-optimized kernel and is configured as a HVM (Hardware Virtual Machine) instance.

How to Get Your CPU Information in Linux

`lscpu` Command

lscpu shows CPU architecture, cores, threads, and virtualization information:

lscpu
Architecture:             x86_64
  CPU op-mode(s):         32-bit, 64-bit
  Address sizes:          46 bits physical, 48 bits virtual
  Byte Order:             Little Endian
CPU(s):                   1
  On-line CPU(s) list:    0
Vendor ID:                GenuineIntel
  Model name:             Intel(R) Xeon(R) CPU E5-2686 v4 @ 2
                          .30GHz
    CPU family:           6
    Model:                79
    Thread(s) per core:   1
    Core(s) per socket:   1
    Socket(s):            1
    Stepping:             1
    BogoMIPS:             4599.99
    Flags:                fpu vme de pse tsc msr pae mce cx8 
                          apic sep mtrr pge mca cmov pat pse3
                          6 clflush mmx fxsr sse sse2 ht sysc
                          all nx rdtscp lm constant_tsc rep_g
                          ood nopl xtopology cpuid tsc_known_
                          freq pni pclmulqdq ssse3 fma cx16 p
                          cid sse4_1 sse4_2 x2apic movbe popc
                          nt tsc_deadline_timer aes xsave avx
                           f16c rdrand hypervisor lahf_lm abm
                           pti fsgsbase bmi1 avx2 smep bmi2 e
                          rms invpcid xsaveopt
Virtualization features:  
  Hypervisor vendor:      Xen
  Virtualization type:    full
Caches (sum of all):      
  L1d:                    32 KiB (1 instance)
  L1i:                    32 KiB (1 instance)
  L2:                     256 KiB (1 instance)
  L3:                     45 MiB (1 instance)
NUMA:                     
  NUMA node(s):           1
  NUMA node0 CPU(s):      0
Vulnerabilities:          
  Gather data sampling:   Not affected
  Itlb multihit:          KVM: Mitigation: VMX unsupported
  L1tf:                   Mitigation; PTE Inversion
  Mds:                    Vulnerable: Clear CPU buffers attem
                          pted, no microcode; SMT Host state 
                          unknown
  Meltdown:               Mitigation; PTI
  Mmio stale data:        Vulnerable: Clear CPU buffers attem
                          pted, no microcode; SMT Host state 
                          unknown
  Reg file data sampling: Not affected
  Retbleed:               Not affected
  Spec rstack overflow:   Not affected
  Spec store bypass:      Vulnerable
  Spectre v1:             Mitigation; usercopy/swapgs barrier
                          s and __user pointer sanitization
  Spectre v2:             Mitigation; Retpolines; STIBP disab
                          led; RSB filling; PBRSB-eIBRS Not a
                          ffected; BHI Retpoline
  Srbds:                  Not affected
  Tsx async abort:        Not affected

Here is a brief explanation of the output above:

1. Basic CPU Info

Architecture: x86_64 (64-bit)
CPU Model: Intel Xeon E5-2686 v4 (2.3 GHz)
Cores/Threads: 1 core, 1 thread (no Hyper-Threading)
Physical CPU (Socket): 1

2. Performance & Features

Cache Sizes:
- L1: 32 KiB (data) + 32 KiB (instructions)
- L2: 256 KiB
- L3: 45 MiB (large, typical for Xeon)
Flags: Supports AVX, AES, SSE4.1/4.2 (useful for encryption/vector ops).

3. Virtualization

Hypervisor: Running on Xen (full virtualization).
Virtualization Support: Yes (Intel VT-x).

4. Security (Vulnerabilities)

Meltdown/Spectre: Mostly mitigated (PTI, Retpolines).
MDS/MMIO: Vulnerable (no microcode fixes).
Spec Store Bypass: Vulnerable (no mitigation).

5. NUMA (Memory)

Single NUMA node (no multi-processor complexity).

The output shows that my machine is a single-core Intel Xeon (in a virtualized/cloud environment) with large L3 cache but has some unpatched CPU vulnerabilities.

`cat /proc/cpuinfo` Command

cat /proc/cpuinfo provides more in-depth details about the CPU:

cat /proc/cpuinfo 
processor    : 0
vendor_id    : GenuineIntel
cpu family    : 6
model        : 79
model name    : Intel(R) Xeon(R) CPU E5-2686 v4 @ 2.30GHz
stepping    : 1
microcode    : 0xd000404
cpu MHz        : 2299.998
cache size    : 46080 KB
physical id    : 0
siblings    : 1
core id        : 0
cpu cores    : 1
apicid        : 0
initial apicid    : 0
fpu        : yes
fpu_exception    : yes
cpuid level    : 13
wp        : yes
flags        : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush mmx fxsr sse sse2 ht syscall nx rdtscp lm constant_tsc rep_good nopl xtopology cpuid tsc_known_freq pni pclmulqdq ssse3 fma cx16 pcid sse4_1 sse4_2 x2apic movbe popcnt tsc_deadline_timer aes xsave avx f16c rdrand hypervisor lahf_lm abm pti fsgsbase bmi1 avx2 smep bmi2 erms invpcid xsaveopt
bugs        : cpu_meltdown spectre_v1 spectre_v2 spec_store_bypass l1tf mds swapgs itlb_multihit mmio_stale_data bhi
bogomips    : 4599.99
clflush size    : 64
cache_alignment    : 64
address sizes    : 46 bits physical, 48 bits virtual
power management:

`nproc` Command

nproc shows the core count:

nproc
1

The above command output shows there is one available processor.

How to Get Your Memory Information in Linux

`free -h` Command

You can use the free -h command to know the total/used/free RAM:

free -h
               total        used        free      shared  buff/cache   available
Mem:           957Mi       406Mi       218Mi       920Ki       522Mi       551Mi
Swap:             0B          0B          0B

Here is a breakdown of the output shared above:

total: The total amount of physical memory (RAM) or swap space available on the system.
used: The amount of memory currently being used by applications and the system. Calculated as: total - free - buffers - cache.
free: The amount of memory that is completely unused.
shared: Memory that may be simultaneously accessed by multiple programs.
buff/cache: Combines two types of memory:
- Buffers: Memory used for block device I/O buffering.
- Cache: Memory used for file system page cache - This memory can be reclaimed when needed by applications.
- available: It includes the 'free' memory plus memory that can be reclaimed from buff/cache. This is the most important column for determining if you have enough memory.

vmstat Command

vmstat stands for Virtual Memory Statistics, a tool to monitor system performance. It provides information about memory usage, CPU activity, Processes, Disk I/O and Swap usage.

You can also use vmstat to extract live information. Here is how you can do that:

vmstat 1 5
procs -----------memory---------- ---swap-- -----io---- -system-- -------cpu-------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st gu
 1  0      0 238264  46120 489056    0    0     3     8   23    0  0  0 82  0 18  0
 0  0      0 238264  46120 489060    0    0     0     0  240  120  0  1 98  0  1  0
 0  0      0 238264  46120 489060    0    0     0     0  239  124  0  0 98  0  2  0
 0  0      0 238264  46120 489060    0    0     0     0  199  101  0  0 95  0  5  0
 0  0      0 238264  46120 489060    0    0     0     0   36   25  0  0 78  0 22  0

Here is what the above command is doing:

Captures 5 snapshots of system performance.
Each snapshot is taken 1 second apart, giving near real-time insights.
Displays key metrics about:
- Memory usage (free, buffered, cached).
- CPU activity (user, system, idle, waiting).
- Processes (running, blocked).
- Disk I/O (blocks read/written).
- Swap usage (if swapping is happening).

Note that, you can replace the interval and number of snapshots accordingly.

Here’s a detailed breakdown of the output above:

Procs:
- r: Number of processes waiting for run time.
- b: Number of processes in uninterruptible sleep
Memory (in KB):
- swpd: Amount of virtual memory used
- free: Amount of idle memory
- buff: Memory used as buffers
- cache: Memory used as cache
Swap:
- si: Memory swapped in from disk (KB/s)
- so: Memory swapped out to disk (KB/s)
IO:
- bi: Blocks received from a block device (blocks/s)
- bo: Blocks sent to a block device (blocks/s)
System:
- in: Number of interrupts per second
- cs: Number of context switches per second
CPU (percentages):
1. us: Time spent running user code
2. sy: Time spent running system code
3. id: Time spent idle
4. wa: Time spent waiting for IO
5. st: Time stolen from a virtual machine
6. gu: Time running guest code (virtual CPU)

From the output, you can see that my system:

Has very low CPU usage (high idle percentage)
Has no swap being used (swpd = 0)
Has about 99MB free memory
Shows minimal IO activity
Is running in a virtualized environment (notice the st (stolen) time column has non-zero value

The first line shows averages since the last reboot, while subsequent lines show the real-time statistics for each second.

`cat /proc/meminfo` Command

cat /proc/meminfo shows detailed memory stats:

cat /proc/meminfo
MemTotal:         980384 kB
MemFree:          245100 kB
MemAvailable:     585896 kB
Buffers:           46184 kB
Cached:           393672 kB
SwapCached:            0 kB
Active:           141404 kB
Inactive:         356376 kB
Active(anon):      47672 kB
Inactive(anon):    29300 kB
Active(file):      93732 kB
Inactive(file):   327076 kB
Unevictable:       36528 kB
Mlocked:           27152 kB
SwapTotal:             0 kB
SwapFree:              0 kB
Zswap:                 0 kB
Zswapped:              0 kB
Dirty:                 0 kB
Writeback:             0 kB
AnonPages:         94488 kB
Mapped:            97936 kB
Shmem:               920 kB
KReclaimable:      95396 kB
Slab:             148672 kB
SReclaimable:      95396 kB
SUnreclaim:        53276 kB
KernelStack:        2444 kB
PageTables:         3224 kB
SecPageTables:         0 kB
NFS_Unstable:          0 kB
Bounce:                0 kB
WritebackTmp:          0 kB
CommitLimit:      490192 kB
Committed_AS:     508912 kB
VmallocTotal:   34359738367 kB
VmallocUsed:        9988 kB
VmallocChunk:          0 kB
Percpu:            14848 kB
HardwareCorrupted:     0 kB
AnonHugePages:         0 kB
ShmemHugePages:        0 kB
ShmemPmdMapped:        0 kB
FileHugePages:         0 kB
FilePmdMapped:         0 kB
Unaccepted:            0 kB
HugePages_Total:       0
HugePages_Free:        0
HugePages_Rsvd:        0
HugePages_Surp:        0
Hugepagesize:       2048 kB
Hugetlb:               0 kB
DirectMap4k:       71680 kB
DirectMap2M:      976896 kB

Here is a detailed breakdown of the output shared above:

Total Memory and Available Memory:
- MemTotal: Total physical RAM available.
- MemFree: Completely unused memory.
- MemAvailable: Memory available for new applications.
Memory Caches and Buffers:
- Buffers: Memory used for block device I/O buffering.
- Cached: Memory used for file system cache.
- SwapCached: Memory pages stored in both RAM and swap.
Active vs Inactive Memory:
- Active: Recently used memory.
- Inactive: Less recently used memory.
- Active(anon): Recently used anonymous memory.
- Active(file): Recently used file-backed memory.
Swap Information:
- SwapTotal: Swap space configured.
- SwapFree: Swap space available.
- Zswap: Compressed swap in RAM.
Other Important Metrics:
- Dirty: Memory waiting to be written to disk.
- Mapped: Files mapped into memory.
- Slab: Kernel data structures cache.
- CommitLimit: Total memory available for allocation.
- Committed_AS: Total memory currently allocated.

A healthy memory usage is indicated by a good amount of available memory, active caching mechanisms in place and no memory pressure (no swap usage needed).

How to Get Your Disk & Filesystem Information in Linux

`tree -d -L 1` Command

tree -d -L 1 shows the file system details from the folder it is executed in. To find the complete file system details, run it from the root / folder:

tree -d -L 1
.
├── bin -> usr/bin
├── bin.usr-is-merged
├── boot
├── dev
├── etc
├── home
├── lib -> usr/lib
├── lib.usr-is-merged
├── lib64 -> usr/lib64
├── lost+found
├── media
├── mnt
├── opt
├── proc
├── root
├── run
├── sbin -> usr/sbin
├── sbin.usr-is-merged
├── snap
├── srv
├── sys
├── tmp
├── usr
└── var

25 directories

The command output of tree -d -L 1 shows a directory tree structure with the following options:

-d: Shows only directories (ignores files)
-L 1: Limits the depth of the tree to one level (only shows the immediate subdirectories)

df -h: mounted filesystems and usage:

  df -h
  Filesystem      Size  Used Avail Use% Mounted on
  /dev/root        29G  2.6G   26G   9% /
  tmpfs           479M     0  479M   0% /dev/shm
  tmpfs           192M  908K  191M   1% /run
  tmpfs           5.0M     0  5.0M   0% /run/lock
  /dev/xvda16     881M  144M  676M  18% /boot
  /dev/xvda15     105M  6.1M   99M   6% /boot/efi
  tmpfs            96M   12K   96M   1% /run/user/1000

The above output from the df -h command shows the following disk space usage information:

Filesystem: The name of the mounted filesystem/device.
Size: Total size of the filesystem.
Used: Amount of space used.
Avail: Amount of space available.
Use%: Percentage of space used.
Mounted on: The mount point where the filesystem is attached

`lsblk` Command

lsblk stands for ‘list block devices’ and shows information about all available block devices like hard drives, SSDs, and so on.

lsblk
NAME     MAJ:MIN RM  SIZE RO TYPE MOUNTPOINTS
loop0      7:0    0 26.3M  1 loop /snap/amazon-ssm-agent/9881
loop1      7:1    0 73.9M  1 loop /snap/core22/1748
loop2      7:2    0 44.4M  1 loop /snap/snapd/23545
loop3      7:3    0 50.9M  1 loop /snap/snapd/24505
loop4      7:4    0 73.9M  1 loop /snap/core22/1963
loop5      7:5    0 27.2M  1 loop /snap/amazon-ssm-agent/11320
xvda     202:0    0   30G  0 disk 
├─xvda1  202:1    0   29G  0 part /
├─xvda14 202:14   0    4M  0 part 
├─xvda15 202:15   0  106M  0 part /boot/efi
└─xvda16 259:0    0  913M  0 part /boot

The output above shows the following details:

NAME: Device name.
MAJ:MIN: Major and minor device numbers.
RM: Removable flag (1 for removable, 0 for fixed).
SIZE: Device size.
RO: Read-only flag (1 for read-only, 0 for read-write).
TYPE: Device type (disk, part for partition, loop for loop device).
MOUNTPOINTS: Where the device is mounted.

`fdisk -l` Command

fdisk -l shows all disk devices and their partitions on your system:

Disk /dev/xvda: 30 GiB, 32212254720 bytes, 62914560 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: E3478E01-32E3-4FC2-8E79-1BCCDE89C2D7

Device        Start      End  Sectors  Size Type
/dev/xvda1  2099200 62914526 60815327   29G Linux filesystem
/dev/xvda14    2048    10239     8192    4M BIOS boot
/dev/xvda15   10240   227327   217088  106M EFI System
/dev/xvda16  227328  2097152  1869825  913M Linux extended boot

The above output shows the partition information for the the main system disk (/dev/xvda) which is 30 GiB in size and has four partitions:

/dev/xvda1: 29G Linux filesystem (main system partition).
/dev/xvda14: 4M BIOS boot partition.
/dev/xvda15: 106M EFI System partition (for UEFI boot).
/dev/xvda16: 913M Linux extended boot partition.

`mount` Command

mount shows all currently mounted filesystems in the format: device/source "on" mount_point "type" filesystem_type (mount_options), displaying where and how each filesystem is attached to your system's directory tree.

Here is an example line from the output of mount:

/dev/xvda1 on / type ext4 (rw,relatime,discard,errors=remount-ro,commit=30)

Some common mount options you’ll see are:

rw: Read-write access.
ro: Read-only access.
nosuid: Disable SUID/SGID bits.
nodev: Prevent device file interpretation.
noexec: Prevent execution of binaries.
relatime: Update access times relatively.

`du -sh *` Command

du -sh * provides a summary of the disk usage for each file and directory in the current directory (good for finding disk hogs):

du -sh *
4.0K    file1.txt
8.0K    file2.txt
12K     directory1
20K     directory2

How to Get Your Hardware Information in Linux

`lshw` Command

The lshw command provides detailed information about the computer's hardware configuration. It can report:

Memory configuration.
Firmware version.
Mainboard configuration.
CPU version and speed.
Cache configuration.
Bus speed and more.

It's particularly useful for system administrators and users who need to gather detailed hardware information. The command can output information in various formats including HTML, XML, JSON, or plain text.

Here is a portion of the output from lshw:

*-pci
          description: Host bridge
          product: 440FX - 82441FX PMC [Natoma]
          vendor: Intel Corporation
          physical id: 100
          bus info: pci@0000:00:00.0
          version: 02
          width: 32 bits
          clock: 33MHz
        *-isa
             description: ISA bridge
             product: 82371SB PIIX3 ISA [Natoma/Triton II]
             vendor: Intel Corporation
             physical id: 1
             bus info: pci@0000:00:01.0
             version: 00
             width: 32 bits
             clock: 33MHz
             capabilities: isa bus_master
             configuration: latency=0

`lspci` Command

lspci displays information about all PCI (Peripheral Component Interconnect) buses and devices connected to your system.

lspci
00:00.0 Host bridge: Intel Corporation 440FX - 82441FX PMC [Natoma] (rev 02)
00:01.0 ISA bridge: Intel Corporation 82371SB PIIX3 ISA [Natoma/Triton II]
00:01.1 IDE interface: Intel Corporation 82371SB PIIX3 IDE [Natoma/Triton II]
00:01.3 Bridge: Intel Corporation 82371AB/EB/MB PIIX4 ACPI (rev 01)
00:02.0 VGA compatible controller: Cirrus Logic GD 5446
00:03.0 Unassigned class [ff80]: XenSource, Inc. Xen Platform Device (rev 01)

From the output, we can see that:

Each line starts with a bus:device.function address (like "00:00.0")
Following the address is the device class and the specific hardware details:
- A Host bridge (Intel 440FX), which manages communications between the CPU and other components.
- An ISA bridge (Intel PIIX3), for legacy device support.
- An IDE interface for storage devices.
- An ACPI bridge for power management.
- A VGA graphics controller (Cirrus Logic).
- A Xen Platform Device (this suggests you're running in a Xen virtualized environment).

The command is particularly useful for:

Troubleshooting hardware issues
Verifying hardware detection
Finding hardware details for driver installation
Checking system configuration

How to Get Your Network Interfaces & Status Information in Linux

`ip a` Command

ip a displays information about all network interfaces on your system:

ip -a
1: lo: 
- This is the loopback interface (localhost)
- MTU (Maximum Transmission Unit) is 65536 bytes
- IP address: 127.0.0.1/8 (IPv4)
- IPv6 address: ::1/128

2. Network Interface (enX0):
enX0: 
- This is your main network interface
- MTU is 9001 bytes
- MAC address (link/ether): 12:16:a6:d3:b3:61
- IPv4 address: 172.31.90.178/20
- IPv6 address: fe80::1016:a6ff:fed3:b361/64 (Link-local)

Here are the key elements in the output:

Interface state (UP/DOWN).
MAC address (link/ether).
IPv4 and IPv6 addresses.
Network scope (host, global, link).
Address validity lifetime (valid_lft).
Broadcast address (brd).

`ip r` Command

ip r shows the system’s routing table:

ip r
default via 172.31.80.1 dev enX0 proto dhcp src 172.31.90.178 metric 100 
172.31.0.2 via 172.31.80.1 dev enX0 proto dhcp src 172.31.90.178 metric 100 
172.31.80.0/20 dev enX0 proto kernel scope link src 172.31.90.178 metric 100 
172.31.80.1 dev enX0 proto dhcp scope link src 172.31.90.178 metric 100

The above ip r output shows my system's routing table with the following routes:

Default Route (Gateway):
- Default via 172.31.80.1: All traffic not matching other rules goes through this gateway.
- Using interface enX0.
- Configured via DHCP.
- Source IP: 172.31.90.178.
Local Network:
- 172.31.80.0/20: Local subnet (covers IPs from 172.31.80.0 to 172.31.95.255)
- Directly connected to enX0 interface
- Kernel-managed route (proto kernel)
- For packets originating from 172.31.90.178
DHCP Route:
- Direct route to DHCP server (172.31.80.1)
- Via interface enX0

All routes have a metric of 100, which determines route priority (lower values are preferred).

netstat -tuln shows active listening ports:

netstat -tuln
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State      
tcp        0      0 127.0.0.54:53           0.0.0.0:*               LISTEN     
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN     
tcp        0      0 127.0.0.53:53           0.0.0.0:*               LISTEN     
tcp6       0      0 :::80                   :::*                    LISTEN     
tcp6       0      0 :::22                   :::*                    LISTEN     
udp        0      0 127.0.0.54:53           0.0.0.0:*                          
udp        0      0 127.0.0.53:53           0.0.0.0:*                          
udp        0      0 172.31.90.178:68        0.0.0.0:*                          
udp        0      0 127.0.0.1:323           0.0.0.0:*                          
udp6       0      0 ::1:323                 :::*

How to Get Your Software & Services Information in Linux

Installed packages

You can check installed packages with dpkg -l, apt list --installed (Debian/Ubuntu). Here is a snippet from the output:

vim-common/noble-updates,noble-security,now 2:9.1.0016-1ubuntu7.8 all [installed,automatic]
vim-runtime/noble-updates,noble-security,now 2:9.1.0016-1ubuntu7.8 all [installed,automatic]
vim-tiny/noble-updates,noble-security,now 2:9.1.0016-1ubuntu7.8 amd64 [installed,automatic]
vim/noble-updates,noble-security,now 2:9.1.0016-1ubuntu7.8 amd64 [installed,automatic]

Service status

systemctl list-units --type=service lists the services. You can also use systemctl status and replace with the one you want.

Here’s the output for cron.service:

systemctl status cron.service
● cron.service - Regular background program processing daemon
     Loaded: loaded (/usr/lib/systemd/system/cron.service; enabled; preset: enabled)
     Active: active (running) since Wed 2025-05-14 19:46:58 UTC; 2 weeks 5 days ago
       Docs: man:cron(8)
   Main PID: 625 (cron)
      Tasks: 1 (limit: 1129)
     Memory: 1.7M (peak: 4.7M)
        CPU: 20.890s
     CGroup: /system.slice/cron.service
             └─625 /usr/sbin/cron -f -P

Jun 03 09:25:01 ip-172-31-90-178 CRON[121748]: pam_unix(cron:session): session closed for user root
Jun 03 09:35:01 ip-172-31-90-178 CRON[121817]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
Jun 03 09:35:01 ip-172-31-90-178 CRON[121818]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Jun 03 09:35:01 ip-172-31-90-178 CRON[121817]: pam_unix(cron:session): session closed for user root
Jun 03 09:45:01 ip-172-31-90-178 CRON[122050]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
Jun 03 09:45:01 ip-172-31-90-178 CRON[122051]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Jun 03 09:45:01 ip-172-31-90-178 CRON[122050]: pam_unix(cron:session): session closed for user root
Jun 03 09:55:01 ip-172-31-90-178 CRON[122318]: pam_unix(cron:session): session opened for user root(uid=0) by root(uid=0)
Jun 03 09:55:01 ip-172-31-90-178 CRON[122319]: (root) CMD (command -v debian-sa1 > /dev/null && debian-sa1 1 1)
Jun 03 09:55:01 ip-172-31-90-178 CRON[122318]: pam_unix(cron:session): session closed for user root
lines 5-21/21 (END)

Processes

ps aux shows all processes with their respective status:

ps aux
USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root           1  0.0  1.4  22556 13952 ?        Ss   May14   0:35 /usr/lib/systemd/systemd --system --deserialize=63
root           2  0.0  0.0      0     0 ?        S    May14   0:00 [kthreadd]
root           3  0.0  0.0      0     0 ?        S    May14   0:00 [pool_workqueue_release]
root           4  0.0  0.0      0     0 ?        I<   May14   0:00 [kworker/R-rcu_g]
root           5  0.0  0.0      0     0 ?        I<   May14   0:00 [kworker/R-rcu_p]
root           6  0.0  0.0      0     0 ?        I<   May14   0:00 [kworker/R-slub_]
.
.
.

Here's an explanation of each column in the ps aux output:

USER: The owner of the process
PID: Process ID number
%CPU: CPU usage percentage
%MEM: Memory usage percentage
VSZ: Virtual Memory Size in kilobytes (total program size)
RSS: Resident Set Size in kilobytes (actual memory used)
TTY: Terminal associated with the process ('?' means no terminal)
STAT: Process state code:
- S: Sleeping
- R: Running
- I: Idle
- Z: Zombie
- T: Stopped
- s: Session leader
- <: High priority
- N: Low priority
START: Time when the process started
TIME: Cumulative CPU time used
COMMAND: The command with all its arguments

`top` and `htop` Commands

top or htop can be used for live usage overview, and for showing a dynamic view of system performance and running processes. Here's what it displays:

System Overview:
- System uptime and number of logged-in users.
- Load average values for the last 1, 5, and 15 minutes.
- Total number of processes and their states (running, sleeping, stopped, zombie)
Resource Usage:
- CPU usage breakdown (user, system, idle, etc.).
- Memory usage (total, free, used, cached).
- Swap space usage
- Process List:Shows a sorted list of running processes (by default sorted by CPU usage)For each process, displays:
  - Process ID (PID).
  - User who owns the process.
  - CPU and memory usage.
  - Process priority and nice value.
  - Memory usage details (virtual, resident, shared).
  - Process status.
  - Running time.
  - Command name.

    top - 10:04:25 up 19 days, 14:17,  1 user,  load average: 0.00, 0.00, 0.00
    Tasks: 104 total,   1 running, 103 sleeping,   0 stopped,   0 zombie
    %Cpu(s):  0.0 us,  0.0 sy,  0.0 ni, 88.0 id,  0.0 wa,  0.0 hi,  0.0 si, 12.0 st 
    MiB Mem :    957.4 total,    247.3 free,    366.1 used,    533.7 buff/cache     
    MiB Swap:      0.0 total,      0.0 free,      0.0 used.    591.3 avail Mem 

        PID USER      PR  NI    VIRT    RES    SHR S  %CPU  %MEM     TIME+ COMMAND                                              
          1 root      20   0   22556  13952   9728 S   0.0   1.4   0:35.08 systemd                                              
          2 root      20   0       0      0      0 S   0.0   0.0   0:00.16 kthreadd                                             
          3 root      20   0       0      0      0 S   0.0   0.0   0:00.00 pool_workqueue_release                               
          4 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 kworker/R-rcu_g                                      
          5 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 kworker/R-rcu_p                                      
          6 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 kworker/R-slub_                                      
          7 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 kworker/R-netns                                      
         10 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 kworker/0:0H-events_highpri                          
         12 root       0 -20       0      0      0 I   0.0   0.0   0:00.00 kworker/R-mm_pe                                      
         13 root      20   0       0      0      0 I   0.0   0.0   0:00.00 rcu_tasks_rude_kthread                               
         14 root      20   0       0      0      0 I   0.0   0.0   0:00.00 rcu_tasks_trace_kthread

The top command updates this information regularly (by default every 3 seconds) and is commonly used for:

Monitoring system performance
Identifying resource-intensive processes
Troubleshooting system slowdowns
Getting a quick overview of system health

You can also interact with top while it's running using various keyboard commands (like 'k' to kill a process, '1' to see cpu cores, etc.).

How to Get Your Logs & Dmesg In formation in Linux

Based on the system configuration, a number of logs are generated. These can be audit logs, system logs, cron logs, and so on. They all carry useful information. Here are some commands that you can use to view logs:

dmesg | less: Kernel ring buffer (hardware issues, boot messages)
journalctl -xe: Recent critical logs (systemd systems)
/var/log/syslog or /var/log/messages: General system logs

How to Get Your Security/User Audit Information in Linux

whoami shows the current user’s username.

whoami
ubuntu

id shows detailed information about a user's identity on the system.

id
uid=1000(ubuntu) gid=1000(ubuntu) groups=1000(ubuntu),4(adm),24(cdrom),27(sudo),30(dip),105(lxd)

Let's break down the output:

User ID (uid): uid=1000(ubuntu) means the user ID is 1000, with username "ubuntu"
Primary Group ID (gid): gid=1000(ubuntu) means the primary group ID is 1000, named "ubuntu"
Supplementary Groups (groups): The user belong to the following groups:
- ubuntu (1000): Your primary group.
- adm (4): For system monitoring tasks.
- cdrom (24): For accessing CD-ROM devices.
- sudo (27): Allows you to execute commands with superuser privileges.
- dip (30): For managing dial-up connections.
- lxd (105): For managing LXD containers.

The id command is useful for checking user and group IDs, verifying group memberships, troubleshooting permissions issues and confirming sudo access.

who displays information about users currently logged into the system:

who
ubuntu   pts/0        2025-06-03 08:45 (39.43.159.5)

The output breakdown is shown below:

Username: "ubuntu"
Terminal: "pts/0" (pseudo-terminal)
Login time: "2025-06-03 08:45"
Remote host: "(39.43.159.5)" - the IP address from where the connection was made
w- shows who is logged in and what they are doing:

w
 10:21:46 up 19 days, 14:35,  1 user,  load average: 0.00, 0.00, 0.00
USER     TTY      FROM             LOGIN@   IDLE   JCPU   PCPU  WHAT
ubuntu   pts/0    39.43.159.5      08:45   44:56   0.00s  0.02s sshd: ubuntu [priv]

Here is the result breakdown:

First line:

10:21:46: Current system time
up 19 days, 14:35: System uptime (how long the system has been running)
1 user: Number of users currently logged in
load average: 0.24, 0.05, 0.02: System load averages for the past 1, 5, and 15 minutes
- Numbers below 1.0 indicate low system load
- Higher numbers indicate more system load/stress

Second line shows the column headers for the user information below:

USER: Username.
TTY: Terminal device being used.
FROM: Remote host from where the user is connected.
LOGIN@: Time when the user logged in.
IDLE: Time since the user's last activity.
JCPU: CPU time used by all processes attached to the tty.
PCPU: CPU time used by the current process.
WHAT: Current process/command being run.

last shows a history of user logins and system reboots:

last
ubuntu   pts/1        39.43.159.5      Tue Jun  3 10:15 - 10:17  (00:02)
ubuntu   pts/0        39.43.159.5      Tue Jun  3 08:45   still logged in
ubuntu   pts/0        39.43.159.5      Tue Jun  3 05:23 - 08:29  (03:06)
ubuntu   pts/0        39.43.159.5      Sun Jun  1 06:32 - 12:24  (05:52)
ubuntu   pts/0        39.43.159.5      Thu May 22 05:39 - 05:58  (00:18)
ubuntu   pts/0        139.135.32.93    Wed May 21 14:45 - 14:47  (00:01)
ubuntu   pts/0        139.135.32.93    Wed May 21 11:58 - 13:49  (01:51)
ubuntu   pts/0        39.43.159.5      Wed May 21 05:05 - 05:12  (00:06)
ubuntu   pts/0        39.43.159.5      Tue May 20 18:41 - 21:45  (03:04)
ubuntu   pts/0        39.43.159.5      Thu May 15 06:12 - 06:12  (00:00)
ubuntu   pts/0        39.43.159.5      Thu May 15 06:05 - 06:12  (00:07)
ubuntu   pts/0        18.206.107.27    Wed May 14 20:06 - 20:08  (00:01)
ubuntu   pts/0        182.185.185.39   Wed May 14 19:48 - 19:50  (00:01)
reboot   system boot  6.8.0-1024-aws   Wed May 14 19:46   still running

wtmp begins Wed May 14 19:46:47 2025

Each line shows:

Username (in this case, all logins are from 'ubuntu' user).
Terminal device (pts/0 indicates a pseudo-terminal, typically used for SSH connections).
Remote host IP address (where the connection came from).
Login time and date.
Logout time or status.
Session duration in parentheses.

sudo -l shows what the current user can do with sudo.

sudo -l
Matching Defaults entries for ubuntu on ip-172-31-90-178:
    env_reset, mail_badpass, secure_path=/usr/local/sbin\:/usr/local/bin\:/usr/sbin\:/usr/bin\:/sbin\:/bin\:/snap/bin,
    use_pty

User ubuntu may run the following commands on ip-172-31-90-178:
    (ALL : ALL) ALL
    (ALL) NOPASSWD: ALL

This output indicates that the 'ubuntu' user has:

Full sudo access (can execute any command)
No password requirement for sudo commands
Complete administrative privileges on the system

Visually Appealing Commands

In this section you’ll learn about two commands that display the information we have seen before in a presentable and aesthetic form.

neofetch - displays system info along with the distribution logo:

btop displays dynamic stats with different modes:

Conclusion

Thank you for reading the article until the end. If you found it helpful, consider sharing it with others.

Stay Connected and Continue Your Learning Journey!

I read every message, come say hi 👋

Connect with me on:
- LinkedIn: I share content related to Linux, Cyber security and DevOps. Leave a recommendation on LinkedIn and endorse me on relevant skills.
- Discord community: Hang around with other devs or share your accomplishments.
- X: I share pre-launch updates and some behind the scenes.
Get access to exclusive content: For one-on-one help and exclusive content go here.

My articles are part of my mission to increase accessibility to quality content for everyone. Each piece takes a lot of time and effort to write. This article will be free, forever. If you've enjoyed my work and want to keep me motivated, consider buying me a coffee.

Thank you once again and happy learning!

Learn Linux for Beginners: From Basics to Advanced Techniques [Full Book]

Zaira Hira — Fri, 12 Jul 2024 13:18:40 +0000

Learning Linux is one of the most valuable skills in the tech industry. It can help you get things done faster and more efficiently. Many of the world's powerful servers and supercomputers run on Linux.

While empowering you in your current role, learning Linux can also help you transition into other tech careers like DevOps, Cybersecurity, and Cloud Computing.

In this handbook, you'll learn the basics of the Linux command line, and then transition to more advanced topics like shell scripting and system administration. Whether you are new to Linux or have been using it for years, this book has something for you.

Important Note: All examples in this book are demonstrated in Ubuntu 22.04.2 LTS (Jammy Jellyfish). Most command line tools are more or less the same in other distributions. However, some GUI applications and commands may differ if you are working on another Linux distribution.

Part 1: Introduction to Linux
- 1.1. Getting Started with Linux
Part 2: Introduction to Bash Shell and System Commands
Part 3: Understanding Your Linux System
- 3.1. Discovering Your OS and Specs
Part 4: Managing Files From the Command line
Part 5: The Essentials of Text Editing in Linux
- 5.1. Mastering Vim: The Complete Guide
- 5.2. Mastering Nano
Part 6: Bash Scripting
Part 7: Managing Software Packages in Linux
Part 8: Advanced Linux Topics
Conclusion

Part 1: Introduction to Linux

1.1. Getting Started with Linux

What is Linux?

Linux is an open-source operating system that is based on the Unix operating system. It was created by Linus Torvalds in 1991.

Open source means that the source code of the operating system is available to the public. This allows anyone to modify the original code, customise it, and distribute the new operating system to potential users.

Why should you learn about Linux?

In today's data center landscape, Linux and Microsoft Windows stand out as the primary contenders, with Linux having a major share.

Here are several compelling reasons to learn Linux:

Given the prevalence of Linux hosting, there is a high chance that your application will be hosted on Linux. So learning Linux as a developer becomes increasingly valuable.
With cloud computing becoming the norm, chances are high that your cloud instances will rely on Linux.
Linux serves as the foundation for many operating systems for the Internet of Things (IoT) and mobile applications.
In IT, there are many opportunities for those skilled in Linux.

What does it mean that Linux is an open-source operating system?

First, what is open source? Open source software is software whose source code is freely accessible, allowing anyone to utilize, modify, and distribute it.

Whenever source code is created, it is automatically considered copyrighted, and its distribution is governed by the copyright holder through software licenses.

In contrast to open source, proprietary or closed-source software restricts access to its source code. Only the creators can view, modify, or distribute it.

Linux is primarily open source, which means that its source code is freely available. Anyone can view, modify, and distribute it. Developers from anywhere in the world can contribute to its improvement. This lays the foundation of collaboration which is an important aspect of open source software.

This collaborative approach has led to the widespread adoption of Linux across servers, desktops, embedded systems, and mobile devices.

The most interesting aspect of Linux being open source is that anyone can tailor the operating system to their specific needs without being restricted by proprietary limitations.

Chrome OS used by Chromebooks is based on Linux. Android, that powers many smartphones globally, is also based on Linux.

What is a Linux Kernel?

The kernel is the central component of an operating system that manages the computer and its hardware operations. It handles memory operations and CPU time.

The kernel acts as a bridge between applications and the hardware-level data processing using inter-process communication and system calls.

The kernel loads into memory first when an operating system starts and remains there until the system shuts down. It is responsible for tasks like disk management, task management, and memory management.

If you are curious about what the Linux kernel looks like, here is the GitHub link.

What is a Linux distribution?

By this point, you know that you can re-use the Linux kernel code, modify it, and create a new kernel. You can further combine different utilities and software to create a completely new operating system.

A Linux distribution or distro is a version of the Linux operating system that includes the Linux kernel, system utilities, and other software. Being open source, a Linux distribution is a collaborative effort involving multiple independent open-source development communities.

What does it mean that a distribution is derived? When you say that a distribution is "derived" from another, the newer distro is built upon the base or foundation of the original distro. This derivation can include using the same package management system (more on this later), kernel version, and sometimes the same configuration tools.

Today, there are thousands of Linux distributions to choose from, offering differing goals and criteria for selecting and supporting the software provided by their distribution.

Distributions vary from one to the other, but they generally have several common characteristics:

A distribution consists of a Linux kernel.
It supports user space programs.
A distribution may be small and single-purpose or include thousands of open-source programs.
Some means of installing and updating the distribution and its components should be provided.

If you view the Linux Distributions Timeline, you'll see two major distros: Slackware and Debian. Several distributions are derived from them. For example, Ubuntu and Kali are derived from Debian.

What are the advantages of derivation? There are various advantages of derivation. Derived distributions can leverage the stability, security, and large software repositories of the parent distribution.

When building on an existing foundation, developers can drive their focus and effort entirely on the specialized features of the new distribution. Users of derived distributions can benefit from the documentation, community support, and resources already available for the parent distribution.

Some popular Linux distributions are:

Ubuntu: One of the most widely used and popular Linux distributions. It is user-friendly and recommended for beginners. Learn more about Ubuntu here.
Linux Mint: Based on Ubuntu, Linux Mint provides a user-friendly experience with a focus on multimedia support. Learn more about Linux Mint here.
Arch Linux: Popular among experienced users, Arch is a lightweight and flexible distribution aimed at users who prefer a DIY approach. Learn more about Arch Linux here.
Manjaro: Based on Arch Linux, Manjaro provides a user-friendly experience with pre-installed software and easy system management tools. Learn more about Manjaro here.
Kali Linux: Kali Linux provides a comprehensive suite of security tools and is mostly focused on cybersecurity and hacking. Learn more about Kali Linux here.

How to install and access Linux

The best way to learn is to apply the concepts as you go. In this section, we'll learn how to install Linux on your machine so you can follow along. You'll also learn how to access Linux on a Windows machine.

I recommend that you follow any one of the methods mentioned in this section to get access to Linux so you may follow along.

Install Linux as the primary OS

Installing Linux as the primary OS is the most efficient way to use Linux, as you can use the full power of your machine.

In this section, you will learn how to install Ubuntu, which is one of the most popular Linux distributions. I have left out other distributions for now, as I want to keep things simple. You can always explore other distributions once you are comfortable with Ubuntu.

Step 1 – Download the Ubuntu iso: Go to the official website and download the iso file. Make sure to select a stable release that is labeled "LTS". LTS stands for Long Term Support which means you can get free security and maintenance updates for a long time (usually 5 years).
Step 2 – Create a bootable pendrive: There are a number of softwares that can create a bootable pendrive. I recommend using Rufus, as it is quite easy to use. You can download it from here.
Step 3 – Boot from the pendrive: Once your bootable pendrive is ready, insert it and boot from the pendrive. The boot menu depends on your laptop. You can google the boot menu for your laptop model.
Step 4 – Follow the prompts. Once, the boot process starts, select try or install ubuntu.

The process will take some time. Once the GUI appears, you can select the language, and keyboard layout and continue. Enter your login and name. Remember the credentials as you will need them to log in to your system and access full privileges. Wait for the installation to complete.
Step 5 – Restart: Click on restart now and remove the pen drive.
Step 6 – Login: Login with the credentials you entered earlier.

And there you go! Now you can install apps and customize your desktop.

For advanced installation, you can explore the following topics:

Disk partitioning.
Setting swap memory for enabling hibernation.

Accessing the terminal

An important part of this handbook is learning about the terminal where you'll run all the commands and see the magic happen. You can search for the terminal by pressing the "windows" key and typing "terminal". You can pin the Terminal in the dock where other apps are located for easy access.

💡 The shortcut for opening the terminal is ctrl+alt+t

You can also open the terminal from inside a folder. Right click where you are and click on "Open in Terminal". This will open the terminal in the same path.

How to use Linux on a Windows machine

Sometimes you might need to run both Linux and Windows side by side. Luckily, there are some ways you can get the best of both worlds without getting different computers for each operating system.

In this section, you'll explore a few ways to use Linux on a Windows machine. Some of them are browser-based or cloud-based and do not need any OS installation before using them.

Option 1: "Dual-boot" Linux + Windows With dual boot, you can install Linux alongside Windows on your computer, allowing you to choose which operating system to use at startup.

This requires partitioning your hard drive and installing Linux on a separate partition. With this approach, you can only use one operating system at a time.

Option 2: Use Windows Subsystem for Linux (WSL) Windows Subsystem for Linux provides a compatibility layer that lets you run Linux binary executables natively on Windows.

Using WSL has some advantages. The setup for WSL is simple and not time-consuming. It is lightweight compared to VMs where you have to allocate resources from the host machine. You don't need to install any ISO or virtual disc image for Linux machines which tend to be heavy files. You can use Windows and Linux side by side.

How to install WSL2

First, enable the Windows Subsystem for Linux option in settings.

Go to Start. Search for "Turn Windows features on or off."
Check the option "Windows Subsystem for Linux" if it isn't already.
Next, open your command prompt and provide the installation commands.
Open Command Prompt as an administrator:
Run the command below:

wsl --install

This is the output:

Note: By default, Ubuntu will be installed.

Once installation is complete, you'll need to reboot your Windows machine. So, restart your Windows machine.

After restarting, you might see a window like this:

Once installation of Ubuntu is complete, you'll be prompted to enter your username and password.

And, that's it! You are ready to use Ubuntu.

Launch Ubuntu by searching from the start menu.

And here we have your Ubuntu instance launched.

Option 3: Use a Virtual Machine (VM)

A virtual machine (VM) is a software emulation of a physical computer system. It allows you to run multiple operating systems and applications on a single physical machine simultaneously.

You can use virtualization software such as Oracle VirtualBox or VMware to create a virtual machine running Linux within your Windows environment. This allows you to run Linux as a guest operating system alongside Windows.

VM software provides options to allocate and manage hardware resources for each VM, including CPU cores, memory, disk space, and network bandwidth. You can adjust these allocations based on the requirements of the guest operating systems and applications.

Here are some of the common options available for virtualization:

Option 4: Use a Browser-based Solution

Browser-based solutions are particularly useful for quick testing, learning, or accessing Linux environments from devices that don't have Linux installed.

You can either use online code editors or web-based terminals to access Linux. Note that you usually don't have full administration privileges in these cases.

Online code editors

Online code editors offer editors with built-in Linux terminals. While their primary purpose is coding, you can also utilize the Linux terminal to execute commands and perform tasks.

Replit is an example of an online code editor, where you can write your code and access the Linux shell at the same time.

Web-based Linux terminals:

Online Linux terminals allow you to access a Linux command-line interface directly from your browser. These terminals provide a web-based interface to a Linux shell, enabling you to execute commands and work with Linux utilities.

One such example is JSLinux. The screenshot below shows a ready-to-use Linux environment:

Option 5: Use a Cloud-based Solution

Instead of running Linux directly on your Windows machine, you can consider using cloud-based Linux environments or virtual private servers (VPS) to access and work with Linux remotely.

Services like Amazon EC2, Microsoft Azure, or DigitalOcean provide Linux instances that you can connect to from your Windows computer. Note that some of these services offer free tiers, but they are not usually free in the long run.

Part 2: Introduction to Bash Shell and System Commands

2.1. Getting Started with the Bash shell

Introduction to the bash shell

The Linux command line is provided by a program called the shell. Over the years, the shell program has evolved to cater to various options.

Different users can be configured to use different shells. But, most users prefer to stick with the current default shell. The default shell for many Linux distros is the GNU Bourne-Again Shell (bash). Bash is succeeded by the Bourne shell (sh).

To find out your current shell, open your terminal and enter the following command:

echo $SHELL

Command breakdown:

The echo command is used to print on the terminal.
The $SHELL is a special variable that holds the name of the current shell.

In my setup, the output is /bin/bash. This means that I am using the bash shell.

# output
echo $SHELL
/bin/bash

Bash is very powerful as it can simplify certain operations that are hard to accomplish efficiently with a GUI (or Graphical User Interface). Remember that most servers do not have a GUI, and it is best to learn to use the powers of a command line interface (CLI).

Terminal vs Shell

The terms "terminal" and "shell" are often used interchangeably, but they refer to different parts of the command-line interface.

The terminal is the interface you use to interact with the shell. The shell is the command interpreter that processes and executes your commands. You'll learn more about shells in Part 6 of the handbook.

What is a prompt?

When a shell is used interactively, it displays a $ when it is waiting for a command from the user. This is called the shell prompt.

[username@host ~]$

If the shell is running as root (you'll learn more about the root user later on), the prompt is changed to #.

[root@host ~]#

2.2. Command Structure

A command is a program that performs a specific operation. Once you have access to the shell, you can enter any command after the $ sign and see the output on the terminal.

Generally, Linux commands follow this syntax:

command [options] [arguments]

Here is the breakdown of the above syntax:

command: This is the name of the command you want to execute. ls (list), cp (copy), and rm (remove) are common Linux commands.
[options]: Options, or flags, often preceded by a hyphen (-) or double hyphen (--), modify the behavior of the command. They can change how the command operates. For example, ls -a uses the -a option to display hidden files in the current directory.
[arguments]: Arguments are the inputs for the commands that require one. These could be filenames, user names, or other data that the command will act upon. For example, in the command cat access.log, cat is the command and access.log is the input. As a result, the cat command displays the contents of the access.log file.

Options and arguments are not required for all commands. Some commands can be run without any options or arguments, while others might require one or both to function correctly. You can always refer to the command's manual to check the options and arguments it supports.

💡Tip: You can view a command's manual using the man command.

You can access the manual page for ls with man ls, and it'll look like this:

Manual pages are a great and quick way to access the documentation. I highly recommend going through man pages for the commands that you use the most.

2.3. Bash Commands and Keyboard Shortcuts

When you are in the terminal, you can speed up your tasks by using shortcuts.

Here are some of the most common terminal shortcuts:

Operation	Shortcut
Look for the previous command	Up Arrow
Jump to the beginning of the previous word	Ctrl+LeftArrow
Clear characters from the cursor to the end of the command line	Ctrl+K
Complete commands, file names, and options	Pressing Tab
Jumps to the beginning of the command line	Ctrl+A
Displays the list of previous commands	history

2.4. Identifying Yourself: The `whoami` Command

You can get the username you are logged in with by using the whoami command. This command is useful when you are switching between different users and want to confirm the current user.

Just after the $ sign, type whoami and press enter.

whoami

This is the output I got.

zaira@zaira-ThinkPad:~$ whoami
zaira

Part 3: Understanding Your Linux System

3.1. Discovering Your OS and Specs

Print system information using the `uname` Command

You can get detailed system information from the uname command.

When you provide the -a option, it prints all the system information.

uname -a
# output
Linux zaira 6.5.0-21-generic #21~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Fri Feb  9 13:32:52 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

In the output above,

Linux: Indicates the operating system.
zaira: Represents the hostname of the machine.
6.5.0-21-generic #21~22.04.1-Ubuntu SMP PREEMPT_DYNAMIC Fri Feb 9 13:32:52 UTC 2: Provides information about the kernel version, build date, and some additional details.
x86_64 x86_64 x86_64: Indicates the architecture of the system.
GNU/Linux: Represents the operating system type.

Find details of the CPU architecture using the `lscpu` Command

The lscpu command in Linux is used to display information about the CPU architecture. When you run lscpu in the terminal, it provides details such as:

The architecture of the CPU (for example, x86_64)
CPU op-mode(s) (for example, 32-bit, 64-bit)
Byte Order (for example, Little Endian)
CPU(s) (number of CPUs), and so on

Let's try it out:

lscpu
# output
Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         48 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  12
  On-line CPU(s) list:   0-11
Vendor ID:               AuthenticAMD
  Model name:            AMD Ryzen 5 5500U with Radeon Graphics
    Thread(s) per core:  2
    Core(s) per socket:  6
    Socket(s):           1
    Stepping:            1
    CPU max MHz:         4056.0000
    CPU min MHz:         400.0000

That was a whole lot of information, but useful too! Remember you can always skim the relevant information using specific flags. See the command manual with man lscpu.

Part 4: Managing Files From the Command line

4.1. The Linux File-system Hierarchy

All files in Linux are stored in a file-system. It follows an inverted-tree-like structure because the root is at the topmost part.

The / is the root directory and the starting point of the file system. The root directory contains all other directories and files on the system. The / character also serves as a directory separator between path names. For example, /home/alice forms a complete path.

The image below shows the complete file system hierarchy. Each directory servers a specific purpose.

Note that this is not an exhaustive list and different distributions may have different configurations.

Here is a table that shows the purpose of each directory:

Location	Purpose
/bin	Essential command binaries
/boot	Static files of the boot loader, needed in order to start the boot process.
/etc	Host-specific system configuration
/home	User home directories
/root	Home directory for the administrative root user
/lib	Essential shared libraries and kernel modules
/mnt	Mount point for mounting a filesystem temporarily
/opt	Add-on application software packages
/usr	Installed software and shared libraries
/var	Variable data that is also persistent between boots
/tmp	Temporary files that are accessible to all users

💡 Tip: You can learn more about the file system using the man hier command.

You can check your file system using the tree -d -L 1 command. You can modify the -L flag to change the depth of the tree.

tree -d -L 1
# output
.
├── bin -> usr/bin
├── boot
├── cdrom
├── data
├── dev
├── etc
├── home
├── lib -> usr/lib
├── lib32 -> usr/lib32
├── lib64 -> usr/lib64
├── libx32 -> usr/libx32
├── lost+found
├── media
├── mnt
├── opt
├── proc
├── root
├── run
├── sbin -> usr/sbin
├── snap
├── srv
├── sys
├── tmp
├── usr
└── var

25 directories

This list is not exhaustive and different distributions and systems may be configured differently.

4.2. Navigating the Linux File-system

Absolute path vs relative path

The absolute path is the full path from the root directory to the file or directory. It always starts with a /. For example, /home/john/documents.

The relative path, on the other hand, is the path from the current directory to the destination file or directory. It does not start with a /. For example, documents/work/project.

Locating your current directory using the `pwd` command

It is easy to lose your way in the Linux file system, especially if you are new to the command line. You can locate your current directory using the pwd command.

Here is an example:

pwd
# output
/home/zaira/scripts/python/free-mem.py

Changing directories using the `cd` command

The command to change directories is cd and it stands for "change directory". You can use the cd command to navigate to a different directory.

You can use a relative path or an absolute path.

For example, if you want to navigate the below file structure (following the red lines):

and you are standing at "home", the command would be like this:

cd home/bob/documents/work/project

Some other commonly used cd shortcuts are:

Command	Description
`cd ..`	Go back one directory
`cd ../..`	Go back two directories
`cd` or `cd ~`	Go to the home directory
`cd -`	Go to the previous path

4.3. Managing Files and Directories

When working with files and directories, you might want to copy, move, remove, and create new files and directories. Here are some commands that can help you with that.

💡Tip: You can differentiate between a file and folder by looking at the first letter in the output of ls -l. A'-' represents a file and a 'd' represents a folder.

Creating new directories using the `mkdir` command

You can create an empty directory using the mkdir command.

# creates an empty directory named "foo" in the current folder
mkdir foo

You can also create directories recursively using the -p option.

mkdir -p tools/index/helper-scripts
# output of tree
.
└── tools
    └── index
        └── helper-scripts

3 directories, 0 files

Creating new files using the `touch` command

The touch command creates an empty file. You can use it like this:

# creates empty file "file.txt" in the current folder
touch file.txt

The file names can be chained together if you want to create multiple files in a single command.

# creates empty files "file1.txt", "file2.txt", and "file3.txt" in the current folder

touch file1.txt file2.txt file3.txt

Removing files and directories using the `rm` and `rmdir` command

You can use the rm command to remove both files and non-empty directories.

Command	Description
`rm file.txt`	Removes the file `file.txt`
`rm -r directory`	Removes the directory `directory` and its contents
`rm -f file.txt`	Removes the file `file.txt` without prompting for confirmation
`rmdir` directory	Removes an empty directory

🛑 Note that you should use the -f flag with caution as you won't be asked before deleting a file. Also, be careful when running rm commands in the root folder as it might result in deleting important system files.

Copying files using the `cp` command

To copy files in Linux, use the cp command.

Syntax to copy files:cp source_file destination_of_file

This command copies a file named file1.txt to a new file location /home/adam/logs.

cp file1.txt /home/adam/logs

The cp command also creates a copy of one file with the provided name.

This command copies a file named file1.txt to another file named file2.txt in the same folder.

cp file1.txt file2.txt

Moving and renaming files and folders using the `mv` command

The mv command is used to move files and folders from one directory to the other.

Syntax to move files:mv source_file destination_directory

Example: Move a file named file1.txt to a directory named backup:

mv file1.txt backup/

To move a directory and its contents:

mv dir1/ backup/

Renaming files and folders in Linux is also done with the mv command.

Syntax to rename files:mv old_name new_name

Example: Rename a file from file1.txt to file2.txt:

mv file1.txt file2.txt

Rename a directory from dir1 to dir2:

mv dir1 dir2

4.4. Locating Files and Folders Using the `find` Command

The find command lets you efficiently search for files, folders, and character and block devices.

Below is the basic syntax of the find command:

find /path/ -type f -name file-to-search

Where,

/path is the path where the file is expected to be found. This is the starting point for searching files. The path can also be/or . which represents the root and current directory, respectively.
-type represents the file descriptors. They can be any of the below:
f – Regular file such as text files, images, and hidden files.
d – Directory. These are the folders under consideration.
l – Symbolic link. Symbolic links point to files and are similar to shortcuts.
c – Character devices. Files that are used to access character devices are called character device files. Drivers communicate with character devices by sending and receiving single characters (bytes, octets). Examples include keyboards, sound cards, and the mouse.
b – Block devices. Files that are used to access block devices are called block device files. Drivers communicate with block devices by sending and receiving entire blocks of data. Examples include USB and CD-ROM
-name is the name of the file type that you want to search.

How to search files by name or extension

Suppose we need to find files that contain "style" in their name. We'll use this command:

find . -type f -name "style*"
#output
./style.css
./styles.css

Now let's say we want to find files with a particular extension like .html. We'll modify the command like this:

find . -type f -name "*.html"
# output
./services.html
./blob.html
./index.html

How to search hidden files

A dot at the beginning of the filename represents hidden files. They are normally hidden but can be viewed with ls -a in the current directory.

We can modify the find command as shown below to search for hidden files:

find . -type f -name ".*"

List and find hidden files

ls -la
# folder contents
total 5
drwxrwxr-x  2 zaira zaira 4096 Mar 26 14:17 .
drwxr-x--- 61 zaira zaira 4096 Mar 26 14:12 ..
-rw-rw-r--  1 zaira zaira    0 Mar 26 14:17 .bash_history
-rw-rw-r--  1 zaira zaira    0 Mar 26 14:17 .bash_logout
-rw-rw-r--  1 zaira zaira    0 Mar 26 14:17 .bashrc

find . -type f -name ".*"
# find output
./.bash_logout
./.bashrc
./.bash_history

Above you can see a list of hidden files in my home directory.

How to search log files and configuration files

Log files usually have the extension .log, and we can find them like this:

 find . -type f -name "*.log"

Similarly, we can search for configuration files like this:

 find . -type f -name "*.conf"

How to search other files by type

We can search for character block files by providing c to -type:

find / -type c

Similarly, we can find device block files by using b:

find / -type b

How to search directories

In the example below, we are finding the folders using the -type d flag.

ls -l
# list folder contents
drwxrwxr-x 2 zaira zaira 4096 Mar 26 14:22 hosts
-rw-rw-r-- 1 zaira zaira    0 Mar 26 14:23 hosts.txt
drwxrwxr-x 2 zaira zaira 4096 Mar 26 14:22 images
drwxrwxr-x 2 zaira zaira 4096 Mar 26 14:23 style
drwxrwxr-x 2 zaira zaira 4096 Mar 26 14:22 webp 

find . -type d 
# find directory output
.
./webp
./images
./style
./hosts

How to search files by size

An incredibly helpful use of the find command is to list files based on a particular size.

find / -size +250M

Here, we are listing files whose size exceeds 250MB.

Other units include:

G: GigaBytes.
M: MegaBytes.
K: KiloBytes
c : bytes.

Just replace with the relevant unit.

find  -type f -size +N

How to search files by modification time

By using the -mtime flag, you can filter files and folders based on the modification time.

find /path -name "*.txt" -mtime -10

For example,

-mtime +10 means you are looking for a file modified 10 days ago.
-mtime -10 means less than 10 days.
-mtime 10 If you skip + or – it means exactly 10 days.

4.5. Basic Commands for Viewing Files

Concatenate and display files using the `cat` command

The cat command in Linux is used to display the contents of a file. It can also be used to concatenate files and create new files.

Here is the basic syntax of the cat command:

cat [options] [file]

The simplest way to use cat is without any options or arguments. This will display the contents of the file on the terminal.

For example, if you want to view the contents of a file named file.txt, you can use the following command:

cat file.txt

This will display all the contents of the file on the terminal at once.

Viewing text files interactively using `less` and `more`

While cat displays the entire file at once, less and more allow you to view the contents of a file interactively. This is useful when you want to scroll through a large file or search for specific content.

The syntax of the less command is:

less [options] [file]

The more command is similar to less but has fewer features. It is used to display the contents of a file one screen at a time.

The syntax of the more command is:

more [options] [file]

For both commands, you can use the spacebar to scroll one page down, the Enter key to scroll one line down, and the q key to exit the viewer.

To move backward you can use the b key, and to move forward you can use the f key.

Displaying the last part of files using `tail`

Sometimes you might need to view just the last few lines of a file instead of the entire file. The tail command in Linux is used to display the last part of a file.

For example, tail file.txt will display the last 10 lines of the file file.txt by default.

If you want to display a different number of lines, you can use the -n option followed by the number of lines you want to display.

# Display the last 50 lines of the file file.txt
tail -n 50 file.txt

💡Tip: Another usage of the tail is its follow-along (-f) option. This option enables you to view the contents of a file as they are being written. This is a useful utility for viewing and monitoring log files in real-time.

Displaying the beginning of files using `head`

Just like tail displays the last part of a file, you can use the head command in Linux to display the beginning of a file.

For example, head file.txt will display the first 10 lines of the file file.txt by default.

To change the number of lines displayed, you can use the -n option followed by the number of lines you want to display.

Counting words, lines, and characters using `wc`

You can count words, lines and characters in a file using the wc command.

For example, running wc syslog.log gave me the following output:

1669 9623 64367 syslog.log

In the output above,

1669 represents the number of lines in the file syslog.log.
9623 represents the number of words in the file syslog.log.
64367 represents the number of characters in the file syslog.log.

So, the command wc syslog.log counted 1669 lines, 9623 words, and 64367 characters in the file syslog.log.

Comparing files line by line using `diff`

Comparing and finding differences between two files is a common task in Linux. You can compare two files right within the command line using the diff command.

The basic syntax of the diff command is:

diff [options] file1 file2

Here are two files, hello.py and also-hello.py, that we will compare using the diff command:

# contents of hello.py

def greet(name):
    return f"Hello, {name}!"

user = input("Enter your name: ")
print(greet(user))

# contents of also-hello.py

more also-hello.py
def greet(name):
    return fHello, {name}!

user = input(Enter your name: )
print(greet(user))
print("Nice to meet you")

Check whether the files are the same or not

diff -q hello.py also-hello.py
# Output
Files hello.py and also-hello.py differ

See how the files differ. For that, you can use the -u flag to see a unified output:

diff -u hello.py also-hello.py
--- hello.py    2024-05-24 18:31:29.891690478 +0500
+++ also-hello.py    2024-05-24 18:32:17.207921795 +0500
@@ -3,4 +3,5 @@

 user = input(Enter your name: )
 print(greet(user))
+print("Nice to meet you")

In the above output:

--- hello.py 2024-05-24 18:31:29.891690478 +0500 indicates the file being compared and its timestamp.
+++ also-hello.py 2024-05-24 18:32:17.207921795 +0500 indicates the other file being compared and its timestamp.
@@ -3,4 +3,5 @@ shows the line numbers where the changes occur. In this case, it indicates that lines 3 to 4 in the original file have changed to lines 3 to 5 in the modified file.
user = input(Enter your name: ) is a line from the original file.
print(greet(user)) is another line from the original file.
+print("Nice to meet you") is the additional line in the modified file.

To see the diff in a side-by-side format, you can use the -y flag:

diff -y hello.py also-hello.py
# Output
def greet(name):                        def greet(name):
    return fHello, {name}!                        return fHello, {name}!

user = input(Enter your name: )                    user = input(Enter your name: )
print(greet(user))                        print(greet(user))
                                        >    print("Nice to meet you")

In the output:

The lines that are the same in both files are displayed side by side.
Lines that are different are shown with a > symbol indicating the line is only present in one of the files.

Part 5: The Essentials of Text Editing in Linux

Text editing skills using the command line are one of the most crucial skills in Linux. In this section, you will learn how to use two popular text editors in Linux: Vim and Nano.

I suggest that you master any one text editor of your choice and stick to it. It will save you time and make you more productive. Vim and nano are safe choices as they are present on most Linux distributions.

5.1. Mastering Vim: The Complete Guide

Introduction to Vim

Vim is a popular text editing tool for the command line. Vim comes with its advantages: it is powerful, customizable, and fast. Here are some reasons why you should consider learning Vim:

Most servers are accessed via a CLI, so in system administration, you don't necessarily have the luxury of a GUI. But Vim has got your back – it'll always be there.
Vim uses a keyboard-centric approach, as it is designed to be used without a mouse, which can significantly speed up editing tasks once you have learned the keyboard shortcuts. This also makes it faster than GUI tools.
Some Linux utilities, for example editing cron jobs, work in the same editing format as Vim.
Vim is suitable for all – beginners and advanced users. Vim supports complex string searches, highlighting searches, and much more. Through plugins, Vim provides extended capabilities to developers and system admins that includes code completion, syntax highlighting, file management, version control, and more.

Vim has two variations: Vim (vim) and Vim tiny (vi). Vim tiny is a smaller version of Vim that lacks some features of Vim.

How to start using `vim`

Start using Vim with this command:

vim your-file.txt

your-file.txt can either be a new file or an existing file that you want to edit.

Navigating Vim: Mastering movement and command modes

In the early days of the CLI, the keyboards didn't have arrow keys. Hence, navigation was done using the set of available keys, hjkl being one of them.

Being keyboard-centric, using hjkl keys can greatly speed up text editing tasks.

Note: Although arrow keys would work totally fine, you can still experiment with hjkl keys to navigate. Some people find this this way of navigation efficient.

💡Tip: To remember the hjkl sequence, use this: hang back, jump down, kick up, leap forward.

The three Vim modes

You need to know the 3 operating modes of Vim and how to switch between them. Keystrokes behave differently in each command mode. The three modes are as follows:

Command mode.
Edit mode.
Visual mode.

Command Mode. When you start Vim, you land in the command mode by default. This mode allows you to access other modes.

⚠ To switch to other modes, you need to be present in the command mode first

Edit Mode

This mode allows you to make changes to the file. To enter edit mode, press I while in command mode. Note the '-- INSERT' switch at the end of the screen.

Visual mode

This mode allows you to work on a single character, a block of text, or lines of text. Let's break it down into simple steps. Remember, use the below combinations when in command mode.

Shift + V → Select multiple lines.
Ctrl + V → Block mode
V → Character mode

The visual mode comes in handy when you need to copy and paste or edit lines in bulk.

Extended command mode.

The extended command mode allows you to perform advanced operations like searching, setting line numbers, and highlighting text. We'll cover extended mode in the next section.

How to stay on track? If you forget your current mode, just press ESC twice and you will be back in Command Mode.

Editing Efficiently in Vim: Copy/pasting and searching

1. How to copy and paste in Vim

Copy-paste is known as 'yank' and 'put' in Linux terms. To copy-paste, follow these steps:

Select text in visual mode.
Press 'y' to copy/ yank.
Move your cursor to the required position and press 'p'.

2. How to search for text in Vim

Any series of strings can be searched with Vim using the / in command mode. To search, use /string-to-match.

In the command mode, type :set hls and press enter. Search using /string-to-match. This will highlight the searches.

Let's search a few strings:

3. How to exit Vim

First, move to command mode (by pressing escape twice) and then use these flags:

Exit without saving → :q!
Exit and save → :wq!

Shortcuts in Vim: Making Editing Faster

Note: All these shortcuts work in the command mode only.

Basic Navigation
- h: Move left
- j: Move down
- k: Move up
- l: Move right
- 0: Move to the beginning of the line
- $: Move to the end of the line
- gg: Move to the beginning of the file
- G: Move to the end of the file
- Ctrl+d: Move half-page down
- Ctrl+u: Move half-page up
Editing
- i: Enter insert mode before the cursor
- I: Enter insert mode at the beginning of the line
- a: Enter insert mode after the cursor
- A: Enter insert mode at the end of the line
- o: Open a new line below the current line and enter insert mode
- O: Open a new line above the current line and enter insert mode
- x: Delete the character under the cursor
- dd: Delete the current line
- yy: Yank (copy) the current line (use this in visual mode)
- p: Paste below the cursor
- P: Paste above the cursor
Searching and Replacing
- /: Search for a pattern which will take you to its next occurrence
- ?: Search for a pattern that will take you to its previous occurrence
- n: Repeat the last search in the same direction
- N: Repeat the last search in the opposite direction
- :%s/old/new/g: Replace all occurrences of old with new in the file
Exiting
- :w: Save the file but don't exit
- :q: Quit Vim (fails if there are unsaved changes)
- :wq or :x: Save and quit
- :q!: Quit without saving
Multiple Windows
- :split or :sp: Split the window horizontally
- :vsplit or :vsp: Split the window vertically
- Ctrl+w followed by h/j/k/l: Navigate between split windows

5.2. Mastering Nano

Getting started with Nano: The user-friendly text editor

Nano is a user-friendly text editor that is easy to use and is perfect for beginners. It is pre-installed on most Linux distributions.

To create a new file using Nano, use the following command:

nano

To start editing an existing file with Nano, use the following command:

nano filename

List of key bindings in Nano

Let's study the most important key bindings in Nano. You'll use the key bindings to perform various operations like saving, exiting, copying, pasting, and more.

Write to a file and save

Once you open Nano using the nano command, you can start writing text. To save the file, press Ctrl+O. You'll be prompted to enter the file name. Press Enter to save the file.

Exit nano

You can exit Nano by pressing Ctrl+X. If you have unsaved changes, Nano will prompt you to save the changes before exiting.

Copying and pasting

To select a region, use ALT+A. A marker will show. Use arrows to select the text. Once selected, exit the marker with with ALT+^.

To copy the selected text, press Ctrl+K. To paste the copied text, press Ctrl+U.

Cutting and pasting

Select the region with ALT+A. Once selected, cut the text with Ctrl+K. To paste the cut text, press Ctrl+U.

Navigation

Use Alt \ to move to the beginning of the file.

Use Alt / to move to the end of the file.

Viewing line numbers

When you open a file with nano -l filename, you can view line numbers on the left side of the file.

Searching

You can search for a specific line number with ALt + G. Enter the line number to the prompt and press Enter.

You can also initiate search for a string with CTRL + W and press Enter. If you want to search backwards, you can press Alt+W after initiating the search with Ctrl+W.

Summary of keybindings in Nano

General
- Ctrl+X: Exit Nano (prompting to save if changes are made)
- Ctrl+O: Save the file
- Ctrl+R: Read a file into the current file
- Ctrl+G: Display the help text
Editing
- Ctrl+K: Cut the current line and store it in the cutbuffer
- Ctrl+U: Paste the contents of the cutbuffer into the current line
- Alt+6: Copy the current line and store it in the cutbuffer
- Ctrl+J: Justify the current paragraph
Navigation
- Ctrl+A: Move to the beginning of the line
- Ctrl+E: Move to the end of the line
- Ctrl+C: Display the current line number and file information
- Ctrl+_ (Ctrl+Shift+-): Go to a specific line (and optionally, column) number
- Ctrl+Y: Scroll up one page
- Ctrl+V: Scroll down one page
Search and Replace
- Ctrl+W: Search for a string (then Enter to search again)
- Alt+W: Repeat the last search but in the opposite direction
- Ctrl+\: Search and replace
Miscellaneous
- Ctrl+T: Invoke the spell checker, if available
- Ctrl+D: Delete the character under the cursor (does not cut it)
- Ctrl+L: Refresh (redraw) the current screen
- Alt+U: Undo the last operation
- Alt+E: Redo the last undone operation

Part 6: Bash Scripting

6.1. Definition of Bash scripting

A bash script is a file containing a sequence of commands that are executed by the bash program line by line. It allows you to perform a series of actions, such as navigating to a specific directory, creating a folder, and launching a process using the command line.

By saving commands in a script, you can repeat the same sequence of steps multiple times and execute them by running the script.

6.2. Advantages of Bash Scripting

Bash scripting is a powerful and versatile tool for automating system administration tasks, managing system resources, and performing other routine tasks in Unix/Linux systems.

Some advantages of shell scripting are:

Automation: Shell scripts allow you to automate repetitive tasks and processes, saving time and reducing the risk of errors that can occur with manual execution.
Portability: Shell scripts can be run on various platforms and operating systems, including Unix, Linux, macOS, and even Windows through the use of emulators or virtual machines.
Flexibility: Shell scripts are highly customizable and can be easily modified to suit specific requirements. They can also be combined with other programming languages or utilities to create more powerful scripts.
Accessibility: Shell scripts are easy to write and don't require any special tools or software. They can be edited using any text editor, and most operating systems have a built-in shell interpreter.
Integration: Shell scripts can be integrated with other tools and applications, such as databases, web servers, and cloud services, allowing for more complex automation and system management tasks.
Debugging: Shell scripts are easy to debug, and most shells have built-in debugging and error-reporting tools that can help identify and fix issues quickly.

6.3. Overview of Bash Shell and Command Line Interface

The terms "shell" and "bash" are often used interchangeably. But there is a subtle difference between the two.

The term "shell" refers to a program that provides a command-line interface for interacting with an operating system. Bash (Bourne-Again SHell) is one of the most commonly used Unix/Linux shells and is the default shell in many Linux distributions.

Till now, the commands that you have been entering were basically being entered in a "shell".

Although Bash is a type of shell, there are other shells available as well, such as Korn shell (ksh), C shell (csh), and Z shell (zsh). Each shell has its own syntax and set of features, but they all share the common purpose of providing a command-line interface for interacting with the operating system.

You can determine your shell type using the ps command:

ps
# output:

    PID TTY          TIME CMD
  20506 pts/0    00:00:00 bash <--- the shell type
  20931 pts/0    00:00:00 ps

In summary, while "shell" is a broad term that refers to any program that provides a command-line interface, "Bash" is a specific type of shell that is widely used in Unix/Linux systems.

Note: In this section, we will be using the "bash" shell.

6.4. How to Create and Execute Bash scripts

Script naming conventions

By naming convention, bash scripts end with .sh. However, bash scripts can run perfectly fine without the sh extension.

Adding the Shebang

Bash scripts start with a shebang. Shebang is a combination of bash # and bang ! followed by the bash shell path. This is the first line of the script. Shebang tells the shell to execute it via bash shell. Shebang is simply an absolute path to the bash interpreter.

Below is an example of the shebang statement.

#!/bin/bash

You can find your bash shell path (which may vary from the above) using the command:

which bash

Creating your first bash script

Our first script prompts the user to enter a path. In return, its contents will be listed.

Create a file named run_all.sh using any editor of your choice.

vim run_all.sh

Add the following commands in your file and save it:

#!/bin/bash
echo "Today is " `date`

echo -e "\nenter the path to directory"
read the_path

echo -e "\n you path has the following files and folders: "
ls $the_path

Let's take a deeper look at the script line by line. I am displaying the same script again, but this time with line numbers.

  1 #!/bin/bash
  2 echo "Today is " `date`
  3
  4 echo -e "\nenter the path to directory"
  5 read the_path
  6
  7 echo -e "\n you path has the following files and folders: "
  8 ls $the_path

Line #1: The shebang (#!/bin/bash) points toward the bash shell path.
Line #2: The echo command displays the current date and time on the terminal. Note that the date is in backticks.
Line #4: We want the user to enter a valid path.
Line #5: The read command reads the input and stores it in the variable the_path.
line #8: The ls command takes the variable with the stored path and displays the current files and folders.

Executing the bash script

To make the script executable, assign execution rights to your user using this command:

chmod u+x run_all.sh

Here,

chmod modifies the ownership of a file for the current user :u.
+x adds the execution rights to the current user. This means that the user who is the owner can now run the script.
run_all.sh is the file we wish to run.

You can run the script using any of the mentioned methods:

sh run_all.sh
bash run_all.sh
./run_all.sh

Let's see it running in action 🚀

6.5. Bash Scripting Basics

Comments in bash scripting

Comments start with a # in bash scripting. This means that any line that begins with a # is a comment and will be ignored by the interpreter.

Comments are very helpful in documenting the code, and it is a good practice to add them to help others understand the code.

These are examples of comments:

# This is an example comment
# Both of these lines will be ignored by the interpreter

Variables and data types in Bash

Variables let you store data. You can use variables to read, access, and manipulate data throughout your script.

There are no data types in Bash. In Bash, a variable is capable of storing numeric values, individual characters, or strings of characters.

In Bash, you can use and set the variable values in the following ways:

Assign the value directly:

country=Netherlands

2. Assign the value based on the output obtained from a program or command, using command substitution. Note that $ is required to access an existing variable's value.

same_country=$country

This assigns the value of country to the new variable same_country.

To access the variable value, append $ to the variable name.

country=Netherlands
echo $country
# output
Netherlands
new_country=$country
echo $new_country
# output
Netherlands

Above, you can see an example of assigning and printing variable values.

Variable naming conventions

In Bash scripting, the following are the variable naming conventions:

Variable names should start with a letter or an underscore (_).
Variable names can contain letters, numbers, and underscores (_).
Variable names are case-sensitive.
Variable names should not contain spaces or special characters.
Use descriptive names that reflect the purpose of the variable.
Avoid using reserved keywords, such as if, then, else, fi, and so on as variable names.

Here are some examples of valid variable names in Bash:

name
count
_var
myVar
MY_VAR

And here are some examples of invalid variable names:

# invalid variable names

2ndvar (variable name starts with a number)
my var (variable name contains a space)
my-var (variable name contains a hyphen)

Following these naming conventions helps make Bash scripts more readable and easier to maintain.

Input and output in Bash scripts

Gathering input

In this section, we'll discuss some methods to provide input to our scripts.

Reading the user input and storing it in a variable

We can read the user input using the read command.

#!/bin/bash
echo "What's your name?"
read entered_name
echo -e "\nWelcome to bash tutorial" $entered_name

2. Reading from a file

This code reads each line from a file named input.txt and prints it to the terminal. We'll study while loops later in this section.

while read line
do
  echo $line
done < input.txt

3. Command line arguments

In a bash script or function, $1 denotes the initial argument passed, $2 denotes the second argument passed, and so forth.

This script takes a name as a command-line argument and prints a personalized greeting.

#!/bin/bash
echo "Hello, $1!"

We have supplied Zaira as our argument to the script.

Output:

Displaying output

Here we'll discuss some methods to receive output from the scripts.

Printing to the terminal:

echo "Hello, World!"

This prints the text "Hello, World!" to the terminal.

2. Writing to a file:

echo "This is some text." > output.txt

This writes the text "This is some text." to a file named output.txt. Note that the > operator overwrites a file if it already has some content.

3. Appending to a file:

echo "More text." >> output.txt

This appends the text "More text." to the end of the file output.txt.

4. Redirecting output:

ls > files.txt

This lists the files in the current directory and writes the output to a file named files.txt. You can redirect output of any command to a file this way.

You'll learn about output redirection in detail in section 8.5.

Conditional statements (if/else)

Expressions that produce a boolean result, either true or false, are called conditions. There are several ways to evaluate conditions, including if, if-else, if-elif-else, and nested conditionals.

Syntax:

if [[ condition ]];
then
    statement
elif [[ condition ]]; then
    statement 
else
    do this by default
fi

Syntax of bash conditional statements

We can use logical operators such as AND -a and OR -o to make comparisons that have more significance.

if [ $a -gt 60 -a $b -lt 100 ]

This statement checks if both conditions are true: a is greater than 60 AND b is less than 100.

Let's see an example of a Bash script that uses if, if-else, and if-elif-else statements to determine if a user-inputted number is positive, negative, or zero:

#!/bin/bash

# Script to determine if a number is positive, negative, or zero

echo "Please enter a number: "
read num

if [ $num -gt 0 ]; then
  echo "$num is positive"
elif [ $num -lt 0 ]; then
  echo "$num is negative"
else
  echo "$num is zero"
fi

The script first prompts the user to enter a number. Then, it uses an if statement to check if the number is greater than 0. If it is, the script outputs that the number is positive. If the number is not greater than 0, the script moves on to the next statement, which is an if-elif statement.

Here, the script checks if the number is less than 0. If it is, the script outputs that the number is negative.

Finally, if the number is neither greater than 0 nor less than 0, the script uses an else statement to output that the number is zero.

Seeing it in action 🚀

Looping and branching in Bash

While loop

While loops check for a condition and loop until the condition remains true. We need to provide a counter statement that increments the counter to control loop execution.

In the example below, (( i += 1 )) is the counter statement that increments the value of i. The loop will run exactly 10 times.

#!/bin/bash
i=1
while [[ $i -le 10 ]] ; do
   echo "$i"
  (( i += 1 ))
done

For loop

The for loop, just like the while loop, allows you to execute statements a specific number of times. Each loop differs in its syntax and usage.

In the example below, the loop will iterate 5 times.

#!/bin/bash

for i in {1..5}
do
    echo $i
done

Case statements

In Bash, case statements are used to compare a given value against a list of patterns and execute a block of code based on the first pattern that matches. The syntax for a case statement in Bash is as follows:

case expression in
    pattern1)
        # code to execute if expression matches pattern1
        ;;
    pattern2)
        # code to execute if expression matches pattern2
        ;;
    pattern3)
        # code to execute if expression matches pattern3
        ;;
    *)
        # code to execute if none of the above patterns match expression
        ;;
esac

Here, "expression" is the value that we want to compare, and "pattern1", "pattern2", "pattern3", and so on are the patterns that we want to compare it against.

The double semicolon ";;" separates each block of code to execute for each pattern. The asterisk "*" represents the default case, which executes if none of the specified patterns match the expression.

Let's see an example:

fruit="apple"

case $fruit in
    "apple")
        echo "This is a red fruit."
        ;;
    "banana")
        echo "This is a yellow fruit."
        ;;
    "orange")
        echo "This is an orange fruit."
        ;;
    *)
        echo "Unknown fruit."
        ;;
esac

In this example, since the value of fruit is apple, the first pattern matches, and the block of code that echoes This is a red fruit. is executed. If the value of fruit were instead banana, the second pattern would match and the block of code that echoes This is a yellow fruit. would execute, and so on.

If the value of fruit does not match any of the specified patterns, the default case is executed, which echoes Unknown fruit.

Part 7: Managing Software Packages in Linux

Linux comes with several built-in programs. But you might need to install new programs based on your needs. You might also need to upgrade the existing applications.

7.1. Packages and Package Management

What is a package?

A package is a collection of files that are bundled together. These files are essential for a particular program to run. These files contain the program's executable files, libraries, and other resources.

In addition to the files required for the program to run, packages also contain installation scripts, which copy the files to where they are needed. A program may contain many files and dependencies. With packages, it is easier to manage all the files and dependencies at once.

What is the difference between source and binary?

Programmers write source code in a programming language. This source code is then compiled into machine code that the computer can understand. The compiled code is called binary code.

When you download a package, you can either get the source code or the binary code. The source code is the human-readable code that can be compiled into binary code. The binary code is the compiled code that the computer can understand.

Source packages can be used with any type of machine if the source code is compiled properly. Binary, on the other hand, is compiled code that is specific to a particular type of machine or architecture.

You can find the architecture of your machine using the uname -m command.

uname -m
# output
x86_64

Package dependencies

Programs often share files. Instead of including these files in each package, a separate package can provide them for all programs.

To install a program that needs these files, you must also install the package containing them. This is called a package dependency. Specifying dependencies makes packages smaller and simpler by reducing duplicates.

When you install a program, its dependencies must also be installed. Most required dependencies are usually already installed, but a few extra ones might be needed. So, don't be surprised if several other packages are installed along with your chosen package. These are the necessary dependencies.

Package managers

Linux offers a comprehensive package management system for installing, upgrading, configuring, and removing software.

With package management, you can get access to an organized base of thousands of software packages along with having the ability to resolve dependencies and check for software updates.

Packages can be managed using either command-line utilities that can be easily automated by system administrators, or through a graphical interface.

Software channels/repositories

⚠️ Package management is different for different distros. Here, we are using Ubuntu.

Installing software is a bit different in Linux as compared to Windows and Mac.

Linux uses repositories to store software packages. A repository is a collection of software packages that are available for installation via a package manager.

A package manager also stores an index of all of the packages available from a repo. Sometimes the index is rebuilt to ensure that it is up to date and to know which packages have been upgraded or added to the channel since it last checked.

The generic process of downloading software from a repo looks something like this:

If we talk specifically about Ubuntu,

Index is fetched using apt update. (apt is explained in next section).
Required files/ dependencies requested according to index using apt install
Packages and dependencies installed locally.
Update dependencies and packages when required using apt update and apt upgrade

On Debian-based distros, you can file the list of repos (repositories) in /etc/apt/sources.list.

7.2. Installing a Package via Command Line

The apt command is a powerful command-line tool, which works with Ubuntu’s "Advanced Packaging Tool (APT)".

apt, along with the commands bundled with it, provides the means to install new software packages, upgrade existing software packages, update the package list index, and even upgrade the entire Ubuntu system.

To view the logs of the installation using apt, you can view the /var/log/dpkg.log file.

Following are the uses of the apt command:

Installing packages

For example, to install the htop package, you can use the following command:

sudo apt install htop

Updating the package list index

The package list index is a list of all the packages available in the repositories. To update the local package list index, you can use the following command:

sudo apt update

Upgrading the packages

Installed packages on your system can get updates containing bug fixes, security patches, and new features.

To upgrade the packages, you can use the following command:

sudo apt upgrade

Removing packages

To remove a package, like htop, you can use the following command:

sudo apt remove htop

7.3. Installing a Package via an Advanced Graphical Method – Synaptic

If you are not comfortable with the command line, you can use a GUI application to install packages. You can achieve the same results as the command line, but with a graphical interface.

Synaptic is a GUI package management application that helps in listing the installed packages, their status, pending updates, and so on. It offers custom filters to help you narrow down the search results.

You can also right-click on a package and view further details like the dependencies, maintainer, size, and the installed files.

7.4. Installing downloaded packages from a website

You may want to install a package you have downloaded from a website, rather than from a software repository. These packages are called .deb files.

Usingdpkgto install packages:dpkg is a command-line tool used to install packages. To install a package with dpkg, open the Terminal and type the following:

cd directory
sudo dpkg -i package_name.deb

Note: Replace "directory" with the directory where the package is stored and "package_name" with the filename of the package.

Alternatively, you can right-click, select "Open With Other Application," and choose a GUI app of your choice.

💡 Tip: In Ubuntu, you can see a list of installed packages with dpkg --list.

Part 8: Advanced Linux Topics

8.1. User Management

There can be multiple users with varying levels of access in a system. In Linux, the root user has the highest level of access and can perform any operation on the system. Regular users have limited access and can only perform operations they have been granted permission to do.

What is a user?

A user account provides separation between different people and programs that can run commands.

Humans identify users by a name, as names are easy to work with. But the system identifies users by a unique number called the user ID (UID).

When human users log in using the provided username, they have to use a password to authorize themselves.

User accounts form the foundations of system security. File ownership is also associated with user accounts and it enforces access control to the files. Every process has an associated user account that provides a layer of control for the admins.

There are three main types of user accounts:

Superuser: The superuser has complete access to the system. The name of the superuser is root. It has a UID of 0.
System user: The system user has user accounts that are used to run system services. These accounts are used to run system services and are not meant for human interaction.
Regular user: Regular users are human users who have access to the system.

The id command displays the user ID and group ID of the current user.

id
uid=1000(john) gid=1000(john) groups=1000(john),4(adm),24(cdrom),27(sudo),30(dip)... output truncated

To view the basic information of another user, pass the username as an argument to the id command.

id username

To view user-related information for processes, use the ps command with the -u flag.

ps -u
# Output
USER       PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root         1  0.0  0.1  16968  3920 ?        Ss   18:45   0:00 /sbin/init splash
root         2  0.0  0.0      0     0 ?        S    18:45   0:00 [kthreadd]

By default, systems use the /etc/passwd file to store user information.

Here is a line from the /etc/passwd file:

root:x:0:0:root:/root:/bin/bash

The /etc/passwd file contains the following information about each user:

Username: root – The username of the user account.
Password: x – The password in encrypted format for the user account that is stored in the /etc/shadow file for security reasons.
User ID (UID): 0 – The unique numerical identifier for the user account.
Group ID (GID): 0 – The primary group identifier for the user account.
User Info: root – The real name for the user account.
Home directory: /root – The home directory for the user account.
Shell: /bin/bash – The default shell for the user account. A system user might use /sbin/nologin if interactive logins are not allowed for that user.

What is a group?

A group is a collection of user accounts that share access and resources. Groups have group names to identify them. The system identifies groups by a unique number called the group ID (GID).

By default, the information about groups is stored in the /etc/group file.

Here is an entry from the /etc/group file:

adm:x:4:syslog,john

Here is the breakdown of the fields in the given entry:

Group name: adm – The name of the group.
Password: x – The password for the group is stored in the /etc/gshadow file for security reasons. The password is optional and appears empty if not set.
Group ID (GID): 4 – The unique numerical identifier for the group.
Group members: syslog,john – The list of usernames that are members of the group. In this case, the group adm has two members: syslog and john.

In this specific entry, the group name is adm, the group ID is 4, and the group has two members: syslog and john. The password field is typically set to x to indicate that the group password is stored in the /etc/gshadow file.

The groups are further divided into 'primary' and 'supplementary' groups.

Primary Group: Each user is assigned one primary group by default. This group usually has the same name as the user and is created when the user account is made. Files and directories created by the user are typically owned by this primary group.
Supplementary Groups: These are extra groups a user can belong to in addition to their primary group. Users can be members of multiple supplementary groups. These groups let a user have permissions for resources shared among those groups. They help provide access to shared resources without affecting the system’s file permissions and keeping the security intact. While a user must belong to one primary group, belonging to supplementary groups is optional.

Access control: finding and understanding file permission

File ownership can be viewed using the ls -l command. The first column in the output of the ls -l command shows the permissions of the file. Other columns show the owner of the file and the group that the file belongs to.

Let's have a closer look into the mode column:

Mode defines two things:

File type: File type defines the type of the file. For regular files that contain simple data it is blank -. For other special file types the symbol is different. For a directory which is a special file, it is d. Special files are treated differently by the OS.
Permission classes: The next set of characters define the permissions for user, group, and others respectively.
– User: This is the owner of a file and owner of the file belongs to this class.
– Group: The members of the file’s group belong to this class
– Other: Any users that are not part of the user or group classes belong to this class.

💡Tip: Directory ownership can be viewed using the ls -ld command.

How to Read Symbolic Permissions or the `rwx` permissions

The rwx representation is known as the Symbolic representation of permissions. In the set of permissions,

r stands for read. It is indicated in the first character of the triad.
w stands for write. It is indicated in the second character of the triad.
x stands for execution. It is indicated in the third character of the triad.

Read:

For regular files, read permissions allow the file to be opened and read only. Users can't modify the file.

Similarly for directories, read permissions allow the listing of directory content without any modification in the directory.

Write:

When files have write permissions, the user can modify (edit, delete) the file and save it.

For folders, write permissions enable a user to modify its contents (create, delete, and rename the files inside it), and modify the contents of files that the user has write permissions to.

Examples of permissions in Linux

Now that we know how to read permissions, let's see some examples.

-rwx------: A file that is only accessible and executable by its owner.

-rw-rw-r--: A file that is open to modification by its owner and group but not by others.
drwxrwx---: A directory that can be modified by its owner and group.

Execute:

For files, execute permissions allows the user to run an executable script. For directories, the user can access them, and access details about files in the directory.

How to Change File Permissions and Ownership in Linux using `chmod` and `chown`

Now that we know the basics of ownerships and permissions, let's see how we can modify permissions using the chmod command.

Syntax ofchmod:

chmod permissions filename

Where,

permissions can be read, write, execute or a combination of them.
filename is the name of the file for which the permissions need to change. This parameter can also be a list if files to change permissions in bulk.

We can change permissions using two modes:

Symbolic mode: this method uses symbols like u, g, o to represent users, groups, and others. Permissions are represented as r, w, x for read, write, and execute, respectively. You can modify permissions using +, - and =.
Absolute mode: this method represents permissions as 3-digit octal numbers ranging from 0-7.

Now, let's see them in detail.

How to Change Permissions using Symbolic Mode

The table below summarize the user representation:

USER REPRESENTATION	DESCRIPTION
u	user/owner
g	group
o	other

We can use mathematical operators to add, remove, and assign permissions. The table below shows the summary:

OPERATOR	DESCRIPTION
+	Adds a permission to a file or directory
–	Removes the permission
\=	Sets the permission if not present before. Also overrides the permissions if set earlier.

Example:

Suppose I have a script and I want to make it executable for the owner of the file zaira.

Current file permissions are as follows:

Let's split the permissions like this:

To add execution rights (x) to owner (u) using symbolic mode, we can use the command below:

chmod u+x mymotd.sh

Output:

Now, we can see that the execution permissions have been added for owner zaira.

Additional examples for changing permissions via symbolic method:

Removing read and write permission for group and others: chmod go-rw.
Removing read permissions for others: chmod o-r.
Assigning write permission to group and overriding existing permission: chmod g=w.

How to Change Permissions using Absolute Mode

Absolute mode uses numbers to represent permissions and mathematical operators to modify them.

The below table shows how we can assign relevant permissions:

PERMISSION	PROVIDE PERMISSION
read	add 4
write	add 2
execute	add 1

Permissions can be revoked using subtraction. The below table shows how you can remove relevant permissions.

PERMISSION	REVOKE PERMISSION
read	subtract 4
write	subtract 2
execute	subtract 1

Example:

Set read (add 4) for user, read (add 4) and execute (add 1) for group, and only execute (add 1) for others.

chmod 451 file-name

This is how we performed the calculation:

Note that this is the same as r--r-x--x.

Remove execution rights from other and group.

To remove execution from other and group, subtract 1 from the execute part of last 2 octets.

Assign read, write and execute to user, read and execute to group and only read to others.

This would be the same as rwxr-xr--.

How to Change Ownership using the `chown` Command

Next, we will learn how to change the ownership of a file. You can change the ownership of a file or folder using the chown command. In some cases, changing ownership requires sudo permissions.

Syntax of chown:

chown user filename

How to change user ownership with `chown`

Let's transfer the ownership from user zaira to user news.

chown news mymotd.sh

Command to change ownership: sudo chown news mymotd.sh.

Output:

How to change user and group ownership simultaneously

We can also use chown to change user and group simultaneously.

chown user:group filename

How to change directory ownership

You can change ownership recursively for contents in a directory. The example below changes the ownership of the /opt/script folder to allow user admin.

chown -R admin /opt/script

How to change group ownership

In case we only need to change the group owner, we can use chown by preceding the group name by a colon :

chown :admins /opt/script

How to switch between users

You can switch between users using the su command.

[user01@host ~]$ su user02
Password:
[user02@host ~]$

How to gain superuser access

The superuser or the root user has the highest level of access on a Linux system. The root user can perform any operation on the system. The root user can access all files and directories, install and remove software, and modify or override system configurations.

With great power comes great responsibility. If the root user is compromised, someone can gain complete control over the system. It is advised to use the root user account only when necessary.

If you omit the username, the su command switches to the root user account by default.

[user01@host ~]$ su
Password:
[root@host ~]#

Another variation of the su command is su -. The su command switches to the root user account but does not change the environment variables. The su - command switches to the root user account and changes the environment variables to those of the target user.

Running commands with sudo

To run commands as the root user without switching to the root user account, you can use the sudo command. The sudo command allows you to run commands with elevated privileges.

Running commands with sudo is a safer option rather than running the commands as the root user. This is because, only a specific set of users can be granted permission to run commands with sudo. This is defined in the /etc/sudoers file.

Also, sudo logs all commands that are run with it, providing an audit trail of who ran which commands and when.

In Ubuntu, you can find the audit logs here:

cat /var/log/auth.log | grep sudo

For a user that does not have access to sudo, it gets flagged in logs and prompts a message like this:

user01 is not in the sudoers file.  This incident will be reported.

Managing local user accounts

Creating users from the command line

The command used to add a new user is:

sudo useradd username

This command sets up a user's home directory and creates a private group designated by the user's username. Currently, the account lacks a valid password, preventing the user from logging in until a password is created.

Modifying existing users

The usermod command is used to modify existing users. Here are some of the common options used with the usermod command:

Here are some examples of the usermod command in Linux:

Change a user's login name:

 sudo usermod -l newusername oldusername

Change a user's home directory:

 sudo usermod -d /new/home/directory -m username

Add a user to a supplementary group:
```
 sudo usermod -aG groupname username
```
Change a user's shell:
```
 sudo usermod -s /bin/bash username
```
Lock a user's account:
```
 sudo usermod -L username
```
Unlock a user's account:
```
 sudo usermod -U username
```
Set an expiration date for a user account:
```
 sudo usermod -e YYYY-MM-DD username
```
Change a user's user ID (UID):
```
 sudo usermod -u newUID username
```
Change a user's primary group:
```
 sudo usermod -g newgroup username
```
Remove a user from a supplementary group:
```
sudo gpasswd -d username groupname
```

Deleting users

The userdel command is used to delete a user account and related files from the system.

sudo userdel username: removes the user's details from /etc/passwd but keeps the user's home directory.
The sudo userdel -r username command removes the user's details from /etc/passwd and also deletes the user's home directory.

Changing user passwords

The passwd command is used to change a user's password.

sudo passwd username: sets the initial password or changes the existing password of username. It is also used to change the password of the currently logged in user.

8.2 Connecting to Remote Servers via SSH

Accessing remote servers is one of the essential tasks for system administrators. You can connect to different servers or access databases through your local machine and execute commands, all using SSH.

What is the SSH protocol?

SSH stands for Secure Shell. It is a cryptographic network protocol that allows secure communication between two systems.

The default port for SSH is 22.

The two participants while communicating via SSH are:

The server: the machine that you want access to.
The client: The system that you are accessing the server from.

Connection to a server follows these steps:

Initiate Connection: The client sends a connection request to the server.
Exchange of Keys: The server sends its public key to the client. Both agree on the encryption methods to use.
Session Key Generation: The client and server use the Diffie-Hellman key exchange to create a shared session key.
Client Authentication: The client logs in to the server using a password, private key, or another method.
Secure Communication: After authentication, the client and server communicate securely with encryption.

How to connect to a remote server using SSH?

The ssh command is a built-in utility in Linux and also the default one. It makes accessing servers quite easy and secure.

Here, we are talking about how the client would make a connection to the server.

Prior to connecting to a server, you need to have the following information:

The IP address or the domain name of the server.
The username and password of the server.
The port number that you have access to in the server.

The basic syntax of the ssh command is:

ssh username@server_ip

For example, if your username is john and the server IP is 192.168.1.10, the command would be:

ssh john@192.168.1.10

After that, you'll be prompted to enter the secret password. Your screen will look similar to this:

john@192.168.1.10's password: 
Welcome to Ubuntu 20.04.2 LTS (GNU/Linux 5.4.0-70-generic x86_64)

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

  System information as of Fri Jun  5 10:17:32 UTC 2024

  System load:  0.08               Processes:           122
  Usage of /:   12.3% of 19.56GB   Users logged in:     1
  Memory usage: 53%                IP address for eth0: 192.168.1.10
  Swap usage:   0%

Last login: Fri Jun  5 09:34:56 2024 from 192.168.1.2
john@hostname:~$ # start entering commands

Now you can execute the relevant commands on the server 192.168.1.10.

⚠️ The default port for ssh is 22 but it is also vulnerable, as hackers will likely attempt here first. Your server can expose another port and share the access with you. To connect to a different port, use the -p flag.

ssh -p port_number username@server_ip

8.3. Advanced Log Parsing and Analysis

Log files, when configured, are generated by your system for a variety of useful reasons. They can be used to track system events, monitor system performance, and troubleshoot issues. They are specifically useful for system administrators where they can track application errors, network events, and user activity.

Here is an example of a log file:

# sample log file
2024-04-25 09:00:00 INFO Startup: Application starting
2024-04-25 09:01:00 INFO Config: Configuration loaded successfully
2024-04-25 09:02:00 DEBUG Database: Database connection established
2024-04-25 09:03:00 INFO User: New user registered (UserID: 1001)
2024-04-25 09:04:00 WARN Security: Attempted login with incorrect credentials (UserID: 1001)
2024-04-25 09:05:00 ERROR Network: Network timeout on request (ReqID: 456)
2024-04-25 09:06:00 INFO Email: Notification email sent (UserID: 1001)
2024-04-25 09:07:00 DEBUG API: API call with response time over threshold (Duration: 350ms)
2024-04-25 09:08:00 INFO Session: User session ended (UserID: 1001)
2024-04-25 09:09:00 INFO Shutdown: Application shutdown initiated

A log file usually contains the following columns:

Timestamp: The date and time when the event occurred.
Log Level: The severity of the event (INFO, DEBUG, WARN, ERROR).
Component: The component of the system that generated the event (Startup, Config, Database, User, Security, Network, Email, API, Session, Shutdown).
Message: A description of the event that occurred.
Additional Information: Additional information related to the event.

In real-time systems, log files tend to be thousands of lines long and are generated every second. They can be very wordy depending on the configuration. Every column in a log file is a piece of information that can be used to track down issues. This makes log files difficult to read and understand manually.

This is where log parsing comes in. Log parsing is the process of extracting useful information from log files. It involves breaking down the log files into smaller, more manageable pieces, and extracting the relevant information.

The filtered information can also be useful for creating alerts, reports, and dashboards.

In this section, you will explore some techniques for parsing log files in Linux.

Text extraction using `grep`

Grep is a built-in bash utility. It stands for "global regular expression print". Grep is used to match strings in files.

Here are some common uses of grep:

Search for a specific string in a file:
```
 grep "search_string" filename
```
This command searches for "search_string" in the file named filename.
Search recursively in directories:
```
 grep -r "search_string" /path/to/directory
```
This command searches for "search_string" in all files within the specified directory and its subdirectories.
Ignore case while searching:
```
 grep -i "search_string" filename
```
This command performs a case-insensitive search for "search_string" in the file named filename.
Display line numbers with matching lines:
```
 grep -n "search_string" filename
```
This command shows the line numbers along with the matching lines in the file named filename.
Count the number of matching lines:
```
 grep -c "search_string" filename
```
This command counts the number of lines that contain "search_string" in the file named filename.
Invert match to display lines that do not match:
```
 grep -v "search_string" filename
```
This command displays all lines that do not contain "search_string" in the file named filename.
Search for a whole word:
```
 grep -w "word" filename
```
This command searches for the whole word "word" in the file named filename.
Use extended regular expressions:
```
 grep -E "pattern" filename
```
This command allows the use of extended regular expressions for more complex pattern matching in the file named filename.

💡 Tip: If there are multiple files in a folder, you can use the below command to find the list of files containing the desired strings.

# find the list of files containing the desired strings
grep -l "String to Match" /path/to/directory

Text extraction using `sed`

sed stands for "stream editor". It processes data stream-wise, meaning it reads data one line at a time. sed allows you to search for patterns and perform actions on the lines that match those patterns.

Basic syntax ofsed:

The basic syntax of sed is as follows:

sed [options] 'command' file_name

Here, command is used to perform operations like substitution, deletion, insertion, and so on, on the text data. The filename is the name of the file you want to process.

sedusage:

1. Substitution:

The s flag is used to replace text. The old-text is replaced with new-text:

sed 's/old-text/new-text/' filename

For example, to change all instances of "error" to "warning" in the log file system.log:

sed 's/error/warning/' system.log

2. Printing lines containing a specific pattern:

Using sed to filter and display lines that match a specific pattern:

sed -n '/pattern/p' filename

For instance, to find all lines containing "ERROR":

sed -n '/ERROR/p' system.log

3. Deleting lines containing a specific pattern:

You can delete lines from the output that match a specific pattern:

sed '/pattern/d' filename

For example, to remove all lines containing "DEBUG":

sed '/DEBUG/d' system.log

4. Extracting specific fields from a log line:

You can use regular expressions to extract parts of lines. Suppose each log line starts with a date in the format "YYYY-MM-DD". You could extract just the date from each line:

sed -n 's/^\([0-9]\{4\}-[0-9]\{2\}-[0-9]\{2\}\).*/\1/p' system.log

Text parsing with `awk`

awk has the ability to easily split each line into fields. It's well-suited for processing structured text like log files.

Basic syntax ofawk

The basic syntax of awk is:

awk 'pattern { action }' file_name

Here, pattern is a condition that must be met for the action to be performed. If the pattern is omitted, the action is performed on every line.

In the coming examples, you'll use this log file as an example:

2024-04-25 09:00:00 INFO Startup: Application starting
2024-04-25 09:01:00 INFO Config: Configuration loaded successfully
2024-04-25 09:02:00 INFO Database: Database connection established
2024-04-25 09:03:00 INFO User: New user registered (UserID: 1001)
2024-04-25 09:04:00 INFO Security: Attempted login with incorrect credentials (UserID: 1001)
2024-04-25 09:05:00 INFO Network: Network timeout on request (ReqID: 456)
2024-04-25 09:06:00 INFO Email: Notification email sent (UserID: 1001)
2024-04-25 09:07:00 INFO API: API call with response time over threshold (Duration: 350ms)
2024-04-25 09:08:00 INFO Session: User session ended (UserID: 1001)
2024-04-25 09:09:00 INFO Shutdown: Application shutdown initiated
  INFO

Accessing columns usingawk

The fields in awk (separated by spaces by default) can be accessed using $1, $2, $3, and so on.

zaira@zaira-ThinkPad:~$ awk '{ print $1 }' sample.log
# output
2024-04-25
2024-04-25
2024-04-25
2024-04-25
2024-04-25
2024-04-25
2024-04-25
2024-04-25
2024-04-25
2024-04-25

zaira@zaira-ThinkPad:~$ awk '{ print $2 }' sample.log
# output
09:00:00
09:01:00
09:02:00
09:03:00
09:04:00
09:05:00
09:06:00
09:07:00
09:08:00
09:09:00

Print lines containing a specific pattern (for example, ERROR)

awk '/ERROR/ { print $0 }' logfile.log

# output
2024-04-25 09:05:00 ERROR Network: Network timeout on request (ReqID: 456)

This prints all lines that contain "ERROR".

Extract the first field (Date and Time)

awk '{ print $1, $2 }' logfile.log
# output
2024-04-25 09:00:00
2024-04-25 09:01:00
2024-04-25 09:02:00
2024-04-25 09:03:00
2024-04-25 09:04:00
2024-04-25 09:05:00
2024-04-25 09:06:00
2024-04-25 09:07:00
2024-04-25 09:08:007
2024-04-25 09:09:00

This will extract the first two fields from each line, which in this case would be the date and time.

Summarize occurrences of each log level

awk '{ count[$3]++ } END { for (level in count) print level, count[level] }' logfile.log

# output
 1
WARN 1
ERROR 1
DEBUG 2
INFO 6

The output will be a summary of the number of occurrences of each log level.

Filter out specific fields (for example, where the 3rd field is INFO)

awk '{ $3="INFO"; print }' sample.log

# output
2024-04-25 09:00:00 INFO Startup: Application starting
2024-04-25 09:01:00 INFO Config: Configuration loaded successfully
2024-04-25 09:02:00 INFO Database: Database connection established
2024-04-25 09:03:00 INFO User: New user registered (UserID: 1001)
2024-04-25 09:04:00 INFO Security: Attempted login with incorrect credentials (UserID: 1001)
2024-04-25 09:05:00 INFO Network: Network timeout on request (ReqID: 456)
2024-04-25 09:06:00 INFO Email: Notification email sent (UserID: 1001)
2024-04-25 09:07:00 INFO API: API call with response time over threshold (Duration: 350ms)
2024-04-25 09:08:00 INFO Session: User session ended (UserID: 1001)
2024-04-25 09:09:00 INFO Shutdown: Application shutdown initiated
  INFO

This command will extract all lines where the 3rd field is "INFO".

💡 Tip: The default separator in awk is a space. If your log file uses a different separator, you can specify it using the -F option. For example, if your log file uses a colon as a separator, you can use awk -F: '{ print $1 }' logfile.log to extract the first field.

Parsing log files with `cut`

The cut command is a simple yet powerful command used to extract sections of text from each line of input. As log files are structured and each field is delimited by a specific character, such as a space, tab, or a custom delimiter, cut does a very good job of extracting those specific fields.

The basic syntax of the cut command is:

cut [options] [file]

Some commonly used options for the cut command:

-d : Specifies a delimiter used as the field separator.
-f : Selects the fields to be displayed.
-c : Specifies character positions.

For example, the command below would extract the first field (separated by a space) from each line of the log file:

cut -d ' ' -f 1 logfile.log

Examples of usingcutfor log parsing

Assume you have a log file structured as follows, where fields are space-separated:

2024-04-25 08:23:01 INFO 192.168.1.10 User logged in successfully.
2024-04-25 08:24:15 WARNING 192.168.1.10 Disk usage exceeds 90%.
2024-04-25 08:25:02 ERROR 10.0.0.5 Connection timed out.
...

cut can be used in the following ways:

Extracting the time from each log entry:

cut -d ' ' -f 2 system.log

# Output
08:23:01
08:24:15
08:25:02
...

This command uses a space as a delimiter and selects the second field, which is the time component of each log entry.

Extracting the IP addresses from the logs:

cut -d ' ' -f 4 system.log

# Output
192.168.1.10
192.168.1.10
10.0.0.5

This command extracts the fourth field, which is the IP address from each log entry.

Extracting log levels (INFO, WARNING, ERROR):

cut -d ' ' -f 3 system.log

# Output
INFO
WARNING
ERROR

This extracts the third field which contains the log level.

Combiningcutwith other commands:

The output of other commands can be piped to the cut command. Let's say you want to filter logs before cutting. You can use grep to extract lines containing "ERROR" and then use cut to get specific information from those lines:

grep "ERROR" system.log | cut -d ' ' -f 1,2 

# Output
2024-04-25 08:25:02

This command first filters lines that include "ERROR", then extracts the date and time from these lines.

Extracting multiple fields:

It is possible to extract multiple fields at once by specifying a range or a comma-separated list of fields:

cut -d ' ' -f 1,2,3 system.log` 

# Output
2024-04-25 08:23:01 INFO
2024-04-25 08:24:15 WARNING
2024-04-25 08:25:02 ERROR
...

The above command extracts the first three fields from each log entry that are date, time, and log level.

Parsing log files with `sort` and `uniq`

Sorting and removing duplicates are common operations when working with log files. The sort and uniq commands are powerful commands used to sort and remove duplicates from the input, respectively.

Basic syntax of sort

The sort command organizes lines of text alphabetically or numerically.

sort [options] [file]

Some key options for the sort command:

-n: Sorts the file assuming the contents are numerical.
-r: Reverses the order of sort.
-k: Specifies a key or column number to sort on.
-u: Sorts and removes duplicate lines.

The uniq command is used to filter or count and report repeated lines in a file.

The syntax of uniq is:

uniq [options] [input_file] [output_file]

Some key options for the uniq command are:

-c: Prefixes lines by the number of occurrences.
-d: Only prints duplicate lines.
-u: Only prints unique lines.

Examples of using `sort` and `uniq` together for log parsing

Let's assume the following example log entries for these demonstrations:

2024-04-25 INFO User logged in successfully.
2024-04-25 WARNING Disk usage exceeds 90%.
2024-04-26 ERROR Connection timed out.
2024-04-25 INFO User logged in successfully.
2024-04-26 INFO Scheduled maintenance.
2024-04-26 ERROR Connection timed out.

Sorting log entries by date:

sort system.log

# Output
2024-04-25 INFO User logged in successfully.
2024-04-25 INFO User logged in successfully.
2024-04-25 WARNING Disk usage exceeds 90%.
2024-04-26 ERROR Connection timed out.
2024-04-26 ERROR Connection timed out.
2024-04-26 INFO Scheduled maintenance.

This sorts the log entries alphabetically, which effectively sorts them by date if the date is the first field.

Sorting and removing duplicates:

sort system.log | uniq

# Output
2024-04-25 INFO User logged in successfully.
2024-04-25 WARNING Disk usage exceeds 90%.
2024-04-26 ERROR Connection timed out.
2024-04-26 INFO Scheduled maintenance.

This command sorts the log file and pipes it to uniq, removing duplicate lines.

Counting occurrences of each line:

sort system.log | uniq -c

# Output
2 2024-04-25 INFO User logged in successfully.
1 2024-04-25 WARNING Disk usage exceeds 90%.
2 2024-04-26 ERROR Connection timed out.
1 2024-04-26 INFO Scheduled maintenance.

Sorts the log entries and then counts each unique line. According to the output, the line '2024-04-25 INFO User logged in successfully.' appeared 2 times in the file.

Identifying unique log entries:

sort system.log | uniq -u

# Output

2024-04-25 WARNING Disk usage exceeds 90%.
2024-04-26 INFO Scheduled maintenance.

This command shows lines that are unique.

Sorting by log level:

sort -k2 system.log

# Output
2024-04-26 ERROR Connection timed out.
2024-04-26 ERROR Connection timed out.
2024-04-25 INFO User logged in successfully.
2024-04-25 INFO User logged in successfully.
2024-04-26 INFO Scheduled maintenance.
2024-04-25 WARNING Disk usage exceeds 90%.

Sorts the entries based on the second field, which is the log level.

8.4. Managing Linux Processes via Command Line

A process is a running instance of a program. A process consists of:

An address space of the allocated memory.
Process states.
Properties such as ownership, security attributes, and resource usage.

A process also has an environment that consists of:

Local and global variables
The current scheduling context
Allocated system resources, such as network ports or file descriptors.

When you run the ls -l command, the operating system creates a new process to execute the command. The process has an ID, a state, and runs until the command completes.

Understanding process creation and lifecycle

In Ubuntu, all processes originate from the initial system process called systemd, which is the first process started by the kernel during boot.

The systemd process has a process ID (PID) of 1 and is responsible for initializing the system, starting and managing other processes, and handling system services. All other processes on the system are descendants of systemd.

A parent process duplicates its own address space (fork) to create a new (child) process structure. Each new process is assigned a unique process ID (PID) for tracking and security purposes. The PID and the parent's process ID (PPID) are part of the new process environment. Any process can create a child process.

Through the fork routine, a child process inherits security identities, previous and current file descriptors, port and resource privileges, environment variables, and program code. A child process may then execute its own program code.

Typically, a parent process sleeps while the child process runs, setting a request (wait) to be notified when the child completes.

Upon exiting, the child process has already closed or discarded its resources and environment. The only remaining resource, known as a zombie, is an entry in the process table. The parent, signaled awake when the child exits, cleans the process table of the child's entry, thus freeing the last resource of the child process. The parent process then continues executing its own program code.

Understanding process states

Processes in Linux assume different states during their lifecycle. The state of a process indicates what the process is currently doing and how it is interacting with the system. The processes transition between states based on their execution status and the system's scheduling algorithm.

The processes in a Linux system can be in one of the following states:

State	Description
(new)	Initial state when a process is created via a fork system call.
Runnable (ready) (R)	Process is ready to run and waiting to be scheduled on a CPU.
Running (user) (R)	Process is executing in user mode, running user applications.
Running (kernel) (R)	Process is executing in kernel mode, handling system calls or hardware interrupts.
Sleeping (S)	Process is waiting for an event (for example, I/O operation) to complete and can be easily awakened.
Sleeping (uninterruptible) (D)	Process is in an uninterruptible sleep state, waiting for a specific condition (usually I/O) to complete, and cannot be interrupted by signals.
Sleeping (disk sleep) (K)	Process is waiting for disk I/O operations to complete.
Sleeping (idle) (I)	Process is idle, not doing any work, and waiting for an event to occur.
Stopped (T)	Process execution has been stopped, typically by a signal, and can be resumed later.
Zombie (Z)	Process has completed execution but still has an entry in the process table, waiting for its parent to read its exit status.

The processes transition between these states in the following ways:

Transition	Description
Fork	Creates a new process from a parent process, transitioning from (new) to Runnable (ready) (R).
Schedule	Scheduler selects a runnable process, transitioning it to Running (user) or Running (kernel) state.
Run	Process transitions from Runnable (ready) (R) to Running (kernel) (R) when scheduled for execution.
Preempt or Reschedule	Process can be preempted or rescheduled, moving it back to Runnable (ready) (R) state.
Syscall	Process makes a system call, transitioning from Running (user) (R) to Running (kernel) (R).
Return	Process completes a system call and returns to Running (user) (R).
Wait	Process waits for an event, transitioning from Running (kernel) (R) to one of the Sleeping states (S, D, K, or I).
Event or Signal	Process is awakened by an event or signal, moving it from a Sleeping state back to Runnable (ready) (R).
Suspend	Process is suspended, transitioning from Running (kernel) or Runnable (ready) to Stopped (T).
Resume	Process is resumed, moving from Stopped (T) back to Runnable (ready) (R).
Exit	Process terminates, transitioning from Running (user) or Running (kernel) to Zombie (Z).
Reap	Parent process reads the exit status of the zombie process, removing it from the process table.

How to view processes

You can use the ps command along with a combination of options to view processes on a Linux system. The ps command is used to display information about a selection of active processes. For example, ps aux displays all processes running on the system.

zaira@zaira:~$ ps aux
# Output
USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
root           1  0.0  0.0 168140 11352 ?        Ss   May21   0:18 /sbin/init splash
root           2  0.0  0.0      0     0 ?        S    May21   0:00 [kthreadd]
root           3  0.0  0.0      0     0 ?        I<   May21   0:00 [rcu_gp]
root           4  0.0  0.0      0     0 ?        I<   May21   0:00 [rcu_par_gp]
root           5  0.0  0.0      0     0 ?        I<   May21   0:00 [slub_flushwq]
root           6  0.0  0.0      0     0 ?        I<   May21   0:00 [netns]
root          11  0.0  0.0      0     0 ?        I<   May21   0:00 [mm_percpu_wq]
root          12  0.0  0.0      0     0 ?        I    May21   0:00 [rcu_tasks_kthread]
root          13  0.0  0.0      0     0 ?        I    May21   0:00 [rcu_tasks_rude_kthread]
*... output truncated ....*

The output above shows a snapshot of the currently running processes on the system. Each row represents a process with the following columns:

USER: The user who owns the process.
PID: The process ID.
%CPU: The CPU usage of the process.
%MEM: The memory usage of the process.
VSZ: The virtual memory size of the process.
RSS: The resident set size, that is the non-swapped physical memory that a task has used.
TTY: The controlling terminal of the process. A ? indicates no controlling terminal.
STAT: The process state.
- R: Running
- I or S: Interruptible sleep (waiting for an event to complete)
- D: Uninterruptible sleep (usually IO)
- T: Stopped (either by a job control signal or because it is being traced)
- Z: Zombie (terminated but not reaped by its parent)
- Ss: Session leader. This is a process that has started a session, and it is a leader of a group of processes and can control terminal signals. The first S indicates the sleeping state, and the second s indicates it is a session leader.
START: The starting time or date of the process.
TIME: The cumulative CPU time.
COMMAND: The command that started the process.

Background and foreground processes

In this section, you'll learn how you can control jobs by running them in the background or foreground.

A job is a process that is started by a shell. When you run a command in the terminal, it is considered a job. A job can run in the foreground or the background.

To demonstrate control, you'll first create 3 processes and then run them in the background. After that, you'll list the processes and alternate them between the foreground and background. You'll see how to put them to sleep or exit completely.

Create Three Processes

Open a terminal and start three long-running processes. Use the sleep command, that keeps the process running for a specified number of seconds.

# run sleep command for 300, 400, and 500 seconds
sleep 300 &
sleep 400 &
sleep 500 &

The & at the end of each command moves the process to the background.

Display Background Jobs

Use the jobs command to display the list of background jobs.

jobs

The output should look something like this:

jobs
[1]   Running                 sleep 300 &
[2]-  Running                 sleep 400 &
[3]+  Running                 sleep 500 &

Bring a Background Job to the Foreground

To bring a background job to the foreground, use the fg command followed by the job number. For example, to bring the first job (sleep 300) to the foreground:

fg %1

This will bring job 1 to the foreground.

Move the Foreground Job Back to the Background

While the job is running in the foreground, you can suspend it and move it back to the background by pressing Ctrl+Z to suspend the job.

A suspended job will look like this:

zaira@zaira:~$ fg %1
sleep 300

^Z
[1]+  Stopped                 sleep 300

zaira@zaira:~$ jobs
# suspended job 
[1]+  Stopped                 sleep 300
[2]   Running                 sleep 400 &
[3]-  Running                 sleep 500 &

Now use the bg command to resume the job with ID 1 in the background.

# Press Ctrl+Z to suspend the foreground job
# Then, resume it in the background
bg %1

Display the jobs again

jobs
[1]   Running                 sleep 300 &
[2]-  Running                 sleep 400 &
[3]+  Running                 sleep 500 &

In this exercise, you:

Started three background processes using sleep commands.
Used jobs to display the list of background jobs.
Brought a job to the foreground with fg %job_number.
Suspended the job with Ctrl+Z and moved it back to the background with bg %job_number.
Used jobs again to verify the status of the background jobs.

Now you know how to control jobs.

Killing processes

It is possible to terminate an unresponsive or unwanted process using the kill command. The kill command sends a signal to a process ID, asking it to terminate.

A number of options are available with the kill command.

# Options available with kill

kill -l
 1) SIGHUP     2) SIGINT     3) SIGQUIT     4) SIGILL     5) SIGTRAP
 6) SIGABRT     7) SIGBUS     8) SIGFPE     9) SIGKILL    10) SIGUSR1
11) SIGSEGV    12) SIGUSR2    13) SIGPIPE    14) SIGALRM    15) SIGTERM
16) SIGSTKFLT    17) SIGCHLD    18) SIGCONT    19) SIGSTOP    20) SIGTSTP
21) SIGTTIN    22) SIGTTOU    23) SIGURG    24) 
...terminated

Here are some examples of the kill command in Linux:

Kill a process by PID (Process ID):
```
 kill 1234
```
This command sends the default SIGTERM signal to the process with PID 1234, requesting it to terminate.
Kill a process by name:
```
 pkill process_name
```
This command sends the default SIGTERM signal to all processes with the specified name.
Forcefully kill a process:
```
 kill -9 1234
```
This command sends the SIGKILL signal to the process with PID 1234, forcefully terminating it.
Send a specific signal to a process:
```
 kill -s SIGSTOP 1234
```
This command sends the SIGSTOP signal to the process with PID 1234, stopping it.
Kill all processes owned by a specific user:
```
 pkill -u username
```
This command sends the default SIGTERM signal to all processes owned by the specified user.

These examples demonstrate various ways to use the kill command to manage processes in a Linux environment.

Here is the information about the kill command options and signals in a tabular form: This table summarizes the most common kill command options and signals used in Linux for managing processes.

Command / Option	Signal	Description
`kill`	`SIGTERM`	Requests the process to terminate gracefully (default signal).
`kill -9`	`SIGKILL`	Forces the process to terminate immediately without cleanup.
`kill -SIGKILL`	`SIGKILL`	Forces the process to terminate immediately without cleanup.
`kill -15`	`SIGTERM`	Explicitly sends the `SIGTERM` signal to request graceful termination.
`kill -SIGTERM`	`SIGTERM`	Explicitly sends the `SIGTERM` signal to request graceful termination.
`kill -1`	`SIGHUP`	Traditionally means "hang up"; can be used to reload configuration files.
`kill -SIGHUP`	`SIGHUP`	Traditionally means "hang up"; can be used to reload configuration files.
`kill -2`	`SIGINT`	Requests the process to terminate (same as pressing `Ctrl+C` in terminal).
`kill -SIGINT`	`SIGINT`	Requests the process to terminate (same as pressing `Ctrl+C` in terminal).
`kill -3`	`SIGQUIT`	Causes the process to terminate and produce a core dump for debugging.
`kill -SIGQUIT`	`SIGQUIT`	Causes the process to terminate and produce a core dump for debugging.
`kill -19`	`SIGSTOP`	Pauses the process.
`kill -SIGSTOP`	`SIGSTOP`	Pauses the process.
`kill -18`	`SIGCONT`	Resumes a paused process.
`kill -SIGCONT`	`SIGCONT`	Resumes a paused process.
`killall`	Varies	Sends a signal to all processes with the given name.
`killall -9`	`SIGKILL`	Force kills all processes with the given name.
`pkill`	Varies	Sends a signal to processes based on a pattern match.
`pkill -9`	`SIGKILL`	Force kills all processes matching the pattern.
`xkill`	`SIGKILL`	Graphical utility that allows clicking on a window to kill the corresponding process.

8.5. Standard Input and Output Streams in Linux

Reading an input and writing an output is an essential part of understanding the command line and shell scripting. In Linux, every process has three default streams:

Standard Input (stdin): This stream is used for input, typically from the keyboard. When a program reads from stdin, it receives data entered by the user or redirected from a file. A file descriptor is a unique identifier that the operating system assigns to an open file in order to keep track of open files.

The file descriptor for stdin is 0.
Standard Output (stdout): This is the default output stream where a process writes its output. By default, the standard output is the terminal. The output can also be redirected to a file or another program. The file descriptor for stdout is 1.
Standard Error (stderr): This is the default error stream where a process writes its error messages. By default, the standard error is the terminal, allowing error messages to be seen even if stdout is redirected. The file descriptor for stderr is 2.

Redirection and Pipelines

Redirection: You can redirect the error and output streams to files or other commands. For example:

# Redirecting stdout to a file
ls > output.txt

# Redirecting stderr to a file
ls non_existent_directory 2> error.txt

# Redirecting both stdout and stderr to a file
ls non_existent_directory > all_output.txt 2>&1

In the last command,

ls non_existent_directory: lists the contents of a directory named non_existent_directory. Since this directory does not exist, ls will generate an error message.
> all_output.txt: The > operator redirects the standard output (stdout) of the ls command to the file all_output.txt. If the file does not exist, it will be created. If it does exist, its contents will be overwritten.
2>&1:: Here, 2 represents the file descriptor for standard error (stderr). &1 represents the file descriptor for standard output (stdout). The & character is used to specify that 1 is not the file name but a file descriptor.

So, 2>&1 means "redirect stderr (2) to wherever stdout (1) is currently going," which in this case is the file all_output.txt. Therefore, both the output (if there were any) and the error message from ls will be written to all_output.txt.

Pipelines:

You can use pipes (|) to pass the output of one command as the input to another:

ls | grep image
# Output
image-10.png
image-11.png
image-12.png
image-13.png
... Output truncated ...

8.6 Automation in Linux – Automate Tasks with Cron Jobs

Cron is a powerful utility for job scheduling that is available in Unix-like operating systems. By configuring cron, you can set up automated jobs to run on a daily, weekly, monthly, or other specific time basis. The automation capabilities provided by cron play a crucial role in Linux system administration.

The crond daemon (a type of computer program that runs in the background) enables cron functionality. The cron reads the crontab (cron tables) for running predefined scripts.

By using a specific syntax, you can configure a cron job to schedule scripts or other commands to run automatically.

What are cron jobs in Linux?

Any task that you schedule through crons is called a cron job.

Now, let's see how cron jobs work.

How to control access to crons

In order to use cron jobs, an admin needs to allow cron jobs to be added for users in the /etc/cron.allow file.

If you get a prompt like this, it means you don't have permission to use cron.

To allow John to use crons, include his name in /etc/cron.allow. Create the file if it doesn't exist. This will allow John to create and edit cron jobs.

Users can also be denied access to cron job access by entering their usernames in the file /etc/cron.d/cron.deny.

How to add cron jobs in Linux

First, to use cron jobs, you'll need to check the status of the cron service. If cron is not installed, you can easily download it through the package manager. Just use this to check:

# Check cron service on Linux system
sudo systemctl status cron.service

Cron job syntax

Crontabs use the following flags for adding and listing cron jobs:

crontab -e: edits crontab entries to add, delete, or edit cron jobs.
crontab -l: list all the cron jobs for the current user.
crontab -u username -l: list another user's crons.
crontab -u username -e: edit another user's crons.

When you list crons and they exist, you'll see something like this:

# Cron job example
* * * * * sh /path/to/script.sh

In the above example,

* represents minute(s) hour(s) day(s) month(s) weekday(s), respectively. See details of these values below:

	VALUE	DESCRIPTION
Minutes	0-59	Command will be executed at the specific minute.
Hours	0-23	Command will be executed at the specific hour.
Days	1-31	Commands will be executed in these days of the months.
Months	1-12	The month in which tasks need to be executed.
Weekdays	0-6	Days of the week where commands will run. Here, 0 is Sunday.

sh represents that the script is a bash script and should be run from /bin/bash.
/path/to/script.sh specifies the path to the script.

Below is a summary of the cron job syntax:

*   *   *   *   *  sh /path/to/script/script.sh
|   |   |   |   |              |
|   |   |   |   |      Command or Script to Execute        
|   |   |   |   |
|   |   |   |   |
|   |   |   |   |
|   |   |   | Day of the Week(0-6)
|   |   |   |
|   |   | Month of the Year(1-12)
|   |   |
|   | Day of the Month(1-31)  
|   |
| Hour(0-23)  
|
Min(0-59)

Cron job examples

Below are some examples of scheduling cron jobs.

SCHEDULE	SCHEDULED VALUE
`5 0 * 8 *`	At 00:05 in August.
`5 4 * * 6`	At 04:05 on Saturday.
`0 22 * * 1-5`	At 22:00 on every day-of-week from Monday through Friday.

It's okay if you are unable to grasp this all at once. You can practice and generate cron schedules with the crontab guru website.

How to set up a cron job

In this section, we will look at an example of how to schedule a simple script with a cron job.

Create a script called date-script.sh which prints the system date and time and appends it to a file. The script is shown below:

#!/bin/bash

echo `date` >> date-out.txt

2. Make the script executable by giving it execution rights.

chmod 775 date-script.sh

3. Add the script in the crontab using crontab -e.

Here, we have scheduled it to run per minute.

*/1 * * * * /bin/sh /root/date-script.sh

4. Check the output of the file date-out.txt. According to the script, the system date should be printed to this file every minute.

cat date-out.txt
# output
Wed 26 Jun 16:59:33 PKT 2024
Wed 26 Jun 17:00:01 PKT 2024
Wed 26 Jun 17:01:01 PKT 2024
Wed 26 Jun 17:02:01 PKT 2024
Wed 26 Jun 17:03:01 PKT 2024
Wed 26 Jun 17:04:01 PKT 2024
Wed 26 Jun 17:05:01 PKT 2024
Wed 26 Jun 17:06:01 PKT 2024
Wed 26 Jun 17:07:01 PKT 2024

How to troubleshoot crons

Crons are really helpful, but they might not always work as intended. Fortunately, there are some effective methods you can use to troubleshoot them.

1. Check the schedule.

First, you can try verifying the schedule that's set for the cron. You can do that with the syntax you saw in the above sections.

2. Check cron logs.

First, you need to check if the cron has run at the intended time or not. In Ubuntu, you can verify this from the cron logs located at /var/log/syslog.

If there is an entry in these logs at the correct time, it means the cron has run according to the schedule you set.

Below are the logs of our cron job example. Note the first column which shows the timestamp. The path of the script is also mentioned at the end of the line. Line #1, 3, and 5 show that the script ran as intended.

1 Jun 26 17:02:01 zaira-ThinkPad CRON[27834]: (zaira) CMD (/bin/sh /home/zaira/date-script.sh)
2 Jun 26 17:02:02 zaira-ThinkPad systemd[2094]: Started Tracker metadata extractor.
3 Jun 26 17:03:01 zaira-ThinkPad CRON[28255]: (zaira) CMD (/bin/sh /home/zaira/date-script.sh)
4 Jun 26 17:03:02 zaira-ThinkPad systemd[2094]: Started Tracker metadata extractor.
5 Jun 26 17:04:01 zaira-ThinkPad CRON[28538]: (zaira) CMD (/bin/sh /home/zaira/date-script.sh)

3. Redirect cron output to a file.

You can redirect a cron's output to a file and check the file for any possible errors.

# Redirect cron output to a file
* * * * * sh /path/to/script.sh &> log_file.log

8.7. Linux Networking Basics

Linux offers a number of commands to view network related information. In this section we will briefly discuss some of the commands.

View network interfaces with `ifconfig`

The ifconfig command gives information about network interfaces. Here is an example output:

ifconfig

# Output
eth0: flags=4163  mtu 1500
        inet 192.168.1.100  netmask 255.255.255.0  broadcast 192.168.1.255
        inet6 fe80::a00:27ff:fe4e:66a1  prefixlen 64  scopeid 0x20
        ether 08:00:27:4e:66:a1  txqueuelen 1000  (Ethernet)
        RX packets 1024  bytes 654321 (654.3 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 512  bytes 123456 (123.4 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

lo: flags=73  mtu 65536
        inet 127.0.0.1  netmask 255.0.0.0
        inet6 ::1  prefixlen 128  scopeid 0x10
        loop  txqueuelen 1000  (Local Loopback)
        RX packets 256  bytes 20480 (20.4 KB)
        RX errors 0  dropped 0  overruns 0  frame 0
        TX packets 256  bytes 20480 (20.4 KB)
        TX errors 0  dropped 0 overruns 0  carrier 0  collisions 0

The output of the ifconfig command shows the network interfaces configured on the system, along with details such as IP addresses, MAC addresses, packet statistics, and more.

These interfaces can be physical or virtual devices.

To extract IPv4 and IPv6 addresses, you can use ip -4 addr and ip -6 addr, respectively.

View network activity withnetstat

The netstat command shows network activity and stats by giving the following information:

Here are some examples of using the netstat command in the command line:

Display all listening and non-listening sockets:
```
 netstat -a
```
Show only listening ports:
```
 netstat -l
```
Display network statistics:
```
 netstat -s
```
Show routing table:
```
 netstat -r
```
Display TCP connections:
```
 netstat -t
```
Display UDP connections:
```
 netstat -u
```
Show network interfaces:
```
 netstat -i
```
Display PID and program names for connections:
```
 netstat -p
```
Show statistics for a specific protocol (for example, TCP):
```
 netstat -st
```
Display extended information:
```
netstat -e
```

Check network connectivity between two devices using `ping`

ping is used to test network connectivity between two devices. It sends ICMP packets to the target device and waits for a response.

ping google.com

ping tests if you get a response back without getting a timeout.

ping google.com
PING google.com (142.250.181.46) 56(84) bytes of data.
64 bytes from fjr04s06-in-f14.1e100.net (142.250.181.46): icmp_seq=1 ttl=60 time=78.3 ms
64 bytes from fjr04s06-in-f14.1e100.net (142.250.181.46): icmp_seq=2 ttl=60 time=141 ms
64 bytes from fjr04s06-in-f14.1e100.net (142.250.181.46): icmp_seq=3 ttl=60 time=205 ms
64 bytes from fjr04s06-in-f14.1e100.net (142.250.181.46): icmp_seq=4 ttl=60 time=100 ms
^C
--- google.com ping statistics ---
4 packets transmitted, 4 received, 0% packet loss, time 3001ms
rtt min/avg/max/mdev = 78.308/131.053/204.783/48.152 ms

You can stop the response with Ctrl + C.

Testing endpoints with the `curl` command

The curl command stands for "client URL". It is used to transfer data to or from a server. It can also be used to test API endpoints that helps in troubleshooting system and application errors.

As an example, you can use http://www.official-joke-api.appspot.com/ to experiment with the curl command.

The curl command without any options uses the GET method by default.

curl http://www.official-joke-api.appspot.com/random_joke
{"type":"general",
"setup":"What did the fish say when it hit the wall?","punchline":"Dam.","id":1}

curl -o saves the output to the mentioned file.

curl -o random_joke.json http://www.official-joke-api.appspot.com/random_joke
# saves the output to random_joke.json

curl -I fetches only the headers.

curl -I http://www.official-joke-api.appspot.com/random_joke
HTTP/1.1 200 OK
Content-Type: application/json; charset=utf-8
Vary: Accept-Encoding
X-Powered-By: Express
Access-Control-Allow-Origin: *
ETag: W/"71-NaOSpKuq8ChoxdHD24M0lrA+JXA"
X-Cloud-Trace-Context: 2653a86b36b8b131df37716f8b2dd44f
Content-Length: 113
Date: Thu, 06 Jun 2024 10:11:50 GMT
Server: Google Frontend

8.8. Linux Troubleshooting: Tools and Techniques

System activity report with `sar`

The sar command in Linux is a powerful tool for collecting, reporting, and saving system activity information. It's part of the sysstat package and is widely used for monitoring system performance over time.

To use sar you first need to install syssstat using sudo apt install sysstat.

Once installed, start the service with sudo systemctl start sysstat.

Verify the status with sudo systemctl status sysstat.

Once the status is active, the system will start collecting various stats that you can use to access and analyze historical data. We'll see that in detail soon.

The syntax of the sar command is as follows:

sar [options] [interval] [count]

For example, sar -u 1 3 will display CPU utilization statistics every second for three times.

sar -u 1 3
# Output
Linux 6.5.0-28-generic (zaira-ThinkPad)     04/06/24     _x86_64_    (12 CPU)

19:09:26        CPU     %user     %nice   %system   %iowait    %steal     %idle
19:09:27        all      3.78      0.00      2.18      0.08      0.00     93.96
19:09:28        all      4.02      0.00      2.01      0.08      0.00     93.89
19:09:29        all      6.89      0.00      2.10      0.00      0.00     91.01
Average:        all      4.89      0.00      2.10      0.06      0.00     92.95

Here are some common use cases and examples of how to use the sar command.

sar can be used for a variety of purposes:

1. Memory usage

To check memory usage (free and used), use:

sar -r 1 3

Linux 6.5.0-28-generic (zaira-ThinkPad)     04/06/24     _x86_64_    (12 CPU)

19:10:46    kbmemfree   kbavail kbmemused  %memused kbbuffers  kbcached  kbcommit   %commit  kbactive   kbinact   kbdirty
19:10:47      4600104   8934352   5502124     36.32    375844   4158352  15532012     65.99   6830564   2481260       264
19:10:48      4644668   8978940   5450252     35.98    375852   4165648  15549184     66.06   6776388   2481284        36
19:10:49      4646548   8980860   5448328     35.97    375860   4165648  15549224     66.06   6774368   2481292       116
Average:      4630440   8964717   5466901     36.09    375852   4163216  15543473     66.04   6793773   2481279       139

This command displays memory statistics every second three times.

2. Swap space utilization

To view swap space utilization statistics, use:

sar -S 1 3

sar -S 1 3
Linux 6.5.0-28-generic (zaira-ThinkPad)     04/06/24     _x86_64_    (12 CPU)

19:11:20    kbswpfree kbswpused  %swpused  kbswpcad   %swpcad
19:11:21      8388604         0      0.00         0      0.00
19:11:22      8388604         0      0.00         0      0.00
19:11:23      8388604         0      0.00         0      0.00
Average:      8388604         0      0.00         0      0.00

This command helps monitor the swap usage, which is crucial for systems running out of physical memory.

3. I/O devices load

To report activity for block devices and block device partitions:

sar -d 1 3

This command provides detailed stats about data transfers to and from block devices, and is useful for diagnosing I/O bottlenecks.

5. Network statistics

To view network statistics, like number of packets received (transmitted) by the network interface:

sar -n DEV 1 3
# -n DEV tells sar to report network device interfaces
sar -n DEV 1 3
Linux 6.5.0-28-generic (zaira-ThinkPad)     04/06/24     _x86_64_    (12 CPU)

19:12:47        IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
19:12:48           lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
19:12:48       enp2s0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
19:12:48       wlp3s0     10.00      3.00      1.83      0.37      0.00      0.00      0.00      0.00
19:12:48    br-5129d04f972f      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
.
.
.

Average:        IFACE   rxpck/s   txpck/s    rxkB/s    txkB/s   rxcmp/s   txcmp/s  rxmcst/s   %ifutil
Average:           lo      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
Average:       enp2s0      0.00      0.00      0.00      0.00      0.00      0.00      0.00      0.00
...output truncated...

This displays network statistics every second for three seconds, helping in monitoring network traffic.

6. Historical data

Recall that previously we installed the sysstat package and ran the service. Follow the steps below to enable and access historical data.

Enable data collection: Edit the sysstat configuration file to enable data collection.

 sudo nano /etc/default/sysstat

Change ENABLED="false" to ENABLED="true".

 vim /etc/default/sysstat
 #
 # Default settings for /etc/init.d/sysstat, /etc/cron.d/sysstat
 # and /etc/cron.daily/sysstat files
 #

 # Should sadc collect system activity informations? Valid values
 # are "true" and "false". Please do not put other values, they
 # will be overwritten by debconf!
 ENABLED="true"

Configure data collection interval: Edit the cron job configuration to set the data collection interval.
```
 sudo nano /etc/cron.d/sysstat
```
By default, it collects data every 10 minutes. You can adjust the interval by modifying the cron job schedule. The relevant files will go to the /var/log/sysstat folder.
View historical data: Use the sar command to view historical data. For example, to view CPU usage for the current day:
```
 sar -u
```
To view data from a specific date:
```
 sar -u -f /var/log/sysstat/sa
```
Replace
with the day of the month for which you want to view the data.

In the below command, /var/log/sysstat/sa04 gives stats for the 4th day of the current month.

sar -u -f /var/log/sysstat/sa04
Linux 6.5.0-28-generic (zaira-ThinkPad)     04/06/24     _x86_64_    (12 CPU)

15:20:49     LINUX RESTART    (12 CPU)

16:13:30     LINUX RESTART    (12 CPU)

18:16:00        CPU     %user     %nice   %system   %iowait    %steal     %idle
18:16:01        all      0.25      0.00      0.67      0.08      0.00     99.00
Average:        all      0.25      0.00      0.67      0.08      0.00     99.00

7. Real-Time CPU Interruptions

To observe real-time interrupts per second served by the CPU, use this command:

sar -I SUM 1 3

# Output
Linux 6.5.0-28-generic (zaira-ThinkPad)     04/06/24     _x86_64_    (12 CPU)

19:14:22         INTR    intr/s
19:14:23          sum   5784.00
19:14:24          sum   5694.00
19:14:25          sum   5795.00
Average:          sum   5757.67

This command helps in monitoring how frequently the CPU is handling interrupts, which can be crucial for real-time performance tuning.

These examples illustrate how you can use sar to monitor various aspects of system performance. Regular use of sar can help in identifying system bottlenecks and ensuring that applications keep running efficiently.

8.9. General Troubleshooting Strategy for Servers

Why do we need to understand monitoring?

System monitoring is an important aspect of system administration. Critical applications demand a high level of proactiveness to prevent failure and reduce the outage impact.

Linux offers very powerful tools to gauge system health. In this section, you'll learn about the various methods available to check your system's health and identify the bottlenecks.

Find load average and system uptime

System reboots may occur which can sometimes mess up some configurations. To check how long the machine has been up, use the command: uptime. In addition to the uptime, the command also displays load average.

[user@host ~]$ uptime 19:15:00 up 1:04, 0 users, load average: 2.92, 4.48, 5.20

Load average is the system load over the last 1, 5, and 15 minutes. A quick glance indicates whether the system load appears to be increasing or decreasing over time.

Note: Ideal CPU queue is 0. This is only possible when there are no waiting queues for the CPU.

Per-CPU load can be calculated by dividing load average with the total number of CPUs available.

To find the number of CPUs, use the command lscpu.

lscpu
# output
Architecture:            x86_64
  CPU op-mode(s):        32-bit, 64-bit
  Address sizes:         48 bits physical, 48 bits virtual
  Byte Order:            Little Endian
CPU(s):                  12
  On-line CPU(s) list:   0-11
.
.
.
output omitted

If the load average seems to increase and does not come down, the CPUs are overloaded. There is some process that is stuck or there is a memory leakage.

Calculating free memory

Sometimes, high memory utilization might be causing problems. To check the available memory and the memory in use, use the free command.

free -mh
# output
               total        used        free      shared  buff/cache   available
Mem:            14Gi       3.5Gi       7.7Gi       109Mi       3.2Gi        10Gi
Swap:          8.0Gi          0B       8.0Gi

Calculating disk space

To ensure the system is healthy, don't forget about the disk space. To list all the available mount points and their respective used percentage, use the below command. Ideally, utilized disk spaces should not exceed 80%.

The df command provides detailed disk spaces.

df -h
Filesystem      Size  Used Avail Use% Mounted on
tmpfs           1.5G  2.4M  1.5G   1% /run
/dev/nvme0n1p2  103G   34G   65G  35% /
tmpfs           7.3G   42M  7.2G   1% /dev/shm
tmpfs           5.0M  4.0K  5.0M   1% /run/lock
efivarfs        246K   93K  149K  39% /sys/firmware/efi/efivars
/dev/nvme0n1p3  130G   47G   77G  39% /home
/dev/nvme0n1p1  511M  6.1M  505M   2% /boot/efi
tmpfs           1.5G  140K  1.5G   1% /run/user/1000

Determining process states

Process states can be monitored to see any stuck process with a high memory or CPU usage.

We saw previously that the ps command gives useful information about a process. Have a look at the CPU and MEM columns.

[user@host ~]$ ps aux
USER         PID %CPU %MEM    VSZ   RSS TTY      STAT START   TIME COMMAND
 runner         1  0.1  0.0 1535464 15576 ?       S  19:18   0:00 /inject/init
 runner        14  0.0  0.0  21484  3836 pts/0    S   19:21   0:00 bash --norc
 runner        22  0.0  0.0  37380  3176 pts/0    R+   19:23   0:00 ps aux

Real-time system monitoring

Real time monitoring gives a window into the realtime system state.

One utility you can use to do this is the top command.

The top command displays a dynamic view of the system's processes, displaying a summary header followed by a process or thread list. Unlike its static counterpart ps, top continuously refreshes the system stats.

With top, you can see well-organised details in a compact window. There a number of flags, shortcuts, and highlighting methods that come along with top.

You can also kill processes using top. For that, press k and then enter the process id.

Interpreting logs

System and application logs carry tons of information about what the system is going through. They contain useful information and error codes that point towards errors. If you search for error codes in logs, issue identification and rectification time can be greatly reduced.

Network ports analysis

The network aspect should not be ignored as network glitches are common and may impact the system and traffic flows. Common network issues include port exhaustion, port choking, unreleased resources, and so on.

To identify such issues, we need to understand port states.

Some of the port states are explained briefly here:

State	Description
LISTEN	Represents ports that are waiting for a connection request from any remote TCP and port.
ESTABLISHED	Represents connections that are open and data received can be delivered to the destination.
TIME WAIT	Represents waiting time to ensure acknowledgment of its connection termination request.
FIN WAIT2	Represents waiting for a connection termination request from the remote TCP.

Let's explore how we can analyze port-related information in Linux.

Port ranges: Port ranges are defined in the system, and range can be increased/decreased accordingly. In the below snippet, the range is from 15000 to 65000, which makes a total of 50000 (65000 - 15000) available ports. If utilized ports are reaching or exceeding this limit, then there is an issue.

[user@host ~]$ /sbin/sysctl net.ipv4.ip_local_port_range
net.ipv4.ip_local_port_range = 15000    65000

The error reported in logs in such cases can be Failed to bind to port or Too many connections.

Identifying packet loss

In system monitoring, we need to ensure that the outgoing and incoming communication is intact.

One helpful command is ping. ping hits the destination system and brings the response back. Note the last few lines of statistics that show packet loss percentage and time.

# ping destination IP
[user@host ~]$ ping 10.13.6.113
 PING 10.13.6.141 (10.13.6.141) 56(84) bytes of data.
 64 bytes from 10.13.6.113: icmp_seq=1 ttl=128 time=0.652 ms
 64 bytes from 10.13.6.113: icmp_seq=2 ttl=128 time=0.593 ms
 64 bytes from 10.13.6.113: icmp_seq=3 ttl=128 time=0.478 ms
 64 bytes from 10.13.6.113: icmp_seq=4 ttl=128 time=0.384 ms
 64 bytes from 10.13.6.113: icmp_seq=5 ttl=128 time=0.432 ms
 64 bytes from 10.13.6.113: icmp_seq=6 ttl=128 time=0.747 ms
 64 bytes from 10.13.6.113: icmp_seq=7 ttl=128 time=0.379 ms
 ^C
 --- 10.13.6.113 ping statistics ---
 7 packets transmitted, 7 received,0% packet loss, time 6001ms
 rtt min/avg/max/mdev = 0.379/0.523/0.747/0.134 ms

Packets can also be captured at runtime using tcpdump. We'll look into it later.

Gathering stats for issue post mortem

It is always a good practice to gather certain stats that would be useful for identifying the root cause later. Usually, after system reboot or services restart, we loose the earlier system snapshot and logs.

Below are some of the methods to capture system snapshot.

Logs Backup

Before making any changes, copy log files to another location. This is crucial for understanding what condition the system was in during time of issue. Sometimes log files are the only window to look into past system states as other runtime stats are lost.

TCP Dump

Tcpdump is a command-line utility that allows you to capture and analyze incoming and outgoing network traffic. It is mostly used to help troubleshoot network issues. If you feel that system traffic is being impacted, take tcpdump as follows:

sudo tcpdump -i any -w

# Where,
# -i any captures traffic from all interfaces
# -w specifies the output filename

# Stop the command after a few mins as the file size may increase
# use file extension as .pcap

Once tcpdump is captured, you can use tools like Wireshark to visually analyze the traffic.

8.10 Diagnosing Hardware Problems

Troubleshooting unexpected issues is a part of the learning process. Sometimes, you may notice frequent segmentation faults (SIGSEGV), overheating, or random crashes across unrelated applications. The issue could either be software or hardware related. While software-related issues depend on the specific application itself, hardware issues can be diagnosed with some standard steps.

In this section, we will discuss how to diagnose and rule out hardware issues related to memory, CPU, system sensors, power supply, and more.

8.10.1 Analyzing Memory Performance

Determine Available RAM

If you feel your system is getting slow and taking longer to finish tasks, check your system's available memory. This will ensure there is enough available memory including the swap memory.

The command to check available memory is free -mh, where -h is for human-readable output and -m is for displaying memory in MB.

free -mh
               total        used        free      shared  buff/cache   available
Mem:            14Gi       5.1Gi       2.4Gi        77Mi       7.3Gi       9.3Gi
Swap:          4.0Gi          0B       4.0Gi

In the above output, look at the "available" column in the "Mem" row. This shows how much RAM is free for use.

Another way to check the memory in real time is to use the top command. There are 2 ways to do this:

When you are in top, press Shift + M to sort the processes by memory usage.
Alternately, press m to see the memory usage in a progress bar like format:

If you see the memory consumed near to 100%, you might want to consider identifying the process that is consuming the memory and take necessary action. You might also want to consider adding more memory to your system.

Run a stress test on your hardware

The memtester command is a utility used for diagnosing memory-related issues by stressing the memory and checking for faults. It is often used in situations where you suspect faulty RAM might be causing system instability or crashes.

Here's how to use it effectively:

First, install memtester.
```
  sudo apt install memtester
```
Determine the amount of RAM to test and the number of passes you’d like your RAM to go through. In the command below, 1G is the amount of RAM to test (1 GB), and 5 is the number of test passes:
```
  sudo memtester 1G 5
```

If all tests pass, your RAM is likely error-free. If errors are reported, your RAM might be faulty and could require replacement or further inspection. You can always run the test again with a different amount of RAM or test passes.

Note that, you shouldn't test too much memory at once, as your system also needs memory for running processes. If you have more RAM than can be tested at once, test in smaller segments sequentially.

Below is a snippet of the memtester output if all tests pass. Notice the ”ok” status for each test.

memtester version 4.5.1 (64-bit)
Copyright (C) 2001-2020 Charles Cazabon.
Licensed under the GNU General Public License version 2 (only).

pagesize is 4096
pagesizemask is 0xfffffffffffff000
want 1024MB (1073741824 bytes)
got  1024MB (1073741824 bytes), trying mlock ...locked.
Loop 1/5:
  Stuck Address       : ok
  Random Value        : ok
  Compare XOR         : ok
  Compare SUB         : ok
  Compare MUL         : ok
  Compare DIV         : ok
  Compare OR          : ok
  Compare AND         : ok
  Sequential Increment: ok
  Solid Bits          : ok
  Block Sequential    : ok
  Checkerboard        : ok
  Bit Spread          : ok
  Bit Flip            : ok
  Walking Ones        : ok
  Walking Zeroes      : ok
  8-bit Writes        : ok
  16-bit Writes       : ok
.
.
.

Below is a snippet of the output if a test fails. Notice the FAILURE status for each test.

memtester version 4.5.1 (64-bit)
Copyright (C) 2001-2020 Charles Cazabon.
Licensed under the GNU General Public License version 2 (only).

pagesize is 4096
pagesizemask is 0xfffffffffffff000
want 1024MB (1073741824 bytes)
got  1024MB (1073741824 bytes), trying mlock ...locked.
Loop 1/5:
  Stuck Address       : testing   1FAILURE: possible bad address line at offset 0x25378a58.
Skipping to next test...
  Random Value        : FAILURE: 0x4df704aaafdf8848 != 0x4df704aaafdfc848 at offset 0x05379a48.
  Compare XOR         : ok
  Compare SUB         : ok
  Compare MUL         : ok
  Compare DIV         : ok
  Compare OR          : ok
  Compare AND         : ok
  Sequential Increment: ok
  Solid Bits          : testing   6FAILURE: 0x00000000 != 0x00004000 at offset 0x05379a48.
  Block Sequential    : testing   3FAILURE: 0x303030303030303 != 0x303030303034303 at offset 0x05379a48.
  Checkerboard        : testing   0FAILURE: 0xaaaaaaaaaaaaaaaa != 0xaaaaaaaaaaaaeaaa at offset 0x05379a48.
  Bit Spread          : testing  12FAILURE: 0xffffffffffffafff != 0xffffffffffffefff at offset 0x05379a48.
  Bit Flip            : testing   0FAILURE: 0x00000001 != 0x00004001 at offset 0x05379a48.
  Walking Ones        : ok
  Walking Zeroes      : testing   0FAILURE: 0x00000001 != 0x00001001 at offset 0x053af9f8.
  8-bit Writes        : -FAILURE: 0x57c7c8ba7d6f5b3b != 0x57c7c8ba7d6f1b3b at offset 0x0537da28.
  16-bit Writes       : -FAILURE: 0xd7768894fbf79099 != 0xd7768894fbf7d099 at offset 0x05379a48.
FAILURE: 0xfffc5633ffefca5d != 0xfffc5633ffefda5d at offset 0x053a5a38.
.
.
.

If errors persist across all test loops, it strongly suggests hardware issues, not transient software glitches.

8.10.2 Identifying Overheating Issues

Overheating can cause unexpected errors and crashes. To diagnose overheating issues, you can use a command line utility lm-sensors.

lm-sensors allow syou monitor hardware health by reading data from various sensors. It provides information about system temperatures, voltages, and fan speeds.

Here's how you can identify and monitor your system temperature using lm-sensors:

First, install lm-sensors:
```
  sudo apt install lm-sensors
```
Detect the available sensors on your system:
```
  sudo sensors-detect
```
Follow the prompts and answer “YES” to detect the available sensors on your system.
Once the available sensors are detected, you can view the temperature of your system using the sensors command:
```
  sensors
```
In the output below, you can see the temperature reading at the edge of the GPU, which is 41.0 degrees Celsius. You can also see other pieces of information like voltage supplied, power consumption and voltage supplied.
```
  amdgpu-pci-0400
  Adapter: PCI adapter
  vddgfx:      731.00 mV 
  vddnb:       687.00 mV 
  edge:         +41.0°C  
  PPT:           7.00 W
```
Using lm-sensors ensures that the system is operating within safe parameters. It helps to detect potential hardware problems early and take corrective actions to prevent hardware damage.

8.10.3 Evaluating Hard Drive Health

Disk errors can also cause application crashes. To identify disk issues, you can run disk check using smartmontools:

First, install smartmontools:
```
  sudo apt install smartmontools
```
Run a quick health check using the command below and replace /dev/sdX with your disk name (check with lsblk).
```
  sudo smartctl -H /dev/sdX
```

Here is the result I got when I ran the command on my disk /dev/nvme0n1:

  sudo smartctl -H /dev/nvme0n1
  smartctl 7.4 2023-08-01 r5530 [x86_64-linux-6.8.0-52-generic] (local build)
  Copyright (C) 2002-23, Bruce Allen, Christian Franke, www.smartmontools.org

  === START OF SMART DATA SECTION ===
  SMART overall-health self-assessment test result: PASSED

You can also run a detailed test:
```
  sudo smartctl -a /dev/nvme0n1
```

The detailed test provides a full report, including:

Temperature
Power-on hours
Error counts
Wear leveling (for SSDs), and more.

8.10.4 Conducting a CPU Stress Test

Faulty CPUs can also lead to a number of performance issues. To test your CPU, you can use the stress-ng utility:

Install stress-ng:
```
  sudo apt install stress-ng
```
Run a CPU stress test:
```
  stress-ng --cpu 4 --timeout 60
```

In the above command, 4 is the number of CPU cores you’d like to test and 60 is the duration in seconds. The command will stress all 4 CPU cores for 60 seconds. Notice the CPU is at 100% load during the test:

If the system crashes during this test, the CPU may be faulty.

8.10.5 Examining System Logs for Errors

systemd is a Linux system manager responsible for booting the system, managing system processes, and handling system services.

journalctl is a command to query the systemd journal logs. It provides detailed logging for system processes, kernel events, user applications, and more.

You can check system logs for hardware-related errors using the command: journalctl -k | grep -iE "error|fault|panic".

In the logs, look for messages about:

Memory faults.
I/O errors.
Hardware timeouts.

Here is what errors in the log file can look like:

Feb 11 10:15:32 hostname kernel: [Hardware Error]: CPU 0: Machine Check: 0 Bank 4: b200000000070f0f
Feb 11 10:15:32 hostname kernel: [Hardware Error]: TSC 0 ADDR fef1c000 MISC 38a0000086 
Feb 11 10:15:32 hostname kernel: [Hardware Error]: PROCESSOR 0:306a9 TIME 1613045732 SOCKET 0 APIC 0 microcode 1f
Feb 11 10:16:45 hostname kernel: EXT4-fs error (device sda1): ext4_find_entry:1453: inode #2: comm ls: reading directory lblock 0
Feb 11 10:17:12 hostname kernel: [drm:drm_atomic_helper_commit_cleanup_done [drm_kms_helper]] *ERROR* [CRTC:36:pipe A] flip_done timed out
Feb 11 10:18:05 hostname kernel: Kernel panic - not syncing: Fatal exception

Conclusion

Thank you for reading the book until the end. If you found it helpful, consider sharing it with others.

This book doesn't end here, though. I will continue to improve it and add new materials in the future. If you found any issues or if you would like to suggest any improvements, feel free to open a PR/ Issue.

Stay Connected and Continue Your Learning Journey!

Your journey with Linux doesn't have to end here. Stay connected and take your skills to the next level:

Follow Me on Social Media:
- X: I share useful short form content there. My DMs are always open.
- LinkedIn: I share articles and posts on tech there. Leave a recommendation on LinkedIn and endorse me on relevant skills.
Get access to exclusive content: For one-on-one help and exclusive content go here.

My articles and books, like this one, are part of my mission to increase accessibility to quality content for everyone. This book will also be open to translation in other languages. Each piece takes a lot of time and effort to write. This book will be free, forever. If you've enjoyed my work and want to keep me motivated, consider buying me a coffee.

Thank you once again and happy learning!

How Do Numerical Conversions Work in Computer Systems? Explained With Examples

Zaira Hira — Wed, 29 May 2024 19:56:06 +0000

Computers perform complex calculations when carrying out their assigned tasks. At the very core, the calculations boil down to operations like comparisons, assignments, and addition.

Have you ever wondered how they are performed under the hood and why they are important? At a fundamental level, a computer works by performing various numerical conversions.

In this article, you'll learn the following concepts:

The importance of numerical systems in computers.
Types of numerical systems.
Numerical conversion techniques.
Application of different numerical systems.
Mini exercises to keep you engaged along the way.

Types of Numerical Systems

Numerical conversion is the process of converting numbers from one numeral system to another. In computer systems, the common numeral systems include decimal (base-10), binary (base-2), hexadecimal (base-16), and octal (base-8).

But What Is a Base?

In mathematics and computer science, the term "base" refers to the number of unique digits or symbols used in a positional numeral system. Each digit's value is multiplied by the base raised to the power of its position in the number, starting from the rightmost digit, which represents the units place.

Here's an explanation of the commonly encountered numeral systems:

Base-2 (Binary):
- Base-2, or binary, uses only two symbols: 0 and 1.
- Each digit's value is a power of 2, with positions increasing from right to left.
Base-10 (Decimal):
- Base-10, or decimal uses ten symbols from 0 to 9.
- Each digit's value is a power of 10, with positions increasing from right to left.
Base-8 (Octal):
- Base-8, or octal, uses eight symbols: 0 to 7.
- Each digit's value is a power of 8, with positions increasing from right to left.
Base-16 (Hexadecimal):
- Base-16, or hexadecimal, uses sixteen symbols: 0 to 9 and A to F (representing 10 to 15).
- Each digit's value is a power of 16, with positions increasing from right to left.
- Below is a table showing the mapping of hexadecimal numbers from 10 with alphabets.

Character	Hexadecimal
A	10
B	11
C	12
D	13
E	14
F	15

This notation is commonly used to simplify the representation of binary-coded values.

Importance of Understanding Numerical Systems in Computers

Learning numeral conversions in computer science is essential for several reasons:

Understanding Data Representation: Computers store and manipulate data using binary (base-2) representation. Knowing how to convert between numeral systems helps in understanding how data is stored and processed at the fundamental level.
Addressing Memory: Memory addresses in computers are frequently represented in hexadecimal format. Knowing how to convert between decimal and hexadecimal is crucial for understanding memory management and for debugging.
Networking and Communication: In networking, IP addresses and MAC addresses are often represented in hexadecimal format. Understanding hexadecimal conversion is thus comes in handy for networking professionals.
Cryptography: In cryptography, hexadecimal numbers are frequently used to represent keys, cipher texts, and other cryptographic data. Understanding numeral conversions helps in understanding cryptographic operations.

Conversion Techniques

In this section, you'll learn techniques to convert one number system to another.

Decimal to Binary

Step-by-Step Conversion Process:

Divide the number by 2: The first step is to divide the number by 2 and record the remainder.
Divide the quotient by 2 repetitively: Divide the quotient from step 1 and record the remainder. Continue to divide and record the remainder till 1 remains as the quotient.
Find the solution in reverse order: Starting from the last quotient that would be 1, go upwards to get the final answer.

Example Conversion:

Let's say you want the binary equivalent of 17, then the process would be like this:

Operation	Result	Remainder
17/2	8	1 ⬆️
8/2	4	0 ⬆️
4/2	2	0 ⬆️
2/2	1 ➡️	0 ⬆️

To find the final answer, follow the arrows. Start from the bottom where the result is 1 and go upwards. You'll get 10001.

So,

$$17_{10} = 10001_{2}$$

Let's try a bigger number 55

Operation	Result	Remainder
55/2	27	1 ⬆️
27/2	13	1 ⬆️
13/2	6	1 ⬆️
6/2	3	0 ⬆️
3/2	1 ➡️	1 ⬆️

So,

$$55_{10} = 110111_{2}$$

Now, your turn:

What is 67 in binary?

Show Answer

How to Create Diagrams as Code with Mermaid, GitHub, and Visual Studio Code

Zaira Hira — Wed, 06 Sep 2023 20:51:45 +0000

Diagrams are an important part of technical documentation. In this article we will explore how we can use code to generate diagrams and leverage them in Markdown.

Here is an example of how a code generated diagram looks like:

Diagram generated by code

No tools, no dragging, aligning and snapping shapes to places. Just code. Interesting right?

Pre-requisites

To follow along with this tutorial, you should have the following:

A basic understanding of Markdown. Have a look at this guide if you aren't familiar with Markdown.
Visual Studio Code with a Markdown previewing extension (covered in detail in coming sections).
GitHub, to understand how you can render diagrams there.

Advantages of Generating Diagrams as Code

There are several advantages of generating diagrams as code rather than using traditional methods to manually 'create' diagrams. Let's see some of them:

Dynamic: Code-generated diagrams are dynamic which means that you can easily update them by updating the code.
Editable: With code-generated images, you are not relying on complex image generating tools. You can simply edit the code in a text editor and get an updated image.
Efficient: Code-generated images are fast to load compared to static images. Also you don't need to host them separately as an image in your website.
Quick to create: You can use a template and quickly come up with diagrams just using code. You don't need to invest the time and effort in learning image creation tools, that are usually overwhelming to start with.

How to Render and View Mermaid Code

There are several ways you can create and view Mermaid diagrams.

VSCode

First, install a Markdown previewer that has the support for Mermaid. Here is an example of such an extension: Markdown Preview Mermaid Support.

Create an empty Markdown file using the extension .md. Write your code in this file and open the preview on the right pane:

Code and preview in VSCode

StackEdit

StackEdit is an online, browser-based Markdown previewer. It provides an editor window and a preview on the right side.

Code and preview in StackEdit

GitHub

You can add Mermaid diagrams to create compelling README files in GitHub. Simply edit the README.md or any other Markdown file in GitHub to render Mermaid diagrams.

Edit code in GitHub

View and render Mermaid diagram in GitHub

How to Generate Diagrams as Code with Mermaid

Mermaid is a JavaScript-based tool that transforms Markdown-style text into dynamic diagrams, allowing you to create and modify them effortlessly.

Mermaid makes it easy to generate diagrams and visuals using simple text and code.

It follows a simple syntax:

```mermaid
    your code goes here
    .
    .
    .

```

With Mermaid, you can generate the following diagram types:

Flowchart
Sequence Diagram
Class Diagram
Gantt
Pie Chart

Let's explore them one by one.

How to Create a Flowchart

A flowchart is a picture that shows the steps of a process using symbols, helping to explain the process in a clear and organized way. A flow chart is composed of nodes that are connected by arrows.

You can render flowcharts in mermaid using this example:

```mermaid
flowchart TD;
    A[Start] --> B[Process 1];
    B --> C[Process 2];
    C --> D[End];
```

Result:

Flowcharts in mermaid

Syntax breakdown:

flowchart: This keyword tells that we are creating a flowchart.
TD: this is the orientation of the flowchart and stands for Top to Down. The other FlowChart orientations are:
– TB - Top to bottom,same as TD.
– BT - Bottom to top
– RL - Right to left
– LR - Left to right
-->: The direction of the arrow connecting the nodes.

How to Create a Sequence Diagram

A sequence diagram is an illustrative representation of interactions between processes, demonstrating their operational flow and the sequence of execution.

You can render sequence diagrams in mermaid using this syntax:

```mermaid
sequenceDiagram
    participant Client
    participant Server
    Client->>Server: Register user
    activate Server
    Server-->>Client: User already exists.
    deactivate Server

```

Result:

Sequence diagram in mermaid

Syntax breakdown:

sequenceDiagram: This keyword specifies that we are making a sequence diagram.
participant: These are the participants or the actors in a sequence diagram.
activate/ deactivate: It is possible to activate and deactivate an actor. Activation shows as a small rectangle between interactions.
-->>: Connecting lines (dashed).
->: Connecting lines (solid).

How to Create a Class Diagram

Class diagrams are used to visualize the structure and relationships of classes, interfaces, and their interactions in object-oriented programming (OOP). You can render class diagrams in Mermaid using this syntax:

```mermaid
classDiagram
    class Animal {
        +name: string
        +age: int
        +makeSound(): void
    }

    class Dog {
        +breed: string
        +bark(): void
    }

    class Cat {
        +color: string
        +meow(): void
    }

    Animal <|-- Dog
    Animal <|-- Cat

```

Result:

Class diagram in mermaid

Syntax breakdown:

Defining classes: An individual class is defined with the class keyword followed by the class name.
Adding class properties: Class properties are added with a + sign. In the example above, +breed is a property.
Adding methods: Methods are recognized by mermaid using (), the round braces. Note that all the properties and methods are grouped under the same class using curly braces.
Defining return types: Return types are defined after the method name/ class name. string and void are the return types in the above code.
Defining relation ships: In OOP, relationships are the specific types of logical connections found on class and object diagrams. The following relationship types are supported in mermaid:

Type	Description
<	--	Inheritance
*--	Composition
o--	Aggregation
-->	Association
--	Link (Solid)
..>	Dependency
..	>	Realization
..	Link (Dashed)

Here is a quick implementation of inheritance:

Inheritance

How to Create a Gantt Chart

A Gantt chart serves as a visual representation in the form of a bar chart. It effectively showcases a project's timeline, revealing the duration required for individual project components to reach completion.

You can render Gantt charts in Mermaid using this example:

```mermaid
gantt
    title Project Schedule
    dateFormat YYYY-MM-DD
    axisFormat %m/%d

    section Planning
    Define Project : 2023-01-01, 7d
    Research : 2023-01-08, 14d
    Define Requirements : 2023-01-22, 7d

    section Development
    Design : 2023-01-29, 21d
    Implementation : 2023-02-19, 28d

    section Testing
    Unit Testing : 2023-03-19, 14d
    Integration Testing : 2023-04-02, 14d

    section Deployment
    Deploy : 2023-04-16, 7d
    User Training : 2023-04-23, 14d

    section Maintenance
    Ongoing Support : 2023-05-07, 30d

```

Result:

Gantt charts in mermaid

Syntax breakdown:

gantt specifies that you want to render a Gantt chart in mermaid.
titlerepresents the project title.
dateFormat YYYY-MM-DD is the input date format for the chart.
axisFormat %m/%d represents the shown date format, the output date, in the x-axis.
section is used to separate different parts of the project.

The x axis represents time and the y records the different tasks and the order in which they are to be completed.

How to Create a Pie Chart

A pie chart, also known as a circle chart, is a round visual representation used for displaying statistical data. It is divided into segments or slices to visually convey the numerical proportions of different categories or values.

Here is an example to create a pie chart in mermaid:

```mermaid
pie
    title Distribution of Expenses
    "Food" : 60
    "Rent" : 15
    "Entertainment" : 10
    "Savings" : 15

```

Result:

Pie charts in mermaid

Syntax breakdown:

In this example:

pie indicates that we are creating a pie chart.
title sets the title for the pie chart.
The items in double quotes represent the categories (e.g., "Food," "Rent," "Entertainment," "Savings").
The numbers after the colons represent the proportions or percentages for each category. In this case, "Food" occupies 60% of the chart, "Rent" 15%, "Entertainment" 10%, and "Savings" 15%.

Wrapping up

In this article, we just scratched the surface of rendering diagrams in Mermaid. The diagrams are highly customizable and allow you to create very complex diagrams as well.

Mermaid supports even more types of diagrams that are not mentioned in this tutorial. To learn more about Mermaid, check out their official documentation.

I hope you found this article helpful. I would love to connect with you on any of these platforms.

See you in the next tutorial, happy coding 😁

How I Got Accepted as an Author at freeCodeCamp

Zaira Hira — Tue, 01 Aug 2023 21:30:58 +0000

Life is full of surprises. Certain actions may lead to a chain of events and you find yourself in a completely different state.

One of my biggest achievements in 2021 was to get accepted as an author at freeCodeCamp.

I have been writing articles on freeCodeCamp's community publication since September 2021 and have published 31 articles. I write mostly on Linux and occasionally on Python and Git. You can check out my author profile here.

I have always enjoyed writing. I used to casually write stories or simply journaled about how my day went. But I never made any of my writings public.

I had been working as an application support admin for a few years. I learned a lot of interesting things during my routine work, and I loved making notes for myself.

I started to share them with my colleagues too, and realized they actually found them useful. I also noticed that people usually run away from documentation. But the drawback is that you may forget things very easily if the tech is complicated and there is a lot going on.

How I Discovered freeCodeCamp

It was a routine day at work when I was scrolling through LinkedIn. I saw that one of my colleagues had recently completed the Responsive Design Course on freeCodeCamp and had shared his certificate there. I congratulated him.

At the same time, I got curious about what freeCodeCamp was. At that time, I was also looking to polish my development skills.

I searched for freeCodeCamp online and followed them on Twitter. I had recently started using Twitter for learning about tech. Over there I saw that people were writing articles which freeCodeCamp was regularly sharing.

I explored some of the author profiles and the popular topics. I had always thought that in order to write a blog online, you had to be famous and have a huge social following. I also believed that the topic should be very complex and should be written about by an expert with years and years of experience.

I got even more curious and approached Abbey on Twitter. I asked how I could join as an author. That was the turning point.

She sent me the publication guide, and I read it a couple of times. Then I decided to apply. There was a Google form with some questions. I shared some of my sample articles that I luckily had. At that time, I was writing as a guest author for a YouTuber's blog.

But...some time went by. I couldn't get a response.👀 To be honest, I was anxious.

I was eagerly waiting for an answer. Just before I was about to give up, I decided to check my spam folder. And there it was – the email that I had been waiting for!

Tips to Help You Get Accepted

If you are keen about writing and showing it to the world, you should definitely apply for freeCodeCamp's author program. Here are some tips to strengthen your application.

Have three writing samples ready – and show your best samples. You can publish some of your sample articles on open publishing platforms like Hashnode and Dev.to to get practice writing technical tutorials before applying to freeCodeCamp.
Don't think that you have to be an expert in every topic or you need to be a tech influencer to apply. Just make sure you do your research and present the information clearly with the primary goal of educating your readers.
There is no harm in applying again with better samples if you didn't qualify the first time. Your application would still be reviewed – just email Abbey or message her on Twitter to give her a heads up.

Also, keep in mind that quality is always better than quantity. Here's a few additional tips related to that.

Plagiarism is never acceptable. So don't think you can get away with it. 👮‍♂️ If you're not sure what constitutes plagiarism, read up on it before you apply. If you think some of your articles contain plagiarism, write some more that don't. It's better for you, and everyone.
Be careful with using GPT. You can use it to assist you – help you create outlines, code samples (which you should test to make sure they work/run), and so on – and not to write the whole article for you. After all, your unique writing style is what makes you stand out. It's worth the extra time, plus you'll learn a whole lot more.

What Did I Get in Return as an Author?

The short answer is – meaningful connections!

The freeCodeCamp community is huge and powerful. I got many follows on my socials being on Twitter and LinkedIn (as freeCodeCamp shares many of the articles they publish on social media).

In the past 12 months, my articles have reached 1,904,450 people! I wasn't expecting to reach so many people when I was starting out.

I have also received appreciation from some of the kind readers. Some of my articles have also been featured in Quincy's weekly newsletter.

Wrapping up

If you have something you would like to share with the developer community, I would highly suggest you start writing. Whether you publish on your own blog, an open publishing site, or you apply and are accepted to freeCodeCamp's publication – I believe you'll see the benefits. You will also meet many great people along the way. 🤗

BTW, I would be happy to connect with you on any of these platforms. 📧

Happy coding :D

How to Write Faster HTML and CSS Code Using Emmet

Zaira Hira — Wed, 26 Jul 2023 14:24:12 +0000

Emmet is an essential development tool that helps you quickly create repetitive structures like lists, tables, or ordered elements with minimal typing. It is more like a shorthand that translates to multiple lines of HTML or a CSS attribute.

Emmet is built-in in many of the text editors like VS Code. However, you can also download it for your favorite text editor from this link. Emmet is also available inside some online IDEs like CodePen.

Look closely at the example below. See how easily we generated ten lines of code using only the keywords ol>li{This is a list}*10?

Emmet doing its magic

We skipped copying and pasting ten lines and then counting to see if there were ten items. Doing this manually also introduces a chance of human error.

In this tutorial, we will learn some Emmet shortcuts that generate HTML and CSS quickly.

Emmet Shortcuts for HTML

How to Create Comments

To comment a single line, press Ctrl + /.

To comment multiple lines, select them and then press Ctrl + /.

Multiplication in Emmet

You can create multiple elements by specifying the count after *.

For example, p*10 would generate 10 p elements.

Create any number of elements using multiplication

How to Create Direct children

You can create direct children using the > symbol.

ol>li would create an li inside the ol element.

Creating direct children

How to Create Siblings

You can create siblings using the+ symbol.

div+p would create a div element and a p element on the same level.

Creating siblings

How to Combine selectors

You can combine the sibling and the child selector to create a structure. For example div+p>span would result in a div and a p element on the same level with a span element as a child of p selector:

Combining selectors

How to code complex structures using brackets `()`

Brackets () can be used to separate different parts of the structure.

By using the Emmet shortcut (main>div)+div>ul>li*5, you can create the following complex structure:

<main>
    <div>div>
main>
<div>
    <ul>
        <li>li>
        <li>li>
        <li>li>
        <li>li>
        <li>li>
    ul>
div>

Creating complex structures

How to use Emmet for setting IDs

If you want to create a p element with the ID of news-section, you can do so using this Emmet: p#news-section.

Emmet for setting IDs

How to use Emmet for setting Classes

If you want to create a p element with the class of news-espanol, you can do so using this Emmet: p.news-espanol.

Emmet for setting Classes

How to create Attributes

It is possible to create elements with certain attributes using Emmet.

The Emmet shortcut button[type=submit] would create this button element: .

Emmet for setting attributes

How to add Content to Elements

It is fairly easy to provide content for your elements using Emmets.

p{This is a paragraph} would create a p element with content "This is a paragraph".

Emmet for setting content

Enumeration

In Emmet, enumeration typically refers to the process of generating multiple HTML elements with sequential numbers or alphabets.

Here is an example of enumeration: ul>li*5{List item number $}.

The $ signifies the point where enumeration starts. The *5 means to repeat this structure five times, with the numbers incrementing for each iteration.

Let's see this in action:

How to generate Lorem Ipsum text

You don't need to Google for "Lorem ipsum" text if you are using Emmet.

Lorem20 would generate a text of 20 characters.

Generate lorem text using Emmet

How to Link CSS

To link a CSS file, type link and press enter.

Link external CSS

How to Link JavaScript

To link JavaScript, use the Emmet shortcut script:src.

Link JavaScript

Emmet Shortcuts for CSS

Just like HTML, Emmet is also available for CSS.

Here are the Emmet shortcuts for some of the commonly used properties:

m10 → margin: 10px;
p-10 → padding: -10px;
m20p → margin: 20%;
w100p → width: 100%;
bgc#ff → background-color: #fff
db → display: block;
dib → display: inline-block;
df → display: flex;
tac → text-align: center;
tar → text-align: right;
c → color: #000;
o → overflow: hidden;
z → z-index: 1;

Wrapping up

If you are into development, I highly recommend learning Emmet.

You will spend less time writing code and get much faster results.

I bet that once you start using it, your development experience will upgrade to a whole new level!

What’s your favorite thing you learned from this tutorial? You can also connect with me on any of these platforms. 📧

See you in the next tutorial, happy coding 😁

How to Use Linux on a Windows Machine – 5 Different Approaches

Zaira Hira — Fri, 09 Jun 2023 15:03:52 +0000

As a developer, you might need to run both Linux and Windows side by side. Luckily, there are a number of ways you can get the best of both worlds without getting different computers for each operating system.

In this article, we'll explore a few ways to use Linux on a Windows machine. Some of them are browser-based or cloud-based that do not need any installation prior to using them.

Here are the methods we'll be discussing:

Dual boot
Windows Subsystem for Linux (WSL)
Virtual Machines (VM)
Browser-based solutions
Cloud-based solutions

Option 1: "Dual-boot" Linux + Windows

With dual boot, you can install Linux alongside Windows on your computer, allowing you to choose which operating system to use at startup.

This requires partitioning your hard drive and installing Linux on a separate partition. With this approach, you can only use one operating system at a time.

If this sounds like the way you want to go, here is a useful tutorial on setting up dual boot on Windows 10.

Option 2: Use Windows Subsystem for Linux (WSL)

Windows Subsystem for Linux provides a compatibility layer that lets you run Linux binary executables natively on Windows.

Using WSL has some advantages:

The setup for WSL is simple and not time consuming.
It is lightweight compared to VMs where you have to allocate resources from the host machine.
You don't need to install any ISO or virtual disc image for Linux machines which tend to be heavy files.
You can use Windows and Linux side by side.

If this sounds like the right option for you, here is a detailed guide on how to install and use WSL.

Option 3: Use a Virtual Machine (VM)

A virtual machine (VM) is a software emulation of a physical computer system. It allows you to run multiple operating systems and applications on a single physical machine simultaneously. Here's a detailed explanation of VMs:

Here are some of the options available for virtualization:

Option 4: Use a Browser-based Solution

Browser based solutions are particularly useful for quick testing, learning, or accessing Linux environments from devices that don't have Linux installed.

You can either use online code editors or web based terminals to access Linux. Note that you usually don't have full administration privileges in these cases.

Online code editors

Online code editors offer editors with built-in Linux terminals. While their primary purpose is coding, you can also utilize the Linux terminal to execute commands and perform tasks.

Replit is an example of an online code editor, where you can write your code and access the Linux shell at the same time.

Replit provides a code editor and a Linux shell.

Web-based Linux terminals

One such example is JSLinux. The screenshot below shows a ready to use Linux environment:

Accessing Linux via JSLinux

Option 5: Use a Cloud-based Solution

Instead of running Linux directly on your Windows machine, you can consider using cloud-based Linux environments or virtual private servers (VPS) to access and work with Linux remotely.

How to Choose the Correct Method

The choice is totally dependent on your use case. But there are some factors that may help you decide which option works the best for you. Let's discuss them:

Access level/elevated privilege: If you require full administrative privileges, it is better to skip the browser-based solutions. WSL, dual booting, VMs, and cloud-based solutions provide you with full administrative control.
Cost: Cloud-based solutions offer services against a subscription fee. This cost varies depending on the choice of operating system, hardware specs of the machine, traffic, and so on. If you are on a tight budget, cloud-based solutions might not be the best.
Scalability: If you are just starting, but plan to do resource exhaustive development in the future, you can always scale up the physical specs of your machine. Some options that support upgrading are cloud-based solutions and VMs. You can add more processors or increase the RAM as per your needs.
Current system's hardware specs: If your current system has lower RAM and storage, running VMs can make the system heavy. It would be better to opt for cloud-based or browser based solutions.
Switching: If you don't plan to use Windows and Linux side by side, dual-boot can be a really good option. It offers a complete and focused Linux experience.

My Setup

I am using Ubuntu VM installed through VMWare workstation player. It is doing a great job as I can frequently switch between two operating systems. It was also simple to setup and I can enjoy the admin privileges as well!

Wrapping Up

I hope you found this article helpful. What’s your favorite thing you learned from this tutorial? I would love to connect with you on any of these platforms. 📧

See you in the next tutorial, happy coding 😁

Bash Scripting Tutorial – Linux Shell Script and Command Line for Beginners

Zaira Hira — Mon, 20 Mar 2023 17:35:58 +0000

In Linux, process automation relies heavily on shell scripting. This involves creating a file containing a series of commands that can be executed together.

In this article, we'll start with the basics of bash scripting which includes variables, commands, inputs/ outputs, and debugging. We'll also see examples of each along the way.

Let's get started. 🚀

Pre-requisites
Introduction
Definition of Bash scripting
Advantages of Bash scripting
Overview of Bash shell and command line interface
How to Get Started with Bash Scripting
How to Run Bash Commands from the Command Line
How to Create and Execute Bash Scripts
Bash Scripting Basics
Comments in bash scripting
Variables and data types in Bash
Input and output in Bash scripts
Basic Bash commands (echo, read, etc.)
Conditional statements (if/else)
Looping and Branching in Bash
While loop
For loop
Case statements
How to Schedule Scripts using cron
How to Debug and Troubleshoot Bash Scripts
Conclusion
Resources for learning more about Bash scripting

Pre-requisites

To follow along with this tutorial, you should have the following accesses:

A running version of Linux with access to the command line.

If you do not have Linux installed or you are just starting out, you can easily access the Linux command line through Replit. Replit is a browser-based IDE where you can access the bash shell in a few minutes.

You can also install Linux on top of your Windows system using WSL (Windows Subsystem for Linux). Here is a tutorial for that.

Introduction

Definition of Bash scripting

By saving these commands in a script, you can repeat the same sequence of steps multiple times and execute them by running the script.

Advantages of Bash scripting

Bash scripting is a powerful and versatile tool for automating system administration tasks, managing system resources, and performing other routine tasks in Unix/Linux systems. Some advantages of shell scripting are:

Automation: Shell scripts allow you to automate repetitive tasks and processes, saving time and reducing the risk of errors that can occur with manual execution.
Portability: Shell scripts can be run on various platforms and operating systems, including Unix, Linux, macOS, and even Windows through the use of emulators or virtual machines.
Flexibility: Shell scripts are highly customizable and can be easily modified to suit specific requirements. They can also be combined with other programming languages or utilities to create more powerful scripts.
Accessibility: Shell scripts are easy to write and don't require any special tools or software. They can be edited using any text editor, and most operating systems have a built-in shell interpreter.
Integration: Shell scripts can be integrated with other tools and applications, such as databases, web servers, and cloud services, allowing for more complex automation and system management tasks.
Debugging: Shell scripts are easy to debug, and most shells have built-in debugging and error-reporting tools that can help identify and fix issues quickly.

Overview of Bash shell and command line interface

The terms "shell" and "bash" are used interchangeably. But there is a subtle difference between the two.

A shell or command-line interface looks like this:

The shell accepts commands from the user and displays the output

In the above output, zaira@Zaira is the shell prompt. When a shell is used interactively, it displays a $ when it is waiting for a command from the user.

If the shell is running as root (a user with administrative rights), the prompt is changed to #. The superuser shell prompt looks like this:

[root@host ~]#

You can determine your shell type using the ps command:

ps

Here is the output for me:

Checking the shell type. I'm using bash shell

In summary, while "shell" is a broad term that refers to any program that provides a command-line interface, "Bash" is a specific type of shell that is widely used in Unix/Linux systems.

Note: In this tutorial, we will be using the "bash" shell.

How to Get Started with Bash Scripting

Running Bash commands from the command line

As mentioned earlier, the shell prompt looks something like this:

[username@host ~]$

You can enter any command after the $ sign and see the output on the terminal.

Generally, commands follow this syntax:

command [OPTIONS] arguments

Let's discuss a few basic bash commands and see their outputs. Make sure to follow along :)

date: Displays the current date

zaira@Zaira:~/shell-tutorial$ date
Tue Mar 14 13:08:57 PKT 2023

pwd: Displays the present working directory.

zaira@Zaira:~/shell-tutorial$ pwd
/home/zaira/shell-tutorial

ls: Lists the contents of the current directory.

zaira@Zaira:~/shell-tutorial$ ls
check_plaindrome.sh  count_odd.sh  env  log  temp

echo: Prints a string of text, or value of a variable to the terminal.

zaira@Zaira:~/shell-tutorial$ echo "Hello bash"
Hello bash

You can always refer to a commands manual with the man command.

For example, the manual for ls looks something like this:

You can see options for a command in detail using man

How to Create and Execute Bash scripts

Script naming conventions

By naming convention, bash scripts end with .sh. However, bash scripts can run perfectly fine without the sh extension.

Adding the Shebang

Below is an example of the shebang statement.

#!/bin/bash

You can find your bash shell path (which may vary from the above) using the command:

which bash

Creating our first bash script

Our first script prompts the user to enter a path. In return, its contents will be listed.

Create a file named run_all.sh using the vi command. You can use any editor of your choice.

vi run_all.sh

Add the following commands in your file and save it:

#!/bin/bash
echo "Today is " `date`

echo -e "\nenter the path to directory"
read the_path

echo -e "\n you path has the following files and folders: "
ls $the_path

Let's take a deeper look at the script line by line. I am displaying the same script again, but this time with line numbers.

  1 #!/bin/bash
  2 echo "Today is " `date`
  3
  4 echo -e "\nenter the path to directory"
  5 read the_path
  6
  7 echo -e "\n you path has the following files and folders: "
  8 ls $the_path

Line #1: The shebang (#!/bin/bash) points toward the bash shell path.
Line #2: The echo command is displaying the current date and time on the terminal. Note that the date is in backticks.
Line #4: We want the user to enter a valid path.
Line #5: The read command reads the input and stores it in the variable the_path.
line #8: The ls command takes the variable with the stored path and displays the current files and folders.

Executing the bash script

To make the script executable, assign execution rights to your user using this command:

chmod u+x run_all.sh

Here,

chmod modifies the ownership of a file for the current user :u.
+x adds the execution rights to the current user. This means that the user who is the owner can now run the script.
run_all.sh is the file we wish to run.

You can run the script using any of the mentioned methods:

sh run_all.sh
bash run_all.sh
./run_all.sh

Let's see it running in action 🚀

Bash Scripting Basics

Comments in bash scripting

Comments start with a # in bash scripting. This means that any line that begins with a # is a comment and will be ignored by the interpreter.

Comments are very helpful in documenting the code, and it is a good practice to add them to help others understand the code.

These are examples of comments:

# This is an example comment
# Both of these lines will be ignored by the interpreter

Variables and data types in Bash

Variables let you store data. You can use variables to read, access, and manipulate data throughout your script.

There are no data types in Bash. In Bash, a variable is capable of storing numeric values, individual characters, or strings of characters.

In Bash, you can use and set the variable values in the following ways:

Assign the value directly:

country=Pakistan

Assign the value based on the output obtained from a program or command, using command substitution. Note that $ is required to access an existing variable's value.

same_country=$country

To access the variable value, append $ to the variable name.

zaira@Zaira:~$ country=Pakistan
zaira@Zaira:~$ echo $country
Pakistan
zaira@Zaira:~$ new_country=$country
zaira@Zaira:~$ echo $new_country
Pakistan

Variable naming conventions

In Bash scripting, the following are the variable naming conventions:

Variable names should start with a letter or an underscore (_).
Variable names can contain letters, numbers, and underscores (_).
Variable names are case-sensitive.
Variable names should not contain spaces or special characters.
Use descriptive names that reflect the purpose of the variable.
Avoid using reserved keywords, such as if, then, else, fi, and so on as variable names.

Here are some examples of valid variable names in Bash:

name
count
_var
myVar
MY_VAR

And here are some examples of invalid variable names:

2ndvar (variable name starts with a number)
my var (variable name contains a space)
my-var (variable name contains a hyphen)

Following these naming conventions helps make Bash scripts more readable and easier to maintain.

Input and output in Bash scripts

Gathering input

In this section, we'll discuss some methods to provide input to our scripts.

Reading the user input and storing it in a variable

We can read the user input using the read command.

#!/bin/bash 

echo "What's your name?" 

read entered_name 

echo -e "\nWelcome to bash tutorial" $entered_name

Reading from a file

This code reads each line from a file named input.txt and prints it to the terminal. We'll study while loops later in this article.

while read line
do
  echo $line
done < input.txt

Command line arguments

In a bash script or function, $1 denotes the initial argument passed, $2 denotes the second argument passed, and so forth.

This script takes a name as a command-line argument and prints a personalized greeting.

echo "Hello, $1!"

We have supplied Zaira as our argument to the script.

#!/bin/bash
echo "Hello, $1!"

Output:

Displaying output

Here we'll discuss some methods to receive output from the scripts.

Printing to the terminal:

echo "Hello, World!"

This prints the text "Hello, World!" to the terminal.

Writing to a file:

echo "This is some text." > output.txt

This writes the text "This is some text." to a file named output.txt. Note that the >operator overwrites a file if it already has some content.

Appending to a file:

echo "More text." >> output.txt

This appends the text "More text." to the end of the file output.txt.

Redirecting output:

ls > files.txt

This lists the files in the current directory and writes the output to a file named files.txt. You can redirect output of any command to a file this way.

Basic Bash commands (echo, read, etc.)

Here is a list of some of the most commonly used bash commands:

cd: Change the directory to a different location.
ls: List the contents of the current directory.
mkdir: Create a new directory.
touch: Create a new file.
rm: Remove a file or directory.
cp: Copy a file or directory.
mv: Move or rename a file or directory.
echo: Print text to the terminal.
cat: Concatenate and print the contents of a file.
grep: Search for a pattern in a file.
chmod: Change the permissions of a file or directory.
sudo: Run a command with administrative privileges.
df: Display the amount of disk space available.
history: Show a list of previously executed commands.
ps: Display information about running processes.

Conditional statements (if/else)

Syntax:

if [[ condition ]];
then
    statement
elif [[ condition ]]; then
    statement 
else
    do this by default
fi

We can use logical operators such as AND -a and OR -o to make comparisons that have more significance.

if [ $a -gt 60 -a $b -lt 100 ]

Let's see an example of a Bash script that uses if, if-else, and if-elif-else statements to determine if a user-inputted number is positive, negative, or zero:

#!/bin/bash

echo "Please enter a number: "
read num

if [ $num -gt 0 ]; then
  echo "$num is positive"
elif [ $num -lt 0 ]; then
  echo "$num is negative"
else
  echo "$num is zero"
fi

The script first prompts the user to enter a number. Then, it uses an if statement to check if the number is greater than 0. If it is, the script outputs that the number is positive. If the number is not greater than 0, the script moves on to the next statement, which is an if-elif statement. Here, the script checks if the number is less than 0. If it is, the script outputs that the number is negative. Finally, if the number is neither greater than 0 nor less than 0, the script uses an else statement to output that the number is zero.

Seeing it in action 🚀

Looping and Branching in Bash

While loop

While loops check for a condition and loop until the condition remains true. We need to provide a counter statement that increments the counter to control loop execution.

In the example below, (( i += 1 )) is the counter statement that increments the value of i. The loop will run exactly 10 times.

#!/bin/bash
i=1
while [[ $i -le 10 ]] ; do
   echo "$i"
  (( i += 1 ))
done

For loop

The for loop, just like the while loop, allows you to execute statements a specific number of times. Each loop differs in its syntax and usage.

In the example below, the loop will iterate 5 times.

#!/bin/bash

for i in {1..5}
do
    echo $i
done

Case statements

case expression in
    pattern1)
        # code to execute if expression matches pattern1
        ;;
    pattern2)
        # code to execute if expression matches pattern2
        ;;
    pattern3)
        # code to execute if expression matches pattern3
        ;;
    *)
        # code to execute if none of the above patterns match expression
        ;;
esac

Here, "expression" is the value that we want to compare, and "pattern1", "pattern2", "pattern3", and so on are the patterns that we want to compare it against.

Let's see an example.

fruit="apple"

case $fruit in
    "apple")
        echo "This is a red fruit."
        ;;
    "banana")
        echo "This is a yellow fruit."
        ;;
    "orange")
        echo "This is an orange fruit."
        ;;
    *)
        echo "Unknown fruit."
        ;;
esac

In this example, since the value of "fruit" is "apple", the first pattern matches, and the block of code that echoes "This is a red fruit." is executed. If the value of "fruit" were instead "banana", the second pattern would match and the block of code that echoes "This is a yellow fruit." would execute, and so on. If the value of "fruit" does not match any of the specified patterns, the default case is executed, which echoes "Unknown fruit."

How to Schedule Scripts using cron

Cron is a powerful utility for job scheduling that is available in Unix-like operating systems. By configuring cron, you can set up automated jobs to run on a daily, weekly, monthly, or specific time basis. The automation capabilities provided by cron play a crucial role in Linux system administration.

Below is the syntax to schedule crons:

# Cron job example
* * * * * sh /path/to/script.sh

Here, the *s represent minute(s) hour(s) day(s) month(s) weekday(s), respectively.

Below are some examples of scheduling cron jobs.

Schedule	Description	Example
`0 0`	Run a script at midnight every day	`0 0 /path/to/script.sh`
`/5`	Run a script every 5 minutes	`/5 /path/to/script.sh`
`0 6 1-5`	Run a script at 6 am from Monday to Friday	`0 6 1-5 /path/to/script.sh`
`0 0 1-7`	Run a script on the first 7 days of every month	`0 0 1-7 /path/to/script.sh`
`0 12 1`	Run a script on the first day of every month at noon	`0 12 1 /path/to/script.sh`

Using crontab

The crontab utility is used to add and edit the cron jobs.

crontab -l lists the already scheduled scripts for a particular user.

You can add and edit the cron through crontab -e.

You can read more about corn jobs in my other article here.

How to Debug and Troubleshoot Bash Scripts

Debugging and troubleshooting are essential skills for any Bash scripter. While Bash scripts can be incredibly powerful, they can also be prone to errors and unexpected behavior. In this section, we will discuss some tips and techniques for debugging and troubleshooting Bash scripts.

Set the `set -x` option

One of the most useful techniques for debugging Bash scripts is to set the set -x option at the beginning of the script. This option enables debugging mode, which causes Bash to print each command that it executes to the terminal, preceded by a + sign. This can be incredibly helpful in identifying where errors are occurring in your script.

#!/bin/bash

set -x

# Your script goes here

Check the exit code

When Bash encounters an error, it sets an exit code that indicates the nature of the error. You can check the exit code of the most recent command using the $? variable. A value of 0 indicates success, while any other value indicates an error.

#!/bin/bash

# Your script goes here

if [ $? -ne 0 ]; then
    echo "Error occurred."
fi

Use `echo` statements

Another useful technique for debugging Bash scripts is to insert echo statements throughout your code. This can help you identify where errors are occurring and what values are being passed to variables.

#!/bin/bash

# Your script goes here

echo "Value of variable x is: $x"

# More code goes here

Use the `set -e` option

If you want your script to exit immediately when any command in the script fails, you can use the set -e option. This option will cause Bash to exit with an error if any command in the script fails, making it easier to identify and fix errors in your script.

#!/bin/bash

set -e

# Your script goes here

Troubleshooting crons by verifying logs

We can troubleshoot crons using the log files. Logs are maintained for all the scheduled jobs. You can check and verify in logs if a specific job ran as intended or not.

For Ubuntu/Debian, you can find cronlogs at:

/var/log/syslog

The location varies for other distributions.

A cron job log file can look like this:

2022-03-11 00:00:01 Task started
2022-03-11 00:00:02 Running script /path/to/script.sh
2022-03-11 00:00:03 Script completed successfully
2022-03-11 00:05:01 Task started
2022-03-11 00:05:02 Running script /path/to/script.sh
2022-03-11 00:05:03 Error: unable to connect to database
2022-03-11 00:05:03 Script exited with error code 1
2022-03-11 00:10:01 Task started
2022-03-11 00:10:02 Running script /path/to/script.sh
2022-03-11 00:10:03 Script completed successfully

Conclusion

In this article, we started with how to access the terminal and then ran some basic bash commands. We also studied what a bash shell is. We briefly looked at branching the code using loops and conditionals. Finally, we discussed automating the scripts using cron followed by some troubleshooting techniques.

Resources for learning more about Bash scripting

If you want to dig deeper into the world of bash scripting, I would suggest you have a look at this 6-hour course on Linux at freeCodeCamp.

What’s your favorite thing you learned from this tutorial? You can also connect with me on any of these platforms. 📧�

See you in the next tutorial, happy coding 😁

Banner image credits: Image by Freepik

Setting a Static IP in Ubuntu – Linux IP Address Tutorial

Zaira Hira — Thu, 02 Mar 2023 21:24:59 +0000

In most network configurations, the router DHCP server assigns the IP address dynamically by default. If you want to ensure that your system IP stays the same every time, you can force it to use a static IP.

That's what we will learn in this article. We will explore two ways to set a static IP in Ubuntu.

Static IP addresses find their use in the following situations:

Configuring port forwarding.
Configuring your system as a server such as an FTP server, web server, or a media server.

Pre-requisites:

To follow this tutorial you will need the following:

Ubuntu installation, preferably with a GUI.
sudo rights as we will be modifying system configuration files.

How to Set a Static IP Using the Command Line

In this section, we will explore all the steps in detail needed to configure a static IP.

Step 1: Launch the terminal

You can launch the terminal using the shortcut Ctrl+ Shift+t.

Step 2: Note information about the current network

We will need our current network details such as the current assigned IP, subnet mask, and the network adapter name so that we can apply the necessary changes in the configurations.

Use the command below to find details of the available adapters and the respective IP information.

ip a

The output will look something like this:

For my network, the current adapter is eth0. It could be different for your system

Note the current network adapter name

As my current adapter is eth0, the below details are relevant.

6: eth0:  mtu 1500 qdisc mq state UP group default qlen 1000
    link/ether 00:15:5d:df:c3:ad brd ff:ff:ff:ff:ff:ff
    inet 172.23.199.129/20 brd 172.23.207.255 scope global eth0
       valid_lft forever preferred_lft forever
    inet6 fe80::215:5dff:fedf:c3ad/64 scope link
       valid_lft forever preferred_lft forever

It is worth noting that the current IP 172.23.199.129 is dynamically assigned. It has 20 bits reserved for the netmask. The broadcast address is 172.23.207.255.

Note the subnet

We can find the subnet mask details using the command below:

ifconfig -a

Select the output against your adapter and read it carefully.

IP is 172.23.199.129 and subnet mask is 255.255.240.0

Based on the class and subnet mask, the usable host IP range for my network is: 172.23.192.1 - 172.23.207.254.

Subnetting is a vast topic. For more info on subnetting and your usable IP ranges, check out this article.

Step 3: Make configuration changes

Netplan is the default network management tool for the latest Ubuntu versions. Configuration files for Netplan are written using YAML and end with the extension .yaml.

Note: Be careful about spaces in the configuration file as they are part of the syntax. Without proper indentation, the file won't be read properly.

Go to the netplan directory located at /etc/netplan.

ls into the /etc/netplan directory.

If you do not see any files, you can create one. The name could be anything, but by convention, it should start with a number like 01- and end with .yaml. The number sets the priority if you have more than one configuration file.

I'll create a file named 01-network-manager-all.yaml.

Let's add these lines to the file. We'll build the file step by step.

network:
 version: 2

The top-level node in a Netplan configuration file is a network: mapping that contains version: 2 (means that it is using network definition version 2).

Next, we'll add a renderer, that controls the overall network. The renderer is systemd-networkd by default, but we'll set it to NetworkManager.

Now, our file looks like this:

network:
 version: 2
 renderer: NetworkManager

Next, we'll add ethernets and refer to the network adapter name we looked for earlier in step#2. Other device types supported are modems:, wifis:, or bridges:.

network:
 version: 2
 renderer: NetworkManager
 ethernets:
   eth0:

As we are setting a static IP and we do not want to dynamically assign an IP to this network adapter, we'll set dhcp4 to no.

network:
 version: 2
 renderer: NetworkManager
 ethernets:
   eth0:
     dhcp4: no

Now we'll specify the specific static IP we noted in step #2 depending on our subnet and the usable IP range. It was 172.23.207.254.

Next, we'll specify the gateway, which is the router or network device that assigns the IP addresses. Mine is on 192.168.1.1.

network:
 version: 2
 renderer: NetworkManager
 ethernets:
   eth0:
     dhcp4: no
     addresses: [172.23.207.254/20]
     gateway4: 192.168.1.1

Next, we'll define nameservers. This is where you define a DNS server or a second DNS server. Here the first value is 8.8.8.8 which is Google's primary DNS server and the second value is 8.8.8.4 which is Google's secondary DNS server. These values can vary depending on your requirements.

network:
 version: 2
 renderer: NetworkManager
 ethernets:
   eth0:
     dhcp4: no
     addresses: [172.23.207.254/20]
     gateway4: 192.168.1.1
     nameservers:
         addresses: [8.8.8.8,8.8.8.4]

Step 4: Apply and test the changes

We can test the changes first before permanently applying them using this command:

sudo netplan try

If there are no errors, it will ask if you want to apply these settings.

Now, finally, test the changes with the command ip a and you'll see that the static IP has been applied.

Static IP applied

How to Set a Static IP Using the GUI

It is very easy to set a static IP through the Ubuntu GUI/ Desktop. Here are the steps:

Search for settings.
Click on either Network or Wi-Fi tab, depending on the interface you would like to modify.
To open the interface settings, click on the gear icon next to the interface name.
Select “Manual” in the IPV4 tab and enter your static IP address, Netmask and Gateway.
Click on the Apply button.

Manually setting a static IP using Ubuntu Desktop.

Verify by using the command ip a

Static IP updated via GUI

Conclusion

In this article, we covered two methods to set the static IP in Ubuntu. I hope you found the article useful.

What’s your favorite thing you learned from this tutorial? Let me know on Twitter!

You can read my other posts here.

Grep Command in Linux – Usage, Options, and Syntax Examples

Zaira Hira — Tue, 17 Jan 2023 22:40:13 +0000

Grep is a useful command to search for matching patterns in a file. grep is short for "global regular expression print".

If you are a system admin who needs to scrape through log files or a developer trying to find certain occurrences in the code file, then grep is a powerful command to use.

In this article, we will discuss the grep command's syntax and its usage with some examples.

Syntax of the `grep` Command

The syntax of the grep command is as follows:

grep [OPTION...] PATTERNS [FILE...]

In the above syntax, grep searches for PATTERNS in each FILE. Grep finds each line that matched the provided PATTERN. It is a good practice to close the PATTERN in quotes when grep is used in a shell command.

In this article, we will discuss the following options that can be used with grep:

-i, --ignore-case: Ignores case distinctions in patterns and input data.
-v, --invert-match: Selects the non-matching lines of the provided input pattern.
-n, --line-number: Prefix each line of the matching output with the line number in the input file.
-w: Find the exact matching word from the input file or string.
-c: Count the number of occurrences of the provided pattern.

In the coming examples, we will use the file fruits.txt with the following content:

apples and pears
citrus – oranges, grapefruits, mandarins and limes
stone fruit – nectarines, apricots, peaches and plums
tropical and exotic – bananas and mangoes
berries – strawBERRIES, raspberries, blueberries, kiwifruit and passionfruit
melons – watermelons, rockmelons and honeydew melons
tomatoes and avocados.

How to Find a Matching String with `grep`

If we want to find the string "fruit" in the file fruits.txt, we can do so like this:

grep "fruit" fruits.txt

Here's the output:

How to Ignore Case Distinctions using `-i`

We can command grep to return results while ignoring the case of the matching string. Let's find the word "berries" from our sample file. It should match all occurrences of "berries" regardless of their case.

 grep -i "berries" fruits.txt

Output:

How to Select the Non-Matching Lines using `-v`

We can reverse the results of the grep command to include non-matching results. Let's say, if we want to get all the lines that do not contain the word "berries", the command would look like this:

grep -v "berries" fruits.txt

The output returned all the lines that do not contain "berries".

How to Find the Line Numbers Against Matching Input using `-n`

There are times when we want to get the line numbers against the matching string. For that, we can supply the -n flag to grep like this:

 grep -n "berries" fruits.txt

Output:

From the output, we can see that the word "berries" occurs on line #5 of the file.

How to Find the Exact Matching Word from the Input File or String using `-w`

If you were to match an exact word rather than just the occurrence of a pattern, you can do so by using the -w flag.

If we grep "fruit" without any flags, it would return any occurrence of the word "fruit". This would include occurrences like "grapefruit", "dragonfruit" and so on like this:

But, if we only want "fruit" in our results, we can use the -w flag like this:

 grep -w  "fruit" fruits.txt

Do you see the difference? The latter result returned only one line where the exact word "fruit" was found.

How to Count the Number of Occurrences of the Provided Pattern using `-c`

We can count the number of times the matched string appears in the file. This is how the -c flag works:

grep -c "fruit" fruits.txt

Output:

The word "fruit" appears 3 times in the file.

Tip: The -c flag can be very useful if you have to count the number of times an error message appeared in a log file.

How to Scan Files for Matching Input

Until now we had discussed how to search for matching patterns in a file or input string. We can also use grep to narrow down the files that contain a matching pattern. We can use the following flags to separate files that contain the matching pattern:

-l, --files-with-matches: Prints the file name that contains the provided matching pattern.
-L, --files-without-match: Prints the file name that does not contain the provided matching pattern.

Let's say we have the following files in our folder:

zaira@Zaira:~/grep-tutorial$ ls -lrt
total 12
-rw-r--r-- 1 zaira zaira 327 Jan 16 19:25 fruits.txt
-rw-r--r-- 1 zaira zaira  47 Jan 16 20:31 vegetables.txt
-rw-r--r-- 1 zaira zaira 194 Jan 16 20:33 more-fruits.txt

The files have the following content:

zaira@Zaira:~/grep-tutorial$ cat fruits.txt
apples and pears
citrus – oranges, grapefruits, mandarins and limes
stone fruit – nectarines, apricots, peaches and plums
tropical and exotic – bananas and mangoes
berries – strawBERRIES, raspberries, blueberries, kiwifruit and passionfruit
melons – watermelons, rockmelons and honeydew melons
tomatoes and avocados.

zaira@Zaira:~/grep-tutorial$ cat more-fruits.txt
passionfruit
dragon fruit
kiwis
pears
grapefruits, mandarins and limes
stone fruit – nectarines, apricots, peaches and plums
tropical and exotic – bananas and mangoes
tomatoes and avocados.

zaira@Zaira:~/grep-tutorial$ cat vegetables.txt
# Vegetables only

Cabbage
Onion
Carrot
Potato

Next, we want to see files that contain the pattern "fruit". We can do that like this:

 grep -l "fruit" *

Note that the * searches for all the files in the folder.

Here is the output::

zaira@Zaira:~/grep-tutorial$  grep -l "fruit" *
fruits.txt
more-fruits.txt

Let's say we want to list files that do not contain the word "fruit". We can achieve that using the command below:

 grep -L "fruit" *

Here is the output:

zaira@Zaira:~/grep-tutorial$  grep -L "fruit" *
vegetables.txt

Wrapping Up

If you want to study the grep command in depth, have a look at the man pages. You can get additional information on the command line with grep --help.

I hope you found this tutorial helpful!

What’s your favorite thing you learned from this tutorial? Let me know on Twitter!

You can read my other posts here.

Image credits:

Image by upklyak on Freepik

How to Make a File in Linux from the Command Line – Create a File in the Terminal

Zaira Hira — Thu, 05 Jan 2023 18:58:23 +0000

Managing files from the command line is one of the most common tasks for a Linux user.

Files are created, edited, deleted, and used by many of the background OS processes. Files are also used by regular users to accomplish daily tasks such as taking notes, writing code, or simply duplicating content.

In this article, we will see three methods through which we can create files using the terminal. The three commands that we'll discuss are touch, cat and echo.

Pre-requisites:

You should have access to the Linux terminal to try out the commands mentioned in this tutorial. You can access the terminal in either of the following ways:

Install a Linux distro of your choice on your system.
Use WSL (Windows Subsystem for Linux), if you want to use Windows and Linux side by side. Here is a guide to do that.
Use Replit which is a browser-based IDE. You can create a Bash project and access the terminal right away.

Method #1: How to Create Files Using the `touch` Command

The touch command creates empty files. You can use it to create multiple files as well.

Syntax of the touch command:

 touch FILE_NAME

Examples of the touch command:

Let's create a single empty file using the syntax provided above.

touch access.log

Next, we'll create multiple files by providing the file names separated with spaces after the touch command.

touch mod.log messages.log security.log

The above command will create three separate empty files named mod.log, messages.log, and security.log.

Method #2: How to Create Files Using the `cat` Command

The cat command is most commonly used to view the contents of a file. But you can also use it to create files.

Syntax of the cat command:

cat > filename

This will ask you to enter the text that you can save and exit by pressing ctrl+c.

cat > sample.txt

When I enter the above command, my terminal output looks like this:

zaira@Zaira:~$ cat > sample.txt
This is a sample file created using the "cat" command
^C

Note the ^C sign, which corresponds to Ctrl+c and signals to the terminal to save and exit.

Here are the contents of the created file:

zaira@Zaira:~$ more sample.txt
This is a sample file created using the "cat" command

Method #3: How to Create Files Using the `echo` Command

The echo command is used to add and append text to files. It also creates the file if it doesn't already exist.

Syntax of the echo command:

echo "some text" > sample.txt

echo "some text" >> sample.txt

The difference between > and >> is that > overwrites the file if it exists whereas >> appends to the existing file.

If you would like to follow along with the video tutorial of this article, here is the link:

Conclusion

In this article, we discussed three different methods to create files in the Linux command line. I hope you found this tutorial helpful.

What’s your favorite thing you learned from this tutorial? Let me know on Twitter!

You can read my other posts here.

Bash If Statement – Linux Shell If-Else Syntax Example

Zaira Hira — Mon, 14 Nov 2022 15:37:54 +0000

When coding, you might need to make decisions based on certain conditions. Conditions are expressions that evaluate to a boolean expression (true or false).

Statements that help to execute different code branches based on certain conditions are known as conditional statements.

if...else is one of the most commonly used conditional statements. Like other programming languages, Bash scripting also supports if...else statements. And we will study that in detail in this blog post.

Syntax of `if` Statements

You can use if statements in a variety of ways. The generic structure of if statements is as follows:

Using an if statement only: if...then...fi
Using and if with an else statement: if...then...else...fi statements
Using multiple else statements with if: if..elif..else..fi

How to Use the `if` Statement

When you are using a single if statement, the syntax is as follows:

if [ condition ]
then
    statement
fi

Note that the spaces are part of the syntax and should not be removed.

Let's go through an example where we are comparing two numbers to find if the first number is the smaller one.

#! /bin/sh

a=5
b=30

if [ $a -lt $b ]
then
        echo "a is less than b"
fi

If you run the above snippet, the condition if [ $a -lt $b ] evaluates to True , and the statement inside the if statement executes

Output:

a is less than b

How to Use the `if .. else` Statement

When you are using an if statement and you want to add another condition, the syntax is as follows:

if [ condition ]
then
    statement
else
    do this by default
fi

Let's see an example where we want to find if the first number is greater or smaller than the second one. Here, if [ $a -lt $b ] evaluates to false, which causes the else part of the code to run.

#! /bin/sh

a=99
b=45

if [ $a -lt $b ]
then
        echo "a is less than b"
else
        echo "a is greater than b"
fi

Output:

a is greater than b

How to Use `if..elif..else` Statements

Let's say you want to add further conditions and comparisons to make the code dynamic. In this case, the syntax would look like this:

if [ condition ]
then
    statement
elif [ condition ] 
then
    statement 
else
    do this by default
fi

To create meaningful comparisons, we can use AND -a and OR -o as well.

In this example, we will determine the type of triangle using these conditions:

Scalene: A triangle where every side is different in length.
Isosceles: A triangle where 2 sides are equal.
Equilateral: A triangle where all sides are equal.

read a
read b
read c

if [ $a == $b -a $b == $c -a $a == $c ]
then
echo EQUILATERAL

elif [ $a == $b -o $b == $c -o $a == $c ]
then
echo ISOSCELES
else
echo SCALENE

fi

In the example above, the script would ask the user to enter the three sides of the triangle. Next, it would compare the sides and decide the triangle type.

3
4
5
SCALENE

Conclusion

You can easily branch your code based on conditions like if..else and make the code more dynamic. In this tutorial, you learned the syntax of if...else along with some examples.

I hope you found this tutorial helpful.

What’s your favorite thing you learned from this tutorial? Let me know on Twitter!

You can read my other posts here.

How to Set an Environment Variable in Linux

Zaira Hira — Wed, 26 Oct 2022 18:38:21 +0000

In programming, you use variables to store information like strings and numbers temporarily.

Variables can be used repeatedly throughout the code or by your operating system to provide values. You can edit them, overwrite them, and delete them.

In this tutorial, I'll teach you what environment variables are and how to set them in Linux.

What Are Environment Variables?

Environment variables are the variables specific to a certain environment. For example, each user in an operating system has its own environment. An admin user has a different environment than other users do, for example.

You might declare an environment variable only required by your user (for example a secret token) that doesn't need to be exposed to other users.

Here are some examples of environment variables in Linux:

USER – This points to the currently logged-in user.
HOME – This shows the home directory of the current user.
SHELL – This stores the path of the current user’s shell, such as bash or zsh.
LANG – This variable points to the current language/locales settings.
MAIL – This shows the location of where the current user’s mail is stored.

These environment variables vary based on the current user session.

How to List Environment Variables in Linux

The command used to display all the environment variables defined for a current session is env.

Here is the output for my session:

root@Zaira:~# env
SHELL=/bin/bash
PWD=/root
LOGNAME=root
HOME=/root
LANG=C.UTF-8
LESSOPEN=| /usr/bin/lesspipe %s
USER=root
SHLVL=1
XDG_DATA_DIRS=/usr/local/share:/usr/share:/var/lib/snapd/desktop
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
MAIL=/var/mail/root
_=/usr/bin/env

How to Print Environment Variables in Linux

There are two ways to print the already defined environment variables:

printenv VARIABLE_NAME
echo $varname

Let's print the value of the variable SHELL using both methods. Here's an example of printing using printenv:

root@Zaira:~# printenv SHELL
/bin/bash

And here's an example of using echo:

root@Zaira:~# echo $SHELL
/bin/bash

How to Set Environment Variables in Linux

The basic syntax to define an environment variable is as follows:

export VARIABLE_NAME=value

Let's define an environment variable, list it, and print its value.

Define the variable JAVA_HOME:

root@Zaira:~# export JAVA_HOME=/usr/bin/java

Verify by listing it:

root@Zaira:~# env
SHELL=/bin/bash
JAVA_HOME=/usr/bin/java
PWD=/root
LOGNAME=root
HOME=/root
LANG=C.UTF-8
LESSCLOSE=/usr/bin/lesspipe %s %s
TERM=xterm-256color
global22=yolo
LESSOPEN=| /usr/bin/lesspipe %s
USER=root
SHLVL=1
XDG_DATA_DIRS=/usr/local/share:/usr/share:/var/lib/snapd/desktop
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games:/snap/bin
MAIL=/var/mail/root
_=/usr/bin/env

Print its value:

root@Zaira:~# echo $JAVA_HOME
/usr/bin/java

However, the variables defined using this method are stored for the current session only. They won't be available for the next session.

Let's verify by opening a new session and printing the variable's value.

zaira@Zaira:/etc$ echo $JAVA_HOME

But, we can make the definitions persistent as shown in the next section.

How to Make Environment Variables Persistent in Linux

To make the JAVE_HOME variable persistent, edit the file .bashrc and define its value in it.

The .bashrc is a script file that's executed whenever a user logs in. It is hidden and located in the user's home directory by default.

I have edited my .bashrc file as follows:

vi ~/.bashrc

Add the definition of the environment variable at the end of the .bashrc file

For the changes to take effect, update the .bashrc file using the source command:

source .bashrc

Let's verify by opening a new session.

root@Zaira:~# echo $JAVA_HOME
/usr/bin/java

How to Create a Persistent Global Variable in Linux

Sometimes you might need to define a global environment variable that is accessible by all users.

For that, we need to first declare a variable and make changes in relevant files where environment variables are read from.

Let's go step by step.

I am logged in as the user Zaira. I am creating a global variable GLOBAL_VARIABLE like this:

zaira@Zaira:~$ export GLOBAL_VARIABLE="This is a global variable"

Edit the following files:
/etc/environment – This file is used to set up system-wide environment variables.

Update the /etc/environment file

For the changes to take effect, use the command source /etc/environment.

/etc/profile – Variables set in this file are read whenever a bash shell is logged in. Edit this file and use the export command:

Update the /etc/profile

Time to test!

Now, I'll switch the user to the root user and verify if I can access the variable GLOBAL_VARIABLE.

root@Zaira:~# echo $GLOBAL_VARIABLE
This is a global variable

It worked! I have been able to access the global variable defined by the user Zaira through the root user as well. The same would apply to other users too. So now you also know how to define global environment variables.

Conclusion

In this tutorial, you learned how to create and define environment variables in Linux. You also learned how to make them persistent so that you can use them across multiple sessions.

What’s your favorite thing you learned here? Let me know on Twitter!

You can read my other posts here.

Image by catalyststuff on Freepik.

Rename a File in Linux – Bash Terminal Command

Zaira Hira — Fri, 30 Sep 2022 15:06:08 +0000

Renaming files is a very common operation whether you are using the command line or the GUI.

Compared to the GUI (or Graphical User Interface), the CLI is especially powerful. This is in part because you can rename files in bulk or even schedule the scripts to rename files at a certain point in time.

In this tutorial, you will see how you can rename files in the Linux command line using the built-in mv command.

How to Use the Linux `mv` Command

You can use the built-in Linux command mv to rename files.

The mv command follows this syntax:

mv [options] source_file destination_file

Here are some of the options that can come in handy with the mv command:

-v , --verbose: Explains what is being done.
-i, --interactive: Prompts before renaming the file.

Let's say you want to rename index.html to web_page.html. You use the mv command as follows:

zaira@Zaira:~/rename-files$ mv index.html web_page.html

Let's list the files and see if the file has been renamed:

zaira@Zaira:~/rename-files$ ls
web_page.html

How to Name Files in Bulk Using `mv`

Let's discuss a script where you can rename files in a bulk using a loop and the mv command.

Here we have a list of files with the extension .js.

zaira@Zaira:~/rename-files$ ls -lrt
total 0
-rw-r--r-- 1 zaira zaira 0 Sep 30 00:24 index.js
-rw-r--r-- 1 zaira zaira 0 Sep 30 00:24 config.js
-rw-r--r-- 1 zaira zaira 0 Sep 30 00:24 blog.js

Next, you want to convert them to .html.

You can use the command below to rename all the files in the folder:

for f in *.js; do mv -- "$f" "${f%.js}.html"; done

Let's break down this long string to see what's happening under the hood:

The first part [for f in *.js] tells the for loop to process each “.js” file in the directory.
The next part [do mv -- "$f" "${f%.js}.html] specifies what the processing will do. It is using mv to rename each file. The new file is going to be named with the original file’s name excluding the .js part. A new extension of .html will be appended instead.
The last part [done] simply ends the loop once all the files have been processed.

zaira@Zaira:~/rename-files$ ls -lrt
total 0
-rw-r--r-- 1 zaira zaira 0 Sep 30 00:24 index.html
-rw-r--r-- 1 zaira zaira 0 Sep 30 00:24 config.html
-rw-r--r-- 1 zaira zaira 0 Sep 30 00:24 blog.html

Conclusion

As you can see, renaming files is quite easy using the CLI. It can be really powerful when deployed in a script.

What’s your favorite thing you learned here? Let me know on Twitter!

You can read my other posts here.

Image by storyset on Freepik

Remove a Directory in Linux – How to Remove folders from the Command Line

Zaira Hira — Wed, 07 Sep 2022 18:18:33 +0000

Linux is a powerful OS with various command line utilities to help you achieve your day-to-day tasks.

Removing folders is a very common operation that allows you to either remove a single folder or remove them recursively (folders within folders).

In this tutorial, we will see how we can delete folders from the command line.

But isn't the GUI enough to delete folders?

It is very simple to delete folders from the file explorer. All we need is to either use the delete button on the keyboard or right-click and delete the folder. But you can be more creative with the CLI (Command Line interface).

With the command line, you can create a script to remove files periodically. You can also remove files that match certain criteria such as modification time or size.

Housekeeping scripts also make use of commands to delete unwanted folders to keep disk space at an optimal level.

In short, you can automate your daily tasks using the command line.

Use caution when removing from the CLI. Once you delete a folder using the CLI, it is permanently deleted as there is no Recycle bin. The only way to recover is through a backup.

How to Delete an Empty Folder using `rmdir`

You use the rmdir command to delete an empty directory. It doesn't work on non-empty directories.

Syntax of the rmdir command:

rmdir [OPTION] DIRECTORY_NAME

We can supply the following options:

-v : displays the detailed output for every directory processed.
--ignore-fail-on-non-empty: ignores if the directory is non-empty.
-p, --parents: removes a directory and its ancestors. For example, 'rmdir -p a/b/c would first remove the folder c, then b and then a

How to Delete a Non-Empty Folder using `rm`

We can not use rmdir on non-empty folders. We have another command for that – rm.

Syntax of the rm command:

 rm [OPTION] [FILE]

We can use the following options:

-r, -R, --recursive: removes directories and their contents recursively.
-d, --dir: removes empty directories. So we can also use rm to remove empty directories like the rmdir command.
-v, --verbose: explains what is being done by mentioning the folder details being processed.
-i: prompts before every removal.

So, the command to delete a non-empty folder would be:

rm -r folder-name

How to Force Delete Directories

If you do not have write permissions, you are unable to delete the folder. If you wish to delete it anyways, use the -f flag. The -f flag doesn't display any prompts.

How to Safely Delete Folders in Linux

To prevent accidental deletion, use the -i interactive flag. This way you'll be prompted before deleting a folder and the contents inside it.

If you want to proceed, just answer the prompts by entering y. On the other hand, enter n to skip the file.

In the example below, note how we descend into the folder and check each file one by one.

Interactively deleting a folder named find-test

Conclusion

Removing directories is useful when you need to remove folders after archiving them, when deleting duplicates, when deleting unused folders, and much more.

All these tasks are targeted at creating more disk space. I hope you found this tutorial helpful.

What’s your favorite thing you learned here? Let me know on Twitter!

Read my other posts here.

Zaira Hira - freeCodeCamp.org

How to Get Information About Your Linux System Through the Command Line

Table of Contents

Why It's Important to Understand Your Linux System

System Administration

Troubleshooting

Security Auditing

Performance Optimization

Proactive Maintenance

How to Get Your OS & Kernel Information in Linux

uname -a Command

uname -r and uname -s Commands

cat /etc/os-release Command

hostnamectl Command

How to Get Your CPU Information in Linux

lscpu Command

cat /proc/cpuinfo Command

nproc Command

How to Get Your Memory Information in Linux

free -h Command

vmstat Command

cat /proc/meminfo Command

How to Get Your Disk & Filesystem Information in Linux

tree -d -L 1 Command

lsblk Command

fdisk -l Command

mount Command

du -sh * Command

How to Get Your Hardware Information in Linux

lshw Command

lspci Command

How to Get Your Network Interfaces & Status Information in Linux

ip a Command

ip r Command

How to Get Your Software & Services Information in Linux

Installed packages

Service status

Processes

top and htop Commands

How to Get Your Logs & Dmesg In formation in Linux

How to Get Your Security/User Audit Information in Linux

Visually Appealing Commands

Conclusion

Learn Linux for Beginners: From Basics to Advanced Techniques [Full Book]

Table of Contents

Part 1: Introduction to Linux

1.1. Getting Started with Linux

What is Linux?

Why should you learn about Linux?

What does it mean that Linux is an open-source operating system?

What is a Linux distribution?

How to install and access Linux

Install Linux as the primary OS

How to use Linux on a Windows machine

Online code editors

Web-based Linux terminals:

Part 2: Introduction to Bash Shell and System Commands

2.1. Getting Started with the Bash shell

Introduction to the bash shell

What is a prompt?

2.2. Command Structure

2.3. Bash Commands and Keyboard Shortcuts

2.4. Identifying Yourself: The whoami Command

Part 3: Understanding Your Linux System

3.1. Discovering Your OS and Specs

Print system information using the uname Command

Find details of the CPU architecture using the lscpu Command

Part 4: Managing Files From the Command line

4.1. The Linux File-system Hierarchy

4.2. Navigating the Linux File-system

Absolute path vs relative path

Locating your current directory using the pwd command

Changing directories using the cd command

4.3. Managing Files and Directories

Creating new directories using the mkdir command

Creating new files using the touch command

Removing files and directories using the rm and rmdir command

Copying files using the cp command

Moving and renaming files and folders using the mv command

4.4. Locating Files and Folders Using the find Command

`uname -a` Command

`uname -r` and `uname -s` Commands

`cat /etc/os-release` Command

`hostnamectl` Command

`lscpu` Command

`cat /proc/cpuinfo` Command

`nproc` Command

`free -h` Command

`cat /proc/meminfo` Command

`tree -d -L 1` Command

`lsblk` Command

`fdisk -l` Command

`mount` Command

`du -sh *` Command

`lshw` Command

`lspci` Command

`ip a` Command

`ip r` Command

`top` and `htop` Commands

2.4. Identifying Yourself: The `whoami` Command

Print system information using the `uname` Command

Find details of the CPU architecture using the `lscpu` Command

Locating your current directory using the `pwd` command

Changing directories using the `cd` command

Creating new directories using the `mkdir` command

Creating new files using the `touch` command

Removing files and directories using the `rm` and `rmdir` command

Copying files using the `cp` command

Moving and renaming files and folders using the `mv` command

4.4. Locating Files and Folders Using the `find` Command

Concatenate and display files using the `cat` command

Viewing text files interactively using `less` and `more`

Displaying the last part of files using `tail`

Displaying the beginning of files using `head`

Counting words, lines, and characters using `wc`

Comparing files line by line using `diff`

How to start using `vim`

How to Read Symbolic Permissions or the `rwx` permissions

How to Change File Permissions and Ownership in Linux using `chmod` and `chown`

How to Change Ownership using the `chown` Command

How to change user ownership with `chown`