Every product, whether it’s software or an application, should go through some sort of security testing process. These checks reveal whether the product is safe or vulnerable to threats.

Penetration testing is a common way to test process or products, but it’s typically more specifically designed with a particular product in mind. On the other hand, a full security audit has a broader scope and covers more ground than penetration testing alone.

In this article, you’ll learn what’s involved in a standard security audit and how these processes can help keep your software secure. By the end, you’ll have a better idea of how security audits can help improve an organisation’s overall security posture.

What is a Security Audit?

A security audit is an overall review of an organization’s security controls, policies, standards, and procedures based on a set of predefined expectations.

These predefined expectations are derived from industry standards such as PCI DSS (Payment Card Industry Data Security Standard), which is a mandatory framework for organizations dealing with credit card transactions. Similarly, HIPAA (Health Insurance Portability and Accountability Act) focuses on the privacy and protection of your health information, making it essential for companies who are handling health-related data.

Apart from these standards, polices, and frameworks, security audits also examine infrastructure components and check application backups or recovery mechanisms to make sure data is protected if accidental or malicious data loss occurs.

The outcome of the audit can help inform the company of its current security posture, and also provides details about possible vulnerabilities and compliance threats.

Usually, security audits are conducted by a group of security experts. They plan the audit well in advance to ensure that it interrupts the company’s day-to-day business as little as possible.

There are two types of security audits:

1. Internal Audit

2. External Audit

In this article, we will discuss internal audits. The main difference between the two processes is that internal audits are conducted by people belonging to the organization, while external audits are conducted by a security team outside the organisation.

Elements of an Internal Audit (Process Flow)

A well-structured security audit provides transparency and ensures that there’s a systematic approach to evaluating the effectiveness of in-place security policies and procedures.

The internal audit process typically follows these five steps:

1. Establishing goals and objectives

2. Conducting a risk assessment

3. Completing a control assessment

4. Assessing compliance

5. Communicating to stakeholders

To understand the elements of an internal audit in more detail, let's break down the process by working through a case study. This will show you how a company or team can plan an internal audit using a systematic approach.

Security Audit Case Study

In order to understand this process more clearly, let’s consider an e-Commerce website called “walkGen.com” as an example.

Case Study: There is a eCommerce website named walkGen.com which primarily sells footwear through their website. Customers can order by logging in using their username and password and can pay using a credit/debit card or UPI. To create a better user experience, the website asks for the user’s name, age, gender, and location for personalized information and design.

With this information in mind, let’s conduct a security audit for the website.

Step 1: Establish Goals and Objectives

This first step involves identifying all the critical assets and services required for the website to run. You’ll also need to define a set of industry standard expectations.

For our walkGen.com website, the critical assets we need to consider include customer data, transaction details, and infrastructure details like hosting server, database, and so on.

Keeping these initial data points in mind, here are some possible goals and objectives:

1. Strength access controls: We want to make sure there are strict authentication and authorization policies in place for any users who sign in.

2. Maintain proper frameworks: Since customers can pay using a credit card, we’ll want to ensure that the site adheres to standards like PCI DSS for payment security.

3. Secure Infrastructure: Assets like hosting servers and databases need to be safeguard from cyber threats and theft.

Now, the key expectations are that we’ll maintain compliance with security standards such as GDPR (General Data Protection Regulation) for protecting customer data as well as PCI DSS (Payment Card Industry Data Security Standard) for monetary transactions. We’ll also need to perform regular maintenance and patch updates for servers and databases.

Step 2: Conduct a Risk Assessment

Risk assessment helps us identify and prioritize threats that may possibly affect walkGen’s critical assets. It helps categorize the risks based on their severity and likelihood.

Identifying and categorizing risks will be the ultimate goal in this step, as it will help walkGen implement effective security measures for critical risks compared to low/information risks.

From the assets and services we identified in the previous step, the possible risks could be:

1. Customer data leakage: Exposure of customer data such as names, email ID, customer delivery addresses, payment details, and so on.

2. Man-in-the-Middle attacks: Intercepting website traffic between the logged in users and the website, leading to stolen login credentials, stored payment card details, and so on.

3. Non-compliance with PCI DSS: Failing to meet the standards required for handling credit card transaction securely.

4. Unauthorized transactions or stolen payment details: payment transaction happening on compromised accounts by cybercriminals.

5. Server vulnerabilities: Weaknesses/loopholes in the hosted web server configuration, third-party software, or cloud network infrastructure.

6. Database exploits: Exploiting vulnerabilities present in the database through penetration testing like SQL Injection, and so on.

7. Missing patch updates: Ignoring/Failing to apply security patches for the OS or walkGen’s applications.

Note: If you look closely, some of these risks seem to overlap with others, and some may seem to be defined in the same way. This is called the “chain link of risk”. But deep down they are different from each other.

Once we’ve identified these risks, the next step is to prioritize them (Critical, Medium, or Low) based on various factors such as severity, likelihood of occurring, potential damage that may happen if the threat occurs, and so on.

3. Man-in-the-Middle attacks
4. Non-Compliance with PCI DSS
5. Unauthorized transactions or stolen payment details
7. Database Exploits | | Medium Risk | 6. Server vulnerabilities | | Low Risk | 7. Missing patch updates |

Step 3: Complete a Controls Assessment

This phase ensures that security checkpoints/controls are implemented for the risks we’ve identified while maintaining compliance standards. If any security controls are missing, the audit documents them and provides optimized preventative measures to safeguard the WalkGen website.

In this particular scenario, some control assessments could be:

• Implementing Multi-Factor Authentication (MFA) or 2 Factor Authentication (2FA) using a one time password (OTP) request to a registered mobile number to prevent accidental exposure of customer data.

• Protecting data using encryption standards such as AES (Advanced Encryption Standard)-256 and TLS (Transport Layer Security) 1.3 for secure data transmission, thus helping eliminate possible man-in-the-middle attacks.

• Also, implementing SIEM (Security Information and Event Management) Tools for event logging to prevent Man-in-the-Middle attacks.

• Account compromise often happens through brute-force attacks when the attacker makes multiple login attempts while trying to guess a user’s password. On the walkGen website, a security plugin has been implemented to block multiple login attempts.

• 

• As stated in the "chain link of risk," the risks of non-compliance with PCI DSS and unauthorized transactions or stolen payment details fall into the same category. We can mitigate these risks by enabling proper authorization during payment transactions and, more importantly, disabling the "Remember my card" option by default, which significantly reduces the risk.

• Server vulnerabilities and missing patch updates also fall into the same category of human-based risks. These risks are mitigated by providing periodic updates and reminders to the respective person in charge for walkGen servers. There is a high likelihood that hosting clouds such as Azure, AWS, or Google Cloud Platform (GCP) may have server vulnerabilities, which we can avoid by keeping them updated to their latest versions.

This phase helps us understand the actual security posture in protecting the website from real time attacks.

Step 4: Assess Compliance

Usually, this phase is merely a continuation of the previous phase. But in an audit, equal weight is given to both risk and compliance. This means that we’ll need to conduct a separate compliance review for all the measures taken to mitigate the risks.

Industry regulations and security standards such as GDPR, PCI DSS, Cybersecurity Best Practices, and ISO 27001 (Information Security Management System) are applicable to WalkGen. Assessing whether these standards are maintained and followed is important – otherwise the website may face threats or heavy fines for non-compliance.

These frameworks are general and foundational frameworks that must be adhered to by almost all companies, regardless of their working model. But there are certain frameworks that are more specific to companies like WalkGen.

One specific framework for walkGen would be ISO 22301 (Business Continuity Management System - BCMS), which helps to ensures that walkGen can continue its operations during cyber attacks (if this happens). It also ensures that the company creates disaster recovery and risk mitigation plans to prepare for the worst case scenario.

Note: If a security control is effective in mitigating the risk but is still not compliant with regulations, it’s considered a red flag in security.

At this stage, 95% of the audit is complete. The team members conducting the audit should have a clear understanding of the risks managed, the security working model, the frameworks implemented, and the security protocols that WalkGen adheres to.

Step 5: Communicate to Stakeholders

This phase concludes the audit process for walkGen and provides the audit results to the relevant security teams and board members, such as founders and the CEO. This report helps them understand the findings and determine the next steps for the website, including how much funding should be allocated for the necessary security measures.

The report for security teams will contain a lot of technical nuances, such as the scope of the website, how vulnerabilities are identified (with code snippets), and a detailed walkthrough of each vulnerability. In contrast, the report for non-technical people, including potentially founders and CEOs, will provide high-level information about the audit and security gaps.

Typically, WalkGen's audit report provides insights such as:

• A summary of identified risks and managed threats

• Implemented security frameworks

• Compliance status: Statements such as "WalkGen meets industry standards like GDPR and ISO 22301."

• Recommendations and further steps, includes suggested security enhancements, security budget requirements, and respective action plans for WalkGen.

Conclusion

Companies should conduct security audits regularly. Each time, they should compare the current results with the previous audit’s results to check the analytics and security status of the organisation.

I hope this article gave you a better idea about what’s involved in a security audit and why they’re necessary for companies to complete on a regular basis. It’s important for companie to stay resilient against cyber threats, reduce potential and legal risks, and maintain customer trust in this evolving digital world.

To make sure that these activities are safe and secure, dev teams need to have a robust security testing framework in place. This helps identify vulnerabilities, prevent cyber threats, and maintain the integrity of digital transactions.

In this article, you will learn all about penetration testing – what it is, why each phase of the process is important, and the tools pentesters use to do their jobs.

What is Penetration Testing?

Penetration Testing is a practice used by security professionals to help companies and teams secure their data. A company gives the security pro permission to try to find vulnerabilities in their system. The security pro then reports any potential weak spots they find to the company so they can fix them. This helps these companies prevent potential attacks before hackers can get access to their data.

If a company fails to conduct pentesting, it can lead to serious consequences like policy violations, hefty compliance regulation fines, loss of customer trust, and a decline in the organization's reputation and overall business value.

There are four phases of penetration testing:

1. Reconnaissance

2. Scanning

3. Exploitation

4. Report Submission

Let’s go through each one so you can learn what’s involved in the entire process.

Reconnaissance: The Art of Information Gathering

Reconnaissance involves gathering information about the target system or network. A pentester’s goal here is to collect as much data as possible about the target, helping them understand the target’s architecture, identify potential vulnerabilities, and develop an effective attack strategy.

In reconnaissance, testing can be conducted in various ways, such as browsing social media for information about the target, using information-gathering tools like theHarvester to crawl websites related to the target domain, and more.

At this stage, all available data—whether technical or non-technical—is gathered without filtering for relevance. The goal is to collect as much information as possible, as even seemingly insignificant details can later prove useful in an attack.

Reconnaissance is crucial for a successful penetration test. So it can be a time-consuming process, often taking anywhere from a few hours to several weeks, depending on the complexity of the target.

Types of Reconnaissance

We can categorize reconnaissance into two main types based on the level of interaction with the target system:

First, we have passive reconnaissance. This involves gathering information from publicly available sources without directly interacting with the target system. Since no direct contact is made, it is stealthy and less likely to alert the target.

At this point, a question may arise: If penetration testing is conducted with prior approval from the target domain, why should we conduct passive reconnaissance to minimize direct interaction when we have the freedom to perform active reconnaissance?

Well, a penetration tester must think from an unethical hacker's perspective. Attackers often rely heavily on passive reconnaissance techniques to gather critical information without alerting the target, making it a crucial phase in ethical hacking as well.

This is why penetration testing should include passive reconnaissance. It helps identify potential information leaks, such as a target company's public announcements or employees posting coding-related doubts on platforms like Substack, which could lead to unauthorized system access.

Active Reconnaissance, on the other hand, involves direct interaction with the target system to extract specific information. Common methods include port scanning, banner grabbing, and network sniffing.

This approach provides more accurate and detailed information, but it comes with a higher risk—the tester’s IP address or digital footprint may be logged by the target system.

For the reconnaissance phase, there are numerous tools available on the internet. But a few are considered highly efficient and popular among penetration testers. Some of these include Medusa and theHarvester.

As an example here, we’ll use theHarvester to gather information on a target domain (Zudio.com) and analyze the different types of data retrieved by the tool.

You can see that the tool crawled the Brave search engine and discovered a couple of IP addresses along with additional subdomains of the target domain (Zudio.com). These findings should be properly documented and included in the target’s reconnaissance report.

Scanning: The Art of Detecting Loopholes

The information a pentester gathers during the reconnaissance phase serves as a crucial input for the scanning phase. This data helps them gain deeper insights into the target system, allowing them to pinpoint areas and filter data that require further analysis.

With a wide range of scanning tools available, pentesters utilize various techniques to:

• Identify open ports, as they can serve as potential entry points.

• Monitor network activity to detect vulnerabilities and security gaps.

Phases of Scanning

Scanning typically involves two key steps:

First, we have port scanning, which identifies open and closed ports on the target system. This helps determine which services are running and are potentially exploitable.

System Ports serve as entry points for a computer system to perform various tasks. Ensuring that all unnecessary ports are closed is crucial for security. Leaving optional ports open can create potential entry points for hackers.

You can use tools like Nmap, Netcat, Masscan for this purpose.

For better understanding, let's scan a sample target domain (192.168.13.136) using Nmap and check which service ports are open.

Next, we have vulnerability scanning, which detects weaknesses in software, configurations, and services. It helps pentesters assess the security risks associated with identified ports and services.

Let’s use the same nmap tool to detect the vulnerabilities from the identified open ports. In the scanning results, you can see that port 21 is open and this port is specifically used for File Transfer Protocol.

Here, we run Nmap on the target address (192.168.13.136) to scan FTP port 21 using the ftp-brute script. This allows us to check whether the FTP service is accessible using default usernames and passwords.

During the scan, we were able to extract additional useful information, including details about the FTP server version (vsftpd 2.3.4). This information can be valuable for identifying potential vulnerabilities in this version.

Finally, the tool successfully identified a vulnerability in the server by discovering valid usernames and passwords from the dictionary list included in the tool.

In general, reconnaissance and scanning are often overlooked by security analysts, assuming they are not important. But these phases provide a valuable dataset and a deeper understanding of the target domain. They help in filtering and directing the exploitation process, allowing penetration testers to focus on specific vulnerabilities instead of blindly attempting various exploits.

Skipping these phases leads to inefficiency, wasting time, resources, and effort. So for successful exploitation, it is essential to conduct thorough information gathering and scanning before proceeding further.

Exploitation: The Art of Attack Simulation

The outcome of the scanning phase gives pentesters a clear understanding of potential entry points, commonly referred to as “open doors”, through identified ports and services. These insights help testers determine which vulnerabilities can be exploited to simulate a real-world cyberattack.

Once vulnerabilities are identified, testers deploy various attack techniques to assess their impact. The goal is to demonstrate how a malicious hacker could gain unauthorized access and compromise the target system. Some common attack methods include:

• SQL Injection – Exploiting database vulnerabilities.

• Cross-Site Scripting (XSS) – Injecting malicious scripts into web applications.

• Buffer Overflow – Overwriting memory to execute malicious code.

• Brute Force Attacks – Cracking weak passwords for system access.

For a clearer understanding, let's explore how database vulnerabilities are exploited using SQL Injection attacks.

Let's say there is a username and password field in a login form. Typically, when a user enters their credentials, the system fetches these input values, constructs a SQL query, and sends it to the server for authentication.

SQL Injection works by manipulating this query to bypass authentication. At a basic level, an attacker can input specially crafted values to alter the query logic. For example, consider the following SQL query:

SELECT * FROM PRODUCTS WHERE USERNAME = " OR 1=1 -- " AND PASSWORD = "1234"

Let’s break down this exploit to see what’s going on:

• The OR 1=1 condition always evaluates to true, meaning the query retrieves all records from the database.

• The -- sequence is a comment operator in SQL, which ignores the rest of the query (including password verification).

As a result, the attacker gains access without valid credentials, effectively bypassing authentication.

Report Submission: The Art of Validation

The final phase of penetration testing involves reporting the vulnerabilities identified during the security test cycle. These reports are crucial for guiding the remediation process, ensuring that the company addresses any weaknesses before they can be exploited.

Penetration testing reports typically include detailed information about the attacks conducted, the respective results, and an assessment of the risks involved. Importantly, the language used in these reports is non-technical, as the findings are often shared with different teams across the organization, including:

• Management

• Higher authorities

• Non-technical teams (like HR, legal, and so on)

These reports must be easily understandable and confidential, as they may contain sensitive information about the organization’s vulnerabilities.

The report should include the following key parameters:

• Number of employees involved

• Start date and end date of the assessment

• List of target domains

• List of open ports (if any)

• List of identified vulnerabilities, categorized by risk level (Critical, High, Medium, Low, Informational)

• Preventive measures to mitigate risks

• List of tools used during the assessment

While the structure and content of these reports may vary from organization to organization, the above parameters are mandatory for a comprehensive security assessment.

The goal is to ensure that stakeholders at all levels of the organization can take appropriate action, whether it's patching a vulnerability, revising a policy, or updating a security strategy.

Conclusion

The penetration testing lifecycle is continuous and it’s something your team must perform periodically. You can’t just do it once, address those concerns, and forget about it.

As new vulnerabilities emerge with the release of updated versions of software, applications, and systems, penetration testing remains essential in identifying and addressing these new risks.

A proactive approach to security through continuous penetration testing is crucial for maintaining a safe and secure digital environment for organizations and their users.

We can define a socket as a quick connection which allows the transmission of data between two processes on the same machine or different machines over a network. It is commonly used in client-server interaction, as sockets allow applications to communicate using the built-in mechanisms of the hardware and operating system.

Many of the today’s most used software – including web browsers, file sharing software, and social media instant messaging applications like WhatsApp and others – fundamentally depend on the concept of sockets.

Usually, a socket program is comprised of two main programs called the client and server. Here, the client acts as the requester, where it requests some data. The server acts as the listener and provides the client the requested data as the response.

In Python, creating a client and server program is a simple task, as Python has many inbuilt modules to help with this.

How to Code the Server

First, let's code our server program. To keep it simple, let's assume that the server listens to the host on a particular port. Whatever data it receives, it just prints and send some random ASCII letters as a response.

# server.py
# Importing neccessary inbuilt modules
import socket
import random
import string

# Creating a socket instance
server_object = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM)

# Connecting to the localhost
ip_address = '127.0.0.1'
port = 5555
server_object.bind((ip_address, port))
server_object.listen()

#Once the client connects to the particular port, the server starts to accept the request.
connection_object, _ = server_object.accept()


if connection_object:
    # Connected to client successfully
    print("SERVER CONNECTED TO CLIENT")

    # sending initial message to the client
    connection_object.send(b"type the message")

    # receiving message from the client
    data_receive = connection_object.recv(1024)

    while data_receive != b'stop':
        print("{}: {}".format("CLIENT MESSAGE: ", data_receive.decode('utf-8')))
        server_input = random.choice(string.ascii_letters)
        connection_object.send(server_input.encode('utf-8'))
        data_receive = connection_object.recv(1024)

In the above code, we created a socket instance for the server. You can see that family=socket.AF_INET defines the address family that this socket can accept – only IPv4 addresses. And type=socket.SOCK_STREAM defines that the socket accepts only TCP (Transmission Control Protocol) connections.

For the server socket instance to listen and accept requests, it needs an IP address and a port. So, we have ip_address = '127.0.0.1' and port = 5555. Here, we have localhost as our IP address as the server and client reside in the same machine.

In the next step, the server instance server_object establishes (binds) an address so that clients can use it to find the server. The bind((ip_address,port)) method assigns a local IP address and a port number to this server_object instance explicitly as the server programs listens on the published port port. A port and local IP address neds to be assigned.

It then starts to actively listens on that particular port. When the client connects to that port from the client side, the server instance accepts the client's request for a connection. It then creates a new connection_object and returns to the server instance.

This connection_object contains all the necessary information about the client and server. Now, we use this connection_object to send a message from the server to client. So we print a SERVER CONNECTED TO CLIENT message if the connection_object is created successfully.

Once the connection_object is created, then the instance sends an initial message type the message in bytes to the client and receives the request from the client.

In the while loop, the connection instance connection_object prints the client message. Then as a response it sends random ASCII letters and waits for the client request. This while loop will execute in the server program until the client sends the request message stop.

How to Code the Client

Up to this point, we have seen the server side code. Now, lets code the client side which is pretty simple.

# client.py

#importing socket module
import socket

# creating socket instance
client_object = socket.socket(family=socket.AF_INET, type=socket.SOCK_STREAM)

# target ip address and port
ip_address = '127.0.0.1'
port = 5555

# instance requesting for connection to the specified address and port
client_object.connect((ip_address,port))

# receiving response from server
data_receive = client_object.recv(1024)

# if response is not null
if data_receive:
    # Connection is successful
    print("CLIENT CONNECTED TO SERVER")
    print(data_receive.decode('utf-8'))


    while data_receive:
        # user input
        client_input = input().encode('utf-8')

        # sending request to the server
        client_object.send(client_input)

        # receiving response from the server
        data_receive = client_object.recv(1024)
        if data_receive:
            print("{}: {}".format("SERVER",data_receive.decode('utf-8')))

In the client side code, we have created a similar socket instance client_object, the target ip_address, and port, just like we created in the server side program.

The next step is to use the client_object instance and connect to the respective target address and port using the connect() method.

Once the connection is successful and the connection_object is created on the server side, then the server sends the response type the message which gets stored in the data_receive in the client side.

Since the server has sent the message, we use this message to confirm that the connection is successful. So, we print CLIENT CONNECTED TO SERVER and then print the message sent by the server type the message.

In the while loop, we first give the input in a string using the input() inbuilt function. Then we convert it to bytes using the encode('utf-8') method and store it in client_input (as the data can be sent only in bytes). We then send the client_input to the server using client_object.send(client_input).

We receive the response data from the server after sending the request to the server. The server will accept and give a response to the client until the user types stop as a request to the server.

Note: We have to first execute the server program and then the client program because when client wants to connect to the target, there should be a server listening, up and running.

Here's the execution of of server.py and then client.py:

Left-Side: Server Program, Right-Side: Client Program

As you can see, once the execution started, the server displayed SERVER CONNECTED TO CLIENT so the user can tell it's working, and sent the initial message to the client.

On the client side, when the client received the message, it displayed CLIENT CONNECTED TO SERVER and also printed type the message received from the client.

Since the client is waiting for the input from the user, once the user entered the input, it sends it to the server and the server prints the client message. It then sent the random ASCII letter as the response to the client.

The flow looped until the client sent the stop message as the request to the server. Once the server received the stop request, it terminated from the socket session.

Conclusion

In this tutorial, We understood socket is one of the most fundamental technologies of computer networking and learnt how to set up a socket program in Python using the socket module in client-side and server-side programs.

“It’s hard enough to find an error in your code when you’re looking for it; it’s even harder when you’ve assumed your code is error-free.”
― Steve McConnell

Errors are inevitable in a programmer's life. In fact, while writing programs, errors can be really helpful in identifying the logic bugs and syntax errors in your code.

But, if you can anticipate an error in a particular set of code lines before execution, then you can handle those errors and make the code error free.

Why Error Handling is Important

Handling or taking care of errors that you're aware of helps the code flow and execute smoothly without any interruptions. If errors occur in any lines of code, the error handling takes care of them and then the code resumes execution.

Let's take an example and understand why we need error handling:

a = 12
b = 6
result = a/b
print(result)
print("I have reached the end of the line")

From the above code, what do you expect ?. Well, the result variable prints 2.0 and on the next line, the console prints I have reached the end of the line. That's what we are excepting.

Let's change value of b from b = 6 to b = 0 and run.

1. a = 12
2. b = 0
3. result = a/b
4. print(result)
5. print("I have reached the end of the line")

When this code gets executed, we will get an error as below:

Error message displayed when b is set to 0

The code didn't print the result value and it also didn't print I have reached the end of the line

The above error messages displays division by zero, which means that if we try to divide any number by 0, we will get this error.

The problem is in line 3. Even though the code didn't print the result value, it should have printed I have reached the end of the line. But, it didn't – why ?

Well, because the Python interpreter stopped at line 3 when the a got divided by 0. At this point, it raised an error in the console and exited the code.

One of the naive solutions to solve this problem can be hard coding the values. If the values of a and b are hard-coded, then running the code will solve this error to some extent.

But the other major problem that may arise is when a user wants to give values of a and b at the time of execution.

a = int(input())
b = int(input())
result = a/b
print(result)
print("I have reached the end of the line")

At this time, there's a high probability that the user will give 0 as the input to b. In order to handle this kind of expected error, we will use certain methods of error handling in order to avoid interrupting the execution flow (even though the user might give any invalid input like 0 as input to b).

How to Use the Try and Except Keywords in Python

Any lines of code that are more prone to errors are kept in try block. If any error occurs, then the except block will take care of those errors.

The code structure looks something like this:

try:
   code that may/may not produce errors
except:
   when error arises, then this block of code exceutes.
   Otherwise, this block of code doesn't exceute

Let's come back to the standard example that we have been discussing. We will handle the division by zero problem using try/except blocks.

Let's insert the lines of code that have a high probability of producing an error. In our case, lines 1-4 of our code have high potential to produce an error. So, we put these four lines in the try block:

try:
  a = int(input())
  b = int(input())
  result = a/b
  print(result)
except:
  print("We caught an error")

print("I have reached the end of the line")

Now, when we give b a value of 0, an error occurs. So, the except block executes and the interpreter prints We caught an error and comes out of the except block and resumes printing I have reached the end of the line.

On the other hand, when we give b a non-zero value, then we print the result value. The code comes out of the try block and resumes printing I have reached the end of the line.

In both cases, we are able to execute until the last line of code without any interruptions.

Apart from try and except, it's quite important to understand the else and finally keywords that come along with try and except.

The else block of code comes after the try and except blocks and executes when no error is raised from the try code block. Similarly, the finally code block comes after the else block and executes whether errors occur or not – this block will execute for sure.

Now that you understand how the try, except, else, and finally code blocks work, the order of flow will be:

try:
   code that may/may not produce errors

except:
   when error arises, then this block of code exceutes

else:
   when error doesn't arise, then this block of code exceutes

finally:
   This block will exceute whether error occurs or not.

On applying the same structure to the number division problem, we get this:

try:
  a = int(input())
  b = int(input())
  result = a/b
  print(result)

except:
   print("We caught an error")

else:
   print("Hurray, we don't have any errors")

finally:
   print("I have reached the end of the line")

When b is assigned 0, then we get an error. So, the except block executes and prints We caught an Error and finally the code block executes and prints I have reached the end of the line.

Code execution flow when error occurs

On the other hand, if b gets 6 for example (or any non-zero value), then we divide the a value by 6 and store it in the result variable. The code then prints the result value.

Then, the else block executes and prints Hurray, we don't have any errors and finally the code block executes and prints I have reached the end of the line.

Code execution flow when no error raises

Summary

Now I hope you understand how you can implement error handling in Python in order to catch potential errors with try/except blocks.

You've also learned how to use the else and finally code blocks that are associated with these error handling methods.

Happy Programming...

Some use looping concepts like Insertion Sort, Bubble Sort, and Selection Sort. On the other hand, you can also sort the same list or array using Merge Sort with the help of recursion.

In this article, you will learn how the Merge Sort algorithm works. You will also learn how recursion plays an important role in Merge Sort.

How Recursion Works

As a prerequisite, we will first understand how recursion works, as it's the backbone of the merge sort algorithm.

First, you should know that a recursive function is one that calls itself until it reaches its desired outcome.

Now, unless you set some condition to stop this process, it'll go on forever – or until your JS code throws an error. This is known as a base case, and it will stop the function from calling itself when its condition is met.

Now that you know the basics of recursion, let's get into the weeds a bit and understand how it works under the hood:

In general, recursion works based on a concept called Principal Mathematical Induction (PMI). In simple words, PMI is a technique which is used to prove certain statements from a basic set of proven statements.

Usually, there are three steps in this process.

Let's prove that the formula of n natural number is P(n)= n*(n+1)/2 using PMI:

To prove that P(n) is true

Step 1: The first step is also known as the base case. During this step, you specify the proven statements. Universally, we know that the sum of the first 1 natural number is 1. Let's consider it the left-hand side of the equation (LHS).

When we apply n = 1 in the the formula, we get p(1) = 1*(1+1)/2 => 1 as the right-hand side (RHS).

Applying 1 in formula

This means that LHS = RHS. This step confirms it's a valid base case.

Step 2: This step is known as the Induction Hypothesis. In this step, we simply assume that this formula is true for some integer k where 1 < k < n. So, we substitute k in the formula and we get p(k) = k*(k+1)/2:

Induction Hypothesis step for some Integer value k

Step 3: This step is known as the Induction Step. At this point, we have to prove that this will work for the integer k+1.

When we substitute k + 1 in the formula and consider it as the LHS, then we get the following:

LHS Statement for P(k+1)

For the RHS, we know that the sum of the natural number k+1 integer is equal to the sum of the natural number of k integer and k+1th integer. We can write it like this:

Sum of k+1 integers is equal to (k+1)th integer and sum of k Integers

The above equation can be rewritten like this:

sum of k natural integers is rewritten as p(k) from step-2

At this point, we know the value of p(k) from step-2 (Induction Hypothesis). When we substitute the value of p(k), we get this:

Substitute p(k) from step-2

We can take the common denominator and simplify the equation, and we get:

Simplifying the equation

On comparing the LHS and RHS, we get the same result and we have proved that this formula works for k+1 integers.

Also, when we substitute n in the place of k+1, we get the end result as:

Substituting k+1 with n

We have proven that formula for n natural number is n*(n+1)/2.

When we observe all the steps in PMI, we did a simple assumption at step-2 and and therefore, we proved the statement at step-3.

This is how recursion works: we initialize a base case for a problem at the start of the function and we assume that this function works for small values of the same problem. Then as the final step, we prove that this works for the existing problem.

How the Merge Sort Algorithm Works

The Merge Sort algorithm works by breaking down an unsorted list into two halves. You then call the merge sort on each of these two parts.

Since, we know recursion works based on Principal Mathematical Induction (PMI), we assume that the smaller lists gets sorted on calling merge sort on these 2 smaller lists.

Note: Before calling recursion in smaller lists, it's more important to define a base case for the function as it acts as a stopping point for the recursive call.

Here, the base case for the merge sort will be if the length of the list is 1. In that case (if the length is 1, it means there's only one item in the list), the list is already sorted so we have to just return the list as it is.

For more clarity, let's take an example and implement merge sort over the unsorted list.

my_list = [3,8,2,7,1,4,5]

In the code above, you can see that the variable my_list contains a list which is unsorted.

Now, since the length of my_list is not 1, we can't assume that it is sorted. So, we call merge sort on first half list1 = [3,8,2] and also call merge sort on second half list2 = [7,1,4,5].

We assume that list1 and list2 are sorted as per the induction step. Now, the list looks like list1 = [2,3,8] and list2 = [1,4,5,7]. Now, all we have to do is just merge those two sorted lists into one using the two pointer technique.

In order to combine and sort the two smaller sorted lists, we take 2 pointers, pointing at the start of each list.

Pointers at start index of list and empty list to append the new list while comparing

Intermediate step in comparing pointer value process

We are comparing the value at each place the pointer points, and whichever is smaller we append that value next in the final list. Then we move the pointer place to the next index.

After looping through this process, when we reach the first final index (in one of the two loops we're joining) we stop the loop. Then we append all the values in the other list (if any remain) to the final list.

Using this technique, we can merge and sort the two small pre-sorted lists.

Working Flow of Merge Sort

When we're using recursion, we assume that it works when we call the same function for a smaller problem. This assumption comes from the induction hypothesis step in PMI.

So when we declare the base case for the problem, we similarly assume that the function will return the correct answer for smaller problems. All we have to do is prove that this works for larger problems, too.

In this algorithm, we declared that the base case is that for a list length of 1, the list is sorted. In the induction hypothesis step, we assumed that algorithm would work for half of the list. In the third step, we just merged the sorted list and proved that this will work on a larger list.

Python Code for the Merge Sort Algorithm

def merge_sort(my_list):

    # Base Case
    if len(my_list) <= 1:
        return my_list

    list_1 = my_list[0:len(my_list) // 2]
    list_2 = my_list[len(my_list) // 2:]

       # Induction Step
    ans_1 = merge_sort(list_1)
    ans_2 = merge_sort(list_2)

    # Sorting and merging two sorted list
    sort_list = sort_two_list(ans_1, ans_2)
    return sort_list



# Separate Function to sort and merge 2 sorted lists
def sort_two_list(list_1, list_2):
    final_list = []
    i = 0
    j = 0
    while i < len(list_1) and j < len(list_2):
        if list_1[i] <= list_2[j]:
            final_list.append(list_1[i])
            i += 1
            continue
        final_list.append(list_2[j])
        j += 1

    while i < len(list_1):
        final_list.append(list_1[i])
        i = i + 1

    while j < len(list_2):
        final_list.append(list_2[j])
        j = j + 1

    return final_list


my_list = [3, 8, 2, 7, 1, 4, 5]
ans = merge_sort(my_list)
print(ans)
# prints [1, 2, 3, 4, 5, 7, 8]

As you see in the above code, I have implemented two separate functions for sorting and merging the two sorted lists. This leads to better readability and easier code reviews.

The main drawback in merge sort is it uses more space. This is because, at every recursive call, a new list gets created and recursion is called on that new list. So space complexity for the worst case scenario is O(n) and time complexity is O(n log n).

Conclusion

Merge Sort is pretty quick in sorting a list recursively from a time complexity perspective. This is useful for counting inversions in lists and it's widely used for external sorting applications than other sorting algorithms.

Happy programming...