JSON was built for humans. It’s readable, accessible, and universal for APIs to consume data or return responses. But in the modern era of AI, a downside of JSON has really come to light: it’s quite verbose.

Every brace, every quote, and every repeated key consumes tokens. If you spend time building apps that talk to large language models (LLMs), you’ll likely know that tokens are the currency of LLM interactions. The more tokens, the more costly your AI solution is going to be.

Now, there is a new kid in town called TOON (Token-Oriented Object Notation). It promises to enable LLMs to talk to structured data more efficiently, intelligently, and cost-effectively.

This article is the result of my curiosity in exploring TOON. I wanted to learn why it’s trending, how it works, and how you can use it today in your JavaScript/TypeScript and Python projects. I hope you find this equally exciting as I do.

You can find all the source code used in this article in this GitHub Repository.

Table of Contents

1. What is Toon?

2. Why is TOON Important Now?

3. JSON vs TOON - Learn With Examples

4. How to Use TOON With JavaScript / TypeScript

5. How to Use Toon With Python?

6. Hold On, JSON Might Still Be Better(In Many Cases)

7. The Future of TOON

8. Before We End…

What is Toon?

TOON is a new data serialization format designed with a code objective:

Reduce the number of tokens when exchanging structured data with language models.

While JSON uses verbose syntax with braces, quotes, and commas, TOON relies on a token-efficient tabular style, which is much closer to how LLMs naturally understand structured data.

Let’s make a quick comparison between JSON and TOON:

Here is some JSON with a users array that contains information about two users (two objects):

{
  "users": [
    { "id": 1, "name": "Alice", "role": "admin" },
    { "id": 2, "name": "Bob", "role": "user" }
  ]
}

If you wanted to represent the same data in TOON, it would look like this:

users[2]{id,name,role}:
  1,Alice,admin
  2,Bob,user

Did you notice the differences?

• No quotes, braces, or colons in TOON.

• The users[2]{id,name,role}: declares an array of two objects with the fields id, name, and role.

• The lines below are simply the data rows.

You can see that TOON visibly reduced the token usage by 30-50%, depending on the data shape.

Why is TOON Important Now?

LLMs like GPT, Gemini, and Claude are token-based systems. Each word, symbol, or chunk costs tokens for input and output. So, if you’re preparing an LLM with structured data input/output like this:

{ "products": [ ... 300, "items" ... ] }

You might waste thousands of tokens in quotes, braces, colons, and repeated keys. TOON solves that by focusing on a compact yet structured representation.

Some of the key benefits of TOON are:

• 30-50% fewer tokens for uniform data sets.

• It has less syntactic clutter, which makes it easier for LLMs to reason about.

• It can be nested as we do with JSON.

• Works well with languages like Python, Go, Rust, and JavaScript.

TOON is a great augmentation to JSON, especially for AI projects, LLMs, and data-heavy prompts. It may not replace JSON entirely, but it’s suitable for use cases where JSON is considered heavyweight for data exchange.

JSON vs TOON – Learn With Examples

Now that you have a basic idea of what TOON does and why it’s helpful, let’s look at some of the most used JSON structures and their equivalent representation in TOON.

1. A Simple Object

Here’s how you’d represent an object with JSON:

{ "name": "Alice", "age": 30, "city": "Bengaluru" }

And here’s how it works with TOON:

name: Alice
age: 30
city: Bengaluru

2. Array of Values

With JSON:

{ "colors": ["red", "green", "blue"] }

With TOON:

colors[3]: red,green,blue

3. Array of Objects

With JSON:

{
  "users": [
    { "id": 1, "name": "Alice" },
    { "id": 2, "name": "Bob" }
  ]
}

With TOON:

users[2]{id,name}:
  1,Alice
  2,Bob

Here, users[2]{id,name} represents the schema, and the lines following it contain the actual data rows.

4. Nested Objects

With JSON:

{
  "user": {
    "id": 1,
    "name": "Alice",
    "profile": { "age": 30, "city": "Bengaluru" }
  }
}

With TOON:

user:
  id: 1
  name: Alice
  profile:
    age: 30
    city: Bengaluru

Indentation represents nesting. It’s almost YAML-like, but it’s still structured.

5. Array of Objects With Nested Fields

With JSON:

{
  "teams": [
    {
      "name": "Team Alpha",
      "members": [
        { "id": 1, "name": "Alice" },
        { "id": 2, "name": "Bob" }
      ]
    }
  ]
}

With TOON:

teams[1]:
  - name: Team Alpha
    members[2]{id,name}:
      1,Alice
      2,Bob

This is still perfectly understandable, and much smaller than the JSON format.

Now that you know a bit about TOON syntax, let’s see how to use it with different programming languages.

How to Use TOON With JavaScript / TypeScript

In most cases, TOON is not meant to be handwritten. Most TOON data will be generated automatically by software, or you’ll need to encode existing data (say, JSON data) into the TOON format.

And there’s good news – TOON already has an official NPM package that you can use in your JavaScript/TypeScript project to convert your JSON data to TOON and vice versa.

Install it using the following command:

npm install @toon-format/toon # Or yarn add, pnpm install, etc

The easiest way to create TOON code is by converting JSON to TOON. You can use the encode() method from the above-mentioned NPM package:

import { encode } from "@toon-format/toon";

const data = {
  users: [
    { id: 1, name: "Alice", role: "admin" },
    { id: 2, name: "Bob", role: "user" },
  ],
};

const toonString = encode(data);
console.log(toonString);

Output:

users[2]{id,name,role}:
  1,Alice,admin
  2,Bob,user

To do the reverse (TOON => JSON), you need to use the decode() method:

import { decode } from "@toon-format/toon";

const toonString = `
users[2]{id,name,role}:
  1,Alice,admin
  2,Bob,user
`;

const jsonObject = decode(toonString);
console.log(jsonObject);

Output:

{
  "users": [
    { "id": 1, "name": "Alice", "role": "admin" },
    { "id": 2, "name": "Bob", "role": "user" }
  ]
}

You can check out this sandbox and try out a few encoding and decoding examples.

How to Use Toon With Python

Using TOON in Python projects is as straightforward as it was with JavaScript/TypeScript. There are Python packages that can encode JSON data to TOON and decode it back to JSON. The python-toon package is the most famous one in recent days.

First, open your terminal and install the python-toon package:

pip install python-toon

Note that if you’re in a virtual environment, you’ll need to activate it first:

python -m venv venv
source venv/bin/activate
pip install python-toon

That’s it! Now you’re all set to use the methods to encode and decode your data to and from TOON. First, let’s encode JSON data to TOON using Python:

from toon import encode

# A channel object
channel = {"name": "tapaScript", "age": 2, "type": "education"}
toon_output = encode(channel)
print(toon_output)

Output:

name: tapaScript
age: 2
type: education

Similarly, we can decode TOON back to JSON:

from toon import decode

toon_string = """
name: tapaScript
age: 2
type: education
"""

python_struct = decode(toon_string)
print(python_struct)

Output:

{"name": "tapaScript", "age": 2, "type": "education"}

Hold On, JSON Might Still Be Better (In Many Cases)

Let’s make it clear that TOON is NOT a universal replacement for JSON. In fact, you should still prefer JSON in many cases, such as when:

• Your data is deeply nested.

• Your data is irregular (for example, varying object shapes).

• Your application needs strict schema validations or type enforcement.

• NON-AI use cases where JSON still stands out and does its job perfectly.

A hybrid approach may even work better. Keep JSON for your application’s data exchange format with APIs, but convert to TOON when it comes to sending data to LLMs.

The Future of TOON

TOON, though in its infancy, is still getting a lot of attention from the developer community. Its early traction is making it unavoidable to talk about.

TOON has already been explored for:

• Less token overhead for structured training data to fine-tune LLMs.

• Compact data exchange in Agent frameworks.

• Faster data serialization and deserialization between the MCP and AI workflow engines.

• With Serverless AI APIs, where cost and speed matter a lot.

Just as JSON has been a standard for the Web’s data exchange, TOON may soon be standardized for AI data interchange. So next time you craft a prompt or pass structured data to an AI model, try it in the TOON format. You may notice the model gets faster and cheaper.

Before We End…

That’s all! I hope you found this article insightful.

Let’s connect:

• Subscribe to my YouTube Channel.

• Check out my FREE courses, 40 Days of JavaScript and 15 Days of React Design Patterns.

• Follow on LinkedIn if you don't want to miss the daily dose of up-skilling tips.

• Join my Discord Server, and let’s learn together.

See you soon with my next article. Until then, please take care of yourself and keep learning.

Python's built-in json module provides a straightforward interface for working with JSON data. You'll use it to convert JSON strings into Python dictionaries and lists that you can manipulate with familiar syntax, and then convert your Python data structures back into JSON when you need to send data to an API or save it to a file.

Beyond basic parsing, you'll often need to handle nested structures, validate data integrity, manage, and transform data formats. This guide covers practical JSON parsing techniques you can use in your projects right away. Let’s get started!

🔗 You can find the code examples on GitHub.

Prerequisites

To follow along with this tutorial, you should have:

• Python 3.7 or later installed on your system

• Basic understanding of Python dictionaries and lists

• Familiarity with Python file operations (opening and reading files)

• A text editor or IDE for writing Python code

Table of Contents

1. Understanding JSON Structure and Basic Parsing

2. How to Work with Nested JSON Objects

3. How to Parse JSON Arrays

4. How to Read JSON from Files

5. How to Handle JSON Parsing Errors

Understanding JSON Structure and Basic Parsing

JSON represents data using a simple syntax with six data types: objects (key-value pairs), arrays, strings, numbers, Booleans, and null.

When Python parses JSON, these types map directly to Python equivalents:

• JSON objects become dictionaries,

• arrays become lists,

• strings remain strings,

• numbers become int or float,

• true and false become True and False, and

• null becomes None.

This direct mapping makes working with JSON in Python intuitive once you understand the correspondence.

Before you start, import the json module that’s built into the Python standard library.

The basic operation in JSON parsing is converting a JSON string into a Python data structure you can work with. Here's how to perform this basic conversion:

import json

json_string = '{"name": "Sarah Chen", "age": 28, "city": "Portland"}'
person = json.loads(json_string)

print(person["name"]) 
print(person["age"])   
print(type(person))

Output:

Sarah Chen
28
<class 'dict'>

Here, the json.loads() function takes a string containing JSON and returns a Python object. The 's' in 'loads' stands for 'string', indicating it works with string data. After parsing, you have a regular Python dictionary that you can access with bracket notation using the JSON keys.

How to Work with Nested JSON Objects

Real-world JSON data rarely comes in flat structures. APIs typically return deeply nested objects containing multiple levels of data. Understanding how to navigate these structures is essential for extracting the information you need.

Consider this example of parsing a weather API response that contains nested objects for location data and current conditions:

import json

weather_data = '''
{
    "location": {
        "city": "Seattle",
        "state": "WA",
        "coordinates": {
            "latitude": 47.6062,
            "longitude": -122.3321
        }
    },
    "current": {
        "temperature_f": 58,
        "conditions": "Partly Cloudy",
        "humidity": 72,
        "wind": {
            "speed_mph": 8,
            "direction": "NW"
        }
    }
}
'''

weather = json.loads(weather_data)

After parsing the JSON string with json.loads(), you can access nested values by chaining dictionary keys together:

city = weather["location"]["city"]
temp = weather["current"]["temperature_f"]
wind_speed = weather["current"]["wind"]["speed_mph"]

print(f"{city}: {temp}°F, Wind {wind_speed} mph")

Output:

Seattle: 58°F, Wind 8 mph

In this example, each level of nesting requires another set of brackets. The expression weather["location"]["city"] first accesses the "location" object, then retrieves the "city" value from within it. You can drill down as many levels as needed, like weather["current"]["wind"]["speed_mph"] which traverses three levels deep. This chaining syntax mirrors how you would access the data in the original JSON structure.

How to Parse JSON Arrays

JSON arrays represent ordered lists of values and appear frequently in API responses when returning collections of items. Python converts JSON arrays into lists, which you can iterate through or access by index.

Here's an example parsing a list of products from an inventory system:

import json

products_json = '''
[
    {
        "id": "PROD-001",
        "name": "Wireless Mouse",
        "price": 24.99,
        "in_stock": true
    },
    {
        "id": "PROD-002",
        "name": "Mechanical Keyboard",
        "price": 89.99,
        "in_stock": false
    },
    {
        "id": "PROD-003",
        "name": "USB-C Hub",
        "price": 34.99,
        "in_stock": true
    }
]
'''

products = json.loads(products_json)

The JSON string starts with a square bracket, indicating an array at the root level. After parsing, products is a Python list containing three dictionaries.

You can now use standard Python list operations on the parsed data. The len() function returns the number of items, and you can iterate through the list with a for loop. Each iteration gives you a dictionary representing one product, which you access using dictionary syntax.

print(f"Total products: {len(products)}")

for product in products:
    status = "Available" if product["in_stock"] else "Out of stock"
    print(f"{product['name']}: ${product['price']} - {status}")

Output:

Total products: 3
Wireless Mouse: $24.99 - Available
Mechanical Keyboard: $89.99 - Out of stock
USB-C Hub: $34.99 - Available

You can also access specific array elements by index and filter the data. List indexing works exactly as it does with any Python list, starting at zero.

first_product = products[0]
print(f"First product ID: {first_product['id']}")

Output:

First product ID: PROD-001

You can also use list comprehensions to filter the parsed data, creating a new list containing only products where the "in_stock" value is True.

available_products = [p for p in products if p["in_stock"]]
print(f"Available: {len(available_products)} products")

Output:

Available: 2 products

How to Read JSON from Files

Most applications read JSON from files rather than hardcoded strings. Configuration files, data exports, and cached API responses typically live in JSON files that your application needs to load at runtime.

The json module comes with the load function for reading files that handles opening and parsing in one step.

This code creates a sample configuration file to demonstrate file reading:

import json

# First, let's create a sample config 
config_data = {
    "api_url": "https://api.example.com/v2",
    "timeout": 30,
    "retry_attempts": 3,
    "enable_logging": True
}

with open('config.json', 'w') as f:
    json.dump(config_data, f, indent=2)

The json.dump() function writes Python data to a file, and the indent=2 parameter formats the JSON with 2-space indentation to make it human-readable. The 'w' mode opens the file for writing, creating it if it doesn't exist or overwriting it if it does.

Now you can read that file back into your application. The json.load() function (without the 's') reads from a file object and parses the JSON in one operation.

with open('config.json', 'r') as f:
    config = json.load(f)

print(f"API URL: {config['api_url']}")
print(f"Timeout: {config['timeout']} seconds")
print(f"Logging: {'Enabled' if config['enable_logging'] else 'Disabled'}")

Note the difference: json.loads() parses strings, while json.load() reads from files.

The with statement ensures that the file closes properly even if an error occurs during reading. After the with block completes, you have a Python dictionary containing all the parsed configuration data.

API URL: https://api.example.com/v2
Timeout: 30 seconds
Logging: Enabled

How to Handle JSON Parsing Errors

JSON parsing can fail for many reasons: malformed syntax, unexpected data types, corrupted files, or network issues when fetching from APIs. Your code must handle these errors gracefully rather than crashing.

The json module raises a JSONDecodeError when it runs into invalid JSON. Here's how to catch and handle these errors appropriately.

The try-except block catches any JSON parsing errors:

• The JSONDecodeError exception provides detailed information about what went wrong: e.msg describes the error, e.lineno indicates which line contains the problem, and e.colno shows the character position. This information helps you debug malformed JSON quickly.

• The function returns None when parsing fails, allowing calling code to check for this and handle the error appropriately.

Let's test this with a few JSON examples:

# Missing closing quote
bad_json1 = '{"name": "Sarah, "age": 28}'
result1 = parse_json_safely(bad_json1)
print(f"Result 1: {result1}\n")

# Missing closing brace
bad_json2 = '{"name": "Sarah", "age": 28'
result2 = parse_json_safely(bad_json2)
print(f"Result 2: {result2}\n")

# Extra comma
bad_json3 = '{"name": "Sarah", "age": 28,}'
result3 = parse_json_safely(bad_json3)
print(f"Result 3: {result3}\n")

# Valid JSON for comparison
good_json = '{"name": "Sarah", "age": 28}'
result4 = parse_json_safely(good_json)
print(f"Result 4: {result4}")

Each malformed JSON string triggers a different error message indicating the specific syntax problem. The error messages help pinpoint exactly where the JSON is invalid. The final example shows that valid JSON parses successfully and returns a dictionary instead of None.

JSON parsing failed: Expecting ',' delimiter
Error at line 1, column 19
Result 1: None

JSON parsing failed: Expecting ',' delimiter
Error at line 1, column 28
Result 2: None

JSON parsing failed: Expecting property name enclosed in double quotes
Error at line 1, column 29
Result 3: None

Result 4: {'name': 'Sarah', 'age': 28}

When reading JSON files, you should also handle file-related errors. The following function load_json_file_safely handles three types of errors:

• FileNotFoundError when the file doesn't exist,

• PermissionError when the application can't read the file, and

• JSONDecodeError when the file contains invalid JSON. Each error type gets its own except block with an appropriate message.

The calling code checks if the result is None and falls back to default values, ensuring the application continues running even when the file can't be loaded.

import json

def load_json_file_safely(filepath):
    try:
        with open(filepath, 'r') as f:
            return json.load(f)
    except FileNotFoundError:
        print(f"Error: File '{filepath}' not found")
        return None
    except PermissionError:
        print(f"Error: Permission denied reading '{filepath}'")
        return None
    except json.JSONDecodeError as e:
        print(f"Error: Invalid JSON in '{filepath}'")
        print(f"  {e.msg} at line {e.lineno}")
        return None

data = load_json_file_safely('missing_file.json')
if data is None:
    print("Using default configuration")
    data = {"timeout": 30, "retries": 3}

If you run the above code, you’ll get the following output:

Error: File 'missing_file.json' not found
Using default configuration

And that’s a wrap! Thank you for making it this far if you’ve following along! 🥳

Conclusion

The json module provides everything you need for working with JSON data in Python. Here’s a summary of what we covered:

• The core functions handle the most common operations: json.loads() parses JSON strings into Python objects, and json.load() reads and parses JSON from files.

• JSON parsing automatically converts between JSON and Python data types. This conversion lets you work with parsed JSON using standard Python syntax.

• You can navigate nested JSON by chaining dictionary keys and list indices together. Access nested values like data['section']['subsection']['field'] by following the structure down through each level.

• Always wrap JSON parsing in try-except blocks when working with external data. The JSONDecodeError exception provides specific information about parsing failures including the error location, helping you debug issues quickly. When reading files, also catch FileNotFoundError and PermissionError to handle common file access problems gracefully.

Get comfortable with these fundamentals and you'll be able to handle most JSON parsing tasks you’ll need for your Python projects. Happy coding!

This tutorial demonstrates how to clean messy JSON and export the results into a new file, based on a predefined schema. The JSON file we’ll be cleaning contains a dataset of 200 synthetic customer records.

In this tutorial, we’ll apply two methods for cleaning the input data:

• With pure Python

• With pandas

You can apply either of these in your code. But the pandas method is better for large, complex data sets. Let’s jump right into the process.

Here’s what we’ll cover:

• Prerequisites

• Add and Inspect the JSON File

• Define the Target Schema

• How to Clean JSON Data with Pure Python

• How to Clean JSON Data with Pandas

• How to Validate the Cleaned JSON

• Pandas vs Pure Python for Data Cleaning

Prerequisites

To follow along with this tutorial, you should have a basic understanding of:

• Python dictionaries, lists, and loops

• JSON data structure (keys, values, and nesting)

• How to read and write JSON files with Python’s json module

Add and Inspect the JSON File

Before you begin writing any code, make sure that the .json file you intend to clean is in your project directory. This makes it easy to load in your script using the file name alone.

You can now inspect the data structure by viewing the file locally or loading it in your script, with Python’s built-in json module.

Here’s how (assuming the file name is “old_customers.json”):

This shows you whether the JSON file is structured as a dictionary or a list. It also prints out the entire file in your terminal. Mine is a dictionary that maps to a list of 200 customer entries. You should always open up the raw JSON file in your IDE to get a closer look at its structure and schema.

Define the Target Schema

If someone asks for JSON data to be cleaned, it probably means that the current schema is unsuitable for its intended purpose. At this point, you want to be clear on what the final JSON export should look like.

JSON schema is essentially a blueprint that describes:

• required fields

• field names

• data type for each field

• standardized formats (for example, lowercase emails, trimmed whitespace, etc.)

Here’s what the old schema versus the target schema looks like:

As you can see, the goal is to delete the ”customer_id” and ”address” fields in each entry and rename the rest from:

• ”name” to ”full_name”

• ”email” to ”email_address”

• ”phone” to ”mobile”

• ”membership_level” to ”tier”

The output should contain 4 response fields instead of 6, all renamed to fit the project requirements.

How to Clean JSON Data with Pure Python

Let’s explore using Python’s built-in json module to align the raw data with the predefined schema.

Step 1: Import json and time modules

Importing json is necessary because we’re working with JSON files. But we’ll use the time module to track how long the data cleaning process takes.

import json
import time

Step 2: Load the file with json.load()

start_time = time.time()
with open('old_customers.json') as file:
    crm_data = json.load(file)

Step 3: Write a function to loop through and clean each customer entry in the dictionary

def clean_data(records):
    transformed_records = []
    for customer in records["customers"]:
        transformed_records.append({
                "full_name": customer["name"],
                "email_address": customer["email"],
                "mobile": customer["phone"],
                "tier": customer["membership_level"],

                })
    return {"customers": transformed_records}

new_data = clean_data(crm_data)

clean_data() takes in the original data (temporarily) stored in the records variable, transforming it to match our target schema.

Since the JSON file we loaded is a dictionary containing a ”customers” key, which maps to a list of customer entries, we access this key and loop through each entry in the list.

In the for loop, we rename the relevant fields and store the cleaned entries in a new list called ”transformed_records”.

Then, we return the dictionary, with the ”customers” key intact.

Step 4: Save the output in a .json file

Decide on a name for your cleaned JSON data and assign that to an output_file variable, like so:

output_file = "transformed_data.json"
with open(output_file, "w") as f:
    json.dump(new_data, f, indent=4)

You can also add a print() statement below this block to confirm that the file has been saved in your project directory.

Step 5: Time the data cleaning process

At the beginning of this process, we imported the time module to measure how long it takes to clean up JSON data using pure Python. To track the runtime, we stored the current time in a start_time variable before the cleaning function, and we’ll now include an end_time variable at the end of the script.

The difference between the end_time and start_time values gives you the total runtime in seconds.

end_time = time.time()
elapsed_time = end_time - start_time

print(f"Transformed data saved to {output_file}")
print(f"Processing data took {elapsed_time:.2f} seconds")

Here’s how long the data cleaning process took with the pure Python approach:

How to Clean JSON Data with Pandas

Now we’re going to try achieving the same results as above, using Python and a third-party library called pandas. Pandas is an open-source library used for data manipulation and analysis in Python.

To get started, you need to have the Pandas library installed in your directory. In your terminal, run:

pip install pandas

Then follow these steps:

Step 1: Import the relevant libraries

import json
import time
import pandas as pd

Step 2: Load file and extract customer entries

Unlike the pure Python method, where we simply indexed the key name ”customers” to access the list of customer data, working with pandas requires a slightly different approach.

We must extract the list before loading it into a DataFrame because pandas expects structured data. Extracting the list of customer dictionaries upfront ensures that we isolate and clean the relevant records alone, preventing errors caused by nested or unrelated JSON data.

start_time = time.time()
with open('old_customers.json', 'r') as f:
    crm_data = json.load(f)

#Extract the list of customer entries
clients = crm_data.get("customers", [])

Step 3: Load customer entries into a DataFrame

Once you’ve got a clean list of customer dictionaries, load the list into a DataFrame and assign said list to a variable, like so:

#Load into a dataframe
df = pd.DataFrame(clients)

This creates a tabular or spreadsheet-like structure, where each row represents a customer. Loading the list into a DataFrame also allows you to access pandas’ powerful data cleaning methods like:

• drop_duplicate(): removes duplicate rows or entries from a DataFrame

• dropna(): drops rows with any missing or null data

• fillna(value): replaces all missing or null data with a specified value

• drop(columns): drops unused columns explicitly

Step 4: Write a custom function to rename relevant fields

At this point, we need a function that takes in a single customer entry – a row – and returns a cleaned version that fits the target schema (“full_name”, “email_address”, “mobile” and “tier”).

The function should also handle missing data by setting default values like ”Unknown” or ”N/A” when a field is absent.

P.S: At first, I used drop(columns) to explicitly remove the “address” and “customer_id” fields. But it’s not needed in this case, as the transform_fields() function only selects and renames the required fields. Any extra columns are automatically excluded from the cleaned data.

Step 5: Apply schema transformation to all rows

We’ll use pandas' apply() method to apply our custom function to each row in the DataFrame. This will creates a Series (for example, 0 → {...}, 1 → {...}, 2 → {...}), which is not JSON-friendly.

As json.dump() expects a list, not a Pandas Series, we’ll apply tolist(), converting the Series to a list of dictionaries.

#Apply schema transformation to all rows
transformed_df = df.apply(transform_fields, axis=1)

#Convert series to list of dicts
transformed_data = transformed_df.tolist()

Another way to approach this is with list comprehension. Instead of using apply() at all, you can write:

transformed_data = [transform_fields(row) for row in df.to_dict(orient="records")]

orient=”records” is an argument for df.to_dict that tells pandas to convert the DataFrame to a list of dictionaries, where each dictionary represents a single customer record (that is, one row).

Then the for loop iterates through every customer record on the list, calling the custom function on each row. Finally, the list comprehension ([...]) collects the cleaned rows into a new list.

Step 6: Save the output in a .json file

#Save the cleaned data
output_data = {"customers": transformed_data}
output_file = "applypandas_customer.json"
with open(output_file, "w") as f:
    json.dump(output_data, f, indent=4)

I recommend picking a different file name for your pandas output. You can inspect both files side by side to see if this output matches the result you got from cleaning with pure Python.

Step 7: Track runtime

Once again, check for the difference between start time and end time to determine the program’s execution time.

end_time = time.time()
elapsed_time = end_time - start_time

#print(f"Transformed data saved to {output_file}")
print(f"Transformed data saved to {output_file}")
print(f"Processing data took {elapsed_time:.2f} seconds")

When I used list comprehension to apply the custom function, my script’s runtime was 0.03 seconds, but with pandas’ apply() function, the total runtime dropped to 0.01 seconds.

Final output preview:

If you followed this tutorial closely, your JSON output should look like this – whether you used the pandas method or the pure Python approach:

How to Validate the Cleaned JSON

Validating your output ensures that the cleaned data follows the expected structure before being used or shared. This step helps to catch formatting errors, missing fields, and wrong data types early.

Below are the steps for validating your cleaned JSON file:

Step 1: Install and import jsonschema

jsonschema is a third-party validation library for Python. It helps you define the expected structure of your JSON data and automatically check if your output matches that structure.

In your terminal, run:

pip install jsonschema

Import the required libraries:

import json
from jsonschema import validate, ValidationError

validate() checks whether your JSON data matches the rules defined in your schema. If the data is valid, nothing happens. But if there’s an error – like a missing field or wrong data type – it raises a ValidationError.

Step 2: Define a schema

As you know, JSON schema changes with each file structure. If your JSON data differs from what we’ve been working with so far, learn how to create a schema here. Otherwise, the schema below defines the structure we expect for our cleaned JSON:

schema = {
    "type": "object",
    "properties": {
        "customers": {
            "type": "array",
            "items": {
                "type": "object",
                "properties": {
                    "full_name": {"type": "string"},
                    "email_address": {"type": "string"},
                    "mobile": {"type": "string"},
                    "tier": {"type": "string"}
                },
                "required": ["full_name", "email_address", "mobile", "tier"]
            }
        }
    },
    "required": ["customers"]
}

• The data is an object that must contain a key: "customers".

• "customers" must be an array (a list), with each object representing one customer entry.

• Each customer entry must have four fields–all strings:

  • "full_name"

  • "email_address"

  • "mobile"

  • "tier"

• The "required" fields ensure that none of the relevant fields are missing in any customer record.

Step 3: Load the cleaned JSON file

with open("transformed_data.json") as f:
    data = json.load(f)

Step 4: Validate the data

For this step, we’ll use a try. . . except block to end the process safely, and display a helpful message if the code raises a ValidationError.

try:
    validate(instance=data, schema=schema)
    print("JSON is valid.")
except ValidationError as e:
    print("JSON is invalid:", e.message)

Pandas vs Pure Python for Data Cleaning

From this tutorial, you can probably tell that using pure Python to clean and restructure JSON is the more straightforward approach. It is fast and ideal for handling small datasets or simple transformations.

But as data grows and becomes more complex, you might need advanced data cleaning methods that Python alone does not provide. In such cases, pandas becomes the better choice. It handles large, complex datasets effectively, providing built-in functions for handling missing data and removing duplicates.

You can study the Pandas cheatsheet to learn more data manipulation methods.

Then, in 2015, a promise-based API request, the Fetch API, was built into JavaScript ES6 to ease the process. After that, libraries like Axios and Alova also appeared.

But why would anyone consider using a third-party API when the lightweight inbuilt Fetch API is an effective option? Well, Axios and Alova provide more than just fetching simple JSON responses. While Axios automates the parsing of JSON and provides shorthand methods for requests, Alova caches responses which prevents making new requests that are redundant.

So which should you stick to – Fetch API, Axios, or Alova?

In this guide, we’ll examine each of these tools based on their features, performance, and project suitability. Walk with me…

Table of Contents

1. Prerequisites

2. The Fetch API

   • Key Features of the Fetch API

   • Limitations of the Fetch API

3. Axios

   • Key Features of Axios

   • Limitations of Axios

4. Alova

   • Key Features of Alova

   • Limitations of Alova

5. Feature-by-Feature Comparison

6. Use Cases and Best Scenarios

   • When to Use Fetch API

   • When to Use Axios

   • When to Use Alova

7. Community and Ecosystem

   • Ecosystem and Integrations

8. Conclusion

Prerequisites

Before you start this tutorial, you should have a basic understanding of JavaScript and ES6+ features, such as async/await, arrow functions, and object destructuring. Being familiar with the fetch() API will also be helpful, as we’ll compare it with Axios and Alova.

You should also have a fundamental knowledge of HTTP methods (GET, POST, PUT, DELETE, PATCH) and handling API responses based on status codes to better understand the API examples.

While this tutorial focuses on JavaScript, some examples use React. So you should be familiar with React and understand the basics of components, state, and hooks (like useState and useEffect). Alova also works with frameworks like Vue and Svelte.

Basic experience with package managers (NPM or Yarn) is useful for installing dependencies like Axios and Alova. And understanding Node.js and browser environments will help, as Alova works in both contexts.

Lastly, familiarity with state management and caching concepts will enhance your understanding of Alova’s features, as it integrates state management and caching directly into API requests.

The Fetch API

Fetch API is a promise-based API request feature in JavaScript that was released to replace the old callback-based XMLHttpRequest (XHP). Unlike the old tool, Fetch API is compatible with modern website features, including service workers and Cross-Origin Resource Sharing (CORS).

With this tool, calling API data is as simple as making a fetch() request on the API URL, as shown below:

fetch("https://fakestoreapi.com/products")

The fetch() returns the server’s promise which is fulfilled with a response object. Then, you pass in some optional arguments to configure the response as JSON or text, attach it to a variable, and use the data.

let products;

  fetch("https://fakestoreapi.com/products")

    .then((res) => res.json())

    .then((data) => {

      products = data

      console.log(products)

    })

    .catch((error) => console.error("Error fetching data:", error))

In the code above, the fetch() requests API data from the URL. The response res gets parsed as JSON res.json. Then, the resulting data is attached to the products variable and logged on the console.

Since Node.js v17.5, the Fetch API has been available natively, eliminating the reliance on external packages like node-fetch, got, or cross-fetch for handling HTTP requests. This native support in both browsers and Node.js removes the need for additional dependencies, reducing the overall bundle size of your application. With this built-in functionality, the Fetch API has become the go-to tool for making asynchronous API calls in JavaScript applications.

Key Features of the Fetch API

Promises-based syntax

As I mentioned earlier, the Fetch API uses a promise-based syntax that sends a promise from the server and executes it with a response object. While the .then chaining can be optimal for simple requests, using several .thens can lead to callback hell and give you a hard time tracking errors. This is why the async/await alternative is a more optimal solution. Check out the code example below:

const fetchData = async () => {

      try {

        const response = await fetch("https://fakestoreapi.com/products");

        if (!response.ok) {

          throw new Error(`HTTP error! Status: ${response.status}`);

        }

        const data = await response.json();

        products = data

        console.log(products); //

      } catch (error) {

        console.error("Error fetching data:", error);

      }

    };

    fetchData();

As shown above, the fetch makes a get request. Then, the server returns an error status if the response is not ok (returns an error status like error 404). Then, the response gets parsed as JSON and used.

Keep in mind that all methods passed on the response are asynchronous, including the fetch() and the json() parsing.

Supports the GET, POST, PUT, PATCH and DELETE methods

GET, used to receive responses, is the Fetch API’s default method. So when you’re using it, you don’t have to define it explicitly or attach a body. But for methods that send requests like POST, PUT, PATCH and DELETE, you must specify their method and attach a body.

All these methods send requests to the backend. You can send data to the server with POST, completely replace an existing resource with new data using PUT, partially update with PATCH, or remove the resource with DELETE.

1. Here’s how you can define a method:

In the code below, I set the POST method to send data to the specified API:

const response = await fetch("https://example.com/products1", {

          method: "POST"

          //...

        });

Apart from posting data, you can also clear data on the server using DELETE:

const response = await fetch("https://example.com/products1", {

      method: "DELETE"

      //...

    });

2. Then, define the header:

Defining the header lets the server understand the type of content you are sending for proper data handling. As shown here, the header asks the server to store the content as a JSON file and set the authorization token to my-classified-token. Keep in mind that the token is the API key that will be used to verify user identity upon use.

const response = await fetch("https://example.com/products1", {

          method: "POST",

          header: {

            "Content-Type": "application-json",

            "Authorization": "Bearer my-classified-token",

          }

          //..

        });

Here is a full list of parameters that can be passed into the header:

3. Next, attach the body:

After specifying the header, you then attach the body. The body is the data being sent to the backend server. It cannot be used with the GET method which only fetches responses. Besides, the information attached should always be in a valid format that matches the content type specified in the headers. You can add as much value as you require to the body.

const response = await fetch("https://example.com/products1", {

          method: "POST",

          header: {

            "Content-Type": "application-json",

            "Authorization": "Bearer my-classified-token",

          },

          body: JSON.stringify({ name: "Laptop", price: 1200 })

        });

Streaming Data

It is also worth noting that the Fetch API facilitates large data handling via streaming. It receives copious data in chunks instead of loading the whole data and buffering in the process. So it data displays real-time as they arrive. Here is a simple example of streaming:

 const fetchData = async () => {

const response = await fetch('https://www.example.com/large-text-file.txt');

      const reader = response.body.getReader();

      const decoder = new TextDecoder();

      while (true) {

        const { done, value } = await reader.read();

        if (done) break;

        const chunk = decoder.decode(value, { stream: true });

        console.log(chunk); // Process the chunk (e.g., display it in UI)

      }

      console.log('Stream complete');

    }

    fetchData();

Fetching Documents with the DOM Parser

Unlike its predecessor, XHP, which can directly return a document, Fetch API can’t achieve the same results without using the DOM Parser. To use it, you have to set the response type to text, then convert to a document using the DOMParser. Here is an example:

fetch("example.xml")

  .then(res => res.text()) // Get raw text

  .then(data => {

    const parser = new DOMParser();

    const doc = parser.parseFromString(data, "text/xml"); // Convert text to Document

    console.log(doc); // Now it's a Document object

  })

  .catch(console.error);

Request Cancellation with AbortController

Previously, the Fetch API couldn’t abort requests. But it is now possible with AbortController and AbortSignal. But the AbortController API is not native either, which means there is extra bundle and set up required.

Limitations of the Fetch API

Response Flexibility or No automatic JSON parsing

Depends on how you see it. Having to specify whether you want your response as JSON res.json() or text res.text() or blob res.blob() lets you set which response type you want from the get go. But it can also be a limitation since most API fetches are in JSON. This means that alternatives like Axios, which sets defaults as res.json(), helps write shorter and cleaner code, and is therefore often preferred by developers.

No built-in request/response interceptors

Unlike Axios, Fetch API does not have built-in methods that intercept and modify requests or responses. This limitation means you have to write boilerplate code to create a custom interceptor.

For instance, via interception, you can attach an Authorization token automatically before sending requests or asking all 401 errors to automatically reload when receiving responses. With the Fetch API, you have to wrap the fetch() in a function to do that, which means more lines of code.

Here is some code built to mimic request/ response interception:

const customFetch = async (url, options = {}) => {

      // Request Interception

      const modifiedOptions = {

          ...options,

          headers: {

              'Content-Type': 'application/json',

              Authorization: Bearer ${localStorage.getItem("token")}, // Interceptor behavior

              ...options.headers

          }

      };



      try {

          const response = await fetch(url, modifiedOptions);



          // Response Interception

          if (!response.ok) {

              console.error("Intercepted Error:", response.status);

          }



          return await response.json();

      } catch (error) {

          console.error("Fetch error intercepted:", error);

          throw error;

      }

  };

  // Usage (No need to set headers manually)

  customFetch('https://api.example.com/data')

      .then(data => console.log(data))

      .catch(error => console.error(error));

Error handling requires additional logic

The Fetch API only rejects network errors, not failed HTTP status codes like 404 or 501. This means that when a fetching request fails, it does not return a 404 Not Found or 500 Internal Server Error unless you configure that with additional code. But Axios does.

fetch('https://jsonplaceholder.typicode.com/invalid-url')

  .then(response => {

    if (!response.ok) { // Manually handle non-2xx responses

      throw new Error(`HTTP Error! Status: ${response.status}`);

    }

    return response.json();

  })

  .then(data => console.log(data))

  .catch(error => console.log('Error:', error.message));

Axios

After XHP was replaced with the Fetch API, Axios emerged in 2016 to address some issues with the new JavaScript-native fetching tool. Built on top of XHP, Axios quickly gained widespread adoption due to combining many Fetch API promise-based features with some methods on the legacy XMLHttpRequest. In no time, it became a popular choice amongst developers.

Axios stands out because it:

• Automates JSON parsing

• Has a built-in method to intercept and modify requests and responses

• Automates error handling

• Automates timeout handling

• Can track upload and download progress

And many more features.

In particular, Axios is widely loved because it reduces boilerplate code. Since most API requests encode data with JSON, Axios sets its default parsing to accordingly, which means you don’t have to define JSON again. And why worry anyway, since developers use far fewer res.text() and res.blob() API responses in comparison.

const fetchData = async () => {

    const response = await axios.get('https://api.example.com/data');

    console.log(response.data); // JSON is already parsed

  };

Now, compare that to a like-for-like fetching with Fetch API:

const fetchData = async () => {

    const response = await fetch('https://api.example.com/data');

    const data = await response.json(); // Extra step

    console.log(data);

  };

Yeah, there’s an extra line, right? That could mean several lines of code for larger codebases.

Key Features of Axios

Automatic JSON Parsing

As explained above, you don’t have to call res.json() again while using Axios, since the method is automatically set. But what happens, in rare cases, when you want to fetch a blob or text using Axios? Then, you have to set the response type accordingly. Here’s how you can do that:

const fetchData = async () => {
    const response = await axios.get('https://api.example.com/data', {
      responseType: 'text', // Treats response as plain text
    });

    console.log(response.data); // Plain text string
  };

Built-in Interceptors to Modify Requests and Responses

Axios comes with its built-in interceptors to intercept and modify API responses or requests. Interceptors can help set authorization tokens for requests or modify global responses and errors before they render. Use the .interceptors.request.use() for requests and .interceptors.response.use() for responses.

import axios from "axios";

  const apiClient = axios.create({

      baseURL: "https://api.example.com",

      headers: {

          'Content-Type': 'application/json'

      }

  });



  // Request Interceptor: Attach Authorization headers

  apiClient.interceptors.request.use(config => {

      config.headers.Authorization = Bearer ${localStorage.getItem("token")};

      return config;

  }, error => Promise.reject(error));

  // Usage: Axios automatically includes the Authorization header

  apiClient.get("/data")

      .then(response => console.log(response.data))

      .catch(error => console.error(error));

To achieve that with Fetch API, you will have to write an interceptor wrapper on your API, which needs far more boilerplate code.

Request cancellation with CancelToken

Although now deprecated, Axios used to have its native request cancellation method known as CancelToken. But now, the AbortController API is regarded as a globally-recognized and reliable method for request abortion.

Error Handling

Axios handles errors better by automatically rejecting all non-2xx status codes like Error 404 and 501. You do not need to check any response.ok message:

axios.get('https://jsonplaceholder.typicode.com/invalid-url')

  .then(response => console.log(response.data))

  .catch(error => {

    console.log('Error Status:', error.response?.status); // Axios auto-rejects non-2xx responses

    console.log('Error Message:', error.message);

  });

Built-in Progress Tracking

Axios incorporates XHP methods like onDownloadProgress and onUploadProgress. This inbuilt feature facilitates tracking download and uploads progress. Whereas with the Fetch API, you’d need ReadableStream to achieve similar results.

Here is an example showing how you can use onUploadProgress:

axios.post(url, data, {

    onUploadProgress: progressEvent => console.log(progressEvent.loaded)

  });

Supports Other Methods, too

Just like the Fetch API, Axios’ default Method is GET. But you can use the POST, PUT, PATCH or DELETE methods using axios.request(). Here’s how:

import axios from "axios";

  axios.request({

      method: "POST",

      url: "https://api.example.com/users",

      body: { name: "Abdullah", age: 25 }, // Request body

      headers: {

          "Authorization": Bearer ${localStorage.getItem("token")},

          "Content-Type": "application/json"

      }

  })

  .then(response => console.log(response.data))

  .catch(error => console.error("Axios Request Error:", error));

Axios also provides a shorthand with methods like axios.get, axios.post, axios.put, axios.patch, and axios.delete, as shown below:

// POST Request

axios.post("https://api.example.com/users",

  { name: "Abdullah", age: 25 }, // Request body

  { headers: { "Content-Type": "application/json" } }

)

.then(response => console.log(response.data))

.catch(error => console.error("Axios POST Error:", error));

// PUT Request

axios.put("https://api.example.com/users/123",

  { name: "Updated Name" }, // Updated data

  { headers: { "Authorization": Bearer ${localStorage.getItem("token")} } }

)

.then(response => console.log(response.data))

.catch(error => console.error("Axios PUT Error:", error));

// DELETE Request

axios.delete("https://api.example.com/users/123", {

  headers: { "Authorization": Bearer ${localStorage.getItem("token")} }

})

.then(response => console.log("User deleted successfully"))

.catch(error => console.error("Axios DELETE Error:", error));

Limitations of Axios

Slightly larger bundle size

Axios adds 35 kb of extra bundle, while FetchAPI adds 0. While Axios clearly offers more features than Fetch in every other metric, you have to make do with the larger bundle size. And in an age where lightweight and fast applications are often preferred, you might not want that load.

Dependency on third-party maintenance

Depending on a third-party option for something as crucial as API might not be desirable. So a native tool like the Fetch API, built within JavaScript, offers more reliability.

Alova

Alova is a request management library that combines simple API fetching with other functionalities like state management, hooks, and caching, amongst many others.

While we use react-query and SWR to process Axios-fetched data, Alova saves you those extra installations and coding by providing these methods natively. The all-in-one alternative not only fetches responses and sends requests, but also merges requests, caches responses, and optimizes them for UI frameworks.

Built in 2022, Alova’s adoption is still early but nonetheless seems promising. It is supported on browsers, Node.js, and most frameworks, including Vue, React, Svelte, and vanilla JavaScript. But it has limited usage for Angular.js.

At just 10kb, it is about 3 times smaller than Axios, making it a more lightweight alternative for building fast applications.

You can also use Alova to either replace react-query to facilitate Axios or be the one-stop-shop for everything API integration-related.

Here is a simple Alova fetch:

const response = await alovaInstance.Get('https://jsonplaceholder.typicode.com').send();

console.log(response); // Response data

When you are fetching Alova on React components, you can use the createAlova() to set parameters and useRequest() to manage state.

import React from "react";

import { createAlova, useRequest } from "alova";

import GlobalFetch from "alova/GlobalFetch";

// Initialize Alova

const alovaInstance = createAlova({

  statesHook: React,

  requestAdapter: GlobalFetch(),

});

// GET request with useRequest

const Profile = () => {

  const { data, loading, error } = useRequest(() => alovaInstance.Get("https://jsonplaceholder.typicode.com"));

  if (loading) return <p>Loading...</p>;

  if (error) return <p>Error fetching profile</p>;

  return <div>Username: {data.username}</div>;

};

export default Profile;

Key Features of Alova

One-Stop Shop

For some functionalities that come built into Alova, Fetch API or Axios might need additional libraries like react-query or SWR to fulfill.

Request Sharing Prevents Redundant Requests

Alova fuses identical requests. Let’s say several components ask for the same data from the API. The Fetch API and Axios send multiple identical requests to the server which creates traffic. But Alova merges them, sends a single request, and shares its response across all components, which reduces network traffic.

State Management

With tools like Fetch API and Axios, you have to manage data, loading, and error states manually. Alova lets you do that on the go within a single line of code. Here is how it looks:

//...

const { data, loading, error } = useRequest(alova.Get("/posts/1"));

//...

Advanced Request Management

Alova offers several request management functionalities with each tailored to specific use cases. With its request management, you can request preload for data to be used later, cache data to prevent reload, manage form submission, handle pagination, and automate refetching when needed. Check out their docs to read more.

Multi-Level Caching

You can also use Alova to cache data, especially when the response isn’t constantly changing and does not need refetching. Unlike react-query that simply stores caches in RAM, Alova offers a more flexible framework.

Its three-pronged caching modes include the memory mode, cache occupying mode, and recovery mode. While memory mode stores data in the RAM, recovery mode persistently stores it in a Local Storage and is made available for longer periods and even offline. Meanwhile, the occupying mode prevents duplicate or redundant requests coming in quick succession.

Independent on any component, cached data can be accessed anywhere in the app if the request URL and parameters match. These features lower traffic going to servers, reduce buffering, and help facilitating a swifter and better user experience.

//...

// Initialize Alova instance

const alovaInstance = createAlova({

  baseURL: "https://jsonplaceholder.typicode.com",

  statesHook: React,

  requestAdapter: GlobalFetch(),

});

// Define GET request

const getPosts = alovaInstance.Get("/posts", {

  cache: {

    mode: "memory", // Caches in memory

    expires: 1000 * 60 * 5, // Expires in 5 minutes

  },

});

const PostList = () => {

  const { data, loading, error } = useRequest(getPosts);

  if (loading) return <p>Loading...</p>;

  if (error) return <p>Error fetching data</p>;

  //...

};

In the example above, Alova caches the response using the memory mode cache: {mode: "memory"} and sets the cache expiration to 5 minutes expires: 1000 * 60 * 5. You can change “memory” to “recovery” if you want a longer storage duration.

Usage Flexibility

You can use Alova with either Axios or the Fetch API. Here is an example where I fetched data using Axios and complemented it with Alova state management.

//...

// Initialize Alova

const alovaInstance = createAlova({

  statesHook: React,

  requestAdapter: GlobalFetch(),

});

// Manage state with Alova

const { data: posts, setData } = useSnapshot([]);

const fetchPosts = async () => {

  const response = await axios.get("https://jsonplaceholder.typicode.com/posts");

  setData(response.data); // Store data in Alova state

};

//...

Supports Other Methods

Alova also makes GET its default option while supporting other methods like POST, PATCH, PUT and DELETE. Here is how to use POST in Alova, for example:

import React, { useState } from "react";

import { createAlova, useRequest } from "alova";

import GlobalFetch from "alova/GlobalFetch";

const alovaInstance = createAlova({

  baseURL: "https://jsonplaceholder.typicode.com",

  statesHook: React,

  requestAdapter: GlobalFetch(),

});

const PostForm = () => {

  const [title, setTitle] = useState("");

  const { send: createPost } = useRequest(alovaInstance.Post("/posts", { title }), { immediate: false });

  createPost().then(console.log)

};

export default PostForm;

Of course, it has shorthand methods too:

import React, { useState } from "react";

import { createAlova, useRequest } from "alova";

import GlobalFetch from "alova/GlobalFetch";

const alova = createAlova({

  baseURL: "https://jsonplaceholder.typicode.com",

  statesHook: React,

  requestAdapter: GlobalFetch(),

});

const { send: createPost } = useRequest(alova.Post("/posts", { title: "New Post" }), { immediate: false });

const { send: updatePost } = useRequest(alova.Put("/posts/1", { title: "Updated Post" }), { immediate: false });

const { send: patchPost } = useRequest(alova.Patch("/posts/1", { title: "Patched Post" }), { immediate: false });

createPost().then(console.log);

updatePost().then(console.log);

patchPost().then(console.log);

Bundle Size

Alova is three times smaller than Axios, but that does not even tell the full story. With Axios and the Fetch API, you need different libraries to handle caching, request deduplication, and retries. But Alova has everything in-built. So using Axios and Fetch API in real code production will always require more bundles than Axios. And overall, Alova facilitates lighter-weight applications compared to Axios and sometimes the Fetch API as well.

Limitations of Alova

Adoption Is Still Low

While writing this article, I had a hard time getting enough resources on Alova. And that is because it only debuted in July 2022 which means adoption is still early. So, troubleshooting Alova might be problematic since there are fewer Alova-themed API communities, as well as fewer StackOverflow answers, Youtube Tutorials, or GitHub contributions.

Potential Stability & Long-Term Maintenance Risks

Newer libraries have a higher risk of abandonment. Axios has been around for years, while Alova is still growing. Besides, it has fewer production use cases and battle-tested applications compared to Axios and Fetch.

Learning Curve

Alova’s learning curve can take some getting used to because it handles API requests differently from tools like Axios or the Fetch API.

Instead of making requests directly, you work with request instances and manage state within Alova’s system. This requires learning new ways to structure API calls and use features like caching and request merging. While it may feel unfamiliar at first, it can help reduce redundant API calls and improve performance once you understand it.

Fewer Third Party Integrations

Alova has fewer third-party libraries built specifically for it, requiring more manual work for compatibility with existing tools.

Feature-by-Feature Comparison

Use Cases and Best Scenarios

Choosing the right HTTP client for your project depends on several factors, including project complexity, dependencies, and performance considerations. Let’s explore when it’s best to use the Fetch API, Axios, or Alova.

When to Use Fetch API

1. Suitable for Lightweight Projects and Simple Requests

The Fetch API is built into modern browsers and is ideal for handling basic HTTP requests without adding dependencies. If your project requires only simple GET, POST, or DELETE requests with minimal configurations, Fetch API is a great choice.

2. When Working in Environments Where Third-Party Libraries Are Restricted

Certain enterprise or security-sensitive applications may restrict the use of external libraries. Since the Fetch API is built into the browser, it remains a viable option when third-party packages like Axios or Alova are not allowed.

3. When Minimal Dependencies Are Preferred

Since Fetch API is native to JavaScript, it does not require installing extra libraries, making it perfect for projects that need to keep dependencies low. This can be particularly beneficial for small lightweight apps or static websites.

When to Use Axios

1. Ideal for Backend-Heavy Applications or Complex APIs

For projects that require multiple API calls, error handling, and efficient request management, Axios is a solid choice. It allows concurrent requests, request cancellation, and improved control over HTTP headers.

2. When Automatic JSON Handling, Interceptors, and Robust Error Handling Are Needed

Axios simplifies working with JSON data by automatically parsing responses. It also provides built-in interceptors for request and response transformations, as well as superior error handling compared to Fetch API.

3. Useful When Working with Node.js in Full-Stack Applications

Axios works both in the browser and in Node.js, making it an excellent choice for full-stack applications where a unified API client is needed across the frontend and backend.

When to Use Alova

1. When Working with Frontend-Heavy Applications (React, Vue, Svelte)

Alova integrates well with frontend frameworks and state management tools, making it a great choice for single-page applications (SPAs) that depend on smooth data fetching, pagination, and updates.

2. Best for Projects Requiring Optimized Caching and Data Synchronization

Alova is designed for performance optimization and better caching strategies. It is suitable for applications that rely on real-time data synchronization and need to minimize redundant network requests.

3. When Performance Optimization and Reduced Network Load Are Priorities

With its intelligent caching mechanisms, Alova can significantly reduce API call frequency, thereby improving the overall performance of the application. It is especially useful in scenarios where network efficiency is crucial, such as mobile applications or progressive web apps (PWAs).

Community and Ecosystem

The community and ecosystem surrounding an HTTP client can impact ease of use, available learning resources, and integration with other tools. Let's explore how the Fetch API, Axios, and Alova are perceived in 2025.

Ecosystem and Integrations

While Fetch API is widely supported, developers often supplement it with additional libraries for improved caching, timeouts, and request queuing. This can lead to increased development effort compared to using an out-of-the-box solution like Axios or Alova.

Meanwhile, Axios benefits from a well-established ecosystem with a variety of plugins and extensions, making it easy to integrate with different backend architectures, authentication systems, and request monitoring tools.

Alova is designed to work seamlessly with modern state management libraries such as React Query and Vue Query. These integrations make it an attractive choice for developers focused on optimizing frontend data fetching strategies.

Conclusion

Choosing between the Fetch API, Axios, and Alova depends on your project’s needs and priorities. The Fetch API is best for lightweight applications that require minimal dependencies, while Axios is a robust choice for full-stack applications and backend-heavy environments. Alova, on the other hand, is an excellent option for optimizing data fetching and caching in frontend-focused applications.

As developers explore new ways to enhance performance and reduce network load, Alova's adoption is expected to grow, particularly in SPAs and PWAs. But Axios remains a reliable and widely adopted solution, while the Fetch API continues to be the fundamental building block for HTTP requests in JavaScript.

In this article, we'll explore the different ways to encode and decode JSON in Golang.

Table of Contents

• How to Send JSON Responses (Encoding)
• How to Use the Marshal Function for JSON Encoding
• How to Use the NewEncoder Function
• How to Parse JSON Requests (Decoding)
• How to Use the Unmarshal Function to Parse JSON Requests
• How to Use NewDecoder Function for JSON Decoding
• Custom JSON Marshaling and Unmarshaling
• How to Use JSON Marshaler
• How to Use JSON Unmarshaler
• Trade-offs
• Use Cases and Recommendations
• Conclusion

How to Send JSON Responses (Encoding)

JSON encoding is the process of converting Go data structures into JSON format.

Encoding refers to the process of converting data from one format to another. In the context of computing and data transmission, encoding typically involves converting data into a standardized format that can be easily stored, transmitted, or processed by different systems or applications.

Think of encoding like packing a suitcase for a trip. You take your clothes (data) and pack them into a suitcase (encoded format) so that they can be easily transported (transmitted) and unpacked (decoded) at your destination.

In the case of JSON encoding, the data is converted into a text-based format that uses human-readable characters to represent the data. This makes it easy for humans to read and understand the data, as well as for different systems to exchange and process the data.

Some common reasons for encoding data include:

• Data compression: Reducing the size of the data to make it easier to store or transmit.
• Data security: Protecting the data from unauthorized access or tampering.
• Data compatibility: Converting data into a format that can be read and processed by different systems or applications.
• Data transmission: Converting data into a format that can be easily transmitted over a network or other communication channels.

In Golang, we can use the encoding/json package to encode JSON data.

How to Use the Marshal Function for JSON Encoding

The Marshal function is the most commonly used method for encoding JSON data in Golang. It takes a Go data structure as input and returns a JSON-encoded string.

package main

import ( 
    "encoding/json"
    "fmt"
    "net/http"
 )

type Person struct { 
    Name string `json:"name"` 
    Age int `json:"age"`
}

func handler(w http.ResponseWriter, r *http.Request) { 
    person := Person{  Name: "John",  Age: 30, } 

    // Encoding - One step
    jsonStr, err := json.Marshal(person) 

    if err != nil {  
        http.Error(w, err.Error(), http.StatusInternalServerError)  
        return 
    } 

    w.Write(jsonStr)
}

func main() { 
    http.HandleFunc("/", handler) 
    http.ListenAndServe(":8080", nil)
 }

Code Explanation:

Imports:

• encoding/json: Provides functions for encoding and decoding JSON.
• fmt: For printing output.

User Struct:

• Defines a struct User with fields Name and Age.
• Struct tags (for example: json:"name") specify the JSON key names.

main Function:

• Creates a User instance.
• Calls json.Marshal to encode the user struct into JSON. This returns a byte slice and an error.
• If there's no error, it converts the byte slice to a string and prints it.

How to Use the NewEncoder Function

The NewEncoder function is used to encode JSON data to a writer, such as a file or network connection.

package main

import ( 
    "encoding/json" 
    "fmt" 
    "net/http"
)

type Person struct { 
    Name string `json:"name"` 
    Age int `json:"age"`
}

func handler(w http.ResponseWriter, r *http.Request) { 
    person := Person{  Name: "John",  Age: 30 } 

    // Encoding - 2 step . NewEncoder and Encode
    encoder := json.NewEncoder(w) 

    err := encoder.Encode(person) 

    if err != nil {  
        http.Error(w, err.Error(), http.StatusInternalServerError)  

        return 
   }}

func main() { 
   http.HandleFunc("/", handler) http.ListenAndServe(":8080", nil)
}

Code Explanation:

Inside the handler:

• The handler function is an HTTP handler that handles incoming HTTP requests.
• w http.ResponseWriter: Used to write the response.
• r *http.Request: Represents the incoming request.
• A Person instance named person was created and initialized with the values Name: "John" and Age: 30.
• A JSON encoder was created using json.NewEncoder(w), which will write the JSON output to the response writer w.
• The person struct was encoded to JSON and written to the response using encoder.Encode(person).
• If an error occurs during encoding, it is sent back to the client as an HTTP error response with a status code 500 Internal Server Error.

How to Parse JSON Requests (Decoding)

JSON decoding is the process of converting JSON data into Go data structures.

Decoding refers to the process of converting data from a standardized format back into its original form. In computing and data transmission, decoding involves taking encoded data and transforming it into a format that can be easily understood and processed by a specific system or application.

Think of decoding like unpacking a suitcase after a trip. You take the packed suitcase (encoded data) and unpack it, putting each item (data) back to its original place, so that you can use it again.

In the case of JSON decoding, the text-based JSON data is converted back into its original form, such as a Go data structure (like a struct or slice), so that it can be easily accessed and processed by the application.

Some common reasons for decoding data include:

• Data extraction: Retrieving specific data from a larger encoded dataset.
• Data analysis: Converting encoded data into a format that can be easily analyzed or processed.
• Data storage: Converting encoded data into a format that can be easily stored in a database or file system.
• Data visualization: Converting encoded data into a format that can be easily visualized or displayed.

Decoding is essentially the reverse process of encoding, and it's an essential step in many data processing pipelines.

In Golang, we can use the encoding/json package to decode JSON data.

How to Use the Unmarshal Function to Parse JSON Requests

The Unmarshal function is the most commonly used method for decoding JSON data in Golang. It takes a JSON-encoded string as input and returns a Go data structure.

package main

import ( 
    "encoding/json" 
    "fmt" 
    "net/http"
)

type Person struct { 
    Name string `json:"name"` 
    Age int `json:"age"`
}

func handler(w http.ResponseWriter, r *http.Request) { 
    var person Person err := json.NewDecoder(r.Body).Decode(&person)

    if err != nil {  
        http.Error(w, err.Error(), http.StatusBadRequest)  
        return
    } 

    fmt.Println(person.Name) 
    // Output: John fmt.Println(person.Age) 
    // Output: 30
}

func main() { 
    http.HandleFunc("/", handler) http.ListenAndServe(":8080", nil)
}

Code Explanation:

Inside the handler:

• The handler function is an HTTP handler that handles incoming HTTP requests.
• w http.ResponseWriter: Used to write the response.
• r *http.Request: Represents the incoming request.
• A variable person of type Person was declared.
• json.NewDecoder(r.Body).Decode(&person): This decodes the JSON request body into the person struct.
• If an error occurs during decoding, it sends back an HTTP 400 error response with a status code 400 Bad Request.
• If decoding is successful, the person struct fields Name and Age are printed using fmt.Println.

How to Use the NewDecoder Function for JSON Decoding

The NewDecoder function is also used to decode JSON data from a reader, such as a file or network connection.

package main

import ( 
    "encoding/json" 
    "fmt" 
    "net/http"
)

type Person struct { 
    Name string `json:"name"` 
    Age int `json:"age"`
}

func handler(w http.ResponseWriter, r *http.Request) { 

    decoder := json.NewDecoder(r.Body) 

    var person Person err := decoder.Decode(&person) 

    if err != nil {  
        http.Error(w, err.Error(), http.StatusBadRequest)  
        return 
       } 

    fmt.Println(person.Name) 
    // Output: John fmt.Println(person.Age) 
    // Output: 30
}

func main() { 
    http.HandleFunc("/", handler) 

    http.ListenAndServe(":8080", nil)
 }

Code Explanation:

Inside the handler function:

• The handler function is an HTTP handler that handles incoming HTTP requests.
• w http.ResponseWriter: Used to write the response.
• r *http.Request: Represents the incoming request.

Create a Decoder:

• decoder := json.NewDecoder(r.Body): Creates a new JSON decoder that reads from the request body.

Declare a Person Variable:

• var person Person: Declares a variable person of type Person.

Decode JSON into Person Struct:

• err := decoder.Decode(&person): Decodes the JSON from the request body into the person struct.
• If an error occurs during decoding, it sends an HTTP 400 error response with the status code 400 Bad Request and returns from the function.

Print the Decoded Values:

• fmt.Println(person.Name): Prints the Name field of the person struct.
• fmt.Println(person.Age): Prints the Age field of the person struct.

Custom JSON Marshaling and Unmarshaling

In some cases, the default JSON encoding and decoding behavior provided by json.Marshal and json.Unmarshal may not be sufficient. For instance, you may need to customize how certain fields are represented in JSON. This is where the json.Marshaler and json.Unmarshaler interfaces come in handy.

How to use JSON Marshaler

The json.Marshaler interface allows you to customize the JSON encoding of a type by implementing the MarshalJSON method. This method returns a JSON-encoded byte slice and an error.

func (p Person) MarshalJSON() ([]byte, error) {
    type Alias Person
    return json.Marshal(&struct {
        Alias
        Age string `json:"age"`
    }{
        Alias: (Alias)(p),
        Age:   strconv.Itoa(p.Age) + " years",
    })
}

In this example, the Age field is converted to a string with a " years" suffix when encoding to JSON.

How to use JSON Unmarshaler

The json.Unmarshaler interface allows you to customize the JSON decoding of a type by implementing the UnmarshalJSON method. This method takes a JSON-encoded byte slice and returns an error.

func (p *Person) UnmarshalJSON(data []byte) error {
    type Alias Person
    aux := &struct {
        Alias
        Age string `json:"age"`
    }{Alias: (Alias)(*p)}

    if err := json.Unmarshal(data, &aux); err != nil {
        return err
    }

    ageStr := strings.TrimSuffix(aux.Age, " years")
    age, err := strconv.Atoi(ageStr)
    if err != nil {
        return err
    }

    p.Age = age
    p.Name = aux.Name
    return nil
}

In this example, the Age field is converted from a string with a " years" suffix to an integer when decoding from JSON.

Trade-offs

From the various methods described above for encoding and decoding JSON. Here are the trade-offs for the most commonly used methods:

json.Marshal and json.Unmarshal:

Pros:

• Ease of Use: Straightforward for encoding (Marshal) and decoding (Unmarshal) JSON.
• Flexibility: Can be used with various types including structs, maps, slices, and more.
• Customization: Struct tags (json:"name") allow customization of JSON keys and other options.

Cons:

• Performance: May not be the fastest method for very large or complex JSON structures.
• Error Handling: Error messages can sometimes be less descriptive for deeply nested or complex data structures.

json.NewEncoder and json.NewDecoder:

Pros:

• Stream-Based: Suitable for encoding/decoding JSON in a streaming manner, which can handle large data sets without consuming a lot of memory.
• Flexibility: Can work directly with io.Reader and io.Writer interfaces, making them useful for network operations and large files.

Cons:

• Complexity: Slightly more complex to use compared to json.Marshal and json.Unmarshal.
• Error Handling: Similar to json.Marshal and json.Unmarshal, error messages can be less clear for complex structures.

Custom Marshaler and Unmarshaler Interfaces (json.Marshaler and json.Unmarshaler):

Pros:

• Customization: Full control over how types are encoded/decoded. Useful for handling complex types or custom JSON structures.
• Flexibility: Allows for implementing custom logic during marshaling/"unmarshaling."

Cons:

• Complexity: More complex to implement and use, as it requires writing custom methods.
• Maintenance: Increases the maintenance burden since custom logic needs to be kept in sync with any changes in the struct or data format.

Use Cases and Recommendations

• Simple Data Structures: Use json.Marshal and json.Unmarshal for straightforward encoding/decoding of simple data structures.
• Large Data Streams: Use json.NewEncoder and json.NewDecoder for working with large data streams or when interacting with files or network operations.
• Custom Requirements: Implement json.Marshaler and json.Unmarshaler interfaces when you need custom behavior for specific types.
• Quick Operations: Use anonymous structs for quick, throwaway operations where defining a full struct type is unnecessary.

Each method has its own strengths and trade-offs, and the best choice depends on the specific requirements of your application.

Conclusion

In conclusion, mastering JSON encoding and decoding is crucial for developing web applications in Golang.

By understanding the different methods available in the encoding/json package, you can choose the most suitable approach based on your specific requirements.

The Marshal and Unmarshal functions offer simplicity and flexibility for general use, while NewEncoder and NewDecoder provide efficient streaming capabilities for large datasets.

For scenarios that demand customized JSON representations, implementing the json.Marshaler and json.Unmarshaler interfaces gives you fine-grained control over the encoding and decoding processes.

Each method has its own strengths and trade-offs, and knowing when and how to use them will enable you handle JSON data effectively in your applications.

ChatGPT's Problem With JSON

Open up the ChatGPT UI and ask it for some JSON. Chances are you will get a response like you can see in the cover photo above: a JSON object presented to you in markdown format, with some text to either side explaining what the JSON shows.

If you try this same prompt in the OpenAI Playground, you can see the JSON is enclosed in three backticks (markdown syntax).

https://platform.openai.com/playground

Now this is great. ChatGPT has gone above and beyond by explaining the response in easy to understand terms – easy to understand, that is...for a human. Not for a machine. Machines need data in a format that sticks to a reliable, consistent, and predictable schema.

Ideally, you'd like to parse the response from ChatGPT in your code and do something useful with it, like this:

// Use the openai package from npm to call ChatGPT
import OpenAI from "openai";

// Create a new instance of the openai client with our API key
const openai = new OpenAI({ apiKey: process.env.OPENAI_KEY });

// Call ChatGPT's completions endpoint and ask for some JSON
const gptResponse = await openai.chat.completions.create({
    model: "gpt-3.5-turbo",
    temperature: 1,
    messages: [
        {
            role: "user",
            content: "Give me the JSON for an object that represents a cat."
        }
    ],
});

// Attempt to read the response as JSON,
// Will most likely fail with a SyntaxError...
const json = JSON.parse(gptResponse.choices[0].message.content);

But this will only work if gptResponse.choices[0].message.content is valid JSON every time.

It would also be nice to have JSON returned that reliably sticks to a schema:

type Cat = {
    name: string,
    colour: "brown" | "grey" | "black",
    age: number
}

// Read the response JSON and type it to our Cat object schema
const json = <Cat>JSON.parse(gptResponse.choices[0].message.content);

Not being able to rely on ChatGPT to return valid JSON in a predictable format can introduce bugs into your application, particularly when consistency is key. This becomes a real problem when you're writing code that relies on real-time responses from ChatGPT to trigger specific actions or updates so we need to find a solution. There are a couple of ways we can approach this...

How Prompt Engineering Can Help

One approach to solving this problem is through prompt engineering.

https://platform.openai.com/playground

Here, we have added some instructions to both the user prompt and the system message. The instructions attempt to force the model to return only the JSON we want, with the format we want.

And for many use cases this works acceptably. You can see from the screenshot above that the "Assistant" response is nothing but JSON, and the JSON does adhere to the schema we described for it.

But this doesn't work 100% of the time.

Here is the exact same pair of prompts with the temperature of the model set above 1. Notice how the colour field in the returned JSON does not now stick to one of the allowed values we specified in the User prompt:

Function Calling to the Rescue

Function calling is a new way to use the ChatGPT API. Instead of getting a message back from the language model, you get a request to call a function.

If you've ever used plugins in the ChatGPT UI, Function Calling is the feature behind the scenes that allow plugins to be integrated with the LLM's responses.

Plugins define the functions that are available to the model and allow it to call those functions in response to user prompts.

When using the API in your own code, you can also take advantage of function calling to better control the data you get back from the model, including forcing it to return JSON data in a predictable format.

Here is a basic request that uses Function Calling to return a message.function_call object in the response instead of a message.content string.

Here, we are simply asking ChatGPT to call a function ("getName") and a description of the function inside the functions: [] array. We are also instructing ChatGPT that we expect it to call this function in the response by setting the value of function_call to the name of our function.

const gptResponse = await openai.chat.completions.create({
    model: "gpt-3.5-turbo-0613",
    messages: [
        {
            role: "user",
            content: "Call the function 'getName' and tell me the result."
        }
    ],
    functions: [
        {
            name: "getName",
            parameters: {
                type: "object",
                properties: {}
            }
        }
    ],
    function_call: { name: "getName" }
});

// Will print "getName"...
console.log(gptResponse.choices[0].message.function_call.name);

It's important to note here that the getName() function does not need to actually exist anywhere in our codebase. All we are doing is telling ChatGPT it exists, and that is it available to be called.

How to add function arguments

Function calling is great because it makes the response from ChatGPT predictable and structured.

In the example above we will get an object back in gptResponse.choices[0].message.function_call that will contain details of how ChatGPT wants to execute our (imaginary) function.

This object looks like the below, with the name of the function and a stringified version of any function arguments it should be called with:

{
  name: "functionName",
  arguments: "{ \"arg1\": \"value\" }",
}

We can take advantage of this second arguments value by describing the shape of the function arguments, and having ChatGPT fill this value in with some JSON that conforms to our shape.

Here is the original example as a function call. Note how we have described the schema of the Cat object to ChatGPT inside the definition of a createCatObject function:

    type Cat = {
        name: string,
        colour: "brown" | "grey" | "black",
        age: number
    }

    const openai = new OpenAI({ apiKey: process.env.OPENAI_KEY });

    const gptResponse = await openai.chat.completions.create({
        model: "gpt-3.5-turbo-0613",
        messages: [
            {
                role: "user",
                content: "Create a new Cat object."
            }
        ],
        functions: [
            {
                name: "createCatObject",
                parameters: {
                    type: "object",
                    properties: {
                        name: {
                            type: "string"
                        },
                        colour: {
                            type: "string",
                            enum: ["brown", "grey", "black"]
                        },
                        age: {
                            type: "integer"
                        }
                    },
                    required: ["name", "colour", "age"]
                }
            }
        ],
        function_call: { name: "createCatObject" }
    });

    const functionCall = gptResponse.choices[0].message.function_call;
    const json = <Cat>JSON.parse(functionCall.arguments);

This completions request will instruct ChatGPT to create a new cat object and send it to the createCatObject() function in a specific format. The last line:

const json = <Cat>JSON.parse(functionCall.arguments);

parses the arguments from ChatGPT into our Cat type, which mirrors the description we gave the model of our the shape of our expected object.

Conclusion

Function calling with ChatGPT not only brings clarity and predictability to the results but also introduces a paradigm shift in how you can interface with machine learning models.

This structured approach ensures that the responses from ChatGPT are scalable, allowing for easier integration into various applications, whether they be simple web apps or more complex machine learning pipelines.

One of the most common responsibilities for front-end developers is handling the data in their front-end applications. You’ll need to be able to retrieve data from an API, manipulate it, and then render it on the screen in a modern web application for user interactions.

Efficient communication between the front-end and back-end is crucial for creating seamless and responsive applications.

Now imagine a scenario where you're working with a backend developer on a project and you're waiting on the API endpoint in order to connect to your frontend. There's a great tool that front-end developers can use to create a mock or dummy API during the development phase. This tool is called a "JSON Server".

In this article, you’ll learn how to use JSON servers for data storage in your React applications. You'll learn about the features and benefits, and a practical implementation in a front-end project.

The application will enable users to view a list of users and their details. The user's data will be created using JSON Server in a JSON file in the frontend application.

What is JSON Server?

JSON is an acronym for JavaScript Object Notation. JSON Server is a lightweight and easy-to-use Node.js tool that simulates a RESTful API using a JSON file as the data source. With JSON Server, front-end developers can create mock APIs without the need to write complex server-side code, or when a backend API isn't ready yet.

This mock API sends requests to an endpoint that will be provided. It responds to HTTP requests, and this makes it ideal for rapid development for front-end developers. JSON Server also enables developers to perform CRUD operations and saves data in JSON files. JSON comes in a key-value pair and is written in this format:

{  
  "name": "Jane",   
  "age": 30,   
  "gender": "Female"
}

The "name", "age", and "gender" are called properties and the "Jane", "30", and "female" are the values of each of the properties.

JSON data file can come in two formats — the array format and the object with nested objects format.

Array Format

[
  {
    "id": 1,
    "name": "John Doe",
    "age": 25
  },
  {
    "id": 2,
    "name": "Jane Smith",
    "age": 40
  }
]

Object with Nested Objects Format

{
  "users": {
    "1": {
      "name": "John Doe",
      "age": 25
    },
    "2": {
      "name": "Jane Smith",
      "age": 30
    }
  }
}

Features of JSON Server

Here are some of the features of JSON Server:

• It is easy and quick to set up. It is also user-friendly for front-end developers and newbie back-end developers.
• It supports common HTTP methods like the GET, POST, PUT, and DELETE methods, just like a real backend API server would.
• With JSON Server, you can perform Create, Read, Update, Delete (CRUD) operations on data to build an interactive application.
• JSON Server offers developers the ability to create custom routes to handle more complex scenarios.

Benefits of Using JSON Server

Here are some of the benefits of using JSON Server:

• JSON Server grants front-end developers the ability to quickly create functional API prototypes that can be tested and modified while waiting for the backend server to be ready.
• Front-end developers can use JSON Server to simulate different scenarios, and error cases during testing to enhance their application.

How to Set up JSON Server in an Application

You'll need Node.js and npm installed on your system, as they're both prerequisites for this setup.

Follow these steps to set up and use JSON Server in your frontend application:

Step #1 - Install JSON Server

To install JSON Server in your application, navigate to your project directory in your terminal or command prompt and type in this command: npm install -g json-server.

This will install the JSON server globally on your system. If you want to install it locally for just a particular project instead, use this command: npm i json-server.

Step #2 - Create a JSON File

Create a JSON file in your project directory that will act as the data source. This JSON file should have a .json file extension. What do I mean? Let's say you want your JSON file name to be 'db', it means you'll create a file called db.json.

Step #3 - Create Data

Define your data inside the JSON file. This JSON data can be an array of objects or an object with nested objects. Each object represents a data entity and should each have a unique id.

Step #4 - Start the Server

Start up the JSON Server by typing this command into your terminal: json-server --watch db.json. This will run on "https://localhost:3000" by default. You can change the port it's running on by specifying a different port number when starting the server using the --port flag.

For example, if you want your server to run on port 8000 instead of the default (3000), use this command while starting the server: json-server --watch db.json --port 8000. You can then view this in your browser on port 8000.

JSON Server will automatically generate RESTful endpoints based on the data you defined in your JSON file.

If you have a JSON file with an array of "users", this is the endpoint that will be automatically generated by the JSON Server:

• GET /users - This retrieves a list of all resource entities of users.
• GET /users/:id - This retrieves a specific user by its id.
• POST /users - This creates a new user.
• PUT /users/:id - This updates a user based on a specified id.
• DELETE /users/:id - This deletes a user based on the specified id.

This pattern makes it easy to interact with the mock API in a RESTful manner just like one would do with a real backend API.

How to Build a Simple Front-end Application Using JSON Server

To further understand how to use the JSON Server in a real project, let's take a look at an example. You'll build a simple React.js application that displays users' data from a JSON data file using the JSON server on the front end.

Ready? Let's get started. Here's a quick look at the app you'll build by the end of this article.

User data app home page

This app will have just two pages. The first page is displayed above, and it is a list of users with their names and email addresses. The other page will contain more details about each user based on the user id when the View full details button is clicked.

Installing Dependencies

Create a React app using CRA or Vite (recommended). I made use of the yarn package manager to create my app using this command: yarn create vite. Choose a project name and select "React" as the framework and "Typescript" as the variant. You can select JavaScript as your variant instead if you wish.

Once this is done, navigate to your project and use the yarn command to install the dependencies. After it has successfully installed the dependencies, run your development server with this command: yarn run dev. Open up your browser, paste the URL (http://127.0.0.1:5173/) and you'll see your app running on the browser.

Install the JSON server in your application by using this command: npm install -g json-server for global installation, or the npm i json-server command for local installation.

The last package to install is react-router to allow page navigation using this command: npm i react-router-dom.

Folder Structure

Now you want your application to have a neat folder structure so follow these guidelines:

First, create a data folder in the root directory of your project. Inside this data folder, you'll create a JSON file called db.json. This is where the JSON data will be defined.

Next, in the src directory, create a components folder. In this folder, you'll create 3 files: Home.tsx, UserDetails.tsx, and UserList.tsx. These are the components that will render the logic and UI of the application.

In the src directory, create a file named useFetch.tsx. This file will contain the code for the API implementation. The main component of your application — the App.tsx file — is where you'll handle pages routing.

Building the App

The first component you'll be modifying is the App.tsx file. Paste the following lines of code in the component:

import Home from './components/Home';
import UserDetails from './components/UserDetails';
import { BrowserRouter as Router, Routes, Route } from 'react-router-dom';

function App() {
  return (
    <Router>
      <Routes>
        <Route path="/" element={<Home />} />
        <Route path="/users/:id" element={<UserDetails />} />
      </Routes>
    </Router>
  )
}

export default App

This is just to provide routes for the application.

Go to your db.json file and define your JSON data inside. Paste these lines of code into the JSON file:

{
  "users": [
    {
      "id": 1,
      "name": "Juliet Oma",
      "email": "julie@yahoo.com",
      "number": "08100000000"
    },
    {
      "id": 2,
      "name": "James Williams",
      "email": "jameswilly@gmail.com",
      "number": "08111111111"
    },
    {
      "id": 3,
      "name": "Ahmed Ali",
      "email": "ahmedali012@gmail.com",
      "number": "09022222222"
    },
    {
      "id": 4,
      "name": "Grace Funsho",
      "email": "gracefunsho@gmail.com",
      "number": "09033333333"
    }
  ]
}

Start up your JSON server using this command: json-server --watch db.json --port 8000. This command will watch the db.json file and wrap the API endpoint running on port 8000. If you check your terminal, this is how it will look like:

If you copy out that Resources URL (http://localhost:8000/users) provided in the terminal and open it up on your browser, you'll see a JSON data displaying all the user data you defined in your db.json file.

JSON file on the browser

Next, you'll need to write code to implement the API. This will be done in the useFetch.tsx file. This is basically a custom React Hook that you'll create to handle asynchronous data fetching from a given URL.

import { useState, useEffect } from 'react';

interface UseFetchResult {
    data: any | null;
    isPending: boolean;
    error: any | null;
}

const useFetch = (url: string): UseFetchResult => {
    const [data, setData] = useState<any | null>(null);
    const [isPending, setIsPending] = useState<boolean>(true);
    const [error, setError] = useState<any | null>(null);

    useEffect(() => {
        setTimeout(() => {
            fetch(url)
                .then(res => {
                    if (!res.ok) {
                        throw Error('Error fetching users data');
                    }
                    return res.json();
                })
                .then(data => {
                    setData(data);
                    setIsPending(false);
                    setError(null);
                })
                .catch(err => {
                    setIsPending(false);
                    setError(err.message);
                });
        }, 1000);
    }, [url]);

    return { data, isPending, error };
}

export default useFetch;

Let's breakdown the code above.

First, you'll need to import two hooks. The useState & useEffect hooks will be used to manage states and perform side effects in the custom hook.

While setting up my React project, I selected Typescript as a variant, so my app is using Typescript. If you did the same, you'll need to declare an interface named useFetchResult which specifies the structure of the result object that the useFetch hook will return. It contains three properties: data, isPending, and error.

The line of code const useFetch = (url: string): UseFetchResult => { ... } defines the useFetch custom hook. It takes a url parameter of type string and returns an object that complies with the UseFetchResult interface.

Next, you need to initialize three variables using the useState hook: data to store the fetched data, isPending to indicate whether the data fetching is in progress, and error state to store any error that occurs during the fetching process. Each has an initial value of null, true, and null respectively.

The useEffect hook is used to perform the data fetching when the URL changes. It runs after the initial render and whenever the URL dependency changes. Inside the useEffect function, a setTimeOut is defined to simulate a 1000 milliseconds (1 second) delay before initiating the data fetching.

The fetch method is used to make a GET request to the specified UR:. The response is checked using res.ok. If the response is not ok, it throws an error. The response is then converted to JSON using the res.json() method and stored in the data variable. The isPending state is set to false to indicate that the data fetching is complete, and the error state is set to null to clear previous error.

If any error occurs during the data fetching process, it is caught in the .catch block. isPending is set to false, and then the error state is updated with the error message.

The custom hook returns an object containing the data, isPending, and error states, allowing other components to access the fetched data and its state.

Creating the List of Users in Tabular form

This is basically the component that will display all user's info in a table. In your UserList.tsx file, paste these lines of code:

import React from 'react';
import { Link } from 'react-router-dom';

interface User {
    id: number;
    name: string;
    email: string;
    number: string;
}

interface UserListProps {
    users: User[];
    name: string;
}

const UserList: React.FC<UserListProps> = ({ users, name }) => {

    return (
        <>
            <section>
                    <section>
                        <h1>Users List</h1>
                    </section>
                    <section>
                        <table>
                            <thead>
                                <tr>
                                    <th>Name</th>
                                    <th>Email</th>
                                    <th>Details</th>
                                </tr>
                            </thead>
                            <tbody>
                                {users.map((user) => (
                                    <tr key={user.id}>
                                        <td>
                                          <p>{user.name}</p>
                                        </td>

                                        <td>
                                            <p>{user.email}</p>
                                        </td>

                                        <td>
                                            <Link to={`/users/${user.id}`}>
                                                <button>
                                                    View full details
                                                </button>
                                            </Link>
                                        </td>
                                    </tr>
                                ))}
                            </tbody>
                        </table>
                    </section>
            </section>
        </>
    );
};

export default UserList;

We started by importing Link from the react-router-dom library.

The next line of code is the interface User, a typescript interface that defines the structure of a user object which has four properties with each property having a specified data type.

Then comes the interface UserListProps which defines the structure of an object that has two properties — users and name. users is an array of objects that matches the User interface.

This component takes an object with the userListProps interface as an argument, destructures users from it, and uses it as props in the component.

Then you have the JSX element to display the list of users in tabular format. We mapped through the list using the map method to render each user in a row.

We then added a button to each row that triggers navigation to a specific route (/users/${user.id}). These routes will display the detailed information of the user whose id is provided as part of the URL.

We used a template literal to create a dynamic URL that includes the user.id value.

Your app is gradually taking shape.

Rendering User List on the Browser

The next component you'll be working on is the Home.tsx component. This is where the user list will be rendered on the screen and it's the home page of your application.

import UserList from './UserList';
import useFetch from '../useFetch';

const Home = (): JSX.Element => {
    const { data: users, isPending, error } = useFetch('http://localhost:8000/users')

    return (
        <section>
            {error && <p>{error}</p>}
            {isPending && <p>Loading users...</p>}
            {users && <UserList users={users} />}
        </section>
    );
};

export default Home;

We first imported the UserList component and the useFetch custom hook from where they're located in the project directory.

The const { data: users, isPending, error } = useFetch('http://localhost:8000/users') line of code calls the useFetch custom hook to fetch data from the specified URL (http://localhost:8000/users). The hook returns an object with three properties: data (the fetched users), isPending (loading status), and error (error message).

We rendered the UI with the JSX elements. If the error state has a value, it renders a paragraph element containing the error message. If the isPending state is true, it renders a paragraph element indicating that users are being loaded. If the users state has data (if it is not null), it renders the UserList component and passes the users data as a prop.

Displaying Users Full Details

We want to add an additional functionality where you can view more details about a particular user on another page by clicking on the View full details button that is in same row with their name.

To do that, paste these lines of code in the UserDetails.tsx component:

import { useParams } from 'react-router-dom';
import useFetch from '../useFetch';

const UserDetails = () => {
    const { id } = useParams();
    const { data: user, error, isPending } = useFetch("http://localhost:8000/users/" + id);

    return (
        <>
            <section>
                {isPending && <p>Loading user details...</p>}

                {error && <p>{error}</p>}

                {user && (
                    <>
                        <h1>User {user.id} details</h1>
                        <h2>{user.name}</h2>
                        <p>{user.email}</p>
                        <p>{user.number}</p>
                    </>
                )}
            </section>
        </>
    );
};

export default UserDetails;

From the code above, we imported the useParams hook from the react-router-dom library. This allows access to URL parameters. We also imported the useFetch custom hook.

The const { id } = useParams(); uses the useParams hook to extract the id parameter from the URL. This is typically used in routes like /users/:id.

The next line of code after that calls the useFetch custom hook to fetch data from a specific user's URL based on the extracted id parameter. The hook returns an object with data, error, and isPending properties.

Then comes the JSX elements to render the users' details. {isPending && <p>Loading user details...</p>} renders a paragraph indicating that the user details are being loaded if isPending is true.

{error && <p>{error}</p>} will render a paragraph displaying an error message, if there is an error. While {user && (...)} renders some user details like id, name, email, and number, if the data is available.

When you go back to your browser, you'll see a list of user data in a table. Each user row in the table will have a button that you can click to open up a page for you to see that specific user's full details.

Now this app won't look so good because it hasn't been styled. So go ahead and style your application using whatever styling technique you want to use. A quick view of my app after styling:

via GIPHY

I know the UI isn't so great but the main focus here was the functionality. Notice how I added a loading animation that displays when the user list and details are waiting to be fetched? You can do this by using an animation library like Framer motion or creating a spinner component.

The loading animation or spinner will be rendered while data is waiting to be fetched instead of just a "Loading....' text. This can make your app look better and help you with user engagement.

You can modify the user data or add more details to your user list like users' job description, their title, and so on in your db.json file and you'll see the changes reflected in the list on the browser.

This app sends a request to the API endpoint provided when you start the JSON server and displays the response on the browser. That's basically how a real backend API server works. In this case, we were able to achieve our functionality using JSON Server.

It is important to know that this JSON mock API can't be used at the production stage. It can only be used in the development stage to create dummy JSON data. This means that you can't ship it to production as the JSON data file only runs on a localhost port.

Wrapping Up

That's all for this article. Here, you learned about the JSON server and how to use it in a front-end React.js application.

You can also perform full CRUD operations with this data from your JSON file on your front end. In this article, I only demonstrated the Read operation. This app can be further improved to enable people Create users, Update users, and Delete users from the JSON database.

If you're looking to learn how to implement a real third-party API in your React application, check out my previous article.

JSON has become the industry standard for data interchange between online services. And it's widely utilized in modern programming languages, including Python.

JSON data is frequently expressed as nested dictionaries, lists, and scalar values such as texts, numbers, booleans, and null. It is named JSON because it closely mimics the syntax used in JavaScript objects.

In this tutorial, you will explore the JSON module in Python and learn how to effectively work with JSON data.

Python's Built-in JSON Module

JSON plays an important role in Python programming because it allows efficient data serialization and deserialization. It enables Python programs to effortlessly communicate with web services, exchange data, and store structured information.

Developers can use JSON to seamlessly link their Python programs with a variety of APIs, databases, and external systems that use JSON for data representation.

If you're looking to learn how to interact with web services using Python, check out my tutorial on the requests module.

The built-in JSON module in Python provides a powerful set of methods and classes that make working with JSON data simple. Developers can use it to encode Python objects into JSON strings and decode JSON strings back into Python objects.

How to Store JSON Data in a File

When working with JSON data in Python, you'll often need to save the data or share it with others. Storing JSON data in a file enables quick retrieval and data persistence.

In this section, you'll learn how to use Python's json.dump() function to save JSON data to a file. This process involves serializing the JSON data and saving it to a file, which you can subsequently read and use as needed.

The json.dump() function

The json.dump() function in Python allows you to store JSON data directly into a file. This function takes two parameters: the data to be serialized and the file object where the data will be written.

To write JSON data to a file, you need to follow a few steps. First, you need to open a file in write mode, specifying the file path. Then, you can use the json.dump() function to serialize the data and write it to the file. Finally, you need to close the file to ensure that all the data is properly saved.

Let's learn how to store data in a file using the horoscope API response as an example.

Assume you have made a GET request to the following URL: https://horoscope-app-api.vercel.app/api/v1/get-horoscope/daily?sign=capricorn&day=today, which provides the daily horoscope for the Capricorn sign.

import requests
import json

# Make the GET request to the horoscope API
response = requests.get("https://horoscope-app-api.vercel.app/api/v1/get-horoscope/daily?sign=capricorn&day=today")
data = response.json()  # Convert the response to JSON

# Store the JSON data in a file
with open("horoscope_data.json", "w") as file:
    json.dump(data, file)

print("Data stored successfully!")

In the code above, you use the requests library to make a GET request to the Horoscope API. You then extract the JSON data from the response using the .json() method. Finally, you open a file named horoscope_data.json in write mode using the with statement, and you use json.dump() to store the data in the file.

Check out this tutorial to learn how to find out your horoscope using Python.

If you open the horoscope_data.json file, you'll see contents similar to below:

{
  "data": {
    "date": "Jun 3, 2023",
    "horoscope_data": "The forecast today is stormy. You may have sensed that there was some tension clouding the conversation at home. Resentments were left unsaid and subtle power games were played without resolution. Today, Capricorn, it all becomes too unbearable for you. Regardless of the risks involved, you will take measures to clear things up."
  },
  "status": 200,
  "success": true
}

How to Retrieve Data from a JSON File

You'll often need to read data from a JSON file. For example, you may need to read configuration settings from a JSON file. Python's JSON module provides the json.load() function, which allows you to read and deserialize JSON data from a file.

In this section, you will learn how to use the json.load() function to retrieve JSON data from a file and work with it in your Python programs.

The json.load() function

The json.load() function accepts a file object as an argument and returns deserialized JSON data in the form of Python objects such as dictionaries, lists, strings, numbers, booleans, and null values.

To read JSON data from a file, you need to open the file in read mode, extract the data using the json.load() function, and store it in a variable for further processing. It's important to ensure that the file being read contains valid JSON data – otherwise, it may raise an exception.

Let's see how you can retrieve the data from the previously created horoscope_data.json file:

import json

# Retrieve JSON data from the file
with open("horoscope_data.json", "r") as file:
    data = json.load(file)

# Access and process the retrieved JSON data
date = data["data"]["date"]
horoscope_data = data["data"]["horoscope_data"]

# Print the retrieved data
print(f"Horoscope for date {date}: {horoscope_data}")

In the code above, you open the file horoscope_data.json in read mode using the with statement. You then use the json.load() function to deserialize the JSON data from the file into the data variable. Finally, you access specific fields of the JSON data (e.g., "date" and "horoscope_data") and process them as needed.

How to Format the JSON Output

When you read data from a JSON file and print it, the output is displayed as a single line, which may not resemble the structured format of JSON.

import json

# Retrieve JSON data from the file
with open("horoscope_data.json", "r") as file:
    data = json.load(file)

print(data)

Output:

{'data': {'date': 'Jun 3, 2023', 'horoscope_data': 'The forecast today is stormy. You may have sensed that there was some tension clouding the conversation at home. Resentments were left unsaid and subtle power games were played without resolution. Today, Capricorn, it all becomes too unbearable for you. Regardless of the risks involved, you will take measures to clear things up.'}, 'status': 200, 'success': True}

The json.dumps() function

The JSON module provides you with a json.dumps() function to serialize Python objects into a JSON formatted string. It provides various options for customization, including formatting the output to make it more human-readable.

The json.dumps() function provides several options to customize the output. The most commonly used is the indent which allows you to specify the number of spaces used for indentation.

import json

# Retrieve JSON data from the file
with open("horoscope_data.json", "r") as file:
    data = json.load(file)

# Format the data
formatted_data = json.dumps(data, indent=2)

print(formatted_data)

Output:

{
  "data": {
    "date": "Jun 3, 2023",
    "horoscope_data": "The forecast today is stormy. You may have sensed that there was some tension clouding the conversation at home. Resentments were left unsaid and subtle power games were played without resolution. Today, Capricorn, it all becomes too unbearable for you. Regardless of the risks involved, you will take measures to clear things up."
  },
  "status": 200,
  "success": true
}

As you can see, the JSON data is now formatted with proper indentation, enhancing its readability. This technique can be applied to any JSON data, allowing you to present JSON output in a more organized and visually appealing way.

The json.tool Command Line Tool

Python's JSON module provides a convenient command line tool called json.tool that allows you to format and pretty-print JSON data directly from the command line. It is a useful utility for quickly visualizing the structure and contents of JSON data in a more readable and organized format.

To use json.tool, you can execute the following command in your command-line interface:

python -m json.tool <input_file> <output_file>

where:

• python -m json.tool invokes the json.tool module using the Python interpreter.
• <input_file> represents the path to the JSON file you want to format.
• <output_file> is an optional argument that specifies the file to which you want to save the formatted JSON output. If not provided, the formatted output will be displayed on the console.

Let's say you have a horoscope_data.json file with the following contents:

{
  "data": {
    "date": "Jun 3, 2023",
    "horoscope_data": "The forecast today is stormy. You may have sensed that there was some tension clouding the conversation at home. Resentments were left unsaid and subtle power games were played without resolution. Today, Capricorn, it all becomes too unbearable for you. Regardless of the risks involved, you will take measures to clear things up."
  },
  "status": 200,
  "success": true
}

Notice that the above JSON file has an indentation of two spaces.

To pretty-print this JSON file using json.tool, you can execute the following command:

python -m json.tool horoscope_data.json

The output will be:

{
    "data": {
        "date": "Jun 3, 2023",
        "horoscope_data": "The forecast today is stormy. You may have sensed that there was some tension clouding the conversation at home. Resentments were left unsaid and subtle power games were played without resolution. Today, Capricorn, it all becomes too unbearable for you. Regardless of the risks involved, you will take measures to clear things up."
    },
    "status": 200,
    "success": true
}

As you can see in the example, executing the json.tool module with the input file path formats the JSON data and displays the formatted output on the console.

You can also redirect the formatted output to an output file by specifying the output file name as the second argument:

python -m json.tool horoscope_data.json formatted_data.json

This command formats the JSON data from horoscope_data.json and saves the formatted output to formatted_data.json.

JSON Encoding Custom Objects

The JSON module in Python allows you to encode and decode custom objects by using JSON encoder and decoder classes. You can define custom serialization and deserialization logic for your objects using these classes.

JSONEncoder class allows you to customize the encoding process. To define how your custom object should be encoded into JSON format, you can extend the JSONEncoder and change its default method.

Here's an example of how you can extend the JSONEncoder class and customize the encoding process for a custom object:

import json


class Person:
    def __init__(self, name, age):
        self.name = name
        self.age = age


class PersonEncoder(json.JSONEncoder):
    def default(self, obj):
        if isinstance(obj, Person):
            return {"name": obj.name, "age": obj.age}
        return super().default(obj)


# Create a custom object
person = Person("Ashutosh Krishna", 23)

# Encode the custom object using the custom encoder
json_str = json.dumps(person, cls=PersonEncoder)

# Print the encoded JSON string
print(json_str)

In this example, you define a custom class Person with name and age attributes. You then create a subclass of JSONEncoder called PersonEncoder and override its default method. Within the default method, you check if the object being encoded is an instance of Person. If it is, you provide a JSON-serializable representation of the object by returning a dictionary containing the name and age attributes. If the object is not of type Person, you call the default method of the superclass to handle other types.

By using json.dumps and specifying the cls parameter as your custom encoder class PersonEncoder, you can encode the person object into a JSON string. The output will be:

{"name": "Ashutosh Krishna", "age": 23}

Similarly, you can specify custom decoding logic in the JSON decoder class, JSONDecoder. To define how JSON data should be decoded into your custom object, extend the JSONDecoder and override its object_hook function.

How to Create JSON from a Python Dictionary

You can use the json.dumps() function provided by the JSON module to create JSON from a Python dictionary. This function takes a Python object, typically a dictionary, and converts it into a JSON string representation.

import json

# Python dictionary
data = {
    "name": "Ashutosh Krishna",
    "age": 23,
    "email": "ashutosh@example.com"
}

# Convert dictionary to JSON string
json_str = json.dumps(data)

# Print the JSON string
print(json_str)

In this example, you have a Python dictionary data representing some data. By calling json.dumps(data), you convert the dictionary into a JSON string. The output will be:

{"name": "Ashutosh Krishna", "age": 23, "email": "ashutosh@example.com"}

How to Create a Python Dictionary from JSON

To create a Python dictionary from JSON data, you can use the json.loads() function provided by the JSON module. This function takes a JSON string and converts it into a corresponding Python object, typically a dictionary.

import json

# JSON string
json_str = '{"name": "Ashutosh Krishna", "age": 23, "email": "ashutosh@example.com"}'

# Convert JSON string to Python dictionary
data = json.loads(json_str)

# Access the dictionary values
print(data["name"])
print(data["age"])
print(data["email"])

In this example, you have a JSON string json_str representing some data. By calling json.loads(json_str), you convert the JSON string into a Python dictionary. You can then access the values in the dictionary using their respective keys.

The output will be:

Ashutosh Krishna
23
ashutosh@example.com

Wrapping Up

Understanding the Python JSON module is necessary for working with JSON data because it is widely used for data exchange and storage in a variety of applications.

You can efficiently handle JSON data, interface with APIs, and deal with configuration files if you learn how to use the JSON module.

PostgreSQL is a powerful open-source relational database management system (RDBMS). It was initially created as a successor to the Ingres database system and was later named "PostgreSQL" (short for "Post-Ingres SQL").

PostgreSQL is known for its robustness, reliability, and scalability, making it a popular choice for large and complex database applications. It offers advanced features such as support for JSON and other non-relational data types as well as support for spatial data.

JSON file support was first introduced in PostgreSQL v9.2, and with every new release, steady improvements are being made.

In this comprehensive guide, you will learn about JSON functions and operators in PostgreSQL. We’ll also go into the basics of storing JSON data in PostgreSQL, how to query JSON data in PostgreSQL to make it readily accessible, and finally, you’ll learn about working with JSON arrays.

What is JSON?

JSON stands for JavaScript Object Notation. It is a common way to store data, especially in web applications. It is pretty similar to HTML or XML and was made for applications to easily read JSON files.

Key-Value Pairs: JSON data is written in key-value pairs surrounded by quotes. Here’s an example of a key-value pair “email”: “jsonlearning@gmail.com”. “Email” here is the key, while “jsonlearning@gmail.com” represents the value. The two are separated by a colon “:”.

Objects: An object is a key-value pair or pairs enclosed in curly brackets. Whenever a key-value pair is enclosed in curly brackets it becomes an object and can be treated as a single unit. Multiple key-value pairs can be added in an object, separated with a comma.

Example of a JSON object:

{“email” : “jsonlearning@gmail.com”, 
“country” : “United Kingdom”}

Arrays: Arrays in JSON are a way to store a collection of values within a single JSON object. An array in JSON is represented by square brackets [ ] containing a comma-separated list of values.

Here’s an example of an array in JSON: [ "apple", "banana", "cherry"].

Arrays in JSON can also be nested, meaning that an array can contain other arrays or objects as values. Here's an example of a nested array:

{ "firstname" : "Claire", 
"location" : "United Kingdom", 
"blog" : [{ "id" : "1", 
"title" : "Welcome to my blog" }, 
{ "id" : "2", 
"title" : "My first programming language" }]}

In this example of nested Arrays, you can see, “blog” is contained in an array, and the array also contains several objects.

JSONB in PostgreSQL

What is the JSONB data type? And how is it different from JSON?

JSONB (JSON Binary) is a data type in PostgreSQL that allows you to store and manipulate JSON data in a more effective and efficient way than the regular JSON data type.

JSONB stores JSON data in a binary format, which enables faster indexing and query performance compared to the regular JSON data type. This is because the binary format allows for more efficient storage and retrieval of JSON data, particularly when dealing with large or complex JSON objects.

In addition, JSONB supports additional indexing options, such as the ability to index specific keys within a JSON object, which allows for even faster queries.

The regular JSON data type in PostgreSQL stores JSON data as plain text, without any binary encoding or special indexing support. This makes it simpler to use but can result in slower query performance when dealing with large or complex JSON objects.

How to create a table with the JSONB data type

You can create a table and give a column a data type of JSON or JSONB, just like you give a column the data type of Int, VARCHAR, or Double. You can just simply give the column a data type of either JSON or JSONB.

Here’s an example of creating a Table Journal and giving the column “diary_information” the data type JSONB.

CREATE TABLE journal (
  id Int NOT NULL PRIMARY KEY, day VARCHAR, 
  diary_information JSONB
);

Because we specified the data type to be of type JSONB, any data held in that column must be a valid JSON.

How to insert JSON data into tables

After creating a table and giving our column the data type JSONB, how do we insert values into the column? Remember the data must be in a valid JSON format.

To insert data to our table, we use this statement:

INSERT INTO journal (id, day, diary_information) 
VALUES 
  (
    1, “Tuesday”, '{"title": "My first day at work", "Feeling": "Mixed                   feeling"}'
  );

If we try to retrieve the information using a select statement SELECT * FROM journal we get the following output, meaning the records have been inserted.

In the next section, we’ll take a look at some Functions and Operators.

Overview of JSON Functions and Operators

Functions and operators allow you to store, manipulate, and query data in JSON format in PostgreSQL.

Here are some commonly used PostgreSQL Functions and operators used in working with JSON files:

• ->: This operator allows you to extract a specific value from a JSON object, you specify the key as a “child” to the “parent”.

For example:

To retrieve a specific value from a JSON object using the -> operator, use it in a SELECT statement as seen below:

SELECT 
  Id, 
  day, 
  diary_information -> 'Feeling' AS Feeling 
FROM 
  journal;

Something to note here is that this operator extracts the field name, with the quote around it.

• ->>: This operator allows you to extract a JSON object field as text without the quotes around it from a JSON object.

For example:

SELECT 
  id, 
  day, 
  dairy_information ->> 'Feeling' as Feeling 
FROM 
  products;

This will extract the value of the "material" key as text from the "features" column in the "products" table.

• json_agg: This function aggregates JSON values into a JSON array.

For example, SELECT json_agg(my_column) FROM my_table; will return a JSON array containing the values in the "my_column" column of the "my_table" table.

• jsonb_set: This function updates a JSON object field with a new value. For example:

UPDATE 
  my_table 
SET 
  json_column = jsonb_set(
    json_column, '{field_name}', '"new_value"'
  ) 
WHERE 
  id = 1;

To update an existing JSON record, we use the function jsonb_set() () in an update statement.

For example, to update the record in the table we created earlier, you can run the following code:

UPDATE 
  journal 
SET 
  diary_information = jsonb_set(
    diary_information, '{Feeling}', '"Excited"'
  ) 
WHERE 
  id = 1;

This will update the "Feeling" key in the "diary_information" column of the "journal" table with the new value "Excited".

• JSONB_BUILD_OBJECT: Manually inserting JSON values can lead to errors, especially if it’s your first time working with JSON data. But with this function, you can input values without having to worry about curly brackets, colons, and the rest of them.

You can use a JSONB_BUILD_OBJECT function to insert a plain text record and this will convert it to JSON data. For example if you run the code:

JSONB_BUILD_OBJECT('Morning', 'Everybody is annoying today', 'Evening', 'Cannot wait to go home’)

This will create a value that looks like this:

{“Morning”: “Everybody is annoying today”, “Evening”: “Cannot wait to go home”}

Using this function in an insert statement:

INSERT INTO journal (id, day, feeling) 
VALUES 
  (
    2, 
    'Wednesday', 
    JSONB_BUILD_OBJECT(
      'Tired', 
      'Everybody is annoying today', 
      'Hungry', 
      'Cannot wait to go home’));

The new record will be added to the table and because we used the JSONB_BUILD__OBJECT function, the values that follow will be in JSON format.

These are the few functions and operators we can cover in this article. You can read more on JSON functions and Operators in PostgreSQL in the official documentation here.

How to Work with JSON Arrays in PostgreSQL

In PostgreSQL, you can store JSON data as a column value in a table, and you can use JSON arrays to store a collection of JSON objects in a single column.

Working with JSON arrays in PostgreSQL involves various operations, such as inserting, querying, and manipulating JSON data. Let's see how those work.

How to insert JSON arrays into tables

To insert JSON arrays into a table in PostgreSQL, you can use the INSERT INTO statement along with the VALUES clause to specify the JSON array as a string value.

Here's an example:

Suppose you have a table called employees with columns id, name, and skills. The skills column stores an array of JSON objects representing the skills of each employee.

To insert a new employee record with the following details:

• id: 1
• name: John
• skills: [{"name": "Python", "level": "Intermediate"}, {"name": "JavaScript", "level": "Expert"}]

You can use the following SQL statement:

INSERT INTO employees (id, name, skills) 
VALUES 
  (
    1, 'John', '[{"name": "Python", "level": "Intermediate"}, {"name":   "JavaScript", "level": "Expert"}]'
  );

How to query JSON arrays using JSON operators

To query JSON arrays in PostgreSQL, you can use the various JSON functions and operators provided by PostgreSQL. These functions allow you to extract specific values or elements from the JSON array and perform various operations on them. Let's look at an example.

How to extract values from a JSON array

Suppose you have a table called employees with a skills column that stores an array of JSON objects representing the skills of each employee.

To extract the names of all employees who have "Python" as one of their skills, you can use the ->> operator to extract the "name" property of each skill object, and the @> operator to check if the resulting array contains the value "Python":

SELECT 
  name 
FROM 
  employees 
WHERE 
  skills @ > '[{"name": "Python"}]' :: jsonb

This is just an example of the many ways in which you can query and manipulate JSON arrays using the JSON operators provided by PostgreSQL.

Conclusion

In conclusion, PostgreSQL support for JSON provides developers with the ability to simplify data models, enhance application performance, and so much more. This also provides a seamless relationship between relational and non-relational data structures.

You have learned about the JSON and JSONB data types, and what key-value pairs, objects, and arrays are in JSON. You also learned about some operators and functions in PostgreSQL to query data in JSON format.

If you learned a thing or two from this article, please share it with others.

JSON files typically contain objects, arrays, strings, and numbers. But you can also have regular expressions in a JSON file. And that’s what we are going to take a look at in this article.

How to Write RegEx inside JSON

Remember each entry in a JSON file is a key:value pair surrounded by double quotes. So, if you want to write RegEx inside your JSON file, you need to specify the key and the value and surrounded both with double quotes.

The key can just be an arbitrary name, and the value would be the regex itself. Here’s an example of a JSON file with regular expressions:

// data.json file

{
  "nigeriaPhone": "^(\\+?234|0)[789]\\d{9}$",
  "email": "^([a-zA-Z0-9_\\-\\.]+)@([a-zA-Z0-9_\\-\\.]+)\\.([a-zA-Z]{2,5})$",
  "password": "^(?=.*[0-9])(?=.*[!@#\\$%^&*])[a-zA-Z0-9!@#\\$%^&*]{8,}$",
  "url": "^(http|https)://[a-zA-Z0-9-\\.]+\\.[a-zA-Z]{2,5}(/[a-zA-Z0-9-._~:/?#[\\]@!$&'()*+,;=]*)?$"
}

Writing RegEx inside a JSON file can be useful when you want to use many regular expressions.

Instead of littering your JavaScript, Python file, or any other programming files with regular expressions, you can put all of them inside a JSON file. You can then import the file into your programming language file and use the regular expressions entries.

That takes us to how to use a JSON file inside a programming language. Specifically, we’ll look at how to do that in JavaScript.

How to use a JSON RegEx in a JavaScript File

To use the RegEx from your JSON file inside a JavaScript file, you need to import it first.

An ordinary import something from somefile won't work if you want to import JSON into a JS file. This is the correct way to do it:

import someName from 'location/file.json' assert { type: 'json' };

So, I'm going to import the JSON file like this:

import validators from './data.json' assert { type: 'json' };

In this case, I’m importing all the entries from the data.json file and naming them validators. This name can be anything.

The assert { type: 'json'} part ensures that what I’m bringing in is a JSON object.

You can then chain any of the keys from the JSON file to validators to see what any of the entries look like:

import validators from './data.json' assert { type: 'json' };

console.log(validators.nigeriaPhone); // ^(\+?234|0)[789]\d{9}$
console.log(validators.email); // ^([a-zA-Z0-9_\-\.]+)@([a-zA-Z0-9_\-\.]+)\.([a-zA-Z]{2,5})$
console.log(validators.password); // ^(?=.*[0-9])(?=.*[!@#\$%^&*])[a-zA-Z0-9!@#\$%^&*]{8,}$
console.log(validators.url); // ^(http|https)://[a-zA-Z0-9-\.]+\.[a-zA-Z]{2,5}(/[a-zA-Z0-9-._~:/?#[\]@!$&'()*+,;=]*)?$

You can even take this further and test what the regular expressions validate. Here’s how I validated a correct Nigerian phone number:

import validators from './data.json' assert { type: 'json' };

const naijaNUmber = '+2348123456789';

if (naijaNUmber.match(validators.nigeriaPhone)) {
  console.log("That's a valid Naija phone number"); // That's a valid Naija phone number
} else {
  console.log("That's not a valid Naija phone number");
}

And here’s how I used the password key to validate a password that is at least eight characters long and contains at least one number and at least one special character:

import validators from './data.json' assert { type: 'json' };

const pword = 'JohnDoe';

if (pword.match(validators.password)) {
  console.log("That's Valid password");
} else {
  console.log(
    'Your password must be 8 characters long with at least 1 number and 1 special character'
  ); // Your password must be 8 characters long with at least 1 number and 1 special character
}

Unfortunately, in some situations, you get an error if you decide to use the JSON regex directly without creating a regex out of it. Here’s an example with test():

import validators from './data.json' assert { type: 'json' };

const naijaPhone = validators.nigeriaPhone;
console.log(naijaPhone); // Output: ^(\+?234|0)[789]\d{9}$

console.log(naijaPhone.test(83412343433));

/* output: dex.js:4 Uncaught TypeError: naijaPhone.test is not a function
    at index.js:4:24
*/

You can see that the output does not show it as a regular expression – it is not surrounded by forward slashes. This means you have to create a regex out of it with the regex constructor:

import validators from './data.json' assert { type: 'json' };

const naijaPhone = validators.nigeriaPhone;
console.log(naijaPhone); // Output: ^(\+?234|0)[789]\d{9}$

// Create a regex out of validators.nigeriaPhone;
const naijaNumberRegex = new RegExp(naijaPhone);
console.log(naijaNumberRegex); // Output: /^(\+?234|0)[789]\d{9}$/

You can see it is now surrounded by forward slashes – that’s a way to identify and even create regex in JavaScript.

Here's how the chrome developer tools distinguishes the two:

It is with that regex you created that you can test the number:

import validators from './data.json' assert { type: 'json' };

const naijaPhone = validators.nigeriaPhone;
console.log(naijaPhone); // Output: ^(\+?234|0)[789]\d{9}$

// Create a regex out of validators.nigeriaPhone;
const naijaNumberRegex = new RegExp(naijaPhone);
console.log(naijaNumberRegex); // Output: /^(\+?234|0)[789]\d{9}$/

// test it out with the regex
console.log(naijaNumberRegex.test(83412343433)); // output: false
console.log(naijaNumberRegex.test(2348033333333)); // output: true

The reason why it worked with the match() method and didn’t work with test() is this:

• when you use the match() method, you can directly pass the regular expression string that you read from a JSON file as an argument to the method, and JavaScript will automatically create a regular expression from the string.

• when you use the test() method, you need to manually create a regular expression from the regular expression string using the new RegExp() constructor. This is because the test() method expects a regular expression object as its argument, not a string.

Conclusion

You’ve seen that there’s no extra caveat involved in writing regular expressions in a JSON file – you still have to follow the same rules for writing JSON.

And when you import that JSON file into a JavaScript file, you don’t need to do anything extra to make it work, other than creating a regex out of it with the new RegExp() constructor in some cases.

Thank you for reading!

JSON (JavaScript Object Notation) is a popular data interchange format that is widely used in web applications, APIs, and databases. It is a lightweight and human-readable format that is easy to parse and generate.

But when dealing with large and complex JSON data, it can be difficult to read and understand the structure of the data. This is where pretty printing comes in. Pretty printing is the process of formatting the JSON data in a way that makes it easier to read and understand.

In this article, we will explore how to pretty print JSON in Python using built-in and third-party libraries. We will also cover best practices used to pretty print JSON, and also talk about it's use cases.

What Does Pretty Print Mean?

In Python, "pretty print" refers to formatting and presenting data structures such as lists, dictionaries, and tuples in a more readable and organized way.

To pretty print JSON in Python, we can use the built-in json module. This module provides a dumps() function that can serialize Python objects into a JSON formatted string.

By default, this function produces a JSON string without any formatting, but we can use the indent parameter to specify the number of spaces to use for indentation.

Here's an example of how to pretty print JSON in Python:

import json

# Sample JSON data
data = {
    "name": "John",
    "age": 30,
    "city": "New York"
}

# Convert the data to a JSON formatted string with 4 spaces of indentation
json_str = json.dumps(data, indent=4)

# Print the pretty-printed JSON string
print(json_str)

Output:

{
    "name": "John",
    "age": 30,
    "city": "New York"
}

As you can see, the indent parameter is set to 4, which produces a JSON string with each level of nesting indented by four spaces. We can adjust this parameter to control the amount of indentation in the output.

Note that the json.dumps() function can also take other optional parameters, such as sort_keys, which can be used to sort the keys in the JSON output. For more information, see the documentation for the json module.

Best Practices for Pretty Print JSON

Use the json module

The json module is a built-in module in Python, which provides methods for working with JSON data. The json.dumps() method is used to serialize Python objects into a JSON formatted string. The json.dumps() method also has an optional indent parameter that can be used to specify the number of spaces to use for indentation.

Here's an example:

import json

data = {
    "name": "John",
    "age": 30,
    "city": "New York"
}

json_str = json.dumps(data, indent=4)
print(json_str)

Output:

{
    "name": "John",
    "age": 30,
    "city": "New York"
}

Use the pprint module

The pprint module is a built-in module in Python that provides a way to pretty print Python data structures. It also works with JSON data. The pprint.pprint() method is used to pretty print JSON data.

Here's an example:

import json
import pprint

data = {
    "name": "John",
    "age": 30,
    "city": "New York"
}

pprint.pprint(data)

Output:

{'age': 30, 'city': 'New York', 'name': 'John'}

Use third-party libraries

There are many third-party libraries available in Python for pretty printing JSON data, such as simplejson, ujson, and json5. These libraries provide additional features such as faster serialization and deserialization, support for additional data types, and more flexible formatting options.

Here's an example using simplejson:

import simplejson as json

data = {
    "name": "John",
    "age": 30,
    "city": "New York"
}

json_str = json.dumps(data, indent=4, sort_keys=True)
print(json_str)

Output:

{
    "age": 30,
    "city": "New York",
    "name": "John"
}

Pretty Print JSON in Python Use Cases

1. Debugging JSON data: When working with JSON data, it can be challenging to read and understand the structure of the data if it is not well-formatted. Pretty printing JSON data in Python can help us to quickly identify any issues with the data and debug our code more efficiently.
2. Displaying JSON data in user interfaces: If we are building a web application or a mobile app that displays JSON data to the user, pretty printing can enhance the user experience by making the data more readable and presentable.
3. Sharing JSON data with team members: If we are working on a project with other team members and need to share JSON data with them, pretty printing the data can make it easier for them to understand the data and work with it.
4. Logging JSON data: If we are logging JSON data in our Python application, pretty printing the data can make it easier to read and analyze the logs.

Conclusion

Pretty printing JSON in Python is an important skill to have for anyone working with JSON data.

In this tutorial, we learned how to use the json module in Python to pretty print JSON as well as the pprint module. With just a few lines of code, we can generate well-formatted JSON output that is easy to read and navigate.

Let's connect on Twitter and on LinkedIn. You can also subscribe to my YouTube channel.

Happy Coding!

In the world of web applications and services, JSON (JavaScript Object Notation) has become a widely used data format for data exchange between different systems.

In Java, it's common to work with JSON data, and an operation you'll frequently perform is converting a JSON object to a string representation.

The JSONObject.toString() method is a powerful tool that Java developers can use to convert JSON objects to strings.

In this article, we will explore how to use JSONObject.toString() method to convert JSON objects to strings in Java. We will also discuss the benefits of using this method, and provide examples of how to use it in practical applications.

What is the JSONObject.toString() Method?

JSON (JavaScript Object Notation) is a lightweight data interchange format that is widely used in web applications. It is easy to read and write, and it can be used to represent complex data structures in a simple and compact format.

In Java, the JSONObject class provided by the org.json package is commonly used to create and manipulate JSON objects. The JSONObject.toString() method is a useful method provided by this class that converts a JSON object to a string representation.

The JSONObject.toString() method takes no arguments and returns a string representation of the JSON object. This string representation is formatted according to the JSON syntax and can be used to transmit the JSON object over a network, store it in a file, or display it on a web page.

Here is the syntax for the JSONObject.toString() method:

public String toString()

To use the JSONObject.toString() method, you first need to create a JSON object using the JSONObject constructor or by parsing a JSON string using the JSONObject static method JSONObject.parse().

Here is an example that demonstrates how to use the JSONObject.toString() method:

import org.json.JSONObject;

public class JSONToStringExample {
    public static void main(String[] args) {
        JSONObject jsonObject = new JSONObject();
        jsonObject.put("name", "John");
        jsonObject.put("age", 30);
        jsonObject.put("married", true);

        String jsonString = jsonObject.toString();

        System.out.println(jsonString);
    }
}

In the above example, we first create a JSONObject instance and add some key-value pairs to it using the put() method. Then, we call the toString() method on the JSONObject instance to get a string representation of the JSON object. Finally, we print the string to the console.

The output of the above code would be:

{"name":"John","age":30,"married":true}

As you can see, the JSONObject.toString() method has converted the JSON object to a string representation that conforms to the JSON syntax. The string representation includes the key-value pairs and the appropriate punctuation marks (braces, commas, and colons) to represent the structure of the JSON object.

Benefits of using the JSONObject.toString() method

1. Easy Serialization: Using the JSONObject.toString() method makes it easy to serialize a JSONObject to a JSON-formatted string, which can then be transmitted over a network or stored in a file. This string representation can also be easily deserialized back into a JSONObject or other JSON-compatible object in the future.
2. Debugging: When debugging an application that uses JSON data, it can be helpful to log the JSON string representation of the JSONObject instance. This can help to diagnose issues related to JSON data processing.
3. Readability: The JSON format is a lightweight and easy-to-read format for storing and exchanging data. By using the JSONObject.toString() method, you can generate a JSON-formatted string that is easy to read and understand by other developers or systems.
4. Cross-platform compatibility: JSON is a widely-used data format that is supported by many programming languages and platforms. By using the JSONObject.toString() method, you can easily generate a JSON-formatted string that can be consumed by other systems or services regardless of the programming language or platform they are using.
5. Flexibility: The JSONObject.toString() method can be used to generate JSON-formatted strings that can represent complex and nested data structures. This flexibility allows you to represent a wide range of data types and structures in a standardized format that can be easily consumed by other systems or services.

Conclusion

The JSONObject.toString() method is a useful method provided by the org.json package in Java that converts a JSON object to a string representation. This method is essential when transmitting JSON data over a network, storing it in a file, or displaying it on a web page.

By following the syntax and examples outlined in this article, you can use the JSONObject.toString() method to easily convert JSON objects to string representations in your Java programs.

Let's connect on Twitter and on LinkedIn. You can also subscribe to my YouTube channel.

Happy Coding!

But JSON files do not officially support comments. This makes providing additional context or explanation for the data challenging.

This article will show you how to include comments in JSON files and why JSON does not natively support comments.

Why JSON Does Not Support Comments?

According to the JSON specification, a JSON document should only contain data structures like arrays and objects and not include comments. This is because JSON is intended to be a simple, easily parsable data format that can be quickly and efficiently processed.

Comments, while useful for providing additional context or explanation for human readers, can add complexity to the parsing process. This slows down performance and increases the risk of errors.

The primary reason why JSON does not support comments is that its creator, Douglas Crockford, deliberately removed them from the format to prevent misuse and keep it as a pure data-only format.

Crockford observed that some people were using comments to store parsing directives, which could break compatibility between different systems. Hence, the decision to remove comments to maintain the simplicity and consistency of the format across various programming languages and environments.

As a result, the only option for adding comments to a JSON file is to use a workaround, such as using custom elements to store comments.

How to Add Comments Inside JSON

When you add comments in the form //, #, or /* */, which are used in popular programming languages, you will notice the error “Comments are not permitted in JSON”.

Then, how can you add comments to a JSON file?

The only way this can be done is to include comments as data pairs in a JSON file. It is not a commonly used or recommended practice, but technically it’s the best way to add comments to your JSON file.

Create a custom element within your JSON object, such as "_comment", to distinguish them from the rest of the data.

{
    "_comment": "Put your JSON comment here"
    "name": "John Doe",
    "age": 35,
    "city": "New York City",
    "isMarried": true,
    "occupation": "Software Engineer",
}

Note: It's not compulsory that you use underscores. You can decide to use two slash such as “//comment” or any other allowed character. The goal is to make it clear that it’s a comment.

It's important to note that this approach may make the JSON file more complex and harder to parse. But if comments are added as custom elements, they will be received and processed like any other data in JSON on the server-side.

You now know how to add comments to your JSON file technically. But how can you add multiple comments? This is possible, but you should remember that JSON does not allow duplicate object keys. You must include unique letters or numbers in the comment element, ensuring it is valid and distinguishable from other elements in the JSON file.

{
    "_comment1": "This is the basic data",
    "name": "John Doe",
    "age": 35,
    "city": "New York City",
    "_comment2": "Marital information",
    "isMarried": true,
    "wifeName": "Jane Doe"
}

When you have nested JSON objects, you can use similar object keys:

{
    "_comment": "This is the basic data",
    "name": "John Doe",
    "age": 35,
    "city": "New York City",
    "maritalInfo": {
        "_comment": "Marital information",
        "isMarried": true,
        "wifeName": "Jane Doe"
    }
}

Conclusion

You now know how to add comments to your JSON file. But because these comments are also processed and can be accessed, you need to be careful when adding comments to JSON files using custom elements.

Thanks for reading. Have fun coding!

In some situations, when you're working locally or when you upload the data file to a server, we might want to read these JSON data from a file.

We'll learn how to do that in this tutorial.

How to Read a JSON File in JavaScript with the Fetch API

One standard method we can use to read a JSON file (either a local file or one uploaded to a server) is with the Fetch API. It uses the same syntax for both. The only difference would be the URL.

For example, suppose we have a local file within our project's folder named data.json that contains the following JSON data:

<!--./data.JSON-->

{
    "id": 1,
    "title": "Hello World",
    "completed": false
}

We can now read this file in JavaScript using the Fetch API method:

<!--./index.js-->

fetch('./data.json')
    .then((response) => response.json())
    .then((json) => console.log(json));

In the above, we have been able to read a local JSON file. But unfortunately, when we run this in a browser, we might get the following CORS error because our file is not on a server:

To fix this, we will make sure our JSON file is on a local or remote server. If we use the Live server on our IDE, we won't get this error. But when we load our file directly, we will get this error.

As I said earlier, suppose we have this JSON file on a remote server and are trying to read this file in JavaScript. The same syntax will work:

<!--./index.js-->

fetch('https://server.com/data.json')
    .then((response) => response.json())
    .then((json) => console.log(json));

The fetch API is the preferable method to use when we want to read a JSON file either from an external server or local file into our JavaScript file.

How to Read a JSON file in JavaScript with the Import Statement

Another method we can use aside from making an HTTP request is the import statement. This method has a few complications, but we will address them all.

Just like in the previous section, suppose we have our JSON file that holds user data, such as user.json:

<!--./user.JSON-->

{
    "id": 1,
    "name": "John Doe",
    "age": 12
}

We can read this JSON data in JavaScript using the import statement this way:

<!---./index.js-->

import data from './data.json';
console.log(data);

Unfortunately, this will throw an error saying we cannot use the import statement outside a module. This is a standard error when we try to use the import statement in a regular JavaScript file, especially for developers who are new to JavaScript.

To fix this, we can add the type="module" script tag in our HTML file where we referenced the JavaScript file, like this:

<html lang="en">
    // ...
    <body>
        <script type="module" src="./index.js"></script>
    </body>
</html>

When we do this, we'll still get another error:

To fix this error, we need to add the file type of JSON to the import statement, and then we'll be able to read our JSON file in JavaScript:

import data from './data.json' assert { type: 'json' };
console.log(data);

This works perfectly as long as we run our files on a local or remote server. But suppose we run this locally – then we would get the CORS error.

Wrapping Up

In this article, we have learned how to read a JSON file in JavaScript and the possible errors we might encounter when using each method.

It's best to use the fetch API method when you want to make an HTTP request. For example, suppose we are fetching data from a mock JSON file which we will eventually pull from an API.

Still, in a situation where we don't need to use an HTTP request, we could use the import statement. We can use the import statement when we use a library like React, Vue, and so on which has to do with modules. This means we won't need to add the type of module, and also, we won't need to add the type of file.

Neither method requires you to install a package or use a library as they are inbuilt. Choosing which method to use is totally up to you.

But a quick tip that differentiates these methods is that the Fetch API reads a JSON file in JavaScript by sending an HTTP request, while the import statement does not require any HTTP request but rather works like every other import we make.

Embark on a journey of learning! Browse 200+ expert articles on web development. Check out my blog for more captivating content from me.

In this article, you'll learn how to read and parse JSON in Python.

What is JSON?

JSON is short for JavaScript Object Notation. It's a simple syntax for storing data in name-value pairs. Values can be different data types as long as they are valid. Non-acceptable types for JSON include functions, dates, and undefined.

JSON files are stored with the .json extension with a valid JSON structure.

Here's what the structure of a JSON file looks like:

{
  "name": "John",
  "age": 50,
  "is_married": false,
  "profession": null,
  "hobbies": ["traveling", "photography"]
}

You'll often use JSON to send and receive data from a server in web applications.

When the data is received, the program reads and parses the JSON to extract specific data. Different languages have their own methods for doing this. We'll look at how to do these in Python here.

How to Read JSON Files

Let's say the JSON in the code block above is stored in a user.json file. Using the open() inbuilt function in Python, we can read that file and assign the content to a variable. Here's how:

with open('user.json') as user_file:
  file_contents = user_file.read()

print(file_contents)
# {
#   "name": "John",
#   "age": 50,
#   "is_married": false,
#   "profession": null,
#   "hobbies": ["travelling", "photography"]
# }

You pass the file path to the open method which opens the file and assigns the stream data from the file to the user_file variable. Using the read method, you can pass the text contents of the file to the file_contents variable.

I used with at the beginning of the expression so that after reading the contents of the file, Python can close the file.

file_contents now contains a stringified version of the JSON. As a next step, you can now parse the JSON.

How to Parse JSON

Python has in-built modules for various operations. For managing JSON files, Python has the json module.

This module comes with many methods. One of which is the loads() method for parsing JSON strings. Then, you can assign the parsed data to a variable like this:

import json

with open('user.json') as user_file:
  file_contents = user_file.read()

print(file_contents)

parsed_json = json.loads(file_contents)
# {
#   'name': 'John',
#   'age': 50,
#   'is_married': False,
#   'profession': None,
#   'hobbies': ['travelling', 'photography']
# }

Using the loads() method, you can see that the parsed_json variable now has a valid dictionary. From this dictionary, you can access the keys and values in it.

Also notice how null from the JSON is converted to None in python. This is because null is not valid in Python.

How to Use json.load() to Read and Parse JSON Files

The json module also has the load method which you can use to read a file object and parse it at the same time. Using this method, you can update the previous code to this:

import json

with open('user.json') as user_file:
  parsed_json = json.load(user_file)

print(parsed_json)
# {
#   'name': 'John',
#   'age': 50,
#   'is_married': False,
#   'profession': None,
#   'hobbies': ['travelling', 'photography']
# }

Instead of using the read method of the file object and using the loads method of the json module, you can directly use the load method which reads and parses the file object.

Wrapping Up

JSON data is commonly known for its simple structure and is popular (a standard in most cases) for information exchange between servers and clients.

Different languages and technologies can read and parse JSON files in different ways. In this article, we've learned how to read JSON files and parse such files using the read method of file objects, and the loads and load methods of the json module.