But this implies that you have no control over the information relating to your identification, who has access to personally identifiable information (PII), and to what extent.

As a result, Decentralized Identity provides identity-related information that is self-controlled, private, and portable. Decentralized identifiers and attestations serve as the main building pieces.

Thanks to Ceramic's decentralized application databases, application developers can reuse data across applications and automatically make them interoperable.

In this article, you will learn about Decentralized Identity, Decentralized Identifiers, Ceramic network, and how to build a decentralized identity profile with Ethereum on Ceramic Networks.

Here's what we'll cover:

• What is a Decentralized Identity?
• What are Decentralized Identifiers?
• What is Ceramic Data Network?
• Why Ceramic Network?
• How to Build a Decentralized Identity Profile with Next.js
• Prerequisites
• Project Setup and Installation
• Install TailwindCSS in Next.js
• Authenticate Users
• Create/Update User Profile
• How to Test the Application
• Conclusion
• References

What is a Decentralized Identity?

Decentralized Identity is a digital identification concept where people, companies, and items are in charge of their data and can share it selectively without relying on a centralized authority.

This is made possible by using decentralized technologies, such as blockchain. These give people control and ownership over the information associated with their identities rather than having it stored on a central server or managed by a third party.

A decentralized identity is a self-owned, independent identity that enables trusted data exchange.

Blockchain-based digital wallets, such as those used to store and handle cryptocurrencies, serve as a practical illustration of decentralized identification. Users of these wallets control the private keys that provide them access to their money and can distribute their public keys to others to accept payments from them.

Users who manage their private keys can conduct transactions with others without relying on a central authority, such as a bank, and keep custody of their money.

What are Decentralized Identifiers?

Decentralized identifiers (DIDs) are issued, held, and controlled by individuals. Since they are kept on peer-to-peer networks or distributed ledgers (blockchains), they are globally unique, highly available, and cryptographically verifiable.

Decentralized identifiers can be associated with individuals, groups, or governmental entities.

DIDs are a vital component of the developing decentralized identity ecosystem. They are designed to offer a uniform process for developing, maintaining, and exchanging digital identities unaffiliated with any one company or piece of technology.

This implies that a DID can be maintained and controlled by the person or entity to which it belongs and utilized across various systems and applications.

In recent years, smart contract platforms like Ethereum have demonstrated the utility of decentralized applications (dApps) that can be assembled like blocks to create new applications. This is especially evident in tokens that build upon one another, in DeFi protocols that use one another, and so on.

Thanks to Ceramic, data on the internet can now have the same kind of composability. Any data type, including profiles, social connections, blog posts, identities, reputations, and so on., can be included. You will learn more about Ceramic Network in the section below.

What is Ceramic Network?

Ceramic is a public, permissionless, open-source protocol that offers computation, state transitions, and consensus for all data structures on the decentralized web.

With the help of stream processing provided by Ceramic, developers can build apps that are strong, safe, trustless, and censorship-resistant using dynamic information – without using unreliable database servers.

Ceramic stores all content in smart documents, which are append-only IPFS logs. Before being anchored in a blockchain for consensus, each commit is verified by a decentralized identification (DID).

All papers in Ceramic are openly discoverable and can be referenced by other documents or queried by any other network user because the system is entirely peer-to-peer.

Why Ceramic Network?

Data interoperability is one of Ceramic Network's key benefits. This platform features a flexible and modular data schema that enables the decentralized and interoperable sharing and combining of various sorts of data.

Developers now have an easier time creating decentralized identification solutions that can be integrated with other programs and systems.

The infrastructure of Ceramic Network is scalable, fault-tolerant, decentralized, and highly available. This enables developers to create robust decentralized identity systems available to users everywhere.

Ceramic Network also provides a set of developer tools and libraries, making it simple to create decentralized identity apps and services. These tools include SDKs, APIs, developer guides, and an expanding ecosystem of open-source tools and libraries.

Now that you have learnt the theories behind decentralized identity, let's take a practical deep dive and get your hands dirty.

How to Build a Decentralized Identity Profile with Next.js

Prerequisites

To go through this tutorial, you'll need some experience with JavaScript and React.js. Experience with Next.js isn't a requirement, but it's nice to have.

Make sure to have Node.js or npm installed on your computer. If you don't, click here.

Also, it'll be very useful to have a basic understanding of blockchain technology and Web3 concepts.

Project Setup and Installation

Navigate to the terminal and cd into any directory of your choice. Then run the following commands:

mkdir decentralized-identity-project
cd decentralized-identity-project
npx create-next-app@latest .

Accept the following options:

Install the @self.id/react and @self.id/web packages using the code snippet below:

npm install @self.id/web @self.id/react

Next, start the app using the following command:

npm run dev

You should have something similar to what is shown below: the default boilerplate layout for Next.js 13.

Install TailwindCSS in Next.js

In this section, you will set up Tailwind CSS in a Next.js project. Install tailwindcss and its peer dependencies via npm, and then run the init command to generate both tailwind.config.js and postcss.config.js.

npm install -D tailwindcss postcss autoprefixer
npx tailwindcss init -p

Navigate to the tailwind.config.js file, and add the paths to your template files with the following code snippet.

/** @type {import('tailwindcss').Config} */

module.exports = {
  content: [
    "./app/**/*.{js,ts,jsx,tsx}",
    "./pages/**/*.{js,ts,jsx,tsx}",
    "./components/**/*.{js,ts,jsx,tsx}",

    // Or if using `src` directory:
    "./src/**/*.{js,ts,jsx,tsx}",
  ],
  theme: {
    extend: {},
  },
  plugins: [],
}

Delete all the CSS styles inside globals.css . Add the @tailwind directives for each of Tailwind’s layers to your globals.css file.

@tailwind base;
@tailwind components;
@tailwind utilities;

Configure the Provider Component

The Provider component must be placed at the top of the application tree to use the hooks detailed below. You can use it to supply an initial state as well as a specific configuration for the Self.ID clients and queries.

Update the _app.js file under the pages folder with the following code snippet:

// Import the Provider component from the "@self.id/react" library.
import { Provider } from "@self.id/react";

// Import the "globals.css" file from the "@/styles" directory.
import "@/styles/globals.css";

// Define the App component as a default export.
export default function App({ Component, pageProps }) {

  // Render the Provider component, which provides authentication and authorization functionality to the application.
  // Pass a client prop to the Provider component, which configures the Ceramic testnet with the "testnet-clay" value.
  // Render the Component with its props inside the Provider component, which allows the application to access the authentication and authorization context.

  return (
    <Provider client={{ ceramic: "testnet-clay" }}>
      <Component {...pageProps} />
    </Provider>
  );
}

In the code snippet above, we:

• Imported a context provider component and global CSS styles and then defined an App component that wraps the entire application with the context provider.
• Configured the context provider with a Ceramic testnet client, which allows the application to access authentication and authorization functionality.
• Finally, the Component is rendered with its props inside the context provider, allowing the application to access the authentication and authorization context.

Build the Layout

Next, navigate to the index.js file under the pages folder and update it with the following code:

// Import the Head component from the "next/head" module.
import Head from "next/head";

// Import the useViewerConnection and useViewerRecord hooks from the "@self.id/react" library.
import { useViewerConnection, useViewerRecord } from "@self.id/react";

// Import the EthereumAuthProvider component from the "@self.id/web" library.
import { EthereumAuthProvider } from "@self.id/web";

// Import the useState hook from the "react" module.
import { useEffect, useState } from "react";


export default function Home() {

  return (
    <>
      <Head>
        <title>
          Decentralized Identity: Build a Profile with NextJs, Ethereum & Ceramic Network
        </title>
        <meta name="description" content="Generated by create next app" />
        <meta name="viewport" content="width=device-width, initial-scale=1" />
        <link rel="icon" href="/favicon.ico" />
      </Head>
      <main className="min-h-screen bg-gray-200">
        <div className="bg-gray-600 py-4 px-4 sm:px-6 lg:px-8 lg:py-6 shadow-lg text-white">
          <div className="container mx-auto px-6 md:px-0">
            <h1 className="text-2xl font-bold text-white text-center">
              Decentralized Identity: Build a Profile with NextJs, Ethereum & Ceramic Network
            </h1>
          </div>
        </div>

        <div className="flex items-center justify-center pt-20 font-sans overflow-hidden">
          <div className="max-w-md w-full mx-auto">
            <div className="bg-white p-10 rounded-lg shadow-lg">
              <form>
                <div className="mb-6">
                  <label
                    className="block text-gray-700 font-bold mb-2"
                    htmlFor="name"
                  >
                    Name
                  </label>
                  <input
                    className="border border-gray-300 p-2 w-full rounded-lg"
                    type="text"
                    name="name"
                    id="name"
                    placeholder="Your name"
                  />
                </div>
                <div className="mb-6">
                  <label
                    className="block text-gray-700 font-bold mb-2"
                    htmlFor="bio"
                  >
                    Bio
                  </label>
                  <textarea
                    className="border border-gray-300 p-2 w-full rounded-lg"
                    name="bio"
                    id="bio"
                    rows="5"
                    placeholder="Write something about yourself"
                  ></textarea>
                </div>
                <div className="mb-6">
                  <label
                    className="block text-gray-700 font-bold mb-2"
                    htmlFor="username"
                  >
                    Username
                  </label>
                  <input
                    className="border border-gray-300 p-2 w-full rounded-lg"
                    type="text"
                    name="username"
                    id="username"
                    placeholder="Your username"
                  />
                </div>
                <div className="flex items-center justify-between">
                  <button
                    className="bg-blue-500 hover:bg-blue-700 text-white font-bold py-2 px-4 rounded"
                    type="submit"
                  >
                    Update Profile
                  </button>
                  <button
                    className="bg-green-500 hover:bg-green-700 text-white font-bold py-2 px-4 rounded"
                    type="button"
                  >
                    Connect Wallet
                  </button>
                </div>
              </form>
            </div>
          </div>
        </div>
      </main>
    </>
  );
}

To start the application, run the following command and navigate to localhost:3000 on your browser; you should have something similar to what is shown below:

How to Authenticate Users

In this section, you will implement user authentication to allow users to connect their wallets and interact with the application.

Update the index.js with the following code:

//..

export default function Home() {

  // State variables for connection, connect function, and disconnect function
  const [connection, connect, disconnect] = useViewerConnection();


  const [isWindow, setIsWindow] = useState(null);


  // State variable for viewer's basic profile data
  const record = useViewerRecord("basicProfile");

  // Function to create EthereumAuthProvider using window.ethereum provider
  async function createAuthProvider() {
    const addresses = await window.ethereum.request({
      method: "eth_requestAccounts",
    });
    return new EthereumAuthProvider(window.ethereum, addresses[0]);
  }

  // Function to connect to viewer's account using created authProvider
  async function connectAccount() {
    const authProvider = await createAuthProvider();
    await connect(authProvider);
  }

  // Rendered JSX code
  return (
    <>
      {/* ... */}
      <div className="flex items-center justify-between">
        {/* ... */}

        {/* Conditionally render a button to connect/disconnect user */}
        {connection.status === "connected" ? (
          <button
            className="bg-red-500 hover:bg-red-700 text-white font-bold py-2 px-4 rounded"
            type="button"
            onClick={() => disconnect()}
          >
            Disconnect
          </button>
        ) : isWindow && "ethereum" in window ? (
          <button
            className="bg-green-500 hover:bg-green-700 text-white font-bold py-2 px-4 rounded"
            type="button"
            disabled={connection.status === "connecting" || !record}
            onClick={() => {
              connectAccount();
            }}
          >
            Connect Wallet
          </button>
        ) : (
          <p className="text-red-500 text-sm italic mt-2 text-center w-full">
            An injected Ethereum provider such as{" "}
            <a href="https://metamask.io/">MetaMask</a> is needed to
            authenticate.
          </p>
        )}
      </div>
    </>
  )
}

In the code snippet above,

• The useViewerConnection hook is used to set up a state variable for the user's connection status, connect and disconnect.
• isWindow to set the initial state of the the window to avoid React hydration error
• The useViewerRecord hook is used to retrieve the user's basic profile data.
• The createAuthProvider function creates an EthereumAuthProvider object using the window.ethereum provider.
• The connectAccount function calls createAuthProvider and connects to the user's account using connect(authProvider).
• The JSX code conditionally renders a button based on the user's connection status and the availability of an ethereum provider in the window object.
• If the user is already connected, the button will enable them to disconnect. If the user is not yet connected and an ethereum provider is available, the button will enable them to connect. But if the user is not connected and no ethereum provider is available, a message will be displayed to inform the user that an injected Ethereum provider like MetaMask is required to authenticate.

Testing out the authentication functionality, you should have something similar to what is shown below:

How to Create or Update a User Profile

In the previous section, you learned how to successfully authenticate users. Next, you will implement functionality to create and update an authenticated user with the following code snippet:

pages/index.js

//...


export default function Home() {
// Use the useState hook to create state variables and functions to update them
  const [name, setName] = useState("");
  const [bio, setBio] = useState("");
  const [username, setUsername] = useState("");

  //...

// Define an asynchronous function called updateProfile to update the profile information
  async function updateProfile() {
    // If any of the required fields are empty, return early and do not update
     if (!name || !bio || !username) {
       return;
     }

     // Use the merge method to update the record with the new information
     await record.merge({
       name,
       bio,
       username,
     });
   }

  // Render the component's UI
  return (
    <>

    {/* ... */}

    <div className="flex items-center justify-center pt-20 font-sans overflow-hidden">
          <div className="max-w-md w-full mx-auto">
            <div className="bg-white p-10 rounded-lg shadow-lg">
              <form>
               {/* ... */}
              </form>
            </div>
            {connection.status === "connected" && record && record.content ? (
              <div className="flex flex-col items-center mt-8">
                <h2 className="text-3xl font-bold mb-6 text-gray-900">
                  Profile Information
                </h2>
                <div className="w-full max-w-md bg-white p-8 rounded-lg shadow-lg">
                  <p className="mb-4">
                    <span className="font-bold text-gray-700 mr-2 text-lg">
                      Name:
                    </span>{" "}
                    <span id="nameOutput" className="text-lg">
                      {record.content.name || "No name set"}
                    </span>
                  </p>

                  <p className="mb-4">
                    <span className="font-bold text-gray-700 mr-2 text-lg">
                      Bio:
                    </span>{" "}
                    <span id="bioOutput" className="text-lg">
                      {record.content.bio || "No bio set"}
                    </span>
                  </p>
                  <p>
                    <span className="font-bold text-gray-700 mr-2 text-lg">
                      Username:
                    </span>{" "}
                    <span id="usernameOutput" className="text-lg">
                      {record.content.username || "No username set"}
                    </span>
                  </p>
                </div>
              </div>
            ) : (
              <div className="mt-8">
                <div className="bg-white p-8 rounded-lg shadow-lg">
                  <p>No profile found.</p>
                </div>
              </div>
            )}

          </div>
        </div>
    </>
  ) 
}

In the code above,

• The component uses the useState hook to manage the state of three variables: name, bio, and username.
• There's an async function called updateProfile that is responsible for merging the current state of the variables into a record.
• If any of the variables is empty, the updateProfile function returns without updating the record.
• There are three conditional statements that render a different UI based on whether a record is found or not.
• The first conditional statement checks whether the record is still loading, and if it is, it displays a Loading... message.
• The second conditional statement checks whether there's no record content and the connection status is connected. If this is true, it displays a No profile found. message.

The third conditional statement checks whether the record content exists. If it does, it displays the profile information, which includes the user's name, bio, and username.

You are almost there. In the form tag, update the name, bio and username input field with the following code:

<div className="mb-6">
  <label
    className="block text-gray-700 font-bold mb-2"
    htmlFor="name"
  >
    Name
  </label>
  <input
    //...
    onChange={(e) => {
      setName(e.target.value);
    }}
  />
</div>
<div className="mb-6">
  <label
    className="block text-gray-700 font-bold mb-2"
    htmlFor="bio"
  >
    Bio
  </label>
  <textarea
    //...
    onChange={(e) => {
      setBio(e.target.value);
    }}
  ></textarea>
</div>
<div className="mb-6">
  <label
    className="block text-gray-700 font-bold mb-2"
    htmlFor="username"
  >
    Username
  </label>
  <input
    //...
    onChange={(e) => {
      setUsername(e.target.value);
    }}
  />
</div>

In the code snippet above, setName, setBio, and setUsername are functions provided by the useState hook that update the state of name, bio, or username.

Next, the Update Profile button.

//...

 <button
     className="bg-blue-500 hover:bg-blue-700 text-white font-bold py-2 px-4 rounded"
     type="submit"
     disabled={!record.isMutable || record.isMutating}
     onClick={() => updateProfile()}
 >
    {record.isMutating ? "Updating..." : "Update Profile"}
</button>

//..

In the code snippet above, the button is disabled when the record is not mutable or is currently mutating.

When the button is clicked, it calls the updateProfile function, which is responsible for updating the user's profile information. If the record mutates, the button will display Updating.... Otherwise, it will display Update Profile.

You can test out the application similar to what is shown below.

Kindly find the complete code on GitHub repository here.

Conclusion

In this post, you learn about Decentralized Identity, Decentralized Identifiers, Ceramic networks, why Ceramic network is useful, and how to build a decentralized identity profile with Ethereum on Ceramic Networks.

References

• Ceramic Network
• Ceramic Documentation
• Decentralized Identity - Ethereum

I'd love to connect with you at Twitter | LinkedIn | GitHub | Portfolio

See you in my next article. Take care!

If you’re reading this then you are a participant in the modern web. The web we are experiencing today is much different than what it was just 10 years ago. How has the web evolved, and more importantly – where is it going next? Also, why do any of these things matter?

If history has taught us anything, these changes will matter a lot.

In this article, I will lay out how the web has evolved, where's it going next, and why this matters.

Think about how the internet affects your life on a daily basis. Consider how society has changed as a result of the internet. Social media platforms. Mobile apps. And now the internet is going through another paradigm shift as we speak.

The Evolution of the Web

The web has evolved a lot over the years, and the applications of it today are almost unrecognizable from its most early days. The evolution of the web is often partitioned into three separate stages: Web 1.0, Web 2.0, and Web 3.0.

What is Web 1.0?

Web 1.0 was the first iteration of the web. Most participants were consumers of content, and the creators were typically developers who build websites that contained information served up mainly in text or image format. Web 1.0 lasted approximately from 1991 to 2004.

Web 1.0 consisted of sites serving static content instead of dynamic HTML. Data and content were served from a static file system rather than a database, and sites didn't have much interactivity at all.

You can think of Web 1.0 as the read-only web.

What is Web 2.0?

Most of us have primarily experienced the web in its current form, commonly referred to as web2. You can think of web2 as the interactive and social web.

In the web2 world, you don’t have to be a developer to participate in the creation process. Many apps are built in a way that easily allows anyone to be a creator.

If you want to craft a thought and share it with the world, you can. If you want to upload a video and allow millions of people to see it, interact with it, and comment on it, you can do that too.

Web2 is simple, really, and because of its simplicity more and more people around the world are becoming creators.

The web in its current form is really great in many ways, but there are some areas where we can do a lot better.

Web 2.0 Monetization and Security

In the web2 world, many popular apps are following a common pattern in their life cycles. Think of some of the apps that you use on a daily basis, and how the following examples might apply to them.

Monetization of Apps

Imagine the early days of popular applications like Instagram, Twitter, LinkedIn, or YouTube and how different they are today. The process usually goes something like this:

1. Company launches an app
2. It onboards as many users as possible
3. Then it monetizes its user base

When a developer or company launches a popular app, the user experience is often very slick as the app continues rising in popularity. This is the reason they are able to gain traction quickly in the first place.

At first, many software companies do not worry about monetization. They strictly focus on growth and on locking in new users – but eventually they have to start turning a profit.

They also need to consider the role of outside investors. Often the constraints of taking on things like venture capital negatively affect the life cycle, and eventually the user experience, of many applications that we use today.

If a company building an application takes in venture capital, its investors often expect a return on investment in the order of magnitude of tens or hundreds of what they paid in.

This means that, instead of going for some sustainable model of growth that they can sustain in a somewhat organic manner, the company is often pushed towards two paths: advertisements or selling personal data.

For many web2 companies like Google, Facebook, Twitter, and others, more data leads to more personalized ads. This leads to more clicks and ultimately more ad revenue. The exploitation and centralization of user data is core to how the web as we know and use it today is engineered to function.

Security and privacy

Web2 applications repeatedly experience data breaches. There are even websites dedicated to keeping up with these breaches and telling you when your data has been compromised.

In web2, you don’t have any control over your data or how it is stored. In fact, companies often track and save user data without their users' consent. All of this data is then owned and controlled by the companies in charge of these platforms.

Users who live in countries where they have to worry about the negative consequences of free speech are also at risk.

Governments will often shut down servers or seize bank accounts if they believe a person is voicing an opinion that goes against their propaganda. With centralized servers, it is easy for governments to intervene, control, or shut down applications as they see fit.

Because banks are also digital and under centralized control, governments often intervene there as well. They can shut down access to bank accounts or limit access to funds during times of volatility, extreme inflation, or other political unrest.

Web3 aims to solve many of these shortcomings by fundamentally rethinking how we architect and interact with applications from the ground up.

What is Web 3.0?

There are a few fundamental differences between web2 and web3, but decentralization is at its core.

Web3 enhances the internet as we know it today with a few other added characteristics. web3 is:

• Verifiable
• Trustless
• Self-governing
• Permissionless
• Distributed and robust
• Stateful
• Native built-in payments

In web3, developers don't usually build and deploy applications that run on a single server or that store their data in a single database (usually hosted on and managed by a single cloud provider).

Instead, web3 applications either run on blockchains, decentralized networks of many peer to peer nodes (servers), or a combination of the two that forms a cryptoeconomic protocol. These apps are often referred to as dapps (decentralized apps), and you will see that term used often in the web3 space.

To achieve a stable and secure decentralized network, network participants (developers) are incentivized and compete to provide the highest quality services to anyone using the service.

When you hear about web3, you'll notice that cryptocurrency is often part of the conversation. This is because cryptocurrency plays a big role in many of these protocols. It provides a financial incentive (tokens) for anyone who wants to participate in creating, governing, contributing to, or improving one of the projects themselves.

These protocols may often offer a variety of different services like compute, storage, bandwidth, identity, hosting, and other web services commonly provided by cloud providers in the past.

People can make a living by participating in the protocol in various ways, in both technical and non-technical levels.

Consumers of the service usually pay to use the protocol, similarly to how they would pay a cloud provider like AWS today. Except in web3, the money goes directly to the network participants.

In this, like in many forms of decentralization, you'll see that unnecessary and often inefficient intermediaries are cut out.

Many web infrastructure protocols like Filecoin, Livepeer, Arweave, and The Graph (which is what I work with at Edge & Node) have issued utility tokens that govern how the protocol functions. These tokens also reward participants at many levels of the network. Even native blockchain protocols like Ethereum operate in this manner.

Native payments

Tokens also introduce a native payment layer that is completely borderless and frictionless. Companies like Stripe and Paypal have created billions of dollars of value in enabling electronic payments.

These systems are overly complex and still do not enable true international interoperability between participants. They also require you to hand over your sensitive information and personal data in order to use them.

Crypto wallets like MetaMask and Torus enable you to integrate easy, anonymous, and secure international payments and transactions into web3 applications.

Networks like Solana offer several hundred digit millisecond latency and transaction costs of a small fraction of a penny. Unlike the current financial system, users do not have to go through the traditional numerous, friction-filled steps to interact with and participate in the network. All they need to do is download or install a wallet, and they can start sending and receiving payments without any gatekeeping.

A new way of building companies

Tokens also brings about the idea of tokenization and the realization of a token economy.

Take, for example, the current state of building a software company. Someone comes up with an idea, but in order to start building they need money in order to support themselves.

To get the money, they take on venture capital and give away a percentage of the company. This investment immediately introduces mis-aligned incentives that will, in the long run, not align well with building out the best user experience.

Also, if the company ever does become successful, it will take a very long time for anyone involved to realize any of the value, often leading to years of work without any real return on investment.

Imagine, instead, that a new and exciting project is announced that solves a real problem. Anyone can participate in building it or investing in it from day one. The company announces the release of x number of tokens, and give 10% to the early builders, put 10% for sale to the public, and set the rest aside for future payment of contributors and funding of the project.

Stakeholders can use their tokens to vote on changes to the future of the project, and the people who helped build the project can sell some of their holdings to make money after the tokens have been released.

People who believe in the project can buy and hold ownership, and people who think the project is headed in the wrong direction can signal this by selling their stake.

Because blockchain data is all completely public and open, purchasers have complete transparency over what is happening. This is in contrast to buying equity in private or centralized businesses where many things are often cloaked in secrecy.

This is already happening in the web3 space.

One example is the app Radicle (a decentralized GitHub alternative) which allows stakeholders to participate in the governance of their project. Gitcoin is another that allows developers to get paid in cryptocurrency for jumping in and working on Open Source issues. Yearn allows stakeholders to participate in decision making and voting on proposals. Uniswap, SuperRare, The Graph, Audius, and countless other protocols and projects have issued tokens as a way to enable ownership, participation, and governance.

DAOs (Decentralized Autonomous Organizations), which offer an alternative way to build what we traditionally thought of as a company, are gaining tremendous momentum and investment from both traditional developers and venture capital firms.

These types of organizations are tokenized and turn the idea of organizational structure on its head, offering real, liquid, and equitable ownership to larger portions of stakeholders and aligning incentives in new and interesting ways.

For example, Friends with Benefits is a DAO of web3 builders and artists, is about a year old, has a market cap of around $125 million as of this writing, and recently received a $10 million round of investment from a16z.

DAOs could encompass an entire post in and of themselves, but for now I'll just say that I think that they are the future of building products and (what we in the past thought of as) companies. Here is a good post outlining the current DAO landscape.

How Identity Works in Web3

In web3, Identity also works much differently than what we are used to today. Most of the time in web3 apps, identities will be tied to the wallet address of the user interacting with the application.

Unlike web2 authentication methods like OAuth or email + password (that almost always require users to hand over sensitive and personal information), wallet addresses are completely anonymous unless the user decides to tie their own identity to it publicly.

If the user chooses to use the same wallet across multiple dapps, their identity is also seamlessly transferable across apps, which lets them build up their reputation over time.

Protocols and tools like Ceramic and IDX already allow developers to build self-sovereign identity into their applications to replace traditional authentication and identity layers. The Ethereum foundation also has a working RFP for defining a specification for "Sign in with Ethereum" which would help provide a more streamlined and documented way to do this going forward. This is also a good thread that outlines some of the ways that this would enhance traditional authentication flows.

How to Build on Web3

I'm a developer who recently transitioned into the web3 space from a traditional development background. So I wanted to start building to get a sense of what the development experience felt like. And I wanted to get an understanding of the types of apps that we can build today.

I dove right in and decided to document some of the things I was doing in a couple of blog posts.

How to Get Into Ethereum, Crypto, and Web3 as a Developer – This is an introduction to the space in general, coming from a developer, for developers looking to break into the industry.

The Complete Guide to Full Stack Ethereum Development – This is a tutorial that teaches you how to build out your first dapp.

The Complete Guide to Full Stack Solana Development with React, Anchor, Rust, and Phantom - This guide dives into Solana to show you how to build a full stack dapp.

If you are interested in learning more about web3 in general, you can check out these posts:

The New Creator Economy - DAOs, Community Ownership, and Cryptoeconomics

The Value Chain of the Open Metaverse

The Rise of Micro-Economies

When you think about developing a decentralized application, a blockchain like Ethereum probably comes to mind.

Blockchain is fantastic for managing state, automating processes via Smart Contracts, and exchanging economic value.

You can follow this tutorial to learn blockchain by building one from scratch yourself if you want to learn more.

But where do you store your application's content? Images? Videos? The application's website front-end composed of all the HTML, CSS, and JS files? Are your application and your users' content loaded from a centralized AWS server?

Storing the content on the blockchain would be expensive and inefficient.

Your blockchain application needs decentralized storage!

In this tutorial, I will introduce you to the InterPlanetary File System, or IPFS. You will learn:

1. How to store and retrieve content from a decentralized storage
2. How to run your IPFS node
3. All about the low-level internals of the IPFS protocol
4. And we'll read a Wikipedia website stored on IPFS

Ready? Let's go.

Table of Contents

• What is the IPFS?
• How to setup an IPFS node
• How to store and retrieve IPFS content using the CLI and HTTP
• What is CID – the IPFS content-based identifier
• How to reverse engineer the IPFS datastore
• How to connect an IPFS node to a decentralized network
• How to exchange data using the peer-to-peer Bitswap protocol
• How to persist content from the peer-to-peer network

What is the IPFS?

The InterPlanetary File System, or IPFS for short, is a peer-to-peer hypermedia protocol designed to make the web faster, safer, and more open.

IPFS is a protocol for storing and sharing content. As in the blockchain world, every user is running its node (server). The nodes can communicate between each other and exchange files.

What is unique about IPFS?

First of all, the IPFS is decentralized because it loads the content from thousands of peers instead of one centralized server. Every piece of data is cryptographically hashed, resulting in a safe, unique content identifier: CID.

Store your website in IPFS to avoid censorship and a single point of failure. Your personal IPFS node goes offline? Don't worry, the website will still load from other nodes across the globe serving it.

For example, suppose your government bans Wikipedia. In that case, you can still access a decentralized version of Wikipedia indexed on April 17th by loading it from the IPFS peer-to-peer network persisted under CID:

"QmT5NvUtoM5nWFfrQdVrFtvGfKFmG7AHE8P34isapyhCxX"

Second, the integrity of IPFS content can be cryptographically verified.

And finally, the IPFS content is de-duplicated. If you tried storing two identical 1MB files in the same IPFS node, they would be stored only once, eliminating the duplication, because their hash would produce an identical CID.

How to Setup an IPFS Node

Install IPFS

Open the official IPFS docs installation page and follow the instructions depending on your operating system (Windows, macOS, Linux). I will be documenting the Ubuntu installation process below.

I prefer compiling the ipfs/go-ipfs repository from scratch to debug the code when needed, and let's be honest: GoLang rocks.

Compile the codebase in Go

Clone the repository and run the install script in the Makefile.

git clone https://github.com/ipfs/go-ipfs.git
cd go-ipfs
git checkout v0.8.0-rc2
make install

Or download pre-compiled IPFS:

sudo snap install ipfs

Validate the installation

Let's be honest. Go is amazing and compiling the codebase yourself is bad-ass and decentralized. The resulted binary will be created in your $GOPATH.

which ipfs
> /home/web3coach/go/bin/ipfs

ipfs version
> ipfs version 0.8.0-rc2

Initialize a new node

Run ipfs init to create your new node. By default, it will create a folder and store all the data in ~/.ipfs You can tweak this by configuring the IPFS_PATH ENV variable.

IPFS_PATH=/home/web3coach/.ipfs_tutorial ipfs init

> generating ED25519 keypair...done
> peer identity: 12D3Koo...dNs
> initializing IPFS node at /home/web3coach/.ipfs_tutorial

Your node is now fully initialized, awaiting your content.

How to Use IPFS

Add content

IPFS can handle all kinds of different data types – from simple strings to images, videos, and websites.

Start by storing a short message hello IPFS world by Web3Coach:

echo "hello IPFS world by Web3Coach. BTW: Ethereum FTW" | ipfs add

The content is now stored and indexed by a cryptographic hash function returning its unique content identifier (CID):

> added QmRBkKi1PnthqaBaiZnXML6fH6PNqCFdpcBxGYXoUQfp6z
> 49 B / 49 B [========] 100%

Your IPFS node will generate the same CID on your local file system as in this tutorial. That's because IPFS hashes the content and returns its unique fingerprint, and as we know, a secure hash function will always return the same output given the same input.

Pin content

When you add content, you add it ONLY to your local node. The content does NOT automatically replicate across the entire network – this is a common confusion between IPFS users and developers.

When you store content via the add command, IPFS will also execute the pin command by default:

ipfs pin add QmRBkKi1PnthqaBaiZnXML6fH6PNqCFdpcBxGYXoUQfp6z

To replicate content, you must take your node online, join the p2p network, and pin the specific CID from another node. You will learn how to do this further in the tutorial and find out what happens in the background.

Read content

Copy-paste the CID to IPFS cat command to read it from disk:

ipfs cat QmRBkKi1PnthqaBaiZnXML6fH6PNqCFdpcBxGYXoUQfp6z
> hello IPFS world by Web3Coach. BTW: Ethereum FTW

The add , pin and cat commands are the most significant IPFS functions, and you just learned them. Congrats, well done!

How IPFS Content Addressing Works

What is QmRBkKi1PnthqaBaiZnXML6fH6PNqCFdpcBxGYXoUQfp6z?

It’s a self-describing content-based identifier.

What does "self-describing" actually mean? It means that by splitting the string following the IPFS specification, you will know everything you need to know about the data it indexes.

• what CID version it is
• how to read the CID string (base32? base58? hex?)
• how data is encoded
• what hash function fingerprinted the data
• the hash function's length

The IPFS team built a convenient website for analyzing a CID:

By parsing the QmRBkKi1P…p6z CID, you discover:

• the CID follows version 0 spec because it starts with Qm
• the QmRBkKi1P…p6z string is encoded using base58btc
• the data "hello IPFS world by Web3Coach. BTW: Ethereum FTW" were encoded as DAG Protobuf under a codec 0x70 before being stored on disk
• the hash code 0x12 signals the data fingerprint obtained using the sha256 hash function, producing a unique 32 byte long digest

"Slightly more complicated" than a simple auto-increment INT in a MySQL table... but extraordinarily potent and future proof. Let me explain.

CID Versions

There are currently two CID versions: v0 and v1.

The CID v0 is not flexible and limited to:

• start with characters "Qm"
• where the CID string is encoded using base58btc
• the data is encoded with dag-pb by default
• can be converted to CID version 1, but not the other way around

The CID v1 leverages several prefixes for maximum interoperability:

CID v1 = Multibase + Multicodec + Multihash

In another words, parsing the binary into a CID v1 string follows this spec:

<base><codec><hash-function><hash-length><hash-digest>

Multihash

To be future-proof and enable different hashing algorithms, IPFS created the following standard:

CODE : SIZE : DIGEST

type DecodedMultihash struct {
   Code   uint64 // 0x12
   Name   string // sha2-256
   Length int    // 32 bytes
   Digest []byte // Digest holds the raw multihash bytes
}

Multihash has many advantages. When computers are more powerful in 5 years, you could use a stronger hash function like sha3-512 as long as you configure the corresponding 0x13 code as the Multihash in the CID prefix – the protocol will be ready to handle it.

Multicodec

The Code attribute tells you how data is encoded before being stored on disk, so you know how to decode it back when the user wants to read it. It could be anything CBOR, Protobuf, JSON…

IPFS maintains a public list of all possible codecs. The most common codecs are:

raw       | ipld      | 0x55 | raw binary
dag-pb    | ipld      | 0x70 | MerkleDAG protobuf
dag-cbor  | ipld      | 0x71 | MerkleDAG cbor

// but you could also encode Ethereum blocks on IPFS!
eth-block | ipld      | 0x90 | Ethereum Block (RLP)

Multibase

The problem with CID v0 and the base58btc encoding is the lack of interoperability between environments. A multibase prefix adds support for different encodings like base32 to achieve DNS-friendly names.

A table with all Multibase encodings:

encoding  | code
base32    | b
base58btc | z
base64    | m

You spot a Multibase encoding based on the first character:

QmRBkKi1PnthqaBaiZnXML6fH6PNqCFdpcBxGYXoUQfp6z

• is CID v0
• the CID string is encoded with base58btc

bafybeibkjmxftowv4lki46nad4arescoqc7kdzfnjkqux257i4jonk44w4

• CID v1
• the CID string is encoded with base32

Both CID versions can retrieve the same content because after you strip the encoding, it's the Multihash that indexes the blocks on the datastore level. In contrast, Multibase is only used to pass the CID correctly in different environments (CLI, URL, DNS).

ipfs cat QmRBkKi1PnthqaBaiZnXML6fH6PNqCFdpcBxGYXoUQfp6z
> hello IPFS world by Web3Coach. BTW: Ethereum FTW

// equivalent to
ipfs cat bafybeibkjmxftowv4lki46nad4arescoqc7kdzfnjkqux257i4jonk44w4
> hello IPFS world by Web3Coach. BTW: Ethereum FTW

Phew. Things got "slightly complex" very quickly.

Speaking of complicated topics, IPFS is powerful because it doesn't treat content as just "data" but as data structures – specifically InterPlanetary Linked Data structure: IPLD. In short, you can implement any file-system, database, or structure on top of IPLD.

For example, you can store all Ethereum blocks on IPFS as long as you set eth-block and eth-tx codecs and register an appropriate Decoder when working with the IPLD graph.

Let's dig into it and explore the default IPLD structure with the DAG Protobuf codec.

How IPFS Stores Content on the File-system

“The ipfs add command will create a Merkle DAG out of the data following the UnixFS data format. Your content is broken down into blocks using a Chunker, and then arranged in a tree-like structure using 'link nodes' to tie them together. The returned CID is the hash of the root node in the DAG.”

Confused?

Rolling back to basics.

Let's explore the node’s data directory

At the beginning of this tutorial, when you initialized your IPFS node with the ipfs init command, you generated the following directory:

export IPFS_PATH=/home/web3coach/.ipfs_tutorial
cd $IPFS_PATH
~/.ipfs_tutorial  tree

.
├── blocks
│   ├── 6Y
│   │   └── CIQA4XCGRCRTCCHV7XSGAZPZJOAOHLPOI6IQR3H6YQ.data
├── config
├── datastore
│   ├── 000002.ldb
│   ├── 000003.log
│   ├── CURRENT
│   ├── CURRENT.bak
│   ├── LOCK
│   ├── LOG
│   └── MANIFEST-000004
├── datastore_spec
├── keystore
└── version

From a very high-level point of view:

• blocks — IPFS stores all the chunked data here, although the go-ipfs flexible interfaces allow you to swap the storage implementation for a different database
• config — Node’s settings (file-system, identity, specs, networking)
• datastore — Indexing and other logic

Don't take my word for it. Create a new file with the following content on your local file system and then add it to IPFS:

hello IPFS world by Web3Coach. Testing DAGs
hello IPFS world by Web3Coach. Testing DAGs
hello IPFS world by Web3Coach. Testing DAGs

ls -la hello_world.txt
> 131 bytes hello_world.txt

ipfs add hello_world.txt
> added QmNtQtxeavDXTjCiWAAaxnhKK3LBYfFfpXUXjdMDYXwKtH

Reverse engineering the go-ipfs codebase, this is what is happening behind the scenes:

IPFS UnixFS adding a new file and converting it to a block

Validate the persistence process by inspecting the blocks directory. You will find the content was written under the Multihash Datastore Key using DAG Protobuf encoding (131 bytes + Protobuf extra encoding).

ls -la blocks/PV/
> 142 CIQAQIXXW2OAQSKZ6AQ2SDEYRZXWPDZNJUAFR3YORYN75I5CQ3LHPVQ.data

vim blocks/PV/CIQA...
<8b>^A^H^B^R<83>^Ahello IPFS world by Web3Coach. Testing DAGs
hello IPFS world by Web3Coach. Testing DAGs
hello IPFS world by Web3Coach. Testing DAGs^X<83>^A

To interact with your raw content, use the ipfs object command.

ipfs object get QmNtQtxeavDXTjCiWAAaxnhKK3LBYfFfpXUXjdMDYXwKtH | jq

{
  "Links": [],
  "Data": "\b\u0002\u0012�\u0001hello IPFS world by Web3Coach. Testing DAGs\nhello IPFS world by Web3Coach. Testing DAGs\nhello IPFS world by Web3Coach. Testing DAGs\u0018�\u0001"
}

• Because the content is only 131 bytes, it fits to one DAG Node
• The Dag Node is persisted as one Block on disk
• The DAG Node has zero links to other Nodes

Time to experiment.

Add the same file again, but configure the Chunker to 64 bytes (or use a bigger file, but a smaller Chunker will demonstrate the concept better).

ipfs add --chunker=size-64 hello_world.txt

> 131 bytes QmTwtTQgrTaait2qWXYjTsEZiF4sT7CD4U87VqQ27Wnsn8

You get a new CID!

IPFS split the content into 4 DAG Nodes and wrote 4 Blocks with data encoded in DAG Protobuf format to disk.

IPFS splits a file into multiple chunks (DAG Nodes + Blocks)

ipfs object get QmTwtTQgrTaait2qWXYjTsEZiF4sT7CD4U87VqQ27Wnsn8 | jq

{
  "Links": [
    {
      "Name": "",
      "Hash": "QmTbsuUYzy3nT6NApb5t7VUq3iQKZXrJJJY2j1miMVgaJU",
      "Size": 72
    },
    {
      "Name": "",
      "Hash": "QmNy9iFF8uU1Cn7trxpSgqxMsjmi4zQ7xgyEgsWff5rnfH",
      "Size": 72
    },
    {
      "Name": "",
      "Hash": "QmdEitCfYgBNxLhxTNvdLaDmTypSAWkGErjw33VZxUbWK3",
      "Size": 11
    }
  ],
  "Data": "\b\u0002\u0018�\u0001 @ @ \u0003"
}

The ultimate test is to retrieve each DAG Node's data and verify the text is split into three chunks:

DAG Protobuf Node 1:

ipfs object get QmTbsuUYzy3nT6NApb5t7VUq3iQKZXrJJJY2j1miMVgaJU | jq

{
  "Links": [],
  "Data": "\b\u0002\u0012@hello IPFS world by Web3Coach. Testing DAGs\nhello IPFS world by \u0018@"
}

DAG Protobuf Node 2:

ipfs object get QmNy9iFF8uU1Cn7trxpSgqxMsjmi4zQ7xgyEgsWff5rnfH | jq

{
  "Links": [],
  "Data": "\b\u0002\u0012@Web3Coach. Testing DAGs\nhello IPFS world by Web3Coach. Testing D\u0018@"
}

DAG Protobuf Node 3:

ipfs object get QmdEitCfYgBNxLhxTNvdLaDmTypSAWkGErjw33VZxUbWK3 | jq

{
  "Links": [],
  "Data": "\b\u0002\u0012\u0003AGs\u0018\u0003"
}

What’s the benefit of splitting the content into multiple chunks and use content-addressing and CIDs?

• Data deduplication
• Decentralization

Next time you want to store a file that would share part of the content with another file, IPFS wouldn't store a duplicate block! It would instead link to an already existing DAG Node and only store the new, unique chunks.

Converting content to a directed acyclic graph with many nodes also helps to load the content in parallel. For example, a blog post, image, entire Wikipedia website can load from multiple IPFS peer nodes. Your node then verifies the integrity of received blocks by re-hashing all the data content and asserting the constructed CID.

You've now learned the bread and butter of IPFS – excellent progress!

One more critical component left: Networking.

How to Connect an IPFS Node to the p2p Network

Every node has its config file generated during the ipfs init execution.

Open it.

vim $IPFS_PATH/config

Other settings aside, you find your node’s Identity (PeerID + Private Key):

"Identity": {
    "PeerID": "12D3KooWCBmDtsvFwDHEr...",
    "PrivKey": "CAESQCj..."
  },

And a list of Bootstrap addresses:

"Bootstrap": [
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmcZf59b...gU1ZjYZcYW3dwt",
    "/ip4/104.131.131.82/tcp/4001/p2p/QmaCpDMG...UtfsmvsqQLuvuJ",
    "/ip4/104.131.131.82/udp/4001/quic/p2p/Qma...UtfsmvsqQLuvuJ",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmNnooD5...BMjTezGAJN",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmQCU2Ec...J16u19uLTa",
    "/dnsaddr/bootstrap.libp2p.io/p2p/QmbLHAnM...Ucqanj75Nb"
  ],

You connect to other peers in the IPFS network by running the ipfs daemon command. Your node will first establish a p2p connection with Protocol Labs (company behind IPFS) bootstrap nodes, and through these bootstrap nodes, you will further find hundreds of other peers.

ipfs daemon 

> Initializing daemon...

Swarm listening on /ip4/127.0.0.1/tcp/4001
Swarm listening on /ip4/127.0.0.1/udp/4001/quic
Swarm listening on /ip4/172.17.0.1/tcp/4001
Swarm listening on /ip4/172.17.0.1/udp/4001/quic
Swarm listening on /ip4/192.168.0.142/tcp/4001
Swarm listening on /ip4/192.168.0.142/udp/4001/quic
Swarm listening on /ip6/::1/tcp/4001
Swarm listening on /ip6/::1/udp/4001/quic
Swarm listening on /p2p-circuit
Swarm announcing /ip4/127.0.0.1/tcp/4001
Swarm announcing /ip4/127.0.0.1/udp/4001/quic
Swarm announcing /ip4/192.168.0.142/tcp/4001
Swarm announcing /ip4/192.168.0.142/udp/4001/quic
Swarm announcing /ip4/88.212.40.160/udp/4001/quic
Swarm announcing /ip6/::1/tcp/4001
Swarm announcing /ip6/::1/udp/4001/quic

API server listening on /ip4/127.0.0.1/tcp/5001
WebUI: http://127.0.0.1:5001/webui

Gateway (readonly) server listening on /ip4/127.0.0.1/tcp/8080
Daemon is ready!

Keep in mind, that by running the IPFS Daemon:

1. Your node connects to the p2p net and can exchange blocks with other nodes
2. Other peers can access the content on your node – as long they know the CIDs
3. Peers will talk to you through TCP, UDP on port: 4001
4. If you have an application, start storing and consuming your node's content via the HTTP API listening on port: 5001.

For application development, I recommend the official ipfs-http-client library in JS exposing all the core commands – add, cat, object and others. It will speed up your coding progress.

I will use curl to interact with the API to keep this tutorial "short."

How to use the IPFS HTTP API:

Add content: :5001/api/v0/add

curl -X POST -F file=@/home/web3coach/go/src/github.com/ipfs/go-ipfs/hello_world.txt "http://127.0.0.1:5001/api/v0/add"

{"Name":"hello_world.txt","Hash":"QmNtQtxeavDXTjCiWAAaxnhKK3LBYfFfpXUXjdMDYXwKtH","Size":"142"}

Read content: :5001/api/v0/cat

curl -X POST "http://127.0.0.1:5001/api/v0/cat?arg=QmNtQtxeavDXTjCiWAAaxnhKK3LBYfFfpXUXjdMDYXwKtH"

hello IPFS world by Web3Coach. Testing DAGs
hello IPFS world by Web3Coach. Testing DAGs
hello IPFS world by Web3Coach. Testing DAGs

See the official HTTP API docs for the complete list of available commands.

How to peer with other IPFS nodes

Fun experiment. Use the ipfs swarm command and check how many nodes you already discovered:

ipfs swarm peers

> 
/ip4/85.70.151.37/tcp/4001/p2p/QmSuCtR...aPq6h4AczBPZaoej
/ip4/91.121.168.96/udp/54001/quic/p2p/QmeC7H...8j2TQ99esS
...
...
...

ipfs swarm peers | wc -l
> 186

Bravo! You are connected to 186 peers forming an unstoppable peer-to-peer web.

What about privacy?

Other peers can access all the content you add to your IPFS node. There is no built-in privacy mechanism, so never add unencrypted, sensitive/personal content to IPFS!

How Nodes Exchange Data Using the Bitswap Protocol

So far, you only interacted with your local content. Imagine you live in a place where the local government decided to block access to Wikipedia. No bueno.

Fortunately, because someone added all the Wikipedia content to IPFS, you can run your node and access its knowledge by requesting the content from peers across the globe.

http://localhost:8080/ipfs/QmT5NvUtoM5nWFfrQdVrFtvGfKFmG7AHE8P34isapyhCxX/wiki/Anasayfa.html

The DAG Service will check the blocks in your datastore, but it won’t find any for QmT5NvUtoM5nWFfrQdVrFtvGfKFmG7AHE8P34isapyhCxX.

The node will therefore create a network request to its peers using the Bitswap protocol via exchange component:

func getBlock(ctx context.Context, c cid.Cid, bs blockstore.Blockstore, fget func() exchange.Fetcher) (blocks.Block, error) {
   err := verifcid.ValidateCid(c) // hash security
   if err != nil {
      return nil, err
   }

   block, err := bs.Get(c)
   if err == nil {
      return block, nil
   }

   if err == blockstore.ErrNotFound && fget != nil {
      f := fget() // Don't load the exchange until we have to

      log.Debug("Blockservice: Searching bitswap")
      blk, err := f.GetBlock(ctx, c)

Internally, the CID is added to a Wantlist :

// Wantlist is a raw list of wanted blocks and their priorities
type Wantlist struct {
   set map[cid.Cid]Entry
}

// Entry is an entry in a want list, consisting of a cid and its priority
type Entry struct {
   Cid      cid.Cid
   Priority int32
   WantType pb.Message_Wantlist_WantType
}

And the PeerManager will iterate over known peers and their peers until it finds an online node capable of providing the wanted Block:

// PeerManager manages a pool of peers and sends messages to peers in the pool.
type PeerManager struct {
   pqLk sync.RWMutex

   peerQueues map[peer.ID]PeerQueue
   pwm        *peerWantManager

   createPeerQueue PeerQueueFactory
   ctx             context.Context

   psLk         sync.RWMutex
   sessions     map[uint64]Session
   peerSessions map[peer.ID]map[uint64]struct{}

   self peer.ID
}

The result?

You can consume the forbidden fruits from Wikipedia directly from localhost:8080:

IPFS loading Wikipedia on your local node

Uncensorable, decentralized storage :)

How to Persist Content from the p2p Network

You must know a crucial thing about IPFS: the content you access from the network will be garbage collected unless you pin it.

Pinning and Garbage Collection

At the beginning of the article you learned that content added to your node via the ipfs add command or its HTTP equivalent is pinned by default.

ipfs pin ls | grep QmNtQtxeavDXTjCiWAAaxnhKK3LBYfFfpXUXjdMDYXwKtH
> QmNtQtxeavDXTjCiWAAaxnhKK3LBYfFfpXUXjdMDYXwKtH recursive

Pinned blocks are marked as NOT TO BE DELETED when the Garbage Collection runs.

Why would the Garbage Collection delete some blocks? To keep your node healthy by controlling its storage size.

By reading Wikipedia or by accessing any other content from the p2p network, IPFS downloads its blocks. As the node's datastore grows in size, a periodic garbage collection process will prune unpinned blocks, so you don't run out of disk space.

If you want your content to be accessible 24/7 on the IPFS network, I recommend that you use a reliable remote provider to pin it: Infura - is the easiest way to get started, and you get 5GB free of decentralized storage.

Follow the getting started docs.

How to pin Wikipedia locally

Verify that the Wikipedia root-level CID (highest DAG node) isn't yet pinned on your node:

ipfs pin ls | grep QmT5NvUtoM5nWFfrQdVrFtvGfKFmG7AHE8P34isapyhCxX
> no output, not pinned

IPFS stores specific versions of Wikipedia in the form of a DAG. I recommend inspecting its graph before pinning:

ipfs object get QmT5NvUtoM5nWFfrQdVrFtvGfKFmG7AHE8P34isapyhCxX | jq

{
  "Links": [
    {
      "Name": "0C-",
      "Hash": "QmSEwJo8Z5bqVX3AhocyimJzPWetr7HgbWbwCg6zbp43AP",
      "Size": 1248085
    },
    {
      "Name": "43I",
      "Hash": "QmPW3kRjncDj145bP9DVNc791FowLPwYHnqbTzfe3whdyZ",
      "Size": 2611324931
    },
    {
      "Name": "58wiki",
      "Hash": "QmRNXpMRzsTHdRrKvwmWisgaojGKLPqHxzQfrXdfNkettC",
      "Size": 12295304394
    },
    {
      "Name": "92M",
      "Hash": "Qmbcvk7jpBTUKdgex139Nvv7BrKocE3pQVKhNJtTU77Qv5",
      "Size": 793
    },
    {
      "Name": "A0index.html",
      "Hash": "QmNqbYogAxH4mmt5WhuKN7NFEUDZ9V3Scxh7QbLwTKBJDk",
      "Size": 191
    }
  ],
  "Data": "\b\u0005\u0012\u0015\u0001\u0000\u0004\u0000\u0000\u0000\u0000\u0000\u0000\u0001\u0000\u0000\b\u0000\u0000\u0000\u0000\u0000\u0000\u0010\u0000(\"0�\u0002"
}

The root DAG object has five links. Four links are relatively small, but one link points to a DAG node with a total size of 12GB. If you inspect this DAG node, you will see 256 more links and a total cumulative (recursive) size of 12GB.

ipfs object stat QmRNXpMRzsTHdRrKvwmWisgaojGKLPqHxzQfrXdfNkettC

NumLinks:       256
BlockSize:      12075
LinksSize:      12034
DataSize:       41
CumulativeSize: 12295304394

Every node with an important pinned article, video, documentary, or a cat meme makes the web more accessible, antifragile, decentralized, and robust.

ipfs pin add QmT5NvUtoM5nWFfrQdVrFtvGfKFmG7AHE8P34isapyhCxX

The pinning process will recursively traverse the entire DAG node, fetch all its links from the Bitswap protocol and then pin every single Block to your local datastore.

Congratulations! In this article, you learned how decentralized storage works behind the scenes.

I worked 47 hours to write this blog post… but you can re-tweet it in just 5 seconds:

In this article, I'll show you how to make an NFT without software engineering skills. Then we will learn how to make unlimited customizable NFTs with Brownie, Python, and Chainlink. And we'll see how to render and sell our creation on the OpenSea NFT marketplace.

If you're looking for a tutorial that uses Truffle, JavaScript, and fun medieval characters, check out how to Build, Deploy, and Sell your NFT here.

What is an NFT?

NFTs (Non-Fungible Tokens) can be summed up with one word: "unique". These are smart contracts deployed on a blockchain that represent something unique.

ERC20 vs ERC721

NFTs are a blockchain token standard similar to the ERC20, like AAVE, SNX, and LINK (technically a ERC677). ERC20s are "fungible" tokens, which means “replaceable” or “interchangeable.”

For example, your dollar bill is going to be worth $1 no matter what dollar bill you use. The serial number on the dollar bill might be different, but the bills are interchangeable and they’ll be worth $1 no matter what.

NFTs, on the other hand, are "non-fungible", and they follow their own token standard, the ERC721. For example, the Mona Lisa is "non-fungible". Even though someone can make a copy of it, there will always only be one Mona Lisa. If the Mona Lisa was created on a blockchain, it would be an NFT.

_Original Image from Wikipedia_

What are NFTs for?

NFTs provide value to creators, artists, game designers and more by having a permanent history of deployment stored on-chain.

You'll always know who created the NFT, who owned the NFT, where it came from, and more, giving them a lot of value over traditional art. In traditional art, it can be tricky to understand what a "fake" is, whereas on-chain the history is easily traceable.

And since smart contracts and NFTs are 100% programmable, NFTs can also have added built-in royalties and any other functionality. Compensating artists has always been an issue, since often times an artist's work is spread around without any attribution.

More and more artists and engineers are jumping on this massive value add, because it's finally a great way for artists to be compensated for their work. And more than just that, NFTs are a fun way to show off your creativity and become a collector in a digital world.

The Value of NFTs

NFTs have come a long way, and we keep seeing record breaking NFT sales, like "Everydays: The First 5,000 Days” selling for $69.3 million.

Image from Twitter

So there is a lot of value here, and it's also a fun, dynamic, and engaging way to create art in the digital world and learn about smart contract creation. So now I'll teach you everything you need to know about making NFTs.

How to Make an NFT

What we are not going to cover

Now, the easiest way to make an NFT is just to go to a platform like Opensea, Rarible, or Mintible and follow their step-by-step guide to deploying on their platform.

You can 100% take this route, however you could be bound to the platform, and you are shoehorned into the functionality the platform has. You can't achieve the unlimited customization, or really utilize any of the advantages NFTs have. But if you're a beginner software engineer, or not very technical, this is the route for you.

If you're looking to become a stronger software engineer, learn some solidity, and have the power to create something with unlimited creativity, then read on!

If you're new to solidity, don't worry, we will go over the basics there as well.

How to Make an NFT with Unlimited Customization

I'm going to get you jump started with this NFT Brownie Mix. This is a working repo with a lot of boilerplate code.

Prerequisites

We need a few things installed to get started:

• Python
• Nodejs and npm
• Metamask

If you're unfamiliar with Metamask, you can follow this tutorial to get it set up.

Rinkeby Testnet ETH and LINK

We will also be working on the Rinkeby Ethereum testnet, so we will be deploying our contracts to a real blockchain, for free!

Testnets are great ways to test how our smart contracts behave in the real world. We need Rinkeby ETH and Rinkeby LINK, which we can get for free from the links to the latest faucets from the Chainlink documentation.

We will also need to add the rinkeby LINK token to our metamask, which we can do by following the acquire LINK documentation.

If you're still confused, you can following along with this video, just be sure to use Rinkeby instead of Ropsten.

When working with a smart contract platform like Ethereum, we need to pay a little bit of ETH, and when getting data from off-chain, we have to pay a little bit of LINK. This is why we need the testnet LINK and ETH.

Awesome, let's dive in. This is the NFT we are going to deploy to OpenSea.

Quickstart

git clone https://github.com/PatrickAlphaC/nft-mix
cd nft-mix

Awesome! Now we need to install the ganache-cli and eth-brownie.

pip install eth-brownie
npm install -g ganache-cli

Now we can set our environment variables. If you're unfamiliar with environment variables, you can just add them into your .env file, and then run:

source .env

A sample .env should be in the repo you just cloned with the environment variables commented out. Uncomment them to use them!

You'll need a WEB3_INFURA_PROJECT_ID and a PRIVATE_KEY . The WEB3_INFURA_PROJECT_ID can be found be signing up for a free Infura account. This will give us a way to send transactions to the blockchain.

We will also need a private key, which you can get from your Metamask. Hit the 3 little dots, and click Account Details and Export Private Key. Please do NOT share this key with anyone if you put real money in it!

export PRIVATE_KEY=YOUR_KEY_HERE
export WEB3_INFURA_PROJECT_ID=YOUR_PROJECT_ID_HERE

Now we can deploy our NFT contract and create our first collectible with the following two commands.

brownie run scripts/simple_collectible/deploy_simple.py --network rinkeby
brownie run scripts/simple_collectible/create_collectible.py --network rinkeby

The first script deploys our NFT contract to the Rinkeby blockchain, and the second one creates our first collectible.

You've just deployed your first smart contract!

It doesn't do much at all, but don't worry – I'll show you how to render it on OpenSea in the advanced part of this tutorial. But first, let's look at the ERC721 token standard.

The ERC721 Token Standard

Let's take a look at the contract that we just deployed, in the SimpleCollectible.sol file.

// SPDX-License-Identifier: MIT
pragma solidity 0.6.6;

import "@openzeppelin/contracts/token/ERC721/ERC721.sol";

contract SimpleCollectible is ERC721 {
    uint256 public tokenCounter;
    constructor () public ERC721 ("Dogie", "DOG"){
        tokenCounter = 0;
    }

    function createCollectible(string memory tokenURI) public returns (uint256) {
        uint256 newItemId = tokenCounter;
        _safeMint(msg.sender, newItemId);
        _setTokenURI(newItemId, tokenURI);
        tokenCounter = tokenCounter + 1;
        return newItemId;
    }

}

We are using the OpenZepplin package for the ERC721 token. This package that we've imported allows us to use all the functions of a typical ERC721 token. This defines all the functionality that our tokens are going to have, like transfer which moves tokens to new users, safeMint which creates new tokens, and more.

You can find all the functions that are given to our contract by checking out the OpenZepplin ERC721 token contract. Our contract inherits these functions on this line:

contract SimpleCollectible is ERC721 {

This is how solidity does inheritance. When we deploy a contract, the constructor is automatically called, and it takes a few parameters.

constructor () public ERC721 ("Dogie", "DOG"){
        tokenCounter = 0;
    }

We also use the constructor of the ERC721, in our constructor, and we just have to give it a name and a symbol. In our case, it's "Dogie" and "DOG". This means that every NFT that we create will be of type Dogie/DOG.

This is like how every Pokemon card is still a pokemon, or every baseball player on a trading card is still a baseball player. Each baseball player is unique, but they are still all baseball players. We are just using type DOG.

We have tokenCounter at the top that counts how many NFTs we've created of this type. Each new token gets a tokenId based on the current tokenCounter.

We can actually create an NFT with the createCollectible function. This is what we call in our create_collectible.py script.

function createCollectible(string memory tokenURI) public returns (uint256) {
        uint256 newItemId = tokenCounter;
        _safeMint(msg.sender, newItemId);
        _setTokenURI(newItemId, tokenURI);
        tokenCounter = tokenCounter + 1;
        return newItemId;
    }

The _safeMint function creates the new NFT, and assigns it to whoever called createdCollectible , aka the msg.sender, with a newItemId derived from the tokenCounter. This is how we can keep track of who owns what, by checking the owner of the tokenId.

You'll notice that we also call _setTokenURI. Let's talk about that.

What are NFT Metadata and TokenURI?

When smart contracts were being created, and NFTs were being created, people quickly realized that it's reaaaally expensive to deploy a lot of data to the blockchain. Images as small as one KB can easily cost over $1M to store.

This is clearly an issue for NFTs, since having creative art means you have to store this information somewhere. They also wanted a lightweight way to store attributes about an NFT – and this is where the tokenURI and metadata come into play.

TokenURI

The tokenURI on an NFT is a unique identifier of what the token "looks" like. A URI could be an API call over HTTPS, an IPFS hash, or anything else unique.

They follow a standard of showing metadata that looks like this:

{
    "name": "name",
    "description": "description",
    "image": "https://ipfs.io/ipfs/QmTgqnhFBMkfT9s8PHKcdXBn1f5bG3Q5hmBaR4U6hoTvb1?filename=Chainlink_Elf.png",
    "attributes": [
        {
            "trait_type": "trait",
            "value": 100
        }
    ]
}

These show what an NFT looks like, and its attributes. The image section points to another URI of what the NFT looks like. This makes it easy for NFT platforms like Opensea, Rarible, and Mintable to render NFTs on their platforms, since they are all looking for this metadata.

Off-Chain Metadata vs On-Chain Metadata

Now you might be thinking "wait... if the metadata isn't on-chain, does that mean my NFT might go away at some point"? And you'd be correct.

You'd also be correct in thinking that off-chain metadata means that you can't use that metadata to have your smart contracts interact with each other.

This is why we want to focus on on-chain metadata, so that we can program our NFTs to interact with each other.

We still need the image part of the off-chain metadata, though, since we don't have a great way to store large images on-chain. But don't worry, we can do this for free on a decentralized network still by using IPFS.

Here's an example imageURI from IPFS that shows the Chainlink Elf created in the Dungeons and Dragons tutorial.

The Chainlink Elf

We didn't set a tokenURI for the simple NFT because we wanted to just show a basic example.

Let's jump into the advanced NFT now, so we can see some of the amazing features we can do with on-chain metadata, have the NFT render on opeansea, and get our Dogie up!

If you want a refresher video on the section we just went over, follow along with the deploying a simple NFT video.

Dynamic and Advanced NFTs

Dynamic NFTs are NFTs that can change over time, or have on-chain features that we can use to interact with each other. These are the NFTs that have the unlimited customization for us to make entire games, worlds, or interactive art of some-kind. Let's jump into the advanced section.

Advanced Quickstart

Make sure you have enough testnet ETH and LINK in your metamask, then run the following:

brownie run scripts/advanced_collectible/deploy_advanced.py --network rinkeby
brownie run scripts/advanced_collectible/create_collectible.py --network rinkeby

Our collectible here is a random dog breed returned from the Chainlink VRF. Chainlink VRF is a way to get provable random numbers, and therefore true scarcity in our NFTs. We then want to create its metadata.

brownie run scripts/advanced_collectible/create_metadata.py --network rinkeby

We can then optionally upload this data to IPFS so that we can have a tokenURI. I'll show you how to do that later. For now, we are just going to use the sample tokenURI of:

https://ipfs.io/ipfs/Qmd9MCGtdVz2miNumBHDbvj8bigSgTwnr4SbyH6DNnpWdt?filename=1-PUG.json

If you download IPFS Companion into your browser you can use that URL to see what the URI returns. It'll look like this:

{
    "name": "PUG",
    "description": "An adorable PUG pup!",
    "image": "https://ipfs.io/ipfs/QmSsYRx3LpDAb1GZQm7zZ1AuHZjfbPkD6J7s9r41xu1mf8?filename=pug.png",
    "attributes": [
        {
            "trait_type": "cuteness",
            "value": 100
        }
    ]
}

Then we can run our set_tokenuri.py script:

brownie run scripts/advanced_collectible/set_tokenuri.py --network rinkeby

And we will get an output like this:

Running 'scripts/advanced_collectible/set_tokenuri.py::main'...
Working on rinkeby
Transaction sent: 0x8a83a446c306d6255952880c0ca35fa420248a84ba7484c3798d8bbad421f88e
  Gas price: 1.0 gwei   Gas limit: 44601   Nonce: 354
  AdvancedCollectible.setTokenURI confirmed - Block: 8331653   Gas used: 40547 (90.91%)

Awesome! You can view your NFT at https://testnets.opensea.io/assets/0x679c5f9adC630663a6e63Fa27153B215fe021b34/0
Please give up to 20 minutes, and hit the "refresh metadata" button

And we can hit the link given to see what it looks like on Opensea! You may have to hit the refresh metadata button and wait a few minutes.

Refresh Metadata

The Random Breed

Let's talk about what we just did. Here is our AdvancedCollectible.sol:

pragma solidity 0.6.6;

import "@openzeppelin/contracts/token/ERC721/ERC721.sol";
import "@chainlink/contracts/src/v0.6/VRFConsumerBase.sol";

contract AdvancedCollectible is ERC721, VRFConsumerBase {
    uint256 public tokenCounter;
    enum Breed{PUG, SHIBA_INU, BRENARD}
    // add other things
    mapping(bytes32 => address) public requestIdToSender;
    mapping(bytes32 => string) public requestIdToTokenURI;
    mapping(uint256 => Breed) public tokenIdToBreed;
    mapping(bytes32 => uint256) public requestIdToTokenId;
    event requestedCollectible(bytes32 indexed requestId); 


    bytes32 internal keyHash;
    uint256 internal fee;
    uint256 public randomResult;
    constructor(address _VRFCoordinator, address _LinkToken, bytes32 _keyhash)
    public 
    VRFConsumerBase(_VRFCoordinator, _LinkToken)
    ERC721("Dogie", "DOG")
    {
        tokenCounter = 0;
        keyHash = _keyhash;
        fee = 0.1 * 10 ** 18;
    }

    function createCollectible(string memory tokenURI, uint256 userProvidedSeed) 
        public returns (bytes32){
            bytes32 requestId = requestRandomness(keyHash, fee, userProvidedSeed);
            requestIdToSender[requestId] = msg.sender;
            requestIdToTokenURI[requestId] = tokenURI;
            emit requestedCollectible(requestId);
    }

    function fulfillRandomness(bytes32 requestId, uint256 randomNumber) internal override {
        address dogOwner = requestIdToSender[requestId];
        string memory tokenURI = requestIdToTokenURI[requestId];
        uint256 newItemId = tokenCounter;
        _safeMint(dogOwner, newItemId);
        _setTokenURI(newItemId, tokenURI);
        Breed breed = Breed(randomNumber % 3); 
        tokenIdToBreed[newItemId] = breed;
        requestIdToTokenId[requestId] = newItemId;
        tokenCounter = tokenCounter + 1;
    }

    function setTokenURI(uint256 tokenId, string memory _tokenURI) public {
        require(
            _isApprovedOrOwner(_msgSender(), tokenId),
            "ERC721: transfer caller is not owner nor approved"
        );
        _setTokenURI(tokenId, _tokenURI);
    }
}

We use the Chainlink VRF to create a random breed from a list of PUG, SHIBA_INU, BRENARD. When we call createCollectible this time, we actually kicked off a request to the Chainlink VRF node off-chain, and returned with a random number to create the NFT with one of those 3 breeds.

Using true randomness in your NFTs is a great way to create true scarcity, and using an Chainlink oracle random number means that your number is provably random, and can't be influenced by the miners.

You can learn more about Chainlink VRF in the documentation.

The Chainlink node responds by calling the fulfillRandomness function, and creates the collectible based on the random number. We then still have to call _setTokenURI to give our NFT the appearance that it needs.

We didn't give our NFT attributes here, but attributes are a great way to have our NFTs battle and interact. You can see a great example of NFTs with attributes in this Dungeons and Dragons example.

Metadata from IPFS

We are using IPFS to store two files:

1. The image of the NFT (the pug image)
2. The tokenURI file (the JSON file which also includes the link of the image)

We use IPFS because it's a free decentralized platform. We can add our tokenURIs and images to IPFS by downloading IPFS desktop, and hitting the import button.

IPFS add a file

Then, we can share the URI by hitting the 3 dots next to the file we want to share, hitting share link and copying the link given. We can then add this link into our set_tokenuri.py file to change the token URI that we want to use.

Persistance

However, if the tokenURI is only on our node, this means when our node is down, no one else can view it. So we want others to pin our NFT. We can use a pinning service like Pinata to help keep our data alive even when our IPFS node is down.

I imagine in the future more and more metadata will be stored on IPFS and decentralized storage platforms. Centralized servers can go down, and would mean that the art on those NFTs is lost forever. Be sure to check where the tokenURI of the NFT you use is located!

I also expect down the line that more people will use dStorage platforms like Filecoin, as using a pinning service also isn't as decentralized as it should be.

Going forward

If you'd like a video walkthrough of the advanced NFT, you can watch the advanced NFT video.

Now you have the skills to make beautiful fun, customizable, interactive NFTs, and have them render on a marketplace.

NFTs are fun, powerful ways to have artists accurately compensated for all the hard work that they do. Good luck, and remember to have fun!

Immutability is blockchain’s greatest strength and biggest barrier. Progressive decentralization could be the answer.

When we released CryptoKitties a year ago, we opted not to fund it up front with an ICO but instead build it on a sustainable revenue model. That model is this: we collect a fee of 3.75% from every transaction in the game. Given that we’d be unable to change the fee once we launched — CryptoKitties is built on the Ethereum blockchain — people often ask how we arrived at that number.

It sounds like a smart, well-reasoned choice. I could spin a compelling story about how we ran simulations with advanced prediction models to find the fee that would yield optimal returns.

But that’s not true.

The truth is we made an educated guess. We picked a number that felt fair and we committed to it.

Immutability is awesome and scary

We easily could have chosen wrong, and since you can’t change something once you add it to the blockchain, that would have been cat-astrophic. Fortunately for CryptoKitties, our community is so passionate and the Kitties are so adorable that 3.75% worked just fine.

Immutability, the inability to be edited, is at once the blockchain’s greatest strength and its largest barrier to meaningful adoption. The pressures of immortal code paralyze developers: you can tinker in a test environment forever, but there will always be real-world variables you can’t anticipate. Covering your eyes and hitting launch is no way to make breakthroughs. It’s more likely to produce breakdowns.

Our fee was just one decision among many: how long should breeding a Kitty take? At what rate should their breeding cooldowns slow? How much should a Gen 0 cat cost? On blockchain, even a seemingly minor choice can pose serious, even critical, consequences.

Decentralization offers everyday people immense benefits: the fairness of permanent and universal rules and the transparency of code and behavior which, combined, create security. However, because it’s often implemented with all-or-nothing immutability, blockchain makes agile development impossible and slows teams to a crawl.

Agility requires iteration. Iterating quickly is key to building the best products, and the best products spark mass adoption.

Enter Progressive Decentralization

We encountered these barriers ourselves building CryptoKitties, which forced us to negotiate including decentralized features while building something that, ya know, works. Since then, we’ve started exploring progressive decentralization in development, an idea we briefly introduced a while ago.

Let’s take a deeper dive now.

Simply put, progressive decentralization advocates easing into decentralization in stages rather than diving in headfirst. What that looks like is building mechanisms into smart contracts that confer special powers to the creators up front, then incrementally lock those powers away in a transparent and systematic way.

The critical condition is that the locking mechanisms must be public and immutable from the start. The creator can’t decide to tweak the terms later and indefinitely extend their power. That balance is vital: done correctly, progressive decentralization allows creators the flexibility to repair their code without compromising the decentralized features of the contract.

Progressive decentralization can take many forms

There’s no one right way to implement progressive decentralization. There are dozens of variables to consider, and the best approach will vary from project to project.

Here are a couple ways developers could approach progressive decentralization:

1. Author multiple contracts with appropriate separation of concerns and the ability to replace some of those contracts. Some decentralized apps (“dapps”) like Decentraland, which features upgradable contracts, are already using this.
2. Configurable variables and permissions to change those values independently. Etheremon, for instance, grants special permissions to groups of users who become moderators.
3. Incorporate a predefined set of ascending levels in the contract, each allowing the creators certain capabilities. The levels can only be increased, never decreased, so backtracking isn’t an option. On level 1, for example, the contract owners can play around with all gameplay variables. At level 2, their capability to modify core variables ends. At the final level, the contract revokes all their special privileges.

To die-hard decentralists, some of this probably sounds too centralized. But this is just the starting point. There are further measures to balance decentralization with iteration. The solution combines transparency of the purpose and the conditions and constraints in the contracts. These constraints could include:

1. Selection: Not everything can be modified, only the specific items that we need to iterate.
2. Range: For many of the questions around game economies, we may have a general idea but not know the precise answer. Limiting configuration to a certain range guarantees users that the iteration will land within a reasonable scope.
3. Direction: Similar to the “levels” concept above, allow certain variables to move only in one direction, decreasing or increasing but never backtracking.

Holding creators accountable

All this sounds great in theory. But how do we ensure creators stay true to their roadmap and reach the fully decentralized version of their contracts? How can users opt-in early with the guarantee that the system is an application of progressive decentralization? How can we know we won’t end up with just another flawed, centralized system?

Progressive decentralization includes tenets to keep creators accountable:

Time- or block-based maturity

Lock certain configuration values, revoke the owner’s capabilities or move to the next level of maturity past a certain time or block number. Once that point is reached, the contract automatically changes.

Imagine, for example, that CryptoKitties had a runway of 360,000 blocks (around 60 days’ time) from the moment it launched to adjust the Kitties’ breeding cooldown variables. We could tweak the cooldown mechanics until that point, giving ourselves the breathing room to perfect the balance, while still guaranteeing players that we wouldn’t have that power indefinitely.

Usage-based maturity

Lock those capabilities once a certain number of users or transactions are completed. This option needs to be carefully thought out to avoid exploits, but we could have, for example, built configurable fees into CryptoKitties that would lock in after 10,000 transactions.

Economic incentive

Align the creator’s incentives with increased decentralization. In this scenario, the creators profit more when the contract becomes more decentralized. Perhaps the fee rises with each level the developer ascends, locking in at the maximum fee when they reach full decentralization. Or, alternatively, perhaps they make no money at all until full decentralization is in place. This financial reward motivates the developer to reach decentralization at a reasonable pace.

There’s no best approach to building on the blockchain

“Progressive decentralization” is really an umbrella encompassing many strategies, mechanisms, and tools to make building on the blockchain more viable. The best way to apply progressive decentralization will always depend on the project and use a mix of the concepts outlined above.

Progressive decentralization is not perfect. The ideal smart contract is simple and straightforward, and these measures add complexity. How and how much to incorporate it is a trade-off that needs to be evaluated on a case-by-case basis.

Although it may anger hardline decentralists, we believe progressive decentralization is far better for users in the long run: by giving developers the flexibility to adjust, the consumer gets a more useful product. That means they’ll actually use it, and once it brings value to their lives, they’ll sing its praises to the people around them. That’s how mass adoption starts.

_Authors: Arthur Camara, Dieter Shirley, and Grady Mitchell_

Recently, Prof. Tim Berners-Lee lifted the veil off a project called Solid. I decided to check it out. In this article, I describe what Solid aims to do, and also how you can get started with it.

What is Solid?

Solid is an attempt to re-decentralize the web.

_Re-_decentralize?

Back in the day, the vision for the web was a decentralized, collaborative read-write space. The first browser (called WorldWideWeb) was also an editor.

However, as it progressed, the design of web applications began to centralize for a variety of reasons. User data became the source of power and income for Internet companies.

Solid is a solution to this.

Solid is a new paradigm for web applications, one that is backwards compatible with the existing web.

Solid is a tech stack, a group of related protocols, implementations, and a growing community. Much like the web.

The separation of app and data

In pre-internet computing, your personal computer stored your data.

As people began using multiple computers, and added smartphones to their lives, the “your data stays with you” model was replaced by “Your data is in one or more massive data centers around the world, managed by the app developer”.

And so, applications were deeply coupled with their data. Creating an application on the web entails managing people’s data at scale.

Apps and their ability to make money are measured by their data silo. Your data is difficult to migrate, since different apps store your data very differently.

The result? Almost every app has walled garden characteristics. This reduces incentives for developers to innovate at the app level. Existing platforms are secured against disruption, since the data lockdown makes it hard for users to move.

Data protection regulations

Some countries have enacted data protection laws. Companies must make your data available, and you can chose to download or delete it.

This attempts to return control over data back to users. But it’s a legal prescription, and not the technical reality. User data still lies with app developers, and the ability to download your data isn’t very useful if you can’t migrate to an alternative.

Pods: Bring your own data

Solid remedies this on the technical side. It allows applications to be built in a way where they read and write data stored on your pod.

You have a pod. Your friends have a pod. Pods store your data. You allow apps to access your pod.

Maybe you have multiple pods. Perhaps separate ones for home and work. Your pod can live on your computer, or be distributed across your devices. Or it could be hosted for you.

And pods store linked data. Your pod can link to something on my pod, or anywhere on the web.

We want applications that run across our devices. But we also want autonomy of our data. And we want the ability for different apps to use the same data and write to it.

The ideas behind Solid

Getting into Solid reminded me of starting out with web development. I remember learning HTML, CSS, JavaScript, and the frameworks of the day, all at the same time.

The only difference: Solid is new, and help is harder to find.

Here’s a collection of day-one concepts you’ll want to know to get started developing for Solid:

(PS: if you just wanna jump in, skip ahead to ‘First steps’)

Linked data

The power of the Solid, and the web generally, is from the way data is hyperlinked together.

In Solid, you store the data you produce wherever you want. Your personal data likely resides on your pod. To refer to this data, you use URLs, like on the web.

This is also a good time to introduce the full-form of Solid: SOcial LInked Data.

Read about Linked Data in the context of Solid

Resource Description Framework

RDF is a way to represent linked data with statements of the form subject-predicate-object. These are also called triples.

RDF is an abstract model. You could even represent RDF in English sentences. Here’s a task on a Todo list:

T1 is a taskT1 is labelled "Write an article about Solid"T1 is due October 5rd 2018T1 is assigned to @itsarnavbT1 is incomplete

Turtle

Turtle is a compact way of representing RDF data, using URLs to represent subject, predicate and object.

That’s repetitive and hard to read, so turtle has a prefix and shorthand system. This gets especially important with longer documents.

You can read more about turtle. Or you could check out a full turtle document here. It’s a detailed public profile of Prof. Ruben Verborgh, who’s a part of the Solid team.

Semantic web

Tim Berners-Lee best explains this:

I have a dream for the Web [in which computers] become capable of analyzing all the data on the Web - the content, links, and transactions between people and computers. A “Semantic Web”, which makes this possible, has yet to emerge, but when it does, the day-to-day mechanisms of trade, bureaucracy and our daily lives will be handled by machines talking to machines. The “intelligent agents” people have touted for ages will finally materialize

First steps

Do these, in any order that works for you.

• Get a pod: Signup with any free pod provider, or run your own server (if that’s your thing).
• Make a Solid app with this tutorial
• Read about these hacks made with Solid
• Read the Solid docs

Go Solid

You can help out the Solid ecosystem by

• contributing to the development of Solid itself, and related infrastructure.
• developing apps using Solid.

But beware, at the moment, learning and developing for Solid requires a lot of trial and error, and asking potentially silly questions. There’s no Stack Overflow to refer to. Debugging some errors might require you to dig into the source.

Here are the communities where you can get help:

• r/solid (I’m one of the mods)
• gitter.im/solid

And finally, my DMs are open: @itsarnavb. I’ll try to answer every question I get, or find someone who can.

And I’ll keep this article up to date with the best resources to learn about Solid.

Further Reading

• Solid website - solid.mit.edu
• Paradigm shifts for the decentralized web - Ruben Verborgh
• One Small Step for the Web - Tim Berners-Lee

At Hashnode we have been experimenting a lot with blockchain and its use-cases. We have been running a developers’ community ourselves, and the idea behind “decentralized communities” fascinates me a lot. The fact that everyone owns the data and controls the platform can give rise to new types of social apps and disrupt the traditional way of building online communities.

Platforms like Steemit have proven that it’s possible to build such communities and reward users for their contributions. But how should someone go about replicating it and launching their own decentralized social platform powered by blockchain?

To answer the question, I took up the challenge of building a decentralized version of HackerNews.

During the process, I evaluated multiple platforms and finally zeroed in on a protocol called Tendermint. Using Tendermint, I have built a prototype called “Mint” which can serve as a boilerplate for building blockchain-powered social apps.

The codebase is on GitHub. You can check out the following links for code and demo:

• Mint Blockchain
• Website Code (Blockchain Front-end)
• Demo

So what does it take to build a blockchain-powered social community where the user-generated data is decentralized? If you are looking for an answer, you have come to the right place. Read on.

Preliminary Observations

Initially, I thought of utilizing an existing platform to build the app. Smart Contract platforms like Ethereum, NEM, NEO, and so on offer storage of assets, but these are not designed to store large amount of data.

HyperLedger Fabric is compelling, but it’s designed to be deployed in private blockchain networks. Hashgraph sounds interesting, but it’s experimental as of now.

Other potential solutions were: Lisk Sidechains, Loom Network, and BigChainDB. The first two are in private alpha (invite-only), while BigChainDB is powered by Tendermint.

So, instead of using BigChainDB, I decided to play around with Tendermint directly and see what was possible.

Why Tendermint

Tendermint is a protocol that takes care of the consensus layer using BFT algorithm while you just focus on writing the business logic.

The beauty of the protocol is that you are literally free to choose any programming language to build an interface (Application Blockchain Interface or simply ABCI) that interacts with the blockchain.

Tendermint handles the most complex aspects of a blockchain such as block production rounds, peer to peer connectivity, gossiping about new blocks, transaction handling, and more. It stores the transactions on the disk using LevelDB and also delivers the confirmed transaction to your ABCI server so that you can create a global state out of it.

Sounds interesting? Let’s see how to create a blockchain app that stores data on chain using Tendermint.

What’s Needed?

Here is what you are going to need:

• Macbook / Ubuntu server
• Golang
• Tendermint
• MongoDB
• And beer… (Coffee lovers can replace this with coffee)

Setting up the Machine

Tendermint is written in Go. So, we need to install the Go language first. Visit this link to check out a few download options. If you are on Ubuntu, you can follow this guide.

By default, Go chooses $HOME/go as your workspace. If you want to use a different location as your workspace, you can set GOPATH variable in ~/.profile . From now on, we’ll refer to this location as GOPATH.

Here is how ~/.profile file looks on my machine:

export GOPATH="$HOME/go" export PATH=~/.yarn/bin:$GOPATH/bin:$PATHexport GOBIN="$GOPATH/bin"

Remember to set GOBIN variable as shown above. This is where the Go binaries will be installed.

Don’t forget to run source ~/.profile after updating the file.

Now we can install Tendermint. Here are the steps:

• cd $GOPATH/src/github.com
• mkdir tendermint
• cd tendermint

And finally,

git clone https://github.com/tendermint/tendermint

This will install the latest version of Tendermint. As I have tested my code against v0.19.7, let’s check out the specific release.

cd tendermintgit checkout v0.19.7

This will put you on v0.19.7. To proceed with the installation, run the following commands:

make get_tools make get_vendor_depsmake install

Congrats! You have installed Tendermint successfully. If everything was installed as intended, the command tendermint version will print out the Tendermint version.

Now, you should go ahead and install MongoDB.

Coding the Blockchain

If you want to understand how Tendermint works, go through this guide. You may also find the following diagram helpful.

_Source: [Tendermint Docs](http://tendermint.readthedocs.io/projects/tools/en/master/introduction.html" rel="noopener" target="blank" title=")

I’ll outline a few important concepts here:

• Tendermint core handles the consensus part.
• You need to write an ABCI server that handles the business logic, validations, and so on. Although you can write this in any language, our language of choice will be Go.
• Tendermint core will interact with your ABCI server via socket connections.
• The ABCI server has many methods (JS developers can think of them as callbacks) that will be invoked by Tendermint core on various events.
• Two important methods are: CheckTx and DeliverTx. The first one is called to validate a transaction, while the latter is called when the Tx is confirmed.
• DeliverTx helps you take necessary actions based on the confirmed transactions. In our case, we’ll use this to create and update our global state stored in MongoDB.
• Tendermint uses BFT consensus. This means more than 2/3 of the validators need to have consensus in order to commit a transaction. So, even if 1/3 of the validators go rogue, the blockchain will still work.
• In a real world scenario (at least in a public deployment), you will most likely add some sort of consensus such as PoS (Proof of State) in addition to BFT consensus. In this case, we’ll just go ahead with simple BFT consensus. I’ll leave adding PoS up to you.

I suggest that you clone the blockchain ABCI server (code-named mint) from GitHub. But before we go ahead, we need to install a dependency management tool called dep.

If you are on a Mac, you can just run brew install dep . For Ubuntu, run the following command.

curl https://raw.githubusercontent.com/golang/dep/master/install.sh | sh

Now you can clone the codebase of mint.

cd $GOPATH/srcgit clone https://github.com/Hashnode/mintcd mintdep ensurego install mint

Sweet! You have now installed mint, which is an ABCI server and works along with Tendermint core.

Now, let me walk you through the whole set-up and all the code.

Entry Point

You can find the code (and entry point) on GitHub here.

The entry point of the app is mint.go . The most important part of the file is the following section:

app = jsonstore.NewJSONStoreApplication(db)srv, err := server.NewServer("tcp://0.0.0.0:46658", "socket", app) if err != nil {  return err }

All the business logic, methods, and so on are defined in the package jsonstore . The above code simply creates a TCP server on port 46658 that accepts socket connections from Tendermint core.

Now let’s look at jsonstorepackage.

Business Logic

Here’s the jsonstore repo.

Our ABCI server does two important things:

• Validates incoming transactions. If a transaction is invalid, it returns an error code and the transaction is rejected.
• Once a transaction is committed (confirmed by > 2/3 of the validators) and stored in LevelDB, the ABCI server updates its global state stored in MongoDB.

We’re going to use mgo for interacting with MongoDB. So, jsonstore.go defines 5 models that correspond to 5 different MongoDB collections.

The code looks like the following:

// Post ...type Post struct {    ID          bson.ObjectId `bson:"_id" json:"_id"`    Title       string        `bson:"title" json:"title"`    URL         string        `bson:"url" json:"url"`    Text        string        `bson:"text" json:"text"`    Author      bson.ObjectId `bson:"author" json:"author"`    Upvotes     int           `bson:"upvotes" json:"upvotes"`    Date        time.Time     `bson:"date" json:"date"`    Score       float64       `bson:"score" json:"score"`    NumComments int           `bson:"numComments" json:"numComments"`    AskUH       bool          `bson:"askUH" json:"askUH"`    ShowUH      bool          `bson:"showUH" json:"showUH"`    Spam        bool          `bson:"spam" json:"spam"`}

// Comment ...type Comment struct {    ID              bson.ObjectId `bson:"_id" json:"_id"`    Content         string        `bson:"content" json:"content"`    Author          bson.ObjectId `bson:"author" json:"author"`    Upvotes         int           `bson:"upvotes" json:"upvotes"`    Score           float64       `bson:"score" json:"score"`    Date            time.Time    PostID          bson.ObjectId `bson:"postID" json:"postID"`    ParentCommentID bson.ObjectId `bson:"parentCommentId,omitempty" json:"parentCommentId"`}

// User ...type User struct {    ID        bson.ObjectId `bson:"_id" json:"_id"`    Name      string        `bson:"name" json:"name"`    Username  string        `bson:"username" json:"username"`    PublicKey string        `bson:"publicKey" json:"publicKey"`}

// UserPostVote ...type UserPostVote struct {    ID     bson.ObjectId `bson:"_id" json:"_id"`    UserID bson.ObjectId `bson:"userID" json:"userID"`    PostID bson.ObjectId `bson:"postID" json:"postID"`}

// UserCommentVote ...type UserCommentVote struct {    ID        bson.ObjectId `bson:"_id" json:"_id"`    UserID    bson.ObjectId `bson:"userID" json:"userID"`    CommentID bson.ObjectId `bson:"commentID" json:"commentID"`}

We also define a few utility functions such as the following:

func byteToHex(input []byte) string {    var hexValue string    for _, v := range input {        hexValue += fmt.Sprintf("%02x", v)    }    return hexValue}

func findTotalDocuments(db *mgo.Database) int64 {    collections := [5]string{"posts", "comments", "users", "userpostvotes", "usercommentvotes"}    var sum int64

for _, collection := range collections {        count, _ := db.C(collection).Find(nil).Count()        sum += int64(count)    }

return sum}

func hotScore(votes int, date time.Time) float64 {    gravity := 1.8    hoursAge := float64(date.Unix() * 3600)    return float64(votes-1) / math.Pow(hoursAge+2, gravity)}

// FindTimeFromObjectID ... Convert ObjectID string to Timefunc FindTimeFromObjectID(id string) time.Time {    ts, _ := strconv.ParseInt(id[0:8], 16, 64)    return time.Unix(ts, 0)}

These will be used subsequently in the code.

Inside CheckTx

Now let’s come to the validation part. How do we accept or reject a transaction? Let’s say someone is trying to sign up, but doesn’t choose a valid username. How can our app validate this?

It’s done via CheckTx function. The signature looks like the following:

func (app *JSONStoreApplication) CheckTx(tx []byte) types.ResponseCheckTx {

 // ... Validation logic}

When a Tendermint node receives a transaction, it invokesCheckTx of ABCI server and passes tx data as a byte array argument. If CheckTx returns a non-zero code, the transaction is rejected.

In our case, clients send Base64 encoded stringified JSON objects to the Tendermint node via an RPC request. So, it is our job to decode the tx and unmarshall the string into a JSON object.

It’s done like this:

var temp interface{}err := json.Unmarshal(tx, &temp)if err != nil {  panic(err)}message := temp.(map[string]interface{})

message object typically looks like the following:

{  body: {... Message body},  publicKey: <Public Key of Sender>,  signature: <message.body is signed with the Private Key>}

First, we need to make sure that said person has indeed submitted the transaction to the blockchain, not someone else claiming to be that person.

The best way to validate is to ask clients to sign the message body with the user’s private key and attach both the public key and the signature to the payload. We’ll use ed25519 algorithm to generate the keys and sign the message in the browser and hit the RPC endpoint. In the CheckTx function we’ll again use ed25519 and verify the message with the help of the user’s public key.

It’s done like this:

pubKeyBytes, err := base64.StdEncoding.DecodeString(message["publicKey"].(string))

sigBytes, err := hex.DecodeString(message["signature"].(string))

messageBytes := []byte(message["body"].(string))isCorrect := ed25519.Verify(pubKeyBytes, messageBytes, sigBytes)if isCorrect != true {  return types.ResponseCheckTx{Code: code.CodeTypeBadSignature}}

In the above example, we use the ed25519 package to validate the message. Various codes such as code.CodeTypeBadSignature are defined inside code package. These are just integers. Just remember that if you want to reject a transaction, you have to return a non-zero code. In our case, if we detect that the message signature is not valid, we return CodeTypeBadSignature which is 4.

The next section of CheckTx deals with various data validations, such as:

• If the user is sending any transaction other than “createUser (Sign up)”, we first check that the user’s public key is present in our database.
• If the user is trying to create a post or comment, it should have valid data such as non-empty title , content , and so on.
• If the user is trying to sign up, the username should have acceptable characters.

The code looks like the following:

// ==== Does the user really exist? ======if body["type"] != "createUser" { publicKey := strings.ToUpper(byteToHex(pubKeyBytes))

 count, _ := db.C("users").Find(bson.M{"publicKey": publicKey}).Count()

 if count == 0 {  return types.ResponseCheckTx{Code: code.CodeTypeBadData} }}// ==== Does the user really exist? ======

codeType := code.CodeTypeOK

// ===== Data Validation =======switch body["type"] {case "createPost": entity := body["entity"].(map[string]interface{})

  if (entity["id"] == nil) || (bson.IsObjectIdHex(entity["id"]. (string)) != true) {  codeType = code.CodeTypeBadData  break }

if entity["title"] == nil || strings.TrimSpace(entity["title"].(string)) == "" {  codeType = code.CodeTypeBadData  break }

if (entity["url"] != nil) && (strings.TrimSpace(entity["url"].(string)) != "") {  _, err := url.ParseRequestURI(entity["url"].(string))  if err != nil {   codeType = code.CodeTypeBadData   break  } }case "createUser": entity := body["entity"].(map[string]interface{})

if (entity["id"] == nil) || (bson.IsObjectIdHex(entity["id"].(string)) != true) {  codeType = code.CodeTypeBadData  break }

r, _ := regexp.Compile("^[A-Za-z_0-9]+$")

if (entity["username"] == nil) || (strings.TrimSpace(entity["username"].(string)) == "") || (r.MatchString(entity["username"].(string)) != true) {  codeType = code.CodeTypeBadData  break }

if (entity["name"] == nil) || (strings.TrimSpace(entity["name"].(string)) == "") {  codeType = code.CodeTypeBadData  break }case "createComment": entity := body["entity"].(map[string]interface{})

if (entity["id"] == nil) || (bson.IsObjectIdHex(entity["id"].(string)) != true) {  codeType = code.CodeTypeBadData  break }

if (entity["postId"] == nil) || (bson.IsObjectIdHex(entity["postId"].(string)) != true) {  codeType = code.CodeTypeBadData  break }

if (entity["content"] == nil) || (strings.TrimSpace(entity["content"].(string)) == "") {  codeType = code.CodeTypeBadData  break }}

// ===== Data Validation =======return types.ResponseCheckTx{Code: codeType}

The code is really simple and pretty self-explanatory. So, I won’t go into the details, and will leave it up to you to read and explore further.

Inside DeliverTx

Once a transaction is confirmed and applied to the blockchain, Tendermint core calls DeliverTx and passes the transaction as a byte array. The function signature looks like the following:

func (app *JSONStoreApplication) DeliverTx(tx []byte) types.ResponseDeliverTx {  // ... Code goes here}

We’ll use this function to construct a MongoDB-based global state. We do this so that our website users can read the data easily.

This function is big and has multiple cases. In this section I’ll just cover only one case which is “Post Creation”. As the rest of the code is similar, I’ll leave it up to you to dig deeper and explore the full code.

Firstly, we’ll go ahead and unmarshall the txdata into a JSON object:

var temp interface{}err := json.Unmarshal(tx, &temp)

if err != nil { panic(err)}

message := temp.(map[string]interface{})

var bodyTemp interface{}

errBody := json.Unmarshal([]byte(message["body"].(string)), &bodyTemp)

if errBody != nil { panic(errBody)}

body := bodyTemp.(map[string]interface{})

For post creation, the message object looks like the following:

{body: {  type: "createPost",  entity: {    id: id,    title: title,    url: url,    text: text,    author: author  }},signature: signature,publicKey: publicKey}

And here is how DeliverTx function creates a new entry in the database when a “createPost” transaction is committed:

entity := body["entity"].(map[string]interface{})

var post Postpost.ID = bson.ObjectIdHex(entity["id"].(string))post.Title = entity["title"].(string)

if entity["url"] != nil { post.URL = entity["url"].(string)}if entity["text"] != nil { post.Text = entity["text"].(string)}

if strings.Index(post.Title, "Show UH:") == 0 { post.ShowUH = true} else if strings.Index(post.Title, "Ask UH:") == 0 { post.AskUH = true}

pubKeyBytes, errDecode := base64.StdEncoding.DecodeString(message["publicKey"].(string))

if errDecode != nil { panic(errDecode)}

publicKey := strings.ToUpper(byteToHex(pubKeyBytes))

var user Usererr := db.C("users").Find(bson.M{"publicKey": publicKey}).One(&user)if err != nil { panic(err)}post.Author = user.ID

post.Date = FindTimeFromObjectID(post.ID.Hex())

post.Upvotes = 1

post.NumComments = 0

// Calculate hot rankpost.Score = hotScore(post.Upvotes, post.Date)

// While replaying the transaction, check if it has been marked as spam

spamCount, _ := db.C("spams").Find(bson.M{"postID": post.ID}).Count()

if spamCount > 0 { post.Spam = true}

dbErr := db.C("posts").Insert(post)

if dbErr != nil { panic(dbErr)}

var document UserPostVotedocument.ID = bson.NewObjectId()document.UserID = user.IDdocument.PostID = post.ID

db.C("userpostvotes").Insert(document)

The actual code block has a switch statement that handles each type of transaction differently. Feel free to check out the code and play around. If something is unclear, feel free to write your queries in the comments below.

Now that we’ve examined two important aspects of the ABCI server, let’s try to run both Tendermint core and our server and see how to send transactions.

In order to run the app, run the following commands from two different terminals.

First, run:

mint

If the command succeeds, you will see the following output in the terminal:

Mint Output

Make sure MongoDB is already running before starting mint. If your terminal is unable to recognize mint command, be sure to run source ~/.profile .

Then start Tendermint in a different terminal:

tendermint node --consensus.create_empty_blocks=false

By default, Tendermint produces new blocks every 3 seconds, even if there are no transactions.

To prevent that we use the flag:

consensus.create_empty_blocks=false

Now that Tendermint is running you can start sending the transactions to it. You need a client that can generate ed25519 keys, sign your requests, and hit the RPC endpoint exposed by Tendermint.

An example request (Node.js) looks like this:

const base64Data = req.body.base64Data;

let headers = {    'Content-Type': 'text/plain',    'Accept':'application/json-rpc'}

let options = {    url: "http://localhost:46657",    method: 'POST',    headers: headers,    json: true,    body: {"jsonrpc":"2.0","method":"broadcast_tx_commit","params": { "tx" : base64Data } ,"id":"something"}}

request(options, function (error, response, body) {    res.json({ body: response.body });});

Note that the RPC endpoint is exposed on port 46657 .

Forming and signing the requests manually can be tedious. So, I suggest that you use Uphack (a HackerNews style website that interacts with the blockchain) to get the full picture.

To install Uphack, follow the steps below:

git clone https://github.com/Hashnode/Uphackcd Uphackyarngulp less // make sure gulp is installed globallynode server.js

You can access the website on http://localhost:3000 . It looks like this on my machine:

Uphack

As you don’t have any data yet, it will look empty initially. Feel free to register an account and submit some posts to visualize the process.

While you are using the app, open up the network tab of your browser and check out the XHR section. The /rpc URL accepts the base64 data and makes request to Tendermint’s RPC endpoint server-side. You can copy the base64 data and paste it into a base64 decoder to see the actual data that’s being sent.

Going into the details of Uphack is out of the scope of this tutorial. However, as mentioned above, the code for Uphack (the client) is open source and the logic is straightforward. If you go through the codebase and examine various endpoints, you will develop a better understanding of the whole process.

Wrapping up

To summarize, we built a blockchain that stores JSON data on chain and accepts transactions in the form of base64. To demonstrate the usage, we also briefly examined Uphack, a HackerNews style website that interacts with the blockchain.

However, here are a few things you should be aware of:

• You have built a single node network. This means you are the only validator. If you are interested in multi-node deployment, check out mint’s documentation. We have deployed a 4-node network so far, and if you wish to become a validator and play around with the blockchain, feel free to reach out to me.
• This arrangement uses BFT consensus. In a real world scenario you will need some consensus algorithm like Proof of Stake, Delegated Proof of Stake, and so on.
• The blockchain RPC endpoint listens on 46657 and the ABCI server runs on 46658 . At any time you can check the blockchain status by visiting localhost:46657/status .
• Right now there is no incentive for becoming a validator and producing blocks. In a (D)PoS setting, the block producers should be rewarded with some token every time they propose a block. It’s left as an exercise to you.
• The ABCI server can be written in any language. For example, check js-abci.

To conclude, I would like to make it clear that I am not a blockchain expert. I am a learner and I am just sharing things I find interesting. Storing data on chain fascinates me and I believe decentralized social communities are one of the prime use-cases of blockchain.

If you spot any inaccuracies anywhere in the codebase or article, feel free to point it out. I’ll appreciate if you use mint & Uphack and provide your feedback. PRs are always welcome!

Let me know what you think in the comments below!