The three concepts that stand out to me as the most fundamental for being able to effectively work with Git at a basic level are:

1. The working directory

2. The staging area

3. The commit history

In this article, we’ll take a new approach to representing these three concepts: by visualizing them in an immersive, 3D game world!

I’ll provide a tangible, visual representation of these key Git concepts which are almost always described in an abstract and confusing way. I hope that this will make them much more intuitive for you to grasp.

Table of Contents

1. Visualize your Working Directory

2. Demystify your Staging Area

3. Literally Walk through your Commit History

4. Summary

5. Try it Yourself

Visualize your Working Directory

What does your brain picture when you think of the “working directory”? I assume it’s something like a folder structure starting at the project root, containing the code files and subfolders that make up the project.

While that is a fair description of the working directory, it is a bit hard to imagine and misses the segmentation that Git applies to your project. Although the current state of your entire project, folder structure, and code files do reside in the working directory, Git doesn’t really need to do much about that unless certain changes are detected in those files.

Git detects and reports changes to the working directory with the Git status command, which shows output like this:

Jack@RAPTOR ~/my-project (main)> git status
On branch main
Your branch is up to date with 'origin/main'.

Changes not staged for commit:
  (use "git add <file>..." to update what will be committed)
  (use "git restore <file>..." to discard changes in working directory)
        modified:   main.py
        modified:   settings.py

Untracked files:
  (use "git add <file>..." to include in what will be committed)
        one.py
        three.py
        two.py

no changes added to commit (use "git add" and/or "git commit -a")

The two relevant sections here are:

1. Changes not staged for commit: Lists existing files tracked by Git which currently contain code changes. In the example above, we see two “modified files”: main.py and settings.py.

2. Untracked files: Lists new files in your project that Git doesn’t know about yet. In the example above, we see three new, untracked files: one.py, two.py, and three.py.

When it comes to understanding Git, thinking of the working directory as the changes Git sees in these two sections – Untracked files and Modified files – is quite helpful.

But the git status command reports these details in the terminal in a purely text-based way, which doesn’t do newer Git users any favors when it comes to wrapping their heads around Git.

Some Git GUI’s do a better job with this (they do provide a safer point-and-click interface, after all), but in my experience, none of them make things obvious at a glance.

Instead, imagine that as a new Git user, you saw this:

A nice big wall with clearly delineated sections for Untracked files and Modified files. Files corresponding to each section are represented as blocks on the wall within that section, clearly labelled with their filename.

More specifically, the blocks representing files one.py, two.py, and three.py are all sitting neatly in the Untracked files section, and the blocks representing the files main.py and settings.py are in the Modified files section.

This makes it abundantly clear to even a total novice that Git is interpreting these files differently and categorizing them in a logical way. It takes the abstract Git concept of the “working directory” and transforms it into a form that almost anyone can wrap their heads around at a glance.

But something is missing here. Let’s say you run the command git add one.py. This stages the untracked file one.py to be included in the next commit. What happens to the block labelled one.py on the wall?

Demystify your Staging Area

To answer that, let’s move on to the mysterious Git “staging area”. But first, where exactly IS the staging area?

Well, technically any staged file changes are still just sitting in the working directory, which makes things a bit confusing.

Here is how Git reports this in the terminal:

Jack@RAPTOR ~/D/git-worlds (main)> git status
On branch main
Your branch is up to date with 'origin/main'.

Changes to be committed:
  (use "git restore --staged <file>..." to unstage)
        new file:   one.py

As you can see from Git’s output, it now includes the section Changes to be committed, which includes the file one.py that was staged with the git add command.

But this is still a bit unclear. Are the staged file changes in one.py still a part of the working directory? Or does Git store them elsewhere?

Well, the answer is… BOTH:

Here you can see that we zoomed out a bit from the previous image, to reveal a third section of the wall labeled Staged files.

Since we ran the command git add one.py, you can see that the corresponding block representing the one.py file moved from the Untracked files column to the Staged files column.

This conveys quite clearly that a file sitting in the staging area is still a part of the working directory (because it is a part of the overall wall), while also being segmented into its own designated space.

From a technical perspective, Git’s staging area is just a file named index which lives in the .git/ folder. Git builds up the code changes specified by the git add command in this file, which is used as the source for those changes the next time the git commit command is run.

But from a workflow perspective, representing the staging area as a section on the “working directory wall” as in the image above makes things more intuitive to understand.

Next, let’s explore how we might visualize things once the staged changes are turned into a new Git commit and become a part of the active branch.

Literally Walk through your Commit History

What does your mind’s eye see when you think of Git’s “commit history”?

Well, the prettiest way Git does it in the terminal is by using the Git log command, such as git log --graph --all, which provides output like:

* commit 88085cff3e2d7657f26eb6479b308526df7d2bba (HEAD -> dev, origin/dev)
| Author: Jacob Stopak <jacob@initialcommit.io>
| Date:   Tue Apr 23 20:31:24 2024 -0700
|
|     Fix command as title clip, ellipses and arrow length in rebase subcommand
|
|     Signed-off-by: Jacob Stopak <jacob@initialcommit.io>
|
*   commit e264605ea26a808c34d4dc2fbc6dad65a8e28c5f
|\  Merge: cb3fa5f b8c071c
| | Author: Jacob Stopak <jacob@initialcommit.io>
| | Date:   Wed Mar 20 19:51:06 2024 -0700
| |
| |     Merge branch 'main' into dev
| |
* | commit cb3fa5f3bdbdcff3d9a8c844cda99d46bf64e337
| | Author: Jacob Stopak <jacob@initialcommit.io>
| | Date:   Sat Mar 9 22:00:49 2024 -0800
| |
| |     Add --staged flag to git restore subcommand
| |
| |     Signed-off-by: Jacob Stopak <jacob@initialcommit.io>
| |
| * commit b8c071cb9a1653748525aa01c2b6bafe06ed9100
|/  Author: Jacob Stopak <jacob@initialcommit.io>
|   Date:   Wed Mar 20 19:50:53 2024 -0700
|
|       Correct license specified in pyproject.toml from MIT to GNU GPLv2
|
|       Signed-off-by: Jacob Stopak <jacob@initialcommit.io>
|
* commit 32a3a3fca583f6c68225b974716e74b557a1a094
| Author: Jacob Stopak <49353917+initialcommit-io@users.noreply.github.com>
| Date:   Tue Aug 22 11:31:38 2023 -0700
|
|     Update README.md

Unfortunately, this is not so pretty at all. This long, garbled list of commit IDs, names, dates, and commit messages is definitely not something most folks would consider user-friendly.

The --graph option supplied above does show the commit relationships by drawing little lines connecting each commit in the terminal, but the purely text-based nature of this is just not intuitive to most people at a glance.

Now consider the following gamified representation of Git’s commit history:

Now we’re talkin’! In this image, each Git commit is represented by a white block with a shortened 6-character commit ID.

Each white commit block points back to its parent commit with an arrow, forming very clear chains of commits that make up Git branches.

You might have noticed that some of the white commit blocks have colored blocks sitting on top of them. The green blocks are branch names, the yellow blocks are Git tags, the blue block is Git’s HEAD pointer, and the red blocks are remote-tracking branches. These are collectively referred to as Git refs.

Besides being able to easily distinguish between them, representing ref types as different colored blocks clarifies another often-confusing Git concept. In Git, branches (along with other refs types) are just “pointers” to a specific commit. It is tempting to think of a branch as a series of connected commits that share a history – and conceptually this is correct – but in Git, a branch is really just a glorified label pointing to a specific commit.

In this gamified world, you can literally walk through your commit history to see, interact with, and examine the code changes in any commit.

Summary

In this article, we explored how Git’s fundamental concepts - the working directory, staging area, and commit history - can be difficult to grasp due to their abstract nature.

To make these concepts more accessible, we introduced a gamified, visual approach that transforms them into something tangible: an immersive game world where files and commits are represented as interactive blocks.

By presenting Git this way, beginner coders, students, and developers of all experience levels can intuitively learn Git concepts and commands, and confidently apply them in professional projects.

Try it Yourself

The images in this post were captured in Devlands, the first and only gamified Git interface and tutorial, which I’m building in Python.

In Devlands, not only can you walk through your codebase… You can also learn Git concepts and commands with a character-guided tutorial, simulate and run Git commands directly in the game, see their results applied in the game world in real time, and use AI to explain code you don’t understand.

If you or someone you know is a visual learner, beginner coder, or newer Git user, consider checking it out!

Simply put, a data structure is a way of organizing and storing data in a computer so that it can be accessed and manipulated reliably. Also, it provides a set of operations such as inserting, deleting, or searching, with their respective computational costs.

Why Are Data Structures Useful?

The value of data structures lies in their ability to approach complex problems in a simpler and more comprehensive way. As they allow for the storage and retrieval of large amounts of data, they become especially important in today's world. Data is being generated at an unprecedented rate. And that data is later processed to extract valuable insights with the goal of developing new products and services that make our lives easier.

Another one of the main advantages of data structures is their ability to reduce the time complexity of certain algorithms. By using the appropriate data structure, you can perform operations such as searching, sorting, and inserting data in sub-linear time.

For example, using a Hash Table instead of a Linked List to store data enables us to access any element of the structure in constant time, reducing the complexity from O(n) to O(1).

In this context, one of the most important aspects of data structures is their ability to organize data in a way that allows for efficient memory usage and reduces the time required to perform specific tasks.

Choosing the right data structure for a large-scale problem is a critical decision that can greatly impact the performance of an algorithm or software application. They also serve as the foundation for many core computer science concepts, such as algorithms, databases, and compilers.

So, in order to fully grasp their significance, you need to understand their inner workings and analyze their strengths and weaknesses.

An example of a Rope data structure. Mainly udes to manage large raw strings. Image gotten from [Wikipedia](https://en.wikipedia.org/wiki/Rope%28datastructure%29)

Different Types of Data Structures

There are numerous types of data structures, including the previously mentioned linked lists, arrays, stacks, queues, trees, and graphs. And each has its own set of use cases and benefits.

What is a Trie Data Structure?

One particularly interesting data structure we will focus on in this article is the Trie, a tree-like data structure employed to store a dynamic set or associative array where the keys are typically strings.

Commonly, we use tries in tasks such as searching and auto-completion, where a large dataset of strings needs to be efficiently stored, checked, or searched.

The Trie is particularly efficient for tasks that involve searching through a large set of strings, as it can perform these tasks with a time complexity of O(m), where m is the length of the searched string. This makes the Trie an attractive choice for applications such as dictionaries, autocomplete features, and IP routing tables.

Despite the numerous advantages offered by Tries, one of the main challenges associated with their implementation is their potential complexity, particularly in their instantiation and visualization.

The visualization of data structures is an essential aspect of understanding their basis and evaluating their performance. But visualizing Tries can become increasingly difficult as the number of nodes and branches grows.

This can lead to confusion and misunderstandings, making it harder for developers and researchers to comprehensively understand the Trie's behavior and throughput. For this reason, this article shows you how to automate the visualization of Trie data structures.

Benefits of Automating the Visualization Process

By automating the visualization process, we unlock a multitude of advantages that can positively impact the overall software development process, transcending the boundaries of traditional visualization techniques.

First and foremost, automation brings with it the gift of efficiency. By eliminating the need for manual visualization and enabling the instantaneous generation of visual representations, we drastically reduce the time spent on creating and understanding Trie diagrams.

This newfound efficiency frees up valuable time for other tasks, allowing us to focus on optimizing and tackling major abstract challenges, ultimately driving progress in the field.

In addition, automation paves the way for consistency and accuracy in visualizing Tries. The human element is prone to errors, leading to misunderstandings and misinterpretations of the underlying data structure. Automating the visualization process removes these potential pitfalls, ensuring that the visual representations generated are trustworthy, precise, and true to the data structure.

Also, automation empowers scalability, enabling the visualization of even the most complex and large-scale datasets. As the size of a Trie increases, so does the difficulty of manual visualization. Automation sidesteps this issue by generating visual representations regardless of the complexity or size of the Trie. This grants researchers and developers an unprecedented level of insight into their systems.

Lastly, automation fosters accessibility and inclusivity in the world of data structure visualization.

Traditionally, creating and interpreting visualizations required a certain level of expertise and experience. But by automating this process, we democratize access to the benefits of visualization. This makes it possible for individuals with varying levels of expertise to analyze and understand Trie data structures.

This, in turn, can inspire collaboration and innovation across a wide range of disciplines and industries.

Here's what we'll cover in this guide:

1. What is a Trie?
2. Main Components of a Trie Data Structure
3. Operations You Can Perform on a Trie
4. How to Implement a Trie
5. How to Visualize a Trie

What is a Trie?

A Trie is a unique and powerful data structure in computer science. It's used to store and manipulate associative arrays or dynamic sets where the keys are typically strings. It is also known as a prefix, radix, or digital tree.

Originating from the term "retrieval," the Trie is a tree-like structure that efficiently organizes and searches through large sets of strings based on their prefixes. This makes it an ideal choice for applications such as dictionaries or spell-checking features.

The Trie comprises nodes connected by branches, each one representing a single character in a string. The root node is usually empty or holds a special value, while the leaf nodes signify the end of a string.

As we traverse from the root node to a leaf node, we construct the string by concatenating the characters represented by each node.

Advantages of the Trie Data Structure

Unlike other tree data structures like binary search trees, Tries don't store the actual keys associated with the nodes. Instead, the position of a node within the Trie structure defines the key.

Example of a Trie containing 4 english words

One of the main advantages of Tries compared to other alternatives like BSTs (Binary Search Trees) is their ability to perform search, insert, and delete operations with a time complexity of O(m). In this case, m is the length of the key (word) being searched, inserted, or deleted.

This is particularly efficient when working with large datasets, as the complexity is independent of the number of keys stored in the Trie. In other words, the performance of these operations does not degrade as the Trie grows in size.

Tries have several other notable features, including:

1. Prefix sharing: A Trie stores strings by sharing their common prefixes, reducing memory usage and providing a compact dataset representation. This feature is particularly beneficial when working with large sets of strings that share many common prefixes.
2. Alphabet-independent size: The size of a Trie is not dependent on the size of the alphabet of the words it contains, making it well-suited for a wide range of applications and datasets.
3. Support for advanced search operations: Tries can perform advanced search operations, such as prefix matching, autocomplete suggestions, and approximate string matching, which are difficult or inefficient to implement using other data structures.
4. Space optimization with compressed Tries: Variations of the Trie data structure, such as Radix Tries, can further optimize memory usage by collapsing multiple nodes with single child nodes into a single node, resulting in a more compact structure.

Still, Tries are not without their drawbacks. One primary concern is their potential space inefficiency. Tries can consume a considerable amount of memory, particularly when working with large alphabets or datasets with few shared prefixes. This issue can be mitigated to some extent through compression techniques and alternative Trie implementations.

In summary, when deciding whether or not to use a Trie, you should consider the specific requirements of your application carefully, the nature of your dataset, and the trade-offs between performance, memory usage, and complexity.

Main Components of a Trie Data Structure

A Trie consists of several fundamental components that work together to compose its basis. Understanding these elements is crucial for solidly comprehending Tries and their full potential.

Here, we will discuss the main parts of a trie, some of them previously mentioned:

Sample Trie to locate and visualize the corresponding elements

1. Nodes: A Trie consists of nodes, each representing a single character in a string. Every Trie has a root node, above colored in red, which is typically empty or holds a special value. The other nodes in the Trie correspond to individual characters of the stored strings.
2. Edges: Edges, or branches, connect the nodes in a Trie. Each edge links a parent node to a child node and represents a transition between two characters in a string. The edges help form the paths from the root node to the leaf nodes, which correspond to complete strings.
3. Leaf Nodes: Leaf nodes (colored in green) are the terminal nodes, signifying the end of a string. Depending on the specific application, they may contain additional information, such as the string's frequency, value, or associated data.
4. Pointers: Each node in a Trie typically contains an array or set of pointers, one for each possible character in the alphabet. These pointers reference the child nodes corresponding to a string's next character. If a pointer is null or is not contained in the set, it indicates that there is no child node for that character.
5. Common Prefixes: One of Trie's defining characteristics is its ability to share common prefixes among stored strings. This feature (optional depending on the applied problem) provides support when performing prefix-based operations and searches.
6. End-of-Word Flag: A marker or end flag is often used to differentiate between complete strings and prefixes. This boolean value is set in a node to indicate that the path from the root node to that node represents a complete string in the Trie.

Operations You Can Perform on a Trie

Now that you've seen the fundamental elements of a Trie, you have a broad idea about their underlying operations. You also understand the trie's value as a data structure.

At this point, you should try to understand technically the complexity of each operation that can be performed on them.

How to Insert a String into a Trie

Imagine the Trie as a tree with many branches, starting from a root node. To insert a string, begin at the root and follow these steps:

• For each character in the string, check if there's an edge (branch) corresponding to the character.
• If there is, move to the child node connected by the edge.
• If not, create a new node for the character, connect it to the current node with an edge, and move to the new node.
• Once you reach the end of the string, mark the final node as the end of a word.

Example: Insert "apple" and "ape" into an empty Trie

How to Search for a String in a Trie

To search for a string in a Trie, start at the root and follow these steps:

• For each character in the string, check if there's an edge corresponding to the character.
• If there is, move to the child node connected by the edge.
• If there isn't, the string is not in the Trie, and the search fails.
• If you reach the end of the string, check if the final node is marked as the end of a word. If it is, the search is successful.

Example: Search for "ape" in the Trie containing "apple" and "ape"

Path followed by the search algorithm to check for the word "ape"

How to Delete a String from a Trie

To delete a string from a Trie, start at the root and follow this process:

• Search for the string, keeping track of the nodes and edges traversed.
• If the string is not found, the deletion fails.
• Remove the end flag from the final node if the string is found.
• Starting from the final node, move back toward the root, deleting nodes and edges that don't have other children or don't belong to other complete words.

Example: Delete "ape" from the Trie containing "apple" and "ape"

Trie after the removal of the word "ape"

By thinking about these operations as traversals through the Trie's branches, we can then more easily account for the memory and time complexity of each one. They are equivalent to tree (or linked list) traversal tasks.

Table of average time and space complexities of Trie operations

As seen in the table, all the operations have the same linear time complexity (being n the length of the word being processed). This means that they only require performing a single traversal across the word's characters to achieve the desired result.

But in terms of space, deletion and search don't need additional memory. This means that they have a constant complexity.

Finally, it is worth pointing out that the worst-case time complexity of the deletion operation could be in the order of O(2n), having to traverse through the branch where the word to be deleted resides once more in order to remove possible unused nodes in the Trie.

How to Implement a Trie

Now we will use Python to implement a Trie specifically designed for a validation problem (but you can tailor it for any given application).

The sample problem our implementation will solve is the following: given a board with NxN squares, we want a user to be able to introduce the coordinates of a specific square using a text message in order to select and operate with it.

For this purpose, the input will consist of a series of alphabetical characters to refer to the board's vertical axis, followed by another set of numerical characters to select the remaining axis.

Sample 6x6 board

In this case, we can use regular expressions or similar techniques to validate if the user's input matches the required structure and format. But if we don't mind our program consuming more memory, we have the alternative of using a Trie.

This choice may, at first glance, seem overly inefficient in terms of memory since we must create and use a specific data structure that consumes space during the entire execution of the program.

But using a Trie provides a constant validation time for all possible inputs, and since in this example the coordinates are short length strings, storing them in memory proves to be reasonable.

Note that in this example, the Trie search operation (validation) runs in "constant" time due to the fixed length of all its words. That is, as the board won't scale up during the program execution, the time complexity for searching a string remains constant.

import random
import math
import urllib, json
from pyvis.network import Network
import matplotlib.pyplot as plt

class TrieNode:
    def __init__(self, inputChar):
        self.char = inputChar
        self.end = False
        self.children = {}

class Trie():
    def __init__(self, startingElements=None):
        self.root = TrieNode("")
        if startingElements!=None:
          for i in startingElements: self.insert(i)
    def insert(self, word):
        node = self.root
        for char in word:
            if char in node.children:
                node = node.children[char]
            else:
                new_node = TrieNode(char)
                node.children[char] = new_node
                node = new_node
        node.end = True      
    def searchAndSplit(self, x):
        node = self.root
        output = ["", ""]
        for char in x:
            if char in node.children:
                node = node.children[char]            
            else:
                return []        
            output[1 if node.end else 0] += node.char
        return output if node.end else []

    def remove(self, root, word, index = 0):
        if not root:
            return None

        if index==len(word):
            if root.end:
                root.end = False
            if root.children=={}:
                del root
                root = None
            return root

        root.children[word[index]] = self.remove(root.children[word[index]], word, index + 1)

        if (root.children=={}) and not root.end:
            del root
            root = None
        return root

Above, you can see the whole implementation. In short, it consists of two classes, TrieNode and Trie, which define the Trie's nodes and Trie structure itself, respectively.

The Trie class provides methods for inserting, searching, splitting (which is a custom feature we will discuss in the visualization section), and removing words from the Trie. The TrieNode class represents individual nodes and their associated attributes. In this case, they are a string containing the corresponding node character and a hash table of characters associated with the pointers to the Trie's subbranches.

How to Visualize a Trie

Finally, after functionally implementing the Trie, we will use the Python library pyvis to define a function inside the Trie class that automatically produces a graph visualization of the data structure. This will result in an HTML file in which we can interact with the elements it contains.

def toGraph(self):      
      g = Network(directed =True)      
      g.show_buttons()

      nodeIndex = 1
      currentNode = 0
      q = [self.root]      
      g.add_node(currentNode, label="", color="red")
      tempLabels = {0:""}
      while q!=[]:
        n = q.pop(0)                     
        for i in n.children.values():
          if i:
            tempLabels[nodeIndex] = tempLabels[currentNode]+i.char
            g.add_node(nodeIndex, label=tempLabels[currentNode]+i.char, color="#48e073" if i.end else "blue")
            g.add_edge(currentNode, nodeIndex)
            nodeIndex+=1
            q.append(i)
        currentNode+=1
      g.show('nx.html')

As you can observe in the toGraph() function, we perform a Breadth-First Search all over the Trie while creating the necessary nodes and edges (connections between them) inside a pyvis.network.Network object.

Also, it gives the end nodes a different color from the rest. This will help us understand the benefits that this visualization provides so we can simplify the execution of functions related to the processing and validation of strings in the previous example.

So to instantiate a Trie with the string coordinates of the example 6x6 board, we will pass a generator Python object to the Trie's constructor to insert each string into the data structure and plot it into a graph:

tr = Trie((''.join(chr(97+int(j)) for j in str(i))+str(k) for k in range(6) for i in range(6)))
tr.toGraph()

According to the coloring rules of the toGraph() function, all nodes containing only an alphabetic character are colored in blue, while all other nodes except the root one are colored in green.

If we increase the board size, we will notice that all the end nodes will have a different color than the nodes with alphabetic characters, as shown in the following graph section of a 16x16 board:

Thus, when performing a search or a BFS traversal, we can take advantage of the end flag to split the coordinates between alphabetic and numeric characters. This avoids an extra post-processing step after validating the user input.

Lastly, we can create a bigger Trie to see how it behaves on a larger scale.

data = json.loads(urllib.request.urlopen("https://raw.githubusercontent.com/dwyl/english-words/master/words_dictionary.json").read())
data = list(data.keys())
tr = Trie(random.sample(data, 100))
tr.toGraph()

By using this code, we are accessing an open-source English words dataset, selecting 100 random words, and inserting them into a new Trie. It gives as visualization output the following graph:

If you would like to work and customize Tries for specific projects interactively, you can access the complete code in the Colab Notebook provided below:

Conclusion

We have seen how visualizing a data structure or a problem input data can significantly enhance our understanding of its relationships and intricacies. It also offers us the chance to introduce new features that otherwise could not have been developed so easily without a proper and automated visioning.

In addition to Tries, the benefits of visualization can be extended to other data structures and algorithms. And it can foster a deeper understanding of the underlying logic and functionality.

Incorporating visualization techniques into the development process can significantly improve the efficiency and creativity of our work in computer science and data analysis.

In conclusion, by embracing visualization as a key component of our development process, we can unlock new opportunities for optimizing, troubleshooting and exploring the full potential of data structures and algorithms in various applications.

About a year ago, I looked high and low for a Python word cloud library that I could use from within my Jupyter notebook. I needed it to be flexible enough to use counts or tfidf when needed or just accept a set of words and corresponding weights.

I was a bit surprised that something like that did not already exist within libraries like plotly. All I wanted to do was to get a quick understanding of my text data and word vectors. I thought that was probably not too much to ask…

Here I am a year later, using my own word_cloud visualization library. Its not the prettiest or the most sophisticated, but it works for most cases. I decided to share it, so that others could use it as well. After installation, here are a few ways you can use it.

Generate word clouds with a single text document

This example show examples of how you can generate word clouds with just one document. While the colors can be randomized, in this example, the colors are based on the default color settings.

By default, the words are weighted by word counts unless you explicitly ask for tfidf weighting. Tfidf weighting makes sense only if you have a lot of documents to start with.

word cloud based on a single document

Generate word clouds from multiple documents

Let’s say you have 100 documents from one news category, and you just want to see what the common mentions are.

word cloud based on multiple documents

Generate word clouds from existing weights

Let’s say you have a set of words with corresponding weights, and you just want to visualize it. All you need to do is make sure that the weights are normalized between [0 - 1].

word cloud from existing weights

Hope you find this useful! Please feel free to propose changes to prettify the output - just open a pull request with your changes.

Links

• See my Jupyter notebook with code examples
• Start using word_cloud library

Install Docker, create your first container, and more

You may have seen Docker somewhere on the internet and you may feel curious about it. Now you want to take it to the next level. How about going on a date with Docker? No — I am not kidding!

But, how do you plan a perfect first date? What should you do? Where to find the right resources? What prerequisites might you need?

The questions are endless. Follow these first-date guidelines, and you’ll be well on your way to landing a second.

Planning your date

Getting to know your date before you start planning is crucial to crafting a memorable first experience. It will likely boost your confidence during the meeting, as well.

It’s never too late to say Hi. Here is a short introduction to my first time meeting Docker if you’re interested.

Okay! So the road map we will follow today looks something like this:

1. Setting up the Venue
2. The Pickup
3. The Conversation

Prelude

The place where we will set up our date is the Docker engine.

Docker engine is no different from any other engine. It is a combination of various components working together.

We will see a few of these components later in this article. But for now, let’s decorate our machine with Docker. The installation process depends upon your operating system.

The key for uninterrupted execution is always to refer to an official website. Docker has this fantastic guide, where you can find out more about installation.

Docker comes in two flavors: Docker Community Edition (CE) and Docker Enterprise Edition (E.E.). For our demonstration purposes, we must stick to the community edition. The Enterprise Edition is paid. And besides, making an expensive reservation for your first date is not the best idea ?

The Venue

The core mantra for any propitious date is to choose the right meeting place. Meeting Docker is no different. First, we should decide on the platform. Cloud? Mac? Linux? Or Windows? Once we know our environment, we can choose the simplest way to set up the meeting place.

Windows

If your choice is Windows, the process for setup is quite simple. Remember our favorite process: Next, Next, Next, and Finish.

Jump to the download page and choose the “Edge channel.” Edge has experimental features and fits best for our learning purposes.

Note! Docker for Windows needs the “Hyper-V” feature, but there is nothing to worry about — Docker will turn it on for you. However, it means that you can’t use Virtual Box.

Linux

When I first started experimenting on a Linux box, I thought it was going to be a nightmare. But to my relief, setting up Docker on a Linux distribution turned out to be simple. Again, there are many ways to install Docker on a Linux distribution.

My favorite is: “Install using the convenience script.” We grab a long shell script from the internet and then pipe it to a shell terminal.

How? Like this:

sudo wget -qO- https://get.docker.com |sh

Done! Yeah, you have Docker on your machine up and running.

Now that the venue is all set, we are ready to proceed. But first, let’s make some inquiries to make sure everything is working fine here. The same as you might inquire with a restaurant before booking a table ?.

Let’s execute the below command in the terminal:

docker version

You should see the installation details. It gives you information about the Client and Server version, the Go-lang version, and so on and so forth.

If the output of your command is somehow like the one below, the sun is shining, and the world is beautiful. Next: time to get ready for the “Pickup.”

Command output

The Pickup

As you may already be aware, Docker uses Images to create containers. These images are large binary immutable files.

An image contains application binaries and their dependencies. It may also contain other components such as a web server or even an OS.

We will need such images to create a container and run an application inside it.

So, where do these images live? Simple question, simple answer: on a Host. This host can be your machine or a registry somewhere on the internet. Docker has its registry called Docker-Hub.

To avoid wandering, you should know the right pickup location for your date.

To make your first date smoother, I found the exact location of an image that I can share with you. Excited enough? Here you go!

docker pull chandrabhan/dotnetconsole

Let’s confirm that we are at the right location. ?

docker image ls

command output

Here you can see various properties of the images on your machine. Cool, it looks like things are going good so far.

Before moving further, let’s break down the previous two commands and understand what’s going on here.

With Docker pull, we asked Docker Daemon to pull an image for us. The Daemon could not find the image in the local registry, so it went ahead and pulled a remote copy from Docker-Hub.

Through docker image ls we saw all available images and their properties.

I must admit, I am getting excited. We’ve done a great job so far. Next, let the interaction begin.

The Conversation

Start the conversation, and things will turn out in your favor.

So again, to make it smoother for you, I want to offer you a few icebreakers.

We will create some containers for .Net Core framework applications. The first one will be a simple console application. The other will be a bit more colorful — a web application.

docker run chandrabhan/dotnetconsole

command output

Magnificent! Did you see what happened there? This stuff is amazing. A .Net core application is running inside a container. Docker went ahead and created a container and executed the application inside it. And as soon as the console application ends, the container exists!

Let’s see how many containers we have in our machine:

docker container ls -a

command output

What you see here is the list of containers on your machine. You can identify a container by its **ID**. The output tells us when the container was **CREATED** and what it’s current **STATUS** is. The container we created is in **Exited** status. Remember? Our application is exited! So is the container.

It looks like the conversation is progressing well, so let’s keep going.

docker run -it --rm -p 5000:80 --name app chandrabhan/aspdotnet

Go to your favorite browser and browse http://localhost:5000. You should see more amazing stuff.

An ASP.Net core web app is running inside a container:

containerized - sample ASP.Net application

Before moving further, let’s see the anatomy of the above command.

command anatomy

Did you notice? This time we did not pull the image, but we were still able to create a container out of it. When you execute docker run, Docker daemon tries to find a local copy of the image. If it cannot, it goes ahead and tries to find it in the default registry (Docker-Hub in this case).

I believe our planning was excellent and so was the execution. Let’s rewind what we did and review how we planned an auspicious date with Docker ?.

Rewinding

1. First, we installed Docker. We saw how the installation process varies depending on the operating system.
2. Then we went ahead and pulled an image from the Docker-Hub registry, a simple .Net Core console app.
3. Once we had this image on our machine, we used Docker to create a container out of this image.
4. We also saw how to pull an image and run a container through a single command.

Here is an illustration that may help you understand a high-level picture.

illustration: Docker client and daemon

Follow up

Out of sight out of mind

Following up is as important as any other stage during the date. Now I should leave you with your date, and would like to hear how it went. Below are a few links that may be useful:

• Github
• Docker hub