A few months ago, I decided to release Caer, a Computer Vision package available in Python. I found the process to be excruciatingly painful. You can probably guess why  — little (and confusing) documentation, lack of good tutorials, and so on.

So I decided to write this article in the hope that it'll help people who are struggling to do this. We’re going to build a very simple module and make it available to anyone around the world.

The contents of this module follow a very basic structure. There are, in total, four Python files, each of which has a single method within it. We’re going to keep this real simple for now.

base-verysimplemodule  --> Base
└── verysimplemodule   --> Actual Module
    ├── extras
    │   ├── multiply.py
    │   ├── divide.py
    ├── add.py
    ├── subtract.py

You will notice that I have a folder called verysimplemodule which, in turn, has two Python files add.py and subtract.py. There is also a folder called extras (which contains multiply.py and divide.py). This folder will form the basis of our Python module.

Bringing out the __init__s

Something that you’ll always find in every Python package is an __init__.py file. This file will tell Python to treat directories as modules (or sub-modules).

Very simply, it will hold the names of all the methods in all the Python files that are in its immediate directory.

A typical __init__.py file has the following format:

from file import method 

# 'method' is a function that is present in a file called 'file.py'

When building packages in Python, you are required to add an __init__.py file in every sub-directory in your package. These sub-directories are the sub-modules of your package.

For our case, we’ll add our __init__.py files to the ‘actual module’ directory verysimplemodule, like this:

from add import add
from subtract import subtract

and we’re going to do the same for the extras folder, like this:

from multiply import multiply
from divide import divide

Once that’s done, we’re pretty much halfway through the process!

How to set up setup.py

Within the base-verysimplemodule folder (and in the same directory as our module verysimplemodule ), we need to add a setup.py file. This file is essential if you intend to build the actual module in question.

Note: Feel free to name the setup.py file as you wish. This file is not name-specific as our __init__.py file is.

Possible name choices are setup_my_very_awesome_python_package.py and python_package_setup.py , but it’s usually best practice to stick with setup.py.

The setup.py file will contain information about your package, specifically the name of the package, its version, platform-dependencies and a whole lot more.

For our purposes, we’re not going to require advanced meta information, so the following code should suit most packages you build:

from setuptools import setup, find_packages

VERSION = '0.0.1' 
DESCRIPTION = 'My first Python package'
LONG_DESCRIPTION = 'My first Python package with a slightly longer description'

# Setting up
       # the name must match the folder name 'verysimplemodule'
        author="Jason Dsouza",
        install_requires=[], # add any additional packages that 
        # needs to be installed along with your package. Eg: 'caer'
        keywords=['python', 'first package'],
        classifiers= [
            "Development Status :: 3 - Alpha",
            "Intended Audience :: Education",
            "Programming Language :: Python :: 2",
            "Programming Language :: Python :: 3",
            "Operating System :: MacOS :: MacOS X",
            "Operating System :: Microsoft :: Windows",

With that done, all we have to do next is run the following command in the same directory as base-verysimplemodule:

python setup.py sdist bdist_wheel

This will build all the necessary packages that Python will require. The sdist and bdist_wheel commands will create a source distribution and a wheel that you can later upload to PyPi.

PyPi — here we come!

PyPi is the official Python repository where all Python packages are stored. You can think of it as the Github for Python Packages.

To make your Python package available to people around the world, you’ll need to have an account with PyPi.

With that done, we’re all set to upload our package on PyPi. Remember the source distribution and wheel that were built when we ran python setup.py ? Well, those are what will actually be uploaded to PyPi.

But before you do that, you need to install twine if you don’t already have it installed. It’s as simple as  pip install twine.

How to upload your package to PyPi

Assuming you have twine installed, go ahead and run:

twine upload dist/*

This command will upload the contents of the dist folder that was automatically generated when we ran python setup.py. You will get a prompt asking you for your PyPi username and password, so go ahead and type those in.

Now, if you’ve followed this tutorial to the T, you might get an error along the lines of repository already exists.

This is usually because there is a name clash between the name of your package and a package that already exists. In other words, change the name of your package — somebody else has already taken that name.

And that’s it!

To proudly pip install your module, fire up a terminal and run:

pip install <package_name> 

# in our case, this is
pip install verysimplemodule

Watch how Python neatly installs your package from the binaries that were generated earlier.

Open up a Python interactive shell and try importing your package:

>> import verysimplemodule as vsm

>> vsm.add(2,5)
>> vsm.subtract(5,4)

To access the division and multiplication methods (remember that they were in a folder called extras ?), run:

>> import verysimplemodule as vsm

>> vsm.extras.divide(4,2)
>> vsm.extras.multiple(5,3)

It’s as simple as that.

Congratulations! You’ve just built your first Python package. Albeit very simple, your package is now available to be downloaded by anyone around the world (so long as they have Python, of course).

What's next?

Test PyPi

The package that we used in this tutorial was an extremely simple module — basic mathematical operations of addition, subtraction, multiplication and division. It doesn’t make sense to upload them directly to PyPi especially since you’re trying this out for the first time.

Lucky for us, there is Test PyPi, a separate instance of PyPi where you can test out and experiment on your package (you will need to sign up for a separate account on the platform).

The process that you follow to upload to Test PyPi is pretty much the same with a few minor changes.

# The following command will upload the package to Test PyPi
# You will be asked to provide your Test PyPi credentials

twine upload --repository testpypi dist/*

To download projects from Test PyPi:

pip install --index-url "https://test.pypi.org/simple/<package_name>"

Advanced Meta Information

The meta information we used in the setup.py file was very basic. You can add additional information such as multiple maintainers (if any), author email, license information and a whole host of other data.

This article will prove particularly helpful if you intend to do so.

Look at other repositories

Looking at how other repositories have built their packages can prove to be super useful to you.

When building Caer, I would constantly look at how Numpy and Sonnet set up their packages. I would recommend taking a look at Caer’s, Numpy’s, and Tensorflow’s repositories if you plan on building slightly more advanced packages.