A few months ago, I decided to release Caer, a Computer Vision package available in Python. I found the process to be excruciatingly painful. You can probably guess why — little (and confusing) documentation, lack of good tutorials, and so on.
So I decided to write this article in the hope that it'll help people who are struggling to do this. We’re going to build a very simple module and make it available to anyone around the world.
The contents of this module follow a very basic structure. There are, in total, four Python files, each of which has a single method within it. We’re going to keep this real simple for now.
base-verysimplemodule --> Base └── verysimplemodule --> Actual Module ├── extras │ ├── multiply.py │ ├── divide.py ├── add.py ├── subtract.py
You will notice that I have a folder called
verysimplemodule which, in turn, has two Python files
subtract.py. There is also a folder called
extras (which contains
divide.py). This folder will form the basis of our Python module.
Bringing out the __init__s
Something that you’ll always find in every Python package is an
__init__.py file. This file will tell Python to treat directories as modules (or sub-modules).
Very simply, it will hold the names of all the methods in all the Python files that are in its immediate directory.
__init__.py file has the following format:
from file import method # 'method' is a function that is present in a file called 'file.py'
When building packages in Python, you are required to add an
__init__.py file in every sub-directory in your package. These sub-directories are the sub-modules of your package.
For our case, we’ll add our __init__.py files to the ‘actual module’ directory
verysimplemodule, like this:
from add import add from subtract import subtract
and we’re going to do the same for the
extras folder, like this:
from multiply import multiply from divide import divide
Once that’s done, we’re pretty much halfway through the process!
How to set up setup.py
base-verysimplemodule folder (and in the same directory as our module
verysimplemodule ), we need to add a
setup.py file. This file is essential if you intend to build the actual module in question.
Note: Feel free to name the
setup.py file as you wish. This file is not name-specific as our
__init__.py file is.
Possible name choices are
python_package_setup.py , but it’s usually best practice to stick with
setup.py file will contain information about your package, specifically the name of the package, its version, platform-dependencies and a whole lot more.
For our purposes, we’re not going to require advanced meta information, so the following code should suit most packages you build:
from setuptools import setup, find_packages VERSION = '0.0.1' DESCRIPTION = 'My first Python package' LONG_DESCRIPTION = 'My first Python package with a slightly longer description' # Setting up setup( # the name must match the folder name 'verysimplemodule' name="verysimplemodule", version=VERSION, author="Jason Dsouza", author_email="<firstname.lastname@example.org>", description=DESCRIPTION, long_description=LONG_DESCRIPTION, packages=find_packages(), install_requires=, # add any additional packages that # needs to be installed along with your package. Eg: 'caer' keywords=['python', 'first package'], classifiers= [ "Development Status :: 3 - Alpha", "Intended Audience :: Education", "Programming Language :: Python :: 2", "Programming Language :: Python :: 3", "Operating System :: MacOS :: MacOS X", "Operating System :: Microsoft :: Windows", ] )
With that done, all we have to do next is run the following command in the same directory as
python setup.py sdist bdist_wheel
This will build all the necessary packages that Python will require. The
bdist_wheel commands will create a source distribution and a wheel that you can later upload to PyPi.
PyPi — here we come!
PyPi is the official Python repository where all Python packages are stored. You can think of it as the Github for Python Packages.
To make your Python package available to people around the world, you’ll need to have an account with PyPi.
With that done, we’re all set to upload our package on PyPi. Remember the source distribution and wheel that were built when we ran
python setup.py ? Well, those are what will actually be uploaded to PyPi.
But before you do that, you need to install
twine if you don’t already have it installed. It’s as simple as
pip install twine.
How to upload your package to PyPi
Assuming you have
twine installed, go ahead and run:
twine upload dist/*
This command will upload the contents of the
dist folder that was automatically generated when we ran
python setup.py. You will get a prompt asking you for your PyPi username and password, so go ahead and type those in.
Now, if you’ve followed this tutorial to the T, you might get an error along the lines of repository already exists.
This is usually because there is a name clash between the name of your package and a package that already exists. In other words, change the name of your package — somebody else has already taken that name.
And that’s it!
pip install your module, fire up a terminal and run:
pip install <package_name> # in our case, this is pip install verysimplemodule
Watch how Python neatly installs your package from the binaries that were generated earlier.
Open up a Python interactive shell and try importing your package:
>> import verysimplemodule as vsm >> vsm.add(2,5) 7 >> vsm.subtract(5,4) 1
To access the division and multiplication methods (remember that they were in a folder called
extras ?), run:
>> import verysimplemodule as vsm >> vsm.extras.divide(4,2) 2 >> vsm.extras.multiple(5,3) 15
It’s as simple as that.
Congratulations! You’ve just built your first Python package. Albeit very simple, your package is now available to be downloaded by anyone around the world (so long as they have Python, of course).
The package that we used in this tutorial was an extremely simple module — basic mathematical operations of addition, subtraction, multiplication and division. It doesn’t make sense to upload them directly to PyPi especially since you’re trying this out for the first time.
Lucky for us, there is Test PyPi, a separate instance of PyPi where you can test out and experiment on your package (you will need to sign up for a separate account on the platform).
The process that you follow to upload to Test PyPi is pretty much the same with a few minor changes.
# The following command will upload the package to Test PyPi # You will be asked to provide your Test PyPi credentials twine upload --repository testpypi dist/*
To download projects from Test PyPi:
pip install --index-url "https://test.pypi.org/simple/<package_name>"
Advanced Meta Information
The meta information we used in the
setup.py file was very basic. You can add additional information such as multiple maintainers (if any), author email, license information and a whole host of other data.
This article will prove particularly helpful if you intend to do so.
Look at other repositories
Looking at how other repositories have built their packages can prove to be super useful to you.
When building Caer, I would constantly look at how Numpy and Sonnet set up their packages. I would recommend taking a look at Caer’s, Numpy’s, and Tensorflow’s repositories if you plan on building slightly more advanced packages.