The easiest way to publish a python package on PyPI using Poetry

In this post, we will use a new package, poetry to create and manage our pypi package. I will show you how you can create and manage a new package easily using poetry python package and then to make it available online.

Advantages of using poetry

The poetry package can do much more than the traditional environment manage such as Virtualenv. Some of them are:

1. Relocatable: Poetry keeps the virtual enviroment separate from the project. By default, it creates the enviroment at the .cache directory at the home directory. One can easily relocate the environment to another location using the command: poetry env use <new env location>
2. Mutable interpreter: You can also change the interpreter of your environment from, say Python 3.6 to Python 3.9 easily using the command: poetry env use python3.9
3. Isolate the development packages: If you are writing a package for the production purpose, we need to also install extra package for the development purpose. We do not need such package for the production. Poetry allows user to separate them using the -D flag. To add a package for the development only, we can use poetry add -D package-name. Later, when we are ready for the production, we can use the no-dev flag to separate the development packages: poetry install --no-dev.
4. Remove redundant packages: While developing a python package and playing with several features, we end up with many redundant packages. Poetry can help to easily remove such redundancies using poetry install --remove-untracked.
5. Manage version and dependencies using pyproject.toml: The pyproject.toml is equivalent to the requirements.txt file in the traditional virtual environment manager. This stores information about the interpreter (requirements.txt file don’t), and the packages and their dependencies.

Install the package poetry

We will first install the package poetry using pip.

pip install poetry

Create a project

Let us create a project:

poetry new miniseed2mat

This will create the miniseed2mat directory with the following content:

$ tree miniseed2mat/
miniseed2mat/
├── README.rst
├── miniseed2mat
│   └── __init__.py
├── pyproject.toml
└── tests
    ├── __init__.py
    └── test_miniseed2mat.py

2 directories, 5 files

The file pyproject.toml contains:

[tool.poetry]
name = "miniseed2mat"
version = "0.1.0"
description = ""
authors = ["Utpal Kumar <utpalkumar@email.com>"]

[tool.poetry.dependencies]
python = "^3.9"

[tool.poetry.dev-dependencies]
pytest = "^5.2"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

Existing Project

If you already have an existing poetry project, you can initialize it using the command inside the prepopulated directory:

poetry init

Adding more dependencies

You can add more dependencies to your project by specifying them in the tool.poetry.dependencies section of the pyproject.toml file.

[tool.poetry.dependencies]
python = ">=3.9,<3.11"
scipy = "^1.7.3"
obspy = "^1.2.2"

Alternatively, you can install them by running the command:

poetry add scipy obspy

Add a script file

Next, we add a script file main.py in the main directory -> miniseed2mat/miniseed2mat/main.py. You can get this script from the previous post Analyzing MiniSEED seismic data in MATLAB.

We specify the entry point to Poetry by adding the following code snippet to the file pyproject.toml.

[tool.poetry.scripts]
mseed2mat = "miniseed2mat.main:convertmseed2mat"

Here, we call the function convertmseed2mat in the main.py script file.

Build the package

Now, we can build the package using the command:

$ poetry build
Building miniseed2mat (0.1.0)
  - Building sdist
  - Built miniseed2mat-0.1.0.tar.gz
  - Building wheel
  - Built miniseed2mat-0.1.0-py3-none-any.whl

This will create a dist folder inside your project with wheel and tar files of your project.

$ tree dist
dist
├── miniseed2mat-0.1.0-py3-none-any.whl
└── miniseed2mat-0.1.0.tar.gz

0 directories, 2 files

Now, your project is ready to be distributed. You can install it by the command

poetry install

And this will install all the dependencies.

Publish your package to PyPI

Now, we are ready to publish the package to PyPI. However, you need an account on PyPI and create an API token.

$ poetry config pypi-token.pypi <TOKEN> 

$ poetry publish

Publishing miniseed2mat (0.1.0) to PyPI
 - Uploading miniseed2mat-0.1.0-py3-none-any.whl 100%
 - Uploading miniseed2mat-0.1.0.tar.gz 100%

Now, you package should be available to install using pip.

Install and test your package

Now, I will install my package using pip and test in some external directory.

cd ~/Downloads
mkdir testminiseed2mat
pip install miniseed2mat

Now, I create a script:

from miniseed2mat.main import convertmseed2mat
mseedfile = "myStream.mseed"
convertmseed2mat(mseedfile, output_mat=None)

Next, we execute the script:

$ python test_env.py 
Output file: myStream.mat

Add more details about your project

You can add more details about your project in the pyproject.toml file:

[tool.poetry]
name = "miniseed2mat"
version = "0.1.7"
description = "Analyzing MiniSEED seismic data in MATLAB"
homepage="https://www.earthinversion.com/utilities/converting-mseed-data-to-mat-and-analyzing-in-matlab/"
authors = ["Utpal Kumar <utpalkumar50@gmail.com>"]
documentation= "https://github.com/earthinversion/convert-mseed2mat"
keywords=["matlab", "seismic waveforms", "python to matlab", "seismology", "miniseed"]
readme="README.md"

[tool.poetry.dependencies]
python = ">=3.9,<3.11"
scipy = "^1.7.3"
obspy = "^1.2.2"

[tool.poetry.dev-dependencies]
pytest = "^5.2"

[build-system]
requires = ["poetry-core>=1.0.0"]
build-backend = "poetry.core.masonry.api"

[tool.poetry.scripts]
mseed2mat = "miniseed2mat.main:convertmseed2mat"

Updating package

If you want to update any changes in your package, you can first make the changes. Then update the version number in the file pyproject.toml and other locations.

poetry build
poetry publish