A Python Package Index (PyPI) is a software repository that stores distribution packages, a description of the package, their metadata and other publishing tools. For experimental reasons it could be necessary to use an internal PyPI, e.g. if there is no possibility to set up a PyPI via GitLab. This post describes how a PyPI simulation can be realized.

Use an Internal Python Package Index (PyPI)

pypiserver is used for this implementation. It is a minimal PyPI compatible server for pip. It offers the same interface as PyPI, allowing standard Python packaging tooling like pip to interact with it as a package index. Also it offers the possibility to use it with an action inside VCF Automation. This is an equivalent approach to the one already described in the post how to mock a web server.

PyPI Server

First the pypiserver is downloaded and then handler action must be coded, which loads the pypiserver and executes its main function. The necessary arguments are passed at the start.

import json
import os.path
import sys
from pypiserver.__main__ import main

def handler(context: dict, inputs: dict) -> dict:

    outputs: dict = {}

    try:

        path: str = os.path.dirname(os.path.abspath(__file__))
        sys.argv[1] = ("run")
        sys.argv.append("-p")
        sys.argv.append("3141")
        sys.argv.append("--welcome")
        sys.argv.append(path + "/welcome.html")
        sys.argv.append(path  + "/packages")
        main()

        outputs = {
            "status": "done",
            "error": None
        }

    except Exception as err:

        outputs = {
            "status": "incomplete",
            "error": repr(err)
        }

    return outputs

Now everything is packed into a zip file. The required distribution packages are added to the package directory. This zip file can be loaded as an action in VCF Automation. The timeout value should be set slightly higher, to give a little more time to experiment.

pypi zip file

Distribution Package Example

A distribution package is an archive file that contains Python packages, modules and other resource files. In this example, beside hello world, calc is used, which contains functions for the four basic arithmetic operations. How to build a distribution package is explained in detail at Python Packaging User Guide.

def add(a,b):
    return a+b

def subtract(a,b):
    return a-b

def multiply(a,b):
    return a * b

def divide(a, b):
    return a / b

PyPI Server Test

Subprocess.run calls the pip command, which makes the packages available to the action. The functions can be used after import.

import json
import socket
import subprocess

def handler(context: dict, inputs: dict) -> dict:

    ipAddress: str = socket.gethostbyname(
        socket.gethostname()
    )

    subprocess.run(["pip", "install", \
        "--index", "http://" + ipAddress + ":3141/simple/", \
        "--trusted-host", ipAddress, \
        "calc==1.0"])

    subprocess.run(["pip", "install", \
        "--index", "http://" + ipAddress + ":3141/simple/", \
        "--trusted-host", ipAddress, \
        "pypi_hello_world"])

    import calc
    import pypi_hello_world

    print(calc.add(1, 2))
    print(calc.subtract(16, 6))
    print(calc.multiply(6, 6))
    print(calc.divide(64, 8))

    pypi_hello_world.hello()

    outputs: dict = {
        "status": "done"
    }

    return outputs

vcf automation pypi test action log window

PyPI Server Connect

A simple connection returns the Welcome.html site.

import socket
import json
from urllib.request import Request, urlopen

def handler(context: dict, inputs: dict) -> dict:

    ipAddress: str = socket.gethostbyname(
        socket.gethostname()
    )

    httpRequest = Request(
        url = "http://" + ipAddress + ":3141/"
    )

    with urlopen(httpRequest) as response:
        print(response.read())

    outputs: dict = {
        "status": "done"
    };

    return outputs

Conclusion

It is very easy to provide an internal PyPI. This approach opens the possibility to work with Python distribution packages independently of other components.