Building and Packaging

What a build actually is

Two commands, no magic:

1. Compile your .c files with the include directory that holds Python.h on the search path.
2. Link them into a shared library with a platform-specific name, and on Windows, against python3XX.lib.

On Linux and macOS you deliberately do not link against libpython: the extension is loaded into a process that already contains the interpreter's symbols, and they are resolved at load time. On Windows the loader requires every import to be resolved at link time, so you link the import library. Build tools handle this difference for you; know it because it explains error messages.

You can see everything the build needs from Python itself:

import sysconfig
print(sysconfig.get_paths()["include"])        # where Python.h lives
print(sysconfig.get_config_var("EXT_SUFFIX"))  # e.g. .cp314-win_amd64.pyd
print(sysconfig.get_config_var("Py_GIL_DISABLED"))   # 1 on a free-threaded build
print(sysconfig.get_platform())

The minimal project

mymod/
├── pyproject.toml
├── setup.py
├── src/
│   └── mymod.c
└── tests/
    └── test_mymod.py

# pyproject.toml
[build-system]
requires = ["setuptools>=77"]
build-backend = "setuptools.build_meta"

[project]
name = "mymod"
version = "0.1.0"
description = "Example C extension"
requires-python = ">=3.13"

[tool.pytest.ini_options]
testpaths = ["tests"]

PEP 517/518 metadata. The build backend is declared here; pip installs it into an isolated environment before building.

# setup.py -- still the right place for anything computed
import sysconfig
from setuptools import Extension, setup

# The Limited API and the free-threaded build are mutually exclusive today.
free_threaded = bool(sysconfig.get_config_var("Py_GIL_DISABLED"))

setup(
    ext_modules=[
        Extension(
            name="mymod",                       # must match PyInit_mymod
            sources=["src/mymod.c"],
            include_dirs=[],                    # extra headers
            libraries=[],                       # e.g. ["z"] for -lz
            library_dirs=[],
            define_macros=[("MYMOD_VERSION", '"0.1.0"')],
            extra_compile_args=[],
            py_limited_api=not free_threaded,
        ),
    ],
)

A setup.py containing only setup() is not deprecated; running python setup.py directly is. Invoke it through pip or build.

Recent setuptools also supports declaring simple extensions purely in pyproject.toml, which is convenient when nothing needs to be computed:

[tool.setuptools]
ext-modules = [
  {name = "mymod", sources = ["src/mymod.c"]}
]

The build/test loop

Reading the output filename

The extension suffix encodes the compatibility contract:

And the wheel tags mirror it: mymod-0.1.0-cp314-cp314-win_amd64.whl (one Python version), mymod-0.1.0-cp313-abi3-win_amd64.whl (3.13 and later), mymod-0.1.0-cp314-cp314t-manylinux_2_28_x86_64.whl (free-threaded).

The Limited API and the stable ABI

By default, an extension is compiled against the full CPython API, which includes macros that read struct fields at fixed offsets. Those offsets change between minor versions, so the resulting binary is version-locked — a new wheel for every Python release.

The Limited API is a documented subset that avoids anything layout-dependent. Opt in before including the header:

#define Py_LIMITED_API 0x030A0000   /* minimum: Python 3.10 */
#include <Python.h>

The matching binary contract is the stable ABI, marketed as abi3: one wheel loads on every CPython from that minimum version forward. In setup.py that is py_limited_api=True on the Extension plus a wheel tag:

# setup.cfg
[bdist_wheel]
py-limited-api = cp310

Two honest caveats. First, Py_LIMITED_API constrains declarations, not semantics: a function whose behaviour on a NULL argument changed between versions will still surprise you, so test on every version you claim to support. Second, Python does not verify the claim — it will happily load a file named .abi3.so that uses non-stable API, and then crash.

Free-threaded builds

The free-threaded interpreter (Lesson 11) is a genuinely different ABI. What the build needs to know:

• Detect it in C with #ifdef Py_GIL_DISABLED, and in Python with sysconfig.get_config_var("Py_GIL_DISABLED").
• You must build separate wheels (cp314t). The stable ABI is not available.
• On Windows, from Python 3.14 the build backend must define Py_GIL_DISABLED=1 explicitly when targeting the free-threaded interpreter; the compiler no longer infers it. Current setuptools does this, but if you drive the compiler yourself, add it.
• cibuildwheel and the manylinux images support free-threaded targets; enabling them is a configuration flag, not a porting project.

Platform notes

Windows

You need the MSVC toolchain that matches your interpreter. CPython 3.14 from python.org is built with Visual Studio 2022 (MSC v.194x); install "Desktop development with C++" from the Visual Studio Build Tools. Then everything is ordinary: pip install -e . finds the compiler through setuptools' registry lookup. The common failure, error: Microsoft Visual C++ 14.0 or greater is required, means exactly what it says.

Useful MSVC flags via extra_compile_args: /O2 (default for release), /W4, /WX, /arch:AVX2. Note that MSVC does not accept the GCC spellings, so guard them:

import sys
warn_flags = ["/W4"] if sys.platform == "win32" else ["-Wall", "-Wextra"]

Linux

You need the development headers: python3-dev (Debian/Ubuntu) or python3-devel (Fedora/RHEL). The absence of them produces fatal error: Python.h: No such file or directory, which is the single most common build failure in the ecosystem. For redistributable wheels, build inside a manylinux container so the glibc symbol versions are old enough.

macOS

Xcode command line tools provide clang. Watch MACOSX_DEPLOYMENT_TARGET and universal2 (arm64 + x86_64) builds if you distribute.

Other build backends

setuptools is fine for a handful of C files. When the native side grows, two alternatives are worth knowing:

meson-python

PEP 517 backend driving Meson. Fast, good dependency detection, real incremental builds. Used by SciPy and NumPy.

scikit-build-core

PEP 517 backend driving CMake. The right answer when you are wrapping a C/C++ library that already builds with CMake.

Both are drop-in from the consumer's point of view — pip install . still works — because the interface is pyproject.toml's build-backend.

When the import fails

Debugging natively

# Linux/macOS: run the whole interpreter under a debugger
gdb --args python -c "import mymod; mymod.crash()"
lldb -- python -c "import mymod; mymod.crash()"

# Address sanitizer, with Python's own allocator out of the way
CFLAGS="-fsanitize=address -g -O1" pip install -e . --no-build-isolation
PYTHONMALLOC=malloc ASAN_OPTIONS=detect_leaks=0 python -m pytest

# Reference-count checking needs a debug interpreter
python -X showrefcount -c "import mymod; mymod.f(1)"

On Windows, build a debug configuration and attach Visual Studio to python.exe; note that a debug extension must be loaded by python_d.exe, since the C runtime differs.

cd code/hello
python -m pip install -e . --no-build-isolation -v
python -c "import hello; print(hello.add_one(41)); print(hello.__file__)"
python -m pytest -q

Further reading

• setuptools — Building Extension Modules. Every Extension() keyword.
• C API Stability. The authoritative statement on the Limited API and stable ABI.
• Python Packaging Guide — Binary Extensions.
• cibuildwheel. Builds and tests wheels for every platform and interpreter in CI, including free-threaded targets.
• pypackaging-native. A clear-eyed survey of why native packaging in Python is hard and what the current options are.