Module Mechanics and Glue Code

What import does to a shared library

1. The import machinery finds a file matching the module name — on CPython 3.14, something like mymod.cpython-314-x86_64-linux-gnu.so, mymod.cp314-win_amd64.pyd, or the ABI-neutral mymod.abi3.so.
2. It loads the shared library (dlopen / LoadLibrary).
3. It looks up one exported symbol: PyInit_<name>, where <name> is the last dotted component. For pkg.sub.mymod, the symbol is PyInit_mymod.
4. It calls that function and expects back either a module object (legacy single-phase) or a PyModuleDef object (modern multi-phase), which it then turns into a module.

PyMODINIT_FUNC

PyMODINIT_FUNC expands to PyObject * plus whatever visibility attributes the platform needs — __declspec(dllexport) on Windows, default visibility elsewhere. Use it and never write the return type by hand, or your symbol may not be exported and the import will fail with the message above even though the code is right.

Single-phase versus multi-phase initialization

The original design had PyInit_foo create the module and populate it, all in one call. PEP 489 split that into two phases, and the split is what makes modern isolation possible.

Everything below is multi-phase. There is no good reason to write a new single-phase module.

PyModuleDef, field by field

typedef struct PyModuleDef {
    PyModuleDef_Base   m_base;      /* always PyModuleDef_HEAD_INIT */
    const char        *m_name;      /* "mymod" */
    const char        *m_doc;       /* module docstring, or NULL */
    Py_ssize_t         m_size;      /* bytes of per-module state; 0 if none */
    PyMethodDef       *m_methods;   /* module-level functions, or NULL */
    PyModuleDef_Slot  *m_slots;     /* multi-phase slots; NULL means single-phase */
    traverseproc       m_traverse;  /* GC: visit PyObject* in module state */
    inquiry            m_clear;     /* GC: clear PyObject* in module state */
    freefunc           m_free;      /* release non-Python resources in state */
} PyModuleDef;

m_size is the field that matters most:

• m_size = 0 — no per-module state. Fine for a module of pure functions that hold nothing.
• m_size = sizeof(my_state) — CPython allocates and zeroes that many bytes per module instance, reachable via PyModule_GetState.
• m_size = -1 — "this module keeps its state in C globals". Legal only with single-phase init, and the thing multi-phase exists to eliminate.

The slots

Per-module state, and why globals are wrong

The natural C instinct is a file-scope static:

static PyObject *MyError;          /* WRONG */
static PyTypeObject *CounterType;  /* WRONG */

There is one of those per process, but a module can be instantiated more than once per process:

• Subinterpreters. interpreters.create() (PEP 734, stdlib in 3.14) and embedding applications both create independent interpreters. Two interpreters sharing one PyObject * global means objects crossing interpreter boundaries — a crash waiting to happen.
• Reinitialization. Py_Initialize/Py_Finalize cycles in an embedding host leave your statics pointing at freed objects.
• Reloading. importlib.reload on your module gives the state no chance to reset.

The fix is to put everything in a struct attached to the module object:

typedef struct {
    PyObject *Error;        /* our exception type    */
    PyObject *CounterType;  /* our heap type         */
    PyObject *cache;        /* a dict we keep around */
} mymod_state;

static inline mymod_state *
get_state(PyObject *module)
{
    void *s = PyModule_GetState(module);
    assert(s != NULL);
    return (mymod_state *)s;
}

And because that struct holds PyObject * fields, the module participates in cycle collection exactly like a type does (Lesson 04):

static int
mymod_traverse(PyObject *module, visitproc visit, void *arg)
{
    mymod_state *st = get_state(module);
    Py_VISIT(st->Error);
    Py_VISIT(st->CounterType);
    Py_VISIT(st->cache);
    return 0;
}

static int
mymod_clear(PyObject *module)
{
    mymod_state *st = get_state(module);
    Py_CLEAR(st->Error);
    Py_CLEAR(st->CounterType);
    Py_CLEAR(st->cache);
    return 0;
}

static void
mymod_free(void *module)
{
    (void)mymod_clear((PyObject *)module);   /* plus any malloc'd resources */
}

Reaching the state from three places

Populating the module

The complete skeleton

This is the template. Every line is load-bearing.

#include <Python.h>

/* ---- per-module state ------------------------------------------------- */

typedef struct {
    PyObject *Error;
    PyObject *CounterType;
} mymod_state;

static inline mymod_state *get_state(PyObject *m) {
    void *s = PyModule_GetState(m);
    assert(s != NULL);
    return (mymod_state *)s;
}

/* ---- a module-level function ------------------------------------------ */

static PyObject *
mymod_scale(PyObject *module, PyObject *const *args, Py_ssize_t nargs)
{
    if (nargs != 2) {
        PyErr_SetString(PyExc_TypeError, "scale() takes exactly 2 arguments");
        return NULL;
    }
    double x = PyFloat_AsDouble(args[0]);
    if (x == -1.0 && PyErr_Occurred()) return NULL;
    double k = PyFloat_AsDouble(args[1]);
    if (k == -1.0 && PyErr_Occurred()) return NULL;

    if (k == 0.0) {
        mymod_state *st = get_state(module);       /* self IS the module */
        PyErr_SetString(st->Error, "scale factor must be non-zero");
        return NULL;
    }
    return PyFloat_FromDouble(x * k);
}

static PyMethodDef mymod_methods[] = {
    {"scale", (PyCFunction)(void(*)(void))mymod_scale, METH_FASTCALL,
     PyDoc_STR("scale(x, k, /)\n--\n\nReturn x * k.")},
    {NULL, NULL, 0, NULL}
};

/* ---- GC hooks for the state ------------------------------------------- */

static int mymod_traverse(PyObject *m, visitproc visit, void *arg) {
    mymod_state *st = get_state(m);
    Py_VISIT(st->Error);
    Py_VISIT(st->CounterType);
    return 0;
}
static int mymod_clear(PyObject *m) {
    mymod_state *st = get_state(m);
    Py_CLEAR(st->Error);
    Py_CLEAR(st->CounterType);
    return 0;
}
static void mymod_free(void *m) { (void)mymod_clear((PyObject *)m); }

/* ---- phase 2: populate the module ------------------------------------- */

static int
mymod_exec(PyObject *module)
{
    mymod_state *st = get_state(module);

    st->Error = PyErr_NewExceptionWithDoc("mymod.Error", "mymod errors.", NULL, NULL);
    if (st->Error == NULL) return -1;
    if (PyModule_AddObjectRef(module, "Error", st->Error) < 0) return -1;

    /* A heap type bound to THIS module instance (Lesson 03). */
    st->CounterType = PyType_FromModuleAndSpec(module, &Counter_spec, NULL);
    if (st->CounterType == NULL) return -1;
    if (PyModule_AddObjectRef(module, "Counter", st->CounterType) < 0) return -1;

    if (PyModule_AddIntConstant(module, "VERSION", 3) < 0) return -1;
    return 0;
}

static PyModuleDef_Slot mymod_slots[] = {
    {Py_mod_exec, mymod_exec},
    {Py_mod_multiple_interpreters, Py_MOD_PER_INTERPRETER_GIL_SUPPORTED},
#if PY_VERSION_HEX >= 0x030D0000
    {Py_mod_gil, Py_MOD_GIL_NOT_USED},           /* Lesson 11 */
#endif
    {0, NULL}
};

static PyModuleDef mymod_def = {
    .m_base     = PyModuleDef_HEAD_INIT,
    .m_name     = "mymod",
    .m_doc      = PyDoc_STR("Example extension module."),
    .m_size     = sizeof(mymod_state),
    .m_methods  = mymod_methods,
    .m_slots    = mymod_slots,
    .m_traverse = mymod_traverse,
    .m_clear    = mymod_clear,
    .m_free     = mymod_free,
};

/* ---- phase 1: the entry point ----------------------------------------- */

PyMODINIT_FUNC
PyInit_mymod(void)
{
    return PyModuleDef_Init(&mymod_def);
}

A complete modern module. code/counter/ in this course is a runnable version with a heap type filled in.

Rules for the exec function

• It can be called more than once per process (once per interpreter, and again after a reload). It must not depend on C globals or on having run before.
• Return −1 with an exception set on failure. Do not attempt to undo partial work; the module object is discarded and m_free runs.
• State memory is zeroed before exec runs, so every pointer starts NULL and m_clear is safe even if exec fails halfway.
• m_traverse, m_clear and m_free are only called when state was actually allocated (m_size > 0).

Packages, submodules, and naming

An extension inside a package is just a module whose full dotted name ends in the extension's name. If you build mypkg._speedups, the file lands next to mypkg/__init__.py and the entry point is PyInit__speedups — note the two underscores, one from the naming convention and one from the prefix. m_name should be the full dotted name "mypkg._speedups", and tp_name for types inside it likewise "mypkg._speedups.Counter".

The common layout is a thin Python package that imports from a private C module, so you can keep pure-Python conveniences (docstrings, __repr__ helpers, a fallback implementation) in Python:

# mypkg/__init__.py
try:
    from ._speedups import scale, Counter
except ImportError:                       # pure-Python fallback
    from ._pure import scale, Counter
__all__ = ["scale", "Counter"]

On the horizon: PEP 793

PEP 793 introduces PyModExport, an alternative entry point that returns an array of slots directly, with no statically allocated PyModuleDef object at all. Its main motivation is letting a single compiled file work with both the default and the free-threaded build. It targets Python 3.15, and PyInit_* becomes soft-deprecated — fully supported and documented, but not gaining new features. Nothing in this lesson becomes wrong; be aware that a second spelling is arriving.

import sys, math, importlib.util

# Extension modules have no __file__-based source and a different spec origin.
print(math.__spec__)
print(math.__spec__.origin)

# Which stdlib modules are C extensions rather than Python?
print(sorted(sys.builtin_module_names)[:15])

# Multi-phase modules report a real loader that participates in the spec protocol:
spec = importlib.util.find_spec("_socket")
print(spec.loader)

# Subinterpreters: a module with C globals is what breaks here (3.14 stdlib).
from concurrent import interpreters
interp = interpreters.create()
interp.exec("import math; print('ok in subinterpreter', math.pi)")

Further reading

• C API — Module Objects. PyModuleDef, every slot, and the PyModule_Add* family.
• Isolating Extension Modules. The definitive how-to for module state and heap types. Read this one twice.
• PEP 489 — Multi-phase extension module initialization.
• PEP 573 — Module State Access from C Extension Methods. Where defining_class came from.
• PEP 793 — PyModExport. The 3.15 entry point.
• Extending Python with C or C++. The official tutorial, still the fastest way to see the whole shape.