Functions and Argument Parsing

PyMethodDef: the table that exposes C to Python

Whether a C function becomes a module-level function or a method on a type, it is registered through the same struct:

typedef struct PyMethodDef {
    const char  *ml_name;   /* the Python-visible name */
    PyCFunction  ml_meth;   /* the C function pointer  */
    int          ml_flags;  /* calling convention      */
    const char  *ml_doc;    /* the docstring           */
} PyMethodDef;

Arrays of these are NULL-terminated and are assigned to m_methods on a module or tp_methods/Py_tp_methods on a type. The first parameter of every C function in the table is conventionally called self, and it is:

• the module object, for a function in m_methods;
• the instance, for a method in tp_methods;
• the type, if METH_CLASS is set;
• NULL, if METH_STATIC is set.

This is worth pausing on: for a module-level function, self is your module, which is how you reach per-module state in Lesson 08 without any globals.

Calling conventions

Modifier flags that combine with the above: METH_CLASS (classmethod), METH_STATIC (staticmethod), METH_COEXIST (allow this method to take precedence over a slot-generated wrapper of the same name).

Choosing

• One argument, no keywords → METH_O.
• No arguments → METH_NOARGS.
• A few positional arguments on a hot path → METH_FASTCALL.
• Anything with keyword arguments and a complex signature → METH_VARARGS | METH_KEYWORDS with PyArg_ParseTupleAndKeywords, unless you are using Argument Clinic (below), in which case take the fastcall version it generates.

PyArg_ParseTuple and friends

The parsing functions take a format string describing the expected arguments and a list of pointers to fill in. They return true on success, and on failure they return false with a suitable TypeError or ValueError already raised — so the caller just returns NULL.

static PyObject *
resize(PyObject *self, PyObject *args, PyObject *kwargs)
{
    static char *kwlist[] = {"width", "height", "mode", NULL};
    int width, height;
    const char *mode = "nearest";     /* default for the optional argument */

    if (!PyArg_ParseTupleAndKeywords(args, kwargs, "ii|s:resize", kwlist,
                                     &width, &height, &mode)) {
        return NULL;
    }
    ...
}

The kwlist array must have exactly one entry per format unit, in order, terminated by NULL. An empty string entry marks a positional-only parameter.

Format-string markers

The format units you will actually use

Custom converters with O&

O& calls a function of yours to turn the object into whatever C value you want, which keeps validation out of the function body:

/* Returns 1 on success, 0 on failure with an exception set.
   Return Py_CLEANUP_SUPPORTED instead of 1 to opt into cleanup calls. */
static int
convert_mode(PyObject *obj, void *addr)
{
    const char *s = PyUnicode_AsUTF8(obj);
    if (s == NULL) return 0;
    if (strcmp(s, "fast") == 0)      { *(int *)addr = MODE_FAST; return 1; }
    if (strcmp(s, "accurate") == 0)  { *(int *)addr = MODE_ACC;  return 1; }
    PyErr_Format(PyExc_ValueError, "unknown mode %R", obj);
    return 0;
}

/* usage: */
int mode = MODE_FAST;
PyArg_ParseTuple(args, "O&:go", convert_mode, &mode);

The lightweight alternative

When you only need "between 1 and 3 objects, no conversion", skip the format string:

PyObject *obj, *callback = NULL;
if (!PyArg_UnpackTuple(args, "ref", 1, 2, &obj, &callback)) return NULL;
/* both are borrowed references */

Argument Clinic

Writing PyArg_ParseTupleAndKeywords by hand is repetitive, is slower than it needs to be, and produces signatures that help() and inspect.signature cannot read. CPython's own solution is Argument Clinic, a preprocessor: you write a declarative signature in a comment block, run a tool, and it generates a fast keyword parser, the docstring, and the __text_signature__ that makes introspection work.

/*[clinic input]
mymod.resize

    width: int
    height: int
    mode: str = "nearest"

Resize the image.
[clinic start generated code]*/

It is a CPython-internal tool rather than a public one, but it ships with the source and is usable in third-party projects. You do not need it, and this course does not assume it — but if you find yourself hand-writing many keyword parsers, know it exists. Its most valuable output is the fast keyword parsing that PyArg_ParseTupleAndKeywords cannot provide for METH_FASTCALL functions.

Building return values

Py_BuildValue is the inverse of PyArg_ParseTuple and returns a new strong reference:

return Py_BuildValue("i", n);                 /* an int              */
return Py_BuildValue("(iid)", x, y, dist);    /* a 3-tuple           */
return Py_BuildValue("{s:i,s:s}", "n", n, "name", name);   /* a dict  */
return Py_BuildValue("y#", buffer, (Py_ssize_t)len);       /* bytes   */

Two units matter for ownership: O takes an object and increfs it, while N takes an object and steals your reference. Use N for something you just created and would otherwise have to decref:

/* Without N you would need a temporary and a Py_DECREF on both paths. */
return Py_BuildValue("(NN)", PyLong_FromLong(a), PyLong_FromLong(b));

For simple cases the direct constructors are clearer and faster: PyLong_FromLong, PyFloat_FromDouble, PyUnicode_FromString, PyUnicode_FromFormat, PyBytes_FromStringAndSize, PyTuple_Pack.

Calling back into Python

Wrapping a library almost always means invoking a Python callable from C. The modern, allocation-light APIs:

All of them return a new strong reference, or NULL with an exception set. Three obligations at every call site:

1. Check for NULL. A Python callback can raise anything.
2. Decref the result, even when you ignore it.
3. Assume the world changed. The callback ran arbitrary code: it may have mutated your containers, released and reacquired the interpreter, imported modules, or triggered a garbage collection. Re-validate anything you cached across the call, and never hold a borrowed reference across one.

PyObject *res = PyObject_CallOneArg(state->callback, item);
if (res == NULL) {
    return -1;               /* propagate; exception already set */
}
Py_DECREF(res);

import timeit

# METH_O (one arg, no parsing) vs METH_VARARGS-style parsing:
print(timeit.timeit("len(x)", setup="x=[1,2,3]", number=2_000_000))
print(timeit.timeit("divmod(7, 3)", number=2_000_000))

# Introspectability is a real consequence of the parsing choice:
import math, inspect
print(inspect.signature(math.hypot))   # Argument Clinic gave this a signature
help(math.pow)                          # note the rendered signature line

Further reading

• C API — Implementing Functions and Methods. PyMethodDef and every METH_ flag.
• C API — Parsing Arguments and Building Values. The complete format-unit table.
• C API — Call Protocol. Vectorcall, and the full set of calling APIs.
• PEP 590 — Vectorcall. The design and the reasoning about allocation cost.
• Argument Clinic How-To.