Type Objects and Slots

A type is a vtable with a Python face

In Lesson 01 you saw that every object carries an ob_type pointer. What it points at is a PyTypeObject: a large C struct — around eighty fields — most of which are function pointers. CPython calls those fields slots.

When Python evaluates a + b, the interpreter does not look up a string named "__add__" in a dict. It reads Py_TYPE(a)->tp_as_number->nb_add and calls it. The dunder name is the Python-visible spelling of the slot; the slot is the real thing. For a class written in Python, CPython installs a generic slot function that then does the dict lookup — the indirection goes the other way. For a class written in C, you fill in the slot directly and the dunder name appears automatically.

This is why extension types are fast, and it is why understanding slots is the core of writing them.

The dunder-to-slot map

The non-function fields

Flags worth knowing

Py_TPFLAGS_DEFAULT

Baseline set of capabilities. Always OR this in.

Py_TPFLAGS_BASETYPE

Permits subclassing from Python. Without it, class Sub(YourType) raises TypeError. Omit it if your tp_dealloc or invariants cannot survive a subclass.

Py_TPFLAGS_HAVE_GC

The type participates in cycle collection and must supply tp_traverse. Lesson 04.

Py_TPFLAGS_IMMUTABLETYPE

The type object itself cannot be modified from Python (no monkey-patching class attributes). Static types get this behaviour implicitly; heap types should usually opt in.

Py_TPFLAGS_MANAGED_DICT / Py_TPFLAGS_MANAGED_WEAKREF

Ask CPython to give instances a __dict__ and weak-reference support without you managing tp_dictoffset/tp_weaklistoffset by hand. Prefer these to the manual offsets on 3.12+.

The instance struct

Your instances are a C struct beginning with the header from Lesson 01, followed by whatever you need:

typedef struct {
    PyObject_HEAD
    PyObject  *name;      /* a Python object field: refcounted, may be NULL */
    long       count;     /* a plain C field: not an object, no refcounting */
    FILE      *handle;    /* a foreign resource: your job to close */
} CounterObject;

The distinction between those three kinds of field drives everything else: object fields need increfs, decrefs, and (Lesson 04) traversal; C fields need nothing; foreign resources need explicit release in tp_dealloc.

The three lifecycle slots

  Counter("abc")
        │
        ▼
  type.__call__          ─── C: type_call()
        │
        ├──► tp_new(type, args, kwds)   allocate raw memory, set safe defaults
        │         returns a new instance (a strong reference)
        │
        └──► tp_init(self, args, kwds)  apply the arguments; returns 0 or -1
                  may be called again later:  obj.__init__(...)

  ... instance lives ...

  refcount reaches 0
        │
        ▼
  tp_dealloc(self)       release every field, then hand memory back

Why two construction phases? Because tp_new is what makes immutable types possible (all the work happens before anyone can see the object) and because subclasses can override one without the other. Practical rules:

• tp_new allocates via type->tp_alloc(type, 0). Use the type passed in, not your own type object, or subclasses break.
• tp_new must leave every field in a state that tp_dealloc can safely handle, because construction can fail afterwards.
• tp_init can be called more than once, or never. Never assume it has run.
• If you need no arguments handling at all, set tp_new = PyType_GenericNew and do the work in tp_init.
• tp_dealloc must end by calling Py_TYPE(self)->tp_free(self) — not free(), and not your own type's tp_free, because self may be an instance of a subclass.

static void
Counter_dealloc(PyObject *op)
{
    CounterObject *self = (CounterObject *)op;
    if (self->handle) { fclose(self->handle); self->handle = NULL; }
    Py_CLEAR(self->name);
    Py_TYPE(self)->tp_free(self);
}

Foreign resources first, then object fields, then the memory itself.

Exposing attributes: members versus getset

PyMemberDef exposes a struct field directly by byte offset. It is the cheap option and requires no code:

static PyMemberDef Counter_members[] = {
    {"count", Py_T_LONG,      offsetof(CounterObject, count), 0,          "call count"},
    {"name",  Py_T_OBJECT_EX, offsetof(CounterObject, name),  Py_READONLY, "the name"},
    {NULL}   /* sentinel */
};

Type codes are Py_T_INT, Py_T_LONG, Py_T_DOUBLE, Py_T_BOOL, Py_T_PYSSIZET, Py_T_STRING, Py_T_OBJECT_EX and friends. Flags include Py_READONLY and Py_AUDIT_READ.

PyGetSetDef is the property equivalent: you supply a getter and optionally a setter, so you can validate, compute, or lazily construct.

static PyObject *
Counter_get_name(PyObject *op, void *closure)
{
    CounterObject *self = (CounterObject *)op;
    return Py_NewRef(self->name);          /* return a STRONG reference */
}

static int
Counter_set_name(PyObject *op, PyObject *value, void *closure)
{
    CounterObject *self = (CounterObject *)op;
    if (value == NULL) {                   /* NULL means: del obj.name */
        PyErr_SetString(PyExc_AttributeError, "cannot delete name");
        return -1;
    }
    if (!PyUnicode_Check(value)) {
        PyErr_SetString(PyExc_TypeError, "name must be a str");
        return -1;
    }
    Py_SETREF(self->name, Py_NewRef(value));
    return 0;
}

static PyGetSetDef Counter_getset[] = {
    {"name", Counter_get_name, Counter_set_name, "the name", NULL},
    {NULL}
};

The trailing void *closure lets one C function serve several attributes by passing a discriminator through the last field of PyGetSetDef.

Static types versus heap types

There are two ways to bring a type into existence from C, and the choice has real consequences.

Static types are what every old tutorial shows, and they are still fine for a simple leaf type in a module you control. Heap types are the direction CPython is moving and are what you want for anything new, especially if you care about subinterpreters or the stable ABI.

Static form

static PyTypeObject CounterType = {
    .ob_base      = PyVarObject_HEAD_INIT(NULL, 0),
    .tp_name      = "counter.Counter",
    .tp_doc       = PyDoc_STR("A counter."),
    .tp_basicsize = sizeof(CounterObject),
    .tp_itemsize  = 0,
    .tp_flags     = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE,
    .tp_new       = Counter_new,
    .tp_init      = Counter_init,
    .tp_dealloc   = Counter_dealloc,
    .tp_methods   = Counter_methods,
    .tp_members   = Counter_members,
};

/* later, during module setup: */
if (PyType_Ready(&CounterType) < 0) return -1;
if (PyModule_AddObjectRef(m, "Counter", (PyObject *)&CounterType) < 0) return -1;

PyType_Ready is not optional: it computes the MRO, inherits unset slots from the base type, and builds tp_dict from your methods, members and getsets. A type object that has not been readied will crash on first use.

Heap form

The same type, expressed as data. Each slot gets an ID of the form Py_tp_*, Py_nb_*, Py_sq_*, Py_mp_*:

static PyType_Slot Counter_slots[] = {
    {Py_tp_doc,     (void *)PyDoc_STR("A counter.")},
    {Py_tp_new,     Counter_new},
    {Py_tp_init,    Counter_init},
    {Py_tp_dealloc, Counter_dealloc},
    {Py_tp_methods, Counter_methods},
    {Py_tp_members, Counter_members},
    {Py_tp_traverse, Counter_traverse},   /* Lesson 04 */
    {Py_tp_clear,    Counter_clear},
    {0, NULL}
};

static PyType_Spec Counter_spec = {
    .name      = "counter.Counter",
    .basicsize = sizeof(CounterObject),
    .itemsize  = 0,
    .flags     = Py_TPFLAGS_DEFAULT | Py_TPFLAGS_BASETYPE
               | Py_TPFLAGS_HAVE_GC | Py_TPFLAGS_IMMUTABLETYPE,
    .slots     = Counter_slots,
};

/* during module exec: */
PyObject *type = PyType_FromModuleAndSpec(module, &Counter_spec, NULL);
if (type == NULL) return -1;
if (PyModule_AddObjectRef(module, "Counter", type) < 0) { Py_DECREF(type); return -1; }
Py_DECREF(type);   /* the module now holds it */

The third argument to PyType_FromModuleAndSpec is the base (or a tuple of bases), or NULL for object. Note that PyType_FromModuleAndSpec returns a strong reference that you must release after handing it to the module.

static void
Counter_dealloc(PyObject *self)
{
    PyTypeObject *tp = Py_TYPE(self);   /* read first */
    PyObject_GC_UnTrack(self);          /* Lesson 04 */
    Py_CLEAR(((CounterObject *)self)->name);
    tp->tp_free(self);
    Py_DECREF(tp);                      /* release the instance's ref to the type */
}

Inheritance and the MRO

To subclass an existing type from C, make the base's struct the first member of yours and set tp_base:

typedef struct {
    PyListObject list;     /* the base object, laid out first */
    int state;
} SubListObject;

/* static form, during module setup, before PyType_Ready: */
SubListType.tp_base = &PyList_Type;

Your tp_init should call the base's: PyList_Type.tp_init(self, args, kwds). PyType_Ready then fills in every slot you left NULL from the base, which is why a subclass that only overrides tp_init still behaves like a list everywhere else.

One consequence worth internalizing: slot inheritance happens once, at type-creation time, not on every call. That is what makes it fast, and it is also why mutating a type's dict from Python after the fact requires CPython to go re-fix the slots — machinery that heap types support and static types deliberately do not.

class Py:
    def __add__(self, other): return "py"

class NoAdd: pass

# A C type has its slot filled; a Python class gets a generic wrapper.
print(type(int.__add__))        # slot wrapper
print(type(Py.__add__))         # plain function

# Types without a slot do not merely fail the lookup -- the operation reports
# unsupported operand types, because the abstract API found a NULL nb_add.
try:
    NoAdd() + NoAdd()
except TypeError as e:
    print(e)

# Members vs getset, visible through the descriptor types:
import datetime
print(type(datetime.timedelta.days))     # getset_descriptor or member_descriptor
print(type(int.numerator))

Further reading

• Defining Extension Types: Tutorial. Builds a type up in four stages; the single best starting point.
• Type Object Structures. The exhaustive reference: every slot, its signature, and its inheritance rules.
• Type Objects. PyType_Spec, PyType_Slot, PyType_FromModuleAndSpec, PyType_GetModuleState.
• Defining Extension Types: Assorted Topics. Attribute management, comparisons, weak references.
• PEP 384 — Defining a Stable ABI. Where PyType_Spec came from and why.