What a Python Object Actually Is

CPython is a C program, and your extension links into it

The reference implementation of Python — the thing you get from python.org, the thing that runs when you type python — is a C program named CPython. It exposes its own internals as a C header, Python.h. A "C extension module" is a shared library (.pyd on Windows, .so on Linux and macOS) that includes that header, links against the interpreter, and is loaded by import exactly like a .py file.

That means your C code runs inside the interpreter process, on the same stack, sharing the same heap, with full access to every interpreter data structure. There is no marshalling boundary, no serialization, no IPC. The flip side is that there is also no sandbox: a mistake in your C code corrupts the interpreter, and the symptom usually appears thousands of instructions later in unrelated code.

Everything in this course is about one question: what invariants does the interpreter expect you to maintain?

The universal object header

Strip away all the machinery and CPython's object model is three lines of C:

/* Include/object.h, simplified */
typedef struct _object {
    Py_ssize_t        ob_refcnt;   /* how many references point here */
    struct _typeobject *ob_type;   /* what kind of thing this is     */
} PyObject;

The header every Python value carries. Nothing in Python escapes it.

Every Python value — an integer, a string, a function, a module, a class, a stack frame, a type — is a block of memory on the heap that begins with those two fields. Concrete types append their own payload after the header:

typedef struct {
    PyObject ob_base;   /* refcnt + type, spelled via the PyObject_HEAD macro */
    double   ob_fval;   /* the actual C double */
} PyFloatObject;

In real CPython source that first line is written with a macro:

typedef struct {
    PyObject_HEAD       /* expands to: PyObject ob_base; */
    double ob_fval;
} PyFloatObject;

This is the classic C "struct-prefix inheritance" trick. Because PyFloatObject starts with a PyObject, a PyFloatObject * can be cast to PyObject * and back, and the C standard guarantees the address arithmetic works out. That is why essentially every function in the C API takes and returns PyObject *: it is the one pointer type that can name any Python value.

  PyObject *p  ────►  ┌──────────────────┐  offset 0
                      │ ob_refcnt        │  Py_ssize_t
                      ├──────────────────┤  offset 8
                      │ ob_type  ────────┼──► PyTypeObject for `float`
                      ├──────────────────┤  offset 16
                      │ ob_fval = 3.14   │  payload, known only to float code
                      └──────────────────┘

Variable-length objects

Objects whose size is decided at allocation time — tuples, strings, ints — use a second header that adds a length field:

typedef struct {
    PyObject   ob_base;
    Py_ssize_t ob_size;   /* number of items, not bytes */
} PyVarObject;

/* spelled in type definitions as: */
typedef struct {
    PyObject_VAR_HEAD
    PyObject *ob_item[1];   /* flexible trailing array */
} PyTupleObject;

The accessor macros are Py_REFCNT(o), Py_TYPE(o), and Py_SIZE(o), with mutating counterparts Py_SET_REFCNT, Py_SET_TYPE, Py_SET_SIZE. Read the fields through the macros rather than touching the struct members directly; the macros are what stay correct across builds, and on free-threaded builds the underlying fields are laid out differently.

Consequence 1: everything is boxed, and that is the cost

There is no such thing as an unboxed Python integer. A Python list of a million ints is a million separate heap allocations plus an array of a million pointers. When Python evaluates a + b, the interpreter does not emit an add instruction; it loads two pointers, reads ob_type from each, finds a function pointer for "addition" in the type, calls it, and that function allocates a third heap object for the result and returns a pointer to it.

This is the single most important fact for anyone writing a C extension for speed. Your extension is faster than Python not because C is a faster language, but because C code can leave the object model: unpack a million Python ints into a flat int64_t[] once, run a tight loop over raw memory, and box only the final answer. If your C loop allocates and frees a Python object per iteration, you have kept all of the cost and thrown away the readability.

Consequence 2: type is runtime data

ob_type points to a PyTypeObject — which is itself a Python object, with its own refcount and its own ob_type (pointing at type). A type object is essentially a large struct of function pointers: "how do I add one of these", "how do I get an attribute", "how do I free one". Lesson 03 takes that struct apart field by field. For now you only need three things:

• Py_TYPE(o) gives you the type of any object.
• Types are compared by pointer identity. Py_IS_TYPE(o, &PyLong_Type) is an exact-type check.
• Every builtin type has a check macro: PyLong_Check(o) accepts subclasses, PyLong_CheckExact(o) does not.

C code that assumes an argument is a particular type without checking is the most common cause of crashing extensions. Python callers can and will pass anything.

/* Wrong: a caller passing a str segfaults the process. */
double bad(PyObject *o) { return ((PyFloatObject *)o)->ob_fval; }

/* Right: verify, then use the accessor. */
static int good(PyObject *o, double *out) {
    if (!PyFloat_Check(o)) {
        PyErr_SetString(PyExc_TypeError, "expected a float");
        return -1;
    }
    *out = PyFloat_AsDouble(o);
    return 0;
}

The three layers of the C API

Functions in Python.h fall into three tiers, and knowing which tier you are in tells you what a call can do.

"Can it run Python code?" turns out to be the question that governs thread safety, reentrancy, and lifetime bugs. Hold that thought; it reappears in Lessons 02, 10, and 11.

Singletons, caching, and identity

None, True, and False are single global objects, addressable from C as Py_None, Py_True, Py_False. There is exactly one of each in the process. CPython also caches small integers (roughly −5 through 256) and interns many short strings, which is why a is b is sometimes surprisingly true in Python.

Since Python 3.12 these shared objects are immortal (PEP 683): their reference count is pinned to a sentinel value and never changes. That is a performance optimization — it removes contention on the hottest objects in the process — but it also means sys.getrefcount(None) returns a meaningless huge number, and any C code that special-cases refcounts on these objects is wrong.

Returning None from a C function has a dedicated macro because forgetting to increment its refcount used to be a classic bug:

Py_RETURN_NONE;     /* equivalent to: return Py_NewRef(Py_None); */
Py_RETURN_TRUE;
Py_RETURN_FALSE;

Where this is going

#include <Python.h>

/* The C implementation of one function.        -- Lesson 06 */
static PyObject *
add_one(PyObject *module, PyObject *arg)
{
    long v = PyLong_AsLong(arg);                 /* Lesson 01: concrete API */
    if (v == -1 && PyErr_Occurred()) return NULL; /* Lesson 05: errors      */
    return PyLong_FromLong(v + 1);               /* Lesson 02: new reference */
}

/* The table that exposes it to Python.          -- Lesson 06 */
static PyMethodDef methods[] = {
    {"add_one", add_one, METH_O, "Return arg + 1."},
    {NULL, NULL, 0, NULL}
};

/* Module definition and multi-phase init.       -- Lesson 08 */
static PyModuleDef_Slot slots[] = {
    {Py_mod_gil, Py_MOD_GIL_NOT_USED},           /* Lesson 11: free-threading */
    {0, NULL}
};

static PyModuleDef moduledef = {
    .m_base = PyModuleDef_HEAD_INIT,
    .m_name = "demo",
    .m_size = 0,
    .m_methods = methods,
    .m_slots = slots,
};

PyMODINIT_FUNC
PyInit_demo(void)                                 /* Lesson 08: entry point */
{
    return PyModuleDef_Init(&moduledef);
}

When a C extension is the right tool

Lesson 14 compares the alternatives properly. The short version, so you can sanity-check your own project now:

• Good fit: a tight numeric or byte-munching loop; wrapping an existing C/C++ library; needing to release the GIL around long work; needing exact control over memory layout.
• Poor fit: code that is I/O-bound; code that spends its time calling back into Python anyway; anything you could get from NumPy vectorization or a library that already exists.
• Consider a tool instead: Cython, pybind11, nanobind, or PyO3 generate the boilerplate for you. Understanding the raw C API is still worth it, because every one of those tools is a thin veneer over exactly the concepts in this course, and their error messages assume you know them.

import ctypes, sys

xs = [1, 2, 3]
raw_refcnt = ctypes.c_ssize_t.from_address(id(xs)).value
print("raw ob_refcnt   :", raw_refcnt)
print("sys.getrefcount :", sys.getrefcount(xs))   # one higher: its own argument

# The type pointer is the next word, and it equals id(type(xs)).
type_ptr = ctypes.c_void_p.from_address(id(xs) + ctypes.sizeof(ctypes.c_ssize_t))
print(type_ptr.value == id(list))

# Sizes show the boxing overhead.
print(sys.getsizeof(1.0), sys.getsizeof(0), sys.getsizeof(2**70))

Further reading

• Python/C API — Introduction. The canonical statement of the object model and the conventions this whole course elaborates on.
• Common Object Structures. PyObject, PyVarObject, and the access macros.
• CPython Include/object.h. Short, readable, and the ground truth for everything above.
• PEP 683 — Immortal Objects.