Reference Counting and Ownership

The invariant

Every object's ob_refcnt counts how many places currently hold a pointer to it and consider that pointer valid. When a piece of code stops needing the object it decrements the count. When the count reaches zero, the object's deallocator runs and the memory is freed.

That is the whole mechanism. There is no compiler support, no scope analysis, no ownership type. It is a hand-maintained invariant, exactly like malloc/free discipline — except that the consequences of getting it wrong are worse, because a premature free in CPython usually does not crash at the point of the bug. It corrupts an object that some unrelated code touches later.

Strong and borrowed references

The API documentation describes every PyObject * in terms of two categories, and you must know which one you are holding at all times.

Strong reference (also: "new reference", "owned reference")

The reference count has been incremented on your behalf. The object is guaranteed alive for as long as you hold it. You are responsible for exactly one Py_DECREF before you lose the pointer.

Borrowed reference

Nobody incremented anything for you. You are looking at a pointer that someone else is keeping alive. You must not decref it, and it is valid only for as long as that other owner keeps it — which may be a very short time.

Crucially, which one you get is a property of the function you called, not of the object's type. From the C API introduction:

"It is important to realize that whether you own a reference returned by a function depends on which function you call only — the plumage (the type of the object passed as an argument to the function) doesn't enter into it!"

There is no way to tell by looking at a pointer. You look it up in the docs, once, and then you write it in a comment.

The common cases, memorized

The pattern behind the second column: a function returns a borrowed reference when it can point into a container that the caller already holds. That is faster — no atomic increment — and it was the dominant style in older CPython. Modern CPython is steadily adding strong-reference variants (the ...Ref suffix) precisely because borrowed references become unsound under free-threading. Lesson 11 covers that; the practical advice today is to prefer the ...Ref variants in new code.

The manipulation macros

Why Py_CLEAR is not optional

This looks equivalent to Py_CLEAR but is not:

Py_DECREF(self->cache);   /* may drop to zero -> deallocator runs -> arbitrary
                             Python code runs -> that code can reach self and
                             read self->cache, which is now a dangling pointer */
self->cache = NULL;

Py_DECREF can run arbitrary Python code, because dropping to zero calls the object's deallocator, which may invoke a __del__ method, release a weak reference and run its callback, or free a container and cascade into more deallocations. Any of that can re-enter your object. Py_CLEAR nulls the field before decrementing, so anything that re-enters sees a consistent, if empty, object.

Stealing references

A handful of functions steal a reference: they take over ownership of a reference you pass in, so you must not decref it afterwards. This exists as an optimization for the extremely common "build a container from freshly created objects" pattern.

PyObject *t = PyTuple_New(3);
if (t == NULL) return NULL;
/* PyTuple_SetItem steals: no decref of the PyLong needed, and on failure
   it disposes of the reference for you. */
PyTuple_SetItem(t, 0, PyLong_FromLong(1));
PyTuple_SetItem(t, 1, PyLong_FromLong(2));
PyTuple_SetItem(t, 2, PyLong_FromLong(3));
return t;

If you already hold a reference that you still need after the call, incref first:

PyObject *item = get_something();     /* strong reference, I still need it */
PyTuple_SetItem(t, 0, Py_NewRef(item));  /* give the tuple its own reference */
... use item ...
Py_DECREF(item);

The stealing functions you will actually meet:

• PyTuple_SetItem, PyTuple_SET_ITEM, PyList_SetItem, PyList_SET_ITEM
• PyModule_Add (steals always, even on error) — contrast PyModule_AddObjectRef, which steals nothing and is the recommended one, and the legacy PyModule_AddObject, which steals only on success and is therefore a leak trap
• Py_BuildValue's N format unit (versus O, which increfs)

Note that PyDict_SetItem and PyObject_SetAttr do not steal. Dicts are the exception to the pattern, which is exactly why it trips people up.

The borrowed-reference trap

Here is the bug that appears in real extension code, in production, for years:

static PyObject *
broken(PyObject *self, PyObject *list)
{
    PyObject *first = PyList_GetItem(list, 0);      /* BORROWED */
    if (first == NULL) return NULL;

    /* Arbitrary Python code runs here. It could be a __len__, a __eq__,
       a callback, a logging handler -- anything. Suppose it clears the list. */
    Py_ssize_t n = PyObject_Length(list);
    if (n < 0) return NULL;

    /* `first` may now be freed memory. The list dropped the only reference. */
    return PyObject_Repr(first);                    /* use-after-free */
}

The list was the owner. When the list dropped the item, the count hit zero and the object was freed. Your borrowed pointer became a dangling pointer at a moment you never wrote any code for.

Two correct fixes, in order of preference:

/* 1. Ask for a strong reference in the first place (Python 3.13+). */
PyObject *first;
int rc = PyList_GetItemRef(list, 0, &first);   /* rc: 1 found, 0 missing, -1 error */
if (rc < 0) return NULL;
...
Py_DECREF(first);

/* 2. Or promote the borrowed reference immediately, before anything else runs. */
PyObject *first = PyList_GetItem(list, 0);
if (first == NULL) return NULL;
Py_INCREF(first);
...
Py_DECREF(first);

Structuring a function so it cannot leak

A C function that builds several objects has many failure points and must release everything acquired so far on each one. The idiom the CPython source uses everywhere is single-exit cleanup with goto, with all pointers initialized to NULL so that Py_XDECREF is always safe.

static PyObject *
pair_of_repr(PyObject *module, PyObject *args)
{
    PyObject *a = NULL, *b = NULL;
    PyObject *ra = NULL, *rb = NULL;
    PyObject *result = NULL;              /* the single return value */

    if (!PyArg_UnpackTuple(args, "pair_of_repr", 2, 2, &a, &b)) {
        return NULL;                      /* a and b were borrowed; nothing owned yet */
    }

    ra = PyObject_Repr(a);                /* strong */
    if (ra == NULL) goto done;

    rb = PyObject_Repr(b);                /* strong */
    if (rb == NULL) goto done;

    result = PyTuple_Pack(2, ra, rb);     /* strong; PyTuple_Pack does NOT steal */

done:
    Py_XDECREF(ra);
    Py_XDECREF(rb);
    return result;                        /* NULL on any failure, with the
                                             exception already set (Lesson 05) */
}

Every owned pointer is released exactly once on every path. result stays NULL unless the last step succeeded.

Note three properties worth copying:

1. All owned pointers are declared and NULL-initialized at the top, so the cleanup block is unconditional.
2. Cleanup uses Py_XDECREF, tolerating the NULLs.
3. The returned object is not in the cleanup list, because ownership transfers to the caller.

Finding refcount bugs

Leaks and over-decrefs are found empirically, not by inspection. The tools, cheapest first:

sys.getrefcount(obj)

Returns one more than the real count (its own argument is a reference). Call your extension function in a loop and check the count of an argument before and after; a steady climb is a leak, a drop is an over-decref.

python -X showrefcount

On a debug build, prints the total live reference count and allocated block count after each interactive statement. Running the same call twice and comparing the delta is the standard quick test.

A debug build (--with-pydebug, which defines Py_DEBUG)

Adds assertions inside the refcount macros, fills freed memory with recognizable bytes, and makes a use-after-free far more likely to abort at the scene of the crime. If you are serious about a C extension, build one and test against it. On Windows, the debug interpreter is python_d.exe and extensions must be built against it separately.

AddressSanitizer / Valgrind

Standard C tooling still works; combine with a Python built using PYTHONMALLOC=malloc so the allocator does not hide the errors.

import sys

obj = object()
print(sys.getrefcount(obj))       # ?

holder = [obj, obj, obj]
print(sys.getrefcount(obj))       # ?

del holder
print(sys.getrefcount(obj))       # ?

d = {"k": obj}
print(sys.getrefcount(obj))       # ?
d.clear()
print(sys.getrefcount(obj))       # ?

Further reading

• C API — Reference Counts. Ownership rules, stealing, and the canonical worked example.
• C API — Reference Counting. Exact semantics of every macro in the table above.
• Extending Python — Reference Counts. A longer prose treatment with the ownership-transfer diagrams.
• Python Extension Patterns — Reference Counts. Paul Ross's book-length treatment, unusually good on the failure modes and on how to detect them.