gh-158803: Fix crash in bytes.join() on a concurrently mutated list

[christianaurichzm]

Fixes #158803

In the free-threaded build, bytes.join(), bytearray.join() and PyBytes_Join() (which share Objects/stringlib/join.h) read items from a list with PySequence_Fast_GET_ITEM(), which returns a borrowed reference, without holding the list's lock. If another thread replaces an item in the meantime, the old item can be freed before join() takes its own reference to it.

str.join() had the same problem and was fixed in gh-119247, which added Py_BEGIN_CRITICAL_SECTION_SEQUENCE_FAST. This change uses the same pattern: the existing body becomes bytes_join_lock_held(), and bytes_join() calls it inside the critical section, as PyUnicode_Join() does with _PyUnicode_JoinArray().

The critical section can be suspended while an item's __buffer__() runs, or while a large result is copied with the thread state detached. Every item is increfed while the lock is held, and the existing size check still raises RuntimeError if the list changes size, so both cases stay safe.

In the default build the macros expand to an empty block.

Verification

• The new test in test_free_threading.test_bytes_object crashes an unpatched free-threaded debug build in most runs (6 to 8 out of 10) and passed 30 out of 30 runs with the fix. The reproducer from the issue segfaults without the fix and runs cleanly for 10 seconds with it.
• Under ThreadSanitizer (clang 21, free-threaded, same configure options and suppressions as CI), test_free_threading, test_bytes and test_capi.test_bytes pass with no reports.
• test_bytes, test_capi.test_bytes and test_free_threading.test_bytes_object pass with -R 3:3 on both builds.
• No new failures in the full test suite on either build.

• Issue: Segfault in bytes.join for mutating list #158803

[christianaurichzm]

The macOS (free-threading) failure is unrelated to this change: the test run was stopped by a SIGINT that escaped test_idle's InterruptTest.test_interrupt_blocking_call. The same failure happened on #158828 and on the Windows NoGIL buildbot (reported in gh-74112), and it is fixed by #158914.

[eendebakpt]

Nice work! The failing CI indeed seems unrelated. Can you merge with main to trigger the CI again?

[christianaurichzm]

@eendebakpt Thanks! Merged main. The Windows failure is the known test_sslproto flake (gh-158646). Could you re-run it?

[eendebakpt]

The added lock will add a little bit of overhead in the FT build, but we can easily gain it back in a followup PR (where we can make use of the fact that the iterator argument is locked). For str.join there is a similar lock (see

cpython/Objects/unicodeobject.c

Line 10439 in 37cc8dd

Py_BEGIN_CRITICAL_SECTION_SEQUENCE_FAST(seq);

)

[christianaurichzm]

Thanks! Happy to look into the follow-up once this lands.