Python: Process Synchronization

When working with Python’s multiprocessing, processes often need to share and update the same data. Without proper control, they can step on each other’s toes—corrupting the data or causing unexpected behavior.

That’s where synchronization tools like locks come in. A special kind of lock called RLock (Reentrant Lock) is designed for cases where a single process might need to lock the same resource more than once. For example, if a function that acquires a lock calls another function that also tries to lock the same resource, a regular Lock would freeze the program. But RLock allows that to happen safely.

This article is all about how to use multiprocessing.RLock. We won’t talk about Lock, Semaphore, or other tools—just RLock. Through clear and realistic examples, you’ll see exactly how and when to use it, especially in situations with nested or repeated locking.

Table of Contents

Creating and Acquiring an RLock

To use a reentrant lock in Python’s multiprocessing, you simply create it with RLock(). It works like a regular Lock, but with one big difference: the same process can acquire it multiple times without causing a deadlock. This is especially helpful when a function that acquires the lock calls another function that also tries to acquire the same lock.

You can use .acquire() and .release() manually, but the cleaner and safer way is to use the with statement, which automatically acquires and releases the lock.

Let’s see a fun example involving a librarian who has to open two cabinets — one inside the other. Both actions require locking, but since it’s the same process calling both functions, we need an RLock.

from multiprocessing import Process, RLock
import time

# Outer cabinet function that also calls the inner one
def open_outer_cabinet(name, lock):

    with lock:
        print(f"{name} opened the outer cabinet.")
        time.sleep(1)
        open_inner_cabinet(name, lock)


# Inner cabinet function that also acquires the same lock
def open_inner_cabinet(name, lock):
    with lock:
        print(f"{name} opened the inner cabinet.")


if __name__ == "__main__":

    lock = RLock()
    names = ["Mary", "Samantha"]

    for name in names:
        Process(target=open_outer_cabinet, args=(name, lock)).start()

In this code, each librarian (process) tries to open both the outer and inner cabinets. Since both actions are protected by the same lock, a regular Lock would block the second acquire. But with RLock, it works smoothly — the same process can safely enter nested locked sections.

Recursive Use Case with RLock

A regular Lock blocks if the same process tries to acquire it again. But RLock allows reentry — the same process can safely lock again, even inside a recursive call.

This is especially useful when writing recursive functions that touch shared state. Let’s look at a fun example where we count how many folders are scanned, including subfolders.

Each process scans its own folder, and the recursive function adds to a shared file counter. Since the same function calls itself while holding the lock, we need RLock for this to work.

from multiprocessing import Process, Value, RLock
import time

# Recursively count folders and subfolders
def count_files(folder_name, counter, lock, depth=0):

    with lock:

        print(f"Scanning {folder_name} at depth {depth}")

        counter.value += 1

        if depth < 2:
            count_files(f"{folder_name}/sub", counter, lock, depth + 1)


if __name__ == "__main__":

    lock = RLock()
    counter = Value('i', 0)
    folders = ["FolderA", "FolderB"]

    for folder in folders:
        Process(target=count_files, args=(folder, counter, lock)).start()

In this example, each folder (like “FolderA”) spawns a process to scan itself and its subfolders. The recursive function safely updates the shared counter using RLock. Without reentrant locking, this would break or hang. With RLock, everything works smoothly — even in nested calls.

Simulating Nested Resources

Sometimes, a task has many layers—each needing access to shared tools or steps. If one function calls another, and both need to lock the same resource, a regular Lock would fail. That’s where RLock shines.

This example shows a chef making a layered cake. The full process includes batter mixing, baking, and decorating. All steps share the same lock, and some are nested inside others.

from multiprocessing import Process, RLock
import time

# Step 1: Make batter (inner task)
def prepare_batter(name, lock):

    with lock:
        print(f"{name} is mixing batter.")
        time.sleep(0.5)


# Step 2: Bake layers (calls prepare_batter)
def bake_layers(name, lock):

    with lock:
        prepare_batter(name, lock)
        print(f"{name} is baking layers.")
        time.sleep(0.5)


# Step 3: Decorate the cake
def decorate(name, lock):

    with lock:
        print(f"{name} is decorating the cake.")
        time.sleep(0.5)


# Main cake-making task
def make_cake(name, lock):

    with lock:
        print(f"{name} started making cake.")
        bake_layers(name, lock)
        decorate(name, lock)
        print(f"{name} finished the cake.")


if __name__ == "__main__":

    lock = RLock()
    names = ["Heather", "Amber"]

    for name in names:
        Process(target=make_cake, args=(name, lock)).start()

Each step is locked, and some steps call other locked functions. Without RLock, this would cause a deadlock. With RLock, the same process can safely enter and exit each locked section—even when they’re nested.

RLock with Shared Values

Sometimes a process needs to update a shared number and then check it again within the same task. With regular Lock, this kind of nested access would cause a deadlock. RLock allows it safely.

In this example, two bank tellers deposit money. After each deposit, they check the balance. Both actions are protected with the same RLock.

from multiprocessing import Process, Value, RLock

# Teller deposits money, then checks the balance
def deposit(name, balance, lock):

    with lock:
        balance.value += 10
        print(f"{name} deposited. Balance: {balance.value}")
        check_balance(name, balance, lock)


# Nested function also tries to access the same locked value
def check_balance(name, balance, lock):

    with lock:
        print(f"{name} sees current balance: {balance.value}")


if __name__ == "__main__":

    balance = Value('i', 100)  # Shared account balance
    lock = RLock()
    names = ["Fred", "George"]

    for name in names:
        Process(target=deposit, args=(name, balance, lock)).start()

Thanks to RLock, each teller can safely lock the shared balance, call another function that also locks it, and avoid crashing or freezing. It’s perfect for step-by-step logic where the same process re-enters locked code.

Using RLock in Manager-Based Shared Lists

When working with a shared list using Manager.list(), multiple processes might try to read and write at the same time. If one function appends and then another edits the same item, we need to guard both steps — even if they happen within the same process. That’s where RLock fits perfectly.

In this example, two writers add and edit chapters in a shared book. Each edit happens inside a nested function, and both use the same RLock.

from multiprocessing import Process, Manager, RLock
import time

# First write a new chapter, then call edit function
def write_chapter(name, book, lock):

    with lock:
        book.append(f"{name} wrote a chapter.")
        time.sleep(0.5)
        edit_chapter(name, book, lock)


# Modify the latest chapter (called from within the lock)
def edit_chapter(name, book, lock):

    with lock:
        book[-1] += f" (edited by {name})"
        print(f"{name} edited the latest chapter.")


if __name__ == "__main__":

    with Manager() as manager:

        book = manager.list()     # Shared book list
        lock = RLock()            # Reentrant lock for nested access
        names = ["Lois", "Joe"]

        processes = []
        for name in names:
            p = Process(target=write_chapter, args=(name, book, lock))
            processes.append(p)
            p.start()

        for p in processes:
            p.join()

        print("\nFinal Book:")

        for chapter in book:
            print(" -", chapter)

Because RLock allows a process to re-acquire the same lock, writers can safely edit chapters inside nested functions. This keeps the list consistent, even when writing and editing happen together.

Conclusion

An RLock (Reentrant Lock) lets the same process safely re-acquire a lock multiple times. This is handy when a function calls another that also needs the lock — like nested steps, recursive logic, or layered tasks.

In this article, we saw how RLock works in realistic and fun cases: librarians unlocking cabinets, bakers layering cakes, writers editing books, and more.

Try using RLock in your own creative setups — maybe: