Python: Queues for Thread Communication

Threads are great when you want parts of your program to run at the same time. But sometimes, these threads need to talk to each other — like one thread making things and another using them. This is where a Queue comes in. A Queue is a special tool that lets threads pass data to each other safely, without messing things up. You don’t have to worry about locking or timing issues — the queue handles it for you. In this guide, we’ll walk through how to use queues for thread communication using fun, easy-to-follow examples. No stress, just clear steps and real code.

What is a Queue?

A Queue is like a safe and organized pipe that threads can use to send and receive data. It helps one thread pass items to another without running into problems like data clashes or timing issues. You don’t need to use locks or worry about multiple threads using it at the same time — the queue handles all that behind the scenes. Python gives us this handy tool in the built-in queue module, making it super easy to create smooth and safe communication between threads.

Creating a Queue

To create a queue for threads, use queue.Queue() from Python’s built-in queue module. This gives you a FIFO queue — first in, first out — which is perfect for safely passing items between threads. You can also set a maxsize to limit how many items the queue can hold at once. This helps control memory or flow between producers and consumers.

Here are two simple examples:

Example 1: Basic Unlimited Queue

import queue

q = queue.Queue()  # This queue can grow as needed
q.put("apple")
q.put("banana")

print(q.get())  # Output: apple
print(q.get())  # Output: banana

This queue doesn’t have a size limit. You can keep adding items, and get() will remove them in the same order.

Example 2: Limited-Size Queue

import queue

q_limited = queue.Queue(2)  # Max 2 items

q_limited.put("cat")
q_limited.put("dog")

print("Added two animals!")

# The next put will wait if no item is removed first
# This will block until there's space:
# q_limited.put("mouse")

This queue only allows 2 items at once. It’s useful when you want to slow down producers until consumers catch up.

These simple setups are the starting point for smooth thread communication.

Putting and Getting Items

Once you have a queue, you can start using it to pass data between threads. The two most common actions are:

• put(item) — puts an item into the queue.
• get() — takes the next item out of the queue.

Both methods are thread-safe and will wait (block) if needed:

• put() will pause if the queue is full (if maxsize is set).
• get() will pause if the queue is empty — it waits for something to arrive.

Example: Fruit Basket

import queue

q = queue.Queue()

# Put some fruits in the queue
q.put("apple")
q.put("banana")

# Take them out
print("I got:", q.get())  # apple
print("I got:", q.get())  # banana

This example shows how items go in and come out in order — like a line at the market. Great for when one thread is producing things and another is ready to pick them up!

Producer-Consumer Example

In many real-world situations, one part of your program makes data (the producer), and another part processes it (the consumer). A Queue is a safe way for them to share that data without crashing into each other.

In the example below, the producer thread creates items and places them in the queue. The consumer thread runs in the background and waits for items to appear. As soon as something is added, it grabs it and prints it.

import threading
import queue
import time

q = queue.Queue()

def producer():

    for i in range(3):
        item = f"Item-{i}"
        q.put(item)
        print(f"Produced {item}")
        time.sleep(1)  # Simulate work


def consumer():

    while True:
        item = q.get()
        print(f"Consumed {item}")
        q.task_done()

# Start the producer thread
threading.Thread(target=producer).start()

# Start the consumer thread (daemon runs in background)
threading.Thread(target=consumer, daemon=True).start()

The producer adds three items, one each second. The consumer waits patiently, picks them up, and processes them. The q.task_done() tells the queue that the item has been handled. This pattern is very useful in games, data pipelines, or anything where work is divided between different workers.

Waiting for Work to Finish

Sometimes you want to wait until all the work in the queue is done — meaning all items have been taken out and fully processed. To do this, use q.join(). This will block (pause) the main thread until every item put into the queue has been handled.

But for join() to work, every time a thread takes something from the queue using get(), it must also call task_done(). This tells the queue, “I’m done with this item.” Once all items have been marked done, join() will stop waiting and the program continues.

Here’s a small example that shows how it works:

import threading
import queue
import time

q = queue.Queue()

def producer():

    for i in range(5):
        q.put(i)
        print(f"Produced {i}")
        time.sleep(0.5)

def consumer():

    while not q.empty():
        item = q.get()
        print(f"Consumed {item}")
        time.sleep(0.5)
        q.task_done()

t1 = threading.Thread(target=producer)
t2 = threading.Thread(target=consumer)

t1.start()
t1.join()  # Wait for producer to finish

t2.start()
q.join()   # Wait for all items to be marked done

print("All work completed.")

This way, you can make sure all items were processed before moving on. It’s clean and safe for thread coordination.

Multiple Producers and Consumers

A great thing about Python’s queue.Queue is that it works smoothly with multiple producers and consumers. You don’t need to add any special locks. All threads can safely share the same queue, putting in and taking out items as needed. This makes it easy to build a team of worker threads.

In the example below, we have two producers and two consumers. Each producer creates two items and puts them into the queue. Meanwhile, the consumers pick up those items and process them. Because the queue handles the thread-safety, everything works nicely without any extra effort:

import threading
import queue

q = queue.Queue()

def producer(name):

    for i in range(2):
        item = f"{name}-Item-{i}"
        q.put(item)
        print(f"{name} produced {item}")

def consumer(name):

    while True:
        item = q.get()
        print(f"{name} consumed {item}")
        q.task_done()


# Start producers
for name in ["P1", "P2"]:
    threading.Thread(target=producer, args=(name,)).start()


# Start consumers
for name in ["C1", "C2"]:
    threading.Thread(target=consumer, args=(name,), daemon=True).start()


# Wait until all tasks are done
q.join()

print("All items processed.")

This setup is powerful for real-world jobs — like downloading files, processing data, or saving results — where many threads can help out together.

Using queue.SimpleQueue

If you just need a basic, thread-safe way for threads to share data — and don’t need extra features like task_done() or join() — then queue.SimpleQueue is a good choice. It works like a regular queue but is a bit lighter and simpler. It’s FIFO (first-in, first-out), thread-safe, and very easy to use.

Here’s a small example. We create a SimpleQueue, put some data in it, and take it out:

from queue import SimpleQueue

q = SimpleQueue()
q.put("data")

print(q.get())

That’s it! No need to worry about max size or signaling when tasks are done. It’s a nice option when you want just the basics for thread communication.

Non-Blocking Operations

Python queues also offer handy shortcuts for non-blocking operations: put_nowait() and get_nowait(). These do the same as put(block=False) and get(block=False) but look cleaner and are easier to remember.

• put_nowait(item) tries to put an item without waiting. If the queue is full (only for limited queues), it raises queue.Full.
• get_nowait() tries to get an item without waiting. If the queue is empty, it raises queue.Empty.

Here’s a simple example:

from queue import Queue, Empty, Full

q = Queue(maxsize=2)

try:

    q.put_nowait("apple")
    q.put_nowait("banana")
    q.put_nowait("cherry")  # Will raise queue.Full if queue is full

except Full:
    print("Queue is full, can't add more items")

try:

    while True:
        item = q.get_nowait()
        print(f"Got: {item}")

except Empty:
    print("Queue is empty, no more items")

For SimpleQueue, which is unlimited in size, put_nowait() will always work:

from queue import SimpleQueue, Empty

sq = SimpleQueue()
sq.put_nowait("data")

try:

    item = sq.get_nowait()
    print(item)

except Empty:
    print("Queue is empty")

These methods are great when you want to try getting or putting without waiting and handle the empty/full cases yourself.

Real-Life Analogy: Pizza Shop

Imagine a busy pizza shop. The chefs are working hard to make pizzas, and the delivery drivers are ready to pick up the orders and deliver them to customers. Here, the pizza queue acts like the shared space where chefs place finished pizzas, and drivers take pizzas to deliver. This way, both chefs and drivers work together smoothly without stepping on each other’s toes.

In this example, the chefs are the producers who put pizzas into the queue. The delivery drivers are the consumers who take pizzas out to deliver. The queue safely handles the handoff between them, so no pizza gets lost or picked up twice.

import threading
import queue

pizza_queue = queue.Queue()

def chef(name):

    for i in range(2):
        pizza = f"{name}'s Pizza-{i}"
        pizza_queue.put(pizza)
        print(f"{name} made {pizza}")


def driver(name):

    while True:
        pizza = pizza_queue.get()
        print(f"{name} delivered {pizza}")
        pizza_queue.task_done()


# Start chef threads
for chef_name in ["Luigi", "Mario"]:
    threading.Thread(target=chef, args=(chef_name,)).start()

# Start driver threads (daemon so they exit when main thread finishes)
for driver_name in ["Toad", "Yoshi"]:
    threading.Thread(target=driver, args=(driver_name,), daemon=True).start()

# Wait until all pizzas are delivered
pizza_queue.join()

This code runs two chefs, Luigi and Mario, each making two pizzas. They place the pizzas into the queue. Two delivery drivers, Toad and Yoshi, keep picking pizzas from the queue and delivering them. The queue ensures everything happens safely and in order.

At the end, pizza_queue.join() waits until all pizzas have been delivered before the program finishes. This simple but fun example shows how queues help threads communicate and work together in real life!

Summary Table

Best Practices

Always make sure to call task_done() after a thread finishes processing an item it got from the queue. This pairs with join() to know when all work is complete.

For consumer threads that run continuously, set daemon=True so they don’t block your program from exiting.

Keep your queue usage simple and focused — its main job is to safely pass data between threads without extra complexity.

Conclusion

Queues are one of the best tools in Python for making thread communication simple and safe. They let threads share data without worrying about conflicts or complicated locks. Whether you are passing small tasks, pieces of data, or signals between threads, a queue keeps everything organized. Using queues means your threads can focus on their work, knowing the data will be passed smoothly and correctly.

By mastering queues, you gain an easy way to coordinate multiple threads in any program. From fun examples like pizza delivery to real tasks like handling jobs in a pipeline, queues help keep your code clean and reliable. When you build your next multithreaded project, try using queue.Queue or queue.SimpleQueue to make your threads talk to each other clearly and safely — it will save you time and headaches.