Multithreading or threading is the solution to running ‘several processes’ at once. Threading gives us parallel execution. In Python you can create threads from within your program.

note: threading is not truly paralel as the operating system still executes thread sequentially.

Related course: Complete Python Programming Course & Exercises

What is a Thread?

A thread is a separate flow of execution. Your program will be doing two things at once (not at the exact same time).

Because many processors these days have multiple cores, you can run a different process on each one. While they may be running on different processors, they are not running at the same time.

To use threading in Python, import the threading module. You can create a thread with this code:

my_thread = threading.Thread(target=thread_function, args = (1,))

The thread will be the execution of a function named thread_function() (it can have any name). This can be a simple function like:

def thread_function(name):
logging.info("Thread %s: started", name)
time.sleep(2)
logging.info("Thread %s: ended", name)

If you want a thread to continue for a long time, you can set a time limit or put it inside a loop.

Threads don’t start themself, the above code only created the thread but didn’t start it. You can start a thread by calling the .start() method.

my_thread.start()

In the Python code below, a thrread is started from the main program. The module logging is used to output some data.

import logging
import threading
import time

def thread_function(name):
logging.info("Thread %s: started", name)
time.sleep(2)
logging.info("Thread %s: ended", name)

if __name__ == "__main__":
format = "%(asctime)s: %(message)s"
logging.basicConfig(format=format, level=logging.INFO, datefmt="%H:%M:%S")

# Create thread (but not start)
logging.info("Main, before thread created")
my_thread = threading.Thread(target=thread_function, args=(1,))

# Start thread
logging.info("Main, before thread started")
my_thread.start()

logging.info("Main, wait for the thread to finish")
logging.info("Ending thread and program")

If you run the program, you’ll see it starts both the main thread and the thread my_thread, and then waits for both to exit.

➜  ~ python3 example.py
15:18:27: Main, before thread created
15:18:27: Main, before thread started
15:18:27: Thread 1: started
15:18:27: Main, wait for the thread to finish
15:18:27: Ending thread and program
15:18:29: Thread 1: ended

Working with many threads

The example code above only has two threads (the thread created and the main thread). You can create more than one thread in your program.

In a multi-threaded program, several processes happen at the same time and each thread is doing something useful for the end user.

You can start multiple threads by applying what you learned above, starting a thread with:

my_thread = threading.Thread(target=thread_function, args = (1,))

One key difference is that we use .join() to wait for the threads to finish. This waits for a thread to stop before it quits your program.

my_thread.join()

The join() method call tells a thread to wait for the other to finish.

import logging
import threading
import time

def thread_function(name):
logging.info("Thread %s: started", name)
time.sleep(2)
logging.info("Thread %s: ended", name)

if __name__ == "__main__":
format = "%(asctime)s: %(message)s"
logging.basicConfig(format=format, level=logging.INFO, datefmt="%H:%M:%S")
logging.info("Main, before thread created")

# Create multiple threads
threads = []
for index in range(5):
logging.info("Thread created")
my_thread = threading.Thread(target=thread_function, args=(1,))
threads.append(my_thread)

# Start thread
my_thread.start()

# use .join to wait for threads to finsih
for index, thread in enumerate(threads):
thread.join()
logging.info("thread %d ended", index)


logging.info("Main, wait for the thread to finish")

This then outputs something like this:

$ python3 threadsexample.py
15:13:31: Main, before thread created
15:13:31: Thread created
15:13:31: Thread 1: started
15:13:31: Thread created
15:13:31: Thread 1: started
15:13:31: Thread created
15:13:31: Thread 1: started
15:13:31: Thread created
15:13:31: Thread 1: started
15:13:31: Thread created
15:13:31: Thread 1: started
15:13:33: Thread 1: ended
15:13:33: Thread 1: ended
15:13:33: Thread 1: ended
15:13:33: Thread 1: ended
15:13:33: Thread 1: ended
15:13:33: thread 0 ended
15:13:33: thread 1 ended
15:13:33: thread 2 ended
15:13:33: thread 3 ended
15:13:33: thread 4 ended
15:13:33: Main, wait for the thread to finish

Threading with queues

we use the modules threading and queue to start several threads. A queue is created by calling Queue(). Then we have an array of urls as parameter for thread construction.

Threads are created in the for loop using the call threading.Thread(). Then they have to be started by calling thead.start(). Every thread calls the method getUrl.

 
import queue
import threading
import urllib.request

def getUrl(q, url):
print('getUrl('+url+') called from a thead.')
q.put(urllib.request.urlopen(url).read())

theurls = ["http://google.com", "http://google.de","http://google.ca"]

threadQueue = queue.Queue()

for u in theurls:
t = threading.Thread(target=getUrl, args = (threadQueue,u))
t.daemon = True
t.start()

output = threadQueue.get()
#print(output)

Graphically this is what happens. Your program (the main thread) starts several new threads which execute a task “in parallel”.

multithreading in python

A program can have any number of threads, each of which work on some data.

If you are a Python beginner, then I highly recommend this book.

Download exercises