Join us and get access to thousands of tutorials and a community of expert Pythonistas.

Unlock This Lesson

This lesson is for members only. Join us and get access to thousands of tutorials and a community of expert Pythonistas.

Unlock This Lesson

Hint: You can adjust the default video playback speed in your account settings.
Hint: You can set your subtitle preferences in your account settings.
Sorry! Looks like there’s an issue with video playback 🙁 This might be due to a temporary outage or because of a configuration issue with your browser. Please refer to our video player troubleshooting guide for assistance.

concurrent.futures vs multiprocessing

In this lesson, you’ll see why you might want to use concurrent.futures rather than multiprocessing. One point to consider is that concurrent.futures provides a couple different implementations that allow you to easily change how your computations are happening in parallel.

In the next lesson, you’ll see which situations might be better suited to using either concurrent.futures or multiprocessing. You’ll also learn about how that ties in with the Global Interpreter Lock (GIL).

tinachoudhary on April 6, 2020

Thanks a lot for the tutorial, it is explained well.

Zarata on May 7, 2020

Well! Per comment two(?) videos ago, now my simple concept of one process per thread, one thread per core is truly all topsy-turvey. The ThreadPoolExecutor is running 7 threads concurrently in one process. Is it fair to presume on one core, or is the mother process doling threads across cores? Is the OS time slicing between the threads so that all complete with near simultaneity? Is the child, Melissa, Dr. Stuben’s long lost daughter? The inquiring mind wants to know! (Who is that man behind the curtain? I can’t ignore him …)

macro84 on Oct. 14, 2021

Strange… it works perfectly fine with with “concurrent.futures.ThreadPoolExecutor() as executor:” but I get error message when using with “concurrent.futures.ProcessPoolExecutor() as executor:” Error message: BrokenProcessPool: A process in the process pool was terminated abruptly while the future was running or pending.

Bartosz Zaczyński RP Team on Oct. 14, 2021

@hlaret It works for me with both pools. What kind of operating system are you on, and which Python version are you using?

macro84 on Nov. 10, 2021

Windows 10 and Python 3.8.3

Bartosz Zaczyński RP Team on Nov. 10, 2021

@macro84 I haven’t used Windows in a long time, but this rings a bell. Did you make sure to wrap your code in the if __name__ == "__main__": condition?

There’s an explanation in the official documentation of the multiprocessing module:

Make sure that the main module can be safely imported by a new Python interpreter without causing unintended side effects (such a starting a new process).


Instead one should protect the “entry point” of the program by using if __name__ == '__main__': as follows:


macro84 on Dec. 20, 2021


Become a Member to join the conversation.