Feature #10351
open
Try more safe process creation method
Added by Evgeny Novikov over 4 years ago.
Updated over 2 years ago.
Category:
Infrastructure of Core
Description
#9186 does not help to avoid hang ups. Indeed, hang ups can happen due to tricky deadlocks happen in child processes. If we will use more safe process creation method.
- Priority changed from Urgent to High
- Target version deleted (
3.0)
By default on Linux Python multiprocessing module starts child processes using "fork" start method. There are a number of issues with this method, as outlined in this article. It is quite possible that "fork" is the reason of rare freezes that happen in Klever from time to time. The solution is to tell multiprocessing to use "spawn" start method instead. Example:
import multiprocessing as mp
def foo(q):
q.put('hello')
if __name__ == '__main__':
mp.set_start_method('spawn')
q = mp.Queue()
p = mp.Process(target=foo, args=(q,))
p.start()
print(q.get())
p.join()
However, there are a few extra restrictions which code must satisfy to in order to be used with "spawn". These restrictions are outlined in the Python documentation. Currently converting native scheduler to use "spawn" is rather easy, but there are a lot of issues that arise during converting Klever.Core. Due to this the implementation is postponed until after the release of Klever 3.0.
- Assignee deleted (
Ilya Shchepetkov)
Also available in: Atom
PDF