merrymercy commented on a change in pull request #6671:
URL: https://github.com/apache/incubator-tvm/pull/6671#discussion_r511923166
##########
File path: python/tvm/auto_scheduler/measure.py
##########
@@ -590,14 +634,20 @@ def local_builder_build(inputs, timeout, n_parallel,
build_func="default", verbo
res : List[BuildResult]
The build results of these MeasureInputs.
"""
- # We use fork and a global variable to copy arguments between processes.
- # This can avoid expensive serialization of TVM IR when using
multiprocessing.Pool
- global GLOBAL_BUILD_ARGUMENTS
-
- GLOBAL_BUILD_ARGUMENTS = (inputs, build_func, timeout, verbose)
-
- pool = NoDaemonPool(n_parallel)
- tuple_res = pool.map(local_build_worker, range(len(inputs)))
+ # This pool is not doing computationally intensive work, so we can use
threads
+ pool = multiprocessing.pool.ThreadPool(n_parallel)
Review comment:
Do we still need serialization with `ThreadPool`?
I guess all these arguments will be shared in memory and will be passed as
references.
If this is true, then we don't need to change any other part of the code in
this file.
In addition, we can choose to use `ProcessingPool` and `ThreadPool` here
according to the OS.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
