I think it's a good advice to remove the "-j 8" option if it doesn't affect the performance much.
Udi Meiri <[email protected]> 于2020年2月12日周三 上午2:20写道： > For me the difference was about 20s longer (40s -> 60s approx). Not > significant IMO > > On Tue, Feb 11, 2020 at 9:59 AM Ahmet Altay <[email protected]> wrote: > >> Should we remove the "-j 8" option by default? Sphinx docs says this is >> an experimental option [1]. I do not recall docs generation taking a long >> time, does this increase significantly without this option? >> >> [1] http://www.sphinx-doc.org/en/stable/man/sphinx-build.html >> >> On Tue, Feb 11, 2020 at 1:16 AM Shoaib Zafar < >> [email protected]> wrote: >> >>> Thanks, Udi and Jincheng for the response. >>> The suggested solution worked for me as well. >>> >>> Regards, >>> >>> *Shoaib Zafar* >>> Software Engineering Lead >>> Mobile: +92 333 274 6242 >>> Skype: live:shoaibzafar_1 >>> >>> <http://venturedive.com/> >>> >>> >>> On Tue, Feb 11, 2020 at 1:17 PM jincheng sun <[email protected]> >>> wrote: >>> >>>> I have verified that this issue could be reproduced in my local >>>> environment (MacOS) and the solution suggested by Udi could work! >>>> >>>> Best, >>>> Jincheng >>>> >>>> Udi Meiri <[email protected]> 于2020年2月11日周二 上午8:51写道： >>>> >>>>> I don't have those issues (running on Linux), but a possible >>>>> workaround could be to remove the "-j 8" flags (2 locations) in >>>>> generate_pydoc.sh. >>>>> >>>>> >>>>> On Mon, Feb 10, 2020 at 11:06 AM Shoaib Zafar < >>>>> [email protected]> wrote: >>>>> >>>>>> Hello Beamers. >>>>>> >>>>>> Just curious does anyone having trouble running >>>>>> ':sdks:python:test-suites:tox:pycommon:docs' command locally? >>>>>> >>>>>> After rebasing with master recently, I am facing sphinx thread fork >>>>>> error with on my macos mojave, using python 3.7.0. >>>>>> I Tried to add system variable "export >>>>>> OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES" (which I found on google) >>>>>> but no luck! >>>>>> >>>>>> Any suggestions/help? >>>>>> >>>>>> Thanks! >>>>>> >>>>>> Console Log: >>>>>> ------------------------------------------------------ >>>>>> .... >>>>>> Creating file target/docs/source/apache_beam.utils.proto_utils.rst. >>>>>> Creating file target/docs/source/apache_beam.utils.retry.rst. >>>>>> Creating file >>>>>> target/docs/source/apache_beam.utils.subprocess_server.rst. >>>>>> Creating file >>>>>> target/docs/source/apache_beam.utils.thread_pool_executor.rst. >>>>>> Creating file target/docs/source/apache_beam.utils.timestamp.rst. >>>>>> Creating file target/docs/source/apache_beam.utils.urns.rst. >>>>>> Creating file target/docs/source/apache_beam.utils.rst. >>>>>> objc[8384]: +[__NSCFConstantString initialize] may have been in >>>>>> progress in another thread when fork() was called. >>>>>> objc[8384]: +[__NSCFConstantString initialize] may have been in >>>>>> progress in another thread when fork() was called. We cannot safely call >>>>>> it >>>>>> or ignore it in the fork() child process. Crashing instead. Set a >>>>>> breakpoint on objc_initializeAfterForkError to debug. >>>>>> >>>>>> Traceback (most recent call last): >>>>>> File >>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/cmd/build.py", >>>>>> line 304, in build_main >>>>>> app.build(args.force_all, filenames) >>>>>> File >>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/application.py", >>>>>> line 335, in build >>>>>> self.builder.build_all() >>>>>> File >>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py", >>>>>> line 305, in build_all >>>>>> self.build(None, summary=__('all source files'), method='all') >>>>>> File >>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py", >>>>>> line 360, in build >>>>>> updated_docnames = set(self.read()) >>>>>> File >>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py", >>>>>> line 466, in read >>>>>> self._read_parallel(docnames, nproc=self.app.parallel) >>>>>> File >>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py", >>>>>> line 521, in _read_parallel >>>>>> tasks.join() >>>>>> File >>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py", >>>>>> line 114, in join >>>>>> self._join_one() >>>>>> File >>>>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py", >>>>>> line 120, in _join_one >>>>>> exc, logs, result = pipe.recv() >>>>>> File >>>>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py", >>>>>> line 250, in recv >>>>>> buf = self._recv_bytes() >>>>>> File >>>>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py", >>>>>> line 407, in _recv_bytes >>>>>> buf = self._recv(4) >>>>>> File >>>>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py", >>>>>> line 383, in _recv >>>>>> raise EOFError >>>>>> EOFError >>>>>> >>>>>> Exception occurred: >>>>>> File >>>>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py", >>>>>> line 383, in _recv >>>>>> raise EOFError >>>>>> EOFError >>>>>> The full traceback has been saved in >>>>>> /Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/tmp/sphinx-err-mphtfnei.log, >>>>>> if you want to report the issue to the developers. >>>>>> Please also report this if it was a user error, so that a better >>>>>> error message can be provided next time. >>>>>> A bug report can be filed in the tracker at < >>>>>> https://github.com/sphinx-doc/sphinx/issues>. Thanks! >>>>>> objc[8385]: +[__NSCFConstantString initialize] may have been in >>>>>> progress in another thread when fork() was called. >>>>>> objc[8385]: +[__NSCFConstantString initialize] may have been in >>>>>> progress in another thread when fork() was called. We cannot safely call >>>>>> it >>>>>> or ignore it in the fork() child process. Crashing instead. Set a >>>>>> breakpoint on objc_initializeAfterForkError to debug. >>>>>> objc[8386]: +[__NSCFConstantString initialize] may have been in >>>>>> progress in another thread when fork() was called. >>>>>> objc[8386]: +[__NSCFConstantString initialize] may have been in >>>>>> progress in another thread when fork() was called. We cannot safely call >>>>>> it >>>>>> or ignore it in the fork() child process. Crashing instead. Set a >>>>>> breakpoint on objc_initializeAfterForkError to debug. >>>>>> .... >>>>>> .... >>>>>> .... >>>>>> 9.54 real 4.59 user 1.69 sys >>>>>> ERROR: InvocationError for command /usr/bin/time >>>>>> scripts/generate_pydoc.sh (exited with code 2) >>>>>> py37-docs run-test-post: commands[0] | >>>>>> /Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/scripts/run_tox_cleanup.sh >>>>>> ___________________________________ summary >>>>>> ____________________________________ >>>>>> ERROR: py37-docs: commands failed >>>>>> >>>>>> > Task :sdks:python:test-suites:tox:pycommon:docs FAILED >>>>>> >>>>>> FAILURE: Build failed with an exception. >>>>>> >>>>>> * What went wrong: >>>>>> Execution failed for task >>>>>> ':sdks:python:test-suites:tox:pycommon:docs'. >>>>>> > Process 'command 'sh'' finished with non-zero exit value 1 >>>>>> ------------------------------------------------------ >>>>>> >>>>>> >>>>>> >>>>>> *Shoaib Zafar* >>>>>> Software Engineering Lead >>>>>> Mobile: +92 333 274 6242 >>>>>> Skype: live:shoaibzafar_1 >>>>>> >>>>>> <http://venturedive.com/> >>>>>> >>>>>
