Should we remove the "-j 8" option by default? Sphinx docs says this is an experimental option [1]. I do not recall docs generation taking a long time, does this increase significantly without this option?
[1] http://www.sphinx-doc.org/en/stable/man/sphinx-build.html On Tue, Feb 11, 2020 at 1:16 AM Shoaib Zafar <shoaib.za...@venturedive.com> wrote: > Thanks, Udi and Jincheng for the response. > The suggested solution worked for me as well. > > Regards, > > *Shoaib Zafar* > Software Engineering Lead > Mobile: +92 333 274 6242 > Skype: live:shoaibzafar_1 > > <http://venturedive.com/> > > > On Tue, Feb 11, 2020 at 1:17 PM jincheng sun <sunjincheng...@gmail.com> > wrote: > >> I have verified that this issue could be reproduced in my local >> environment (MacOS) and the solution suggested by Udi could work! >> >> Best, >> Jincheng >> >> Udi Meiri <eh...@google.com> 于2020年2月11日周二 上午8:51写道： >> >>> I don't have those issues (running on Linux), but a possible workaround >>> could be to remove the "-j 8" flags (2 locations) in generate_pydoc.sh. >>> >>> >>> On Mon, Feb 10, 2020 at 11:06 AM Shoaib Zafar < >>> shoaib.za...@venturedive.com> wrote: >>> >>>> Hello Beamers. >>>> >>>> Just curious does anyone having trouble running >>>> ':sdks:python:test-suites:tox:pycommon:docs' command locally? >>>> >>>> After rebasing with master recently, I am facing sphinx thread fork >>>> error with on my macos mojave, using python 3.7.0. >>>> I Tried to add system variable "export >>>> OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES" (which I found on google) but >>>> no luck! >>>> >>>> Any suggestions/help? >>>> >>>> Thanks! >>>> >>>> Console Log: >>>> ------------------------------------------------------ >>>> .... >>>> Creating file target/docs/source/apache_beam.utils.proto_utils.rst. >>>> Creating file target/docs/source/apache_beam.utils.retry.rst. >>>> Creating file >>>> target/docs/source/apache_beam.utils.subprocess_server.rst. >>>> Creating file >>>> target/docs/source/apache_beam.utils.thread_pool_executor.rst. >>>> Creating file target/docs/source/apache_beam.utils.timestamp.rst. >>>> Creating file target/docs/source/apache_beam.utils.urns.rst. >>>> Creating file target/docs/source/apache_beam.utils.rst. >>>> objc[8384]: +[__NSCFConstantString initialize] may have been in >>>> progress in another thread when fork() was called. >>>> objc[8384]: +[__NSCFConstantString initialize] may have been in >>>> progress in another thread when fork() was called. We cannot safely call it >>>> or ignore it in the fork() child process. Crashing instead. Set a >>>> breakpoint on objc_initializeAfterForkError to debug. >>>> >>>> Traceback (most recent call last): >>>> File >>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/cmd/build.py", >>>> line 304, in build_main >>>> app.build(args.force_all, filenames) >>>> File >>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/application.py", >>>> line 335, in build >>>> self.builder.build_all() >>>> File >>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py", >>>> line 305, in build_all >>>> self.build(None, summary=__('all source files'), method='all') >>>> File >>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py", >>>> line 360, in build >>>> updated_docnames = set(self.read()) >>>> File >>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py", >>>> line 466, in read >>>> self._read_parallel(docnames, nproc=self.app.parallel) >>>> File >>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py", >>>> line 521, in _read_parallel >>>> tasks.join() >>>> File >>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py", >>>> line 114, in join >>>> self._join_one() >>>> File >>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py", >>>> line 120, in _join_one >>>> exc, logs, result = pipe.recv() >>>> File >>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py", >>>> line 250, in recv >>>> buf = self._recv_bytes() >>>> File >>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py", >>>> line 407, in _recv_bytes >>>> buf = self._recv(4) >>>> File >>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py", >>>> line 383, in _recv >>>> raise EOFError >>>> EOFError >>>> >>>> Exception occurred: >>>> File >>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py", >>>> line 383, in _recv >>>> raise EOFError >>>> EOFError >>>> The full traceback has been saved in >>>> /Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/tmp/sphinx-err-mphtfnei.log, >>>> if you want to report the issue to the developers. >>>> Please also report this if it was a user error, so that a better error >>>> message can be provided next time. >>>> A bug report can be filed in the tracker at < >>>> https://github.com/sphinx-doc/sphinx/issues>. Thanks! >>>> objc[8385]: +[__NSCFConstantString initialize] may have been in >>>> progress in another thread when fork() was called. >>>> objc[8385]: +[__NSCFConstantString initialize] may have been in >>>> progress in another thread when fork() was called. We cannot safely call it >>>> or ignore it in the fork() child process. Crashing instead. Set a >>>> breakpoint on objc_initializeAfterForkError to debug. >>>> objc[8386]: +[__NSCFConstantString initialize] may have been in >>>> progress in another thread when fork() was called. >>>> objc[8386]: +[__NSCFConstantString initialize] may have been in >>>> progress in another thread when fork() was called. We cannot safely call it >>>> or ignore it in the fork() child process. Crashing instead. Set a >>>> breakpoint on objc_initializeAfterForkError to debug. >>>> .... >>>> .... >>>> .... >>>> 9.54 real 4.59 user 1.69 sys >>>> ERROR: InvocationError for command /usr/bin/time >>>> scripts/generate_pydoc.sh (exited with code 2) >>>> py37-docs run-test-post: commands[0] | >>>> /Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/scripts/run_tox_cleanup.sh >>>> ___________________________________ summary >>>> ____________________________________ >>>> ERROR: py37-docs: commands failed >>>> >>>> > Task :sdks:python:test-suites:tox:pycommon:docs FAILED >>>> >>>> FAILURE: Build failed with an exception. >>>> >>>> * What went wrong: >>>> Execution failed for task ':sdks:python:test-suites:tox:pycommon:docs'. >>>> > Process 'command 'sh'' finished with non-zero exit value 1 >>>> ------------------------------------------------------ >>>> >>>> >>>> >>>> *Shoaib Zafar* >>>> Software Engineering Lead >>>> Mobile: +92 333 274 6242 >>>> Skype: live:shoaibzafar_1 >>>> >>>> <http://venturedive.com/> >>>> >>>
