Re: Sphinx Docs Command Error (:sdks:python:test-suites:tox:pycommon:docs)

[Ahmet Altay]

Sent https://github.com/apache/beam/pull/10874 to remove this flag.

On Tue, Feb 11, 2020 at 5:38 PM jincheng sun 
wrote:

> I think it's a good advice to remove the "-j 8" option if it doesn't
> affect the performance much.
>
>
> Udi Meiri  于2020年2月12日周三 上午2:20写道：
>
>> For me the difference was about 20s longer (40s -> 60s approx). Not
>> significant IMO
>>
>> On Tue, Feb 11, 2020 at 9:59 AM Ahmet Altay  wrote:
>>
>>> Should we remove the "-j 8" option by default? Sphinx docs says this is
>>> an experimental option [1]. I do not recall docs generation taking a long
>>> time, does this increase significantly without this option?
>>>
>>> [1] http://www.sphinx-doc.org/en/stable/man/sphinx-build.html
>>>
>>> On Tue, Feb 11, 2020 at 1:16 AM Shoaib Zafar <
>>> shoaib.za...@venturedive.com> wrote:
>>>
 Thanks, Udi and Jincheng for the response.
 The suggested solution worked for me as well.

 Regards,

 *Shoaib Zafar*
 Software Engineering Lead
 Mobile: +92 333 274 6242
 Skype: live:shoaibzafar_1

 


 On Tue, Feb 11, 2020 at 1:17 PM jincheng sun 
 wrote:

> I have verified that this issue could be reproduced in my local
> environment (MacOS) and the solution suggested by Udi could work!
>
> Best,
> Jincheng
>
> Udi Meiri  于2020年2月11日周二 上午8:51写道：
>
>> I don't have those issues (running on Linux), but a possible
>> workaround could be to remove the "-j 8" flags (2 locations) in
>> generate_pydoc.sh.
>>
>>
>> On Mon, Feb 10, 2020 at 11:06 AM Shoaib Zafar <
>> shoaib.za...@venturedive.com> wrote:
>>
>>> Hello Beamers.
>>>
>>> Just curious does anyone having trouble running
>>> ':sdks:python:test-suites:tox:pycommon:docs' command locally?
>>>
>>> After rebasing with master recently, I am facing sphinx thread fork
>>> error with on my macos mojave, using python 3.7.0.
>>> I Tried to add system variable "export
>>> OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES" (which I found on google)
>>> but no luck!
>>>
>>> Any suggestions/help?
>>>
>>> Thanks!
>>>
>>> Console Log:
>>> --
>>> 
>>> Creating file target/docs/source/apache_beam.utils.proto_utils.rst.
>>> Creating file target/docs/source/apache_beam.utils.retry.rst.
>>> Creating file
>>> target/docs/source/apache_beam.utils.subprocess_server.rst.
>>> Creating file
>>> target/docs/source/apache_beam.utils.thread_pool_executor.rst.
>>> Creating file target/docs/source/apache_beam.utils.timestamp.rst.
>>> Creating file target/docs/source/apache_beam.utils.urns.rst.
>>> Creating file target/docs/source/apache_beam.utils.rst.
>>> objc[8384]: +[__NSCFConstantString initialize] may have been in
>>> progress in another thread when fork() was called.
>>> objc[8384]: +[__NSCFConstantString initialize] may have been in
>>> progress in another thread when fork() was called. We cannot safely 
>>> call it
>>> or ignore it in the fork() child process. Crashing instead. Set a
>>> breakpoint on objc_initializeAfterForkError to debug.
>>>
>>> Traceback (most recent call last):
>>>   File
>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/cmd/build.py",
>>> line 304, in build_main
>>> app.build(args.force_all, filenames)
>>>   File
>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/application.py",
>>> line 335, in build
>>> self.builder.build_all()
>>>   File
>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>>> line 305, in build_all
>>> self.build(None, summary=__('all source files'), method='all')
>>>   File
>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>>> line 360, in build
>>> updated_docnames = set(self.read())
>>>   File
>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>>> line 466, in read
>>> self._read_parallel(docnames, nproc=self.app.parallel)
>>>   File
>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>>> line 521, in _read_parallel

Re: Sphinx Docs Command Error (:sdks:python:test-suites:tox:pycommon:docs)

[jincheng sun]

I think it's a good advice to remove the "-j 8" option if it doesn't affect
the performance much.


Udi Meiri  于2020年2月12日周三 上午2:20写道：

> For me the difference was about 20s longer (40s -> 60s approx). Not
> significant IMO
>
> On Tue, Feb 11, 2020 at 9:59 AM Ahmet Altay  wrote:
>
>> Should we remove the "-j 8" option by default? Sphinx docs says this is
>> an experimental option [1]. I do not recall docs generation taking a long
>> time, does this increase significantly without this option?
>>
>> [1] http://www.sphinx-doc.org/en/stable/man/sphinx-build.html
>>
>> On Tue, Feb 11, 2020 at 1:16 AM Shoaib Zafar <
>> shoaib.za...@venturedive.com> wrote:
>>
>>> Thanks, Udi and Jincheng for the response.
>>> The suggested solution worked for me as well.
>>>
>>> Regards,
>>>
>>> *Shoaib Zafar*
>>> Software Engineering Lead
>>> Mobile: +92 333 274 6242
>>> Skype: live:shoaibzafar_1
>>>
>>> 
>>>
>>>
>>> On Tue, Feb 11, 2020 at 1:17 PM jincheng sun 
>>> wrote:
>>>
 I have verified that this issue could be reproduced in my local
 environment (MacOS) and the solution suggested by Udi could work!

 Best,
 Jincheng

 Udi Meiri  于2020年2月11日周二 上午8:51写道：

> I don't have those issues (running on Linux), but a possible
> workaround could be to remove the "-j 8" flags (2 locations) in
> generate_pydoc.sh.
>
>
> On Mon, Feb 10, 2020 at 11:06 AM Shoaib Zafar <
> shoaib.za...@venturedive.com> wrote:
>
>> Hello Beamers.
>>
>> Just curious does anyone having trouble running
>> ':sdks:python:test-suites:tox:pycommon:docs' command locally?
>>
>> After rebasing with master recently, I am facing sphinx thread fork
>> error with on my macos mojave, using python 3.7.0.
>> I Tried to add system variable "export
>> OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES" (which I found on google)
>> but no luck!
>>
>> Any suggestions/help?
>>
>> Thanks!
>>
>> Console Log:
>> --
>> 
>> Creating file target/docs/source/apache_beam.utils.proto_utils.rst.
>> Creating file target/docs/source/apache_beam.utils.retry.rst.
>> Creating file
>> target/docs/source/apache_beam.utils.subprocess_server.rst.
>> Creating file
>> target/docs/source/apache_beam.utils.thread_pool_executor.rst.
>> Creating file target/docs/source/apache_beam.utils.timestamp.rst.
>> Creating file target/docs/source/apache_beam.utils.urns.rst.
>> Creating file target/docs/source/apache_beam.utils.rst.
>> objc[8384]: +[__NSCFConstantString initialize] may have been in
>> progress in another thread when fork() was called.
>> objc[8384]: +[__NSCFConstantString initialize] may have been in
>> progress in another thread when fork() was called. We cannot safely call 
>> it
>> or ignore it in the fork() child process. Crashing instead. Set a
>> breakpoint on objc_initializeAfterForkError to debug.
>>
>> Traceback (most recent call last):
>>   File
>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/cmd/build.py",
>> line 304, in build_main
>> app.build(args.force_all, filenames)
>>   File
>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/application.py",
>> line 335, in build
>> self.builder.build_all()
>>   File
>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>> line 305, in build_all
>> self.build(None, summary=__('all source files'), method='all')
>>   File
>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>> line 360, in build
>> updated_docnames = set(self.read())
>>   File
>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>> line 466, in read
>> self._read_parallel(docnames, nproc=self.app.parallel)
>>   File
>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>> line 521, in _read_parallel
>> tasks.join()
>>   File
>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py",
>> 

Re: Sphinx Docs Command Error (:sdks:python:test-suites:tox:pycommon:docs)

[Udi Meiri]

For me the difference was about 20s longer (40s -> 60s approx). Not
significant IMO

On Tue, Feb 11, 2020 at 9:59 AM Ahmet Altay  wrote:

> Should we remove the "-j 8" option by default? Sphinx docs says this is an
> experimental option [1]. I do not recall docs generation taking a long
> time, does this increase significantly without this option?
>
> [1] http://www.sphinx-doc.org/en/stable/man/sphinx-build.html
>
> On Tue, Feb 11, 2020 at 1:16 AM Shoaib Zafar 
> wrote:
>
>> Thanks, Udi and Jincheng for the response.
>> The suggested solution worked for me as well.
>>
>> Regards,
>>
>> *Shoaib Zafar*
>> Software Engineering Lead
>> Mobile: +92 333 274 6242
>> Skype: live:shoaibzafar_1
>>
>> 
>>
>>
>> On Tue, Feb 11, 2020 at 1:17 PM jincheng sun 
>> wrote:
>>
>>> I have verified that this issue could be reproduced in my local
>>> environment (MacOS) and the solution suggested by Udi could work!
>>>
>>> Best,
>>> Jincheng
>>>
>>> Udi Meiri  于2020年2月11日周二 上午8:51写道：
>>>
 I don't have those issues (running on Linux), but a possible workaround
 could be to remove the "-j 8" flags (2 locations) in generate_pydoc.sh.


 On Mon, Feb 10, 2020 at 11:06 AM Shoaib Zafar <
 shoaib.za...@venturedive.com> wrote:

> Hello Beamers.
>
> Just curious does anyone having trouble running
> ':sdks:python:test-suites:tox:pycommon:docs' command locally?
>
> After rebasing with master recently, I am facing sphinx thread fork
> error with on my macos mojave, using python 3.7.0.
> I Tried to add system variable "export
> OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES" (which I found on google)
> but no luck!
>
> Any suggestions/help?
>
> Thanks!
>
> Console Log:
> --
> 
> Creating file target/docs/source/apache_beam.utils.proto_utils.rst.
> Creating file target/docs/source/apache_beam.utils.retry.rst.
> Creating file
> target/docs/source/apache_beam.utils.subprocess_server.rst.
> Creating file
> target/docs/source/apache_beam.utils.thread_pool_executor.rst.
> Creating file target/docs/source/apache_beam.utils.timestamp.rst.
> Creating file target/docs/source/apache_beam.utils.urns.rst.
> Creating file target/docs/source/apache_beam.utils.rst.
> objc[8384]: +[__NSCFConstantString initialize] may have been in
> progress in another thread when fork() was called.
> objc[8384]: +[__NSCFConstantString initialize] may have been in
> progress in another thread when fork() was called. We cannot safely call 
> it
> or ignore it in the fork() child process. Crashing instead. Set a
> breakpoint on objc_initializeAfterForkError to debug.
>
> Traceback (most recent call last):
>   File
> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/cmd/build.py",
> line 304, in build_main
> app.build(args.force_all, filenames)
>   File
> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/application.py",
> line 335, in build
> self.builder.build_all()
>   File
> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
> line 305, in build_all
> self.build(None, summary=__('all source files'), method='all')
>   File
> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
> line 360, in build
> updated_docnames = set(self.read())
>   File
> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
> line 466, in read
> self._read_parallel(docnames, nproc=self.app.parallel)
>   File
> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
> line 521, in _read_parallel
> tasks.join()
>   File
> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py",
> line 114, in join
> self._join_one()
>   File
> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py",
> line 120, in _join_one
> 

Re: Sphinx Docs Command Error (:sdks:python:test-suites:tox:pycommon:docs)

[Ahmet Altay]

Should we remove the "-j 8" option by default? Sphinx docs says this is an
experimental option [1]. I do not recall docs generation taking a long
time, does this increase significantly without this option?

[1] http://www.sphinx-doc.org/en/stable/man/sphinx-build.html

On Tue, Feb 11, 2020 at 1:16 AM Shoaib Zafar 
wrote:

> Thanks, Udi and Jincheng for the response.
> The suggested solution worked for me as well.
>
> Regards,
>
> *Shoaib Zafar*
> Software Engineering Lead
> Mobile: +92 333 274 6242
> Skype: live:shoaibzafar_1
>
> 
>
>
> On Tue, Feb 11, 2020 at 1:17 PM jincheng sun 
> wrote:
>
>> I have verified that this issue could be reproduced in my local
>> environment (MacOS) and the solution suggested by Udi could work!
>>
>> Best,
>> Jincheng
>>
>> Udi Meiri  于2020年2月11日周二 上午8:51写道：
>>
>>> I don't have those issues (running on Linux), but a possible workaround
>>> could be to remove the "-j 8" flags (2 locations) in generate_pydoc.sh.
>>>
>>>
>>> On Mon, Feb 10, 2020 at 11:06 AM Shoaib Zafar <
>>> shoaib.za...@venturedive.com> wrote:
>>>
 Hello Beamers.

 Just curious does anyone having trouble running
 ':sdks:python:test-suites:tox:pycommon:docs' command locally?

 After rebasing with master recently, I am facing sphinx thread fork
 error with on my macos mojave, using python 3.7.0.
 I Tried to add system variable "export
 OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES" (which I found on google) but
 no luck!

 Any suggestions/help?

 Thanks!

 Console Log:
 --
 
 Creating file target/docs/source/apache_beam.utils.proto_utils.rst.
 Creating file target/docs/source/apache_beam.utils.retry.rst.
 Creating file
 target/docs/source/apache_beam.utils.subprocess_server.rst.
 Creating file
 target/docs/source/apache_beam.utils.thread_pool_executor.rst.
 Creating file target/docs/source/apache_beam.utils.timestamp.rst.
 Creating file target/docs/source/apache_beam.utils.urns.rst.
 Creating file target/docs/source/apache_beam.utils.rst.
 objc[8384]: +[__NSCFConstantString initialize] may have been in
 progress in another thread when fork() was called.
 objc[8384]: +[__NSCFConstantString initialize] may have been in
 progress in another thread when fork() was called. We cannot safely call it
 or ignore it in the fork() child process. Crashing instead. Set a
 breakpoint on objc_initializeAfterForkError to debug.

 Traceback (most recent call last):
   File
 "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/cmd/build.py",
 line 304, in build_main
 app.build(args.force_all, filenames)
   File
 "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/application.py",
 line 335, in build
 self.builder.build_all()
   File
 "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
 line 305, in build_all
 self.build(None, summary=__('all source files'), method='all')
   File
 "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
 line 360, in build
 updated_docnames = set(self.read())
   File
 "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
 line 466, in read
 self._read_parallel(docnames, nproc=self.app.parallel)
   File
 "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
 line 521, in _read_parallel
 tasks.join()
   File
 "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py",
 line 114, in join
 self._join_one()
   File
 "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py",
 line 120, in _join_one
 exc, logs, result = pipe.recv()
   File
 "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
 line 250, in recv
 buf = self._recv_bytes()
   File
 

Re: Sphinx Docs Command Error (:sdks:python:test-suites:tox:pycommon:docs)

[Shoaib Zafar]

Thanks, Udi and Jincheng for the response.
The suggested solution worked for me as well.

Regards,

*Shoaib Zafar*
Software Engineering Lead
Mobile: +92 333 274 6242
Skype: live:shoaibzafar_1




On Tue, Feb 11, 2020 at 1:17 PM jincheng sun 
wrote:

> I have verified that this issue could be reproduced in my local
> environment (MacOS) and the solution suggested by Udi could work!
>
> Best,
> Jincheng
>
> Udi Meiri  于2020年2月11日周二 上午8:51写道：
>
>> I don't have those issues (running on Linux), but a possible workaround
>> could be to remove the "-j 8" flags (2 locations) in generate_pydoc.sh.
>>
>>
>> On Mon, Feb 10, 2020 at 11:06 AM Shoaib Zafar <
>> shoaib.za...@venturedive.com> wrote:
>>
>>> Hello Beamers.
>>>
>>> Just curious does anyone having trouble running
>>> ':sdks:python:test-suites:tox:pycommon:docs' command locally?
>>>
>>> After rebasing with master recently, I am facing sphinx thread fork
>>> error with on my macos mojave, using python 3.7.0.
>>> I Tried to add system variable "export
>>> OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES" (which I found on google) but
>>> no luck!
>>>
>>> Any suggestions/help?
>>>
>>> Thanks!
>>>
>>> Console Log:
>>> --
>>> 
>>> Creating file target/docs/source/apache_beam.utils.proto_utils.rst.
>>> Creating file target/docs/source/apache_beam.utils.retry.rst.
>>> Creating file target/docs/source/apache_beam.utils.subprocess_server.rst.
>>> Creating file
>>> target/docs/source/apache_beam.utils.thread_pool_executor.rst.
>>> Creating file target/docs/source/apache_beam.utils.timestamp.rst.
>>> Creating file target/docs/source/apache_beam.utils.urns.rst.
>>> Creating file target/docs/source/apache_beam.utils.rst.
>>> objc[8384]: +[__NSCFConstantString initialize] may have been in progress
>>> in another thread when fork() was called.
>>> objc[8384]: +[__NSCFConstantString initialize] may have been in progress
>>> in another thread when fork() was called. We cannot safely call it or
>>> ignore it in the fork() child process. Crashing instead. Set a breakpoint
>>> on objc_initializeAfterForkError to debug.
>>>
>>> Traceback (most recent call last):
>>>   File
>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/cmd/build.py",
>>> line 304, in build_main
>>> app.build(args.force_all, filenames)
>>>   File
>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/application.py",
>>> line 335, in build
>>> self.builder.build_all()
>>>   File
>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>>> line 305, in build_all
>>> self.build(None, summary=__('all source files'), method='all')
>>>   File
>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>>> line 360, in build
>>> updated_docnames = set(self.read())
>>>   File
>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>>> line 466, in read
>>> self._read_parallel(docnames, nproc=self.app.parallel)
>>>   File
>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>>> line 521, in _read_parallel
>>> tasks.join()
>>>   File
>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py",
>>> line 114, in join
>>> self._join_one()
>>>   File
>>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py",
>>> line 120, in _join_one
>>> exc, logs, result = pipe.recv()
>>>   File
>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>>> line 250, in recv
>>> buf = self._recv_bytes()
>>>   File
>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>>> line 407, in _recv_bytes
>>> buf = self._recv(4)
>>>   File
>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>>> line 383, in _recv
>>> raise EOFError
>>> EOFError
>>>
>>> Exception occurred:
>>>   File
>>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>>> line 383, in _recv
>>> raise EOFError
>>> EOFError
>>> The 

Re: Sphinx Docs Command Error (:sdks:python:test-suites:tox:pycommon:docs)

[jincheng sun]

I have verified that this issue could be reproduced in my local environment
(MacOS) and the solution suggested by Udi could work!

Best,
Jincheng

Udi Meiri  于2020年2月11日周二 上午8:51写道：

> I don't have those issues (running on Linux), but a possible workaround
> could be to remove the "-j 8" flags (2 locations) in generate_pydoc.sh.
>
>
> On Mon, Feb 10, 2020 at 11:06 AM Shoaib Zafar <
> shoaib.za...@venturedive.com> wrote:
>
>> Hello Beamers.
>>
>> Just curious does anyone having trouble running
>> ':sdks:python:test-suites:tox:pycommon:docs' command locally?
>>
>> After rebasing with master recently, I am facing sphinx thread fork error
>> with on my macos mojave, using python 3.7.0.
>> I Tried to add system variable "export
>> OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES" (which I found on google) but
>> no luck!
>>
>> Any suggestions/help?
>>
>> Thanks!
>>
>> Console Log:
>> --
>> 
>> Creating file target/docs/source/apache_beam.utils.proto_utils.rst.
>> Creating file target/docs/source/apache_beam.utils.retry.rst.
>> Creating file target/docs/source/apache_beam.utils.subprocess_server.rst.
>> Creating file
>> target/docs/source/apache_beam.utils.thread_pool_executor.rst.
>> Creating file target/docs/source/apache_beam.utils.timestamp.rst.
>> Creating file target/docs/source/apache_beam.utils.urns.rst.
>> Creating file target/docs/source/apache_beam.utils.rst.
>> objc[8384]: +[__NSCFConstantString initialize] may have been in progress
>> in another thread when fork() was called.
>> objc[8384]: +[__NSCFConstantString initialize] may have been in progress
>> in another thread when fork() was called. We cannot safely call it or
>> ignore it in the fork() child process. Crashing instead. Set a breakpoint
>> on objc_initializeAfterForkError to debug.
>>
>> Traceback (most recent call last):
>>   File
>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/cmd/build.py",
>> line 304, in build_main
>> app.build(args.force_all, filenames)
>>   File
>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/application.py",
>> line 335, in build
>> self.builder.build_all()
>>   File
>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>> line 305, in build_all
>> self.build(None, summary=__('all source files'), method='all')
>>   File
>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>> line 360, in build
>> updated_docnames = set(self.read())
>>   File
>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>> line 466, in read
>> self._read_parallel(docnames, nproc=self.app.parallel)
>>   File
>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
>> line 521, in _read_parallel
>> tasks.join()
>>   File
>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py",
>> line 114, in join
>> self._join_one()
>>   File
>> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py",
>> line 120, in _join_one
>> exc, logs, result = pipe.recv()
>>   File
>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>> line 250, in recv
>> buf = self._recv_bytes()
>>   File
>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>> line 407, in _recv_bytes
>> buf = self._recv(4)
>>   File
>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>> line 383, in _recv
>> raise EOFError
>> EOFError
>>
>> Exception occurred:
>>   File
>> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
>> line 383, in _recv
>> raise EOFError
>> EOFError
>> The full traceback has been saved in
>> /Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/tmp/sphinx-err-mphtfnei.log,
>> if you want to report the issue to the developers.
>> Please also report this if it was a user error, so that a better error
>> message can be provided next time.
>> A bug report can be 

Re: Sphinx Docs Command Error (:sdks:python:test-suites:tox:pycommon:docs)

[Udi Meiri]

I don't have those issues (running on Linux), but a possible workaround
could be to remove the "-j 8" flags (2 locations) in generate_pydoc.sh.


On Mon, Feb 10, 2020 at 11:06 AM Shoaib Zafar 
wrote:

> Hello Beamers.
>
> Just curious does anyone having trouble running
> ':sdks:python:test-suites:tox:pycommon:docs' command locally?
>
> After rebasing with master recently, I am facing sphinx thread fork error
> with on my macos mojave, using python 3.7.0.
> I Tried to add system variable "export
> OBJC_DISABLE_INITIALIZE_FORK_SAFETY=YES" (which I found on google) but no
> luck!
>
> Any suggestions/help?
>
> Thanks!
>
> Console Log:
> --
> 
> Creating file target/docs/source/apache_beam.utils.proto_utils.rst.
> Creating file target/docs/source/apache_beam.utils.retry.rst.
> Creating file target/docs/source/apache_beam.utils.subprocess_server.rst.
> Creating file
> target/docs/source/apache_beam.utils.thread_pool_executor.rst.
> Creating file target/docs/source/apache_beam.utils.timestamp.rst.
> Creating file target/docs/source/apache_beam.utils.urns.rst.
> Creating file target/docs/source/apache_beam.utils.rst.
> objc[8384]: +[__NSCFConstantString initialize] may have been in progress
> in another thread when fork() was called.
> objc[8384]: +[__NSCFConstantString initialize] may have been in progress
> in another thread when fork() was called. We cannot safely call it or
> ignore it in the fork() child process. Crashing instead. Set a breakpoint
> on objc_initializeAfterForkError to debug.
>
> Traceback (most recent call last):
>   File
> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/cmd/build.py",
> line 304, in build_main
> app.build(args.force_all, filenames)
>   File
> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/application.py",
> line 335, in build
> self.builder.build_all()
>   File
> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
> line 305, in build_all
> self.build(None, summary=__('all source files'), method='all')
>   File
> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
> line 360, in build
> updated_docnames = set(self.read())
>   File
> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
> line 466, in read
> self._read_parallel(docnames, nproc=self.app.parallel)
>   File
> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/builders/__init__.py",
> line 521, in _read_parallel
> tasks.join()
>   File
> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py",
> line 114, in join
> self._join_one()
>   File
> "/Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/lib/python3.7/site-packages/sphinx/util/parallel.py",
> line 120, in _join_one
> exc, logs, result = pipe.recv()
>   File
> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
> line 250, in recv
> buf = self._recv_bytes()
>   File
> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
> line 407, in _recv_bytes
> buf = self._recv(4)
>   File
> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
> line 383, in _recv
> raise EOFError
> EOFError
>
> Exception occurred:
>   File
> "/Users/shoaib/.pyenv/versions/3.7.0/lib/python3.7/multiprocessing/connection.py",
> line 383, in _recv
> raise EOFError
> EOFError
> The full traceback has been saved in
> /Users/shoaib/Projects/beam/newbeam/sdks/python/test-suites/tox/pycommon/build/srcs/sdks/python/target/.tox-py37-docs/py37-docs/tmp/sphinx-err-mphtfnei.log,
> if you want to report the issue to the developers.
> Please also report this if it was a user error, so that a better error
> message can be provided next time.
> A bug report can be filed in the tracker at <
> https://github.com/sphinx-doc/sphinx/issues>. Thanks!
> objc[8385]: +[__NSCFConstantString initialize] may have been in progress
> in another thread when fork() was called.
> objc[8385]: +[__NSCFConstantString initialize] may have been in progress
> in another thread when fork() was called.