Re: Proposal for pyproject.toml Support in Apache Beam Python

The gen_protos.py will be called while building a sdist, wheel or an editable installation. We use pytest through the tox package and during the tox build process, gen_protos.py is called during either wheel or sdist creation. For building sdist, the process now changed from `python setup.py

Re: Proposal for pyproject.toml Support in Apache Beam Python

On Thu, Oct 12, 2023 at 2:04 PM Anand Inguva wrote: > I am in the process of updating the documentation at > https://cwiki.apache.org/confluence/display/BEAM/Python+Tips related to > setup.py/pyproject.toml changes, but yes you can't call setup.py directly > because it might fail due to the lack

Re: Proposal for pyproject.toml Support in Apache Beam Python

I am in the process of updating the documentation at https://cwiki.apache.org/confluence/display/BEAM/Python+Tips related to setup.py/pyproject.toml changes, but yes you can't call setup.py directly because it might fail due to the lack of presence of beam python's build time dependencies. With

Re: Proposal for pyproject.toml Support in Apache Beam Python

Does this change any development practices? E.g. if I clone the repo, I'm assuming I couldn't run "setup.py test" anymore. What about the generated files (like protos, or the yaml definitions copied from other parts of the repo)? On Thu, Oct 12, 2023 at 12:27 PM Anand Inguva via dev wrote: >

Re: Proposal for pyproject.toml Support in Apache Beam Python

The PR https://github.com/apache/beam/pull/28385 is merged today. If there are any observed failures, please comment on the PR and I will follow up with a forward fix. Thanks. On Fri, Sep 1, 2023 at 2:30 PM Anand Inguva wrote: > Since there is positive feedback from the dev community, I am

Re: Proposal for pyproject.toml Support in Apache Beam Python

Since there is positive feedback from the dev community, I am going ahead and implementing this proposal for Python SDK. @aus...@apache.org Initially let's move forward with the setuptools as backend for building package and as part of the future work, we can find a better backend than

Re: Proposal for pyproject.toml Support in Apache Beam Python

I've thought about this a ton, but haven't been in a position to undertake the work. Thanks for bringing this up, @Anand Inguva ! I'd point us to https://python-poetry.org/ ... [ which is where I'd look take us, but I'm also not able to do all the work, so my suggestion/preference doensn't

Re: Proposal for pyproject.toml Support in Apache Beam Python

+1 Hi Anand, I appreciate this effort. Managing python dependencies has been a major pain point for me, and I think this approach would help. Kerry On Mon, Aug 28, 2023 at 10:14 AM Anand Inguva via dev wrote: > Hello Beam Dev Team, > > I've compiled a design document >

Re: Proposal for pyproject.toml Support in Apache Beam Python

Thanks Anand! This generally sounds good to me. I left a few questions before giving a full +1, but it definitely seems like we need some migration effort here and this seems like a good route. Thanks, Danny On Mon, Aug 28, 2023 at 10:14 AM Anand Inguva via dev wrote: > Hello Beam Dev Team, >

Proposal for pyproject.toml Support in Apache Beam Python

Hello Beam Dev Team, I've compiled a design document [1] proposing the integration of pyproject.toml into Apache Beam's Python build process. Your insights and feedback would be