Dave, Could we get help? I'm not sure how to ask the Infrastructure team about the python3-venv package. Is there an alias?
And with regards to the requirements.txt. I agree. It seems the project uses it in some places, but also has these dependencies listed here that are built. They seem to be used in the "tools/rules/pex/BUILD" script. Perhaps there is a reason they need to be referenced like this? Seems like it's maybe something originally setup that can now be improved. https://github.com/apache/incubator-heron/blob/fab089ce2c9cdbdd2d6a7ef7cda6cf22b201469c/WORKSPACE#L178-L240 On Mon, Jan 3, 2022 at 4:57 PM Dave Fisher <[email protected]> wrote: > > > > On Jan 3, 2022, at 1:25 PM, Nicholas Nezis <[email protected]> > wrote: > > > > I don't think this should be a blocker to cutting the release. Removing > PEX > > might have various consequences. > > > > We should still attempt to remove it if we think this is the issue, but I > > suspect we were running into other issues. > > 1. Building on non AMD64 architecture > > a. I'm happy to help build locally > > 2. Building on Jenkins which doesn't have the python3-venv package. > > a. Can we request the package be installed? I'm not sure how we ask > the > > build infrastructure team to add it. > > I’m surprised that venv is missing from the python setup. That’s somewhat > core to Infra’s process. > > - Send an email to [email protected] > - Create an INFRA Jira at issues.apache.org/jira login with your apache > id. > > > b. We can convert our build script to use Docker for the build. This > > might take a bit more effort compared to adding python3-venv > > > > Often we have had issues with Python compatibility. And some of those > > issues were related to the move from Python 2 to Python 3. Another > > complicating factor is that we often have trouble updating some of the > > dependencies because instead of using `pip3 install` to download a > > dependency, we instead download the source and compile the packages. This > > may provide more flexibility to run on different architectures, but also > > can add an extra layer of complexity depending on each package's needs. > > Why wouldn’t we just use a requirements.txt with pip3? > > > > > So while I think the move from Pex to the more native Bazel Python binary > > rules might be better, I don't know if it directly addresses the > perceived > > issues. > > > > On Sun, Jan 2, 2022 at 1:12 AM Ning Wang <[email protected]> wrote: > > > >> it seems like pex has been painful to maintain. It would be great to try > >> the official rule. Thanks! > >> > >> On Sat, Jan 1, 2022 at 11:30 AM Saad Ur Rahman <[email protected] > > > >> wrote: > >> > >>> I will assist as much as I can with the migration. > >>> > >>> On Sat, Jan 1, 2022 at 12:47 PM Josh Fischer <[email protected]> > >> wrote: > >>> > >>>> Hey All, > >>>> > >>>> After doing work on the next RC we've found some issues related to > >> Python > >>>> and Pex. I'm thinking we need to remove the custom Pex rules from > >> repo > >>>> and depend on native bazel rules. I'll create a branch within the > >> Heron > >>>> repo to start work on it. I'll most likely have some questions for > >>>> everyone, and I'd like feedback as we go along. > >>>> > >>>> Thanks > >>>> > >>>> On Fri, Dec 24, 2021 at 6:53 PM Saad Ur Rahman < > >> [email protected]> > >>>> wrote: > >>>> > >>>>> Thank you, Josh, that would be very interesting indeed. > >>>>> > >>>>> On Fri., Dec. 24, 2021, 5:48 p.m. Josh Fischer, <[email protected] > >>> > >>>>> wrote: > >>>>> > >>>>>> Saad, > >>>>>> > >>>>>> I welcome any help. If you are interested I can walk you through > >> the > >>>>>> process. > >>>>>> > >>>>>> On Fri, Dec 24, 2021 at 3:42 PM Saad Ur Rahman < > >>>> [email protected]> > >>>>>> wrote: > >>>>>> > >>>>>>> Thank you, Josh. Please let me know if there is anything I can do > >>> to > >>>>>> help. > >>>>>>> > >>>>>>> On Fri., Dec. 24, 2021, 2:52 p.m. Josh Fischer, < > >>> [email protected] > >>>>> > >>>>>>> wrote: > >>>>>>> > >>>>>>>> Nick, > >>>>>>>> I would appreciate the help if you are willing. > >>>>>>>> > >>>>>>>> Saad, > >>>>>>>> > >>>>>>>> Ok, I’ll start the process Sunday or Monday. > >>>>>>>> > >>>>>>>> > >>>>>>>> > >>>>>>>> On Thu, Dec 23, 2021 at 7:19 PM Saad <[email protected]> > >>>>> wrote: > >>>>>>>> > >>>>>>>>> I feel we are in a good place to cut a release as well. > >>>>>>>>> > >>>>>>>>> On Thu., Dec. 23, 2021, 6:44 p.m. Nicholas Nezis, < > >>>>>>>>> [email protected]> > >>>>>>>>> wrote: > >>>>>>>>> > >>>>>>>>>> Yes I think we should cut a release now. The remaining PRs > >>> can > >>>>> wait > >>>>>>> and > >>>>>>>>> may > >>>>>>>>>> change APIs. > >>>>>>>>>> > >>>>>>>>>> I'd like to help with publishing any release artifacts that > >>> we > >>>>>>> haven't > >>>>>>>>>> published previously. Specifically thinking of HeronPy and > >>> Helm > >>>>>>>>> artifacts. > >>>>>>>>>> > >>>>>>>>>> On Wed, Dec 22, 2021, 9:59 AM Josh Fischer < > >>>> [email protected]> > >>>>>>>> wrote: > >>>>>>>>>> > >>>>>>>>>>> Hi, > >>>>>>>>>>> > >>>>>>>>>>> The kubernetes scheduler PRs are now completed and > >>> merged. I > >>>>>> know > >>>>>>> we > >>>>>>>>>> have > >>>>>>>>>>> a couple of PRs that were recently opened, do we need to > >>> wait > >>>>> for > >>>>>>> any > >>>>>>>>> of > >>>>>>>>>>> those before we start the voting process? > >>>>>>>>>>> > >>>>>>>>>>> - Josh > >>>>>>>>>>> > >>>>>>>>>> > >>>>>>>>> > >>>>>>>> -- > >>>>>>>> Sent from A Mobile Device > >>>>>>>> > >>>>>>> > >>>>>> -- > >>>>>> Sent from A Mobile Device > >>>>>> > >>>>> > >>>> > >>> > >> > >
