I agree with the points made. It is probably best to have all of the build scripts running in Docker containers so that there is better control over the build environment. I believe ASF infrastructure also wants builds to run in containers going forward.
The removal of PEX based build rules is important, and we should get started on it. I suspect the process will be lengthy with new issues coming to light as we proceed. Another issue to consider is the time since the last release. On Mon., Jan. 3, 2022, 4:25 p.m. Nicholas Nezis, <[email protected]> wrote: > I don't think this should be a blocker to cutting the release. Removing PEX > might have various consequences. > > We should still attempt to remove it if we think this is the issue, but I > suspect we were running into other issues. > 1. Building on non AMD64 architecture > a. I'm happy to help build locally > 2. Building on Jenkins which doesn't have the python3-venv package. > a. Can we request the package be installed? I'm not sure how we ask the > build infrastructure team to add it. > b. We can convert our build script to use Docker for the build. This > might take a bit more effort compared to adding python3-venv > > Often we have had issues with Python compatibility. And some of those > issues were related to the move from Python 2 to Python 3. Another > complicating factor is that we often have trouble updating some of the > dependencies because instead of using `pip3 install` to download a > dependency, we instead download the source and compile the packages. This > may provide more flexibility to run on different architectures, but also > can add an extra layer of complexity depending on each package's needs. > > So while I think the move from Pex to the more native Bazel Python binary > rules might be better, I don't know if it directly addresses the perceived > issues. > > On Sun, Jan 2, 2022 at 1:12 AM Ning Wang <[email protected]> wrote: > > > it seems like pex has been painful to maintain. It would be great to try > > the official rule. Thanks! > > > > On Sat, Jan 1, 2022 at 11:30 AM Saad Ur Rahman <[email protected]> > > wrote: > > > > > I will assist as much as I can with the migration. > > > > > > On Sat, Jan 1, 2022 at 12:47 PM Josh Fischer <[email protected]> > > wrote: > > > > > > > Hey All, > > > > > > > > After doing work on the next RC we've found some issues related to > > Python > > > > and Pex. I'm thinking we need to remove the custom Pex rules from > > repo > > > > and depend on native bazel rules. I'll create a branch within the > > Heron > > > > repo to start work on it. I'll most likely have some questions for > > > > everyone, and I'd like feedback as we go along. > > > > > > > > Thanks > > > > > > > > On Fri, Dec 24, 2021 at 6:53 PM Saad Ur Rahman < > > [email protected]> > > > > wrote: > > > > > > > > > Thank you, Josh, that would be very interesting indeed. > > > > > > > > > > On Fri., Dec. 24, 2021, 5:48 p.m. Josh Fischer, < > [email protected] > > > > > > > > wrote: > > > > > > > > > > > Saad, > > > > > > > > > > > > I welcome any help. If you are interested I can walk you through > > the > > > > > > process. > > > > > > > > > > > > On Fri, Dec 24, 2021 at 3:42 PM Saad Ur Rahman < > > > > [email protected]> > > > > > > wrote: > > > > > > > > > > > > > Thank you, Josh. Please let me know if there is anything I can > do > > > to > > > > > > help. > > > > > > > > > > > > > > On Fri., Dec. 24, 2021, 2:52 p.m. Josh Fischer, < > > > [email protected] > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > Nick, > > > > > > > > I would appreciate the help if you are willing. > > > > > > > > > > > > > > > > Saad, > > > > > > > > > > > > > > > > Ok, I’ll start the process Sunday or Monday. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > On Thu, Dec 23, 2021 at 7:19 PM Saad < > [email protected]> > > > > > wrote: > > > > > > > > > > > > > > > > > I feel we are in a good place to cut a release as well. > > > > > > > > > > > > > > > > > > On Thu., Dec. 23, 2021, 6:44 p.m. Nicholas Nezis, < > > > > > > > > > [email protected]> > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > Yes I think we should cut a release now. The remaining > PRs > > > can > > > > > wait > > > > > > > and > > > > > > > > > may > > > > > > > > > > change APIs. > > > > > > > > > > > > > > > > > > > > I'd like to help with publishing any release artifacts > that > > > we > > > > > > > haven't > > > > > > > > > > published previously. Specifically thinking of HeronPy > and > > > Helm > > > > > > > > > artifacts. > > > > > > > > > > > > > > > > > > > > On Wed, Dec 22, 2021, 9:59 AM Josh Fischer < > > > > [email protected]> > > > > > > > > wrote: > > > > > > > > > > > > > > > > > > > > > Hi, > > > > > > > > > > > > > > > > > > > > > > The kubernetes scheduler PRs are now completed and > > > merged. I > > > > > > know > > > > > > > we > > > > > > > > > > have > > > > > > > > > > > a couple of PRs that were recently opened, do we need > to > > > wait > > > > > for > > > > > > > any > > > > > > > > > of > > > > > > > > > > > those before we start the voting process? > > > > > > > > > > > > > > > > > > > > > > - Josh > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > -- > > > > > > > > Sent from A Mobile Device > > > > > > > > > > > > > > > > > > > > > -- > > > > > > Sent from A Mobile Device > > > > > > > > > > > > > > > > > > > > >
