Re: [Numpy-discussion] CI Testing
On Wed, Sep 12, 2018 at 7:30 PM Stanley Seibert wrote: > > > On Wed, Sep 12, 2018 at 7:32 PM, Charles R Harris < > charlesr.har...@gmail.com> wrote: > >> >> >> On Wed, Sep 12, 2018 at 6:26 PM Stanley Seibert >> wrote: >> >>> If you go beyond the free tier, you can connect self-managed build >>> workers to the same system, but the build agent is written in C#, so I'm >>> not sure how portable it is to ARM or PPC yet. >>> >> >> Did you sign up as open source? And if so, were you able to do so as an >> organization? >> > > Yes, it was pretty straightforward. They seem to be using the simplistic > definition of "public repository == open source", so there was no > verification step. I was able to create a project associated with the > numba organization on Github. It seems to still require that you have > Microsoft accounts, and manage permissions on those, rather than inheriting > the settings from your Github repositories. > So is the account set up in your name? What do you mean "manage permissions", did you need to add names to the account via Microsoft and does everyone accessing the account need to have a Microsoft account? It would be helpful if you could go through the procedure step-by-step. Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] CI Testing
I'm still trying to wrap my head around the security model here. The onboarding wizard makes it pretty easy to get started, but the UI afterwards has a lot of complexity for managing fine grained permissions. As I understand it, I made a "numba" project with my Microsoft account, but I can add other Microsoft accounts to the project and give them varying access to administer the project. None of these user accounts or permissions connect directly to Github accounts, so that is annoying if you have a large core dev team you want to give permission to manage builds. They will all need Microsoft accounts, and you will have to grant them admin access to the project. (Still trying to figure out how to do that for Numba...) Azure Pipeline's connection to Github itself (to post CI status under PRs, etc) can be done either by granting permission via a Github user's account, or by installing it as an "app" in the Github organization, which is the route I opted for. On Thu, Sep 13, 2018 at 8:13 AM, Charles R Harris wrote: > > > On Wed, Sep 12, 2018 at 7:30 PM Stanley Seibert > wrote: > >> >> >> On Wed, Sep 12, 2018 at 7:32 PM, Charles R Harris < >> charlesr.har...@gmail.com> wrote: >> >>> >>> >>> On Wed, Sep 12, 2018 at 6:26 PM Stanley Seibert >>> wrote: >>> If you go beyond the free tier, you can connect self-managed build workers to the same system, but the build agent is written in C#, so I'm not sure how portable it is to ARM or PPC yet. >>> >>> Did you sign up as open source? And if so, were you able to do so as an >>> organization? >>> >> >> Yes, it was pretty straightforward. They seem to be using the simplistic >> definition of "public repository == open source", so there was no >> verification step. I was able to create a project associated with the >> numba organization on Github. It seems to still require that you have >> Microsoft accounts, and manage permissions on those, rather than inheriting >> the settings from your Github repositories. >> > > So is the account set up in your name? What do you mean "manage > permissions", did you need to add names to the account via Microsoft and > does everyone accessing the account need to have a Microsoft account? It > would be helpful if you could go through the procedure step-by-step. > > Chuck > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion > > ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] CI Testing
Another minor annoyance is that the link on the Github "Checks" page that says "View More Details on Azure Pipelines" takes you to a login page, which then forwards you to the public (no login required) pipeline build results page. As a result, people might not realize you can view the build results for public projects without a Microsoft account, so you'll probably want to put a build status badge with a direct link somewhere prominent in the README. On Thu, Sep 13, 2018 at 9:25 AM, Stanley Seibert wrote: > I'm still trying to wrap my head around the security model here. The > onboarding wizard makes it pretty easy to get started, but the UI > afterwards has a lot of complexity for managing fine grained permissions. > As I understand it, I made a "numba" project with my Microsoft account, but > I can add other Microsoft accounts to the project and give them varying > access to administer the project. None of these user accounts or > permissions connect directly to Github accounts, so that is annoying if you > have a large core dev team you want to give permission to manage builds. > They will all need Microsoft accounts, and you will have to grant them > admin access to the project. (Still trying to figure out how to do that > for Numba...) > > Azure Pipeline's connection to Github itself (to post CI status under PRs, > etc) can be done either by granting permission via a Github user's account, > or by installing it as an "app" in the Github organization, which is the > route I opted for. > > On Thu, Sep 13, 2018 at 8:13 AM, Charles R Harris < > charlesr.har...@gmail.com> wrote: > >> >> >> On Wed, Sep 12, 2018 at 7:30 PM Stanley Seibert >> wrote: >> >>> >>> >>> On Wed, Sep 12, 2018 at 7:32 PM, Charles R Harris < >>> charlesr.har...@gmail.com> wrote: >>> On Wed, Sep 12, 2018 at 6:26 PM Stanley Seibert wrote: > If you go beyond the free tier, you can connect self-managed build > workers to the same system, but the build agent is written in C#, so I'm > not sure how portable it is to ARM or PPC yet. > Did you sign up as open source? And if so, were you able to do so as an organization? >>> >>> Yes, it was pretty straightforward. They seem to be using the >>> simplistic definition of "public repository == open source", so there was >>> no verification step. I was able to create a project associated with the >>> numba organization on Github. It seems to still require that you have >>> Microsoft accounts, and manage permissions on those, rather than inheriting >>> the settings from your Github repositories. >>> >> >> So is the account set up in your name? What do you mean "manage >> permissions", did you need to add names to the account via Microsoft and >> does everyone accessing the account need to have a Microsoft account? It >> would be helpful if you could go through the procedure step-by-step. >> >> Chuck >> >> ___ >> NumPy-Discussion mailing list >> NumPy-Discussion@python.org >> https://mail.python.org/mailman/listinfo/numpy-discussion >> >> > ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] CI Testing
On Thu, Sep 13, 2018 at 8:30 AM Stanley Seibert wrote: > Another minor annoyance is that the link on the Github "Checks" page that > says "View More Details on Azure Pipelines" takes you to a login page, > which then forwards you to the public (no login required) pipeline build > results page. As a result, people might not realize you can view the build > results for public projects without a Microsoft account, so you'll probably > want to put a build status badge with a direct link somewhere prominent in > the README. > > On Thu, Sep 13, 2018 at 9:25 AM, Stanley Seibert > wrote: > >> I'm still trying to wrap my head around the security model here. The >> onboarding wizard makes it pretty easy to get started, but the UI >> afterwards has a lot of complexity for managing fine grained permissions. >> As I understand it, I made a "numba" project with my Microsoft account, but >> I can add other Microsoft accounts to the project and give them varying >> access to administer the project. None of these user accounts or >> permissions connect directly to Github accounts, so that is annoying if you >> have a large core dev team you want to give permission to manage builds. >> They will all need Microsoft accounts, and you will have to grant them >> admin access to the project. (Still trying to figure out how to do that >> for Numba...) >> >> Azure Pipeline's connection to Github itself (to post CI status under >> PRs, etc) can be done either by granting permission via a Github user's >> account, or by installing it as an "app" in the Github organization, which >> is the route I opted for. >> > Does Azure Pipeline respond to the Github hooks? We have hooks for all the other testing sites. Thanks for the continuing updates, you are one of the first explorers and anything you can report back is of great help to the rest of us. >> On Thu, Sep 13, 2018 at 8:13 AM, Charles R Harris < >> charlesr.har...@gmail.com> wrote: >> >>> >>> >>> On Wed, Sep 12, 2018 at 7:30 PM Stanley Seibert >>> wrote: >>> On Wed, Sep 12, 2018 at 7:32 PM, Charles R Harris < charlesr.har...@gmail.com> wrote: > > > On Wed, Sep 12, 2018 at 6:26 PM Stanley Seibert > wrote: > >> If you go beyond the free tier, you can connect self-managed build >> workers to the same system, but the build agent is written in C#, so I'm >> not sure how portable it is to ARM or PPC yet. >> > > Did you sign up as open source? And if so, were you able to do so as > an organization? > Yes, it was pretty straightforward. They seem to be using the simplistic definition of "public repository == open source", so there was no verification step. I was able to create a project associated with the numba organization on Github. It seems to still require that you have Microsoft accounts, and manage permissions on those, rather than inheriting the settings from your Github repositories. >>> >>> So is the account set up in your name? What do you mean "manage >>> permissions", did you need to add names to the account via Microsoft and >>> does everyone accessing the account need to have a Microsoft account? It >>> would be helpful if you could go through the procedure step-by-step. >>> >>> Chuck ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal to accept NEP-18, __array_function__ protocol
On Sat, Sep 8, 2018 at 7:12 PM Charles R Harris wrote: > On Wed, Aug 1, 2018 at 6:27 PM Stephan Hoyer wrote: > >> I propose to accept NEP-18, "A dispatch mechanism for NumPy’s high level >> array functions": >> http://www.numpy.org/neps/nep-0018-array-function-protocol.html >> >> Since the last round of discussion, we added a new section on "Callable >> objects generated at runtime" clarifying that to handle such objects is out >> of scope for the initial proposal in the NEP. >> >> If there are no substantive objections within 7 days from this email, >> then the NEP will be accepted; see NEP 0 for more details. >> >> > I've merged the PR. What next? > > Chuck > I rolled back Chuck's merge of the "Acceptance" PR for this NEP because (1) it was not clear that if had reached consensus and (2) I still wanted to make a few changes based on this discussion. I have now drafted these revisions to the NEP to clarify its stance around backwards compatibility, and the type of the "types" argument: https://github.com/numpy/numpy/pull/11943 I have *not* included any form of the explicit "end-user opt-in" requested by Nathaniel. I don't think it is warranted based on the scope of anticipated future changes; see my earlier post [1] for details. Nobody has seriously suggested that we would release this functionality and later eliminate the ability to override NumPy functions entirely in the future. Of course, requiring an onerous explicit opt-in would be fine while this feature is initially under development, and until we are sure that checks for overriding arbitrary NumPy functions are fast enough to be generally viable. In particular, we should verify that __array_function__ does not meaningfully impact performance for major downstream components of the SciPy stack. But I think running standard benchmark suites (e.g., with ASV) and user testing of release candidates would be enough. If you have still have major objections to this version of proposal, please raise them unambiguously -- preferably with an formal veto [2]. Best, Stephan [1] https://mail.python.org/pipermail/numpy-discussion/2018-August/078669.html [2] https://docs.scipy.org/doc/numpy/dev/governance/governance.html ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal to accept NEP-18, __array_function__ protocol
Thanks for adding the clarifications. They read well to me, and I think make clear to a project like astropy what to expect (and I expect we'll be use it!). -- Marten ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
Re: [Numpy-discussion] Proposal to accept NEP-18, __array_function__ protocol
> On Thursday, Sep 13, 2018 at 7:30 PM, Stephan Hoyer (mailto:sho...@gmail.com)> wrote: > > > On Sat, Sep 8, 2018 at 7:12 PM Charles R Harris (mailto:charlesr.har...@gmail.com)> wrote: > > > > > > On Wed, Aug 1, 2018 at 6:27 PM Stephan Hoyer > (mailto:sho...@gmail.com)> wrote: > > > > > > > > > > > > > I propose to accept NEP-18, "A dispatch mechanism for NumPy’s high level > > > array functions": > > > > > > > > > > > > > > > http://www.numpy.org/neps/nep-0018-array-function-protocol.html > > > > > > > > > > > > > > > > > > > Since the last round of discussion, we added a new section on "Callable > > > objects generated at runtime" clarifying that to handle such objects is > > > out of scope for the initial proposal in the NEP. > > > > > > If there are no substantive objections within 7 days from this email, > > > then the NEP will be accepted; see NEP 0 for more details. > > > > > > > I've merged the PR. What next? > > > > Chuck > > I rolled back Chuck's merge of the "Acceptance" PR for this NEP because (1) > it was not clear that if had reached consensus and (2) I still wanted to make > a few changes based on this discussion. > > I have now drafted these revisions to the NEP to clarify its stance around > backwards compatibility, and the type of the "types" argument: > https://github.com/numpy/numpy/pull/11943 > > > > > I have *not* included any form of the explicit "end-user opt-in" requested by > Nathaniel. I don't think it is warranted based on the scope of anticipated > future changes; see my earlier post [1] for details. Nobody has seriously > suggested that we would release this functionality and later eliminate the > ability to override NumPy functions entirely in the future. > > > > > Of course, requiring an onerous explicit opt-in would be fine while this > feature is initially under development, and until we are sure that checks for > overriding arbitrary NumPy functions are fast enough to be generally viable. > In particular, we should verify that __array_function__ does not meaningfully > impact performance for major downstream components of the SciPy stack. But I > think running standard benchmark suites (e.g., with ASV) and user testing of > release candidates would be enough. > > If you have still have major objections to this version of proposal, please > raise them unambiguously -- preferably with an formal veto [2]. > > Best, > Stephan > > > > > [1] https://mail.python.org/pipermail/numpy-discussion/2018-August/078669.html > [2] https://docs.scipy.org/doc/numpy/dev/governance/governance.html > > > > ___ > NumPy-Discussion mailing list > NumPy-Discussion@python.org > https://mail.python.org/mailman/listinfo/numpy-discussion +1 from me. The edits are readable and clean, and a good compromise. Best Regards, Hameer Abbasi ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
[Numpy-discussion] PR to add "weights" option to np.quantile
Hi all, I have a long-standing PR to add a "weights" option to np.quantile/percentile: https://github.com/numpy/numpy/pull/9211 For a little background, there are quite a few ways to define "quantile" to begin with. Numpy defines it the same way as R's default (Type 7): https://www.rdocumentation.org/packages/stats/versions/3.5.0/topics/quantile What PR 9211 does is introducing a "weights" option while staying consistent with the Type 7 definition of quantile. There is certainly support on adding this feature, but as also can be perused from the thread, there are doubts on 1.) what is the right way to define "weighted" quantile? 2.) whether Numpy should support other types of quantiles in the first place, etc. I've stated my answers to those questions in the thread. This PR has fallen off the radar and needs a little popular jolt to get some movement. Please chime in to keep it alive. Best, Chun ___ NumPy-Discussion mailing list NumPy-Discussion@python.org https://mail.python.org/mailman/listinfo/numpy-discussion
