Re: [openstack-dev] [pbr] support v_version
Excerpts from Gaetan's message of 2018-01-15 19:29:01 +0100: > > > > > > > I guess it somehow didn't used my build of the pbr package when > > > running in gitlab pipeline. > > > > It sounds like the CE pipeline is not building packages in the same way? > > Or is using an old version of pbr? > > > > I guess it was the pbr version from pypi.python.org, not my customized > build > published on our internal pypi > https://books.sonatype.com/nexus-book/3.0/reference/pypi.html > > > > Do you plan on releasing PBR soon on pypi? > > > I have to build myself and push it on our internal nexus pypi, but I > > think > > > the > > > safest way is to wait for an official pbr release on pypi.python.org :) > > > > Unfortunately we're approaching a significant set of feature freeze > > deadlines for this OpenStack development cycle. Because it is very > > difficult to "pin" a library like pbr that appears in the setup_requires > > list for projects in our CI pipelines, and this next release of pbr > > would have quite a few changes, we have decided in our meeting today > > to be cautious and delay the next release for a few weeks. > > > Have you ever considered using pipenv/pipfile ? I started using it a few > months ago it helps a lot with dependency freezing We have a system in place to constrain most of the libraries we use (using pip's -c option) but pip does not pay attention to that list in all code paths when installing things listed in setup_requires. > > I have scheduled a reminder to tag the pbr release during the first > > week of March, after the OpenStack release is safely completed. > > > > I'm sorry if this causes inconvenience. We're going to work on > > ensuring we have more frequent releases so we reduce our chance of > > ending up in a similar situation again in the future. > > > > That is not a problem by itself, I still have this self-hosted Pypi > repository > in the meantime. OK, good. > To react on the IRL log, indeed, the proposal I make in this thread is > purely optional, > for my need, gitlab handle correctly the protection against unwanted > publication. > Just hope it will be useful for other project, probably not for OpenStack > itself. > > Thanks a lot for supporting external projects :) > > Gaetan Thanks for working with us to improve pbr! :-) Doug __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [pbr] support v_version
> > > > I guess it somehow didn't used my build of the pbr package when > > running in gitlab pipeline. > > It sounds like the CE pipeline is not building packages in the same way? > Or is using an old version of pbr? > > I guess it was the pbr version from pypi.python.org, not my customized build published on our internal pypi https://books.sonatype.com/nexus-book/3.0/reference/pypi.html > > Do you plan on releasing PBR soon on pypi? > > I have to build myself and push it on our internal nexus pypi, but I > think > > the > > safest way is to wait for an official pbr release on pypi.python.org :) > > Unfortunately we're approaching a significant set of feature freeze > deadlines for this OpenStack development cycle. Because it is very > difficult to "pin" a library like pbr that appears in the setup_requires > list for projects in our CI pipelines, and this next release of pbr > would have quite a few changes, we have decided in our meeting today > to be cautious and delay the next release for a few weeks. > > Have you ever considered using pipenv/pipfile ? I started using it a few months ago it helps a lot with dependency freezing I have scheduled a reminder to tag the pbr release during the first > week of March, after the OpenStack release is safely completed. > > I'm sorry if this causes inconvenience. We're going to work on > ensuring we have more frequent releases so we reduce our chance of > ending up in a similar situation again in the future. > That is not a problem by itself, I still have this self-hosted Pypi repository in the meantime. To react on the IRL log, indeed, the proposal I make in this thread is purely optional, for my need, gitlab handle correctly the protection against unwanted publication. Just hope it will be useful for other project, probably not for OpenStack itself. Thanks a lot for supporting external projects :) Gaetan __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [pbr] support v_version
Excerpts from Gaetan's message of 2018-01-15 17:29:01 +0100: > First, thanks a lot for your support and your kindness ! Really appreciate > that :) > > > > Do you know where I need to hack PBR to fix it? > > > > So 'pbr' correctly parses the prefixed tags, but it's just the output > > packages (sdists, wheels) that always unversioned? If so, this sounds > > correct. Python packaging, as defined in PEP-440 [1], doesn't use the > > 'v' prefixes, so it doesn't really make sense to stick them in here. > > Out of curiosity, what's your rationale for modifying the package name? > > > > The package name is not changed actually. With the patch in PBR that has > been merged, one could add a tag named "v1.0.0" on mypackage package, > and the sdist will generate a distribution package > > mypackage-0.0.4.tar.gz > > > So I think (hope?) this is still PEP440 compliant. > > I tried this feature on another software that also uses pbr and there is no > problem: v version works great with sdist/bdist/wheel packages. > > I use it inside a Gitlab CE pipeline on a tag pipeline (CI is triggered when > a git tag that follows the "v*" regular expression), and instead of > creating > a package > > mypackage-0.0.4-py2.py3-none-any.whl > > it created > > mypackage-0.0.3.dev3-py2.py3-none-any.whl. > > > When I retried manually on my development environment, pbr works > perfectly again on the same code. > I guess it somehow didn't used my build of the pbr package when > running in gitlab pipeline. It sounds like the CE pipeline is not building packages in the same way? Or is using an old version of pbr? > > Do you plan on releasing PBR soon on pypi? > I have to build myself and push it on our internal nexus pypi, but I think > the > safest way is to wait for an official pbr release on pypi.python.org :) Unfortunately we're approaching a significant set of feature freeze deadlines for this OpenStack development cycle. Because it is very difficult to "pin" a library like pbr that appears in the setup_requires list for projects in our CI pipelines, and this next release of pbr would have quite a few changes, we have decided in our meeting today to be cautious and delay the next release for a few weeks. I have scheduled a reminder to tag the pbr release during the first week of March, after the OpenStack release is safely completed. I'm sorry if this causes inconvenience. We're going to work on ensuring we have more frequent releases so we reduce our chance of ending up in a similar situation again in the future. Doug [1] http://eavesdrop.openstack.org/irclogs/%23openstack-oslo/%23openstack-oslo.2018-01-15.log.html __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [pbr] support v_version
First, thanks a lot for your support and your kindness ! Really appreciate that :) > > Do you know where I need to hack PBR to fix it? > > So 'pbr' correctly parses the prefixed tags, but it's just the output > packages (sdists, wheels) that always unversioned? If so, this sounds > correct. Python packaging, as defined in PEP-440 [1], doesn't use the > 'v' prefixes, so it doesn't really make sense to stick them in here. > Out of curiosity, what's your rationale for modifying the package name? > The package name is not changed actually. With the patch in PBR that has been merged, one could add a tag named "v1.0.0" on mypackage package, and the sdist will generate a distribution package mypackage-0.0.4.tar.gz So I think (hope?) this is still PEP440 compliant. I tried this feature on another software that also uses pbr and there is no problem: v version works great with sdist/bdist/wheel packages. I use it inside a Gitlab CE pipeline on a tag pipeline (CI is triggered when a git tag that follows the "v*" regular expression), and instead of creating a package mypackage-0.0.4-py2.py3-none-any.whl it created mypackage-0.0.3.dev3-py2.py3-none-any.whl. When I retried manually on my development environment, pbr works perfectly again on the same code. I guess it somehow didn't used my build of the pbr package when running in gitlab pipeline. Do you plan on releasing PBR soon on pypi? I have to build myself and push it on our internal nexus pypi, but I think the safest way is to wait for an official pbr release on pypi.python.org :) > Second point, to go to the end of the logic of my change, I would > > like to propose an optional way (in setup.cfg?) to **prevent** any > > tag without the 'v' prefix, ie, where a bare version tag like `1.0.0` > > is not to be considered as a valid version. > > That way, on system such as gitlab or github: > > - repository owners "protect" tags with pattern "v*", ie, all tags > > for release such as "v1.0.0", ... cannot be pushed by anyone but the > > owners/masters > > - other developer can still push other tags for other purpose > > So this could be used to prevent pbr reading the tags, but it won't > stop anyone from creating them in the first place (i.e. "protect" > tags). Yes, I agree this is not really mandatory. Gitlab tag protection should be enough. I am using a "protected environment variables" on gitlab, and indeed, the credentials to push on Pypi are only sent when the pipeline is triggered on such a "protected" branch or "protected tag". So we "protect" only tags starting with a "v*" and only this triggered pipeline can publish to pypi (we use Nexus). This allows other developers to add any tags not started with v (only repository owners can create tags starting with a "v*"). Note this "v*" regular expression is configurable and seem to default/good practice on GitLab CE/EE. > We can do this but it would be a separate feature and, to be > honest, I'd suggest using Git hooks or some form of access control as a > better way to do this (Note: it seems GitLab already supports something > similar [2]). > Yes this is what I actually use :) Thanks In short: pbr v_version seems to work great, just hoping for the official PBR release on pypi.python.org :) Thanks Gaetan Semet __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [pbr] support v_version
On Tue, 2018-01-09 at 10:25 +0100, Gaetan wrote: > Hello > > I have submitted this patch ([1]) that add support for v_version in > PBR. Basically I can tag v1.0.0 instead of 1.0.0 to release 1.0.0. > > However, after rework it appears PBR does not behaves well, even if > the unit tests pass: > On tag for instance v1.0.0, the result packages in named > `-1.0.0.dev1`. > > Do you know where I need to hack PBR to fix it? So 'pbr' correctly parses the prefixed tags, but it's just the output packages (sdists, wheels) that always unversioned? If so, this sounds correct. Python packaging, as defined in PEP-440 [1], doesn't use the 'v' prefixes, so it doesn't really make sense to stick them in here. Out of curiosity, what's your rationale for modifying the package name? > Second point, to go to the end of the logic of my change, I would > like to propose an optional way (in setup.cfg?) to **prevent** any > tag without the 'v' prefix, ie, where a bare version tag like `1.0.0` > is not to be considered as a valid version. > That way, on system such as gitlab or github: > - repository owners "protect" tags with pattern "v*", ie, all tags > for release such as "v1.0.0", ... cannot be pushed by anyone but the > owners/masters > - other developer can still push other tags for other purpose So this could be used to prevent pbr reading the tags, but it won't stop anyone from creating them in the first place (i.e. "protect" tags). We can do this but it would be a separate feature and, to be honest, I'd suggest using Git hooks or some form of access control as a better way to do this (Note: it seems GitLab already supports something similar [2]). Stephen [1] https://www.python.org/dev/peps/pep-0440/ [2] https://gitlab.com/gitlab-org/gitlab-ce/issues/18471 __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [pbr] support v_version
Excerpts from Jeremy Stanley's message of 2018-01-15 14:40:14 +: > On 2018-01-09 10:25:56 +0100 (+0100), Gaetan wrote: > > I have submitted this patch ([1]) that add support for v_version > > in PBR. Basically I can tag v1.0.0 instead of 1.0.0 to release > > 1.0.0. > [...] > > Looks like the patch you linked has since merged. Any issues with it > so far? > > > Second point, to go to the end of the logic of my change, I would > > like to propose an optional way (in setup.cfg?) to **prevent** any > > tag without the 'v' prefix, ie, where a bare version tag like > > `1.0.0` is not to be considered as a valid version. > [...] > > I'm not heavily involved in PBR maintenance so my ideas may be > terrible, but have you considered just making it possible to set a > version pattern option in setup.cfg so that this sort of filtering > is more easily generalized? That seems reasonable. It may even help solve the package filename problem. Doug __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
Re: [openstack-dev] [pbr] support v_version
On 2018-01-09 10:25:56 +0100 (+0100), Gaetan wrote: > I have submitted this patch ([1]) that add support for v_version > in PBR. Basically I can tag v1.0.0 instead of 1.0.0 to release > 1.0.0. [...] Looks like the patch you linked has since merged. Any issues with it so far? > Second point, to go to the end of the logic of my change, I would > like to propose an optional way (in setup.cfg?) to **prevent** any > tag without the 'v' prefix, ie, where a bare version tag like > `1.0.0` is not to be considered as a valid version. [...] I'm not heavily involved in PBR maintenance so my ideas may be terrible, but have you considered just making it possible to set a version pattern option in setup.cfg so that this sort of filtering is more easily generalized? -- Jeremy Stanley signature.asc Description: PGP signature __ OpenStack Development Mailing List (not for usage questions) Unsubscribe: openstack-dev-requ...@lists.openstack.org?subject:unsubscribe http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-dev
