Re: Tying Dockerhub into development and release management

[Matteo Merli]

On Wed, Feb 6, 2019 at 3:41 PM Justin Mclean  wrote:
>
> Hi,
>
> > If projects want to make convenience binaries available for installation
> > via Docker and DockerHub, then it seems like we need an official Apache
> > DockerHub repository. Do we have one of those, or are folks just publishing
> > to personal repos?
>
> A quick look shows HTTP, Maven, Tomcat, Casandra, Solr, Groovy and a number 
> of other Apache TLP using docker hub. Well they may be it’s hard to know who 
> is publishing them.
>
> All are using Apache branding and most are marked as "Docker Official Images” 
> [1]
>
> None seem to be publishing nightly there but there are binary versions of 
> released source code in all these projects.
>
> I can also see ignite [2] published under an apache ignite account and pulsar 
> [2] under an apache pulsar account and nutch [3] published under an apache 
> account.
>
> Pulsar is publishing RCs and looks like it was doing so during incubation :-(
> 4. https://hub.docker.com/r/apachepulsar/pulsar

Just a quick clarification. We have the push to DockerHub as part of the steps
performed by the release manager *after* the release is approved (in
the incubator
phase, that was after IIPMC approval). The images are simply a
repackaged version
of the official release binary tgz.

Regarding RCs tag, the only one published there was
"2.0.0-rc1-incubating" and the
reason was that this was the name of the official release (Because of
the jump from
1.x to 2.x, we wanted to communicate the stability to expect from that release).
Ref: 
https://lists.apache.org/thread.html/b40a4791d0810b6c11f4cb9630ca6a4510ac0ed27336796dee54ec1b@%3Cgeneral.incubator.apache.org%3E



>
> 1. https://hub.docker.com/search?q=Apache=image
> 2. https://hub.docker.com/r/apacheignite/ignite
> 3. https://hub.docker.com/r/apacheignite/ignite
> 4. https://hub.docker.com/r/apachepulsar/pulsar
> 5. https://hub.docker.com/r/apache/nutch
> 6. https://hub.docker.com/u/apache
> 7. https://hub.docker.com/r/apache/yetus/tags
> 8. https://hub.docker.com/r/apache/syncope/tags
> 9. https://hub.docker.com/r/apache/airflow/tags
> -
> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> For additional commands, e-mail: general-h...@incubator.apache.org
>

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Aw: Re: Tying Dockerhub into development and release management

[Jochen Theodorou]

> Gesendet: Freitag, 08. Februar 2019 um 04:58 Uhr
> Von: "Dave Fisher" 
> An: general@incubator.apache.org
> Betreff: Re: Tying Dockerhub into development and release management
[...]
> > On Feb 7, 2019, at 7:51 PM, Chris Lambertus  wrote:
[...]
> >> On Feb 7, 2019, at 6:47 PM, Justin Mclean  wrote:
> >> 
> >> Hi,
> >> 
> >>> Infra does not police what projects deploy on their dockerhub repos. Do 
> >>> we need to?
> >> 
> >> Well from a casual glance I can see several projects that seem to be 
> >> putting releases constructed from unapproved source code up there. I’ve 
> >> not looked in detail so may be mistaken. I guess sit depends if that 
> >> concerns you or not.
> > 
> > I hear you loud and clear. It’s not a question of if it concerns “me” i.e. 
> > Infra, but more if it concerns Legal. Based on 
> > www.apache.org/legal/release-policy.html it seems like Infra may need to 
> > clamp down on what’s going on with the dockerhub repos and builds. As I 
> > alluded to before, we’ve generally left this to the good will of the 
> > project. If it’s being abused and the project is “releasing” artifacts via 
> > dockerhub that have not been vetted through the ASF release policy, then we 
> > do need to take action. Thanks for bringing this to our attention. Could 
> > you please send a list of any “offenders” that you’ve found to 
> > private@infra?

Question: "releasing artifacts that have not been vetted through the ASF 
release policy" does also include Docker images based on official releases? (I 
know the thread also talked about nightlies and such, that is why it is not 
100% clear to me here)

bye Jochen

-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Tying Dockerhub into development and release management

[Dave Fisher]

Sent from my iPhone

> On Feb 7, 2019, at 7:51 PM, Chris Lambertus  wrote:
> 
> 
> 
>> On Feb 7, 2019, at 6:47 PM, Justin Mclean  wrote:
>> 
>> Hi,
>> 
>>> Infra does not police what projects deploy on their dockerhub repos. Do we 
>>> need to?
>> 
>> Well from a casual glance I can see several projects that seem to be putting 
>> releases constructed from unapproved source code up there. I’ve not looked 
>> in detail so may be mistaken. I guess sit depends if that concerns you or 
>> not.
> 
> I hear you loud and clear. It’s not a question of if it concerns “me” i.e. 
> Infra, but more if it concerns Legal. Based on 
> www.apache.org/legal/release-policy.html it seems like Infra may need to 
> clamp down on what’s going on with the dockerhub repos and builds. As I 
> alluded to before, we’ve generally left this to the good will of the project. 
> If it’s being abused and the project is “releasing” artifacts via dockerhub 
> that have not been vetted through the ASF release policy, then we do need to 
> take action. Thanks for bringing this to our attention. Could you please send 
> a list of any “offenders” that you’ve found to private@infra?

Does DockerHub provide a way to limit some containers from public view? If so 
then such unapproved artifacts should be hidden first. A general announcement 
could then be made.

Regards,
Dave

> 
> Thanks,
> 
> 
> -Chris
> ASF Infra
> 
> 
> -
> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> For additional commands, e-mail: general-h...@incubator.apache.org
> 


-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Tying Dockerhub into development and release management

[Chris Lambertus]

> On Feb 7, 2019, at 6:47 PM, Justin Mclean  wrote:
> 
> Hi,
> 
>> Infra does not police what projects deploy on their dockerhub repos. Do we 
>> need to?
> 
> Well from a casual glance I can see several projects that seem to be putting 
> releases constructed from unapproved source code up there. I’ve not looked in 
> detail so may be mistaken. I guess sit depends if that concerns you or not.

I hear you loud and clear. It’s not a question of if it concerns “me” i.e. 
Infra, but more if it concerns Legal. Based on 
www.apache.org/legal/release-policy.html it seems like Infra may need to clamp 
down on what’s going on with the dockerhub repos and builds. As I alluded to 
before, we’ve generally left this to the good will of the project. If it’s 
being abused and the project is “releasing” artifacts via dockerhub that have 
not been vetted through the ASF release policy, then we do need to take action. 
Thanks for bringing this to our attention. Could you please send a list of any 
“offenders” that you’ve found to private@infra?

Thanks,


-Chris
ASF Infra


-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Tying Dockerhub into development and release management

[Justin Mclean]

Hi,

> Infra does not police what projects deploy on their dockerhub repos. Do we 
> need to?

Well from a casual glance I can see several projects that seem to be putting 
releases constructed from unapproved source code up there. I’ve not looked in 
detail so may be mistaken. I guess sit depends if that concerns you or not.

> Most projects historically have played by the rules. This touches a bit on 
> what it really means to be an Apache Project. My thought here is that the 
> Incubator needs to be a bit more circumspect with regards to onboarding 
> projects before they fully understand the Apache Way. We’ve seen this play 
> out with a significant percentage of projects over the last few years. I 
> don’t know what the solution here is, but the gap between “old school” 
> projects that fall into the Apache Way fairly easily, and the “new school” 
> github-based projects that fly fast and loose seems to be growing by the day.

The incubator does need to do a better job of making the GitHub (and other 
platform)  limitations clear before the project joins. Quite often we’re not 
even aware of how the project was using GitHub, Docker or other channels before 
it joined, it only that after they join that these things become issues and 
then it’s too late.

Thanks,
Justin
-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Tying Dockerhub into development and release management

[Chris Lambertus]

> On Feb 7, 2019, at 5:53 PM, Justin Mclean  wrote:
> 
> Hi,
> 
>> Infra manages the u/apache dockerhub org. We provide this service with the 
>> express caveat that projects note that these are UNOFFICIAL releases, and 
>> CONVENIENCE BINARIES ONLY.
> 
> By that I assume you mean only connivance binaries only created from an 
> official approved/voted on ASF releases. Does that also suggest that release 
> candidates, snapshot and nightlys (even if tagged and/or clearly described) 
> would not be allowed there

Infra has little control over what the project publishes. We expect their good 
will and knowledge of the Apache Release Policy will guide them. As these 
releases are UNOFFICIAL, they could have dockerhub packages built for any 
branch or tag on their repo. This is a social construct, not a technical one, 
but that of course means absolutely nothing to the end user consumer of these 
packages. 

Infra does not police what projects deploy on their dockerhub repos. Do we need 
to?

Most projects historically have played by the rules. This touches a bit on what 
it really means to be an Apache Project. My thought here is that the Incubator 
needs to be a bit more circumspect with regards to onboarding projects before 
they fully understand the Apache Way. We’ve seen this play out with a 
significant percentage of projects over the last few years. I don’t know what 
the solution here is, but the gap between “old school” projects that fall into 
the Apache Way fairly easily, and the “new school” github-based projects that 
fly fast and loose seems to be growing by the day.

> If so doesn’t that just encourage podlings to go make their own 
> https://hub.docker.com/r/apache/ and put them there? And if 
> they do what do we do about it? If a TLP project or podling is found to be 
> using u/apache and placing unapproved releases there what should be done? 
> (Would you like to be informed for starters?)  

I don’t see how this “encourages” any action one way or the other. Many new 
podlings find the restrictions imposed by joining the ASF to be extremely 
detrimental to their community, and are often very surprised to find that 
things that “used to work” on github are now verboten due to our legal policies.


>> Until that juxtaposition is well and truly addressed, these discussions will 
>> continue to happen ad nauseam.
> 
> Well that is problematic.

I agree. This is a topic that comes up frequently, but has yet to be really 
addressed as far as I know.

> Thanks,
> Justin

-Chris
ASF Infra


-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Tying Dockerhub into development and release management

[Justin Mclean]

Hi,

> Infra manages the u/apache dockerhub org. We provide this service with the 
> express caveat that projects note that these are UNOFFICIAL releases, and 
> CONVENIENCE BINARIES ONLY.

By that I assume you mean only connivance binaries only created from an 
official approved/voted on ASF releases. Does that also suggest that release 
candidates, snapshot and nightlys (even if tagged and/or clearly described) 
would not be allowed there

 If so doesn’t that just encourage podlings to go make their own 
https://hub.docker.com/r/apache/ and put them there? And if 
they do what do we do about it? If a TLP project or podling is found to be 
using u/apache and placing unapproved releases there what should be done? 
(Would you like to be informed for starters?)   

> Until that juxtaposition is well and truly addressed, these discussions will 
> continue to happen ad nauseam.

Well that is problematic.

Thanks,
Justin
-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Tying Dockerhub into development and release management

[Chris Lambertus]

> On Feb 7, 2019, at 4:54 PM, Justin Mclean  wrote:
> 
> Hi,
> 
>> I assume that "https://hub.docker.com/u/apache 
>> " is an Apache controlled
>> location, so publishing Apache images from there is fine provided they obey
>> our policies (release policy, website policy etc).
> 
> I believe INFA has something to do with setting that up, but I think we 
> should extend what be consider “official" to anything with apache in it’s 
> user name, like apachepulsar.


Hi, 

Infra manages the u/apache dockerhub org. We provide this service with the 
express caveat that projects note that these are UNOFFICIAL releases, and 
CONVENIENCE BINARIES ONLY. How much projects comply with this is unknown to me. 
It is a common thread in discussions of this nature because the party line is 
“the ASF releases code” but the reality is that “people consume binaries.”

Until that juxtaposition is well and truly addressed, these discussions will 
continue to happen ad nauseam. I have not reviewed the threads on legal, I just 
popped in over here because Dave Fisher pointed it out to me. 

We (Infra) always point out http://www.apache.org/legal/release-policy.html as 
the defining document that describes official release channels. 

-Chris
ASF Infra



-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Tying Dockerhub into development and release management

[Justin Mclean]

Hi,

> Ignore me on this thread. I'll take my ignorance off to a special corner
> and let it beat me up a bit more.

The podlings are already there waiting for you :-)

So what can we offer on guidance to podlings around:
1) Making official releases available on docker (or other platforms)?
2) Want to provide nightly or snapshots on those platforms?

Thanks,
Justin

Re: Tying Dockerhub into development and release management

[Justin Mclean]

Hi,

> I assume that "https://hub.docker.com/u/apache 
> " is an Apache controlled
> location, so publishing Apache images from there is fine provided they obey
> our policies (release policy, website policy etc).

I believe INFA has something to do with setting that up, but I think we should 
extend what be consider “official" to anything with apache in it’s user name, 
like apachepulsar.

> I get more confused in other areas (PyPI, npm) where we don't have a clear
> namespace for acts of the foundation. Is
> https://pypi.org/project/apache-beam/  
> an act of the PMC or an act of some
> random folk who may or may not overlap with the PMC.

Given the confusion, perhaps only allow project names with apache in them to be 
controlled by PMCs and/or assume they are? Much like the domain name policy we 
have. If so they would need to follow ASF release policy. If it’s named apache 
there a good chance users think it official and it probably is run by the PMC 
so policy applies?

Thanks,
Justin

Re: Tying Dockerhub into development and release management

[Hen]

On Thu, Feb 7, 2019 at 4:43 PM Hen  wrote:

>
>
> On Thu, Feb 7, 2019 at 1:49 PM Justin Mclean 
> wrote:
>
>> Hi,
>>
>> > 2. The PPMC should not publish software outside of Apache controlled
>> locations.
>>
>> I’m trying to find where the above has come from as I can find anything
>> in the release or distribution policies. [1] says “It is appropriate to
>> distribute official releases through downstream channels, but inappropriate
>> to distribute unreleased materials through them.”, [4] say this is OK "In
>> all such cases, the binary/bytecode package must have the same version
>> number as the source release and may only add binary/bytecode files that
>> are the result of compiling that version of the source code release.”
>>
>> So everything there  to me is saying it’s OK to distribute versions of
>> release software on platforms like docker and I not sure where the "Apache
>> controlled locations only” restriction has come from.
>>
>
> I'm trying to understand from the thread on legal-discuss on the subject.
> Which frankly I think I'm failing to do.
>
> I see these DockerHub accounts:
>
> https://hub.docker.com/_/httpd   (HTTP Server from Docker?)
> https://hub.docker.com/r/bitnami/apache/  (HTTP Server from Bitnami)
> https://hub.docker.com/u/apache (various images from the ASF)
>
> I assume that "https://hub.docker.com/u/apache; is an Apache controlled
> location, so publishing Apache images from there is fine provided they obey
> our policies (release policy, website policy etc).
>
> On the other hand, Docker and Bitnami do not have to obey our release
> policy for their publishing locations, just our license/trademark-policy.
>
> Assuming the ASF control /apache, which I think believe do, Docker works.
> Though "_/httpd" is confusing as to who that's coming from.
>
> I get more confused in other areas (PyPI, npm) where we don't have a clear
> namespace for acts of the foundation. Is
> https://pypi.org/project/apache-beam/ an act of the PMC or an act of some
> random folk who may or may not overlap with the PMC. It seems it's the
> latter (ie: Pypi packages are not an act of the PMC and therefore don't
> have to obey our release policy, just our license and trademark policy - I
> think that's nuts btw).
>
>
Tried to re-read the thread on legal-discuss and I'm more confused now than
before (though I did note that /u/apache is Apache controlled by Infra).

Ignore me on this thread. I'll take my ignorance off to a special corner
and let it beat me up a bit more.

Hen

Re: Tying Dockerhub into development and release management

[Hen]

On Thu, Feb 7, 2019 at 1:49 PM Justin Mclean 
wrote:

> Hi,
>
> > 2. The PPMC should not publish software outside of Apache controlled
> locations.
>
> I’m trying to find where the above has come from as I can find anything in
> the release or distribution policies. [1] says “It is appropriate to
> distribute official releases through downstream channels, but inappropriate
> to distribute unreleased materials through them.”, [4] say this is OK "In
> all such cases, the binary/bytecode package must have the same version
> number as the source release and may only add binary/bytecode files that
> are the result of compiling that version of the source code release.”
>
> So everything there  to me is saying it’s OK to distribute versions of
> release software on platforms like docker and I not sure where the "Apache
> controlled locations only” restriction has come from.
>

I'm trying to understand from the thread on legal-discuss on the subject.
Which frankly I think I'm failing to do.

I see these DockerHub accounts:

https://hub.docker.com/_/httpd   (HTTP Server from Docker?)
https://hub.docker.com/r/bitnami/apache/  (HTTP Server from Bitnami)
https://hub.docker.com/u/apache (various images from the ASF)

I assume that "https://hub.docker.com/u/apache; is an Apache controlled
location, so publishing Apache images from there is fine provided they obey
our policies (release policy, website policy etc).

On the other hand, Docker and Bitnami do not have to obey our release
policy for their publishing locations, just our license/trademark-policy.

Assuming the ASF control /apache, which I think believe do, Docker works.
Though "_/httpd" is confusing as to who that's coming from.

I get more confused in other areas (PyPI, npm) where we don't have a clear
namespace for acts of the foundation. Is
https://pypi.org/project/apache-beam/ an act of the PMC or an act of some
random folk who may or may not overlap with the PMC. It seems it's the
latter (ie: Pypi packages are not an act of the PMC and therefore don't
have to obey our release policy, just our license and trademark policy - I
think that's nuts btw).

Hen

Re: Tying Dockerhub into development and release management

[Justin Mclean]

Hi,

> 2. The PPMC should not publish software outside of Apache controlled 
> locations.

I’m trying to find where the above has come from as I can find anything in the 
release or distribution policies. [1] says “It is appropriate to distribute 
official releases through downstream channels, but inappropriate to distribute 
unreleased materials through them.”, [4] say this is OK "In all such cases, the 
binary/bytecode package must have the same version number as the source release 
and may only add binary/bytecode files that are the result of compiling that 
version of the source code release.”

So everything there  to me is saying it’s OK to distribute versions of release 
software on platforms like docker and I not sure where the "Apache controlled 
locations only” restriction has come from.

Now the main issue with snapshots and nightly is this [2] "Projects SHALL 
publish official releases and SHALL NOT publish unreleased materials outside 
the development community.”, [3] explains it’s about legal protection of (P)PMC 
members, and [4] prohibits links to such releases. The distribution policy [5] 
also states that unreleased material "MAY be distributed to consenting members 
of a development community.”

So I believe  the question here with docker is:  Is using a tag enough to 
satisfy that the PPMC is only distributing to consenting members of a 
development community?

Thanks,
Justin


1, https://issues.apache.org/jira/browse/LEGAL-270
2. http://www.apache.org/legal/release-policy.html#publication
3. http://www.apache.org/legal/release-policy.html#why
4. http://www.apache.org/legal/release-policy.html#what
5. https://www.apache.org/dev/release-distribution.html#unreleased
-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Tying Dockerhub into development and release management

[Roman Shaposhnik]

Hey Justin -- any chance you can take a look at the proposed policy for
*snapshot* Docker hub artifacts I proposed up-thread?

Thanks,
Roman.

On Thu, Feb 7, 2019 at 6:13 AM Justin Mclean 
wrote:

> Hi,
>
> > AFAIK everyone releasing binary convenience to dockerhub believed they
> were authorized.
>
> I didn't have a problem with them if they are based on actual released
> software, although it seem it’s now not clear that in line with policy or
> not, lets alone if a PPMC can  release tagged nightly or snapshots there.
>
> Thanks,
> Justin
>
>
> -
> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> For additional commands, e-mail: general-h...@incubator.apache.org
>
>

Re: Tying Dockerhub into development and release management

[Justin Mclean]

Hi,

> AFAIK everyone releasing binary convenience to dockerhub believed they were 
> authorized.

I didn't have a problem with them if they are based on actual released 
software, although it seem it’s now not clear that in line with policy or not, 
lets alone if a PPMC can  release tagged nightly or snapshots there.

Thanks,
Justin


-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Tying Dockerhub into development and release management

[Dave Fisher]

Hey - not everyone agrees with Justin’s and Henry’s conclusions.

I don’t have time to argue every point.

AFAIK everyone releasing binary convenience to dockerhub believed they were 
authorized.

Pulsar had ONE release that was current for a short time that mistakenly kept 
an RC in the name. It was a valid release. Please review the vote thread.

Please stop these allegations and work to clarify the policy!

Until tomorrow.

Regards,
Dave

Sent from my iPhone

> On Feb 6, 2019, at 4:10 PM, Dave  wrote:
> 
> Seems like some well established projects need some schooling from The
> Incubator :-)
> 
> 
> On Wed, Feb 6, 2019 at 6:41 PM Justin Mclean 
> wrote:
> 
>> Hi,
>> 
>>> If projects want to make convenience binaries available for installation
>>> via Docker and DockerHub, then it seems like we need an official Apache
>>> DockerHub repository. Do we have one of those, or are folks just
>> publishing
>>> to personal repos?
>> 
>> A quick look shows HTTP, Maven, Tomcat, Casandra, Solr, Groovy and a
>> number of other Apache TLP using docker hub. Well they may be it’s hard to
>> know who is publishing them.
>> 
>> All are using Apache branding and most are marked as "Docker Official
>> Images” [1]
>> 
>> None seem to be publishing nightly there but there are binary versions of
>> released source code in all these projects.
>> 
>> I can also see ignite [2] published under an apache ignite account and
>> pulsar [2] under an apache pulsar account and nutch [3] published under an
>> apache account.
>> 
>> Pulsar is publishing RCs and looks like it was doing so during incubation
>> :-(
>> 
>> There are more project published under an apache named account here. [6]
>> That account would look official to an outsider from its given details.
>> 
>> A couple seem to be / may be pointing to master [7] or latest [9] and
>> others are releasing snapshots.[8] No wonder podlings get confused.
>> 
>> Thanks,
>> Justin
>> 
>> 1. https://hub.docker.com/search?q=Apache=image
>> 2. https://hub.docker.com/r/apacheignite/ignite
>> 3. https://hub.docker.com/r/apacheignite/ignite
>> 4. https://hub.docker.com/r/apachepulsar/pulsar
>> 5. https://hub.docker.com/r/apache/nutch
>> 6. https://hub.docker.com/u/apache
>> 7. https://hub.docker.com/r/apache/yetus/tags
>> 8. https://hub.docker.com/r/apache/syncope/tags
>> 9. https://hub.docker.com/r/apache/airflow/tags
>> -
>> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
>> For additional commands, e-mail: general-h...@incubator.apache.org
>> 
>> 


-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Tying Dockerhub into development and release management

[Hen]

On Wed, Feb 6, 2019 at 8:34 PM Justin Mclean 
wrote:

> HI,
>
> > 2. The PPMC should not publish software outside of Apache controlled
> > locations.
>
> We have TLP / PMCs doing that already
>
> > 3. Third parties may publish software based on Apache's, but they must
> not
> > cause user confusion (i.e. respect trademarks).
>
> And we have this as well (see docker links).
>
> All a bit of a mess really.
>
> Justin



Agreed :)

I think PMCs are trying to do the right thing for the public and we need to
come up with structure for how PMCs should publish installables on 3p
locations.

Hen

>

Re: Tying Dockerhub into development and release management

[Justin Mclean]

HI,

> 2. The PPMC should not publish software outside of Apache controlled
> locations.

We have TLP / PMCs doing that already

> 3. Third parties may publish software based on Apache's, but they must not
> cause user confusion (i.e. respect trademarks).

And we have this as well (see docker links).

All a bit of a mess really.

Justin

Re: Tying Dockerhub into development and release management

[Dave]

Seems like some well established projects need some schooling from The
Incubator :-)


On Wed, Feb 6, 2019 at 6:41 PM Justin Mclean 
wrote:

> Hi,
>
> > If projects want to make convenience binaries available for installation
> > via Docker and DockerHub, then it seems like we need an official Apache
> > DockerHub repository. Do we have one of those, or are folks just
> publishing
> > to personal repos?
>
> A quick look shows HTTP, Maven, Tomcat, Casandra, Solr, Groovy and a
> number of other Apache TLP using docker hub. Well they may be it’s hard to
> know who is publishing them.
>
> All are using Apache branding and most are marked as "Docker Official
> Images” [1]
>
> None seem to be publishing nightly there but there are binary versions of
> released source code in all these projects.
>
> I can also see ignite [2] published under an apache ignite account and
> pulsar [2] under an apache pulsar account and nutch [3] published under an
> apache account.
>
> Pulsar is publishing RCs and looks like it was doing so during incubation
> :-(
>
> There are more project published under an apache named account here. [6]
> That account would look official to an outsider from its given details.
>
>  A couple seem to be / may be pointing to master [7] or latest [9] and
> others are releasing snapshots.[8] No wonder podlings get confused.
>
> Thanks,
> Justin
>
> 1. https://hub.docker.com/search?q=Apache=image
> 2. https://hub.docker.com/r/apacheignite/ignite
> 3. https://hub.docker.com/r/apacheignite/ignite
> 4. https://hub.docker.com/r/apachepulsar/pulsar
> 5. https://hub.docker.com/r/apache/nutch
> 6. https://hub.docker.com/u/apache
> 7. https://hub.docker.com/r/apache/yetus/tags
> 8. https://hub.docker.com/r/apache/syncope/tags
> 9. https://hub.docker.com/r/apache/airflow/tags
> -
> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> For additional commands, e-mail: general-h...@incubator.apache.org
>
>

Re: Tying Dockerhub into development and release management

[Justin Mclean]

Hi,

> If projects want to make convenience binaries available for installation
> via Docker and DockerHub, then it seems like we need an official Apache
> DockerHub repository. Do we have one of those, or are folks just publishing
> to personal repos?

A quick look shows HTTP, Maven, Tomcat, Casandra, Solr, Groovy and a number of 
other Apache TLP using docker hub. Well they may be it’s hard to know who is 
publishing them.

All are using Apache branding and most are marked as "Docker Official Images” 
[1]

None seem to be publishing nightly there but there are binary versions of 
released source code in all these projects.

I can also see ignite [2] published under an apache ignite account and pulsar 
[2] under an apache pulsar account and nutch [3] published under an apache 
account.

Pulsar is publishing RCs and looks like it was doing so during incubation :-(

There are more project published under an apache named account here. [6] That 
account would look official to an outsider from its given details.

 A couple seem to be / may be pointing to master [7] or latest [9] and others 
are releasing snapshots.[8] No wonder podlings get confused.

Thanks,
Justin

1. https://hub.docker.com/search?q=Apache=image
2. https://hub.docker.com/r/apacheignite/ignite
3. https://hub.docker.com/r/apacheignite/ignite
4. https://hub.docker.com/r/apachepulsar/pulsar
5. https://hub.docker.com/r/apache/nutch
6. https://hub.docker.com/u/apache
7. https://hub.docker.com/r/apache/yetus/tags
8. https://hub.docker.com/r/apache/syncope/tags
9. https://hub.docker.com/r/apache/airflow/tags
-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Tying Dockerhub into development and release management

[Dave]

Thanks, Hen. Seems like the "Apache controlled locations" bit is important
here.

If projects want to make convenience binaries available for installation
via Docker and DockerHub, then it seems like we need an official Apache
DockerHub repository. Do we have one of those, or are folks just publishing
to personal repos?

I recently made a Docker image available for the unreleased Apache Roller
6.0.0-SNAPSHOT on my personal DockerHub account, which I believe is kosher,
but where should the Roller project publish the convenience binary/Docker
image for the official 6.0.0 release?

Thanks,
Dave


On Wed, Feb 6, 2019 at 4:22 PM Roman Shaposhnik 
wrote:

> On Wed, Feb 6, 2019 at 10:12 PM Hen  wrote:
>
> > On Tue, Feb 5, 2019 at 6:45 AM Roman Shaposhnik 
> > wrote:
> >
> > > On Tue, Feb 5, 2019 at 2:48 PM Dave  wrote:
> > >
> > > > I totally agree with you that Docker images should be built from
> > official
> > > > source releases, unless they are clearly marked as unofficial
> SNAPSHOT
> > > > releases and intended for testing. I'm just repeating what I've heard
> > > over
> > > > and over again from various ASF members that the only official
> release
> > is
> > > > the source release; I'd don't agree with that point of view.
> > > >
> > > > I'm curious what "built from the official source releases". Does that
> > > mean
> > > > that you must create Docker images by downloading the official source
> > > > release, verifying it's hash and then building image?  Or, are you
> > > allowed
> > > > to build your Docker images from the same SCM tag as was used to
> create
> > > the
> > > > source release?
> > > >
> > >
> > > I think an acceptable solution could be:
> > >* make sure that your :latest tag either points to a Docker scratch
> > > container
> > >  or a container that simply prints Incubator disclaimer and exists
> > >* introduce a tagging scheme for nightly builds (personally I'm
> quite
> > > fond
> > >  of tagging nightly docker builds with SHAs from your git tree from
> > > which
> > >  you build the image)
> > >* introduce :snapshot tag that points at the latest tag from
> previous
> > > item
> > >
> > > I feel that this could be passable for IPMC.
> > >
> > >
> > I remain confused on this topic.
> >
>
> We're not talking about "official" release binaries (whatever that means at
> ASF).
> We're talking about snapshot binaries that need to be available for
> developers.
> I don't think the rest of your reasoning applies *to this* particular
> discussion.
>
>
> >
> > The legal-discuss thread leads me to think the current state is:
> >
> > 1. The PPMC release some source code. They may release convenience
> binaries
> > on the Apache distribution urls, or in Maven Central (via Infra's
> support),
> > and those binaries must be built from the release soruce.
> > 2. The PPMC should not publish software outside of Apache controlled
> > locations.
> > 3. Third parties may publish software based on Apache's, but they must
> not
> > cause user confusion (i.e. respect trademarks).
> > 4. The PPMC may link to the software (including binaries) published by a
> > third party, but they should flag that it does not come from Apache and
> > should not treat it as the default user experience.
> >
> > All of which means PPMCs must not use PyPI, NuGet, NPM, DockerHub, etc.
> > unless Infra actively support a mechanism of doing so (which they
> > definitely do for Maven).
> >
> > (Though I'm confused as to whether #2 is a must not, should not, or can
> if
> > they wish to)
> >
> > Hen
> >
>

Re: Tying Dockerhub into development and release management

[Roman Shaposhnik]

On Wed, Feb 6, 2019 at 10:12 PM Hen  wrote:

> On Tue, Feb 5, 2019 at 6:45 AM Roman Shaposhnik 
> wrote:
>
> > On Tue, Feb 5, 2019 at 2:48 PM Dave  wrote:
> >
> > > I totally agree with you that Docker images should be built from
> official
> > > source releases, unless they are clearly marked as unofficial SNAPSHOT
> > > releases and intended for testing. I'm just repeating what I've heard
> > over
> > > and over again from various ASF members that the only official release
> is
> > > the source release; I'd don't agree with that point of view.
> > >
> > > I'm curious what "built from the official source releases". Does that
> > mean
> > > that you must create Docker images by downloading the official source
> > > release, verifying it's hash and then building image?  Or, are you
> > allowed
> > > to build your Docker images from the same SCM tag as was used to create
> > the
> > > source release?
> > >
> >
> > I think an acceptable solution could be:
> >* make sure that your :latest tag either points to a Docker scratch
> > container
> >  or a container that simply prints Incubator disclaimer and exists
> >* introduce a tagging scheme for nightly builds (personally I'm quite
> > fond
> >  of tagging nightly docker builds with SHAs from your git tree from
> > which
> >  you build the image)
> >* introduce :snapshot tag that points at the latest tag from previous
> > item
> >
> > I feel that this could be passable for IPMC.
> >
> >
> I remain confused on this topic.
>

We're not talking about "official" release binaries (whatever that means at
ASF).
We're talking about snapshot binaries that need to be available for
developers.
I don't think the rest of your reasoning applies *to this* particular
discussion.


>
> The legal-discuss thread leads me to think the current state is:
>
> 1. The PPMC release some source code. They may release convenience binaries
> on the Apache distribution urls, or in Maven Central (via Infra's support),
> and those binaries must be built from the release soruce.
> 2. The PPMC should not publish software outside of Apache controlled
> locations.
> 3. Third parties may publish software based on Apache's, but they must not
> cause user confusion (i.e. respect trademarks).
> 4. The PPMC may link to the software (including binaries) published by a
> third party, but they should flag that it does not come from Apache and
> should not treat it as the default user experience.
>
> All of which means PPMCs must not use PyPI, NuGet, NPM, DockerHub, etc.
> unless Infra actively support a mechanism of doing so (which they
> definitely do for Maven).
>
> (Though I'm confused as to whether #2 is a must not, should not, or can if
> they wish to)
>
> Hen
>

Re: Tying Dockerhub into development and release management

[Hen]

On Tue, Feb 5, 2019 at 6:45 AM Roman Shaposhnik 
wrote:

> On Tue, Feb 5, 2019 at 2:48 PM Dave  wrote:
>
> > I totally agree with you that Docker images should be built from official
> > source releases, unless they are clearly marked as unofficial SNAPSHOT
> > releases and intended for testing. I'm just repeating what I've heard
> over
> > and over again from various ASF members that the only official release is
> > the source release; I'd don't agree with that point of view.
> >
> > I'm curious what "built from the official source releases". Does that
> mean
> > that you must create Docker images by downloading the official source
> > release, verifying it's hash and then building image?  Or, are you
> allowed
> > to build your Docker images from the same SCM tag as was used to create
> the
> > source release?
> >
>
> I think an acceptable solution could be:
>* make sure that your :latest tag either points to a Docker scratch
> container
>  or a container that simply prints Incubator disclaimer and exists
>* introduce a tagging scheme for nightly builds (personally I'm quite
> fond
>  of tagging nightly docker builds with SHAs from your git tree from
> which
>  you build the image)
>* introduce :snapshot tag that points at the latest tag from previous
> item
>
> I feel that this could be passable for IPMC.
>
>
I remain confused on this topic.

The legal-discuss thread leads me to think the current state is:

1. The PPMC release some source code. They may release convenience binaries
on the Apache distribution urls, or in Maven Central (via Infra's support),
and those binaries must be built from the release soruce.
2. The PPMC should not publish software outside of Apache controlled
locations.
3. Third parties may publish software based on Apache's, but they must not
cause user confusion (i.e. respect trademarks).
4. The PPMC may link to the software (including binaries) published by a
third party, but they should flag that it does not come from Apache and
should not treat it as the default user experience.

All of which means PPMCs must not use PyPI, NuGet, NPM, DockerHub, etc.
unless Infra actively support a mechanism of doing so (which they
definitely do for Maven).

(Though I'm confused as to whether #2 is a must not, should not, or can if
they wish to)

Hen

Re: Tying Dockerhub into development and release management

[Roman Shaposhnik]

On Tue, Feb 5, 2019 at 2:48 PM Dave  wrote:

> I totally agree with you that Docker images should be built from official
> source releases, unless they are clearly marked as unofficial SNAPSHOT
> releases and intended for testing. I'm just repeating what I've heard over
> and over again from various ASF members that the only official release is
> the source release; I'd don't agree with that point of view.
>
> I'm curious what "built from the official source releases". Does that mean
> that you must create Docker images by downloading the official source
> release, verifying it's hash and then building image?  Or, are you allowed
> to build your Docker images from the same SCM tag as was used to create the
> source release?
>

I think an acceptable solution could be:
   * make sure that your :latest tag either points to a Docker scratch
container
 or a container that simply prints Incubator disclaimer and exists
   * introduce a tagging scheme for nightly builds (personally I'm quite
fond
 of tagging nightly docker builds with SHAs from your git tree from
which
 you build the image)
   * introduce :snapshot tag that points at the latest tag from previous
item

I feel that this could be passable for IPMC.

Thanks,
Roman.

Re: Tying Dockerhub into development and release management

[Dave]

I totally agree with you that Docker images should be built from official
source releases, unless they are clearly marked as unofficial SNAPSHOT
releases and intended for testing. I'm just repeating what I've heard over
and over again from various ASF members that the only official release is
the source release; I'd don't agree with that point of view.

I'm curious what "built from the official source releases". Does that mean
that you must create Docker images by downloading the official source
release, verifying it's hash and then building image?  Or, are you allowed
to build your Docker images from the same SCM tag as was used to create the
source release?

Dave

On Tue, Feb 5, 2019 at 5:23 AM Justin Mclean 
wrote:

> Hi,
>
> > My understanding is that only source-code releases are official releases.
> > Binaries are just a convenience and not official, so I don't think you
> have
> > to worry about making images available via DockerHub, even if that is the
> > primary way most people install.
>
> -1 the PMC  can’t release unapproved code to the general public, this
> mandated by ASF release policy and has been previous discussed here [1].
>
> "It is appropriate to distribute official releases through downstream
> channels, but inappropriate to distribute unreleased materials through
> them.”
>
> Docker images MUST be built from our official source releases.
>
> Put it this way if a PPMC could just publish what they wanted  to docker
> at any time why bother with official release at all.
>
> Thanks,
> Justin
>
> 1. https://issues.apache.org/jira/browse/LEGAL-270
> -
> To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
> For additional commands, e-mail: general-h...@incubator.apache.org
>
>

Re: Tying Dockerhub into development and release management

[Justin Mclean]

Hi,

> My understanding is that only source-code releases are official releases.
> Binaries are just a convenience and not official, so I don't think you have
> to worry about making images available via DockerHub, even if that is the
> primary way most people install.

-1 the PMC  can’t release unapproved code to the general public, this mandated 
by ASF release policy and has been previous discussed here [1].

"It is appropriate to distribute official releases through downstream channels, 
but inappropriate to distribute unreleased materials through them.”

Docker images MUST be built from our official source releases.

Put it this way if a PPMC could just publish what they wanted  to docker at any 
time why bother with official release at all.

Thanks,
Justin

1. https://issues.apache.org/jira/browse/LEGAL-270
-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Re: Tying Dockerhub into development and release management

[Dave]

My understanding is that only source-code releases are official releases.
Binaries are just a convenience and not official, so I don't think you have
to worry about making images available via DockerHub, even if that is the
primary way most people install.

Dave

On Sun, Feb 3, 2019 at 1:00 PM lewis john mcgibbney 
wrote:

> Hi general@,
> Over at the SDAP podling we are currently involved in a bit of a situation
> regarding the use of Dockerhub in our development and release management.
> The primary mechanism for the deployment of SDAP’s NEXUS component is
> Docker. It is therefore necessary for us to have available Docker images
> which can be consumed by people trying to consume and deploy the SDAP
> software.
> The issue we have run into however is that the availability of Docker
> images in Dockerhub appears to be in violation of Apache release policy. Is
> this true for all images e.g. tagged and version controlled, hosted within
> Dockerhub or is it acceptable for us to publish version controlled images
> in Dockerhub?
> Can anyone share experiences facilitating the use of Dockerhub throughout
> development cycles and for staging release candidates which include Docker
> artifacts?
> Thank you
> --
> http://home.apache.org/~lewismc/
> http://people.apache.org/keys/committer/lewismc
>

Re: Tying Dockerhub into development and release management

[Ross Gardler]

These would not be official releases. One or more of your community can create 
Docker builds from apache released source. Ideally the build files will be part 
of the ASF project.

Dockerhub has GitHub integration, so all it takes is for someone in the 
community to create the account and connect it to the repo.

Get Outlook for Android<https://aka.ms/ghei36>


From: lewis john mcgibbney 
Sent: Sunday, February 3, 2019 10:00:39 AM
To: general@incubator.apache.org
Cc: d...@sdap.apache.org
Subject: Tying Dockerhub into development and release management

Hi general@,
Over at the SDAP podling we are currently involved in a bit of a situation
regarding the use of Dockerhub in our development and release management.
The primary mechanism for the deployment of SDAP’s NEXUS component is
Docker. It is therefore necessary for us to have available Docker images
which can be consumed by people trying to consume and deploy the SDAP
software.
The issue we have run into however is that the availability of Docker
images in Dockerhub appears to be in violation of Apache release policy. Is
this true for all images e.g. tagged and version controlled, hosted within
Dockerhub or is it acceptable for us to publish version controlled images
in Dockerhub?
Can anyone share experiences facilitating the use of Dockerhub throughout
development cycles and for staging release candidates which include Docker
artifacts?
Thank you
--
https://eur02.safelinks.protection.outlook.com/?url=http:%2F%2Fhome.apache.org%2F~lewismc%2Fdata=02%7C01%7C%7C6d9017f7dbc04dc1824c08d68a018b3a%7C84df9e7fe9f640afb435%7C1%7C0%7C636848136560161169sdata=d52cOLjekmHhjncZcdI8pMnKhlXmcdxuIOaSmBFf%2F%2Bk%3Dreserved=0
https://eur02.safelinks.protection.outlook.com/?url=http%3A%2F%2Fpeople.apache.org%2Fkeys%2Fcommitter%2Flewismcdata=02%7C01%7C%7C6d9017f7dbc04dc1824c08d68a018b3a%7C84df9e7fe9f640afb435%7C1%7C0%7C636848136560161169sdata=t4CngAQFIONjFr078NHVXCXsyOk1dmKQucQvwdp2F8M%3Dreserved=0

Re: Tying Dockerhub into development and release management

[Justin Mclean]

Hi,

> The issue we have run into however is that the availability of Docker
> images in Dockerhub appears to be in violation of Apache release policy. Is
> this true for all images e.g. tagged and version controlled, hosted within
> Dockerhub or is it acceptable for us to publish version controlled images
> in Dockerhub?

If they consist of code from a voted on release then that’s fine. This has been 
discussed a bit recently,  see these two legal JIRAs [1][2]. In short “It is 
appropriate to distribute official releases through downstream channels, but 
inappropriate to distribute unreleased materials through them.”

However I assume you asking about what to do with release candidates, my guess 
is to make sure they are not the default latest release and come up with a 
tagging scheme.

Thanks,
Justin

1. https://issues.apache.org/jira/browse/LEGAL-270
2. https://issues.apache.org/jira/browse/LEGAL-427




-
To unsubscribe, e-mail: general-unsubscr...@incubator.apache.org
For additional commands, e-mail: general-h...@incubator.apache.org

Tying Dockerhub into development and release management

[lewis john mcgibbney]

Hi general@,
Over at the SDAP podling we are currently involved in a bit of a situation
regarding the use of Dockerhub in our development and release management.
The primary mechanism for the deployment of SDAP’s NEXUS component is
Docker. It is therefore necessary for us to have available Docker images
which can be consumed by people trying to consume and deploy the SDAP
software.
The issue we have run into however is that the availability of Docker
images in Dockerhub appears to be in violation of Apache release policy. Is
this true for all images e.g. tagged and version controlled, hosted within
Dockerhub or is it acceptable for us to publish version controlled images
in Dockerhub?
Can anyone share experiences facilitating the use of Dockerhub throughout
development cycles and for staging release candidates which include Docker
artifacts?
Thank you
-- 
http://home.apache.org/~lewismc/
http://people.apache.org/keys/committer/lewismc