Re: Proposal to add "release requirements" check

[Jarek Potiuk]

It's an interesting (but not practical) idea to base the set of active
branches on the links in documentation pages.I would never try to do
that really and it was never anywhere close to my proposal - scanning
download pages is only to check if they have proper links/closer.lua
script, nothing else.

I think I am not looking for a perfect solution from day one which
will solve all cases. Problem with that is the same as with DAOs -
assuming that we have them is nice, but practically that does not work
as it requires upfront work from the projects. This makes them
essentially useless because projects do not have them and they are not
updated. It's a bit of a chicken-egg problem for all systems that
require every independent "project" in the organisation to implement
something in order to see the benefits of it. You cannot really make
use of a DAO to get "common view" of the projects until you are sure
that a) they are there b) they are updated regularly and that requires
the interested "projects" keep them updated as part of their
processes. I have no doubt implementing a new common process across
all the projects of ASF is next to impossible if you want to add a
"new" process to follow.  I am not "that" stupid to not realize that.

My idea is different - to tap as much as possible into existing
processes and make it a one-off effort for the project to comply. The
idea is that If it is flagged at board review time (automatically)
that the project does not follow rules, they will get notified and
after they correct their existing process/configuration/process it
will continue to work without change (and they will see if they are
compliant).

Tapping into existing processes:

1) updating URL to release page at "release submission page" seems
doable (and there are various ways you can make it easy and natural)
2) "downloads.apache.org" and "archives.apache.org" are already
updated (same process) - detecting artifacts that should be removed
from downloads - seems quite possible. They can look differently for
different projects. But my assumption is (something to work out over
time) that there are just a few patterns of versions to follow.

So coming back to point 1)  we might ask the projects to (again at
release submission time) choose which pattern they use (Semver or
other) - and even more explicitly ask them if there are releases that
should be made obsolete/inactive. That will require a bit of guessing
and mistakes initially when we add more projects, but - again - there
is a board review process that we can tap into. Every time there is a
board review time, we can look at the projects that are coming and add
a pattern or two if missing - and during the first review to ask them
about old releases. Eventually after a full round, the only effort
will be to add new projects that graduate, but then they will have
enough patterns to choose from at the first release

This means that we can have an "eventually consistent" system after
sufficient time passes - the important thing is that we can do it
gradually and over time. Even if we add a few projects a month,
eventually we will get there.

BTW. I actually tried to submit DAO for Airflow and failed miserably
(it was a long time ago and I don't remember details but tooling
set-up was not worth it and I gave up finally - also because I could
not see what DAO submission brings us or the ASF (precisely the
chicken-egg problem)

On Mon, Nov 8, 2021 at 10:54 PM sebb  wrote:
>
> On Mon, 8 Nov 2021 at 17:49, Jarek Potiuk  wrote:
> >
> > > However this does not solve the issue of stale download pages: how do
> > you know when a release version is no longer supported?
> >
> > What do you mean?
> >
> > Is there a requirement for that that I missed? I believe the
> > requirement is only about removing artifacts from "downloads.a.o" and
> > this is a completely different issue.
>
> That *is* what I am referring to.
> In order to remove artifacts from downloads.a.o, it needs to be
> established whether or not the release is still supported.
>
> If a release does not appear in any download page, then it is a
> candidate for removal (unless it is brand new).
>
> However if it does appear as a link to downloads.a.o in a download
> page, that does not necessarily mean that it is still a supported
> release.
>
> This is because projects don't always keep their download pages up to
> date, especially where there are individual pages for each release.
>
> For example, Airflow has download pages for 2.2.1 and 2.2.0; these
> both link to files on downloads.a.o.
>
> I suppose it is possible that there are plans to support updates to
> 2.2.0 separately from 2.2.1, but that seems unlikely, assuming that
> the project uses semantic versioning or something similar.
>
> > On Mon, Nov 8, 2021 at 6:42 PM sebb  wrote:
> > >
> > > On Mon, 8 Nov 2021 at 17:11, Jarek Potiuk  wrote:
> > > >
> > > > Yeah. We will have different URLs especially as more artifacts are
> > > > produced (airflow has 3 

Re: Proposal to add "release requirements" check

[sebb]

On Mon, 8 Nov 2021 at 17:49, Jarek Potiuk  wrote:
>
> > However this does not solve the issue of stale download pages: how do
> you know when a release version is no longer supported?
>
> What do you mean?
>
> Is there a requirement for that that I missed? I believe the
> requirement is only about removing artifacts from "downloads.a.o" and
> this is a completely different issue.

That *is* what I am referring to.
In order to remove artifacts from downloads.a.o, it needs to be
established whether or not the release is still supported.

If a release does not appear in any download page, then it is a
candidate for removal (unless it is brand new).

However if it does appear as a link to downloads.a.o in a download
page, that does not necessarily mean that it is still a supported
release.

This is because projects don't always keep their download pages up to
date, especially where there are individual pages for each release.

For example, Airflow has download pages for 2.2.1 and 2.2.0; these
both link to files on downloads.a.o.

I suppose it is possible that there are plans to support updates to
2.2.0 separately from 2.2.1, but that seems unlikely, assuming that
the project uses semantic versioning or something similar.

> On Mon, Nov 8, 2021 at 6:42 PM sebb  wrote:
> >
> > On Mon, 8 Nov 2021 at 17:11, Jarek Potiuk  wrote:
> > >
> > > Yeah. We will have different URLs especially as more artifacts are
> > > produced (airflow has 3 of those each with different download pages).
> > >
> > > But this should be straightforward when you can add the url when
> > > registering the release (as Sebb proposed). According to the release
> > > process you should register each such "release type" separately and
> > > adding a URL there will do the job nicely. And initially it can be
> > > optional and once we figure out how to communicate it best and maybe
> > > try with some projects first we can make it obligatory at release
> > > registration. We could make it convenient - for example an easy way to
> > > copy a previously added URL from selected previous release or
> > > autocomplete from previous entries - making the process of entry as
> > > smooth (or even smoother) than it is today.
> >
> > Or you could just ask for the URL if the release is not already on a
> > download page.
> > This would mean parsing the page of course, but could be used to
> > provide immediate feedback if the page is non-conforming.
> >
> > Asking for the page might also nudge projects to move to generic pages
> > rather than one per release.
> >
> > However this does not solve the issue of stale download pages: how do
> > you know when a release version is no longer supported?
> >
> > > J.
> > >
> > >
> > > On Mon, Nov 8, 2021 at 4:19 PM sebb  wrote:
> > > >
> > > > On Mon, 8 Nov 2021 at 11:29, Hans Van Akelyen
> > > >  wrote:
> > > > >
> > > > > Hi,
> > > > >
> > > > > I think a certain level of uniformity would not be bad for this.
> > > > > Having a fixed path to the downloads for each project makes it 
> > > > > simpler for
> > > > > the users to "shop" for Apache projects.
> > > > >
> > > > > Having .a.o/download/ as a link that should point to the 
> > > > > resources
> > > > > makes it simpler for everyone.
> > > >
> > > > I agree, but expect a lot of complaints if you try to get this enforced.
> > > > (and there's not much point unless it is enforced).
> > > >
> > > > > Kind Regards,
> > > > > Hans
> > > > >
> > > > > On Mon, 8 Nov 2021 at 12:19, Justin Mclean  
> > > > > wrote:
> > > > >
> > > > > > Hi,
> > > > > >
> > > > > > > However that assumes that you know where to find the download 
> > > > > > > page(s).
> > > > > > > It also assumes that projects update their download page(s) to 
> > > > > > > only
> > > > > > > show current releases.
> > > > > >
> > > > > > It does indeed assume that, and a few other things. Currently policy
> > > > > > doesn't specify what the download page URL should be, but once 
> > > > > > known I
> > > > > > would guess it wouldn’t change often.
> > > > > >
> > > > > > Kind Regards,
> > > > > > Justin
> > > > > >
> > > > > >
> > > > > >
> > > > > > -
> > > > > > To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
> > > > > > For additional commands, e-mail: dev-h...@community.apache.org
> > > > > >
> > > > > >
> > > >
> > > > -
> > > > To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
> > > > For additional commands, e-mail: dev-h...@community.apache.org
> > > >
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
> > > For additional commands, e-mail: dev-h...@community.apache.org
> > >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
> > For additional comman

Re: Proposal to add "release requirements" check

[Hans Van Akelyen]

I was also talking more from a user perspective.
It's not that hard to create a one pager that contains the latest version,
release dates, release notes and keys for all packages the project provides.

In the case of Airflow it's pretty much what you put in announcements, and
this one is fairly obvious to find.
On some other projects you really have to start digging.

And I am not sure I would want yet another xml/json/yaml config file that
has to be maintained when creating a release.
eg. you should update your DOAP file too (
https://projects.apache.org/project.html?airflow) as Dave said, many
projects fail to do so.

Maybe the download page url should be configurable (use the DOAP?), but I
would rather have a uniform way of finding that page.

On Mon, 8 Nov 2021 at 19:11, Dave Fisher  wrote:

>
>
> > On Nov 8, 2021, at 9:48 AM, Jarek Potiuk  wrote:
> >
> >> However this does not solve the issue of stale download pages: how do
> > you know when a release version is no longer supported?
> >
> > What do you mean?
> >
> > Is there a requirement for that that I missed? I believe the
> > requirement is only about removing artifacts from "downloads.a.o" and
> > this is a completely different issue.
>
> There is no definitive rule about the path to the download page. There is
> no standard and with over 200 projects, 35 podlings, and lots of products
> change is difficult.
>
> There is one definite requirement that can be checked. Whether the
> releases in svn are properly formed.
>
> We do this in the Incubator with the Clutch Analysis.
> https://svn.apache.org/repos/asf/incubator/public/trunk/clutch2.py - L1132
> Output found by clicking on a podling on
> https://incubator.apache.org/clutch/
>
> We also have code in ASF Pelican -
> https://github.com/apache/infrastructure-pelican/blob/master/plugins/asfdata.py#L362
> That was used to produce this example for Hadoop -
> https://template.staged.apache.org/downloads.html
> Using this template -
> https://github.com/apache/template-site/blob/main/content/downloads.ezmd
> Control is -
> https://github.com/apache/template-site/blob/main/asfdata.yaml#L110
>
> To review projects you might find out where their download page is by
> reviewing the project DOAP file, but we’ve never been able to get all
> projects to maintain one.
>
> Maybe it’s time to more carefully define what the ask is here.
>
> >
> > On Mon, Nov 8, 2021 at 6:42 PM sebb  wrote:
> >>
> >> On Mon, 8 Nov 2021 at 17:11, Jarek Potiuk  wrote:
> >>>
> >>> Yeah. We will have different URLs especially as more artifacts are
> >>> produced (airflow has 3 of those each with different download pages).
> >>>
> >>> But this should be straightforward when you can add the url when
> >>> registering the release (as Sebb proposed). According to the release
> >>> process you should register each such "release type" separately and
> >>> adding a URL there will do the job nicely. And initially it can be
> >>> optional and once we figure out how to communicate it best and maybe
> >>> try with some projects first we can make it obligatory at release
> >>> registration. We could make it convenient - for example an easy way to
> >>> copy a previously added URL from selected previous release or
> >>> autocomplete from previous entries - making the process of entry as
> >>> smooth (or even smoother) than it is today.
> >>
> >> Or you could just ask for the URL if the release is not already on a
> >> download page.
> >> This would mean parsing the page of course, but could be used to
> >> provide immediate feedback if the page is non-conforming.
> >>
> >> Asking for the page might also nudge projects to move to generic pages
> >> rather than one per release.
> >>
> >> However this does not solve the issue of stale download pages: how do
> >> you know when a release version is no longer supported?
> >>
> >>> J.
> >>>
> >>>
> >>> On Mon, Nov 8, 2021 at 4:19 PM sebb  wrote:
> 
>  On Mon, 8 Nov 2021 at 11:29, Hans Van Akelyen
>   wrote:
> >
> > Hi,
> >
> > I think a certain level of uniformity would not be bad for this.
> > Having a fixed path to the downloads for each project makes it
> simpler for
> > the users to "shop" for Apache projects.
> >
> > Having .a.o/download/ as a link that should point to the
> resources
> > makes it simpler for everyone.
> 
>  I agree, but expect a lot of complaints if you try to get this
> enforced.
>  (and there's not much point unless it is enforced).
> 
> > Kind Regards,
> > Hans
> >
> > On Mon, 8 Nov 2021 at 12:19, Justin Mclean 
> wrote:
> >
> >> Hi,
> >>
> >>> However that assumes that you know where to find the download
> page(s).
> >>> It also assumes that projects update their download page(s) to only
> >>> show current releases.
> >>
> >> It does indeed assume that, and a few other things. Currently policy
> >> doesn't specify what the download page URL should be, but once
>

Re: Proposal to add "release requirements" check

[Dave Fisher]

> On Nov 8, 2021, at 9:48 AM, Jarek Potiuk  wrote:
> 
>> However this does not solve the issue of stale download pages: how do
> you know when a release version is no longer supported?
> 
> What do you mean?
> 
> Is there a requirement for that that I missed? I believe the
> requirement is only about removing artifacts from "downloads.a.o" and
> this is a completely different issue.

There is no definitive rule about the path to the download page. There is no 
standard and with over 200 projects, 35 podlings, and lots of products change 
is difficult.

There is one definite requirement that can be checked. Whether the releases in 
svn are properly formed.

We do this in the Incubator with the Clutch Analysis. 
https://svn.apache.org/repos/asf/incubator/public/trunk/clutch2.py - L1132
Output found by clicking on a podling on https://incubator.apache.org/clutch/

We also have code in ASF Pelican - 
https://github.com/apache/infrastructure-pelican/blob/master/plugins/asfdata.py#L362
That was used to produce this example for Hadoop - 
https://template.staged.apache.org/downloads.html
Using this template - 
https://github.com/apache/template-site/blob/main/content/downloads.ezmd
Control is - https://github.com/apache/template-site/blob/main/asfdata.yaml#L110

To review projects you might find out where their download page is by reviewing 
the project DOAP file, but we’ve never been able to get all projects to 
maintain one.

Maybe it’s time to more carefully define what the ask is here.

> 
> On Mon, Nov 8, 2021 at 6:42 PM sebb  wrote:
>> 
>> On Mon, 8 Nov 2021 at 17:11, Jarek Potiuk  wrote:
>>> 
>>> Yeah. We will have different URLs especially as more artifacts are
>>> produced (airflow has 3 of those each with different download pages).
>>> 
>>> But this should be straightforward when you can add the url when
>>> registering the release (as Sebb proposed). According to the release
>>> process you should register each such "release type" separately and
>>> adding a URL there will do the job nicely. And initially it can be
>>> optional and once we figure out how to communicate it best and maybe
>>> try with some projects first we can make it obligatory at release
>>> registration. We could make it convenient - for example an easy way to
>>> copy a previously added URL from selected previous release or
>>> autocomplete from previous entries - making the process of entry as
>>> smooth (or even smoother) than it is today.
>> 
>> Or you could just ask for the URL if the release is not already on a
>> download page.
>> This would mean parsing the page of course, but could be used to
>> provide immediate feedback if the page is non-conforming.
>> 
>> Asking for the page might also nudge projects to move to generic pages
>> rather than one per release.
>> 
>> However this does not solve the issue of stale download pages: how do
>> you know when a release version is no longer supported?
>> 
>>> J.
>>> 
>>> 
>>> On Mon, Nov 8, 2021 at 4:19 PM sebb  wrote:
 
 On Mon, 8 Nov 2021 at 11:29, Hans Van Akelyen
  wrote:
> 
> Hi,
> 
> I think a certain level of uniformity would not be bad for this.
> Having a fixed path to the downloads for each project makes it simpler for
> the users to "shop" for Apache projects.
> 
> Having .a.o/download/ as a link that should point to the 
> resources
> makes it simpler for everyone.
 
 I agree, but expect a lot of complaints if you try to get this enforced.
 (and there's not much point unless it is enforced).
 
> Kind Regards,
> Hans
> 
> On Mon, 8 Nov 2021 at 12:19, Justin Mclean  
> wrote:
> 
>> Hi,
>> 
>>> However that assumes that you know where to find the download page(s).
>>> It also assumes that projects update their download page(s) to only
>>> show current releases.
>> 
>> It does indeed assume that, and a few other things. Currently policy
>> doesn't specify what the download page URL should be, but once known I
>> would guess it wouldn’t change often.
>> 
>> Kind Regards,
>> Justin
>> 
>> 
>> 
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
>> For additional commands, e-mail: dev-h...@community.apache.org
>> 
>> 
 
 -
 To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
 For additional commands, e-mail: dev-h...@community.apache.org
 
>>> 
>>> -
>>> To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
>>> For additional commands, e-mail: dev-h...@community.apache.org
>>> 
>> 
>> -
>> To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
>> For additional commands, e-mail: dev

Re: Proposal to add "release requirements" check

[Jarek Potiuk]

> However this does not solve the issue of stale download pages: how do
you know when a release version is no longer supported?

What do you mean?

Is there a requirement for that that I missed? I believe the
requirement is only about removing artifacts from "downloads.a.o" and
this is a completely different issue.

On Mon, Nov 8, 2021 at 6:42 PM sebb  wrote:
>
> On Mon, 8 Nov 2021 at 17:11, Jarek Potiuk  wrote:
> >
> > Yeah. We will have different URLs especially as more artifacts are
> > produced (airflow has 3 of those each with different download pages).
> >
> > But this should be straightforward when you can add the url when
> > registering the release (as Sebb proposed). According to the release
> > process you should register each such "release type" separately and
> > adding a URL there will do the job nicely. And initially it can be
> > optional and once we figure out how to communicate it best and maybe
> > try with some projects first we can make it obligatory at release
> > registration. We could make it convenient - for example an easy way to
> > copy a previously added URL from selected previous release or
> > autocomplete from previous entries - making the process of entry as
> > smooth (or even smoother) than it is today.
>
> Or you could just ask for the URL if the release is not already on a
> download page.
> This would mean parsing the page of course, but could be used to
> provide immediate feedback if the page is non-conforming.
>
> Asking for the page might also nudge projects to move to generic pages
> rather than one per release.
>
> However this does not solve the issue of stale download pages: how do
> you know when a release version is no longer supported?
>
> > J.
> >
> >
> > On Mon, Nov 8, 2021 at 4:19 PM sebb  wrote:
> > >
> > > On Mon, 8 Nov 2021 at 11:29, Hans Van Akelyen
> > >  wrote:
> > > >
> > > > Hi,
> > > >
> > > > I think a certain level of uniformity would not be bad for this.
> > > > Having a fixed path to the downloads for each project makes it simpler 
> > > > for
> > > > the users to "shop" for Apache projects.
> > > >
> > > > Having .a.o/download/ as a link that should point to the 
> > > > resources
> > > > makes it simpler for everyone.
> > >
> > > I agree, but expect a lot of complaints if you try to get this enforced.
> > > (and there's not much point unless it is enforced).
> > >
> > > > Kind Regards,
> > > > Hans
> > > >
> > > > On Mon, 8 Nov 2021 at 12:19, Justin Mclean  
> > > > wrote:
> > > >
> > > > > Hi,
> > > > >
> > > > > > However that assumes that you know where to find the download 
> > > > > > page(s).
> > > > > > It also assumes that projects update their download page(s) to only
> > > > > > show current releases.
> > > > >
> > > > > It does indeed assume that, and a few other things. Currently policy
> > > > > doesn't specify what the download page URL should be, but once known I
> > > > > would guess it wouldn’t change often.
> > > > >
> > > > > Kind Regards,
> > > > > Justin
> > > > >
> > > > >
> > > > >
> > > > > -
> > > > > To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
> > > > > For additional commands, e-mail: dev-h...@community.apache.org
> > > > >
> > > > >
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
> > > For additional commands, e-mail: dev-h...@community.apache.org
> > >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
> > For additional commands, e-mail: dev-h...@community.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
> For additional commands, e-mail: dev-h...@community.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
For additional commands, e-mail: dev-h...@community.apache.org

Re: Proposal to add "release requirements" check

[sebb]

On Mon, 8 Nov 2021 at 17:11, Jarek Potiuk  wrote:
>
> Yeah. We will have different URLs especially as more artifacts are
> produced (airflow has 3 of those each with different download pages).
>
> But this should be straightforward when you can add the url when
> registering the release (as Sebb proposed). According to the release
> process you should register each such "release type" separately and
> adding a URL there will do the job nicely. And initially it can be
> optional and once we figure out how to communicate it best and maybe
> try with some projects first we can make it obligatory at release
> registration. We could make it convenient - for example an easy way to
> copy a previously added URL from selected previous release or
> autocomplete from previous entries - making the process of entry as
> smooth (or even smoother) than it is today.

Or you could just ask for the URL if the release is not already on a
download page.
This would mean parsing the page of course, but could be used to
provide immediate feedback if the page is non-conforming.

Asking for the page might also nudge projects to move to generic pages
rather than one per release.

However this does not solve the issue of stale download pages: how do
you know when a release version is no longer supported?

> J.
>
>
> On Mon, Nov 8, 2021 at 4:19 PM sebb  wrote:
> >
> > On Mon, 8 Nov 2021 at 11:29, Hans Van Akelyen
> >  wrote:
> > >
> > > Hi,
> > >
> > > I think a certain level of uniformity would not be bad for this.
> > > Having a fixed path to the downloads for each project makes it simpler for
> > > the users to "shop" for Apache projects.
> > >
> > > Having .a.o/download/ as a link that should point to the 
> > > resources
> > > makes it simpler for everyone.
> >
> > I agree, but expect a lot of complaints if you try to get this enforced.
> > (and there's not much point unless it is enforced).
> >
> > > Kind Regards,
> > > Hans
> > >
> > > On Mon, 8 Nov 2021 at 12:19, Justin Mclean  
> > > wrote:
> > >
> > > > Hi,
> > > >
> > > > > However that assumes that you know where to find the download page(s).
> > > > > It also assumes that projects update their download page(s) to only
> > > > > show current releases.
> > > >
> > > > It does indeed assume that, and a few other things. Currently policy
> > > > doesn't specify what the download page URL should be, but once known I
> > > > would guess it wouldn’t change often.
> > > >
> > > > Kind Regards,
> > > > Justin
> > > >
> > > >
> > > >
> > > > -
> > > > To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
> > > > For additional commands, e-mail: dev-h...@community.apache.org
> > > >
> > > >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
> > For additional commands, e-mail: dev-h...@community.apache.org
> >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
> For additional commands, e-mail: dev-h...@community.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
For additional commands, e-mail: dev-h...@community.apache.org

Re: Proposal to add "release requirements" check

[Jarek Potiuk]

Yeah. We will have different URLs especially as more artifacts are
produced (airflow has 3 of those each with different download pages).

But this should be straightforward when you can add the url when
registering the release (as Sebb proposed). According to the release
process you should register each such "release type" separately and
adding a URL there will do the job nicely. And initially it can be
optional and once we figure out how to communicate it best and maybe
try with some projects first we can make it obligatory at release
registration. We could make it convenient - for example an easy way to
copy a previously added URL from selected previous release or
autocomplete from previous entries - making the process of entry as
smooth (or even smoother) than it is today.

J.


On Mon, Nov 8, 2021 at 4:19 PM sebb  wrote:
>
> On Mon, 8 Nov 2021 at 11:29, Hans Van Akelyen
>  wrote:
> >
> > Hi,
> >
> > I think a certain level of uniformity would not be bad for this.
> > Having a fixed path to the downloads for each project makes it simpler for
> > the users to "shop" for Apache projects.
> >
> > Having .a.o/download/ as a link that should point to the resources
> > makes it simpler for everyone.
>
> I agree, but expect a lot of complaints if you try to get this enforced.
> (and there's not much point unless it is enforced).
>
> > Kind Regards,
> > Hans
> >
> > On Mon, 8 Nov 2021 at 12:19, Justin Mclean  wrote:
> >
> > > Hi,
> > >
> > > > However that assumes that you know where to find the download page(s).
> > > > It also assumes that projects update their download page(s) to only
> > > > show current releases.
> > >
> > > It does indeed assume that, and a few other things. Currently policy
> > > doesn't specify what the download page URL should be, but once known I
> > > would guess it wouldn’t change often.
> > >
> > > Kind Regards,
> > > Justin
> > >
> > >
> > >
> > > -
> > > To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
> > > For additional commands, e-mail: dev-h...@community.apache.org
> > >
> > >
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
> For additional commands, e-mail: dev-h...@community.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
For additional commands, e-mail: dev-h...@community.apache.org

Re: Proposal to add "release requirements" check

[sebb]

On Mon, 8 Nov 2021 at 11:29, Hans Van Akelyen
 wrote:
>
> Hi,
>
> I think a certain level of uniformity would not be bad for this.
> Having a fixed path to the downloads for each project makes it simpler for
> the users to "shop" for Apache projects.
>
> Having .a.o/download/ as a link that should point to the resources
> makes it simpler for everyone.

I agree, but expect a lot of complaints if you try to get this enforced.
(and there's not much point unless it is enforced).

> Kind Regards,
> Hans
>
> On Mon, 8 Nov 2021 at 12:19, Justin Mclean  wrote:
>
> > Hi,
> >
> > > However that assumes that you know where to find the download page(s).
> > > It also assumes that projects update their download page(s) to only
> > > show current releases.
> >
> > It does indeed assume that, and a few other things. Currently policy
> > doesn't specify what the download page URL should be, but once known I
> > would guess it wouldn’t change often.
> >
> > Kind Regards,
> > Justin
> >
> >
> >
> > -
> > To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
> > For additional commands, e-mail: dev-h...@community.apache.org
> >
> >

-
To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
For additional commands, e-mail: dev-h...@community.apache.org

Re: Proposal to add "release requirements" check

[sebb]

On Mon, 8 Nov 2021 at 11:19, Justin Mclean  wrote:
>
> Hi,
>
> > However that assumes that you know where to find the download page(s).
> > It also assumes that projects update their download page(s) to only
> > show current releases.
>
> It does indeed assume that, and a few other things. Currently policy doesn't 
> specify what the download page URL should be, but once known I would guess it 
> wouldn’t change often.

If only.

Several projects use a different URL for each release. For example
https://daffodil.apache.org/releases/
https://db.apache.org/derby/derby_downloads.html
https://guacamole.apache.org/releases/

Some projects use different pages for source and binaries, e.g.
https://ant.apache.org/bindownload.cgi

> Kind Regards,
> Justin
>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
> For additional commands, e-mail: dev-h...@community.apache.org
>

-
To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
For additional commands, e-mail: dev-h...@community.apache.org

Re: Proposal to add "release requirements" check

[Hans Van Akelyen]

Hi,

I think a certain level of uniformity would not be bad for this.
Having a fixed path to the downloads for each project makes it simpler for
the users to "shop" for Apache projects.

Having .a.o/download/ as a link that should point to the resources
makes it simpler for everyone.

Kind Regards,
Hans

On Mon, 8 Nov 2021 at 12:19, Justin Mclean  wrote:

> Hi,
>
> > However that assumes that you know where to find the download page(s).
> > It also assumes that projects update their download page(s) to only
> > show current releases.
>
> It does indeed assume that, and a few other things. Currently policy
> doesn't specify what the download page URL should be, but once known I
> would guess it wouldn’t change often.
>
> Kind Regards,
> Justin
>
>
>
> -
> To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
> For additional commands, e-mail: dev-h...@community.apache.org
>
>

Re: Proposal to add "release requirements" check

[Justin Mclean]

Hi,

> However that assumes that you know where to find the download page(s).
> It also assumes that projects update their download page(s) to only
> show current releases.

It does indeed assume that, and a few other things. Currently policy doesn't 
specify what the download page URL should be, but once known I would guess it 
wouldn’t change often.

Kind Regards,
Justin



-
To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
For additional commands, e-mail: dev-h...@community.apache.org

Re: Proposal to add "release requirements" check

[sebb]

On Mon, 8 Nov 2021 at 00:49, Justin Mclean  wrote:
>
> Hi,
>
> > Purging old releases needs at least two data sources:
> > - the list of files on downloads.apache.org  
> > (or the dist.a.o source).
> > This is trivial to obtain.
> > - the list of current versions for each release. This is the most
> > difficult part.
>
> The check dist area script I wrote attempts to do this. [1] It get the list 
> of released version from the download page and matches them up with what is 
> in the dist area.

However that assumes that you know where to find the download page(s).
It also assumes that projects update their download page(s) to only
show current releases.

Once you have that information, the rest is relatively easy (unless
the download page uses Javascript)

> To be complete you also need to look in the archives (which I’ve not done 
> yet).  It does have some of the issues you mention but it’s a start and 
> worked for abut 90% of Incubating projects. having something that works for 
> project project is probably a good start and it can be refined from there.
>
> Kind Regards,
> Justin
>
> 1. https://github.com/justinmclean/ReleaseChecker/blob/main/checkdistarea.py

-
To unsubscribe, e-mail: dev-unsubscr...@community.apache.org
For additional commands, e-mail: dev-h...@community.apache.org