[
https://issues.apache.org/jira/browse/SLING-7262?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16346554#comment-16346554
]
Bertrand Delacretaz edited comment on SLING-7262 at 1/31/18 10:26 AM:
----------------------------------------------------------------------
Trying to summarize the various initiatives going on, as well as our
requirements, please correct/complete as needed.
*Inputs*
* Either the GitHub API to list all Sling repositories as currently used at
[https://github.com/apache/sling-aggregator]
* Or the Gitbox OPML at [https://gitbox.apache.org/repos/asf?a=opml] as
currently used by [https://github.com/oliverlietz/apache-sling-aggregator]
*Desired Outputs*
* 1) A manifest for the Google Repo tool, which requires just the repository
names and their paths.
* 2) Our Jenkins jobs, currently generated manually by running scripts from
https://github.com/apache/sling-tooling-jenkins
* 3) A list suitable for updating [http://sling.apache.org/repolist.html] (or
a similar canonical Web page)
* 4) A Maven reactor pom that allows for building everything or sets of
modules, probably based on Maven profiles
For 1) I haven't checked if the Repo manifest format officially allows us to
add extra information, currently we have an additional "groups" attribute, if
that's supported we might add "tags", "badges" and "descriptions" as needed.
For 3) the repositories need to be grouped by functional areas (already
partially done) and should also have tags or badges for contrib/core/legacy and
similar classifications. It would be nice to have a brief description of each
repository as well in that list, maybe the first few lines of their README
(excluding "belongs to Apache Sling" of course)
*Environment*
The generation of these files needs to run as a Jenkins build so that they can
be regenerated automatically.
Commiting the output of Jenkins jobs to Git is easy if the target is the
{{asf-site}} branch, for that we just need to tie the jobs to nodes having the
{{git-websites}} label. Committing the output to the {{asf-site}} branch of the
[https://github.com/apache/sling-site] repository seems to be a good solution
to me and other scripts can pick it up there at a stable URL, even if the raw
result is not visible on the website.
*Here's what we currently have "in production"*
* [https://github.com/apache/sling-aggregator] generates a Repo manifest,
needs to be updated manually by running a script and pushing the result when
new Git repos are created
* [http://sling.apache.org/repolist.html] is generated from that Repo
manifest. It's missing tags/badges and descriptions
* Our Jenkins jobs are generated from
[https://github.com/apache/sling-tooling-jenkins] and optionally use
information from a {{.sling-module.xml}} file described at
[https://cwiki.apache.org/confluence/display/SLING/Sling+module+descriptor]
* I don't think we have an official Maven aggregator pom at this point
* [~olli] has been working on
[https://github.com/oliverlietz/apache-sling-aggregator] with useful things in
various branches of that repository, which generate a Repo manifest, a Maven
pom and a GitHub page.
See
[https://github.com/apache/sling-org-apache-sling-launchpad-contrib-testing]
for an example of a sling:contrib badge that can easily be parsed from the
README and is displayed there, as opposed to {{.sling-module.xml}} which has no
impact on the README page on GitHub:
{code:java}
curl -s
https://raw.githubusercontent.com/apache/sling-org-apache-sling-launchpad-contrib-testing/master/README.md
| grep sling:badge
{code}
We should consolidate all that and ideally generate all the outputs from a
single master list that's automatically updated at regular intervals.
was (Author: bdelacretaz):
Trying to summarize the various initiatives going on, as well as our
requirements, please correct/complete as needed.
*Inputs*
* Either the GitHub API to list all Sling repositories as currently used at
https://github.com/apache/sling-aggregator
* Or the Gitbox OPML at https://gitbox.apache.org/repos/asf?a=opml as currently
used by https://github.com/oliverlietz/apache-sling-aggregator
*Desired Outputs*
* 1) A manifest for the Google Repo tool, which requires just the repository
names and their paths.
* 2) A list suitable for updating http://sling.apache.org/repolist.html (or a
similar canonical Web page)
* 3) A Maven reactor pom that allows for building everything or sets of
modules, probably based on Maven profiles
For 1) I haven't checked if the Repo manifest format officially allows us to
add extra information, currently we have an additional "groups" attribute, if
that's supported we might add "tags", "badges" and "descriptions" as needed.
For 2) the repositories need to be grouped by functional areas (already
partially done) and should also have tags or badges for contrib/core/legacy and
similar classifications. It would be nice to have a brief description of each
repository as well in that list, maybe the first few lines of their README
(excluding "belongs to Apache Sling" of course)
*Environment*
The generation of these files needs to run as a Jenkins build so that they can
be regenerated automatically.
Commiting the output of Jenkins jobs to Git is easy if the target is the
{{asf-site}} branch, for that we just need to tie the jobs to nodes having the
{{git-websites}} label. Committing the output to the {{asf-site}} branch of the
https://github.com/apache/sling-site repository seems to be a good solution to
me and other scripts can pick it up there at a stable URL, even if the raw
result is not visible on the website.
*Here's what we currently have "in production"*
* https://github.com/apache/sling-aggregator generates a Repo manifest, needs
to be updated manually by running a script and pushing the result when new Git
repos are created
* http://sling.apache.org/repolist.html is generated from that Repo manifest.
It's missing tags/badges and descriptions
* Our Jenkins jobs are generated from
https://github.com/apache/sling-tooling-jenkins and optionally use information
from a {{.sling-module.xml}} file described at
https://cwiki.apache.org/confluence/display/SLING/Sling+module+descriptor
* I don't think we have an official Maven aggregator pom at this point
* [~olli] has been working on
https://github.com/oliverlietz/apache-sling-aggregator with useful things in
various branches of that repository, which generate a Repo manifest, a Maven
pom and a GitHub page.
See https://github.com/apache/sling-org-apache-sling-launchpad-contrib-testing
for an example of a sling:contrib badge that can easily be parsed from the
README and is displayed there, as opposed to {{.sling-module.xml}} which has no
impact on the README page on GitHub:
{code}
curl -s
https://raw.githubusercontent.com/apache/sling-org-apache-sling-launchpad-contrib-testing/master/README.md
| grep sling:badge
{code}
We should consolidate all that and ideally generate all the outputs from a
single master list that's automatically updated at regular intervals.
> Generate reusable lists of Git repositories for machine consumption
> -------------------------------------------------------------------
>
> Key: SLING-7262
> URL: https://issues.apache.org/jira/browse/SLING-7262
> Project: Sling
> Issue Type: Task
> Components: Build and Source Control
> Reporter: Robert Munteanu
> Priority: Major
>
> We currently have two 'clients' that query the github repos and the
> .sling-module.xml descriptors:
> * the jenkins job creation DSL script
> * the repo manifest creation script
> We've also discussed in SLING-7161 the possibility of listing all
> repositories on the website.
> To minimize the effort of talking to github in each location and reduce the
> number of calls to Github ( for rate limiting purposes ) we should generate a
> JSON/XML file that contains all information about the git repos. This file is
> in turns read by various "clients" that need the information.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
