Re: the new http://projects.apache.org/ is live
Hervé BOUTEMY wrote: As the title says, projects is now projects-new: http://projects.apache.org Nice! Please report if there is any issue There's an old typo I spotted long ago; I would have fixed it myself, but I got lost when trying to locate the site sources. https://projects.apache.org/projects.html in the left block you have "By Progamming [sic] language". Regards, Andrea.
the new http://projects.apache.org/ is live
Hi, As the title says, projects is now projects-new: http://projects.apache.org The old projects is available at https://projects-old.apache.org/ (there is currently a bug in config for http://projects-old.apache.org/ that should be worked on tomorrow, tracked in https://issues.apache.org/jira/browse/INFRA-9453 ) Please report if there is any issue Regards, Hervé
Re: [Projects New] Why are output files stored in SVN?
On 12 July 2015 at 19:54, Hervé BOUTEMY wrote: > true, everything can be regenerated > > but there are IMHO multiple ideas behind storing the full site in svn: > - not serve http from projects-vm but from classical resilient httpd (just let > projects-vm just run the cron jobs) If projects-vm fails, then the resilient httpd will display stale data. I cannot see the point of that. > - have a centralized history of whole data (be it source rdf in a central > place or derived json files) Again, having derived data in SVN is not a good idea. Sorry, but I don't see a good reason to store this data in SVN. > Regards, > > Hervé > > Le dimanche 12 juillet 2015 16:58:29 sebb a écrit : >> Why does projects new need to store changes to the files it generates in >> SVN? >> >> AFAICT, all (or nearly all) the files are generated from data that is >> already in SVN. >> >> There are perhaps a few files that contain historic data that is not >> otherwise available. >> These could be stored as template files in SVN in case the working >> files need to be regenerated. >> But otherwise, I don't see the point in storing changes to derived data in >> SVN. >> >> We should be striving to reduce data duplication, not increase it... >
Re: svn commit: r1690454 - /comdev/projects.apache.org/scripts/README.txt
On 12 July 2015 at 19:49, Hervé BOUTEMY wrote:
> Le dimanche 12 juillet 2015 15:41:13 sebb a écrit :
>> On 12 July 2015 at 14:56, Hervé BOUTEMY wrote:
>> > Le dimanche 12 juillet 2015 14:14:49 sebb a écrit :
>> >> On 12 July 2015 at 13:58, wrote:
>> >> > Author: hboutemy
>> >> > Date: Sun Jul 12 12:58:09 2015
>> >> > New Revision: 1690454
>> >> >
>> >> > URL: http://svn.apache.org/r1690454
>> >> > Log:
>> >> > updated explanation on import scripts vs update scripts
>> >> >
>> >> > Modified:
>> >> > comdev/projects.apache.org/scripts/README.txt
>> >> >
>> >> > Modified: comdev/projects.apache.org/scripts/README.txt
>> >> > URL:
>> >> > http://svn.apache.org/viewvc/comdev/projects.apache.org/scripts/README.
>> >> > tx
>> >> > t?rev=1690454&r1=1690453&r2=1690454&view=diff
>> >> > ===
>> >> > ==
>> >> > = --- comdev/projects.apache.org/scripts/README.txt (original)
>> >> > +++ comdev/projects.apache.org/scripts/README.txt Sun Jul 12 12:58:09
>> >> > 2015
>> >> >
>> >> > @@ -43,3 +43,8 @@ various sources:
>> >> >in: data/projects.xml + projects' DOAP files
>> >> >out: site/json/projects/*.json + site/json/foundation/projects.json
>> >> >
>> >> >+ site/doap/{committeeId}/{project}.rdf
>> >> >
>> >> > +
>> >> > +NOTICE: what prevents import scripts to be added to cron?
>> >> > +1. parse committees.py requires committee-info.txt, which is not
>> >> > available on project-vm (require authentication)
>> >>
>> >> Agreed this means the script must be run by a process with suitable karma
>> >>
>> >> > +2. both scripts not only update files but sometimes need to add new
>> >> > files
>> >> > (new committees or new projects) or move + (projects going to Attic
>> >> > or
>> >> > retired committees)
>> >>
>> >> Why should it make a difference if files need to be added?
>> >> There is already a cronjob that detects new files and runs 'svn add' for
>> >> them
>> >
>> > the real problem is when moving a file is required: if we blindly add new
>> > files for files that should be moved, not only do we get too much files,
>> > but when removing the old file we loose history (that could have been
>> > useful)
>> In which case, why not run 'svn add' and 'svn mv' as part of the
>> script where necessary?
> we need to detect when a project moves to Attic
That must already be known to the script else how does the person
running the script know what to do?
> then integrate svn commands to the python script
That should be fairly easyl.
> feasible, but adds complexity...
>
>>
>> This would be safer than the cronjob which assumes that all new files
>> are to be added to SVN.
>
Re: [Projects New] Why are output files stored in SVN?
true, everything can be regenerated but there are IMHO multiple ideas behind storing the full site in svn: - not serve http from projects-vm but from classical resilient httpd (just let projects-vm just run the cron jobs) - have a centralized history of whole data (be it source rdf in a central place or derived json files) Regards, Hervé Le dimanche 12 juillet 2015 16:58:29 sebb a écrit : > Why does projects new need to store changes to the files it generates in > SVN? > > AFAICT, all (or nearly all) the files are generated from data that is > already in SVN. > > There are perhaps a few files that contain historic data that is not > otherwise available. > These could be stored as template files in SVN in case the working > files need to be regenerated. > But otherwise, I don't see the point in storing changes to derived data in > SVN. > > We should be striving to reduce data duplication, not increase it...
Re: svn commit: r1690454 - /comdev/projects.apache.org/scripts/README.txt
Le dimanche 12 juillet 2015 15:41:13 sebb a écrit :
> On 12 July 2015 at 14:56, Hervé BOUTEMY wrote:
> > Le dimanche 12 juillet 2015 14:14:49 sebb a écrit :
> >> On 12 July 2015 at 13:58, wrote:
> >> > Author: hboutemy
> >> > Date: Sun Jul 12 12:58:09 2015
> >> > New Revision: 1690454
> >> >
> >> > URL: http://svn.apache.org/r1690454
> >> > Log:
> >> > updated explanation on import scripts vs update scripts
> >> >
> >> > Modified:
> >> > comdev/projects.apache.org/scripts/README.txt
> >> >
> >> > Modified: comdev/projects.apache.org/scripts/README.txt
> >> > URL:
> >> > http://svn.apache.org/viewvc/comdev/projects.apache.org/scripts/README.
> >> > tx
> >> > t?rev=1690454&r1=1690453&r2=1690454&view=diff
> >> > ===
> >> > ==
> >> > = --- comdev/projects.apache.org/scripts/README.txt (original)
> >> > +++ comdev/projects.apache.org/scripts/README.txt Sun Jul 12 12:58:09
> >> > 2015
> >> >
> >> > @@ -43,3 +43,8 @@ various sources:
> >> >in: data/projects.xml + projects' DOAP files
> >> >out: site/json/projects/*.json + site/json/foundation/projects.json
> >> >
> >> >+ site/doap/{committeeId}/{project}.rdf
> >> >
> >> > +
> >> > +NOTICE: what prevents import scripts to be added to cron?
> >> > +1. parse committees.py requires committee-info.txt, which is not
> >> > available on project-vm (require authentication)
> >>
> >> Agreed this means the script must be run by a process with suitable karma
> >>
> >> > +2. both scripts not only update files but sometimes need to add new
> >> > files
> >> > (new committees or new projects) or move + (projects going to Attic
> >> > or
> >> > retired committees)
> >>
> >> Why should it make a difference if files need to be added?
> >> There is already a cronjob that detects new files and runs 'svn add' for
> >> them
> >
> > the real problem is when moving a file is required: if we blindly add new
> > files for files that should be moved, not only do we get too much files,
> > but when removing the old file we loose history (that could have been
> > useful)
> In which case, why not run 'svn add' and 'svn mv' as part of the
> script where necessary?
we need to detect when a project moves to Attic
then integrate svn commands to the python script
feasible, but adds complexity...
>
> This would be safer than the cronjob which assumes that all new files
> are to be added to SVN.
[Projects New] Why are output files stored in SVN?
Why does projects new need to store changes to the files it generates in SVN? AFAICT, all (or nearly all) the files are generated from data that is already in SVN. There are perhaps a few files that contain historic data that is not otherwise available. These could be stored as template files in SVN in case the working files need to be regenerated. But otherwise, I don't see the point in storing changes to derived data in SVN. We should be striving to reduce data duplication, not increase it...
Re: svn commit: r1690454 - /comdev/projects.apache.org/scripts/README.txt
On 12 July 2015 at 14:56, Hervé BOUTEMY wrote:
> Le dimanche 12 juillet 2015 14:14:49 sebb a écrit :
>> On 12 July 2015 at 13:58, wrote:
>> > Author: hboutemy
>> > Date: Sun Jul 12 12:58:09 2015
>> > New Revision: 1690454
>> >
>> > URL: http://svn.apache.org/r1690454
>> > Log:
>> > updated explanation on import scripts vs update scripts
>> >
>> > Modified:
>> > comdev/projects.apache.org/scripts/README.txt
>> >
>> > Modified: comdev/projects.apache.org/scripts/README.txt
>> > URL:
>> > http://svn.apache.org/viewvc/comdev/projects.apache.org/scripts/README.tx
>> > t?rev=1690454&r1=1690453&r2=1690454&view=diff
>> > =
>> > = --- comdev/projects.apache.org/scripts/README.txt (original)
>> > +++ comdev/projects.apache.org/scripts/README.txt Sun Jul 12 12:58:09 2015
>> >
>> > @@ -43,3 +43,8 @@ various sources:
>> >in: data/projects.xml + projects' DOAP files
>> >out: site/json/projects/*.json + site/json/foundation/projects.json
>> >
>> >+ site/doap/{committeeId}/{project}.rdf
>> >
>> > +
>> > +NOTICE: what prevents import scripts to be added to cron?
>> > +1. parse committees.py requires committee-info.txt, which is not
>> > available on project-vm (require authentication)
>> Agreed this means the script must be run by a process with suitable karma
>>
>> > +2. both scripts not only update files but sometimes need to add new files
>> > (new committees or new projects) or move + (projects going to Attic or
>> > retired committees)
>>
>> Why should it make a difference if files need to be added?
>> There is already a cronjob that detects new files and runs 'svn add' for
>> them
> the real problem is when moving a file is required: if we blindly add new
> files
> for files that should be moved, not only do we get too much files, but when
> removing the old file we loose history (that could have been useful)
In which case, why not run 'svn add' and 'svn mv' as part of the
script where necessary?
This would be safer than the cronjob which assumes that all new files
are to be added to SVN.
Re: svn commit: r1690454 - /comdev/projects.apache.org/scripts/README.txt
Le dimanche 12 juillet 2015 14:14:49 sebb a écrit :
> On 12 July 2015 at 13:58, wrote:
> > Author: hboutemy
> > Date: Sun Jul 12 12:58:09 2015
> > New Revision: 1690454
> >
> > URL: http://svn.apache.org/r1690454
> > Log:
> > updated explanation on import scripts vs update scripts
> >
> > Modified:
> > comdev/projects.apache.org/scripts/README.txt
> >
> > Modified: comdev/projects.apache.org/scripts/README.txt
> > URL:
> > http://svn.apache.org/viewvc/comdev/projects.apache.org/scripts/README.tx
> > t?rev=1690454&r1=1690453&r2=1690454&view=diff
> > =
> > = --- comdev/projects.apache.org/scripts/README.txt (original)
> > +++ comdev/projects.apache.org/scripts/README.txt Sun Jul 12 12:58:09 2015
> >
> > @@ -43,3 +43,8 @@ various sources:
> >in: data/projects.xml + projects' DOAP files
> >out: site/json/projects/*.json + site/json/foundation/projects.json
> >
> >+ site/doap/{committeeId}/{project}.rdf
> >
> > +
> > +NOTICE: what prevents import scripts to be added to cron?
> > +1. parse committees.py requires committee-info.txt, which is not
> > available on project-vm (require authentication)
> Agreed this means the script must be run by a process with suitable karma
>
> > +2. both scripts not only update files but sometimes need to add new files
> > (new committees or new projects) or move + (projects going to Attic or
> > retired committees)
>
> Why should it make a difference if files need to be added?
> There is already a cronjob that detects new files and runs 'svn add' for
> them
the real problem is when moving a file is required: if we blindly add new files
for files that should be moved, not only do we get too much files, but when
removing the old file we loose history (that could have been useful)
Regards,
Hervé
Re: svn commit: r1690454 - /comdev/projects.apache.org/scripts/README.txt
On 12 July 2015 at 13:58, wrote:
> Author: hboutemy
> Date: Sun Jul 12 12:58:09 2015
> New Revision: 1690454
>
> URL: http://svn.apache.org/r1690454
> Log:
> updated explanation on import scripts vs update scripts
>
> Modified:
> comdev/projects.apache.org/scripts/README.txt
>
> Modified: comdev/projects.apache.org/scripts/README.txt
> URL:
> http://svn.apache.org/viewvc/comdev/projects.apache.org/scripts/README.txt?rev=1690454&r1=1690453&r2=1690454&view=diff
> ==
> --- comdev/projects.apache.org/scripts/README.txt (original)
> +++ comdev/projects.apache.org/scripts/README.txt Sun Jul 12 12:58:09 2015
> @@ -43,3 +43,8 @@ various sources:
>in: data/projects.xml + projects' DOAP files
>out: site/json/projects/*.json + site/json/foundation/projects.json
>+ site/doap/{committeeId}/{project}.rdf
> +
> +NOTICE: what prevents import scripts to be added to cron?
> +1. parse committees.py requires committee-info.txt, which is not available
> on project-vm (require authentication)
Agreed this means the script must be run by a process with suitable karma
> +2. both scripts not only update files but sometimes need to add new files
> (new committees or new projects) or move
> + (projects going to Attic or retired committees)
Why should it make a difference if files need to be added?
There is already a cronjob that detects new files and runs 'svn add' for them
>
