Re: [IMPORTANT] - Migration to ci-builds.a.o
Pushing to 'asf-site' and 'asf-staging*' branches will be allowed automatically when running on the git-websites labeled node. I also just opened a pull request for site building to put the job configuration in version control instead of being part of the job: https://github.com/apache/jena-site/pull/17 IMO it's a bit nicer to have the job config in git. On 2020/07/24 12:13:21, Andy Seaborne wrote: > I've forgotten the details of how security for site update. > > Is it that is it runs on the right node (label "git-websites"), it just > works? i.e. no credential files needed? > > > > The other builds: > > They have started adding worker nodes and I've started copying jobs > over. It seems slower than the main server at the moment and there is a > massive build queue but they are all waiting on something so Jena jobs > are starting within a few seconds. > > I'm trying to reduce the number of things that are kept (artifacts) and > setting the various JDK jobs to poll only hourly. All this this can be > changed, it's just a chance to review setup and play nice. > > Jobs on the old server are still there. > > Andy > > > > On 17/07/2020 21:58, Andy Seaborne wrote: > > Thanks! > > > > Fingers-crossed - copying the job across will work. It's not like the > > site build is unique for Jena. > > > > Good timing WRT the recent release as well. > > > > Andy > > > > On 17/07/2020 21:31, Roy Lenferink wrote: > >> If any help is needed with the re-creation of the jena-site job let me > >> know > >> ! > >> > >> Roy > >> > >> Op vr 17 jul. 2020 om 22:21 schreef Andy Seaborne > >> > >>> Jenkins is on the move. > >>> > >>> I've asked for a Folder on the new system. > >>> > >>> I am so glad we have a simple build setup! > >>> > >>> Andy > >>> > >>> > >>> On 16/07/2020 17:33, Gavin McDonald wrote: > >>>> Hi All, > >>>> > >>>> This NOTICE is for everyone on builds.apache.org. We are migrating to a > >>> new > >>>> Cloudbees based Client Master called https://ci-builds.apache.org. The > >>>> migrations of all jobs needs to be done before the switch off date of > >>> 15th > >>>> August 2020, so you have a maximum of 4 weeks. > >>>> > >>>> There is no tool or automated way of migrating your jobs, the > >>>> differences in the platforms, the plugins and the setup makes it > >>> impossible > >>>> to do in a safe way. So, you all need to start creating new jobs on > >>>> ci-infra.a.o and then , when you are happy, turn off your old builds on > >>>> builds.a.o. > >>>> > >>>> There are currently 4 agents over there ready to take jobs, plus a > >>> floating > >>>> agent which is shared amongst many masters (more to come). I will > >>>> migrate > >>>> away 2 more agents from builds.a.o to ci-builds.a.o every few days, and > >>>> will keep an eye of load across both and adjust accordingly. > >>>> > >>>> If needed, create a ticket on INFRA jira for any issues that crop > >>>> up, or > >>>> email here on builds@a.o. there may be one or two plugins we need to > >>>> install/tweak etc. > >>>> > >>>> We will be not using 'Views' at the top level, but rather we will take > >>>> advantage of 'Folders Plus' - each project will get its own Folder and > >>> have > >>>> authorisation access to create/edit jobs etc within that folder. I will > >>>> create these folders as you ask for them to start with. This setup > >>>> allows > >>>> for credentials isolation amongst other benefits, including but not > >>> limited > >>>> to exclusive agents (Controlled Agents) for your own use , should you > >>> have > >>>> any project targeted donations of agents. > >>>> > >>>> As with other aspects of the ASF, projects can choose to just enable > >>>> all > >>>> committers access to their folder, just ask. > >>>> > >>>> We will re-use builds.apache.org as a CNAME to ci-builds.a.o but will > >>> not > >>>> be setting up any forwarding rules or anything like that. > >>>> > >>>> So, please, get started *now *on this so you can be sure we are all > >>>> completed before the final cutoff date of 15th August 2020. > >>>> > >>>> Any questions - I expect a few (dozen :) ) - ask away and/or file INFRA > >>>> tickets. > >>>> > >>>> Hadoop and related projects have their own migration path to follow, > >>>> same > >>>> cut off date, Cassandra, Beam, CouchDB have already migrated and are > >>> doing > >>>> very well in their new homes. > >>>> > >>>> Lets get going ... > >>>> > >>> > >> >
Re: [IMPORTANT] - Migration to ci-builds.a.o
If any help is needed with the re-creation of the jena-site job let me know ! Roy Op vr 17 jul. 2020 om 22:21 schreef Andy Seaborne > Jenkins is on the move. > > I've asked for a Folder on the new system. > > I am so glad we have a simple build setup! > > Andy > > > On 16/07/2020 17:33, Gavin McDonald wrote: > > Hi All, > > > > This NOTICE is for everyone on builds.apache.org. We are migrating to a > new > > Cloudbees based Client Master called https://ci-builds.apache.org. The > > migrations of all jobs needs to be done before the switch off date of > 15th > > August 2020, so you have a maximum of 4 weeks. > > > > There is no tool or automated way of migrating your jobs, the > > differences in the platforms, the plugins and the setup makes it > impossible > > to do in a safe way. So, you all need to start creating new jobs on > > ci-infra.a.o and then , when you are happy, turn off your old builds on > > builds.a.o. > > > > There are currently 4 agents over there ready to take jobs, plus a > floating > > agent which is shared amongst many masters (more to come). I will migrate > > away 2 more agents from builds.a.o to ci-builds.a.o every few days, and > > will keep an eye of load across both and adjust accordingly. > > > > If needed, create a ticket on INFRA jira for any issues that crop up, or > > email here on builds@a.o. there may be one or two plugins we need to > > install/tweak etc. > > > > We will be not using 'Views' at the top level, but rather we will take > > advantage of 'Folders Plus' - each project will get its own Folder and > have > > authorisation access to create/edit jobs etc within that folder. I will > > create these folders as you ask for them to start with. This setup allows > > for credentials isolation amongst other benefits, including but not > limited > > to exclusive agents (Controlled Agents) for your own use , should you > have > > any project targeted donations of agents. > > > > As with other aspects of the ASF, projects can choose to just enable all > > committers access to their folder, just ask. > > > > We will re-use builds.apache.org as a CNAME to ci-builds.a.o but will > not > > be setting up any forwarding rules or anything like that. > > > > So, please, get started *now *on this so you can be sure we are all > > completed before the final cutoff date of 15th August 2020. > > > > Any questions - I expect a few (dozen :) ) - ask away and/or file INFRA > > tickets. > > > > Hadoop and related projects have their own migration path to follow, same > > cut off date, Cassandra, Beam, CouchDB have already migrated and are > doing > > very well in their new homes. > > > > Lets get going ... > > >
Re: website move
On 2020/03/08 10:29:17, Andy Seaborne wrote: > > > On 08/03/2020 09:40, Roy Lenferink wrote: > > Hi Andy, > > > > Good to be pro-active on this! However, I had yet to send reply to this > > list that the actual move was > > already done :) See [1] for the actual change. > > :-) > > > To verify: on jena.a.o the 'Edit this page' button is visible (meaning the > > site is served from Git), and > > this correctly redirects the user to the jena-site repository on GitHub. > > Yes - "edit" is there. > > https://jena.apache.org/documentation/javadoc/ was 404. > > Fixed by change the layout but (for future reference), is it possible to > have content in :master:/documentation/javadoc/_index.md and then not > have it lost when branch javadoc is gitpubsub'ed. i.e does the > additional javadoc branch simply copy in more HTML pages or replace the > directory? > > If it is copy-in, I'll change the change to preserve bookmarks. Only a single source of truth is preserved. It is not possible to retain both. I discussed this with infra (for MINA) on Slack here: https://the-asf.slack.com/archives/CBX4TSBQ8/p1582565890030900 However something I am investigating and that possibly can be useful for Jena is to have a virtual domain 'docs.jena.apache.org' in place which will be served using e.g. a branch called 'jena-docs' (instead of javadoc). DNS won't be in place so people cannot visit docs.jena.apache.org. Using RewriteRules (.htaccess) the exact location of the documentation can be preserved. In that case we can change URLs to let https://jena.apache.org/documentation/javadoc/ function again. https://jena.apache.org/documentation/javadoc/ => content served from asf-site branch (generated with Hugo) http://jena.apache.org/documentation/javadoc/jena/ => served because a .htaccess (present on the asf-site branch) contains a RewriteRule to docs.jena.a.o/documentation/javadoc/jena/ Lucene does the same. > > > Is polling scm (build job) something we want to set-up or are starting with > > manually triggering the > > build-job after changes are made? > > Personally, I'd prefer to change to polling SCM. > > We have typically published the site with the release, having used > "staging" to view changes. trunk had the role of "proposed". > > If everyone is happy and setup for Hugo, we can change to master being > the ready-to-go real thing. Personally I'd say to have the polling in place. This ensures the latest changes will always be in place. Why would we want to have something on the master branch but not served on the site ? ;) - Roy > > Andy > > > > > [1] https://github.com/apache/infrastructure-puppet/pull/1734 > > > > On 2020/03/08 08:55:37, Andy Seaborne wrote: > >> Request made to move from svn+CMS to gitpubsub: > >> > >> https://issues.apache.org/jira/browse/INFRA-19939 > >> > >> Andy > >> >
Re: website move
Hi Andy, Good to be pro-active on this! However, I had yet to send reply to this list that the actual move was already done :) See [1] for the actual change. To verify: on jena.a.o the 'Edit this page' button is visible (meaning the site is served from Git), and this correctly redirects the user to the jena-site repository on GitHub. Is polling scm (build job) something we want to set-up or are starting with manually triggering the build-job after changes are made? [1] https://github.com/apache/infrastructure-puppet/pull/1734 On 2020/03/08 08:55:37, Andy Seaborne wrote: > Request made to move from svn+CMS to gitpubsub: > > https://issues.apache.org/jira/browse/INFRA-19939 > > Andy >
Re: [RESULT][VOTE] Move jena from cms/svn to hugo/git
Perfect. Thanks Andy! I used the following command for svn to git conversion: git svn clone --trunk="https://svn.apache.org/repos/asf/jena/site/trunk/; --authors-file=authors.txt --ignore-paths="(content/documentation/javadoc|content/documentation/temp)" jena-site Best way to move forward is for a Jena committer to clone [1] and push the 3 different branches to the Apache remote [2]. Pull requests can not be created since they need a base branch. And creating the branch means adding an initial commit which is not that handy when having the already existing history we want to keep. Is that something you can help out with Andy? ;) [1] https://github.com/rlenferink/jena-site [2] https://github.com/apache/jena-site On 2020/03/01 10:07:01, Andy Seaborne wrote: > > Looks good - the log counts are about right for the TLP + the incubator > site. (I had failed to find out how to stitch the incubator site to the > TLP site across the SVN rename - out of curiosity, how did you do that?) > > Repo [2] created: > > https://gitbox.apache.org/repos/asf/jena-site.git > https://github.com/apache/jena-site > > Andy >
Re: [RESULT][VOTE] Move jena from cms/svn to hugo/git
After fiddling around a bit I managed to get the SVN history included as well (had to run it multiple times because of a temporary javadoc location at /content/documentation/temp). All content is included except 'content/documentation/javadoc' and 'content/documentation/temp'. I have renamed the 'release-docs' branch to be called 'javadoc' which indeed makes more sense. Feel free to browse around here [1]. Whenever agreed with jena-site can be requested [2] and the 'master', 'asf-site' and 'javadoc' branches can be pushed. After that I'll create a Jenkins build job and make sure jena.a.o will be moved from the CMS to the git repository. [1] https://github.com/rlenferink/jena-site [2] https://gitbox.apache.org/setup/newrepo.html On 2020/02/26 11:10:56, Andy Seaborne wrote: > That would not carry over the history although the history would include > all the javadoc over time. Importing svn would keep the history > > I don't know if a partial import can be done, without > content/documentation/javadoc/ > > "git svn --ignore-paths=" looks hopeful (I've never run "git svn" before). > > History for the website would be good but personally I don't see it as a > blocker. > > Andy >
Re: [RESULT][VOTE] Move jena from cms/svn to hugo/git
Valid point. The reason for not including the history was all the javadoc stuff. But the --ignore-paths can maybe help with that. I'll have a look if possible. Roy Op wo 26 feb. 2020 om 12:11 schreef Andy Seaborne : > That would not carry over the history although the history would include > all the javadoc over time. Importing svn would keep the history > > I don't know if a partial import can be done, without > content/documentation/javadoc/ > > "git svn --ignore-paths=" looks hopeful (I've never run "git svn" before). > > History for the website would be good but personally I don't see it as a > blocker. > > Andy > > On 25/02/2020 13:05, Roy Lenferink wrote: > > [1] has just been updated. Easiest way to move forward (after the > jena-site repository has been > > requested and created) is to clone [1], add the asf remote and push the > 3 different branches to the > > asf remote. > > > > The 3 branches explained: > > - 'master' => website sources > > - 'asf-site' => generated website > > - 'release-docs' => javadoc documentation available on > /documentation/javadoc/* > > > > Let me know if help is needed! > > > > [1] https://github.com/rlenferink/jena-site > > > > On 2020/02/25 12:24:47, Roy Lenferink wrote: > >> Hi, > >> > >> The vote has passed with the following results: > >> > >> +1 Bruno P. Kinoshita (binding) > >> +1 Andy Seaborne (binding) > >> +1 Adam Soroka (binding) > >> +1 Rob Vesse (binding) > >> +1 Chris Tomlinson (binding) > >> +1 Roy Lenferink (non-binding) > >> > >> No -1 votes have been cast. > >> > >> To continue I'll need some help from a PMC member to request the > jena-site repository here [1] > >> > >> Once requested I'll make sure to have the repository with the migrated > content up to date [2] after which a committer can push the branches to the > Apache remote. I need someone with commit > >> privileges to help initially because of branch creation, cleanup of SVN > etc. which requires having > >> commit rights to the repository (I'm willing to help out with this so > let me know when help is needed). > >> > >> After having the site in place, I'll create a Jenkins job, test whether > it works and open a pull request > >> against the infrastructure-puppet repository to make the actual switch > from svn to git. > >> > >> Best, > >> Roy > >> > >> [1] https://gitbox.apache.org/setup/newrepo.html > >> [2] https://github.com/rlenferink/jena-site > >> >
Re: [RESULT][VOTE] Move jena from cms/svn to hugo/git
[1] has just been updated. Easiest way to move forward (after the jena-site repository has been requested and created) is to clone [1], add the asf remote and push the 3 different branches to the asf remote. The 3 branches explained: - 'master' => website sources - 'asf-site' => generated website - 'release-docs' => javadoc documentation available on /documentation/javadoc/* Let me know if help is needed! [1] https://github.com/rlenferink/jena-site On 2020/02/25 12:24:47, Roy Lenferink wrote: > Hi, > > The vote has passed with the following results: > > +1 Bruno P. Kinoshita (binding) > +1 Andy Seaborne (binding) > +1 Adam Soroka (binding) > +1 Rob Vesse (binding) > +1 Chris Tomlinson (binding) > +1 Roy Lenferink (non-binding) > > No -1 votes have been cast. > > To continue I'll need some help from a PMC member to request the jena-site > repository here [1] > > Once requested I'll make sure to have the repository with the migrated > content up to date [2] after which a committer can push the branches to the > Apache remote. I need someone with commit > privileges to help initially because of branch creation, cleanup of SVN etc. > which requires having > commit rights to the repository (I'm willing to help out with this so let me > know when help is needed). > > After having the site in place, I'll create a Jenkins job, test whether it > works and open a pull request > against the infrastructure-puppet repository to make the actual switch from > svn to git. > > Best, > Roy > > [1] https://gitbox.apache.org/setup/newrepo.html > [2] https://github.com/rlenferink/jena-site >
[jira] [Commented] (JENA-1849) Website Download not functioning
[ https://issues.apache.org/jira/browse/JENA-1849?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17044410#comment-17044410 ] Roy Lenferink commented on JENA-1849: - Thanks for reporting. This seems like an ASF wide issue. Other projects (e.g. [http://celix.apache.org/download.cgi)] are experiencing the same. > Website Download not functioning > > > Key: JENA-1849 > URL: https://issues.apache.org/jira/browse/JENA-1849 > Project: Apache Jena > Issue Type: Bug >Reporter: Andreas Plank >Priority: Critical > Attachments: Screenshot_20200225_134130_apache_jena_download.png > > > The web site [https://jena.apache.org/download/index.cgi] is not functioning > properly (see screenshot) > !Screenshot_20200225_134130_apache_jena_download.png! > > I did not know where else to report it. Thanks for repairing the issue. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[RESULT][VOTE] Move jena from cms/svn to hugo/git
Hi, The vote has passed with the following results: +1 Bruno P. Kinoshita (binding) +1 Andy Seaborne (binding) +1 Adam Soroka (binding) +1 Rob Vesse (binding) +1 Chris Tomlinson (binding) +1 Roy Lenferink (non-binding) No -1 votes have been cast. To continue I'll need some help from a PMC member to request the jena-site repository here [1] Once requested I'll make sure to have the repository with the migrated content up to date [2] after which a committer can push the branches to the Apache remote. I need someone with commit privileges to help initially because of branch creation, cleanup of SVN etc. which requires having commit rights to the repository (I'm willing to help out with this so let me know when help is needed). After having the site in place, I'll create a Jenkins job, test whether it works and open a pull request against the infrastructure-puppet repository to make the actual switch from svn to git. Best, Roy [1] https://gitbox.apache.org/setup/newrepo.html [2] https://github.com/rlenferink/jena-site
Re: [VOTE][LAZY] Move from CMS/SVN to Hugo/Git
Adding my +1 as well before closing the vote On 2020/02/24 09:47:27, Rob Vesse wrote: > +1 > > Glad to see progress on this > > Rob >
Re: [DISCUSS] Moving Jena site from Apache CMS/SVN to Git
On 2020/02/21 18:24:08, Andy Seaborne wrote: > Is the following description correct and complete? > > 1/ We have repo "jena-site" > > I prefer the website to be separate from the code and not in the > source-release so we are free to publish the site at anytime. This is indeed my suggestion as well because sites most probably need to be updated separate from releases. > > 2/ We have a branch of the "jena-site:asf-site" that is the published > webside > > 3/ There is a "jena-site:release-docs" branch for javadoc > Let's say this has /javadoc If we have the following configured in the gitwcsub.cfg (INFRA stuff) /www/jena.apache.org: $gitbox/jena-site This means that the 'asf-site' contains a 'content' folder with the actual site content. /www/jena.apache.org/content/documentation/javadoc: $gitbox/jena-site:release-docs This means that everything from the root of the release-docs branch will be served on jena.apache.org/documentation/javadoc The release-docs branch will contain an exact copy of the following: https://svn.apache.org/repos/infra/websites/production/jena/content/documentation/javadoc/ > > 4/ The site is built by a Jenkins job at ASF. The job is triggered manually. > e.g. > https://builds.apache.org/job/celix-site/ > > It is restricted to run on 'git-websites' > > 5/ It runs a shell script - see the one for Celix - > > In essence the job does: > > * pull master > * run Hugo to get the pages in /temp-??/output > * switch to 'asf-site' > * copy /temp-??/output to /content > /content/ is the site root. > * > * commit > * push -u > > Is there an example of a project that does javadoc? > Looks to me like at > > git co release-docs > cp -r /javadoc /content/documentation/ > Copying the release-docs to the asf-site branch is not even necessary. The config above (the INFRA part) defines that on the production machine content is being served from 2 branches. It _is_ possible to copy it to the asf-site branch and then serve everything from the asf-site, but that is just an additional step which I wouldn't do personally (we already have all the documentation available on the release-docs branch, no need to copy it over. Just let INFRA handle the merging of the actual content. > It emails rlenfer...@apache.org -- no need to change that :-) Let's keep that :) > > We need to modify the script because it does not handle separate javadoc. > > 6/ There is no "jena.staging.apache.org". It is replaced by people > running "hugo" locally to get "/target/content" : "hugo server" puts it > on http://localhost:1313 The staging domain is indeed gone. An alternative could be to use Netlify (netlify has free plans for OSS projects). Netlify will build pull requests and comment with a link to the preview. Celix has this enabled as well. See this pull request as example: https://github.com/apache/celix-site/pull/7 (of course cgi scripts are not working but this is expected behavior) But maybe with the new site build solution that INFRA is currently working on the staging domain will be re-introduced as well ;) This is something we probably will find out in a couple of weeks. > > Andy >
Re: [DISCUSS] Moving Jena site from Apache CMS/SVN to Git
On 2020/02/19 23:06:31, Andy Seaborne wrote: > > > On 19/02/2020 19:06, Andy Seaborne wrote: > > I am finding some conversion issues around boxes and > > > > https://jena.apache.org/tutorials/rdf_api.html#ch-Statements > > vs > > http://jena.roylenferink.nl/tutorials/rdf_api.html#ch-Statements > > > > The example is doubly boxed and the fint size keeps jumping. > > > > Yes, the Jena site markup is a bit yuk - it is a conversion itself - but > > we have the content we have. > > I've cleared up some problems - on staging now. > Pulling in the latest changes from svn seemed to have solved the problem. The 'old' page shows it as a element as well. I don't know if this is necessary because the 'new' page drops that element. This should be fixable but IMO loosing the blockquote isn't that much of a problem. The page is still pretty readable (updated the demo page as well so can have a look yourself :) ). > > > > There are changes around whitespace and tables. sometimes better, > > sometimes not. And I have seen an odd "Title" change where ":" becomes "-" > > The titles have been restored to there original ones. Good catch!
Re: [DISCUSS] Moving Jena site from Apache CMS/SVN to Git
(inline comments) On 2020/02/19 17:04:56, Andy Seaborne wrote: > > Sorry, I haven't had time to investigate all the machinery but if the > javadoc is a different branch or repo, how does publish happen? And not > happen for changes during the development cycle? > Publishing right now happens by checking out the 'production' site and adding the javadocs to there. Publishing in the git way would happen by adding the generated javadocs to a separate branch or repository. Right now jena.a.o is served using SVN (config here [1]). In these config files it is possible to specify multiple lines and specify a subfolder as well. E.g. what beam is doing here [2]. The following could make sense for jena: /www/jena.apache.org: $gitbox/jena-site /www/jena.apache.org/content/documentation/javadoc: $gitbox/jena-site:release-docs This means: - That the 'master' branch will contain the sources for the site - That the 'asf-site' branch will contain the generated output for the site - That the 'release-docs' branch will contain additional files which will be served at http://jena.apache.org/documentation/javadoc/ > > I know infra is working on a CMS replacement and/or guidelines. Is this > it? Is it compatible? I thought they were going to have a buildbot2. > Infra currently offers pelican and jekyll as 'replacements' for the CMS. However, Infra is also testing a new build approach were the project can more or less define how their site is build [3]. This could be a script to e.g. retrieve and execute Hugo. I already had a quick chat with Infra about how to implement support for Hugo (e.g. next to Jekyll and Pelican) but we decided that the new build approach will probably be a better fit. > > > > Good to hear the community is interested in this :) > > TBC: I am interested - I can't speak for the community. My bad. I made the quick assumption by taking 2 responses and calling that 'the community', which is almost never the case, oops! > Let me know if any additional information is needed. Roy [1] https://github.com/apache/infrastructure-puppet/blob/5bd3b7cc8bd6d8f2698a6e0cac1f421dd960ddf0/modules/svnwcsub/files/svnwcsub.conf#L115 [2] https://github.com/apache/infrastructure-puppet/blob/5bd3b7cc8bd6d8f2698a6e0cac1f421dd960ddf0/modules/gitwcsub/files/config/gitwcsub.cfg#L49-L50 [3] https://lists.apache.org/thread.html/rf0d4fef9b65e1ef346bc71b66253f9e54f15e048be6d011751a24ae0%40%3Cdev.community.apache.org%3E
[VOTE][LAZY] Move from CMS/SVN to Hugo/Git
Hi Jena community, After last weeks proposal [1] I'd like to start a vote for moving over from the current Apache CMS (and SVN) to use Hugo and git :) Short recap about Hugo: Hugo is a static site generator which is already having more stars on GitHub than Jekyll. It is easy to get started with (single static binary without dependencies) and it is really doing well performance wise. [ ] +1 for moving over from the Apache CMS / SVN to Hugo / Git. [ ] 0 I don't have a strong opinion on this [ ] -1 for not moving over, in this case please explain why This vote will be open for the usual 72 hour period. Lazy consensus applies: if no -1 votes are being cast within the voting period, the vote passes. Best regards, Roy [1] https://lists.apache.org/thread.html/r1a46369ea6e9df4f0b0971934563b9e8433cb33fe42290240969bbe0%40%3Cdev.jena.apache.org%3E
Re: [DISCUSS] Moving Jena site from Apache CMS/SVN to Git
Hi all, I think most (definitely valid) points have been addressed. I'll start a vote for this shortly. Roy On 2020/02/15 13:28:11, Roy Lenferink wrote: > Hi Jena community, > > I noticed Jena is still serving its site from SVN with help of the Apache > CMS. At the Apache Celix > project we decided to move away from the CMS and move towards git for serving > our website [1]. > This because the CMS was introduced in 2010 and is currently not accepting > any new projects. With > the move to Hugo we're sure that we still can update the site when the ASF > CMS will be > decommissioned. > > The master branch contains the actual website sources whereas the generated > website is served > from the asf-site branch. Hugo is being used as static website generator. > Content is written in > Markdown. > > I had a look at Jena its site and it was quite easy to migrate Jena its site > to use Hugo as well. An > example of Hugo its output is (for demo purposes only) available on [2]. > These are the existing > markdown files with a slightly changed header. > > Moving to git can enlarge the visibility of how projects are functioning. > Next to that, new contributors > can simply create a pull request against the website repository if they find > anything they want to > improve. I've updated the 'Improve this page' to 'Edit this page' which will > open the GitHub webpage > to edit the currently viewed page (currently returning a 404 because the > apache/jena-site repository > does not exist). > > If the community is interested in this move what I propose: > - Someone from the PMC to request a new git repository for the website (e.g. > jena-site) via [4] > - Me creating the pull request from the repository I am temporarily using for > the website contents [5] > to the official jena-site repository > - Create a Jenkins job to automatically build the site after changes happen > on the master branch. > - When the pull request is reviewed and merged ask INFRA to move over from > the current svnpubsub > to the gitpubsub approach and remove Jena from the Apache CMS. > > Next to that Jena is also providing javadocs on its site. My proposal for > this is to serve them from a > separate branch (e.g. release-docs). This way people who only want to clone > the site can clone the > the master branch using --single-branch and will not be bothered with cloning > all the javadocs. > > All the URLs are still the same so the SEO score is not influenced in any way. > > I'd like to hear everyone's opinion on this :) > > Best regards, > Roy > > [1] https://github.com/apache/celix-site > [2] http://jena.roylenferink.nl/ > [3] http://celix.apache.org/contributing/releasing.html > [4] https://gitbox.apache.org/setup/newrepo.html > [5] https://github.com/rlenferink/jena-site >
Re: [DISCUSS] Moving Jena site from Apache CMS/SVN to Git
Hi Andy, Thanks for your answer! I can confirm it is easier to use Hugo on Windows compared to Jekyll (I am generating and testing using the Ubuntu shell on Windows). Just downloading the tarball containing the Hugo binary and you're good to go. With Jekyll the process is a bit more complex as it requires some specific Ruby stuff. (see my inline comments below as well) On 2020/02/16 17:06:59, Andy Seaborne wrote: > Hi Roy - looks good! Thanks for pushing this along.# > > We had discussed this using Jekyll (because GH uses Jekyll) but I read > that installation on Windows is easier for Hugo (the speed issue does not > look like such an advantage for us) - and that that is helpful to get more > contributions. CMS and "improve this page" editing has had a reasonable > amount of use over the years but we do need to migrate away from CMS. And this is something that will be kept in place, however instead of a redirect to the Apache CMS it will open the GitHub editor with the file for the page the person is viewing at that moment. > > Was conversion quite smooth? Any discoveries? > What's the markdown like? I don't know of anything special except of course > tables. > Most of it was just putting the pieces together. The CMS skeleton.html could --with a few minor changes-- be re-used as Hugo template. The pages of the website were already written in markdown and only the header of the file needed to be changed (old: [1], new [2]), the content of the pages could just be re-used. Basically whatever syntax markdown supports is supported by Hugo [3]. > Managing javadoc separately would be nice. Putting a lot of javadoc into > svn is a bit clunky. > Well, in the end the data still needs to be stored somewhere. However, in this case I'd suggest to put it in a separate branch because in that way it is still possible for others to clone the site without the huge javadoc collection. An alternative could be a complete separate repository, e.g. jena-release-docs, which contains all the javadoc documentation, but this is just a matter of preference. > > Andy > Good to hear the community is interested in this :) Best, Roy [1] https://svn.apache.org/repos/asf/jena/site/trunk/content/getting_involved/reviewing_contributions.mdtext [2] https://raw.githubusercontent.com/rlenferink/jena-site/110a42f109b25b9771f55a500cfffcb0593544f0/source/getting_involved/reviewing_contributions.md [3] https://github.com/adam-p/markdown-here/wiki/Markdown-Cheatsheet
Re: [DISCUSS] Moving Jena site from Apache CMS/SVN to Git
Hi Claude, Thanks for your email and thoughts! I wasn't aware of Disqus and its privacy policy. After checking I think you are talking about the following page [1]? On that page the following 2 statements are present: - "To use Hugo’s Disqus template, you first need to set a single value in your site’s config.toml or config.yml" - "To add Disqus, include the following line in templates where you want your comments to appear" However, using Disqus is not something we are doing [2]. This can be verified by e.g. doing the following: Roy@BTO-ROY:~/workspace/asf/jena-site$ hugo | EN -+-- Pages| 230 Paginator pages| 0 Non-page files | 0 Static files | 136 Processed images | 0 Aliases | 0 Sitemaps| 1 Cleaned | 0 Total in 465 ms Roy@BTO-ROY:~/workspace/asf/jena-site$ grep -ri disqus Hugo writes the following [3] and how to disable specific options in the config.toml file [4]. I think this clarifies your doubts, can you confirm? An alternative would be using e.g. Jekyll which I did some tests with as well. Jekyll can be extended with a lot of different plugins, however Hugo is easier to setup (single static binary) and faster to generate the site compared to Jekyll [5] (no metrics from me personally). Roy [1] https://gohugo.io/templates/internal/#configure-disqus [2] https://github.com/rlenferink/jena-site/blob/master/config.toml [3] https://gohugo.io/about/hugo-and-gdpr/ [4] https://gohugo.io/about/hugo-and-gdpr/#disable-all-services [5] https://forestry.io/blog/hugo-vs-jekyll-benchmark/ On 2020/02/16 11:34:28, Claude Warren wrote: > I think this can be the first step in making the entire site deployment > process more automated and keeps us moving forward with supported > technologies. > > However, I note that the Hugo documentation states: "Hugo ships with an > internal Disqus template". I am concerned that the Discus connection may > not meet with the Apache Foundation's approach to privacy and data > sharing. I know there are problems with surveys and polls hosted by 3rd > parties. > > So: > +1 for moving to site to git. > -1 for using Hugo unless it can be deployed without 3rd party data sharing > or unless there is approval from Apache legal. > > Claude > > On Sat, Feb 15, 2020 at 1:28 PM Roy Lenferink wrote: > > > Hi Jena community, > > > > I noticed Jena is still serving its site from SVN with help of the Apache > > CMS. At the Apache Celix > > project we decided to move away from the CMS and move towards git for > > serving our website [1]. > > This because the CMS was introduced in 2010 and is currently not accepting > > any new projects. With > > the move to Hugo we're sure that we still can update the site when the ASF > > CMS will be > > decommissioned. > > > > The master branch contains the actual website sources whereas the > > generated website is served > > from the asf-site branch. Hugo is being used as static website generator. > > Content is written in > > Markdown. > > > > I had a look at Jena its site and it was quite easy to migrate Jena its > > site to use Hugo as well. An > > example of Hugo its output is (for demo purposes only) available on [2]. > > These are the existing > > markdown files with a slightly changed header. > > > > Moving to git can enlarge the visibility of how projects are functioning. > > Next to that, new contributors > > can simply create a pull request against the website repository if they > > find anything they want to > > improve. I've updated the 'Improve this page' to 'Edit this page' which > > will open the GitHub webpage > > to edit the currently viewed page (currently returning a 404 because the > > apache/jena-site repository > > does not exist). > > > > If the community is interested in this move what I propose: > > - Someone from the PMC to request a new git repository for the website > > (e.g. jena-site) via [4] > > - Me creating the pull request from the repository I am temporarily using > > for the website contents [5] > > to the official jena-site repository > > - Create a Jenkins job to automatically build the site after changes > > happen on the master branch. > > - When the pull request is reviewed and merged ask INFRA to move over from > > the current svnpubsub > > to the gitpubsub approach and remove Jena from the Apache CMS. > > > > Next to that Jena is also providing javadocs on its site. My proposal for > > this is to serve them from a > > separate branch (e.
[DISCUSS] Moving Jena site from Apache CMS/SVN to Git
Hi Jena community, I noticed Jena is still serving its site from SVN with help of the Apache CMS. At the Apache Celix project we decided to move away from the CMS and move towards git for serving our website [1]. This because the CMS was introduced in 2010 and is currently not accepting any new projects. With the move to Hugo we're sure that we still can update the site when the ASF CMS will be decommissioned. The master branch contains the actual website sources whereas the generated website is served from the asf-site branch. Hugo is being used as static website generator. Content is written in Markdown. I had a look at Jena its site and it was quite easy to migrate Jena its site to use Hugo as well. An example of Hugo its output is (for demo purposes only) available on [2]. These are the existing markdown files with a slightly changed header. Moving to git can enlarge the visibility of how projects are functioning. Next to that, new contributors can simply create a pull request against the website repository if they find anything they want to improve. I've updated the 'Improve this page' to 'Edit this page' which will open the GitHub webpage to edit the currently viewed page (currently returning a 404 because the apache/jena-site repository does not exist). If the community is interested in this move what I propose: - Someone from the PMC to request a new git repository for the website (e.g. jena-site) via [4] - Me creating the pull request from the repository I am temporarily using for the website contents [5] to the official jena-site repository - Create a Jenkins job to automatically build the site after changes happen on the master branch. - When the pull request is reviewed and merged ask INFRA to move over from the current svnpubsub to the gitpubsub approach and remove Jena from the Apache CMS. Next to that Jena is also providing javadocs on its site. My proposal for this is to serve them from a separate branch (e.g. release-docs). This way people who only want to clone the site can clone the the master branch using --single-branch and will not be bothered with cloning all the javadocs. All the URLs are still the same so the SEO score is not influenced in any way. I'd like to hear everyone's opinion on this :) Best regards, Roy [1] https://github.com/apache/celix-site [2] http://jena.roylenferink.nl/ [3] http://celix.apache.org/contributing/releasing.html [4] https://gitbox.apache.org/setup/newrepo.html [5] https://github.com/rlenferink/jena-site
