On Thursday, March 08, 2012 11:31:17 PM Christian Müller wrote: > Hi Dan, > > looks great! I also like that the new site passed the W3C validation and > that it renders the images of our blog posts much nicer. I will try it out > when it's online. > I didn't know that INFRA has mandated this move. Are there some resources > where I can read more about it?
There was a note sent out to private@camel on Feb 8th. You may have missed it. You should be able to find it from: https://mail-search.apache.org/pmc/ using your apache login creds. One other note: this distribution directory will need to be flipped to svnpubsub as well, but that's an easier thing to deal with as adding things there is a manual process anyway. Dan > > Thanks for the time you spend on it, > Christian > > On Wed, Mar 7, 2012 at 6:54 PM, Daniel Kulp <dk...@apache.org> wrote: > > As some of you are aware, the Apache Infrastructure team has mandated > > that all projects move to the new svnpubsub process for publishing > > their websites > > by the end of the year. Camel (as are all Confluence based sites) is > > affected by this mandate. We currently use a multi-step rsync process > > where Confluence exports the space to HTML, that gets rsynced to an area > > on people.apache.org once an hour. A cron process in someone's > > crontab on people.apache.org then runs to rsync it to the appropriate > > place (/www/camel.apache.org) once an hour or so as well. Then, > > another rsync will sync from there to the live sites. This causes a > > lot of delays in publishing (can be a few hours between change and > > live), but also involves a > > LOT of disk IO to sync things all over the place. The svnpubsub > > process > > is > > a lot faster as the site changes are committed to svn and anything that > > needs it (the live site) can listen for the changes and update > > immediately. > > > > Anyway, over the last couple of months, a bit of work has been done with > > various projects to start helping projects transition to svnpubsub. Joe > > Schaefer has been working with the Maven folks so the "mvn site:deploy" > > stuff can deploy via svnpubsub. Obviously the CMS uses it heavily. > > > > We also now have a "solution" for Confluence based sites based on work > > I've done for CXF's site. Confluence has a SOAP interface for > > retrieving information and rendering pages. Well, if I see a SOAP > > interface (even a crappy one like Confluence's)...... ;-) > > > > Seriously, I have a program now that can render an entire confluence > > space by using the SOAP API's and Velocity (which is what the current > > AutoExport stuff uses, so migration is easy). However, it does more > > than that by also > > recording the modified times, checking the RSS feed for changes first, > > tracking {children} and {include} tags, etc... Thus, if you change a > > page that is "included" in another (think about the "Book in One Page" > > page), those pages will also get re-rendered. If you add/delete a > > page, any page > > that uses the {children} tag to generate a tableof contents will > > automatically re-render. > > > > It ALSO cleans up the resulting HTML via tagsoup and some custom cleanup > > code. The Confluence generated HTML is aweful with invalid attributes, > > bad links, etc... They are now "mostly" cleaned up. > > > > I've uploaded a "build" of the site to: > > http://people.apache.org/~dkulp/camel/ > > so you can see that the result is pretty much identical to the live > > site. > > A couple of things are actually better such as the image links for the > > blog entries. Also, the new page actually validates with the w3c > > validator: > > > > > > http://validator.w3.org/check?uri=http%3A%2F%2Fpeople.apache.org%2F~dkul > > p%2Fcamel%2F<http://validator.w3.org/check?uri=http%3A%2F%2Fpeople.apach > > e.org%2F%7Edkulp%2Fcamel%2F> > > > > > > To run this, a buildbot build will be setup to run the process once an > > hour to generate new html if it detects a change (rss feed). Once run, > > you get a > > commit message to the commits list and the changes are live immedately. > > Thus, changes are now "at most" one hour till they are on the live site. > > However, any commiter can checkout the stuff and run it manually if they > > need/want things live immiately. > > > > > > For CXF, this new process is now "live" (since Monday). I've filed a > > ticket with INFRA to start the process for Camel. (requires a content > > area in the web svn repo for the live content, then a buildbot build, > > then some configs to make it all live) It's definitely still a "work > > in progress", but it's a good start. For example, it doesn't track > > the blog/news entries > > so currently if you add a blog (for a release), you would need to > > manually trigger an Index page update. However, the code is there so > > it's something > > we can add/enhance. I also want to update it to render pages in > > parallel > > if > > possible to make it a bit quicker. > > > > For camel, the main "pom" and scripts are in: > > http://svn.apache.org/repos/asf/camel/website/ > > and there is a README there. The code for the stuff is grabbed via an > > svn:externals to the area in CXF so I just need to update the code in > > one > > place: > > http://svn.apache.org/repos/asf/cxf/web/ > > I want to avoid "forking" the code as I *DO* know that if/when they move > > to Confluence 4.x (currently on 3.4.x), it will need some updating as > > the SOAP API's changes a bit. > > > > > > Anyway, I'm hoping to have Camel flipped over by the end of the week or > > early next week. > > > > BTW: if you are interested, it has to render 644 pages for the full > > Camel website. Takes about 15 minutes to do right now, but like I > > said, once it's all setup, it can do incremental updates which is MUCH > > quicker. 644 pages is quite a bit. > > > > > > -- > > Daniel Kulp > > dk...@apache.org - http://dankulp.com/blog > > Talend Community Coder - http://coders.talend.com -- Daniel Kulp dk...@apache.org - http://dankulp.com/blog Talend Community Coder - http://coders.talend.com
